Forum for the Future of Higher Education
CyberCampus Project
Technical Document 2.5
The Simulation Engine: Performance Indicators
This paper describes work in progress on the Higher Education simulation project funded by the Alfred P. Sloan Foundation. Contents may not be used or cited without permission. Limited distribution is provided to obtain comments and criticisms, and to assist potential development partners. Copyright © 1997 by JHHEG.
Table of Contents
1. Introduction
2. Overview
3. Academic Support Functions
3.1 Libraries:
3.2 Information technology infrastructure:
3.3 Student Life
3.4 Enrollment Management
4. Non-Academic Functions
4.1 Intercollegiate Athletics
4.2 Institutional Advancement Results
4.3 Administrative Effectiveness
4.4 O&M Effectiveness
5. Financial Performance
5.1 Surplus or Deficit
5.2 Tuition Discounting
5.3 Deferred Maintenance Backlog
5.4 Deviation from Long-Run Financial Equilibrium
5.5 Asset Accumulation
5.6 Assets to Liabilities Ratio
5.7 Percent of Expenditures on "Overhead" Activities
6. Institutional Performance Indicators
6.1 Prestige
6.2 Departmental Academic Standing
6.3 Education Quality
6.4 Research Performance
6.5 Student Diversity
6.5 Faculty Diversity
6.6 Average Class Size
6.6 Average Teaching Load
6.7 Graduation Rates
7. Attitudes Toward the Institution
7.1 Media Attitudes
7.2 Student Morale
7.3 Faculty Morale
7.4 Staff Morale
7.5 Alumni Morale
8. Trustee Evaluation
Higher Education is a computer-based simulation game under development that targets both the institutional professional and the interested layperson to participate in leadership challenges in a college or university setting. Players set, monitor, and modify a variety of institutional parameters and policies, allocate resources as they see fit, and watch as results continually unfold. The game provides an opportunity to experiment and succeed or fail in a safe and entertaining fantasy environment. While Higher Education is necessarily a caricature of real academic life, it is grounded in authentic data and will provide serious lessons in higher education. The game will be driven by a sophisticated simulation engine that models five broad areas:
1. enrollment management
2.
3.
4.
5.
This paper describes the models for computing performance indicators and associated variables. It is intended to provide an overview for technical readers, especially those who will be responsible for game development. The elements discussed herein pertain to the simulation engine, not to the player interface. Many will be transparent to the player. In particular, the definitions and formulas describe the kinds of quantities that will be computed within the simulation. They do not necessarily describe the game as it will be seen by the player. The perfromance indicator definitions provide a menu from which items can be selected during design of the player interface. Items not selected need not be computed unless they are required as inputs to other needed quantities.
The performance indicator variables fall into six broad categories: performance in academic support areas (e.g., libraries); performance in non-academic areas (e.g., administration); financial performance (e.g., surplus or deficit); general performance indicators (e.g., institutional prestige); attitudes toward the institution (e.g., student morale); and trustee evaluation (the game’s overall scoring system).
A variable may be driven by a simple or dual logistic response function. Some variables will be passed through exponential smoothing functions. The weights associated with each function will be read from the game’s database. Some may be modified during play.
The simple logistic function transforms a list of input variables into a single output variable. It uses the following equation to produce an s-shaped response curve:
where the xi are input variables, wi are weights, and w0 is the value of the weighted sum that produces a result midway between the floor and ceiling. The value of w0 may equal a constant or the weighted sum’s initial value ("initial condition"); this value may be multiplied by a random factor to provide variation from play to play. For some variables, the weighted sum may include interactions as well as linear terms. The simple logistic curve is illustrated in Figure 1, for a single driver variable, w0=0, floor=0, and ceiling=1.
Figure 1: Simple Logistic Function
The dual logistic function is used when the output variable does not necessarily have maximum slope midway between the extremes —e.g., where there is a range of relative insensitivity around the middle of the curve. This curve, which is computed by summing two simple logistics, is illustrated in Figure 2. Notice that the excursions above and below the midpoint need not be symmetric.
Figure 2: Dual Logistic Function
The exponential smoothing
where yt is the variable to be smoothed, yt–1 is its value for the prior year, and xt the current value of the driver variable—e.g., the output of one of the aforementioned response functions. The smoothing parameter, l, must be between 0 and 1. The closer l is to 1, the faster the response.
Definition: the degree to which library operations provide effective support for teaching and research programs. Includes both collection development and service provision without differentiation.
Depends on: current and cumulative past expenditures for library operations per faculty FTE; staff morale.
Response function: simple logistic; midpoint based on the initial conditions.
Smoothing: yes.
Definition: the degree to which the information technology infrastructure supports teaching and research. Includes both equipment resources and service provision without differentiation.
Depends on: current and cumulative past expenditures on information technology per faculty FTE; weighted average percent faculty effort devoted to IT innovation (averaged over departments and faculty groups, weighted by FTEs); staff morale.
Response function: dual logistic (to allow a disproportionate response to strong efforts); midpoint based on the initial conditions.
Definition: the effectiveness of student life programs and student services. Includes extracurricular activities, intramural athletics, counseling and medical care, and other services provided to students.
Depends on: current student life expenditures per weighted student FTE (traditional undergraduates will have the heaviest weight); extracurricular rating of admitted traditional undergraduates; staff morale.
Definition: the effectiveness of enrollment management activities. Includes the admissions and financial aids offices, brochures, advertising, etc.. Feeds into the enrollment management model, which determines student application and yield rates.
Depends on: current enrollment management expenditures per admissions place (traditional undergraduates will have the heaviest weight); staff morale.
Response function: dual logistic (to provide strong effects at both extremes); midpoint based on the initial conditions.
Definition: effectiveness in intercollegiate athletic competition. The index value translates into wins and losses in the key sports of football and basketball (which occur in the fall and winter trimesters, respectively)—which in turn affect morale and attitudes toward the institution. (Specific wins and losses will be programed as part of the player interface.)
Depends on: current expenditures on athletics in relation to a norm based on the level of competition chosen by the player (e.g., NCAA Division 1A, etc.); the athletics rating of admitted traditional undergraduates.
Response function: dual logistic; midpoint based on the competition-level norm (the dual logistic allows for disproportionate effects at both extremes).
Definition: the dollar value of gift income. Total gift income will be divided into gifts for current use, gifts for endowment, and gifts for plant according to player-determined percentages.
Depends on: current expenditures on institutional advancement; staff morale; institutional prestige; alumni, student, and faculty morale; absence of adverse administrative outcomes.
Response function: dual logistic; midpoint based on the initial conditions; the logistic output; upper and lower bounds (which are dollar values) to be determined based on the initial conditions.
Definition: the effectiveness of administrative activities. High values translate into high faculty, staff and student morale. Low values translate into lower morale and also increase the probability of adverse administrative outcomes (e.g., process breakdowns or audit problems, which will be generated as part of the player interface).
Depends on: current expenditures on administration as a fraction of total expenditures; recent rate of change in administrative expenditures (high rates of change reduce effectiveness, other things being equal); staff morale; faculty morale.
Response function: dual logistic (disproportionate negative consequences to inadequate budgets) ; midpoint based on the initial conditions; midpoint may change as a result of a player-generated decision to engage in administrative restructuring (the midpoint might rise for a year while the restructuring is being done, then fall to reflect the benefits of restructuring).
Definition: the effectiveness of operations and maintenance activities. High values translate into high faculty, staff and student morale. Low values translate into lower morale and also increase the probability of adverse physical events (e.g., building and equipment malfunctions, to be programed as part of the player interface). The index value also operates through the physical plant model to increase or decrease the deferred maintenance backlog.
Response function: dual logistic (disproportionate negative consequences to be associated with low budgets) ; midpoint based on the initial conditions; midpoint may change as a result of the aforementioned player-generated decision to engage in administrative restructuring.
The following represent key financial indicators that will be of particular interest to the player. Other quantities described in connection with the resource allocation model also should be available for review.
Definition: indicates whether the institution is adding to or subtracting from its operating reserve.
Equals: surplus or deficit as a percent of total expenditures.
Response function: none.
Smoothing: no.
Definition: represents the degree to which the institution discounts its sticker price through financial aid.
Equals: institutional financial aid as a percent of tuition revenue, by student level.
Definition: the size of the deferred maintenance backlog as calculated in the plant model.
Equals: the deferred maintenance backlog as a percent of plant replacement value.
Definition: the degree to which the institution departs from long-run financial equilibrium. Equilibrium requires that the surplus/deficit and rate of change in the deferred maintenance backlog be zero, and that the growthrates of income and expense be equal.
Depends on: current growthrates of income and expense as determined by player policy and exogenous factors; beginning-of-year budget and asset values; growthrates of intercollegiate athletics income and the deferred maintenance backlog calculated from current policies and parameters.
Definition: the degree to which the institution is accumulating total assets on an inflation-adjusted basis.
Depends on: total assets as shown on the balance sheet (sum of the operating reserve, endowment market value, plant value, and unexpended plant reserve); prior year’s value; and the inflation rate.
Definition: the degree to which the institution is burdened by debt.
Equals: ratio of total funds balances to the sum of general, plant and bank debt. (Bank debt exists when the operating reserve is negative.)
Definition: the degree to which the institution spends on activities that do not directly benefit students and faculty.
Equals: sum of expenditures for institutional advancement, administration, operations and maintenance, and other operation expense, as a percent of total operating expenditures.
Definition: prestige as defined by external constituencies including the media. High prestige generates favorable media ratings and word of mouth.
Depends on: the weighted average of the five best departmental academic standing scores* (the weights reflect the importance of the departments to institutional prestige); selection ratio and average academic ability rating for entering traditional undergraduates; libraries effectiveness index; ratio of FTE students per FTE faculty; total expenditures per FTE student.
Response function: dual logistic (allows for strong effects at the extremes).
Definition: average departmental academic standing: a measure of the institution’s average academic prowess as opposed to the aura perceived overall.
Equals: the weighted average of departmental academic standing scores to the institutional level (weights are the same as for institutional prestige).
Definition: the aggregation of departmental education quality scores to the institutional level. In addition to its other benefits, high quality may result in favorable comments in the media.
Equals: the weighted sum of the departmental education quality scores (the weights reflect the importance of the departments to institutional education quality).
Definition: the aggregation of departmental research performance scores to the institutional level. In addition to its other benefits, a high score may result in a favorable comments in the media and high placement in the government scientific research rankings.
Equals: the weighted sum of the departmental research performance scores (the weights reflect the importance of the departments to institutional research performance).
Definition: the degree to which student diversity goals have been met— calculated for each student level and gender-ethnic group, by student level, by gender-ethnic group, and overall.
Equals: the ratio of FTE students to the associated diversity target (in terms of FTEs) for the student level and gender-ethnic group. The aggregate indices equal the ratios of the sums of these two quantities.
Definition: the degree to which faculty diversity goals have been met— calculated for each faculty group (or perhaps aggregations of groups) and gender-ethnic group, by faculty group, by gender-ethnic group, and overall.
Equals: the ratio of FTE faculty to the associated diversity target (in terms of FTEs) for the faculty group and gender-ethnic group. The aggregate indices equal the ratios of the sums of these two quantities.
Definition: average class size by course level (undergraduate and graduate).
Equals: the weighted average of class sizes over departments and course types, using the number of courses as weights (undergraduate courses come in three types, graduate courses one type).
Definition: average teaching load for tenure-line faculty.
Equals: weighted average of teaching loads over departments and faculty groups, using faculty FTEs as weights.
Definition: percent of entering students who graduate (in five years if I can devise an algorithm to cut the calculation there), by student level.
Equals: weighted average of graduation rates over departments and student levels using student FTEs as weights.
Definition: attitudes of the mass media toward the institution.
Depends on: institutional prestige; smoothed education quality; smoothed research performance; athletic performance; absence of adverse administrative events.
Response function: dual logistic.
Definition: attitudes of students toward the institution (as opposed to attitudes toward individual departments, calculated in the academic operations model); calculated separately for each student level and then aggregated.
Depends on: the weighted sum of the department-level student morale scores for student levels (with FTEs as weights); unmet student residence demand; the student life index; athletics effectiveness; O&M effectiveness; and the deferred maintenance backlog. Aggregation over student levels uses FTEs as weights.
Response function: dual logistic; none for aggregation over student levels.
Definition: attitudes of faculty toward the institution (as opposed to attitudes toward individual departments, calculated in the academic operations model); calculated separately for each faculty group and then aggregated.
Depends on: the weighted sum of the department-level faculty morale scores by for faculty groups (with FTEs as weights); institutional prestige; academic quality of entering undergraduate students (academic quality of entering masters and doctoral students are included in the departmental morale score); libraries effectiveness; administrative effectiveness; O&M effectiveness; the deferred maintenance backlog; and media attitudes. Aggregation over faculty groups uses FTEs as weights.
Response function: dual logistic; none for aggregation over faculty groups.
Definition: attitudes of staff toward the institution.
Depends on: staff salaries in relation to market; the most recent year-to-year change in total staff FTEs; administrative effectiveness; negative mid-year budget adjustments; O&M effectiveness; the deferred maintenance backlog; and media attitudes. The year-to-year change in total staff FTEs equals the change in total staff compensation divided by (1 + inflation rate + real staff salary improvement).
Definition: attitudes of alumni toward the institution.
Depends on: institutional prestige; education quality; smoothed resources devoted to institutional advancement; student morale; athletic performance; media attitudes.
The simulated trustees will provide feedback to the player in his or her role as President. Evaluations will be provided for particular performance areas, and for overall performance. The latter will serve as the player’s overall success measure. The performance scores will be calculated frequently. The summary score will be made available to the player continuously, the detail will be made available on demand.
The evaluations will be computed according to the following formula:
where E[a] is the evaluation for activity area a, E[•] is the overall evaluation, and the xi are the driver variables. The quantity (xi - xi 0)+ represents a positive deviation of the driver from its target (xi 0 > 0); (xi - xi 0)–, taken as a positive quantity, represents a negative deviation. The powers k determine the shape of the response (e.g., k=1 for linearity) The w-values are weights; they add to one for each i-summation and for the a-summation. The activity-level function rewards positive excursions from the targets and penalizes negative ones, perhaps with different weights and response patterns. The overall score is the weighted average of the activity-level scores.
The candidate activity categories and driver variables to be associated with each category are shown in the following table. (The list will be refined in subsequent discussions.) The target variables will be set equal to the initial value of the driver (perhaps displaced randomly) in the case of general play. Scenario play will assign targets appropriate to the scenario.
Candidate Activity Areas and Driver Variables
General institutional Performance • prestige • education quality • research performance • student diversity • faculty diversity • athletics record
Attitudes Toward the Institution • media attitudes • student morale • faculty morale • staff morale • alumni morale
Admissions (trad’l undergrads) • applications – number – average academic rating – ave. non-academic rating • matriculations – average academic rating – ave. non-academic rating • yield rate
Management • graduation rate (trad. undergrads) • average class size (undergrad c’ses) • average teaching load • information technology utilization • administrative performance • annual gift receipts
Finance • current surplus/deficit • deviation from LRFE (–) • asset accumulation • asset to liability ratio • deferred maintenance backlog (–) • percent exp. on overhead (–)
