Transcript β R - Michigan State University

Inference by College
Admission Departments:
Bayesian or Cursed?
Michael Conlin Michigan State University
Stacy Dickert-Conlin Michigan State University
Cornell University– October 2009
Optional SAT I Policy
Colleges with optional SAT I policy
allow applicants to submit their SAT I
score but do not require them to submit
their score.
(As of Spring 2009, more than 800 colleges
have SAT- or ACT- optional policies.)
Research Question
What does the admission
department at the college infer
when an applicant chooses not to
submit her SAT I score?
This inference influences whether or
not the school accepts the applicant.
Bayesian vrs. Cursed Equilibrium
Most game theoretic papers assume the
inference is based on Bayes Rule.
Eyster & Rabin (Econometrica 2005) document
extensive experimental evidence and develop a
simple model of how players underestimate the
relationship between other players’ actions and
their private information. They call this
psychologically motivated equilibrium concept a
cursed equilibrium.
Eyster and Rabin (page 1633)
“The
primary motivation for defining
cursed equilibrium is not based on
learning or any other foundational
justification, but rather on it
pragmatic advantages as a powerful
empirical tool to parsimoniously
explain data in a variety of context.”
Why is this important?
Think of any private information game –
including those resulting in adverse selection,
signaling and screening. Most of these games
apply Bayesian Nash Equilibria to predict
outcome. The Bayesian Nash Equilibrium
Concept assumes that players apply Bayes
Rule at all information sets that are reached with
positive probability. Expected outcomes could
change significantly if this were not the case.
Example using Spence
Education Model
Nature
Low Productivity Worker
High Productivity Worker
e=0
Firm
e=0
e=12
e=12
e=16
e=16
Suppose a separating equilibrium exists where the low productivity type
gets 0 years of education and the high productivity type gets 16 years of
education.
Voluntary Disclosure/ Verifiable
Cheap Talk Example


Student i has the following
probability distribution in term
of SAT I scores.
When disclosure is costless,
Bayesian Nash Equilibrium
results in every type except
the worst disclosing and the
worst being indifferent
between disclosing and not
disclosing.
SAT I Score
Probability
1300
0.2
1200
0.4
1100
0.3
1000
0.1
Voluntary Disclosure /
Verifiable Cheap Talk Models

Comments:
Distribution depends on student characteristics that are
observable to the school such as high school GPA.
With positive disclosure costs, the “unraveling” is not
complete and only the types with the lower SAT I scores
do not disclose.

Assumptions:
Common Knowledge.
Colleges use Bayesian Updating to Infer SAT I Score of
those who do not Submit/Disclose
Colleges’ incentives to admit an applicant is only a
function of his/her actual SAT I score (not whether the
applicant submits the score)
Eyster and Rabin applied to Voluntary
Disclosure / Verifiable Cheap Talk
College correctly predicts the
distribution of the applicant’s actions
but underestimates the degree these
actions are correlated with the
applicant’s SATI score (i.e., private
information).
SAT I Score
Probability
1300
0.2
1200
0.4
1100
0.3
1000
0.1
“Fully” Cursed Equilibrium (χ=1)– College infers if applicant doesn’t disclose
that his/her expected SAT I score is
1300(.2)+1200(.4)+1100(.3)+1000(.1)=1170
“Partially” Cursed Equilibrium (χ=.4 for example)– College infers if applicant
doesn’t disclose that his/her expected SAT I score is
(.4)1170 + (1-.4) [(1100(.3)+1000(.1))/.4] = 1113
Why is this important in the
context of SATI optional policy?
If,
(1) a college underestimates the relationship
between an applicant’s decision to not submit and
their SATI score (i.e., private information) and
(2) certain types of applicants are less likely to
submit,
then the SATI optional policy is likely to affect the
demographic characteristics of the student body.
Overview of Environment
SAT EXAMS: College Board administers SAT I & SAT II: Subject exams 7 times/year
between October and June. SAT I was a 2-part verbal and math exam. There are 20
SAT II exams.
EARLY DECISION: Submit application by mid-November & sign a written
agreement (with guidance counselor) that states that upon admittance the
applicant will attend if she can “afford it” (agreement not legally binding). Notified
in December of acceptance. The deadline for regular admission applications is
January 1st and these applicants are notified of acceptance or denial between
March and April. An accepted applicant that chooses to enroll fills out enrollment
forms and makes a deposit by May 1st.
Applicant’s Early Decision and
SAT Submission Decisions
Applicant i’s expected utility conditional on attending the
college is
μ(Zi, εap, εen)= μ(Zi) + εap + εen
where,
μ(Zi) captures individual–specific preferences that depends on observable variables (Zi),
εap represents unobservable individual–specific preferences known to the applicant at the time
she submits the application, and
εen represents unobservable individual–specific preferences not known to the applicant until she
makes the enrollment decision.

Applicant i’s expected utility if doesn’t attend
UR if apply regular admission
UR-C if apply early decision
Applicant’s Early Decision and
SAT Submission Decisions
Applicant i’s expected utility from not applying early
decision to the college and submitting SATI scores
is:
where,
is the applicant’s expectation of the probability she will attend,
εs represents the “cost” of submitting which is known to the applicant at the
time she submits application, and
K represents fixed cost of applying.
Applicant’s Early Decision and
SAT Submission Decisions
Assuming UR=0 (a normalization) and E(εen)=0, an
applicant applies early decision and does not
submit if :
CONDITION 1
CONDITION 2
CONDITION 3
is not binding
College’s Objective Function
To account for the college’s concern for the
quality of its current and future students and
the understanding that future student quality
depends on the college’s ranking, we allow
the college’s objective function to depend on
the perceived ability of the incoming
students, the “reported” ability of these
students, the demographic characteristics of
the student body and yield rate.
College’s Objective Function
where,
ΠP(ZP) captures how the college’s payoff is affected by
the perceived student body’s ability level,
ΠR(ZR) captures how the college’s payoff is affected by
the reported student body’s ability level,
ΠD(ZD) captures how the college’s payoff is affected by
the demographic characteristics of the student body,
f(YR) captures how the yield rate affects the college’s
payoff
College’s Acceptance Decision
College accepts applicant i if:
where,
denotes the expected probability an accepted applicant i enrolls
where k equals ed (ned) if applicant i applies early decision (regular
decision) and l equals s (ns) if the applicant i submits (does not
submit) her SATI score,
Z+i, Z-i, Zri are expected characteristics of student body if applicant i is
accepted and enrolls, is accepted but does not enroll, and is rejected.
College’s Acceptance Decision
The probability applicant i is accepted if she
applies early and does not submit is:
with the expected student body characteristics
being the same if applicant i is accepted but
chooses not to enroll (Z-i) or if applicant i is
rejected (Zri).
Applicant’s Enrollment Decision

An accepted applicant i who does not apply early
decision will enroll if:
μ(Zi) + εap + εen > UR

An accepted applicant i who does apply early
decision will enroll if:
μ(Zi) + εap + εen > UR – C
Applicant’s Enrollment Decision
The probability an early decision applicant i, who
does not submit, enrolls conditional on being
accepted is:
Pe(Zi,e,ns)=
Prob[εap+εen>-μ(Zi)-C, Conditions 1,2&3 hold] /
Prob[Conditions 1,2&3 hold].
Generating a Likelihood Function
1.
2.
3.
4.
Parameterize the cursed equilibrium concept
proposed by Eyster and Rabin;
Make assumptions about the applicants’
expectations of attending and the college’s
expectations of enrollment;
Impose functional form restrictions on the
college’s and applicants’ objective functions;
Make distributional assumptions for the random
components.
Parameterizing cursed equilibrium
We assume college’s belief of applicant i’s SAT score if
she does not submit is
SATi,un + (1- )SATi,cond
where,
SATi,un is the belief of applicant i’s score if the college
does not condition on her choosing not to submit and
SATi,cond is the belief if the college does condition on
her choosing not to submit.
Parameterizing Eyster and Rabin’s
cursed equilibrium concept

We construct SATi,un


regress SATI scores for applicants who submit on their
observable characteristics.
use the resulting coefficient estimates to construct fitted
values for all applicants, i, who do not submit.
•

assume the fitted value for applicant i is what would be inferred if
the college believes that applicant i’s decision to not submit did
not depend on her actual SATI score.
We assume SATi,cond is applicant i’s actual SATI score.
Eyster and Rabin applied to Voluntary
Disclosure / Verifiable Cheap Talk
College correctly predicts the
distribution of the other
players’ actions but
underestimates the degree
these actions are correlated
with the other players’ private
information.
SAT I Score
Probability
1300
0.2
1200
0.4
1100
0.3
1000
0.1
“Fully” Cursed Equilibrium (χ=1)– College infers if applicant doesn’t disclose
that his/her expected SAT I score is
1300(.2)+1200(.4)+1100(.3)+1000(.1)=1170
“Partially” Cursed Equilibrium (χ=.4 for example)– College infers if applicant
doesn’t disclose that his/her expected SAT I score is
(χ)SATun+ (1-χ)SATcond = (.4)1170 + (1-.4) [(1100(.3)+1000(.1))/.4] = 1113
Applicants’ expectations of attending
College’s expectations of enrollment
Assume college uses the information they have on
each applicant to accurately predict their enrollment
probabilities.
For those who submit,
For applicants who chose not to submit their SATI scores,
the college’s expected probability of enrollment is
similar to the predicted probability except the college
infers an SATI score of SATi,un + (1- )SATi,cond while
the predicted probability, Pe(Z,k,l), is based on their
actual score.
Functional Form Restrictions
μ(Zi) = βμ•Zi
f(YRai) = βYR YRai , f(YRri) = βYR YRri
ΠP(Z+iP)= βP•Z+iP , ΠP(Z-iP)= βP•Z-iP
ΠR(Z+iR)= βR•Z+iR , ΠR(Z-iR)= βR•Z-iR and
ΠD(Z+iD)= βD•Z+iD , ΠD(Z-iD)= βD•Z-iD
Functional Form Restrictions
Assuming the college’s expectations of Z-ij, YRri,
and Nsb do not vary across applicants, these
functional form restrictions result in:
where βP’= βP/(Nsb+1), βR’= βR/(Nsb+1), βD’=βD/(Nsb+1),
βYR’= βYR YRri/(N+1) and βint is a function of Z-ij (for all j),
YRri, Nsb and N.
Functional Form Restrictions
Note that not all the parameters are
identified. For example, suppose high
school GPA affects both perceived quality
and ranking, then can only identify
βPGPA+βRGPA. We can identify βP and βR
associated with the SATI scores because
some applicants submit while others do
not.
Distributional Assumptions
εap, εen, εs and εqi are standard
normal and independent.
Likelihood Function
Goal is to estimate a vector of structural parameters
θ = { , βμ, βe, βks, βksat, βint, βP’, βR’, βD’, βYR’ and C}.
Derive parametric expressions, as a function of observed data
and structural parameters, for



the probabilities an applicant applies early decision and/or
submits her SATI score
the probability the college accepts an applicant
the probability an accepted applicant enrolls.
Expressed with standard normal and standard multivariate
normal distributions.
Data





Application data for 2 liberal arts schools in
the north east
Each with approximately 1800 students
enrolled.
Both report a typical SAT I score in the
upper 1200s/1600
College X: 2 years ≈ 5 years after the
optional policy was instituted.
College Y the year after the optional policy
was instituted.
College Board Data
SAT scores for those who elected not
to submit them to the college.
 Student Descriptive Questionnaire
(SDQ)

SAT II Scores
 Self Reported income
 High school GPA
 High school activities

Descriptive Statistics for College X
Early Decision
Applicants more
likely accepted
Early Decision
Applicants more
likely to enroll
Applicants that apply early
decision and don’t submit
have lower SATI scores
Number of Applicants
324
122
5216
895
Descriptive Statistics for College X
Female applicants
from private high
school less likely to
submit
Applicants who apply
early are weaker
students
Descriptive Statistics for College X
Legacies and high
income applicants
are more likely to
apply early decision
Descriptive Statistics for
College Y
Similar to College X except:
1.
2.
Early Decision applicants are less likely to be
accepted (~.65 compared to ~.85 for College
X) and enroll conditional on acceptance (~.92
compared to ~.98 for College X)
Measure of academic performance are slightly
worse for applicants who do not submit their
SAT I scores.
Results from reduced form papers
(Dickert-Conlin & Chapman; Conlin,Dickert-Conlin & Chapman)
1.
Applicants are behaving strategically by choosing
not to reveal their SAT I scores if they are below a
value one might predict based on their other
observable characteristics.
Results from reduced form papers
SATIun
Results from reduced form papers
(Dickert-Conlin & Chapman; Conlin,Dickert-Conlin & Chapman)
1.
2.
Applicants are behaving strategically by choosing
not to reveal their SAT I scores if they are below a
value one might predict based on their other
observable characteristics.
College admission departments are behaving
strategically by rewarding applicants who do
submit their SAT I scores when their scores will
raise the college’s average SAT I score reported to
U.S. News and World Report and rewarding
applicants who do not submit when their SAT I
scores will lower the college’s reported score.
Applicants who submit their SAT I
score are less likely to be
accepted by College X if their SAT
I score is below 1388 and are
more likely to be accepted if their
score is above 1388. Average
SAT I score at College X is 1281.
Results from reduced form papers
Set of regressors include SATII score, ACT score, private high school, female,
Subsequent
measures
offamily
college
aredecision
not correlated
HS GPA,
HS class rank,
incomeperformance
bracket, legacy, early
indicator,
race, and
geographic
location, HS decisions
extracurricular in
activities.
with scores
submission
a similar manner.
Table 2: Structural Parameter Estimates
College X has more
years of experience
with optional policy
but admission director
is relatively new.
Expect probability of
acceptance to increase with
expectation of enrollment but
difference in coefficient is
worrisome.
Estimates large as expected based on enrollment
probabilities of early decision and early decision enrollment
probability higher for College X.
Parameters of
College’s Payoff
Function
Table 2 Structural Parameter Estimates
Utility Parameters
Table 2 Structural Parameter Estimates
Table 2 Structural Parameter Estimates
Table 3: Model Fit College X
Table 3: Model Fit College Y
Model fits the other observables well in terms
of whose accepted and those who enrolled.
Table 6: Counterfactuals : Student Body
Composition for Applicants who don’t Submit
Table 7: Counterfactuals : Student
Not much
difference
because
of
inability
of
model
Body Composition
to fit acceptance and enrollment probabilities.
Table 8: Counterfactuals College X
Student Body Performance Measures
for Applicants who do not Submit
Table 8: Counterfactuals College X
Student Body Performance Measures
for Applicants who do not Submit
Better students
transfer to Cornell
Table 8: Counterfactuals College Y
Student Body Performance Measures
for Applicants who do not Submit
Conclusions
1.
2.
The empirical results suggest that the colleges
underestimates the relationship between an
applicant’s decision to submit her SATI score
and her actual SATI score.
For those applicants who choose not to
submit, this affects their acceptance decision
and also the demographic characteristics of
the expected student body.