Transcript X - Michigan State University

Inference: Bayesian,
Cursed, …
Michael Conlin Michigan State University
Stacy Dickert-Conlin Michigan State University
Jeff Wooldridge Michigan State University
2009 Summer Meetings
Optional SAT Policies
“I SOMETIMES think I should write a handbook for college
admission officials titled “How to Play the U.S. News & World
Report Ranking Game, and Win!” I would devote the first
chapter to a tactic called “SAT optional.”
The idea is simple: tell applicants that they can choose
whether or not to submit their SAT or ACT scores.
Predictably, those applicants with low scores or those who
know that they score poorly on standardized aptitude tests
will not submit. Those with high scores will submit. When
the college computes the mean SAT or ACT score of its
enrolled students, voilà! its average will have risen. And so
too, it can fondly hope, will its status in the annual U.S.
News & World Report’s college rankings.”
Colin Diver, President of Reed College, New York Times, 2006
U.S. News & World Report
(Criteria and weights for rankings colleges)
Research Question
What is the college’s
inference for applicants who
choose not to submit their
SAT I scores?
College Data

Application data for 2 liberal arts schools in
north east
Submitted SATI Scores, Submitted SATII
Scores, Submitted ACT Scores, High School
GPA, Private High School, Race, Gender,
Residence, Legacy
Acceptance and Enrollment Decisions.
 Performance Measures for those who Enroll.

College Board Data
SAT I scores for those who elected
not to submit them to the college.
 SAT II scores
 Student Descriptive Questionnaire
(SDQ)

Self Reported income
 High school GPA
 High school activities

Summary Statistics

15.3 percent of the 7,023 applicants to
College X choose not to submit SAT I
scores.

24.1 percent of the 3,054 applicants to
College X choose not to submit SAT I
scores.
Summary Statistics for College X
N=324
N=122
N=5216
N=895
Summary Statistics for College X
Conclusions from Prior Research
(based on estimates from reduced form)
1.
College admission departments are behaving
strategically by more (less) likely accepting applicants
WHO DO NOT SUBMIT their SAT I scores if submitting
their scores would decrease (increase) the average SAT
I score the colleges report to the ranking organizations.
2.
Applicants are behaving strategically by choosing not to
reveal their SAT I scores if they are below a value one
might predict based on their other observable
characteristics.
These reduced form results are robust to different
assumptions regarding college’s inference for those
applicants who do not submit.
Voluntary Disclosure Example


Student i has the following
probability distribution in term
of SAT I scores.
When disclosure is costless,
Bayesian Nash Equilibrium
results in every type except
the worst disclosing and the
worst being indifferent
between disclosing and not
disclosing.
SAT I Score
Probability
1300
0.2
1200
0.4
1100
0.3
1000
0.1
Expected SAT I Score
1300(.2)+1200(.4)+1100(.3)+1000(.1)
=1170
Voluntary Disclosure Models

Comments:
Distribution depends on student characteristics that
are observable to the school such as high school
GPA.
With positive disclosure costs, the “unraveling” is not
complete and only the types with the lower SAT I
scores do not disclose.

Assumptions:
Common Knowledge.
Colleges use Bayesian Updating to Infer SAT I Score
of those who do not Submit/Disclose
Colleges’ incentives to admit an applicant is only a
function of his/her actual SAT I score (not whether the
applicant submits the score)
Voluntary Disclosure: Theory
o Eyster and Rabin (Econometrica,
2005) propose a new equilibrium
concept which they call a Cursed
Equilibrium. College correctly
predicts the distribution of the other
players’ actions but underestimates
the degree these actions are
correlated with the other players’
private information.
SAT I Score
Probability
1300
0.2
1200
0.4
1100
0.3
1000
0.1
“Fully” Cursed Equilibrium (χ=1)– College infers if applicant doesn’t disclose
that his/her expected SAT I score is
1300(.2)+1200(.4)+1100(.3)+1000(.1)=1170
“Partially” Cursed Equilibrium (χ=.4 for example)– College infers if applicant
doesn’t disclose that his/her expected SAT I score is
(1-.4) [(1100(.3)+1000(.1))/.4]+ (.4)1170 = 1113
Model and Structural Estimation
Notation




Known to the applicant at
the time she submits her
application
Known to the applicant at
the time of enrollment
decision
μ(Xi)+εap +εen ,expected utility from attending
College X for applicant i
UR ,expected utility if applicant does not
attend College X and does not apply early
decision at College X (normalized to zero).
UR-C , expected utility if applicant does not
attend College X and does apply early
decision at College X.
εs , unobserved cost of submitting SAT I
Whether to Apply Early Decision
and/or Submit SATI Score
College’s Objective Function
To account for the college’s concern for the
quality of its current and future students and
the understanding that future student quality
depends on the college’s ranking, we allow
the college’s objective function to depend on
the perceived ability of the incoming students,
the “reported” ability of these students, and
the demographic characteristics of the student
body.
Unobserved (to us)
quality of applicant i
College accepts applicant i if:
Pe(Xi,k,l) [ΠP(X+iP)+εqi + ΠR(X+iR)+ ΠD(X+iD)]
+(1- Pe(Xi,k,l)) [ΠP(X-iP)+ ΠR(X-iR)+ ΠD(X-iD)]
>ΠP(XriP)+ ΠR(XriR)+ ΠD(XriD)
Probability i attends
(function of whether
apply early and submit)
Or
X’s are expected characteristics
of incoming class
Pa(Xi,k,l) =Prob{εqi>ΠP(X-iP)-ΠP(X+iP)+ΠR(X-iR)-ΠR(X+iR)
+ΠD(X-iD)-ΠD(X+iD)+f(YRri)-f(YRai)/Pe(Xi,k,l) }
Can relate X+ij-X-ij to Xij for j=P,R,D
College accepts applicant i if:
Pa(Xi,k,l) =Prob{εqi>βP•XiP+βR•XiR+βD•XiD+ βYR/Pe(Xi,k,l)}
What about perceived quality of applicant who
does not submit SAT I? Depends on what
college infers.
What about perceived quality of
applicant who does not submit?
College accepts applicant i if:
Pa(Xi,k,l) =Prob{εqi>βP•XiP+βR•XiR+βD•XiD+ βYR/Pe(Xi,k,l)}
Based on Eyster and Rabin for those who don’t submit,
assume SATiP = χ SATi,unconditional + (1- χ)SATi,conditional
where χ is a parameter to be estimated.
Applicant’s Decision to Enroll if Accepted


An accepted applicant who applies early
decision will enroll if μ(X)+ εap+εen> -C
An accepted applicant who doesn’t apply early
decision enrolls if μ(X)+ εap+εen> 0
(Probability that these conditions hold determine Pe(Xi,k,l).)
Estimation
Assuming εap,εen,εq,εs~N(0,1) and are
independent, can derive a maximum
likelihood function and obtain parameter
estimates of, among other things, (χ,C,βYR)
using SML.
χ =0.266 (for College X)
Future Work

Consider “Counterfactuals”
What if χ=0? How would that affect
admission decisions and quality of student
body?
 What if the College had to report their “true”
average SAT I scores to the ranking
organizations? How would that affect their
admission decisions and quality of student
body?
