Transcript The Efficient Exam

The Efficient Exam
Shlomo Yitzhaki
Hebrew University
Talk’s Structure
•
•
•
•
Characterization of Grades in Exams
Monotonic correlation
Properties of Gini’s Mean Difference
Properties of the Efficient Exam
Characterization of grades
• Grades are an Ordinal Variable
• It is as if we are measuring height of
people standing behind a screen
• We ask who has the X centimeter and
those that are taller than X respond
positively.
• Height is the number of positive answers.
• It is impossible to plot a cumulative
distribution of grades
Characterization of Grades
• If cumulative distributions of two groups
intersect, then there are two alternative
legitimate exams that will result in
contradicting ranking of average grades.
• Hence, one can improve her country
performance in international exams like
PISA, by pointing out the alternative exam.
Monotonic Correlation
• It is assumed that we are examining a unidimensional ability
• Otherwise we have to examine whether
the correlation is monotonic.
• The method to do that is based on plotting
Concentration curves (A variant of Lorenz
curve for two variables).
• The Method is already published
(Economics Letters, 2012).
Properties of GMD
• Gini’s Mean Difference can be
decomposed in a way that makes the
decomposition of the variance a special
case.
• This way one can find the implicit
assumptions behind the variance.
• Properties described in a 540 pages book
• Entitled “The Gini Methodology” by
Springer Statistics N. Y. 2013.
GMD vs. Variance
•
•
•
•
•
•
Variance = cov(X, X)
GMD = 4 cov(X, F(X))
Note that F is uniform [0, 1].
Gini covarince cov(X, F(Y)), cov(Y,F(X))
They don’t have to have the same sign.
Known in economics as “Index number
problem.
Properties of GMD
• ANOVA Translates into ANOGI
• Two correlation coefficients between two
variables two Gini Covariance, two
regression coefficients, mixed GMD-OLS
regression, etc..
• If the two correlations between two
variables are equal, then we get an
identical decomposition of the variance of
a sum of random variabes
Properties of the Efficient exam
• Because of the limited number of
questions, There is “Binning”
• Main proposition: To maximize betweengroup variability, the distribution of grades
in the “efficient exam” should be Uniform.
• No proof is presented in this talk.
A sketch of the proof
• The proof is based on the proposition that
the distribution of the cumultive distribution
is uniform [0, 1].
• Using Lorenz curve then the question is
what is the optimal size of a “bin”
• Two stages: Every “bin” should be
positive. Mid-point is optimal
Transvariation
• Two possible ways to rank groups:
• According to average grade
• According to transvariation: The probability
of a randomly selected member of the high
(low) average group to be better than the
randomly selected member of the lower
(higher) average group.
• Under efficient exam both criteria are
equivalent.
Applications
• The arguments are relevant to any test
based on ordinal variable.
• I owe this point to Emil
• This is the reason why I was invited
Thank you
• For your Patience