Transcript ACCUPLACER’S Computer Scored Essay

A Placement Validity Study
for Freshman Composition
and College Algebra
Gary Greer
University of Houston Downtown
([email protected])
Annual Conference on The First-Year Experience
February 22, 2004 1:45-2:45 PM
Addison TX
Purpose of this study



To check the validity of our cut scores.
To be able to translate our student’s
placement score into a probability that
the student will succeed.
At U. of Houston Downtown we used
grades and placement scores to check
ACCUPLACER’s accuracy in forecasting
student performance.
Setting Cut Scores

NCME recommends in The Code of Fair
Test Practices in Education that test users
monitor the behavior of cut scores in
placement. (see
www.apa.org/science/fairtestcode.html)

Documentation should include
measurement of the relation between
predictor and criterion.
Measurement




An Accuplacer placement score is a
predictor.
Course grade is the criterion.
Course grade can be predicted by
placement score.
Success is defined as a course grade B or
grade C. A probability is associated with
the grade predicted.
Samples

844 students (405 Males; 439 Females) took the
placement test and Freshman Composition.

1089 students (432 Males; 657 Females) took
the placement test and College Algebra.

Findings from these samples generalize to
future students taking Freshman Composition
and College Algebra. Probability of making a B
is related to a certain placement score.
Grade Distributions and Placement
Scores
A
Freshman
Composition
Avg SS Score
College
Algebra
Avg EA Score
B
184
270
21.8% 32.0%
C
D
F
178
21.1%
39
4.6%
173
20.5%
90.7
88.4
92.3
89
193
220
8.2% 17.7% 20.2%
161
14.8%
426
39.1%
60.6
54.9
98.5
79.4
93.3
67.9
62.3
Totals
844
100%
1089
100%
Research Questions - Freshman Composition
1.
2.
3.
What is the probability that a student with
Sentence Skill score “X” will make a B in
Freshman Composition?
What is the probability that a student with
Sentence Skill score “X” will make a C in
Freshman Composition?
What is the correlation between placement
score and course grade?
Research Questions – College Algebra
1.
2.
3.
What is the probability that a student with
Elementary Algebra score “X” will make a
B in College Algebra?
What is the probability that a student with
Elementary Algebra score “X” will make a
C in College Algebra?
What is the correlation between placement
score and course grade?
Placement Score and Probability of Success
Sentence
Sentence
Probability
Skills score for Skills for
of Success
C or better
B or better
Elementary
Algebra score for
C or better
.80
114
.75
94
.70
77
.65
62
.60
47
119
104
.55
33
96
86
.50
20
72
69
.45
6
49
51
.40
25
33
.35
1
15
Elementary
Algebra score for
B or better
103
.30
72
.25
39
Decision Probabilities


To both maximize correct decisions and
minimize incorrect decisions, place students
who have probability of success>.50 (1
success out of 2)
Or choose .67 to get 2 successes out of 3 or
.75 to get 3 successes out of 4
True and False Decisions


Correct Decisions determined by
• Placing a student into a course who
succeeds – true positive
• Denying placement to a student who
would not succeed – true negative
Incorrect Decisions determined by
• Placing a student into a course who will
not succeed – false positive
• Denying placement to a student who
would succeed – false negative
Biserial Correlations and % of Correct Placements
Correlation
% correctly
% correctly
between
Predictor
placed for C or placed for B or
predictor and
higher
higher
criterion
Sentence
.09
75
Skills
Sentence
.05
54
Skills
Elementary
.14
61
Algebra
Elementary
.09
74
Algebra
Conclusions

We present statistics of Accuplacer’s placement
accuracy to faculty every three years.

We are satisfied when faculty are satisfied.

Data collection and submission for a V study has
never been easier.
Thank You