Transcript Chapter 8

Measuring Achievement
and Aptitude: Applications
for Counseling
Session 7
Definitions

Achievement tests
– Provides information about what an individual
has learned or acquired.
 Provide
information to help individuals
understand their academic strengths and
limitations.

Aptitude test
– Predict future performance or ability to learn
 Often
come to counseling because they are
trying to make a decision about their future.
Six Areas of Assessment Using
Achievement
 Survey
achievement batteries
 Individual achievement tests
– Typically cover the areas of reading,
math, and spelling

Diagnostic tests
– Diagnosis learning disabilities
– Assess achievement strengths and
limitations
Six Areas--2

Criterion-referenced tests
– Measure knowledge or comprehension to
determine if certain criterion or standard has
been met

Minimum-level skills
– Measure skills for promotion, entrance, or
graduation
– Level is established before test is administered

Subject area tests
– Example: test that covered knowledge of
assessment strategies in counseling
Achievement Battery
TerraNova





TerraNova
–
–
–
–
–
–
Administered to children K – 12.
Combination of norm and criterion referenced
Schools can select from basic battery
Schools can select format
Spanish version
Construction involved Item Response Theory
–
–
–
–
Norm-referenced
Criterion-referenced
Objective master
Performance level
–
–
–
–
National percentiles
National percentile ranges
Stanines
Grade equivalents
–
–
–
Reliability r= mid .80 to mid .90
Recently published not much on construct validity yet
Content validity
Information Provided
Provides a profile
Scores provided
Psychometric properties

Item analysis
Discussion
Would you use this test?
Provide rationale?
What do others think ?
Aptitude Assessments
Hood & Johnson (1997) argued that
counselors in a variety of fields need
to be knowledgeable about the
predominant scholastic aptitudes
tests.
 If you do not have a working
knowledge of these instruments may
not be viewed as credible.

SAT

Scholastic Assessment Test
– First adminstered in 1926 as the Scholastic Aptitude
Tests
– 1994 was revised and renamed SAT
– 2004 added an essay—writing sample

Consists of two tests
– SAT I: Reasoning test

3 hour test that measures verbal and mathematical
reasoning abilities
– SAT II: Subject tests

Consists of single subject tests
–
–
–
–
Writing
Math level 1
Biology
French
Interpretation
 Score
ranges between 200 and 800
on both the Verbal and Mathematical
sections of SAT I
– Mean is 500
– SD is 100
 ETS
(Education Testing Services)
uses complex formula to equate
scores on different versions.
Psychometric Properties of
Aptitude Tests
These tests are used to make decisions
that often have a significant influence on
people’s lives and, therefore, the validity
of these instruments deserves analysis.
 GRE

– Combined scores have validity coefficients of
.31-.37
– Undergraduate grade point averages have r =
.35-.39
– Adding the GRE and Grade point results in r =
.49 -.63
Discussion
 Would
you use this test?
 Provide
 What
rationale?
do others think?
Work Samples Assessment
Another way of assessing
vocational/career aptitudes
 Philosophical base

– Work performance can best be assessed
by using a sample of the actual work
the individual would perform
Work Sample Assessments

Valpar Component Work Sample (VCWS)
– 23 individual work samples on computer
– Criterion-referenced scoring
– Speed test (completed in certain amount of
time_
– Norms on nondisabled and disabled workers
– Comes with built in computer scoring system

SAGE system
– Non computer and computer versions
Test Preparation and Performance

Do workshops really increase scores?
– Mixed review
– Depends on individuals test taking sophistication—can they learn to
learn to meet exams requirement or test format (logical problem
solving)
– Individual who have experience in taking standardized test have a
distinct advantage (Anastasi, 1981)
– Manuals and
– sample tests have been constructed to level the playing field and
provide persons with such experiences.

Coaching
– Coaching programs do make a significant positive difference in scores
(contested but not yet disproven)
– Less expensive ways (travel experiences, tutoring, trips to museums
– The closer the coaching material to the actual test content the greater
the improvement in test scores
– The more time individuals spend reviewing the material, the
more likely it is that they will cover the material on the test.
Exam Results
2
1
0
92.25
106.5
108
116.25
Mean = 122.3
s.d. = 17.08
Mode = {106,5; 108; 150}
Median = 118.5
117
118.5
126
129
130.5
133.5
142.5
Z= data point – mean
standard deviation
150
Raw and Z-scores
Raw scores
92.25
106.5
106.5
108
108
116.25
117
118.5
126
129
130.5
133.5
142.5
150
150
Mean = 122.3
s.d. = 17.08
Z-scores
-1.75934498
-0.925046612
-0.925046612
-0.837225731
-0.837225731
-0.354210886
-0.310300446
-0.222479565
0.216624839
0.392266601
0.480087482
0.655729244
1.182654529
1.621758933
1.621758933