Transcript 4. pp slides: scales and norms

Scales, Transformations,
& Norms
Norms
Norm-Referenced Test : one of the most useful ways of
describing a person’s performance on a test is to compare
his/her test score to the test scores of some other persons or
group of people.
• Norms are average scores computed for a large
representative sample of the population.
• The arithmetic average (mean) is used to judge whether a
score on the scale is above or below the average relative to
the population of interest.
Norms (cont.)
• No single population can be regarded as the normative
group.
• a representative sample is required to ensure meaningful
comparisons are made.
• When norms are collected from the test performance of
groups of people these reference groups are labeled
normative or standardized samples.
Norms (cont.)
• The normative sample selected as the normative group,
depends on the research question in particular.
• It is necessary that the normative sample selected be
representative of the examinee and of the research
question to be answered, in order for meaningful
comparisons to be made.
• For example: tests measuring attitudes towards
federalism having norm groups consisting of only
students in the province of Quebec might be very useful
for interpretation regionally in Quebec, however their
generalizability in other parts of the country (Yukon,
Toronto, Ontario) would be suspect.
Sample Groups
Although the three terms below are used interchangeably,
they are different.
Standardized Sample - is the group of individuals on
whom the test is standardized in terms of scoring
procedures, administration procedures, and developing the
tests norms. (e.g., sample used in technical manual)
Normative Sample - can refer to any group from which
norms are gathered. Norms collected after test is published
Reference Group - any group of people against which test
scores are compared. (e.g., a designated group such as
students in 3090.03 or World Champions)
Types of Norms
Norms can be Developed:
• Locally
• Regionally
• Nationally
Normative Data Can be Expressed By:
• Percentile Ranks
• Age Norms
• Grade Norms
Local Norms
• Test users may wish to evaluate scores on the
basis of reference groups drawn from specific
geographic or institutional setting.
For Example
Norms can be created for employees of a particular
company or the students of a certain university.
• Regional & National norms examine much
broader groups.
Subgroup Norms
• When large samples are gathered to represent broadly
defined populations, norms can be reported in aggregate or can
be separated into subgroup norms.
• Provided that subgroups are of sufficient size and fairly
representative of their categories, they can be formed in terms
of:
- Age
- Sex
- Occupation
- Education Level
Or any other variable that may have a significant impact on
test scores or yield comparisons of interest.
Percentile Ranks
• The most common form of norms and is the simplest method of
presenting test data for comparative purposes.
• The percentile rank represents the percentage of the norm group
that earned a raw score less than or equal to the score of that
particular individual.
For example, a score at the 50th percentile indicates that the
individual did as well or better on the test than 50% of the norm
group.
• When a test score is compared to several different norm groups,
percentile ranks may change.
For example, a percentile rank on a mathematical reasoning test
may be lower when comparing it to math grade students, than music
students.
Age Norms
• Method of describing scores in terms of the average or
typical age of the respondents achieving a specific test score.
• Age norms can be developed for any characteristic that
changes systematically with age.
• In establishing age norms, we need to obtain a representative
sample at each of several ages and measure the particular age
related characteristic in each of these samples.
• It is important to remember that there is considerable
variability within the same age, which means that some
children at one age will perform similar to children at other
ages.
Grade Norms
• Most commonly used in school settings.
• Similar to age norms except the baseline is grade level rather
than age.
• It is important to remember that there is considerable
variability within individuals of different grade, which means
that some children in one grade will perform similar to or
below children in other grades.
• One needs to be extremely careful when interpreting grade
norms not to fall into the trap of saying that, just because a
child obtains a certain grade-equivalent on a particular test,
he/she is the same grade in all areas.
Evaluating Suitability of a Normative
Sample
• How large is the normative sample?
• When was the sample gathered?
• Where was the sample gathered?
• How were individuals identified and selected?
• What was the composition of the normative sample?
- age, sex, ethnicity, education level, socioeconomic status
Caution When Interpreting Norms
• Norms are not based on samples that adequately
represent the type of population to which the examinee’s
scores are compared.
• Normative data can become outdated very quickly.
• The size of the sample taken.
Setting Standards/Cutoffs
• Rather than finding out how you stand compared to
others, it might be useful to compare your performance on
a test to some external standard.
For Example - if most people in class get an F on a test and
you get a D, your performance in comparison to the
normative group is good. However, overall your score is
not good.
Criterion-Referenced Tests - assesses your performance
against some set of standards. (e.g., school tests, Olympics)
Cutoff Scores - 1 SD?, 2 SD?
Raw Scores
• Raw scores are computed for instruments using Likert scales
(interval or ordinal) by assigning scores to responses and totaling
the scores of the items.
- For positively phrased items, e.g., “ I think things will turn out right”
5=Always, 4=Often, 3=Sometimes, 2=Seldom, 1=Never
- For negatively phrased items, e.g., “ I think things will turn out right”
1=Always, 2=Often, 3=Sometimes, 4=Seldom, 5=Never
• The raw score would be the sum of the scores for pertinent items.
• The problem with raw scores are that they are fairly meaningless
without some sort of benchmark with which to make a comparison
(e.g., What would a raw score of 30 on an Optimism scale mean?)
Transformations
• Raw scores (i.e., simplest counts of behaviour sampled by
a measuring procedure) do not always provide useful
information.
• It is often necessary to reexpress, or transform raw scores
into some more informative scale.
•The simplest form of transformation is changing raw
scores to percentages.
For Example
If a student answers 35 questions out of 50 correctly on a test, that
student’s score could be reexpressed as a score of 70%.
Linear Transformations
• Changes the units of measurement, while leaving the
interrelationship unaltered.
• An advantage of this procedure is that the normally
distributed scores of tests with different means and score
ranges can be meaningfully compared and averaged.
• Most familiar linear transformation is the z score.
Standard Scores
• Standard scores allow each obtained score to be
compared to the same reference value.
• In order to facilitate comparison between obtained scores
and the scores of other individuals (i.e., the normative
sample), as well as comparison among the various scales
and instruments.
• Standard scores are calculated from raw scores such that
each scale and subscale will have the same mean (or
average) score and standard deviation.
For example, IQ scores are transformed so that the
average score is 100, with a SD of 15.
Z Scores
•A z-score tells how many standard deviations someone is above or
below the mean. Simply put, the mean of the distribution is given the
z value of zero (0) and its standard deviation is counted by ones.
•A z-score of -1.4 indicates that someone is 1.4 standard deviations
below the mean. Someone who is in that position would have done as
well or better than 8% of the students who took the test.
• To calculate a z-score, subtract the mean from the raw score and
divide that answer by the standard deviation. (i.e., raw score =15,
mean = 10, standard deviation = 4. Therefore 15 minus 10 equals 5. 5
divided by 4 equals 1.25. Thus the z-score is 1.25.)
Test Your Knowledge:
Problem 1 –
Assume that a test has a raw-score mean of 62 and a
standard deviation of 9. If the test-taker obtains a raw
score of 71 on the test, what would their z-score be?
Problem 2:
Test has a raw-score mean of 62 and a
standard deviation of 9. Test-taker
obtains a raw-score of 53. What
would be their z-score?
T Scores
T-Scores (or standardized scores) are a conversion
(transformation) of raw individual scores into a standard
form, where the conversion is made without knowledge of
the population's mean and standard deviation.
•
• The scale has a mean set at 50 and a standard deviation
at 10.
T = 50 + l0 x z score
• An advantage of using a T-Scores is that none of the
scores are negative.
Test your knowledge:
Johnny obtains a z-score of -2.0. What
numerical value would his score be if
converted to T-score units?
Area Transformations
• Area transformations do more than simply put scores on a new and
more convenient scale -- it changes the point of reference.
• Area transformations adjust the mean and standard deviation of
the distribution into convenient units.
• Advantages of area transformations are obvious. Out of the infinite
number of possible empirical distributions of test scores, the normal
distribution is most frequently assumed and approximated. It is also
most frequently studied, in considerably greater detail than other
possible test score distributions.
• Normalization thus allows the application of knowledge concerning
properties of standard normal distribution toward the interpretation
of the obtained scores.
Skewness
• Skewness is the nature and extent to which symmetry is
absent.
Positive Skewness - when relatively few of the scores fall at
the high end of the distribution.
For Example - positively skewed examination results may indicate
that a test was too difficult.
Negative Skewness - when relatively few of the scores fall
at the low end of the distribution.
For Example - negatively skewed examination results may indicate
that a test was too easy.
Normal Distribution Curve
• Many human variables fall on a normal or close to normal curve
including IQ, height, weight, lifespan, and shoe size.
• Theoretically, the normal curve is bell shaped with the highest
point at its center. The curve is perfectly symmetrical, with no
skewness (i.e., where symmetry is absent). If you fold it in half at the
mean, both sides are exactly the same.
•From the center, the curve tapers on both sides approaching the X
axis. However, it never touches the X axis. In theory, the distribution
of the normal curve ranges from negative infinity to positive infinity.
•Because of this, we can estimate how many people will compare on
specific variables. This is done by knowing the mean and standard
deviation.
Normal Distribution
The bell-shaped curve has the following properties:
1. bilaterally symmetrical (right and left halves are mirror images)
3. the limits of the curve are plus and minus infinity, so the tails of
the curve will never quite touch the baseline
4. about 68% of the total area of the curve lies between one
standard deviation below the mean and one standard deviation
above the mean
5. about 95% of the total area of the curve lies between two
standard deviations below the mean and two standard deviations
above the mean
6. about 99.8% of the total area of the curve lies between three
standard deviations below the mean and three standard deviations
above the mean.
Standard Deviations
The standard deviation represents the average distance
each score is from the mean.
Use of Standard Deviations with Norms:
• Knowing the average of a population allows for a determination as
to whether a particular respondent scored above or below that
average, but does not indicate how much above or below average the
score falls. Standard Deviation plays a role in this.
• Scores within 1 SD of average are pretty much in the middle cluster
of the population. Scores between 1 & 2 SDs from the average are
moderately above or below the average , and scores 2 SDs from the
average are markedly for above or below the average.