Transcript Scale Scoring - HRSBSTAFF Home Page

Scale Scoring
A New Format for
Provincial Assessment Reports
http://www.ela.ednet.ns.ca
Agenda
9:00 - 10:15 Introduction to Scaled Scores
10:15 -10:30 Break
10:30 -12:00 Scaled Scores continued
LSP website
Using the Results
12:00 -1:00 Lunch
1:00 - 3:00
LSP writing
Beginning in the 2007-2008 school year, all
provincial assessments will be reported on a
common scale.
Why Use a Common Scale?
Imagine trying to accurately compare a
student’s or a school’s achievement when
given the following raw scores:
 letter grades
 numerical marks
 rubric levels
 percentage grades
What is a Scale?
A scale is an arbitrarily established set of numbers
used for measurement.
Different scales may be used to measure the same
thing, but they are calibrated differently. (e.g. rulers,
thermometers, speedometers, bathroom scales)
What is a Scale?
A scale, simply put, is a mathematical
conversion of raw scores to a
common scale or a derived score.
A scaled score is a conversion of a student's raw
score on a test to a common scale that allows for a
numerical comparison between students.
Provincial assessments use multiple versions of a test
over the years. The scale is used to control slight
variations from one version of a test to the next.
Scaled scores, when equated, are particularly useful
for comparing test scores over time.
Equating: A Statistical Process
Equating measures the difficulty of each assessment (content,
cognition, difficulty) and adjusts the cut score to
account for differences among assessment forms.
The difficulty level of tests in different years can be
equated and this enables us to compare assessment
results from year to year.
Advantages of Scale Scores
Scaled scores allow better comparison of assessment
results:
• from year to year
• from one grade level assessment to another
• of student achievement in different subject areas
Future Advantages
Because each successive year’s assessments are
statistically equated with a baseline reference year,
the scaled scores can also be used to reflect
growth.
For example, if a future scaled score is higher than
the reference year scale score (2007-2008), it can
be interpreted as an improvement from that base
year.
Cautions for Comparisons
Many factors impact upon a school’s results.
No single assessment result tells the whole story of a
student’s or school’s achievement.
Sample size does matter! Schools with fewer students will
show more variability in scores from one year to the next.
Scale Scoring
Cut score
Provincial
Mean
One
standard
deviation
above
65 %
of students who wrote
this assessment
One
standard
deviation
below
200
300
384
400
500
Scale Scores
600
700
800
Things to Consider
Distribution of Scores:
- Scale ranges from 200 to 800
- Provincial mean is set to 500
- Provincial standard deviation is set to 100
- On most provincial assessments, about 65% of
students will place between 400 and 600
Cut Score
A cut score defines the point at which a
certain required level of achievement has been
demonstrated on the assessment.
Cut scores are set by a panel of educators
who look at the actual test questions to
determine levels of acceptable performance.
A cut score can be above or below the mean.
Cut Score
A cut score differentiates between students who are
meeting expectations
or
not yet meeting expectations.
Things to Consider
The cut score for the ELA was set at 384
Higher scores mean stronger performance relative to
other students who wrote the same assessment.
Interpreting Results
Overall Score
Cut Score:
How close or how far away from the cut score (384)
did the student perform?
Did the student meet expectations for this assessment or has
the student not yet met expectations for this assessment?
REMEMBER: The cut score only applies to the overall score
(section 1) the student received.
Using the Results
Overall Score
Students Meeting Expectations (385 and above)
If the overall score is above the cut score (384) the student is
meeting expectations.
Benchmarks have been established to help determine the
level of performance.
If a student’s overall score falls between 385 - 600, his/her
performance is considered to be satisfactory.
If a student’s overall score falls between 601- 800, his/her
performance is considered to be stronger.
Using the Results
Overall Score
Students Not Yet Meeting Expectations
(Overall Score below 384)
For every student who is not yet meeting expectations (overall score is
below 384) teachers will be expected to:
• provide on-going, in-class support
• contact parents/guardians to discuss support options
Using the Results
Overall Score
Students Not Yet Meeting Expectations (Overall Score below 384)
If a student’s overall score falls below 384, his/her score is more than one standard
deviation from the mean. Therefore, he/she will require the most support and formal
tracking of that support.
For every student who is not yet meeting expectations and whose score is below 384,
the teacher is expected to:
•
•
Develop a Literacy Support (LS) document
Support student within the class structure
The LS document will be on-going for students, with transition meetings at the end of
each grade.
Interpreting Results
Overall Score
Provincial Mean (500)
How close or how far away from the mean did the
student perform?
How did the student perform relative to others that wrote
the assessment?
How does the student’s performance relative to others
compare from subject to subject?
How did the student perform this time compared to the
time before?
Using the Results
The Reading Performance section:
• Provides information on student performance in
the various genres, cognitive levels and some
specific outcomes .
• Allows for comparison to the provincial mean of
500.
• May help teachers isolate concerns and
develop strategies to address these indicators.
• was not factored into a student’s overall score
• is intended to provide to teachers information
for planning purposes only.
Using the Results
The Writing Performance section:
• Provides information on student performance in
the elements of writing: ideas, organization,
matters of correctness
• May help teachers identify concerns and trends.
Using the Results
Schools are responsible for tracking Literacy Support
(LS) and for monitoring support.
Teachers and Principals are expected to:
• review LS document and monitor support
• ensure parent/guardian signatures on the LS
document
• ensure that the completed LS document is kept in
the student’s cum file
Using Your Data: Classroom Level
Teachers are encouraged to use the student reports
to identify trends and patterns in literacy
development.
Class / grade profiles may be used for classroom
instructional decision making and for targeting
support to individual students.
Using the Results: School Level
In April, The Minister of Education will release the
provincial, board and school results in the Minister’s
Report to Parents.
Using the Results: School Level
Next Steps:
• Principals have been asked to verify with the
DOE that the information sent to them is
accurate.
• NOTE: Assessment information is now sent to
principals electronically via the Student Records
website. The blue forms will no longer be sent
to schools.
• Print Report for School Administration and
Teachers. Put results for students who do not
meet expectations in the blue pocket folders and
place them in the cum file.
• Parent Reports will be sent to schools once
verification with the DOE is complete. Please do
not send home the school reports.
• Gather classroom based assessment
information to support LSP writing.
• Commence LSP writing using the electronic
template.
IMPLEMENTATION DEADLINE: APRIL 30, 2008
• In May, there will be a random Audit of LSPs at
the grade 6 level.
• Late May / June: complete the Student Progress
and Transition sections of the LSP and meet
with Junior High teachers.
• Print a hard copy of the updated LSP and place
it in the blue pocket folder in the cum file.
• Forward the electronic templates to the Junior
High school.
http://hrsbstaff.ednet.ns.ca/lsp/
Let’s walk through
the set up of
a Literacy
Support Plan
www.hrsbstaff.ednet.ns.ca/lsp
For more information
Program of Learning Assessment for Nova
Scotia (PLANS), Nova Scotia Department of
Education, Provincial Assessment Reports: A
New Format (2007)
http://www.ela.ednet.ns.ca