Transcript Using Results from a Practice Test in Data Directorx

Using Results from a Practice Test in
Data Director to impact SAT
Instructional Decisions
PSAT/NMSQT and SAT Practice Tests
Emily McEvoy, Assessment Consultant, MISD
November 17, 2015
KEY DELIVERABLES
• Receive a foundational understanding of the SAT Suite of Assessments and
the key changes to the redesign
• Be able to interpret raw score conversions to a test score, section scores,
sub-scores and cross section scores
• Learn to use the District Assessment and Item frequency reports in Data
Director to identify the student successes and challenges
• Identify current instructional strengths and alignment to the SAT.
FOR OUR TIME TOGETHER
• Take responsibility for your own learning
• Support the learning of your colleagues
• If you need to take a call or have a side conversation, please step out in the
hall
• If you have a question, please ask
For K–12 Students, Educators, and Parents
Not for MI
MI State Assessments
SPRING TEST DATES
11th Grade
April 12th - SAT w/ essay
April 26th - Make-up date
April 13th - WorkKeys
April 27th – Make-up date
April 11th – April 29th (3 weeks) - M-Step
9th & 10th Grade
April 12th or 13th – PSAT
April 26th or 27th – Make-up date
PSAT AND SAT PRACTICE TESTS
Number Grid Responses
• Data Director recognizes ONLY 4
digit answers as correct.
• I.E. 300 and 0300 are not the
same answer or 2 and 0002
• Make sure to use the number
grid directions attached to the
practice tests in Data Director
FOCUS READ
Individually read Section 2: Using SAT Suite of Assessments Scores and
Reporting to Inform Instruction and Appendix: Standard Reports Explained
In the reports, identify:
√ = pertinent information
! = Important for teachers
? = Hmmm... I have questions about this
TABLE DISCUSSION
Discuss your thoughts about the reports with your table.
At your tables, select important elements from the reports at a building level.
Make sure to take a break too!
Elements we can’t recreate as of now, Nov 17th
• Instructional Planning Report
• Three performance groups related to the benchmarks
• Cohort Longitudinal Report
• Benchmark by Institution Report or Demographic
• Score Trends by Institution Report or Demographic
Important Elements from the Building Reports we
can create using the Data Director Results
• Overall Test Score
• School Mean and SD
• County Mean and SD
• Section Scores
• Evidence-based Reading and
Writing Mean and SD
• Math Mean and SD
• Test Scores
• Reading Mean and SD
• Writing Mean and SD
• Math Mean and SD
* SD is Standard Deviation
• Question Analysis
• Student responses
• Difficulty level
• Related cross-test scores and
subscores
How to find the converted scores in Data Director?
MEAN AND STANDARD DEVIATION
https://www.youtube.com/watch?v=cgxPcdPbujI
FIND THE STANDARD DEVIATION
Macomb County
PSAT Practice Test #1
• Plot all of the calculations on the bell
curve.
• -2, -1, Mean, 1 and 2
• Where on the scale of scores would the
curve fall?
Mean
400
590
724
-133
857
-133
990
+133
1123
+ 133
1600
ANOTHER EXAMPLE
Macomb County
PSAT Practice Test #1
• Plot all of the calculations on the bell
curve.
• -2, -1, Mean, 1 and 2
• Where on the scale of scores would the
curve fall?
Mean
Plot and answer the following for all scores:
• Describe the spread of scores.
• Where are most students falling on the
scale?
200 273
432
352
- 79
-79
511
+79
590
+ 79
800
LUNCH
Important Elements from the Building Reports we
can create using the Data Director Results
• Overall Test Score
• School Mean and SD
• County Mean and SD
• Section Scores
• Evidence-based Reading and
Writing Mean and SD
• Math Mean and SD
• Test Scores
• Reading Mean and SD
• Writing Mean and SD
• Math Mean and SD
* SD is Standard Deviation
• Question Analysis
• Student responses
• Difficulty level
• Related cross-test scores and
subscores
In Data Director:
Which pre-built reports could I use?
DISTRICT ASSESSMENT REPORT
Prebuilt report in Data Director
Using the Raw Scores
DISTRICT ASSESSMENT REPORT
Page 1 and 2
Please NOTE:
• District means
County
• There are items on
both assessments
that contribute to
the scores for the
Analysis of Science
and His/S.S.
DISTRICT ASSESSMENT REPORT
Page 1 and 2
Please NOTE:
• District means
County
• There are items on
both assessments
that contribute to
the scores for the
Analysis of Science
and His/S.S.
PSAT Practice Test #1
Total Number of Items
Test Section, Subscore, Content Dimension and Item Difficulty
The rest of the summaries are in your handout
SAT/PSAT
Subscores, Domains and Content Dimensions:
Reading
Writing and Language
• Subscores
• Content Dimension
• Words in Context
• Command of Evidence
• Content Dimension
• Expression of Ideas (subscore)
Math
• Information and Ideas
• Domains (subscores)
• Rhetoric
• Heart of Algebra
• Synthesis
• Problem Solving and Data
Analysis
• Passport to Advanced Math
• Additional Topics (only SAT)
• Development
• Organization
• Effective Language use
• Standard English Conventions
(subscore)
• Sentence structure
• Conventions of Usage
• Conventions of Punctuation
Identify a high and low performing Content Domain or Dimension
using the District Assessment Report (pg. 1 and 2)
Select the low performing items
in the low performing content
domain/dimension chosen.
DISTRICT ASSESSMENT REPORT
RESPONSE FREQUENCY
Select the high performing items
in the high performing content
domain/dimension chosen.
DISTRICT ASSESSMENT REPORT
RESPONSE FREQUENCY
USING THE RESPONSE FREQUENCY
In the low performing Content Domain or Dimension:
Identify 10 questions within the content domain or dimension chosen where the
majority of the students answered incorrect. This is when one of the other answer
choices (not the correct answer) has a higher percent of students selecting it.
Steps:
• Use the test booklet to review the test questions.
• Use the distractors to identify student misconceptions.
• Why did students choose B over D (correct answer)?
• Draw conclusions about why the students' performance was low on the item.
• Stay away from items students guessed on (responses will be evenly distributed
among answer choices).
• Make sure to pick the best questions to analyze so that conclusions can be
drawn.
USING THE RESPONSE FREQUENCY
Do the same for the higher performing Content Domain or Dimension
• Identify 10 questions within the content domain or dimension chosen where
the majority of the students answer correctly.
• What strengths are students able to demonstrate on these questions?
• Share common patterns and themes across the low performing items vs. the
higher performing items
• Next Steps: How will this analysis effect what we do in the classroom?
instruction? Assessment structure/format? etc.