Transcript Using Data to Improve Student Learning in the Classroom

Making the Connection:
How to Use Assessment to
Increase Learning
The Oregon DATA Project
“As to methods, there may be a million
and then some, but principles are few.
The man who grasps principles can
successfully select his own methods.
The man who tries methods, ignoring
principles, is sure to have trouble.”
Ralph Waldo Emerson
Objectives
 Establish the intended connection
between testing and teaching.
 Compare current assessment practices to
those recommended for identifying student
needs, learning goals, and subsequent
instructional strategies.
 Practice linking analysis of assessment
results with instructional planning to
increase learning.
Formative Assessment
Living Likert:
 Read the following statement.
 Stand at your place along the line.
Strongly agree -------> Strongly Disagree
Evidence exists that
formative assessments
positively impact student
achievement.
Does formative assessment lead to improved
student achievement?
 Most frequently cited analysis is Black and Wiliam,
1998.
 Looked at classroom formative assessment practices
defined as point-in-time daily and weekly assessment of
student learning in the classroom.
 Consisted of informal and formal observation,
questioning, quizzes, rubrics, unit exams, etc.
 Curriculum based measurement (CBM) and progress
monitoring were studied for groups needing intensive
intervention or belonging to special populations.
Unpacking formative
assessment
Where the learner is
going
Teacher
Clarifying, sharing
and understanding
learning intentions
Peer
Learner
Where the learner is
Engineering effective
discussions, tasks, and
activities that elicit evidence
of learning
How to get there
Providing feedback
that moves learners
forward
Activating students as learning
resources for one another
Activating students as owners
of their own learning
7
And one big idea
Where the learner is
going
Teacher
Peer
Where the learner is
How to get there
Using evidence of
achievement to adapt what
happens in classrooms to
meet learner needs
Learner
8
Evaluating the Evidence –
John Hattie (2009) meta-analysis of over
800 studies...
Teacher – Student relationships - .72
Professional development (as a PLC)- .63
Effective Teacher Instruction- .75
Vocabulary programs - .67
Assessment as Formative Feedback - .73
The CCSS require a paradigm shift
toward assessment as a multi-faceted
process of formative assessment
development.
How will you know if each student is
learning, becomes a significant
question for you and the collaborative
team.
Tim Kanold, 2012
Definitions
Assessment refers to all those activities that
provide information to be used as
feedback to modify teaching and learning
activities.
Such assessment becomes formative
assessment when the evidence is actually
used to adapt the teaching to meet student
needs.
Black and Wiliam
Balancing “of” and “for”
Assessment for learning to make
instructional decisions and monitor
student progress; i.e., formative.
Assessment of learning to evaluate
students’achievement and overall
program effectiveness; i.e., summative.
Key Ideas
• Formative assessment is much closer to instruction
than to assessment.
•Formative assessment is something you DO,
not something you GIVE.
•You have done formative assessment when you can
answer the question: “Does each of my students
know and can they do what I expect them to have
learned today?” with evidence.
At the heart of assessment:
 We use tests or assessments to collect
overt (visible) evidence to make
inferences about covert (unseen) status
of student skills and knowledge.
 Restated-we use a limited sample of test
items so that we can generalize student
performance on a content standard.
 Different ways of interpreting and
operationalizing content standards.
These may require different types of
instruction.
Assessment for the
sake of assessment?
Or,
Assessment with a purpose!
Testing/Teaching
Connection:
What are the advantages of
connecting testing and teaching?
Classroom use of assessments:
 Decisions about curriculum alignment
 Decisions about students’ prior
knowledge
 Decisions about how long to teach
something
 Decisions about effectiveness of
instruction
Use of assessment What assessments do
you have in place?
Decisions about
curriculum alignment
Decisions about students’
prior knowledge
Decisions about how
long to teach something
Decisions about
effectiveness of
instruction
Frequency
What happens when we are not
clear on the standard or
common curricular goal?
What is the target?
Discuss
How is Test-triggered Clarity
different from
“Teaching to the Test”?
Advantage of using tests to clarify curricular
goals:
 More accurate task analysis-what are my
students expected to know and do?
 When you begin with the end in mind you
have a better chance of getting there!
 Can identify “enabling subskills” or
“enabling knowledge”
AKA unwrapping the standards!
 Clearer instruction and explanations
 More appropriate practice activities
Curriculum Alignment Decisions
 To whose interpretation are you teaching?
 Clarify expectations using
 Sample Tests, Scoring Guides and Work
Samples
 Test Specifications and Blueprints
 Smarter Balanced Assessment
 Augment OAKS results with local
assessment results to clarify alignment
within and across grade levels
http://www.azed.gov/standards-practices/mathematics-standards/
________________________________
Grade Level
Power Standard:
!
!
!
!
Participating Collaborators:
(Simple)
ELA Rdg. Lit
ELA Rdg. Inf. Text
ELA Spk./Listen.
ELA Language
Building:
Date:
Big Idea:
Power Standard:
!
Unwrapping Template (Simple)
Essential Questions:
Big Idea:
Power Standard:
SKILLS (verbs)
Essential Questions:
CONCEPTS (Nouns)
Students will know:
Bloom’s Level
(place(Place
Bloom’s
level Level
after each
verb verb
Bloom’s
after each
phrase:
See! )
phrase)
SKILLS (verbs)
And will be able to:
(place Bloom’s level after each verb
phrase: See )
Bloom’s Level
Higher-order thinking
CONCEPTS (Nouns)
Big Idea:
What does this standard require students to…
Students will know:
And will
be able to:
Students will KNOW? (Concepts)
Be able to DO? (Skills/Reasoning)
Level 6: Creating
(Putting togethe r ideas
or elements to develop
an original idea o r
engage in creative
thinking).
Common verbs:
Generate, hypothesize,
plan, design, pro duce,
construct, formulate,
invent, transform,
Level 6: Creating
originate
(Putting together
Level 5:ideas
Evaluating
or elements to develop
(Judging
the value of
an original idea or
materials and
engageideas,
in creative
methods by developing
thinking).
and
applying
standards
Common verbs:
andhypothesize,
criteria).
Generate,
Common
verbs: Check,
plan, design,
produce,
detect,
monitor, j udge,
construct,
formulate,
appraise,
invent, critique,
transform,
defend, justify, assess,
originate
convince,
Levelprioritize,
5: Evaluating
discriminate,
(Judging the
value
Level 4: of
Analyzing
ideas, materials and
(Breaking
inform ation
methods by developing
down into
its component
and applying
standards
elements).
and criteria).
Common
Common
verbs: verbs:
Check,
det ermine,
detect, Differentiate,
monitor, judge,
discriminate,
critique, appraise, distinguish,
integrate,
defend,organize,
justify, assess,
outline,
solve,
prioritize,
convince,
experiment, investigate
discriminate,
3: Applying
Level 4:Level
Analyzing
Higher-order thinking
!
______________________________
Standards:
Content Area
Unwrapping
Template
! Math
What could a sample test item look like?
Where does this align with our curriculum?
Capistrano Unified School District
Lower-order thinking
NOTES to record while you work…
What are some possible learning targets?
Lower-order thinking
(Using
strategies,
(Breaking
information
concepts,
principles and
down into
its component
theories in new
elements).
situations).
Common
verbs:
Common
verbs: apply,
Differentiate,
determine,
execute,
implement,
discriminate,
distinguish,
construct,
model,
organize,
integrate,
outline,display,
solve, illustrate, carry
out, investigate
experiment,
Level
2: Understanding
Level
3: Applying
(Understanding of given
(Using strategies,
information).
concepts,
principles and
theoriesCommon
in new verbs:
Summarize, clarify,
situations).
paraphrase,
represent,
Common
verbs: apply,
translate,
exemplify,
execute,
implement,
construct,
illustrate,
model,categori ze,
display,match,
illustrate,
carry
distinguish,
out,
conclude, reorga nize
Level 2:Level
Understanding
1: Remembering
(Understanding
given ition of
(Recall orofrecogn
information).
specific information).
Common
verbs: verbs: tell,
Common
Summarize,
clarify,label, recite,
list, define,
paraphrase,
recall,represent,
retrieve, record,
translate,
exemplify,
relate,
recogni ze,
illustrate,
categorize,
identify,
describe,
match, examine,
distinguish,
locate, match,
conclude,
say,reorganize
show, tell, write.
Level 1: Remembering
(Recall or recognition of
specific information).
Common verbs: tell,
list, define, label, recite,
recall, retrieve, record,
relate, recognize,
identify, describe,
examine, locate, match,
say, show, tell, write.
Using Blueprints
Adapting the use of the Ishikawa
Ishikawa
Fishbone
Analysis:
Looking for
related
causes and
antecedents
that may
relate to a
specific effect
or outcome
Ishikawa Fishbone: Cause & Effect
Diagram Modified for Task Analysis
Content/skills
Content/skills
Student Engagement
Tasks
Student Engagement
Tasks
Standard
Student Engagement Tasks
Content/skills
Student Engagement Tasks
Content/skills
So many targets, what
is a teacher to do?
What about generalizability?
 Scenario 1, page 24
 Clarify nature of curriculum content standard by
analyzing measures used to assess standard
 Look at various ways it is assessed
 Teach toward the skills or knowledge a test represents,
not toward test itself.
 Extend, apply, etc. for generalizability (for more on this
tab pages 23-25)
Prioritize and align!
 Objectives for instruction
 content and skills you
plan to teach
 Actual instruction that
preceded assessment
 content and skills you
actually taught
 Decisions or conclusions
you plan to make using
interpretation of resulting
scores
Curriculum
Instruction
Evaluate/Assess
Advantages of assessing prior learning:
 Economizes instructional planning
 Many standards, not enough time,
 teach what is needed, not what is already known by
students
 Gives teacher the lay of the landscape
 Diversity of learners
 Diversity of prior knowledge/readiness to learn
 Provides connections from which to build
new knowledge and skills when you include
key enabling skills and subskills or bodies of
knowledge
Assessing prior knowledge:
It’s more than just a pretest
Brainstorm with your team a quick list
of pre-assessment strategies.
Be prepared to share at least 2!
Simple, but powerful model…
Pre-Test
Data
?
Instruction
Post Test
Data
Instruction
Post-Test
Data
Do you use assessment practices to
determine how long to focus on a particular
set of objectives?
 Economizes instructional planning
 Move on when students are ready, not when
the unit planner indicates
 Many standards, not enough time, “steal” back
time where possible
 Time saved in an easily mastered unit can be
used for units with unexpected difficulty
The Dipstick Assessment:
How long do I need to teach this set of
skills/concepts?
 Item-sampling method for quick
assessment
 Different students complete different
subsamples of items from your unit test (a
couple of items each)
 Takes less than five minutes to administer to
students
 Gives quick fix on status of entire class—not
intended for inferences about individual
students
Using tests to determining
instructional effectiveness:
 Use classroom assessment to evaluate your
own instructional effectiveness
 Use cohort and growth from OAKS to triangulate
on instructional effectiveness
 Strong inferences come from simple model:
 Pretest
 Posttest
 Compare results
Simple, but powerful model…
Pre-Test
Data
?
Instruction
Post Test
Data
Instruction
Post-Test
Data
These concepts can be integrated into your
action research and data teams processes
Data Teams Process:
 Examine student work collaboratively
observe, hypothesize, predict
 Develop interventions hypothesize,
predict
 Adjust teaching strategies test
hypothesis
 Monitor results gather data, explain,
observe
Testing/Teaching
Connection:
How will you alter instruction
as a result of what you’ve
learned through assessment?
At the heart of a lesson plan
 Standard(s) stated in student language
 Instructional objectives
 Time allotment
 Materials and resources needed
 Outline of content
 Outline of procedures (teacher behaviors and
expected student actions)
 Plan for assessing objectives
Identify the content and
skills you plan to teach, and
therefore, assess
Instruction
What do you expect students to
know and be able to do as a
result of instruction?
Align with OR content
standards/ CCSS/ Diploma
Essential Skills
Responsive to student’s
readiness, identified
strengths and weaknesses
Plan instructional activities
and student engagement
strategies
What will you do to
teach concepts and
skills?
What will students
do to engage in
learning?
Engage in instruction
Assess instruction &
learning
Assessment
How do you expect
students to
demonstrate their
learning?
Identify content/skills
and expected
cognitive demand
Determine item
formats &
Develop items
Develop scoring
and/or rubrics
What do you expect students to know
and be able to do? How do you expect
them to demonstrate it?
Use the standards language to
determine content and skills:
 nouns identify content
 verbs identify skills & level of
cognitive demand
Identify the content and skills
you plan to teach, and
therefore, assess
What do you expect students
to know and be able to do as a
result of instruction?
Align with OR content
standards/
CCSS/Diploma Essential
Skills
Responsive to student’s
readiness, identified
strengths and weaknesses
How do you expect
students to demonstrate
their learning?
Identify content/skills
and expected
cognitive demand
Plan instructional
activities and student
engagement strategies
What will you do to
teach concepts and
skills?
What will students
do to engage in
learning?
Engage in instruction
Assess instruction &
learning
Determine item
formats &
Develop items
Develop scoring
and/or rubrics
Use a data team process to analyze classroom
level data for patterns and outliers.
Notes on patterns of performance
Exceeds
Meets
Nearly
Meets
Far
To Go
Notes on outliers
How does pattern analysis apply to
instructional/assessment planning?
Notes on patterns of performance
Exceeds
Look at curriculum alignment issues that
may be associated with weaknesses
Extend learning deep into higher levels
of Bloom’s
Meets
To move these students, work on
generalizability—increase student
engagement and use deeper levels of
Bloom’s
Nearly
Meets
Increase student engagement in grade
appropriate content and skill-based
activities
Far
To Go
Use explicit instruction in terms of how
one concept or skill builds to the next,
can’t assume anything in terms of how
these students connect! Model
connections and keep them engaged!
Notes on outliers
Connecting Assessment to Instruction
 Identify a student from each group whose responses or
performance are representative of the group’s
performance
 Describe each representative performance
 Describe each student’s learning needs based on this
detail of performance
 Determine how you will differentiate instruction
 Deal with the students whose performance in a category
is fuzzy, needs more data to determine
Describe representative performance
Far Below
Nearly
Meets
Meets
Exceeds
Bill
Destiny
Jeremy
Ashley
Drew vertical and
horizontal axis
No labels or scale
No title
Counts for 4 of 5
categories were
incorrect
No sign of a
counting or
frequency strategy
for determining the
bar heights
Chart included
labeled horizontal
and vertical axis
Chart include a
general title
Counts for 2 of 5
categories were
incorrect
Addition used to
tally or count, but
errors in addition
that determined bar
heights
Chart was labeled
appropriately for
vertical and
horizontal axis
General title
included
One category had
errors in count
Evidence of how
student arrived at
bar heights correct
for 4 of 5 categories
Chart was labeled
appropriately for
vertical and
horizontal axis
Specific title
No errors in counts
for categories
Evidence of
strategy to
determine bar
height, no errors in
calculations
Describe learning needs based on representative
performance and curricular learning goals
Far Below
Nearly Meets
Meets
Exceeds
Bill
Intensive instruction
using modeling to teach
tallying strategies
Guided practice in
tallying using simple
scenarios
Follow up with practice
in grade level
expectations of
tallying/counting/calculat
ing as appropriate
Step by step linear
instructions for creating a
complete graph or chart
Modeling of the step by
step
Destiny
Jeremy
Student review of
calculations to find error
and correct.
Model creating charts
from different scenarios
that require student to
apply the skill and
knowledge in different
ways or to create
different types of charts
Provide guided practice
with different scenarios
that require student to
apply the skill and
knowledge in different
ways or to create
different types of charts
Ashley
Model analyzing
multiple representations
of the data
Identify elements that are
common (what are the
requirements to
appropriately represent
the data)
Provide scenarios that
allow extensions for
students to develop one
or two more visual
representations.
Challenge them to
include essentials and to
think creatively about
how they can enhance the
interpretability of the
charts they create.
Specific instruction to
correct errors in tally
and count strategies
Brief guided practice
with tally and count
strategies
Independent practice
tallying and counting
Step by step (linear)
instructions for
creating a complete
graph or chart to use
for a reference
Determine differentiated learning
strategies
Far Below
Nearly Meets
Meets
Exceeds
Planning Instructional Strategies:
Linking Strategies to Assessment
 Given what you expect students to know and be
able to do, what strategies will you use for
instruction?
 How will you know that both adults and students
are implementing strategies?
 What evidence from teachers (lesson plan and
assessment)?
 What evidence from students (engagement and
assessment)?
 This is where released items can help you target
expected actions.
Identify the content and skills
you plan to teach, and
therefore, assess
What do you expect students
to know and be able to do as a
result of instruction?
Align with OR content
standards/CCSS/Diplom
a Essential Skills
Responsive to student’s
readiness, identified
strengths and weaknesses
How do you expect
students to demonstrate
their learning?
Identify content/skills
and expected
cognitive demand
Plan instructional
activities and student
engagement strategies
What will you do to
teach concepts and
skills?
What will students
do to engage in
learning?
Engage in instruction
Assess instruction &
learning
Determine item
formats &
Develop items
Develop scoring
and/or rubrics
Identify instructional activities and
student engagement strategies
Instructional Activities
What will the teacher do to
facilitate learning?
How will you monitor
teacher activity?
Student Engagement
Strategies
What will the student do as
an active participant in
his/her learning?
How will you know students
are engaged?
Knowledge
ELA Action Words
Math Action Words
Define, describe, identify, label,
list, name, read, reproduce, state,
view
Define, identify, name, select, state,
order, what is?
Comprehen Describe, explain, cite, discuss,
express, report, restate, review,
sion
Convert, estimate, explain, express,
factor, generalize, give example,
identify representation of concept
Application Demonstrate, dramatize, illustrate,
sketch, write
Apply, choose, compute, employ,
interpret, graph, modify, operate,
plot, practice, solve, use
Analysis
Analyze, assemble, categorize,
compare, contrast, criticize,
diagram, distinguish, focus,
outline, question
Compare, contrast, correlate,
differentiate, discriminate, examine,
infer, maximize, minimize,
prioritize, subdivide, test
Synthesis
Anticipate, collaborate,
communicate, compose, contrast,
create, devise, express, facilitate,
write
Arrange, collect, construct, design,
develop, formulate, organize, set
up, prepare, plan, propose
Evaluation
Argue, choose, compare, decide,
defend assertion, decide, judge,
support
Appraise, assess, defend estimate,
evaluate, judge, predict, rate,
validate, verify
translate, interpret, summarize
How do you gauge and compare
implementation (what you do) and
outcomes (what students do as a result
of your actions)?
Objectives
 Establish the intended connection
between testing and teaching.
 Compare current assessment practices to
those recommended for identifying student
needs, learning goals, and subsequent
instructional strategies.
 Practice linking analysis of assessment
results with instructional planning to
increase learning.
Personal Commitment
• Now that I KNOW where my students are
progress wise, how I will CHANGE my
instructional practice to more effectively
improve my students’ learning?
• How will I monitor results and make
adjustments?