Transcript PPT - Dr. Karin Hess - High Schools That Work Ohio

Are your students thinking
more deeply or just working
harder?
Infusing Rigor into HighQuality Instruction &
Assessment
Dr. Karin Hess [email protected]
1
www.karin-hess.com
Session Overview
O Develop a shared understanding of the concept of
cognitive rigor
O Dispel some common DOK misconceptions
O Use DOK & the Hess Cognitive Rigor Matrices to:
O Examine what rigor/DOK looks like in action
O Sample videos, assessments/tasks, & rubrics
O Karin’s Coaching Tips: Analyzing assessment
tasks/rubrics & instruction in light of current
research
Provide tools & strategies for current & future work at
www.karin-hess.com - Event Materials – Topic #1 –
Password: _______________________
O
Source: Teaching for Rigor: A Call for a Critical Instructional Shift
Marzano & Toth (March 2014)
3
Track your reflections as we work…
Ways I am refining my
thinking about
DOK/rigor…
O ?
O ?
STRATEGIC scaffolding
strategies for getting
students to deeper
thinking…
O ?
O ?
Before we begin…
O Take a minute to jot down 2-3
words/phrases that come to
mind when you think of
“cognitive rigor” as it relates
to instruction, learning,
and/or assessment.
The Hess Cognitive Rigor Matrix
integrates Bloom + Webb
Different states/schools/teachers use
different models to describe cognitive rigor.
Each addresses something different.
O Bloom – What type of thinking (verbs) is
needed to complete a task?
O Webb – How deeply do you have to
understand the content to successfully
interact with it? How complex is the content?
6
Bloom’s Taxonomy [1956] &
Bloom’s Cognitive Process Dimensions [2001]
Knowledge -- Define, duplicate,
label, list, name, order, recognize,
relate, recall
Remember Retrieve knowledge from
long-term memory, recognize, recall,
locate, identify
Comprehension -- Classify, describe,
discuss, explain, express, identify,
indicate, locate, recognize, report,
review, select, translate
Understand -- Construct meaning,
clarify, paraphrase, represent,
translate, illustrate, give examples,
classify, categorize, summarize,
generalize, predict
Application -- Apply, choose,
demonstrate, dramatize, employ,
illustrate, interpret, practice, write
Apply -- Carry out or use a procedure
in a given situation; carry out or use
Analysis -- Analyze, appraise, explain
calculate, categorize, compare,
criticize, discriminate, examine
Analyze -- Break into constituent
Synthesis -- Rearrange, assemble,
collect, compose, create, design,
develop, formulate, manage, write
Evaluate -- Make judgments based
on criteria, check, detect
inconsistencies/fallacies, critique
Evaluation -- Appraise, argue,
assess, choose, compare, defend,
estimate, explain, judge, predict, rate,
core, select, support, value
Create -- Put elements together to
form a coherent whole, reorganize
7
elements into new patterns/
structures
/apply to an unfamiliar task
parts, determine how parts relate
Webb’s Depth-of-Knowledge Levels
O DOK-1 – Recall & Reproduction - Recall of a fact, term,
principle, concept, or perform a routine procedure
O DOK-2 - Basic Application of Skills/Concepts - Use of
information, conceptual knowledge, select appropriate
procedures for a task, two or more steps with decision points
along the way, routine problems applying 2+ concepts,
organize/display data, interpret/use simple graphs
O DOK-3 - Strategic Thinking - Requires reasoning, developing a
plan or sequence of steps to approach problem; requires
some decision making and justification; abstract, complex, or
non-routine; often more than one possible answer or
approach
O DOK-4 - Extended Thinking - An original investigation or
application to real world; requires time to research, problem
solve, and process multiple conditions of the problem or task;
OR non-routine manipulations, across disciplines/content
areas/multiple sources
8
DOK Misconception #1:
All kids can’t do this; or Kids don’t need
scaffolding to get “up” there.
Engaging in “a complex task” with supports/ scaffolding is
an essential step along the way to proficiency (Vygotsky’s
ZPD)
O Do it with others first; DOK 3 and 4 are not meant to
only be done alone/independently, especially at first
O Oral language & meaningful discourse support deeper
thinking and increase initial exposures to the content
and student engagement. This is NOT cheating!
O One strategy: Plan questioning & formative probes from
DOK 1-2-3-4 over the course of a lesson or unit of
study. Consider all DOK levels in your planning.
9
Vygotsky: Zone of Proximal Development
(What a child can do with assistance today)
What a child can
do independently
now: “ENTRY”
Actual
Development
Area
The
ZONE
What a child can
do independently
tomorrow/future
Potential
Development
Area
LEARNING PROGRESSIONS ZONE:
Dynamic area
Causes development to move forward
Social interaction essential (scaffolding)
Karin Hess (2008). Using learning progressions as a schema to monitor progress across grades.
10
DOK Misconception #2:
Webb’s DOK model is a taxonomy
O Bloom’s is a taxonomy, intended to be a hierarchy
O Primary Weaknesses of Bloom: generic verbs (void of
content); some of the same verbs at different levels
O Webb’s DOK model is nominative:
O It names how you interact with content; is contentspecific
O It differentiates varying levels of engagement with
content and suggests what tasks might look like
O DOK 4 is not better than DOK 3, or DOK 2, or DOK 1
11
11
DOK Misconception #3
Bloom verbs & levels = Webb DOK
The DOK “Wheel of Misfortune” implies that a DOK level is
indicated by a particular verb or set of verbs.
Norman Webb, “It’s what comes after the verb, that
indicates the complexity of a task (content).”
12
12
What does (cognitive) rigor
mean?
13
DOK Misconception #4:
DOK is about difficulty
O The intended student learning outcome (not grade level)
determines the DOK level. What mental processing must
occur? DOK = Complexity, not difficulty!
O While verbs may appear to point to a DOK level, it is what
comes after the verb that is the best indicator of the
rigor/DOK level and complexity of the task.
O Describe the information contained in graphics or data tables in
the text; or the rule for rounding a number
O Describe how the two story characters are alike and different.
O Describe the data or text evidence that supports your solution,
your reasoning, or your conclusions
O Describe varying perspectives on global climate change using
supporting scientific evidence, and identify the most significant
14
effects it might have on the planet in 100 yrs.
14
What is actually being assessed?
What mental processing is required?
O Can you define or spell erosion? (term/definitional)
O Which one is an example/non-example of erosion?
(conceptual)
O Can you demonstrate/explain what happens during
erosion? (procedural/descriptive)
O Use the data to identify a trend … and some possible
effects of erosion on this land mass over the next 5 years.
Support your reasoning with data analysis.
O What did the author mean when he said, “after that, her
popularity began to erode”? Explain why you think so
using your interpretation of the text? (contextual/multistep, requiring analysis of supporting evidence)
15
The Hess Cognitive Rigor Matrix Applies
Webb’s DOK to Bloom’s Cognitive Process Dimensions
Depth +
Thinking
Level 1
Recall &
Reproduction
Level 2
Skills &
Concepts
Level 3
Strategic
Thinking
Level 4
Extended
Thinking
Remember
- Recall, locate basic
facts, details, events
Understand
- Select appropriate
words to use when
intended meaning is
clearly evident
- Specify, explain
relationships
- summarize
– identify main ideas
- Explain, generalize,
or connect ideas
using supporting
evidence (quote,
example, data …)
- Explain how
concepts or ideas
specifically relate
to other content
domains or
concepts
Apply
- Use language structure
(pre/suffix) or word
relationships
(synonym/antonym) to
determine meaning
– Use context to identify
meaning of word
- Obtain and interpret
information using text
features
- Use concepts to
solve non-routine
problems
- Devise an
approach among
many alternatives
to research a novel
problem
Analyze
- Identify whether
information is contained
in a graph, table, text
feature, etc.
– Compare literary
elements, terms, facts,
events
– analyze format,
organization, & text
structures
- Analyze or interpret
author’s craft
(literary devices,
viewpoint, or
potential bias) to
critique a text
– Analyze multiple
sources
- Analyze
complex/abstract
themes
– Cite evidence and
develop a logical
argument for
conjectures
- Evaluate
relevancy,
accuracy, &
completeness of
information
- Synthesize
information within
one source, data set,
or text
- Synthesize
information across
multiple sources or
texts
Evaluate
Create
“UG” = unsubstantiated
generalization
- Brainstorm ideas about
a topic
- Generate conjectures
based on observations or
prior knowledge
16
17
DOK Misconception #5:
All DOK levels can be assessed with a
multiple choice question
O That’s just dumb!
O “Weaker” DOK 3 multiple choice items are possible;
but does selecting the best option (e.g., locate
supporting evidence for a theme) provide as much
insight as seeing HOW a student formulates and
reveals thinking?
O By their nature, DOK 3 and 4 questions/tasks are
more open-ended, generally take longer to respond
to/solve, and may have more than one “appropriate
right answer”
18
18
“Evidence-Based” Items may provide more
insights than traditional MC
19
2. The DOK
Instruction
& Assessment
Matrix Instructional
Decisions…
Paths
Selected Response
Each standard has an assigned Depth
Performance
of Knowledge.
Constructed Response
Tasks/Projects
DOK 2
DOK 1
Skills and
Concepts
Recall and Reproduction
Remember
Understand
Recall, locate basic
facts, definitions,
details, events
The DOK determines the cognitive
level of instruction.
DOK 4
DOK 3
Extended Thinking
Reasoning and
Thinking
Select appropriate
words for use when
intended meaning is
clearly evident.
Explain relationships
Summarize
State central idea
Use context for word
meanings
Use information using
text features
Apply
Analyze
Explain, generalize or
connect ideas using
supporting evidence
(quote, evidence, data)
Use concepts to solve
non-routine problems and
justify
Analyze or interpret author’s
craft (e.g., literary devices,
viewpoint, or potential bias) to
critique a text
.
Cite evidence and develop a
logical argument for
conjectures based on one text
or problem
Evaluate
-Explain how concepts or
ideas specifically relate to
other content domains.
Devise an approach
among many alternatives
to research a novel
problem
Analyze multiple sources
or multiple text
Analyze complex abstract
themes
Evaluate relevancy,
accuracy and
completeness of
information across texts
or sources
20
Create
.
Develop a complex model or
approach for a given situation
Develop an alternative solution
Synthesize across multiple
sources/ texts
Articulate a new voice, theme,
or perspective
20
DOK Misconception #6:
Higher order thinking = deeper learning
O What we have thought of as “higher order”
(analysis, evaluation, creative) thinking might only
be engaging or fun…and not always deeper
O Many critical thinking examples do not go deep or
ever get to DOK 3 or 4 (e.g., interpret/solve and
justify)
O Shift our thinking from “higher order” to deeper
learning, and that can mean:
O deeper understanding
O deeper application
O deeper analysis, etc.
The Hess CRM illustrates this
shift
21
21
Some general rules of thumb…
O If there is one correct answer, it is probably level DOK
1 or DOK 2
O DOK 1: you either know it (can recall it, locate it, do it) or you
don’t know it
O DOK 2 (conceptual): apply one concept, then make a decision
before going on applying a second concept; express
relationship (if-then; cause-effect), making connections; HOW I
did it
O If more than one possible answer/approach, requiring
evidence, it is DOK 3 or 4
O DOK 3: Must interpret, provide supporting evidence and
reasoning (not just HOW solved, but WHY it works– explain
reasoning for each step/decision made)
O DOK 4: all of “3” + use of multiple sources/data/ texts; initiate
22
& complete an investigation
DOK Misconception #7:
Multi-step or longer tasks, multiple texts, or
complex texts always means deeper thinking
O DOK 2 is not simply more than one step, it’s applying
more than one concept; DOK 2 is still routine/typical
(main idea, word problems, etc.)
O Simply reading more complex texts, but NOT delving
deeply into the text’s meaning/discourse style/etc., is
likely to still be DOK 1 or 2
O DOK 3 requires some aspect of open-endedness and
interpretation with justification or support; DOK 4 =
interpretation draws from analyzing multiple sources
O DOK is not cumulative! More = Deep [1+1+1 still =1!]
23
23
Examining Rubrics
What are the intended DOK levels?
24
Math Content Standards & Math Practices
Depth +
Thinking
Level 1
Remember
Know math facts,
terms, principles
Understand
Attend to precision
Level 2
Level 3
Level 4
Model with
mathematics
Construct viable
arguments
Estimate, predict,
observe, explain
relationships
Geometry proof
Integrate concepts
across math
domains or content
areas
Calculate,
measure, make
conversions
Use formulas
Make sense of
routine problems
Make sense of nonroutine problems
Design & conduct a
project
Identify a pattern
Locate information
in table, diagram
Use tools strategically
Classify, organize data,
extend a pattern
Reason abstractly
Analyze multiple
sources of
Evidence to solve
problem
Evaluate expressions,
locate/plot points
Represent math
relationships
Apply
Analyze
Generalize a pattern
Evaluate
Critique the
reasoning of others
Create
Design a model for
a new perspective
Design a complex
model with multiple
constraints
25
DOK examples from different
content areas
26
Depth +
Thinking
Remember
Level 1
Level 2
Level 3
Level 4
What is slope?
What is negative
space?
Understand
Read, write, and
represent these
fractions
Describe why negative
space is used.
Which term ____?
Explain how you
solved this problem.
Why control variables
in the investigation?
Find examples of…
Construct a math
argument to show
equivalence using
area, set, and linear
models
Interpret the theme.
Apply
Convert fraction to a
decimal
Organize these data
to support your
solution (e.g., graph,
diagram)
Conduct the
investigation, interpret
results, and support
conclusions with data
Design & conduct
an investigation,
based on a new
question raised.
Compare these
methods/mediums.
Are there design flaws?
Justify your
interpretation using
what you know about
fair tests.
Analyze more than
one product,
drawing from
multiple source
materials for the
analyses
UG - Which team, artist, play, etc. … is the
best?
How would you rank
these ___? Justify your
rankings using data
that supports your
criteria.
Some say the NFL
settlement for player
brain injury is not
adequate. Evaluate
both sides using data to
determine the validity of
this claim.
How would you
demonstrate this
technique?
Create scenario
supported by the data in
display.
Integrate multiple
source materials with
27a
intent to develop
novel product
Add these fractions
Analyze
Evaluate
Create
What kind of graph or
model is this?
Which data point
shows ____?
Is this a …
Which graph shows how
the data would be
displayed?
Create an equivalent
fractions. card game
27
For each assessment task (and rubric)…ask
O What is its purpose? (What understandings -
O
O
O
O
content/skill-are being assessed? is there a ‘right’
answer?)
What is the implied/intended rigor? (What mental
processing would you expect students to engage in?
Use the CRM Tools to find descriptors.)
Which standards does it REALLY elicit and assess?
(content + intended rigor)
Does the scoring guide/rubric match content +
intended rigor?
What would student responses tell a teacher if
students could/could not do all or part of this task?
(open-ended tasks, reasoning used) – next
instructional decisions are clear
28
Hess Formative Assessment Analysis Tool #10
What will this assessment uncover?
29
Hess’ Performance Assessment Validation Tool #9A
30
Take-Away Messages: Cognitive Rigor &
Some Implications for Assessment
O Begin with daily DOK3 classroom discourse!
O Assessing only at the highest DOK level (the
“ceiling”) will miss opportunities to know what
students do & don’t know – go for a range; end
“high” in selected/prioritized content
O Performance assessments can offer varying levels
of DOK embedded in a larger, more complex (“rich”)
tasks
O Planned formative assessment strategies and tools
can/should focus on differing DOK levels
31
Cognitive Rigor: Shifting Teacher & Student Roles
© Karin Hess, 2013, Linking Research with Practice
DOK
Levels
Teacher Role
Student Role
1
• Questions to focus attention (Who?
What? Where? How? When?)
• Directs, leads, demonstrates,
defines
• Acquires vocabulary, facts, rules
• Memorizes, recites, quotes
• Practices, restates
2
• Questions to differentiate/ classify,
draw out inferences, check
conceptual understanding (Why?
What conditions? Give
example/non-example?)
• Explains relationships, sorts, classifies,
compares, organizes
• Makes predictions based on estimates,
observations; proposes
3
• Questions to probe reasoning and
underlying thinking (How do you
know? What is the hard evidence?
• Uncovers relevant, accurate, credible
information or flaws in a design
• Develops supporting (hard) evidence for
conclusions or claims
• Tests ideas, solves non-routine problems
4
• Questions to extend thinking,
explore alternative sources, broaden
perspectives (What are the potential
biases? Can you propose an
alternative model?
• Initiates, transfers, and constructs
knowledge
• Modifies, creates, elaborates based on
multiple sources
• Investigates real-world problems and issues
32
Some Related Resources
O
O
O
O
O
O
New York City Department of Ed website – Common Core library: preK-12
assessments in ELA and mathematics, some of which Karin helped to
develop and pilot www.weteachnyc.org
National Math Science initiative – gr 3 – HS; also has ELA and SS
performance tasks with DOK designations - www.nms.org
Read Works - gr k-8; short literary & informational texts with CC Qs
www.readworks.org
Jim Burke – Middle & High School ELA resources
www.englishcompanion.com
Dan Meyer blog – “3-act” math PAs for ES-MS-HS; kids build the problems by
deciding what’s needed to solve them – good strategic thinking required;
http://blog.mrmeyer.com/category/3acts/
Links and resources on www.karin-hess.com
O Norman Webb video clip – on how he developed the DOK model
O Karin’s YouTube videos
O Karin’s blog, newsletter, classroom poster set - “Rock the Rigor” & many
other resources
33