Transcript Developing Expertise with Classroom Assessment in K

Developing Expertise with
Classroom Assessment in
K-12 Science
Maryl Gearhart, Sam Nagashima, Jennifer Pfotenhauer,
Cheryl Schwab, Craig Strang
Center for the Assessment and Evaluation of
Student Learning (CAESL)
University of California, Berkeley
University of California, Los Angeles
Lawrence Hall of Science
1
Developing Expertise with Classroom
Assessment in K-12 Science
UCLA
Joan Herman
Lawrence Hall of
Science
Sam Nagashima
Lynn Barakos
Ellen Osmundson
Craig Strang
WestEd
U.C. Berkeley
Diane Carnahan
Maryl Gearhart
Karen Cerwin
Jennifer Pfotenhauer
Kathy DiRanna
Cheryl Schwab
Jo Topps
2
Overview of Session
Presentations
•
•
•
•
Project purpose and context
Framework for quality classroom assessment
PD model - the assessment portfolio
Research design - focus on one research tool
Small groups
•
•
Sample assessment portfolios
Research tool: Practice Profiles
Reconvene
•
•
•
•
Revisions of assessment portfolio
Preliminary research findings
PD model - assessment leadership
Q&A
3
Purposes of R & D
Classroom assessment expertise
PD strategy - assessment portfolio
Research tools
Framework to guide PD and research
New forms of collaboration
4
Background
Psychometric standards
•
Student Evaluation Standards (2002)
•
Standards for Educational and Psychological
Testing (1999)
•
Buros (1990)
5
Standards for the teaching profession
•
National Board for Professional Teaching
Standards (NBPTS)
•
California Standards for the Teaching
Profession (CSTP)
•
Performance Assessment for California
Teachers (PACT)
6
Conceptions of Classroom practice
•
General models: Stiggins….
•
Assessment that supports learning:
Black & Wiliam, Shepard, Wiggins ….
•
Cognitive models: Knowing What
Students Know ….
•
Multi-leveled assessment systems:
Shepard, Wilson
7
Science Education Resources
•
AAAS, Benchmarks for Science Literacy
•
National Science Education Standards
•
NRC, Classroom Assessment
8
CAESL Mission
Center for Assessment and
Evaluation of Student Learning
(CAESL)
•
strengthening K-12 science assessment
at all levels of the system
9
10
CAESL Quality Classroom Assessment
QUALITY GOALS FOR
STUDENT LEARNING AND
PROGRESS
QUALITY
TOOLS
QUALITY
USE
CAESL Quality Classroom Assessment
QUALITY GOALS FOR
STUDENT LEARNING AND
PROGRESS
QUALITY
TOOLS
Valid for Purpose
• Developmentally
Sound Content
• Clear Expectations
• Reliable & Accurate
• Fair & Unbiased
QUALITY
USE
Sound Interpretation
• Quality Criteria
Support Student Learning
• Guide for Instruction
• Quality Application
• Instructional Materials
Refinements
• Quality Analysis
• Assessment Refinements
• Student Involvement and
Accountability
• Timely Feedback
• Equitable
CAESL Quality Classroom Assessment
CLASSROOM ASSESSMENT SYSTEM
• Longitudinal to track student progress
• Multileveled
•Coordinated and strategic
•Feasible
Assessment Portfolio
Purpose
• design, implement, evaluate an
assessment plan for a curriculum unit
Support for
•
individual learning and reflection
coaching
collaboration
•
leadership
•
•
Team organization
•
•
•
cross site unit teams
K-12 district teams
local coaching
14
Portfolio sections
Section I: The plan
Section II: Implementation
Section III: Evaluation
15
I: The plan
Conceptual Flow
• Establish learning goals
• Context for assessment planning
• Deepen knowledge
16
QuickTi me™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
17
Building (Identifying) the Conceptual Flow
Grade Level Concept
Unit Organizing
Concept
Unit topic:
Unit concept:
Subconcept
Lesson Concept:
Lesson Activity:
18
19
I: The plan
Record of Assessments in
Instructional Materials (RAIM)
• Concepts assessed
• Expected Student Responses (ESRs)
• Any need for assessment refinement
• Key assessments to track progress
20
Section I A 2: Record of Assessment in Ins tructional Materials (RAIM2)
How are assessments organized and used in the unit of instruction to mak e sense of and understand student learning?
Concept Assesse d
Pre

The structures of plants are
roots, stems, leaves, and
flowers
Assessment #2

The structures of plants are
roots, stems, and leaves.

Plants need water, sun, air,
and something to grow in
Type of
Assessm ent
Page #
Use or
Refine
Expected Student Response
Expected Student Response
HIGH
MEDIUM and LOW
•What kinds of understandings do you
expect from students?
•W hat kinds of understandings do
you expect from stu dents?
Note possible alternative conceptions
Observation
s
Drawing
Not in
text book
New
Parts should be labeled correctly.
Roots in dirt. Drawings are in color,
detailed, and looks like plants.
Complete sentences about the plant
that describe color, leaves, names,
etc.
booklet
Not in
text book
New
Pg 149
in
Compl
ete
Book
of
Scienc
e
All parts of a plant are labeled
correctly and all jobs of plants are
correctly labeled.
All parts of a plant are labeled
correctly. Some of the jobs are
correctly labeled.
Assessment #3

Plants can change
depending on the
environment. They have
characteristics that are the
same and different and grow
in different environments.
21
II: Implementation
Selected assessments
Documentation of:
•
•
•
•
Concepts assessed
Expected Student Responses (ESR)
Analysis of student work
Instructional follow-up and feedback
22
III: Evaluation
Prompts to guide review
and revision
Revised tasks
Revised criteria
23
Research Questions
How does assessment expertise
develop?
How does the portfolio support the
development of assessment
expertise?
24
Design and Data Sources
•
Longitudinal
•
Nested design
All participants:
Surveys
Assessment portfolios
Cases:
Observations
Interviews
25
Research tool:
Practice Profiles
Purpose: Capture development of
assessment expertise over time
Unit of Analysis: Teacher practice
regarding one assessment
26
Designing the Profiles
•
Audience
•
Comprehensiveness
•
Detail
•
Assumptions
27
Draft Profile Levels
•
Draft levels of emerging
assessment expertise
Inexperienced ->
Developing ->
Maturing ->
Expert
28
CAESL Quality Classroom Assessment
QUALITY GOALS FOR
STUDENT LEARNING AND
PROGRESS
QUALITY
TOOLS
Valid for Purpose
• Developmentally
Sound Content
• Clear Expectations
• Reliable & Accurate
• Fair & Unbiased
QUALITY
USE
Sound Interpretation
• Quality Criteria
Support Student Learning
• Guide for Instruction
• Quality Application
• Instructional Materials
Refinements
• Quality Analysis
• Assessment Refinements
• Student Involvement and
Accountability
• Timely Feedback
• Equitable
Sound Interpretation
•
Draft components
Quality criteria
Quality application of criteria
Quality analysis
30
Small groups
•
Review sample portfolio
•
Review draft profile for
Sound Interpretation
31
Portfolio Revisions
I: Greater focus and integration
RAIM fewer assessments
Integrated CF & RAIM
Tradeoff
Comprehensive <- -> Focused
32
33
Portfolio Revisions
II: Scaffolding the analysis
Preliminary criteria for H M L
Criteria revision using student work
Models of whole class analysis
Models of target student analysis
Tradeoff
Resources <- -> Prescriptions
34
Research Questions
How does assessment expertise
develop?
How does the portfolio support
the development of
assessment expertise?
35
Findings from ‘03-’04 survey
Quality of assessment tool
Assessment use
36
Teachers feel most comfortable with:
•
•
Using assessment results as evidence of
student progress to design developmentally
appropriate instruction
Evaluating the alignment of assessments with
learning goals
Teachers feel less comfortable with:
•
•
Evaluating scientific soundness of assessment
content
Judging whether assessments capture full
range of alternative conceptions
37
Leadership Context
•
Districts committed to K-12
science
•
Districts committed to
reform/inquiry-based science
38
Challenges of Leadership
•
•
•
•
Various, developing levels of assessment
expertise
Various, developing levels of leadership
expertise
Science not the priority, erosion of reform
Quality assessment of reform science
really not the priority
39
Goals of Leadership
•
•
•
Growth & improvement of individual
practice in instruction & assessment
Leading/guiding individual growth in
others
Influencing systemic change in
districts: developing/implementing
quality assessment systems
40
Dimensions of Leadership
(Program Elements Matrix)
•
•
•
•
•
Professional Development
District Policy
Parent/Community
Involvement/Support
Develop/Implement Quality
Assessment System
Leadership Development &
Capacity Building
41
Leadership Activities
•
•
•
•
•
CF & RAIM with Reform Unit with a Buddy
(or a few)
Participate in FOSS ASK Project
Serve on and influence district
committees
Presentations to school board, district
administrators, parent groups
Develop benchmark assessments(!?)
42