Transcript Assessment. Brian Couch 2015

Assessment:
More than just grades

Brian Couch
University of Nebraska-Lincoln
Kate Semsar
University of Colorado-Boulder
Adapted in part from materials by Clarissa Dirks (Evergreen State College), Michelle
Withers (West Virginia University), and Jenny Knight (University of Colorado-Boulder)
Assessment as a learning tool
“Ongoing assessment plays a key role
– possibly the most important role –
in shaping classroom standards and
increasing learning gains.”
Black and Wiliam, 1998
Learning Objectives
You will be able to…
 Distinguish between formative and summative
assessment
 Use Bloom’s Taxonomy to evaluate learning objectives
and assessments
 Use backward design to align learning outcomes with
formative and summative assessments
 Demonstrate the different possible uses of formative
assessments
 Discuss the uses of concept assessments
Think-Pair-Share
 How do you know when you know something?
 How do you know when your students know something?
 How do your students know when they know something?
Assessment provides a means for gauging student understanding.
Individual
In your opinion, what is the role of assessment?
Share your ideas with your group. Write your ideas
on whiteboards.
Purposes of assessment
 Assessment provides meaningful information
for instructors and students
 Instructor:
- communicates course expectations
- can determine level of student mastery
- can use information to inform teaching practices
 Student:
- clarifies course expectations
- can determine own level of mastery
- can identify areas for improvement
Different types of assessment
Summative assessment
vs.
Formative assessment
What’s the difference?
Assessment spectrum
Different types of assessment
Summative assessments: provide information about
mastery
 Exams, papers, presentations
 Typically occur at the end of teaching
 Usually part of grade for the class
Formative assessments: provide feedback to both the
instructor and students during the learning process
 In-class work, homework, pre-class online
assignments
 Can be for a grade or for participation
Assessment with a purpose
Backward Design:
Learning goals drive assessment and instruction
What should
students know
or be able to do
by the end of
your course?
Learning goals
How will you
know if they
get there?
Summative
assessments
Alignment is key!
What will you
do to get
them there?
Learning
activities
(including formative
assessments)
Terminology review
Learning Goal: Broad description of what students will
understand and learn: not necessarily assessable with
single question.
Example: Understand how chromosomes align and
separate during the process of meiosis (the
production of sperm/egg cells)
Learning Objective: specific, action-oriented description
of what students will be able to do: assessable.
Example: Predict the probability of a certain
phenotype among children of two individuals, one
with a sex chromosomal abnormality
Review: Bloom’s Taxonomy
Practice recognizing Bloom’s levels
Order these from lowest to highest
order; discuss with neighbors
(1) Knowledge, (2) Comprehension,
(3) Application, (4) Analysis,
(5) Evaluation, (6) Synthesis
1. Compare the mechanisms for regulating
transcription in bacteria and eukaryotes.
2. Define transcription.
3. Design an experiment to determine
whether all of an organism's mRNA
sequences are encoded in its DNA.
4. Describe the process by which
nucleotides are added to the RNA.
The best order (from lowest
order to highest order) is:
a) 4,2,1,3,5,6
b) 2,4,6,5,1,3
c) 6,2,4,5,3,1
d) 2,1,5,6,4,2
e) 2,4,1,6,5,3
5. Predict the possible affects on protein
function as a consequence of a specific
mutation in the DNA
6. Diagram a DNA duplex in the process of
transcription showing base-pairing and
strand polarity for all polynucleotides.
Apply to your own course
 Use Bloom’s Taxonomy to evaluate objectives and
assessments
 Use the principles of backward design to align learning
outcomes with summative assessments
What is your level of experience with learning
objectives?
A. I didn’t know what they were until I started reading
“Scientific Teaching.”
B.
I am familiar with them but have not used them in my
courses.
C.
I write learning objectives for my courses and use
them only for myself.
D. I write learning objectives for my courses and share
them with my students.
Activity: Learning objectives and assessments
PART I: Blooming your objectives
 Find (or make up on the spot) one or more learning
objective(s) for a topic you teach
 “Bloom” the level of your objective (independently)
 Now share with your neighbor.
 Do you agree with each other’s ratings?
 Group reflect: what kinds of problems arose in your
discussion?
Use the Bloom’s table in your binder
Alignment Table
Learning Goal
What will students
learn?
Students will
understand the
transfer of
information from
DNA to proteins
Learning Objective
(content + behavior)
Summative Assessment
(exam question)
If they have learned How will students demonstrate
it, what will
they know it or are able to do
students know and
it?
be able to do?
Predict changes in
amino acid
sequences caused
by mutations
Deva has cystic fibrosis. By
looking at this section of her
DNA sequence and comparing it
to her mother’s DNA sequence,
find her mutation, and predict
the amino acid sequence that
will result. Suggest a different
DNA mutation in the same
codon that would NOT have
resulted in cystic fibrosis
PART II: Aligning objectives/outcomes with
assessments
 Find an exam question that you think addresses
the objective you were just discussing.
 Share the question with your neighbor. Is your
question aligned with your learning objective?
 How can you tell?
 If your neighbor and you disagree on the
alignment, why?
Bloom’s in practice
Return to the exam questions that you just
aligned with objectives
Modify your original learning objective and
exam question so that they are both at a
higher level than they were before. If they
were already at a the highest level, practice
rewriting them to a lower level.
 What were some key strategies you used
to level-up or level-down?
Alignment of exams with concept maps
Course
1
How helpful for
your learning were
the concept maps?
2
3
4
Alignment of exams with concept maps
Course
1
How helpful for
your learning were
the concept maps?
2
3
4
“The concept maps were very helpful for exam
3 and 2 because the test questions were more
congruent with the use of the concept maps.
The first test was more matching, and the
concept maps did not really help with that as
much.”
“I did not find them useful most of
time but thought they were a form
of busy work.”
Alignment of exams with concept maps
Course
1
How helpful for
your learning were
the concept maps?
2
3
4
“The concept maps were very helpful for exam
3 and 2 because the test questions were more
congruent with the use of the concept maps.
The first test was more matching, and the
concept maps did not really help with that as
much.”
“I did not find them useful most of
time but thought they were a form
of busy work.”
What kinds of questions do you usually use on
exams/final assessments?
A. Multiple choice
B. Free response (short, drawing)
C. Long essays
D. A mixture of the above
E. Papers, or other
Add formative assessment to your aligned
objective and exam question
Learning
Goal
What will
students
learn?
Learning
Objective
If they have
learned it, what
will students
know and be able
to do?
Summative
Assessment
(Learning Activity)
Formative
Assessment
How will students What will students do to
demonstrate they learn it?
know it or are
able to do it?
Assessment throughout the learning cycle
Assessments can be used throughout an instructional
unit.
clicker
questions
pre-class
in-class
worksheet
homework
assignment
homework
assignment
unit exam
in-class
pre-class worksheet
Time
practice
questions
Individual: Pre-Assessment
What are concepts or skills that your students need
to know when they enter your course in order to be
successful?
To what degree are you confident that students have
an adequate level of mastery of these concepts and
skills?
Group: Pre-Assessment
What are some ways that we can collect information
about incoming student knowledge?
List a few ideas on whiteboards
Assessments allow opportunities for
“deliberate practice”
The information obtained from assessments can help
students know where they are at and figure out where
they need to go
K. Anders Ericsson
EnGaugements
When you ask a student to do
something, they are simultaneously
engaged in learning and can gauge their
progress by whether or how well they
can perform.
– Handelsman et al.
Scientific Teaching.
Assessment helps determine whether
instruction is at the appropriate level
The Montillation of Traxoline
It is very important to learn
about traxoline. Traxoline is a
new form of zionter. It is
montilled in Ceristanna. The
Ceristannians found that they
could gristerlate large amounts of
fervon and then bracter it to
quasel traxoline. This new, more
efficient bracterillation process
has the potential to make
traxoline one of the most useful
products within the molecular
family of lukizes snezlaus.
QUIZ:
1. What is traxoline?
2. Where is it montilled?
3. How is traxoline quaseled?
4. Why is traxoline important?
Assessment also communicates course expectations.
People often don’t know what they know
AND/OR
think they know something but don’t!
 Incorrect Conceptions
 Private Universe
 Minds of our Own
 Tiny World
What are some of the incorrect ideas that people
expressed?
 Assessments help students distinguish
what they know and don’t know
Example method:
Group brainstorm
Genetic diseases, like Phenlyketonuria (PKU), confirm that there is a
link between an individual’s DNA and that individual’s proteins.
Below is a DNA molecule and the amino acid sequence that would
result from translating the DNA sequence.
3’CGTTTTACCAAACCGAGTACTGAG
5’GCAAAATGGTTTGGCTCATGACTC
TRP-PHE-GLY-SER
Which nucleotides are responsible for this particular
sequence of amino acids?
As a group, write down what you know about DNA and proteins
on one side of the white board.
On the other side, write what else you need to know to be able
to answer this question.
 Assessments can aid in the
construction of new knowledge
Example method:
Group work followed by
report-out
Based on your understanding of natural selection and traits
that vary along a continuum:
1. Explain the changes that occurred in the tree and
dinosaur populations over time.
2. Based on your knowledge of evolution, propose a
plausible mechanism by which the tree and dinosaur
populations can change over time.
These represent the
average for an entire
population
(AAAS 1999)
Students hold a wide range of different
conceptions
Certain misconceptions
appear consistently
across years and impede
the development of
conceptual mastery
Fraction of students
answering correctly in two
consecutive semester
r = 0.93
Couch, Wood, Knight (2015) CBE-Life Sci Educ.
 Assessments can help students
confront misconceptions
Example method:
clickers
As the acorn grows into the tree, from where
does the majority of the biomass come?
A. Air
B. Soil
C. Water
D. Sun
Formative Assessment is Beneficial to Me
100
80
Percent Students
60
strongly agree
40
agree
20
strongly disagree
disagree
0
-20
-40
JiTT
MB-pre
Pre-Class
CQ
ICA
In-Class
MB-post HW/Q
Post-Class
Formative Assessment Type
Key Factors
- Logistics
- Alignment
- Follow-through
A great resource for
finding different
ideas for formative
assessment
Formative assessment in practice
 What is an area of your course where you
think you and your students might benefit
from formative assessment?
 What type of assessment would you
administer? What information would it
provide to you and the students? How
might you use this information to alter
your teaching?
Additional tools: Concept assessments
• Typically multiple-choice or multiple-T/F format
• Measure conceptual understanding, not just facts
• Focus on critical concepts that faculty value
• Use common misconceptions as distractor items
• Rigorously developed and tested with student interviews
A link to a comprehensive list of concept assessments across
STEM disciplines:
http://www.asbmb.org/uploadedFiles/Education/TeachingStrategies/
Concept_Inventory/Concept%20Inventories%202%202%202015.pdf
Example of use of concept assessments: objectively measure
learning gain
Genetics Concept Assessment (GCA)
Smith, Wood, Knight, 2008. The Genetics Concept
Assessment: A New Concept Inventory for Gauging
Student Understanding of Genetics. CBE Life Sci. Educ.
7, 422-430.
25 multiple choice questions that address 9 Learning
Goals commonly cited by faculty as critical for student
understanding of genetics
Students take the GCA on the first day of class (pre)
and again at the final (post)
100
post test higher than pretest
(p<0.05 in all cases)
90
Percent Correct
80
70
Post test
Pre-test
60
50
Post
Pre
40
30
20
10
0
Majors #1
Majors #2
Majors #3
Course
Majors #4
Non-majors #1 Nonmajors #2
Subset of GCA Questions are still difficult on post test
These areas can be targeted
for instructional change
Question
Number
25
23
24
21
22
19
20
17
18
15
16
13
14
11
12
9
10
8
7
6
5
4
3
2
1
Percent Correct
100
90
80
70
60
50
40
30
20
10
0
Post test
Pre-test
Assessing attitudes and skills
 Affective domain:
http://www.asbmb.org/uploadedFiles/Education/TeachingStrategies/Concept_Inventory/
Student%20Views%20Attitudes%20Affective%20Instruments%201%2011%20201
5.pdf
 Science process skills:
http://www.asbmb.org/uploadedFiles/Education/TeachingStrategies/Concept_Inventory/
Student%20Skills%20Inventories%201%2011%202015.pdf
Program for Assigning Student Groups
www.catme.org