Transcript Assessing Student Learning to Improve Teaching

Assessing Student
Learning to Improve
Teaching
Jeff Bell and Jim Bidlack
Assessing Student
Learning
• This session will provide some
suggestions for how your current
assessments can be “tweaked” to
provide tools to assess learning objects
or any change in teaching style or
technique
Why Assess Student
Learning?
• To assign grades (not the point of this
talk)
• To determine the effectiveness of the
instruction (the point of this talk)
• Assumption: to improve instruction
you need feedback on the
effectiveness of the instruction
Why Not Use
• Grades forGrades?
the course, an exam, or
even a paper usually are affected by
too many different factors to give a
good indication of the effectiveness of a
specific learning task
• Purpose is to assign a value to a
student, not the instructor or
instruction
• Need to define very specific learning
objectives, and then measure student
achievement of those objectives
Example Using
Grades- The Use of
Clickers
• Clickers are used in large lectures to
quiz the students during the lecture
• Jim Bidlack looked at the effectiveness
of this strategy in a Biology class
Problems
• If no statistical difference (the most likely
outcome) there is no evidence to guide
changes
• Were students coming to class?
• Did they do better on “clicker questions”
but worse on other questions?
• Did exam questions cover the same
learning objectives as the clicker
questions?
• Did changes in class size, students, etc.,
affect the results?
Specific Learning
Objectives
• Specific Learning Objectives (SLOs) -
narrowly defined to cover one specific skill or
knowledge
• Should describe what the student can ‘do’
• Not – “understands probability,” but, “Can
use probability theory to calculate the
expected frequency of unordered events”
instead
Genetics Examples
• Can use the ratios of phenotypes
produced in a monohybrid cross to
determine the genotypes of the parents
and the mode of inheritance of the trait
• Can predict the probable outcome of
phenotypes in a monohybrid cross from
the genotypes of the parents and mode
of inheritance
How to Get SLOs
• SLOs should be very specific and it
should be obvious how to measure
them
• Examination of your current tests and
assignments will provide you with
many of the actual SLOs for your
courses
• If you don’t have any assessments for
a SLO then your students may not
really be learning that SLO
How to Measure
SLOs?
• Item analysis of exam questions or
assignment tasks
• Surveys
• To measure the effect of an instructional
change will also need
• Pre-test and post-tests, or
• Control groups and experimental
groups
Item Analysis
• Typical exams and assignments have
multiple SLOs, but can be designed
so that each individual SLO can be
assessed
• SLO: Can use probability theory to
calculate the expected frequency of
unordered events
• Exam question: The probability that
in a family with seven children,
three of the children will be girls, is?
• Assessment: Fall’05 – 47/49 (96%)
• SLO: Can use a pedigree, the laws
of inheritance for an autosomal
gene, and the product, sum and
conditional rules to calculate the risk
of affected children in a specific
mating.
• Exam question: An older women
who has no genetic disease has a
sister with the recessive disease
cystic fibrosis; both her parents are
normal. How likely is it that she is
not a carrier for the genetic
disease?
• Assessment: Fall’05 – 30/49 (61%)
Process
• Write the exam so each question
covers one SLO
• When grading each student keep track
on a separate sheet of success or
failure for each SLO
• Tally the success rate for each SLO
• For assignments, use a rubric and
follow the same process as above
Surveys
• You can ask your students what they
know
• If the question is specific, they usually
do know what they can do, or not do
• “Before taking this course, I could use a
chi-square statistical test to test a
hypothesis about expected ratios in a
genetic cross and understood what the
p value the test returned meant.”
• Before taking this course, given the
probability of the independent individual
events I could predict the probability of
any sequence or set of outcomes (e.g.,
the probability of a family with five
children having three boys and two
girls).
• Pre – 2.27 (5=strongly agree)
• Post – 4.23 (3 students disagreed)
• Exam – 96% (2 students failed)
Pre and post test
• Test them before instruction and after
and then use item analysis
• Problem: Can not usually grade them
on the pre-test – effort on pre-test is
usually not as great as on post-test
• Good if you want to always show a
positive effect
• Does work for low effort recall type
questions
Spring ‘02
Control and
Experimental Groups
• Best if done at the same time with different
sections – logistically difficult and may have a
placebo effect
• Different semesters – have to get baseline data
before trying your innovation
• Point of this talk - you need to start now even
if you don’t know what you’re going to do next
year
• Bonus – baseline data may tell you where you
really need something new!