Transcript Chavis - LSUA Center for Teaching Excellence

2015 Fall Teaching Institute
Louisiana State University at Alexandria
August 20, 2015
Why do We Assess? Improving
Student Learning through
Performance Based Measures
Dr. Kim Chavis, Ph.D.
North Carolina A & T State University
[email protected]
Words to Consider
"Assessment is an ongoing process aimed
at understanding and improving student
learning. It involves making our
expectations explicit and public; setting
appropriate criteria and high standards for
learning quality; systematically gathering,
analyzing, and interpreting evidence to
determine how well performance matches
those expectations and standards; and
using the resulting information to document,
explain, and improve performance.” '
Dr. Tom Angelo, Reassessing (and Defining)
Assessment. The AAHE Bulletin, 48(2), November
1995, pp.7-9.
THERE IS A DIFFERENCE BETWEEN WHAT
IS DESIGNED, TAUGHT, AND LEARNED
Learned curriculum
Designed curriculum
Official course outlines
Course catalog
What have they learned
What can they do
What do they care about
Taught curriculum
Found in syllabi
NATIONAL MOVE TOWARD
ASSESSMENT OF LEARNING
 What
is assessment?
“the methods that an institution employs to
gather evidence and evaluate quality
(Accrediting Commission for Community and Junior Colleges)
 What
are student learning outcomes?
“statements which define what a student
should be able to do after the completion of a
course or program”
(http://online.bc.cc.ca.us/courseassessment)
There Has Been a Paradigm Shift in the
Way That We View Educational
Effectiveness
Learning Outcomes:
In terms of the
learning students
are able to
demonstrate
Rather than inputs
such as number of
credit hours or
curricular content
ASSESSMENT: WHY BOTHER?
Three main purposes for assessment
1.
To demonstrate quality and excellence and ensure the
same level of quality continues
2.
To identify areas needing attention, support and
development and decide how to improve those areas
3.
To plan changes that will improve policies, procedures,
services, curriculum, resources, teaching, campus
climate and ultimately improve student learning.
IN THE PRINCIPLES FOR
ACCREDITATION
Comprehensive standard 3.3.1
The institution identifies expected
outcomes, assesses the extent to which it
achieves these outcomes, and provides
evidence of improvement based on
analysis of the results in each of the
following areas: (Institutional
Effectiveness)
ASSESSMENT STAIRS
WHY BOTHER WITH
LEARNING OUTCOMES?
Supporters assert…
They help students learn more effectively
 Students know what to expect from a
particular course or program
 Learning outcomes help instructors
 Design their materials more effectively
 Select the appropriate teaching strategy
and technology
 Design examinations which reflect what
was taught

PROGRAMS NEED TO BE ASSESSED
Programs are an aggregation of courses
with a common program mission.
The heart of assessment is embedded in the
courses.
The program’s product is the graduate.
10
ESTABLISH CLEAR PROGRAM
LEARNING DOCUMENTS
Programs should clearly define
Program Mission
Program Goals and Objectives
Learning Outcomes for each Goal
Assessment Methods
Expected Results
Actual Results
Use of Results to Improve the Program
A Mission…
A mission indicates the purpose of the
program or its reason for being
A mission states what
it hopes to achieve in
the future
A mission affirms its principles and
beliefs, nature, and values.
A Sample Mission…
The Forestry and Ecology program is designed to
educate broad-based, ecologically sensitive
resource managers, enabling them to succeed
as professional managers and practice wise
stewardship of forests or other natural resources
for multiple uses. As part of an Historically Black
College or University, the program addresses the
needs of capable students who are
underrepresented in the forestry profession as
well as the needs of the minority, forestlandowner community that have historically
been under-served by the forestry profession.
A program
goal…
A goal is broadly stated
A goal is accomplished via objectives
A goal is not directly measurable
Example: To understand and apply the concepts
of bioinformatics in professional life
A program
educational
objective…
Objectives describe knowledge, skills,
abilities, attitudes, or dispositions
An objective is directly measurable
An objective is specific and tied to a
series of linked learning outcomes
Educational objectives are sometimes considered
synonymous with learning outcomes, but learning
outcomes are more detailed, behavioral in nature, and
stated using precise verbs.
A program
learning
outcome…
Describes what…
we want the students in the program to
know
we want the students to be able to do
values or attitudes we want to instill in
our students
A program learning outcome is capable of
being assessed
Overlapping Course Student Learning Outcomes Become Program
Student Learning Outcomes
Course
SLOs
Course
SLOs
Program
SLOs
Course
SLOs
Course
SLOs
Course
SLOs
Course
SLOs
COURSE MAPPING
Course mapping helps to develop a
sense of how current course offerings in
the program cover different learning
outcomes
A course map is a table with the
learning outcomes on one axis and the
courses in the major on the other axis
ADVANTAGES OF CURRICULUM
MAPPING IN ASSESSING STUDENT
LEARNING OUTCOMES
Curriculum Mapping causes…
Thought about LEARNING INPUTS – Things that
are in place before learning begins
[prerequisites, placement, class size, facilities &
equipment.
Thought about LEARNING PROCESSES – Time on
task, active learning opportunities.
Thought about LEARNING CONTEXT –
Professional v. liberal arts program, professional
expectations, attitudes towards learning.
Simple Course Map
http://www.bridgew.edu/AssessmentGuidebook/chapter4.cfm
More
Complex
Course
Map
Indicating
Level of
Emphasis
on Learning
Outcome
http://www.bridgew.edu/AssessmentGuidebook/chapter4.cfm
MYTHS ABOUT ASSESSMENT
It is not my job
I already assess; I call it grading
You cannot assess what we do in academia
I do not have the time to do it
I do not have the expertise to do it
Assessment might reveal “bad” news
It is a passing fad
YOU CANNOT ASSESS WHAT YOU
HAVE NOT DEFINED: ESTABLISH
CLEAR LEARNING OUTCOMES
Are your learning outcomes clear?
Are they shared outcomes?
Can you talk about them easily?
Closing the Loop
What do the results of
assessment mean?
How are we going to use
the results to improve
student learning?
UNTIL THE RESULTS OF ASSESSMENT OF
STUDENT LEARNING ARE USED, THEY HAVE
NO VALUE
This may take the form of revision of
course content of individual courses,
revision of curricula, changes in
methods of teaching, inclusion of
experiential approaches
The changes must in turn be assessed
for effectiveness themselves
Don’t forget –
Assessment is
not useful
until it is
used!
Learning Outcomes…
Behaviors that demonstrate
• desired knowledge (cognitive domain)
• attitudes or dispositions (affective domain)
• acquired skills (psychomotor domain)
Fundamentally, the key is what will
students be able to do as a result of the
instruction.
The Cognitive Domain
Knowledge
Application
Comprehension
Bloom’s Classification
Analysis
Evaluation
Synthesis
Knowledge…
Recalling or remembering
something without
necessarily understanding,
using, or changing it
Count
Label
Outline
Record
Define
List
Quote
Repeat
Describe
Match
Read
Select
Draw
Name
Recall
State
Identify
Point to
Recite
Write
Comprehension…
Understanding something
that has been
communicated without
relating it to anything else
Associate
Compute
Convert
Defend
Discuss
Distinguish
Estimate
Explain
Extend
Extrapolate
Generalize
Infer
Rewrite
Summarize
Paraphrase Predict
Application…
Using a general concept to
solve problems in a
particular situation or using
learned material in new and
concrete situations
Add
Apply
Calculate
Classify
Complete
Compute
Divide
Examine
Graph
Interpolate
Manipulate Modify
Operate
Produce
Show
Solve
Analysis…
Breaking something down
into its parts (identification
of parts, analysis of
relationships among parts,
recognition of organizing
principles)
Analyze
Arrange
Breakdown
Combine
Design
Detect
Develop
Diagram
Illustrate
Outline
Point out
Relate
Select
Separate
Subdivide
Utilize
Synthesis…
Creating something new by
putting parts of different
ideas together to make a
whole
Categorize
Combine
Compile
Compose
Create
Design
Devise
Generate
Group
Integrate
Modify
Order
Organize
Plan
Prescribe
Propose
Rearrange
Reconstruct Reorganize
Revise
Evaluation…
Judging the value of
material or methods as they
might be applied in
particular situations or
judging with the use of
definite criteria
Appraise
Assess
Compare
Conclude
Contrast
Criticize
Determine
Grade
Interpret
Judge
Justify
Measure
Rank
Rate
Support
Test
Cognitive Domain – Learning Outcomes Related to Knowledge
Knowledge
Comprehension
Application
Analysis
Synthesis
Evaluation
Student
remembers or
recognizes
information as
communicated
with little
personal
assimilation
Student grasps
the meaning
behind the
information and
interprets or
comprehends
the information
Student uses
information to
relate and apply
it to a new
situation with
minimal
instructor input
Student
discriminates,
organizes, and
scrutinizes
assumptions in
an attempt to
identify
evidence for a
conclusion
Student
creatively
applies
knowledge and
analysis to
integrate
concepts or
construct an
overall theory
Student judges
or evaluates
information
based upon
standards and
criteria, values,
and opinions
Cite
Label
List
Enumerate
Identify
Imitate
Match
Name
Quote
Recall
Reproduce
State
Write
Convert
Define
Describe
Discuss
Estimate
Explain
Generalize
Identify
Illustrate
Locate
Paraphrase
Restate
Summarize
Apply
Chart
Compute
Demonstrate
Determine
Dramatize
Establish
Make
Manipulate
Prepare
Project
Solve
Use
Analyze
Compare
Contrast
Correlate
Diagram
Dissect
Differentiate
Distinguish
Infer
Investigate
Limit
Outline
Separate
Assemble
Create
Construct
Design
Develop
Formulate
Generate
Hypothesize
Initiate
Invent
Modify
Reframe
Synthesize
Access
Appraise
Conclude
Critique
Decide
Defend
Diagnose
Evaluate
Judge
Justify
Rank
Recommend
Support
From Assessing Student Learning Workshop, Janet Fulks, Bakersfield College, 2004
Psychomotor Domain – Learning Outcomes Related to Skills
Observe
Model
Recognize
Standards
Correct
Apply
Coach
Students
translate
sensory input
into physical
tasks or
activities
Students are
able to replicate
a fundamental
task or skill
once shown
Students
recognize
standards or
criteria
important to
perform a task
or skill correctly
Students use
standards to
evaluate their
own
performances
and make
corrections
Students apply
this skill or
ability to real
life situations
Students are
able to instruct
or train others
to perform this
skill in other
situations
Hear
Identify
Observe
See
Smell
Taste
Touch
Watch
*
Attempt
Copy
Follow
Imitate
Mimic
Model
Reenact
Repeat
Reproduce
Show Try
Check
Detect
Discriminate
Differentiate
Distinguish
Notice
Perceive
Recognize
Select
Adapt
Adjust
Alter
Change
Correct
Customize
Develop
Improve
Manipulate
Modify
Practice
Revise
Build
Compose
Construct
Create
Design
Originate
Produce
Demonstrate
Exhibit
Illustrate
Instruct
Teach
Train
From Assessing Student Learning Workshop, Janet Fulks, Bakersfield College, 2004
Affective Domain – Learning Outcomes Related to Attitudes, Behaviors, and
Values
Receiving
Responding
Valuing
Organizing
Characterizing
Students become
aware of an
attitude, behavior,
or value
Students exhibit a
reaction or
change as a result
of exposure to an
attitude, behavior,
or value
Students
recognize value
and display this
through
involvement or
commitment
Students
determine a new
value or behavior
as important or a
priority
Students integrate
consistent behavior
as a naturalized
value in spite of
discomfort or cost
and the value is
recognized as part
of the person’s
character
Accept
Attend
Describe
Explain
Locate
Observe
Realize
Receive
Recognize
Behave
Comply
Cooperate
Discuss
Examine
Follow
Model
Present
Respond
Show
Studies
Accept
Adapt
Balance
Choose
Differentiate
Defend
Influence
Prefer
Recognize
Seek
Value
Adapt
Adjust
Alter
Change
Customize
Develop
Improve
Manipulate
Modify
Practice
Revise
Authenticate
Characterize
Defend
Display
Embody
Habituate
Internalize
Produce
Represent
Validate
Verify
From Assessing Student Learning Workshop, Janet Fulks, Bakersfield College, 2004
TO DEVELOP A
LEARNING OUTCOME,
BUILD IT IN PARTS…
Insert information
and/or
Students
will be
skills to be
learned
Insert
verb
able
to
from
list
learning of…
the ability to…
competence with…
knowledge of…
understanding of…
Insert what
student must do
demonstrate
to demonstrate
the learning
by
Example:
Students will be able to demonstrate knowledge of plant nutrient
requirements by identifying deficiency symptoms and listing actions
to alleviate them.
EXAMPLES…
 Students
will be able to demonstrate
knowledge of correct syntax by editing
written narratives from other students
 Students
will be able to demonstrate
understanding of human circulatory
systems by drawing diagrams illustrating
the path that blood follows in the body
 Students
will be able to demonstrate
the ability to use logic by constructing a
deductive argument from a real life
setting
POOR LEARNING OUTCOMES…





Students will get an introduction to
bioinformatics and its practical
applications.
Students will demonstrate proficiency by
conducting a review of published
research.
Students will demonstrate an
understanding of the connections
between religion and politics.
Students will demonstrate proficiency with
conversational Dutch by the end of the
semester.
Students will demonstrate an appreciation
for music of the Band Era.
LEARNING OUTCOMES ARE NOT
THE SAME AS COURSE
OBJECTIVES OR GOALS
Objectives



Present valuable skills, tool,
or content that enable a
student to do well in the
course
Focus on content that is
important in the classroom
or what the faculty will do
(input into the course)
Can be numerous, specific,
and detailed
Outcomes

Represent overarching
products of the course

Expresses higher level
thinking as observed
behaviors, skills, and useable
knowledge

Can be assessed as an end
product and evaluated
against specific criteria
SAMPLES FOR COMPARISON
Goal (the purpose of the course in more general terms)
The goal of the Scientific Writing course is to teach the basic
tenets of ethical research, sound scientific writing,
presentation of data, and development of proposals in
response to funding opportunities.
Objectives (details course contents or activities but not what the student
will do to demonstrate learning)
Learn how to organize the parts of a scientific paper that is to be
submitted as a manuscript for publication.
Learning Outcomes (describes what a
student will know or do at the end)
Students will demonstrate competency
in presenting data by developing
graphs for both continuous and noncontinuous data.
MINI-QUIZ
ARE THESE GOALS, OBJECTIVES, OR LEARNING
OUTCOMES?

This course introduces the concept of human
nutrition and its relationship with health
interventions, educational training, and policy.
 Students will demonstrate knowledge of nutrition by
documenting a nutritional problem, determining a
strategy to correct it, and drafting a set of preventative
strategies.
 Concepts of nutritional requirements and how they
vary with age, gender, and activity will be described.
GOOD LEARNING OUTCOMES ARE
ASSESSABLE!
 Define
expectations
 Develop
 Measure
criteria
the
outcome
YOU CANNOT ASSESS WHAT
YOU HAVE NOT DEFINED:
ESTABLISH CLEAR LEARNING GOALS
 Are
your learning goals clear?
 Are
they shared goals?
 Can
you talk about them easily?
STUDENT LEARNING
OUTCOMES: QUESTIONS TO
CONSIDER
 How
would you ensure students have
the opportunity to learn this?
 How would you evaluate this in a way
that is NOT dependent on a course
instructor’s opinion or grade?
 How would you compile the results of
your evaluation?
 How would you share the results?
 If you were dissatisfied with your
findings, what would you do about it?
OUTCOME DEVELOPMENT EXERCISE
 Use
the form and lists of active verbs in
each domain to describe the attitudes,
skills, and knowledge that you would like
your students to know or do as a result of
your course.
 Divide
them into domains
 Affective
 Psychomotor
 Cognitive
 Try
to get at least three in each section
47
DRAFT YOUR OWN LEARNING OUTCOMES
Type of learning
learning of…
the ability to…
competence with…
Students will
knowledge of…
demonstrate…
understanding of…
Active verb
by
Insert
verb
from
list
Knowledge, skill or attitude
of
Insert information
and/or
skills to be
learned
Action or behavior
Insert what
student must do
to demonstrate
the learning
Try to do
ten of them
WHY AREN’T GRADES
ENOUGH?
 Grades
evaluate a limited set of objectives
which may or may not be related to the
program objectives
 They tend to be inconsistent from
section-to-section and from term-to-term to
be a valid overall program assessment tool
 There is a conflict of interest when the instructor is the
only evaluator of whether students have met
program objectives
 Grades do not reflect long term learning and
accomplishments
 Many times they do little to capture perceptions,
attitudes, and skills
Knowledge
Recalling or remembering
something without necessarily
understanding, using, or
changing it
Comprehension
Understanding something that
has been communicated without
relating it to anything else
Good Assessment Tools:
Students will identify, describe, or recount in their own words…
-Written tests and essays
- Oral tests
- Reports
- Making lists
- Role play focused on accurate information
- One minute papers
Application…
Using a general concept to solve
problems in a particular situation or
using learned material in new and
concrete situations
Good assessment tools:
Students will apply, demonstrate, solve…
- Problem scenario
- Problem set
- Demonstration of skill
- Role play
- Lab report
- Field experience
- Internship report
Analysis…
Breaking something down into its
parts (identification of parts,
analysis of relationships among
parts, and recognition of
organizing principles)
Good assessment tools:
Students will categorize, examine, induce…
- Case study
- Journal
- Field experience/internship reflection
- Complex problems
- Conduct experiments and test hypotheses
- Observations
- Examinations of an object, writing, process
Synthesis…
Creating something new by
putting parts of different ideas
together to make a whole
Good assessment tools:
Students will design, integrate, relate…
- Project
- Research thesis
- Essay
- Design an experiment
- Annotated bibliography
- Create a video or poster
- Portfolio
Evaluation…
Judging the value of materials or
methods as they might be applied
in particular situations or judging
with the use of definite criteria
Good assessment tools:
Students will conclude, critique, justify…
- Debate
- Essay
- Report
- Journal
- Write and editorial/position paper
- Comment on author’s perspectives
- Book review
- Prepare a committee/team briefing paper