Transcript Assessment and Service-learning

Assessment of Service-learning Courses in Higher Education:
Understanding the Rationale and Techniques
Kathy O’Byrne, Ph.D.
UCLA Center for Community Learning
May 5, 2010
Agenda
 Welcome and Introductions
 Definitions of assessment and service learning
 Learning outcomes
 Research options: unit of analysis
 Integration into your syllabus & examples
 Resources
 Questions and Discussion
Your campus and your experience
 History of your service
 History with assessment?
learning program?
 Institutional resources?
 Disciplines?
 Campus leadership?
 Service learning courses or
others?
 Research and publications?
 Faculty development
experience?
Key components of assessment
 Assessment is linked to learning




goals/outcomes.
Meaningful work in service learning
courses is designed to elicit higher
order thinking
Multiple forms of evidence
Assessment activities should draw
on the affective and cognitive
domains of learning
Self-assessment is often part of the
overall design
 Student involvement and feedback is
important data to guide future
learning
 Consider an examination of
processes as well as outcomes
 Pre-planning and integrated with
instruction
 Expectations apparent to students
Definitions and Common Language
Assessment practices are a reflection of
your values.
Assessment is a means of understanding
the learning process, and improving the
quality of learning that occurs.
Assessment is the systematic application
of tools and techniques to better
understand and improve the process of
learning by everyone involved.
Assessment is a mind set that asks
questions– good questions, hard
questions, about what and how students
are learning.
Assessment is an ongoing process aimed at understanding and improving
student learning. It involves making our expectations explicit and public;
setting appropriate criteria and high standards for learning quality;
systematically gathering, analyzing and interpreting evidence to determine
how well performance matches those expectations and standards; and
using the resulting information to document, explain, and improve
performance.
Assessment…can help us focus our collective attention, examine our
assumptions, and create a shared academic culture dedicated
to…improving the quality of higher education.
What is service learning?
 Equal parts “service” and “learning”
 Reflection is key
 Embedded in academic, credit-bearing courses
 Connection to graded assignments
 Reciprocal benefit
 Collaborative approach to projects
Examples of strong service learning programs
 Tulane University
 University of San Diego
 Kapiolani Community
College
 University of Washington
 UCLA
Options for assessment: unit of analysis
 Students: learning
outcomes, knowledge,
skills, attitudes, behaviors
 Faculty: changes in
teaching and instruction
 Community: impact of
the project, qualities of
the partnership
Students
Faculty
Community
Potential service-learning outcomes for students
Communication
Skills (written and
oral)
Interpersonal Skills
Cultural awareness/
Appreciation of
diversity
Improved use of
technology
Logical reasoning
Application of
theory
Self-understanding
Critical thinking
Investigation of
Power relationships
Analysis of social
institutions
Explication of
values
Identify and frame
problems
Active listening
Negotiation skills
Public dialogue on
shared issues
Creative conflict to
produce growth
Political
imagination:
reimaging futures
Mentoring
Commitment to
service/service ethic
Career
Development
Values clarification
Character Education
Civic Engagement
Perspective
Transformation
Other?
Student learning and
assessment issues
1.
2.
3.
Define student learning outcomes: knowledge,
skills, attitudes or values?
Systemic issues: what is the role of the student, faculty,
sites? How will students demonstrate learning?
What data will you use? Graded assignments? Pre-post
instruments? Other?
Service-learning outcomes
for assessment
1.
2.
3.
4.
5.
Understand and apply knowledge
Develop identity and values
Develop community awareness and practice community
engagement
Demonstrate a commitment to lifelong learning and
action
Awareness of social justice issues
Summarize overall plan
1.
2.
3.
4.
Expectations/assumptions about students
Learning outcomes: what they should know and/or know
how to do
Teaching strategies in support of outcomes
Assessment Strategies: who? when? how? where? what?
Five Assessment techniques:
Strategy #1: Critical Incidents




Describe an incident that was “critical” to you.
It can be surprising, frightening, enlightening or
something that made you proud
What made it significant?
What did you learn or how did it change your
worldview/assumptions?
Strategy #2: Concept mapping
 One main concept from the course
 Additional cards or post-its that students arrange in a patter
 Related concepts connected or close to one another
 Policy, diversity, social justice, the role of government,
economic factors, etc.
Strategy #3: Problem based learning
 Based on training from medical schools
 Students write about an issue related to their service
learning work.
 Have to provide a potential solution for change in the
community, that includes classroom/curriculum
learning and connections to the service learning
site/staff/meaningful work they performed.
Strategy #4: Focus Groups
 Small groups of students who all worked at the same site
answer structured, pre-planned questions about the
organization, the community, the work they accomplished,
and the future of the organization.
 If change is needed, who would have to get involved to make
that happen? Elected officials? Residents? Others?
Strategy #5: Pre-post instruments
 Can focus on knowledge, attitudes, beliefs, values related
to the work of the organization.
 Can have students complete the pre-test and return it
after they complete the post-test to compare their
answers.
 Can break down differences according to gender, majors,
other characteristics including grades.
Other options for assessing service learning:
faculty or academic departments
 Departmental views for integrating service learning into a
major or minor;
 Faculty transformation after teaching a service learning
course
 Research and publication on discipline-based service learning
Assessment with
Community Partners
 Satisfaction
 Outcomes for community
 Impact
residents or agency clients
 Connection of projects to
public policy
 Research that informs
decision-making
 Visibility
 Building the organization’s
capacity
Example: Sample questions for service
learning community partners
 Rate student learning: cultural awareness, appreciation
of diversity, communication, critical thinking,
negotiation, career exploration, understanding of nonprofits and public policy issues, etc.
 General: understanding of their responsibilities, how
well prepared were students?
 What was the meaningful work that was performed?
Your syllabus
 Definition of service learning
 Learning outcomes
 Community partners and
criteria
 Job descriptions for each site
 Connection to graded
assignments
 Logistics
Some learning outcomes












Course Competencies and Learning Outcomes:
Students in this course will learn:
Definitions of civic engagement and service learning;
Current and historical national trends in higher education to promote democratic action and civic
engagement;
The diverse traditions of service and the history of service movements;
How civic engagement and participation in public life contribute to overall quality of life in the
community;
An overview of the issues raised by theories and empirical studies of links between individual
motivation and success, access to resources and distribution of wealth;
A description of the national movement for reform of undergraduate education;
Trends in research and scholarship that are impacted by this movement;
Definitions and research on sustainable environments, including (but not limited to) issues of water
usage, endangered species and native habitats, transportation, and pollution.
The role of community-based organizations that support a variety of sustainable environment
programs, non-profits and governmental agencies;
The potential range of personal choices that individuals can make to promote sustainable
environments in Los Angeles.
Principles of university/community
partnerships (CCPH)










Principles of good community-campus partnerships:
Partners have agreed upon mission, values, goals, and measurable outcomes for the
partnership.
The relationship between partners is characterized by mutual trust, respect, genuineness, and
commitment.
The partnership builds upon identified strengths and assets, but also addresses areas that need
improvement.
The partnership balances power among partners and enables resources among partners to be
shared.
There is clear, open and accessible communication between partners, making it an ongoing
priority to listen to each need, develop a common language, and validate/clarify the meaning
of terms.
Roles, norms, and processes for the partnership are established with the input and agreement
of all partners.
There is feedback to, among, and from all stakeholders in the partnership, with the goal of
continuously improving the partnership and its outcomes.
Partners share the credit for the partnership's accomplishments.
Partnerships take time to develop and evolve over time.
A “menu” of service learning options:
Based on time available, resources, and
experience
 An “Appetizer”: Pre-post research on knowledge,
attitudes, etc.
 An “Entrée”: Ongoing collection of data throughout the
academic term or a comparison of service learners to nonservice learners
 A “Dessert”: small scale study at the end of a course.
Examples
 Attitude toward Korean immigrant neighborhoods before and
after the service learning experience
 Compare the learning outcomes of two sections in a large general
education course (service learning and non)
 In what way would you say you have changed your mind—if only a
little bit—about one issue, as a result of the service learning
work?
Resources: Publications and Conferences
Journal of Applied Learning in Higher Education):
2009 Ash and Clayton
 Generating, Deepening, and Documenting
Learning: The Power of Critical Reflection
in Applied Learning
 This article will consider the meaning of critical
reflection and principles of good practice for designing it
effectively and will present a research-grounded, flexible
model for integrating critical reflection and assessment.
http://www.missouriwestern.edu/appliedlearning/iss
ues.asp

1999: Where’s the Learning in Service Learning?
Janet Eyler and Dwight Giles.






National Service Learning Clearinghouse
Bonner Foundation
Campus Compact


Tenth International Research Conference on
Service-learning and Community Engagement
International Perspectives: Crossing Boundaries
through Research
October 28-30, 2010
Indianapolis, IN .
The Indiana University-Purdue University Indianapolis
(IUPUI) Center for Service and Learning, Indiana
Campus Compact, and the International Association for
Research on Service-learning and Community
Engagement.
October 28-30, 2010 at the Crowne Plaza at Historic
Union Station in Indianapolis, IN.
The featured theme for the 10th annual conference is
“International Perspectives: Crossing
Boundaries through Research.” Service-learning is
valued as an active learning strategy across the globe;
however, little is known about the ways that servicelearning is similar or different in varied contexts.
Understanding service-learning and community
engagement from diverse cultural perspectives will add
insight necessary for comparative research and to improve
practice.
.
Questions
 Reactions
 Discussion
 Relevance
?
 Need for further information
 Next steps for your service-learning course!
Contact information
 Kathy O’Byrne, Ph.D.
 UCLA Center for
Community Learning
 [email protected]