Transcript Youth Risk and Resiliency Framework - NC 4-H

Evaluation Basics
for Youth Programs
Ben Silliman, Ph.D.
Dept. of 4-H Youth Development
Reasons to Love
Program Evaluation
Personal: feedback on
personal effectiveness
Practical: information
for improving delivery
Professional: process
and outcomes data to
show accountability,
impact
Understanding Evaluation
Name the best thing(s)
you’ve seen/experienced
in a youth-at-risk
program
Name the worst thing(s)
you’ve seen/experienced
in a youth-at-risk
program
Dimensions of Evaluation
Good things come from…
A clear vision of what is
happening
A positive climate for youth
Programs that make a
difference
Organizations that run
effective programs
Dimensions of Evaluation
Good things come from
research-based…
Environmental scanning
Program evaluation
Outcome evaluation
Organizational/
professional evaluation
Environmental Scanning
What do people need?
What resources do people have?
Needs and Asset Mapping
Demographic Data
Community Economic and Social Surveys
Focus Groups
Targeted Needs Assessments
Community Coalition consensus
Program Evaluation
How good is the program?
Positive Climates/Best Practices
Safe Environments
Structure Appropriate for
Supportive Relationships
Social Inclusion
Standards of Conduct
Skill Learning Focused
Contribution-Oriented
Connections to Family, Community
Program Climate
Physical and Psychological Safety
o Safe, healthy facilities
o Conflict resolution
o Wellness habits
Program Climate
Appropriate Structure
o Clear consistent boundaries
o Age-based opportunities
o Effective monitoring
Program Climate
Supportive Relationships
o Warmth, affirmation
o Communication
o Positive guidance
Program Climate
Opportunities to Belong
o Acceptance, inclusion
o Cultural diversity/ tolerance
o Teamwork
Program Climate
Positive Social Norms
o High expectations for ethics,
behavior
o High achievement orientation
o Service orientation
Program Climate
Efficacy and Mattering
o Growth in confidence and
autonomy
o Ability to make a difference
o Meaningful changes effected
with individuals, organizations,
communities
Program Climate
Opportunities for Skill Building
o Mastery of practical skills in many
domains
o Ability to relate and navigate
varied cultures
o Improvements in vocationrelated skills
Program Climate
Integration with Family, School,
and Community
o Parent involvement and family
activities at home
o Increased academic performance
and school leadership
o Youth-adult partnership
Outcome Evaluation
What’s happening with the kids?
Logic Model:
Building a Game Plan
The Big Picture: Winning Legacy
Giant Steps: A Habit of Playing Well
Small Steps: Mastering the Fundamentals
The Right Stuff: Good Preparation
Setting the Table: Good Resources
The Logic of Evaluation
Inputs
Investments
of resources
Money
Staff
Volunteers
Equipment
Supplies
Effects of
constraints
Laws
Regulations
Funder
requirements
Outputs
Outcomes
Products of programs
Classes taught
Materials
distributed
Hours of service
Participants
served
Short-term outcomes
 Knowledge
Attitude
 Skills
 Aspirations
Long-term outcomes
(impacts)
 Behavioral change
 Social-economic
change
 Cultural change
Evaluating Outcomes
in Small Steps
Knowledge
Attitudes
Skills
Aspirations
Outcome Domains
Physical
 Health
 Safety
 Wellness
 Risk Reduction
Outcome Domains
Intellectual/Cognitive
 Critical thinking/
problem solving
 Academic achievement
 Project skills/mastery
 Decision-making
 Creative thinking
Outcome Domains
Emotional
 Coping
 Stress management
 Optimism
 Self-efficacy
 Character
Organizational/Professional
Evaluation
How (well) do leaders make things happen?
Organizational Processes
Serving People
Designing and Producing Better
Goods and Services
Building an Efficient and Effective
Workforce
Six Sigma Processes
Define the project goals and customer
(internal and external) requirements
Measure and determine customer needs
and specifications; benchmark competitors
and industry
Analyze the process options to meet the
customer needs
Design (detailed) the process
to meet the customer needs
Verify the design performance
and ability to meet customer needs
Professional Evaluation
Professional Standards
Novice-to-Expert Rubrics and
Certification for General &
Specialized Competencies
Individual and Team goals,
performance standards, and
professional development
Continuous quality improvement
Human Subjects
Satisfaction, description is
program feedback
Individual performance
change is consent-required
feedback
Replicating and Sustaining
Programs
Stage 1: Program running as
planned, having intended effect
(Process Evaluation/Formative
Evaluation/Quality Control)
Stage 2: Change in knowledge,
attitude, skill, or aspiration
(Outcome Evaluation/Summative
Evaluation/Achievements &
Impact)