Transcript Logic Model - 2005 National Conference on Tobacco or Health

Outcome-based Planning and
Evaluation
Gloria Latimer, Ed.S, Director of Community Programs
Jason Vahling, M.P.H., Community Program Specialist
Recommended Framework for
Evaluation in Public Health Practice*
*MMWR, Vol. 48, No. RR-11
Logic Model
Engages evaluation stakeholders
 Educates stakeholders about realistic
expectations for change
 Describes your program
 Focuses your evaluation design

Logic Model (continued)
Directs your data gathering efforts
 Justifies your conclusions
 Communicates evaluation results

Logic Models
Input:
A resource dedicated to or used by the
program.
Activity:
Type of service the program provides to
fulfill its mission. What the program does
with the inputs-how it goes about
transforming them into products.
Output:
The direct product of program operation.
Outcome:
Benefit to participants during or after
participating in the program.
Logic Model
INPUTS
RESOURCES
 money
 staff
 volunteer
 equipment
ACTIVITIES
OUTPUTS
SERVICES
PRODUCTS
 medicine
assistance
 training
 education
 counseling
CONSTRAINTS
 laws
 regulations
 funders’
requirements
 mentoring
 transportation
services
 case
management
 classes taught
 counseling
sessions
conducted
 educational
materials
distributed
 hours of
service
delivered
 participants
served
 meds
distributed
OUTCOMES
BENEFITS
FOR PEOPLE
 new
knowledge
 increased
skills
 changed
attitudes or
values
 modified
behavior
 improved
condition
 altered status
Identifying Outcomes




Envision the expected impacts of the program
on its participants/audience
Ask yourself – “What has changed for
individuals, organizations, or the community as
a result of this program?”
Look beyond what the staff does and ask “So
What?”
Use the “So That” chain of logic
“So That” Chain of Logic
Teacher training & development
of school-based curriculum
So that
Increased awareness, knowledge, attitudes,
and skills related to youth initiation
So that
Decreased social norms that
support tobacco use among youth
So that
Reduced initiation among youth
Selecting Outcomes to Measure



Which outcomes are most important &
most closely related to your core
business?
Which outcomes are the most
meaningful to the participants?
Which outcomes will provide the best
information for decision-making &
program improvement?
Selecting Outcomes to Measure
(continued)


Which outcomes can the program be
expected to influence in a non-trivial
way?
Which outcomes are most likely
achievable given the resources
available?
Colorado’s Logic Model
Community Public Health Programs
are Essential Components of
STEPP’s Program




Local public health agencies have core
functions
For social norms to change, involvement must
extend to where people live, work and play
Policy is an effective public health strategy and
voluntary and legislated policy interventions
are supported (clean indoor air, tobacco free
schools, illegal sales to minors as examples)
Industry influence can be more effectively
blocked at local level
The Challenge



Community Programs: Funding to 46 (from 8
in 2000) public health agencies serving 63
counties in the state through non-competitive
grants
Coordination:
– Federal/State/State Partners/Local Partners
Collaboration
– Defining roles
– Avoiding duplication
Colorado’s Experience: Linking
Outcomes Across Points of
Intervention
National
State
Local
Statewide Objectives




Based on Colorado’s Logic Models
Provides Colorado with the ability to broadly
evaluate program effectiveness
Demonstrates accountability to program
stakeholders including state and local officials,
policymakers, and community leaders
Policy focused
Statewide Objectives…cont.



Ensures that programs adhere to the
recommended CDC framework
Maximizes training and technical assistance
resources by focusing on several key goals &
objectives
Facilitates additional regional and county level
data collection
Example: Secondhand Smoke
Linked Objective
National:
Increase to 100% the proportion of worksites with
formal policies that prohibit smoking or limit it to
separately ventilated areas.
State:
Increase the proportion of worksites in Colorado that
have an official policy that prohibits smoking:
•In public areas by 20% to 85%.
•In work areas by 20% to 98%.
Local:
By June 30, 2003, increase by/to X% the proportion
of worksites that adopt formal policies that prohibit
smoking.
Example: Cessation Linked
Objective
National:
Increase to 75% the proportion of adult smokers who
stopped smoking for a day or longer in the last year.
State:
Increase the proportion of smokers who report that
they:
•Quit smoking for 1 day or longer in the past 12
months by 50% to 66%.
•Quit smoking cigarettes regularly 6-12 months ago
by 25% to 5.6%.
Local:
By June 30, 2003, increase by/to X% the proportion of
quit attempts among adult tobacco users in X
city/county.
Example: Youth Linked Objective
National:
Increase to 51% the proportion of jurisdictions with a
5% or less illegal buy rate among minors.
State:
Increase by 10% the number of retailers who refuse
to sell tobacco products to minors.
Local:
By June 30, 2003, by/to X% the proportion of
tobacco retailers that sell tobacco products to minors
in X city/county.
Conclusion
Logic models helped guide Colorado’s
programmatic and evaluation decisions
 Logic models were used to develop
Colorado’s statewide objectives
 Statewide objective help to link outcomes
at the national, state, and local levels

Assumptions




Every local program had the capacity to
evaluate the outcomes proposed by STEPP
The framework allowed local programs enough
flexibility to feel vested in the decision making
process because they could choose their own
activities
Local programs had familiarity with the logic
model
A database format would simplify the process
Lessons Learned




Not all local programs have data collection
capacity…strike a balance
Involve local stakeholders in the decision
making process
It takes time for local partners to understand
and apply logic model principles
Not all local programs are technologically
oriented
Questions, Comments, and
Concerns
THANK YOU!
[email protected]
303-692-2513
[email protected]
303-692-2578