Transcript Train-the-Trainer Workshop: Evaluation of Demand Reduction

The Fundamentals of Evaluating
Prevention Programs
Colleen Ryan,
Canada’s Drug Strategy
Secretariat, Health
Canada
Health
Canada
Santé
Canada
Presentation Overview
2
•
Why Evaluate?
•
Why Don’t We Evaluate?
•
Role of Evaluation and the MEM
•
Challenges in the Field of
Prevention
•
Basic Steps in Evaluation and
What Will Be Accomplished in This Meeting
Health
Canada
Santé
Canada
Why Evaluate?
3

Decision-making and overall good management of initiatives

Program/project improvement

Understanding what work’s and what doesn’t

Tracking unanticipated outcomes

Overall accountability for investment

Public relations/fundraising
Health
Canada
Santé
Canada
Why Don’t We Evaluate?
4

Time

Financial Resources

Expertise

We already “know” the program is effective

Philosophy

Long-term change vs. short-term funds

Fear
Health
Canada
Santé
Canada
Evaluation and the MEM
5

The MEM as an important instrument of Evaluation

22 recommendations issued in the
2nd Round Evaluation

?? recommendations issued
in the 3rd Round Evaluation

Capacity issues (financial and
technical) identified as primary
reason for lack of progress to date
Health
Canada
Santé
Canada
Challenges in the Field of Prevention
6

Need to establish “perceived” risk for substance
use/abuse at the outset

Difficult to measure longer-term
impacts

Costly to track cohorts over a
prolonged period of time

Challenges in establishing valid
control or comparison groups
Health
Canada
Santé
Canada
Basic Steps in Evaluation
7

Step 1:
Identify goals (anticipated outcomes) – Logic Model

Step 2:
Describe the project/program – Logic Model

Step 3:
Identify the evaluation questions you want answered
– Logic Model and Performance Indicators

Step 4:
Identify data sources and data collection tools Performance Indicators

Step 5:
Collect the information (focus of next meeting)

Step 6:
Organize the information (focus of next meeting)

Step 7:
Analyze the data (focus of next meeting)

Step 8:
Report the results – reassess goals and program
content (focus of next meeting)
Health
Canada
Santé
Canada