Transcript Document

Monitoring and evaluation
during SALW Awareness
programmes
Name, Organisation, Event
Location, Date
© SEESAC, 2006
Part 1 – 30 minutes
1.
2.
3.
Define monitoring and evaluation
Explain the differences between them
Explain why they are important
© SEESAC, 2006
Part 2 - 45 minutes
1.
Describe the different levels at which
monitoring and evaluation can be carried
out
© SEESAC, 2006
Part 3 – 40 minutes
1.
2.
3.
List some common tools for doing
monitoring and evaluation
Explain why it is best to use a mix of
tools
List some common criteria for selecting
monitoring and evaluation tools
© SEESAC, 2006
Part 1
© SEESAC, 2006
Exercise 1
1.
2.
3.
4.
What is monitoring?
What is evaluation?
What are the similarities and
differences?
Is it important to do M&E, and if so, why?
© SEESAC, 2006
What are monitoring and
evaluation?

An informal explanation:



Monitoring and evaluation are two forms of
activity which aim to measure how your work
is progressing
Measurement during a project = monitoring
Measurement after a project = evaluation
© SEESAC, 2006
A formal definition (SASP 2)…

Monitoring and evaluation (M&E) =
“Collecting and analysing information….
…to determine whether those groups engaged by a SALW
Awareness programme have,…..
…as a result of the intervention,….
…changed their awareness of, and attitudes and behaviour
towards, SALW,….
… in line with the stated programme objectives.”
© SEESAC, 2006
Table 9, (SASP 2)…
WHAT IT INVOLVES
WHEN IT OCCURS WITHIN
THE PROGRAMME CYCLE
Ongoing.
Monitoring
Tracking progress towards the
achievement of objectives, in order
to identify what is working and what
isn’t working so well, allowing a
degree of adaptability in strategy
and tactics as appropriate.
Evaluation
A more structured and formal
process of reviewing achievements,
in order to make judgements about
past effectiveness and learn from
experience to improve future
practice.
At fixed times – for projects lasting
more than 18 months, this would
normally include a mid-term review
as well as an evaluation conducted
at the completion of the project.
© SEESAC, 2006
Why are they important?






Test for effectiveness
Better appreciation of area where working
Learn how events have affected the work
Adapt programme both during project lifecycle (M) and before next phase (E)
Identify good practice to use elsewhere
Information to share with others
© SEESAC, 2006
The programme cycle
Feasibility study
Planning for
monitoring and evaluation
Needs and
capacity assessment
Evaluation
Analysing
and planning
Designing
Monitoring
and reviewing activities and materials
Field-testing
Implementing
activities
The time difference…
- Monitoring -
RESEARCH
+
ANALYSIS
DESIGN
IMPLEMENT
- Evaluation -
EVALUATE
© SEESAC, 2006
Part 2
© SEESAC, 2006
Imagine you are doing SALW
Awareness …
© SEESAC, 2006
Levels of M&E (SASP 2)
LEVEL OF
EVALUATION
PURPOSE
KEY QUESTIONS
Activities
To assess how well the programme
has been organised and whether
resources have been used efficiently.
- Are we sending people the
correct messages?
- Are the messages reaching the
right people?
Outcomes
To identify changes in knowledge,
attitudes and behaviour among target
groups that can be reasonably
attributed to the programme.
- Are there any signs that
knowledge, attitudes and beliefs
are changing?
- Are there any signs that
behaviour is changing?
Impact
To explore how a particular
programme may have made a
difference to the lives of specific
groups of people, e.g. better security.
- What impact has the
programme had in terms of
security / casualties etc.?
© SEESAC, 2006
Activity monitoring example..

Monitoring a TV spot


Watch TV!
Monitoring a community
meeting

Phone the local organiser
afterwards
© SEESAC, 2006
Outcome monitoring example…






Balkan Youth Union (BYU)
5th April 2003, central Belgrade
BYU and children destroyed 500 toy
weapons
Puppet show
T-shirts
MUP information leaflets to support
collection
© SEESAC, 2006
© SEESAC, 2006
Outcome monitoring example…

Outcome evaluation by:


Letters to BYU (hundreds)
Media coverage of interviews with the public
© SEESAC, 2006
Impact monitoring and evaluation…

Difficult




Other factors
Casualty figures
Crime levels
Observe weapons visibility
© SEESAC, 2006
Six questions…
1.
2.
3.
4.
5.
6.
How has your awareness programme reduced
the number of weapons casualties resulting
from weapons in target communities?
Are the messages being promoted reaching the
right people?
Are there any signs of changes in practice or
behaviour?
How has your awareness programme changed
security in targeted communities?
Are there any signs that knowledge, attitudes
and beliefs are changing?
Are the messages being promoted the right
ones?
© SEESAC, 2006
Part 3
© SEESAC, 2006
Things to consider:








Cost
Staffing
Skill-levels
Representativeness
Geographic coverage
Depth of explanation
Access to social groups
Level of participation
© SEESAC, 2006
Common M&E Tools





Interviews
Focus groups
Questionnaires
Secondary (desk) research
Participatory methods
© SEESAC, 2006
Map exercise…
Group Exercise…
Group 1:
 Cheap
 Give a deep understanding of target group’s feelings
about SALW
 Capture women’s views well
Group 2:
 Allow generalisations to be made about the wider
population
 Allow respondents to participate
Group 3:
 Build the skills of respondents
 Capture information about unexpected impacts
© SEESAC, 2006