Transcript Conducting a Basic Efficiency Resource analysis

WHERE BER STARTED
Evaluation of Oxfam GB’s
Global Climate Change
Campaign
Solution to evaluation
challenge of
considering value for
money of a complex,
multi-unit,
international campaign
1
EVALUATION CHALLENGES
Simplifying complex multi-unit programs
Many evaluations are about ROI, but ROI is
difficult to assess in social contexts
Nothing is good or bad, except in comparison to
something else
2
THE BER SOLUTION
Provide a simple framework for evaluating complex
multi-component programs, campaigns, or activities
Build on the basic concepts of SROI to evaluate unit's
impact compared to their resources
Offers a relative perspective on performance where
units of analysis are judged in comparison to their
peer units, operating under similar conditions
3
THEORETICAL FOUNDATIONS: MATRIX ANALYSIS
FRAMEWORKS
• Boston Consulting Group
• General Electric Grid
• Customer satisfaction quadrant
analysis by Andreasen
• Bloc modeling techniques used by
social network analysts
• Multi dimensional scaling
• SROI
4
BER VARIABLES
input constitutes a program’s resources which may be measured by
its budget, number of staff, pool of talent, social capital, or any
measure of capacity, concrete or abstract.
Output measures a program’s
High
impact, and will vary
according to a program's
purpose – may include
Input
behaviour change; public
Low
awareness; policy change;
reduced inequality; improved
environmental health… or any
other measure depending on
a program's goals
Below
Average
Efficiency
Average
Efficiency
Average
Efficiency
Above
Average
Efficiency
Low
High
Output
5
CONDUCTING A BER ANALYSIS
The example in following section is fictional and for illustrative
purposes.
6
1. SELECTING UNITS
OF ANALYSIS
For example: Coalition/partnership building; intra
organisational coordination; lobbying and advocacy;
media relations; online engagment; public
mobilization; research...



Input
Program budgets (perceived and 
real)
Number of staff

Number and level of staff


Output
How often a lobbying keyword
appeared in policy
Number of widgets produced
Number of people engaged
Perceptions of impact
7
2. DATA TYPES
• Quantitative input data may include budgets,
number of staff, or combined multi-dimensional
resource measure
• Quantitative output data may include process
evaluation measures such as the number of
people engaged by a campaign or media hits
• Qualitative measures can include perceived
program investments and perceived output
achieved
8
3. MEASUREMENT TOOLS
1. Units of Analysis
3. Perceived Input (Budgets)
•Most
•Average
•Least
•I don't know
2. Perceived Output (impact)
•Most
•Average
•Least
•I don't know
Research papers
•Most
•Average
•Least
•I don't know
•Most
•Average
•Least
•I don't know
Lobbying and
advocacy
•Most
•Average
•Least
•I don't know
•Most
•Average
•Least
•I don't know
Online engagement
Organisation X has influenced policies or legislation because of the following
activities:
Completely
Disagree
1
2
3
4
5
Completely
Agree
6
I can't say
Online engagement
Research papers
Lobbying and advocacy
9
4. VISUALIZATION APPROACHES
High
Low
Low
High
10
4. VISUALIZATION APPROACHES

High

Media relations
Public relations


Lobbying and advocacy
Coalition/partnership
building


Research papers
Public mobilization
Input

Low

Intra organisational
coordination
Online engagement
Low
High
Output
11
4. VISUALIZATION APPROACHES
12
CASE STUDY: OXFAM GB’S GLOBAL CLIMATE
CHANGE CAMPAIGN
High
4.5
Lobbying and advocacy
Partnerships with others
Media engagement
4.0
Impact
Global Oxfam affiliates
Policy analysis
Climate hearings
Staff in UNFCCC delegations
Public campaigns/mobilization
Rapid news dissemination
Effective com at policy events
Research papers
Perceived impact
3.5
Low
Visual stunts & media relations
Online campaigning
Adopt a negotiator
Using celebrities
Oxfam internal program links
3.0
2.0
Low
2.5
3.0
Perceived resourcing
Investment
3.5
High
13
5. INTERPRETATION
Use BER is as a starting point for deeper discussions into
the performance of intervention units, their challenges,
opportunities, and operating environment
Understand the units of analysis and the informants who
shared their perceptions
Not all units within a program operate under the same
conditions
Some units contribute indirect effects, by empowering
other units
14
LIMITATIONS AND RISKS
It is easy to draw conclusions from the simple
visualizations that would never stand in the
face of a deeper understanding of the reality
behind charts
Kotler et al. (2005) noted, reliance on matrix
approaches prompted a number of companies to
sell off strategic assets and plunge into
businesses that they lacked the experience to
manage
15