Transcript Performance indicators for M&E of capacity building

UNFCCC Meeting on experiences with performance
indicators for monitoring and evaluation of capacity
building in developing countries
Rio De Janeiro, Brazil
6-7 November, 2008
Jamil H.Chowdhury
Bangladesh M & E Network
Dhaka, Bangladesh
Thing to be remember
“ In the evaluation of capacity development
the impact metaphor should be avoided.
The militaristic impact metaphor fails to
capture the essential features of capacity
development which is a process of
change and growth.”
(Horton, D. 2002)
1
2
Common confronting question in M & E of capacity building
Has the capacity building been well defined?
Are the widespread examples of M & E of capacity building?
What tools we should use for accessing national capacity in a
integrated way?
Do we have any standardized approaches to monitoring and
evaluating capacity building interventions?
Do we have standardized logical framework or gold standard for
assessing country capacity?
Are we clear about the relationship between capacity and
performance i.e. what elements of capacity are critical to
performance?
Do we understand the difficulty in quantifying many elements of
capacity before attributing capacity change to any single
intervention?
Can we separate capacity measurement from the process of
building capacity itself?
Rationale for M & E of capacity building
Identifying existing needs and gaps in the process of
implementation.
Understanding the status of implementation of
multilateral decisions.
Identifying needs for technical and financial support.
Assessing prioritization of climate change issue in
national and local planning and decision making.
Assessing arrangement for mainstreaming climate
change related capacity building within the context of
wider sustainable development.
Determine the level of country commitment and
ownership.
3
Issues of consideration for evaluating country capacity
Country level performance monitoring for assessing capacity
may be a preferred choice and easy to implement on a
regular basis.
Evaluating a country capacity is complex and
methodologically challenging.
Evaluation must be done for specific desired changes or
outcome variables in performance with an interval of every
three-five years.
Capacity evaluation tend to get mix up with performance
measurement.
Capacity indicators must be well identified and well defined
before they are evaluated.
Quality of capacity building remain usually unaddressed.
Measuring cost of capacity building at any point of time
remains a great challenge.
4
Suggested indicators for assessing country capacity
There are always unlimited range of national capacity that may
include:
 Level of national response or political commitment.
 Policy and regulatory changes.
 Budgetary provisions for capacity building.
 Ability and level of procurement of external resources for
enhancing capacity.
 Building organization or strengthening institutions both at
public and private sector.
 Capacity of vulnerability and adaptation assessment and
implementation of mitigating options.
5
Suggested indicators of country capacity (contd..)
Functional availability of climate change secretariat and
focal points.
 Coverage of climate change issues by the leading news
papers.
 Institutional arrangement for technology transfer.
 Level of implementations of adaptations measures.
 Level of interministerial or multi sector co-ordination to
address climate change issues.
 Type of bi-lateral and multilateral agreement signed for
strengthening capacity.
 Availability of baseline data on current capacity/capacity
needs assessment for monitoring future progress.

6
Suggested indicators of country capacity (contd..)


Instances of developing/strengthening research and
training institutions.
Level of involvement of civil society, academician and
other stakeholders.

Availability of national plan of action and national
strategies (inclusions of climate change agenda in the
national development plan).

Plan for development of technical human resources
(metrological, hydrological land, water & forest
management).
Training & awareness of print and electronic media
journalists for expanded advocacy.

7
Common barriers to measurement of capacity






Capacity building is rarely considered an important
management issue and always seen in isolation.
Most people seem uncertain as to what, when and how to
monitor when it comes to capacity building.
Absence of baseline data to compare the level of progress.
High staff turnover and repeated transfer of technical/
skilled human resources especially at the public sector.
Lack of availability of quality data or failure to identify
data sources.
Lack of proper documentation of best practices and
lessons learnt.
8
Suggested methodologies
Combine qualitative and quantitative methods.
Review national and policy and plan documents.
Review organizational records/documents (if
available)
Interview key professionals/focal points or conduct
focus groups with them.
Review research reports/secondary data.
Review leading newspaper reports for last 1 year for
monitoring change and media coverage.
9
Suggested methodologies (contd…)
Use scoring system for relevant indicators.
Engage external evaluators.
Consider participatory self assessment methods.
Make separate capacity assessment of
organizations/institutions as part of assessment of
national capacity.
Try Appreciative Inquiry (AI) for relevant
organizations/institutions for identifying strengths.
10
The way forward
For appropriate methods of M & E of capacity
building
we need to:

Assess the current level of capacity.

Identify the gaps in performance and capacity.

Map what inputs are available to support the current
capacity.

Determine which interventions will address the gaps
in capacity and performance.

Define performance objectives and determine the
outcome measures to evaluate progress.
11
The way forward (contd…..)

Plan long term financial support to develop capacity
building / training organizations.

Plan long term support for capacity building of
research organizations for systematic research and
monitoring of climate change issues.

Develop a time frame for periodic evaluation of
outcome.
12
Example of a success story
Best known example of capacity building is in the
EPI programme in the Bangladesh health sector.
Reasons for success
It started as a project with long time continued support
from UNICEF, WHO & other few INGOs
The project was mainstreamed into the health system
after 5-7 years.
The external and technical support continued even after
integration into the system.
Dynamic leadership at the initial stages brought success.
Strong BCC campaign reinforced the effort.
Successful community involvement and strong
community support helped sustain the mainstream.
13
Example of a failure case
Bangladesh bureau of statistics has been generating data on
various development indicators for more than 30 years, but
with little institutional capacity attainment.
Reasons for failure
 It failed to produce quality data to satisfy national &
international users.
 Lack of institutional leadership and ownership at any point
of time (the head of the institutions frequently changed).
 Lack of long time technical support from international
agencies.
 Lack of accountability and good governance.


Misdirected energy and waste of resource without
institutional capacity building goal.
Lack of institutional commitment.
14
End