Transcript Drafting the new drug strategy and action plan

The CZ example of
evaluating and drafting
of drug strategy
Lucia Kiššová
Jindřich Vobořil
Drug policy formulated in strategic documents
•
•
•
•
•
•
•
1990 – 1992 Commission of CSFR for Drugs
December 1992 – Christmas memorandum to the Government
1993 – set up of the Interministerial Drug Commission
1993 – 1996 – First Drug Policy Program
1998 – 2000 – Second Drug Policy Program
2001 – 2004 – National Drug Strategy
2005 – 2009 – National Drug Strategy and 2 Action plans
– 2005-2006 action plan
– 2007-2009 action plan
• 2010 – 2018 – National Drug Strategy and 3 Action Plans
– 2010-2012 action plan
– 2013-2015 action plan
– 2016-2018 action plan
• National strategy:
– More general, defines main goals and basic principles of drug
policy for the covered period, drug policy intervention areas
• Action plans:
– More specific, define activities, responsible institutions and
deadlines
Evaluation of drug policy 2005-2009
• Subject of evaluation
– National strategy 2005-2009
– Action plan 2007-2009
• Time frame
– 05/2009 – 03/2010
• Internal evaluation – National Drug Committee secretariat – body
responsible for drug policy coordination
• Aim of the evaluation – to learn:
– The extend to which strategic goals where achieved
(How has the drug situation changed)
– What were the main achievements and failures of drug policy in 5 years
Concept of evaluation
Based on the (simplified) policy cycle model
Findings and recommendations
What have we learnt in connection to drafting
new drug strategy
• Formal structure of strategic documents is convenient, most of the
strategic goals are relevant for the future
• The strategy and action plan were not fully cohesive/linked-up
– One strategic goal had no activity that could lead (directly or indirectly) to its
fulfillment
– Even though strategic goals were mostly focused on changing the drug
situation, the majority of activities were focused on coordination and
organizational framework
– In some areas majority of planed activities had limited potential to change the
situation, so even if implemented the strategic goal could not be achieved
– Only small no. of activities were essential to achieve strategic goals
• The character and potential of planned activities defined in Action plan
2007-2009 and also the level of successful implementation corresponds
with the success in achieving strategic goals and areas
Incidention matrix – all activities that contribute to the achievement of goals
Incidention matrix – activities that are essential to achieve goals
- prevention
Strategy goal – decrease the level ofeg.
experimental
use
Not achieved:
•
•
high level of experimental use – cannabis, alcohol …
The consumption of illegal drugs increased and of legal drug stabilized on a high
level
Prevention in Action plan 2007-2009
•
•
•
•
No activity leaded directly to specific strategic goals
High % (70) of activities focused on organizational-coordination frame – only 30 %
of activities were focused on solving a problem
Only 35 % - implemented activities in 2007-2009 and 50 % in 2005-2006
No improvement was noticed in this area (experts opinion); number of strong
points from 2004 were weakened (SWOT)
Strategy goal – Decrease the risks of drug use
Achieved:
•
•
•
Infection diseases and other health consequences on a low level
Stabilization of problem drug users
Relatively stable network of drug services
Harm-reduction in Action plan 2007-2009
•
•
•
Had the highest no. of activities leading directly to achieving most of strategic goals
Highest % (65) activities, which were specifically focused on problem solving,
Fulfilled/implemented 78 % of activities 2005-2006 (most from all 4 pillars), 41 % in
2007-2009
• Therefore more attention should be paid to better link action plan and
strategy – planned activities with strategic goals
• While planning check weather planed activities contribute to the
achievement of goals and weather the activities are realistic
• Define limited no. of priorities for specific time period in order to
concentrate attention, personal and financial resources
• Formulate activities clearly – unclear vague definition causes variety of
interpretation, it complicated implementation and evaluation
– Positive correlation was find between clear formulation of activity and the
level of achievement
• Czech drug policy can rely on the professional potential of experts,
availability of relevant information and on the institutional framework
(network of services for drug users, institutions responsible for drug policy
implementation.
– We should rely on this factors in implementing future drug policy
• The level of coordination of relevant institutions plays positive but also a
negative role in drug policy implementation.
– The future drug policy should pay attention to strengthen the area of
cooperation and coordination
• Negative factors that complicate the goal achievement is formality in
activity fulfillment, not realistic activities defined in action plan.
– In the future this should be eliminated by paying attention to set realistic
activities, that is possible to implement and more accurate definition of how
the activity could be achieved
• The Czech drug policy is negatively influenced by limited financial
resources, given priority to other public policies, low interest in drug field
by politicians and finally high level of social tolerance to use of legal drugs
and underestimation of risks related to drug use.
Drafting the new drug strategy
and action plan
• The process of drafting new strategic documents and the process of
evaluation were conducted in parallel, involved the same people therefore
the transfer of knowledge was easier
• The process of drafting new documents and evaluation was coordinated
by National Drug Committee Secretariat
• Strategy:
– We decided to keep the strategy and its structure more-less the same
– According to findings from drug situation monitoring we updated strategic
goals
• Action plan:
– Expert working groups were established
– Priorities of drug policy for the period covered by action plan (3 years) were
set by the secretariat of NDC
Establishment of working groups
• Working group for each intervention drug policy areas (7 areas)
– 7 areas – Prevention, Treatment, Harm reduction, Law enforcement,
Coordination and financing, Information, research, evaluation and
International cooperation
• Identify all relevant stakeholders for each area
• Each working group had to ideally consist of:
– ministries, NGOs, professional organization, individual experts, university
(Centre of adictology)
• 15-20 people per one group
• 60 experts all together involved in the process
Work of working groups
•
•
•
•
Participated in the evaluation and worked with outputs of evaluation
Met cca. 3x
Conducted SWOT analysis of situation in relevant drug policy area
Identified main problems (W) and strengths (S) – experts ranked W,S
according to their importance
• Experts than tried to find solutions to identified main problems by method
of brainstorming
– There was a possibility to send ideas also by e-mail after the meeting of
working group
– There was a criterion that proposed activities should cover defined priorities
of drug policy for the period of three years.
• Proposed solutions/ideas created the basis for defining activities for action
plan
• Secretariat of National Drug Committee made the selection of proposed
activities/ideas and drafted the new action plan.
The process of negotiation/commenting on action plan
• Draft of action plan was discussed:
– in the working groups that participated in the process of evaluation and
drafting
– in the Committee of regions
– Committee of Department representatives (ministries, institution)
– Then in the National Drug Committee (members are ministers and other high
level representatives of other institution)
• After that the draft action plan was submitted to the Government
Drug Policy Coordination
• Pros and cons of this process
(-) More work and more effort needed
(-) More time consuming
(-) Demands good negotiations and facilitation skills
(+) Involves all relevant stakeholders and experts from the low level to
the high level management
(+) Helps to steer the discussion, exchange views and therefore helps
to find commonly acceptable drug policy
(+) Supports the feeling of shared responsibility for drug policy
• Still keep in mind:
– Drug policy is still a public policy
– Even if you try to be it as evidence based and objective as possible
there will still be different points of views, individual interests of
ministries or individuals, political influence
Thank you and
[email protected]
[email protected]
Detailed information on the
evaluation of CZ drug strategy and
action plan
Background information for BC participants
National Drug Strategy 2005-2009
Main goals
• Decrease the level of any drug use prevalence and the potential risks
• To combat organized crime
Specific goals:
I. Stabilize / decrease the no. of
problem drug users
II. Stop the increase of experimental
drug use
III. Stabilize / decrease drug
consumption
IV. Decrease the risks of drug use
V. Guarantee availability of drug
services
VI. Decrease the accessibility of
drugs
Technical-organizational goals:
VII. Effective financing
VIII. Coordination
IX. Information to general public
X. International cooperation
XI. Evaluation of activities
Action plan 2007-2009
7 drug areas:
•
•
•
•
Prevention
Treatment
Harm reduction
Law enforcement
•
•
•
Coordination and financing
Information, research, evaluation
International cooperation
Action plan 2007-2009 had 43 objectives and 170 activities
Example of a structure of action plan
Aim of evaluation
• To learn :
– The extend to which strategic goals where achieved (How has the drug
situation changed)
– What were the main achievements and failures of drug policy in 5 years
• Organizational aspects:
– internal evaluation – National Drug Committee secretariat (a body responsible
for coordination and implementation of drug policy)
– 1,5 employee, no budget
– The Board of Drug Strategy Evaluation (5 experts) was established:
• Provided methodological guidance of the evaluation process and evaluation
tools
• Supervision of final outcome/findings and final version of evaluation report
Concept of evaluation
Based on the (simplified) policy cycle model
Stakeholders perception on the structure and
formulation of evaluated drug strategy and
action plan
We were about to draft new strategic documents so in the frame of
evaluation we asked stakeholders on their perception of current strategy
and action plan (We wanted to find out weather we should keep the
structure for the new strategy or change it)
• Method used
– Online questionnaire
• Target group:
– Representants of relevant ministries (33 people)
– Regions (regional drug coordinators - 14)
– Members of professional organizations (8 people)
Example
• Questions asked in the questionnaire:
– Were the strategic goals realistic?
– Did strategic documents clearly define a way (measurements) for reaching
strategic goals?
– Were the priorities clearly defined?
– Do you perceive a current action plan as an effective tool for reaching the
strategic goals?
– How well did the action plan reflected current drug situation and trends?
– How useful you find for your work the drug strategy and action plan?
– Do you find the structure of current strategy and action plan convenient for
you?
Evaluation of the process of
formulation
How the strategic documents were formulated
• During the process of drafting and adopting strategic documents in 2004
there was a personal change in the position of the main actor/stakeholder
responsible for drafting strategic documents.
• National strategy 2005-2009 was drafted by the previous stakeholder, the
following action plan 2005-2007 by new one.
• In those two documents some inconsistencies were found – drug policy
areas in the action plan and its activities did not fully corresponded to the
National Strategy. With the change of the main responsible stakeholder
original purpose/structure and content of action plan has been
affected/changed.
• Therefore we decided to focus our evaluation also on the process of
formulation of strategic documents.
Evaluation questions
Formulation
Q1: Have changes occurred within the formulation process?
Q2: To what extend is the Action plan and Strategy cohesive/linked-up?
Q3: What is the character/type of activities planed in Action plan?
Formulation
Q1: Have changes occurred within the formulation process?
Have strategic documents changed compared to their planed design?
• Methods used
– Comparison of official (and internal) documents – what structure and
contend was planed in 2004 what was approved in 2005
– Semi-structure interview with the former stakeholder
• Main finding
– There was a change in the original purpose and content of action plan
– Originally the action plan should contain the area Use of legal drugs, but it
was abandoned – alcohol and tobacco is still not integrated part of drug
policy (a failure of drug policy)
– There was also a change in the originally planed area Public relation for
Information, research, evaluation – therefore no links existed between
activities in the action plan and a specific strategic goal
Q2: To what extend is the Action plan and Strategy cohesive/linked up?
Q3: What is the character/type of activities planed in Action plan?
For Q2 we wanted to know how many activities really lead to strategic goals.
For Q3 we assessed the activities from the point of their formulation, their
potential to solve a problem and their significance for drug situation
• Methods used:
– Analysis of documents based on the idea of logframe analysis – it helps to link
and clarify mutual relationship among goals, objectives, outputs and activities
– Internal working group was set up (5 people) for this purpose – the working
group assessed each activity of action plan in 5 different criteria
Example
• Criteria assessed by the working group for each activity:
– formulation – was the activity clear and understandable? (yeas/no)
– context of problem solving:
• is the activity focused on solving a problem? (eg. Implementation of specific
prevention program)
• or is the activity focused on creating conditions/framework for solving the problem?
(e.g. Regular meetings of prevention committee)
– significance for drug situation:
• Activity with high potential to change the drug situation (eg. To adopt stricter
regulations for free selling of medicine containing pseudoefedrine)
• Activity with partial potential to change the drug situation (eg. To publish a
methodological manual for prevention)
• Activity with no potential to change the drug situation (eg. To realize a questionnaire
survey among teachers)
– relevance to strategic goals:
• Each activity was then assessed from the point of their relevance to individual strategic
goals (we had 11 strategic goals). The group assessed weather the activity was crucial
for reaching the strategic goal or contributed to the strat. goal only partially or did not
contribute at all.
• This way we found out that activities formulated in the area of prevention were rather
formal, had no direct link with strategic goals and very low potential to change the
situation.
Process of Implementation
evaluation
Evaluation questions
Q1: How successful was the implementation of Action Plan 2007-2009?
Q2: What factors influenced the process of implementation (positively and
negatively)?
Q3: What are the conditions for implementation of drug policy on the
regional level?
Q1: How successful was the implementation of Action Plan?
(monitoring of implementation)
• Methods used:
– Top-down approach - Objectives set on the central level are assessed from
the point of – how these have been achieved, what hindered their
achievement, and what is the consistency of outputs and outcomes with set
objectives.
– A questionnaire was sent to relevant stakeholders/actors – (16 ministries and
institutions directly responsible for activity implementation, 6 collaborating
subjects – professional organizations, 14 regions)
– An internal working group of the body responsible for evaluation assessed
the information submitted
Example
Implementation
• Structure of the questionnaire – asked for each activity (eg.):
–
–
–
–
–
–
–
–
–
Was the activity implemented? (yeas-no-partially)
Was it implemented on time? (yeas-no)
Describe the process of activity implementation (how it was implemented)
Describe problems in implementation/reasons for not implementing the
activity
Describe the outcome of activity implementation (weather the activity met
its original purpose)
If applicable, describe alternative outcomes of activity implementation or
non intended outcomes.
Provide data for defined indicators (indicators were set for each activity)
If possible, provide data of financial expenditures needed for activity
implementation
Do you plan to continue with this activity in the future action plan?
• Working group than assessed the information submitted and compared
it with real outputs/objectives planed in Action plan
• Eg. A stakeholder informed us that data on alcohol are regularly
collected and published in annual report therefore the implementation
of an activity is fulfilled. However the working group found that in reality
there is no such annual report and therefore marked the activity as not
implemented.
• This way we decreased the formality in fulfillment of activities and made
the evaluation more objective (eg. In the area of prevention declared
implemented activities decreased from cca 45 % down to 35 %)
Implementation
Q2: What factors influenced the process of implementation
(positively and negatively)?
• Methods used:
– expert groups for 7 drug areas – according to areas of action plan (cca. 60
people) (15-20 people per one expert group). Expert groups conducted
SWOT analysis for each area of action plan (in 2009).
– Comparison of SWOT analysis conducted in 2004 and 2009 (comparison of
opportunities and threads). Experts assessed weather the opportunity
identified in 2004 was taken or not and weather the thread identified in
2004 occurred within 5 years or not)
– On-line anonymous questionnaire (35 respondents) – state organizations,
local government, professional organizations – asking respondents to mark
the level of influence of selected factors (positively, negatively)
Example
• The questionnaire asked stakeholders to mark the level of influence to the
implementation of factors listed bellow (-3 very negatively, +3 very
positively):
–
–
–
–
–
–
–
–
–
–
–
–
Formulation of activity
Realistic planning
Allocated financial resources
Institutional framework
Personal resources
Available information needed for activity achievement
Level of motivation of professionals and institutions in drug field
Level of professional knowledge of people responsible for activity
implementation
Level of cooperation of relevant stakeholders/institutions
Level of coordination
Level of flexibility of responsible actors to act and react on new trends in drug
situation
Other – please mention
Implementation
Q3: What are the conditions for implementation of drug policy on
the regional level?
• Methods used:
– On-line questionnaire among regional drug coordinators (14 regions)
• Questions focused on the area of:
– Extend to which goals of national drug strategy has been introduced into
regional strategic documents (how many and which goals became a part of
regional strategic documents)
– Cooperation with state organizations
– Position of regional drug coordinator/possibilities to influence drug policy
implementation (where is the regional coordinator situated within
organization structure, what other areas except drug policy is drug
coordinator responsible for, weather regional drug coordinator is a member
of all relevant regional coordination bodies and working groups/committees
etc.)
– Development of conditions for implementation of regional drug policy in the
last 5 years- personal, political, financial, institutional (indicate weather the
situation in listed conditions worsened or got better)
Outcome evaluation
of CZ drug policy 2005-2009
Evaluation questions
Q2: What was the development in each drug area in the last 5 years?
Q1: What is the level of achievement of specific strategic goals?
Q3: What is the level of achievement of technical-organizational strategic
goals?
Outcome evaluation
Q1: What was the development in each drug area in the last 5 years?
• 7 drug areas – Prevention, Treatment, Harm reduction, Law
enforcement, Coordination and financing, Information, research,
evaluation and International cooperation
• Methods used:
– Expert working groups for each drug area (7 groups), 60 experts (15-20
experts in each working group)
– Each expert group assesses elements of SWOT analysis from 2004
• Is the strength still strong point or has it become week point?
• have the week points been eliminated?
• have we used identified opportunities or have the threads occurred or
not…?
– Expert groups than identified new elements of SWOT for 2009
Q2: What is the level of achievement of specific strategic goals?
• 6 specific strategic goals - focused on the change of drug situation:
I. Stabilize / decrease the no. of problem drug users
II. Stop the increase of experimental drug use
III. Stabilize / decrease drug consumption
IV. Decrease the risks of drug use
V. Guarantee availability of drug services
VI. Decrease the accessibility of drugs
• Methods used:
– Analysis of quantitative data/indicators of drug situation (data provided by
NFP)
Example
• We assessed the development in the following indicators:
– Goal I. – no. of problem drug users and their characteristics (average age,
injecting use, primary drug), no.of problem drug users in treatment (in-outpatient, substitution treatment)
– Goal II. – prevalence of drug use in general population, in school population,
development in the institutional framework and support to primary
prevention programmers
– Goal III. – alcohol and tobacco consumption, consumption of other illegal
drugs
– Goal IV. – drug related death (direct/indirect), drug related infection diseases,
no. of distributed needles and syringes, no.of tests for infection diseases, and
other intervention of low threshold facilities.
– Goal V. – availability and accessibility of services for drug users, socioeconomic characteristics of drug users in treatment
– Goal VI. – drug seizures, seizures of drug precursors (medicines containing
pseudoephedrine), laboratories for pervitin production, drug related law
offences, subjectively perceived availably of drugs in the marked
Q3: What is the level of achievement of technical-organizational
strategic goals?
• 5 Technical-organizational goals:
VII. Effective financing
VIII. Coordination
IX. Information to general public
X. International cooperation
XI. Evaluation of activities
• Methods used:
– Combination of results and findings from all parts of evaluation (evaluation
of formulation process, implementation and from expert working groups)
Success and failures of drug policy
• Finally the evaluation report concludes the success and failures of drug
policy within the years 2004-2009.
• For this we used synthesis of information and findings gained from each
step of the whole evaluation process.
Lucia Kiššová
[email protected]
Tel: 00420 725 721 467