Transcript THE EMCDDA AND THE REITOX NETWORK

Quality Assurance at the European
Monitoring Centre for Drugs and Drug
Addiction
Sandrine Sleiman
European Conference on Quality in Official Statistics, Rome
8-11 July 2008
Contents
• Scope of EMCDDA activities
• Quality assurance: operating framework
• Challenges and results
Who we are
• A decentralised EU agency
• Formally established in 1993
• Based in Lisbon, Portugal (operating
since 1995)
• The hub of drug-related information in
the EU
Guiding principles and priorities: 20072009
Principles
Priorities
• Scientific excellence
• Partnership
• Good governance and
efficiency
• Consolidating monitoring and
reporting activities
• Enhancing data analysis
• Communicating more
effectively
Collecting and analysing existing data
Areas:
• Epidemiology — drug situation, monitoring new trends, five key
indicators
• Interventions — health, social and criminal justice responses, best
practice in demand and supply reduction
• Legislation — national and Community strategies and policies
• Policy — political and institutional framework, coordination
Five Key indicators
• Drug use in the general population (students and young adults):
prevalence and patterns of drug use in the general population
• Drug users accessing treatment (TDI)
• Drug related infectious diseases (HIV, HCV, HBV)
• Drug related deaths and mortality of drug users
• Problem drug use: prevalence and patterns of problem drug use
Requires commitment from MS to invest in data collection on a routine basis
Type of data collected
• Annual registry and epidemiological data
• Descriptive and qualitative data (non-annual)
• Data from research studies
• Institutional data
• Database cases
• Coverage and availability of prevention, treatment
and harm reduction
• Statistics from law enforcement sources
Collecting and analysing existing data
• Reitox national focal points
(NFPs), national experts
• Annual reporting to EMCDDA
(national reports, standard
tables, questionnaires)
• Expert groups for developing
indicators (nominated by NFPs)
• Ad hoc working groups
Reporting system: standard templates
• National reports and selected issues (based on detailed
guidelines)
• Standard Tables (ST) for quantitative information
• Sturctured Questionnaires
supplementing ST and NR
(SQ)
for
qualitative
information
Quality Management at the EMCDDA
Data submission in
FONTE, including
quality control at
national level
National Focal
Points
&
Quality assurance at EMCDDA level
Quality
Control
by
Scientific
Units
Quality
Assurance
by
QA team
National
data
providers
National data
providers
Data
Validation
Data collection
instruments
Quality
overviews
Data collection instruments,
protocols and guidelines
National
Focal
Points
Main quality related outputs & reports

Annual Quality reports
• Implementation Needs Profiles on the 5 key indicators
• EU experts groups ad-hoc documents
• Regular reporting to EC on implementation of 5 KI and on
quality of reporting
Annual Quality reports
Quality criteria
Completeness, :insight, reliability, usefulness,
internal consistency
Quality reports includes feedback on :
National reports
1.
2.
3.
4.
5.
Global
evaluation
Method.
Aspects
By section :
level of details,
clarity, strong/
weak points,
suggestions for
improvements,
discrepencies
Compliance
with formal
r equirements
Layout
presentation
incl. deadline
Structured
questionnaires
Standard tables
- Level of details
(insuf., suff.,
good)
- Conciseness
(insuf., suff.
good)
- Discrepencies
with NR
,
C1
C2
C3
C4
M1
M2
M3
D1
+ Number of
tables uploaded;
st
Date of 1
upload;
Number of
uploads.
Criteria for National reports
•
Completeness: the report contains all the necessary and existent
information in order to provide an overview of the situation
•
Insight: the report includes complete and significant information,
giving an interpretation to the reported information, according to
social and political contexts.
•
Reliability: the extent to which the information in the report allows
comparisons (between different time periods).
•
Usefulness: information oriented to the targets; acceptable and
pertinent to the report objectives; no redundant information is
presented.
•
Internal consistency: the extent to have coherent information in the
report or to describe the reasons for a lack of internal consistency.
Criteria for data checking in STs
CNDA: Not clear whether all
data in the ST are new or not
MDA: Quantitative information
missing
CQUA: Quantitative information
needed clarification
MME: Methodological information
missing
CQUAL: Qualitative information
needed clarification
MRE: References missing
OTH: Others
CREF: References needed
clarification
DNR: Discrepancy between NR
and ST
Criteria for SQ
Level of details
Insufficient, sufficient,
good
Conciseness
Insufficient, sufficient,
good
Discrepencies with NR
Yes - No
General comment
Any other important
feedback
Implementation Needs Profiles
• Concerns 5 key epidemiological indicators
• Focuses on activities related to indicators and partially on
content
• Is a general analysis, helps to identify problematic areas, status
quo or improvement in data collection
Criteria used
• Most recent data
• Reference
• Description
• Weaknesses
• Activities implemented or ongoing/planned
• Actions needed
Quality assurance: work in progress
Quality Assurance…
•
•
•
is mentioned as a priority in the EU action plan
is mentioned in the external evaluation report
is part of the EMCDDA’s 3-year work programme
Quality documents should be …
•
•
•
•
considered as real tools for the EMCDDA and the REITOX network (as
well as for external actors)
aimed at scientific excellence and not administrative purposes
an overview of strengths and weaknesses of data or collection
processes
…
Quality assurance: work in progress
• Production of 5 KI methodological packages
• Definition of precise assessment criteria for 5 KI
based on :
•
•
•
•
minimum implementation standards
specific to each indicator
sensitive to national contexts
somewhere between pragmatism and perfection
• Redefinition of templates and guidelines with
adaptation of quality criteria
Quality assurance : work in progress
Understand
needs and
capacity to
deliver from the
Reitox network
Promise what can
be delivered
Priorities for value
delivery: revision of
reporting guidelines
Change capability
to match future
needs:
- Quality feedbacks
- Capacity
development
activities
Change capability
to match promise
Deliveries
matching
standards and
priorities
Case study: national reports
Success
• Recommendations for improvement are
generally applied as regards adherence to
guidelines:
-
Document layout
References to standard tables and structured questionnaires
Sources and use of Harvard for bibliography
Methodological information is more precise
Efforts in writing the National report
Case study: national reports
Gaps
•
•
•
•
Deadlines missed
Conciseness
Lack of insight on trends and interpretation of results
In some countries, it is still too early to have
information on trends
• Work on quality assurance depends on external
factors, such as Member States’ involvement in
quality issues
THANK YOU FOR YOUR ATTENTION
[email protected]
http://www.emcdda.europa.eu