Transcript Right click this link to PowerPoint Document

What do we mean by evidence?
Some pointers from the Irish reform
experience
Dr. Richard Boyle
Senior Research Officer
Institute of Public Administration
Dublin
Evidence for Healthcare Reforms conference, Limerick,
17th October 2008
Historical context
•
•
•
•
•
1960s - rational analysis
1970s - evaluation of individual projects
1980s - focus on control of public expenditure
- EU influence
1990s - public service modernisation
- value for money
- expenditure reviews
2000s - a focus on outcomes
Evidence – a tale of two reviews
Smoking ban
 Water fluoridation

Waiting lists ARE falling
Number of inpatient waiters
300,000
2001/02
2002/03
2003/04
2004/05
2005
250,000
6+ month
200,000
March 04 milestone
150,000
100,000
9+ month
50,000
12+ month
0
AMJ J ASOND J FMAMJ J A SOND J FMAMJ J A SOND J FMAMJ J A SOND J F MAMJ J A SOND
Source : MORI Ireland
But the public disagree......
% who think waiting times are getting...
70
60
62
Longer
56
52
50
52
49
40
Same
30
26
20
23
29
25
24
Shorter
10
8
4
0
2000 (S)
4
2001 (W)
2002 (S)
7
6
2002 (W)
2003 (S)
Base: All -Spring 2003 (1,000) Winter 2002 (1,002), Spring 2002 (1,041) Winter
2001 (1,021) and Spring 2000 (1,046) (Source: MORI Ireland)
What do we mean by evidence?
Randomised trials – the gold
standard??
 Empirical data or findings
 Information that we can trust
 Multi-disciplinary perspectives health economists, physicians,
behavioral psychologists, health
sociologists, and patient groups

Three criteria for judging evidence


Relevance in relation to a given assertion
Sufficiency – corroboration with other
instances of the same kind of evidence or
other kinds of evidence
 Veracity – the process of gathering
evidence has been free from distortion
and as far as possible uncontaminated by
vested interests
(Thomas, 2004: 5)
Creating Evidence - Some
Recent Trends
•
•
•
•
Performance indicators and targets
Output statements
Value for money and policy reviews
OECD review of the Irish public service
– At a stage of performance reporting rather
than managing for performance
– Performance dialogue between departments
and agencies is needed
– Align the many and various performance
reporting exercises
– Improve the quality of performance indicators
Evidence and the policy process: do
politicians use evidence?
Rarely
 Selectively

Politicians receive the evidence

Can’t be taken for granted

Engagement is prompted when
evidence is disseminated to key
interests/advocates, and the media,
where it is simplified and translated
Politicians can interpret and make
sense of the evidence

Most effective when they are
supported in this process
– Direct attention to particular pieces of
information
– Framing what it means in resource
terms
– Offering suggestions for questioning
Politicians view the evidence as
being significant
Outputs and outcomes – where does
the main interest lie?
 Ex ante or ex post?
 More trust in higher quality evidence
– central review and quality control

Policy and evidence: improving the
linkages

Evidence informing policy
– HRB review of impact of health research
– Children’s well being indicators
– Longitudinal studies

Policy informing evidence
– Setting the research agenda
– Mechanisms/structures for getting
evidence used
Where do we want to go?

From decision makers making judgements
based on a mixture of experience, intuition
and access to evidence-informed
judgements
 Towards judgements based on
authenticated, clearly structured and
widely shared sources of evidence
 But it is not a straight or an easy road.
Evidence in and of itself is not the answer.