Transcript Knowledge Translation

Knowledge Translation:
The steep path between evidence
generation and application
Brian Haynes
Health Information Research Unit
Dep’t of Clinical Epidemiology and Biostatistics
McMaster University
KNOWLEDGE IS THE ENEMY OF
DISEASE
The application of what we know
will have a bigger impact on health
and disease than any single drug or
technology likely to be introduced
in the next decade.
Sir Muir Gray, UK National Library for Health
Knowledge Translation…
…the organization, retrieval,
appraisal, refinement,
dissemination, and uptake of
knowledge (eg, important new
knowledge from health research)
Generalizable knowledge for better
clinical practice and healthcare
• knowledge from research
(sometimes called evidence)
• knowledge from the analysis of
routinely collected and audit data
(sometimes called statistics)
• knowledge from the experience of
clinicians and patients.
Cost-effectiveness of warfarin*
• Warfarin for atrial fibrillation
– $25CDN saved per stroke averted
• Aspirin for atrial fibrillation
– $65CDN saved per stroke
*Gustafsson C, et al. Cost effectiveness of primary
stroke prevention in atrial fibrillation: Swedish
national perspective. BMJ. 1992;305:1457-60.
What proportion of patients with
atrial fibrillation do not receive
anticoagulants?
50%
Bradley BC, et al. Frequency of anticoagulation
for atrial fibrillation and reasons for its nonuse at a Veterans Affairs medical center. Am J
Cardiol. 2000 Mar 1;85(5):568-72.
In Hamilton, Ontario,
“The Clot Capital of the Universe,”
the proportion of medical inpatients receiving
clot prevention according to guidelines is…
…33%
Current guideline adherence for diabetes
Intervention:
Ophthalmology assessment…
Proteinuria assessment…
Foot assessment…
HbA1c…
Cholesterol assessment…
Smoking status assessment…
46%
35%
30%
16%
55%
25%
-
80%
82%
72%
87%
68%
87%
In all, 73% of microalbuminuric
patients were not on ACE-I/ARB.
Hypertensive type II diabetic
patients were often left untreated
and only a minority of those treated
were optimally controlled. The
importance of an elevated systolic
pressure is underestimated and the
number of antihypertensive drugs
prescribed, insufficient. Screening
and treatment of albuminuria are
inadequate.
The routine application of what
we know can prevent or minimise:
• unknowing variation in clinical practice
• errors of commission and omission
• unsatisfactory patient experience
Evidence (from research) is necessary
but, of course, not sufficient…
...it has to be combined with the
circumstances of the individual
patient and the values of each
patient. But without evidence it
is improbable that patients,
professionals, and those who
manage resources, will to make
good decisions.
researchers
decision makers
generation synthesis policy
application
a
5
c
3
1
4
b
decisions
2
 Knowledge Translation 
MRC
Steps
from evidence generation to clinical application
CIHR
Steps: 1. generation of evidence from research; 2. evidence summary and
synthesis; 3. forming clinical policy; 4. application of policy; 5. individual
clinical decisions, including a) patient’s circumstances, b) patient’s wishes,
and c) evidence from research
Step 1. Generating Research Evidence
Barrier
• too little research
addressing “real
world” problems
Solutions
• large, simple
randomized trials
• “head to head”
comparisons
Step 2. Synthesizing Research Evidence
Barrier
• size and noise of
the research
enterprise
Solutions
• research into
rating, abstracting,
and synthesizing
research
How much synthesis do we need?
“..at least 10 000 Cochrane reviews
are needed to cover a substantial
proportion of the studies relevant
to health care that have already
been identified”
Susan Mallett & Mike Clarke
ACP Journal Club. 2003 Jul-Aug;139:A11.
When will we have our 10,000 reviews?
Growth of Cochrane Reviews and Protocols
2003
Non-Cochrane reviews: >50% of all reviews
2500 completed mid-2005
protocols
2000 completed mid-2004
1995
reviews
“…between 2010 and 2015”.
Mallett&Clarke, ACPJC 2003
Step 3. Developing Policy
Barrier
• problems in
developing
evidence-based
clinical and health
policy
Solutions
• national drug and
technology assessment
agencies
• local leadership
Step 4. Applying evidence in practice
Barrier
• poor access to
current best
evidence and
guidelines
Solutions
• development and
testing of information
systems that integrate
evidence and guidelines
with patient care
(eg Diabetes InCHARGE)
The McMaster PLUS project
• only a tiny proportion of all research is
“ready for application”
• only a tiny fraction of the “ready”
research is “relevant” to the practice of
a given clinician
• only a tiny proportion of the “relevant”
research for a given practitioner is
“interesting” in the sense of being
something new, important, and
actionable.
Evidence-Based Journals
Critical Appraisal Filters
50,000 articles/y
from 120 journals
~2,500 articles/y
meet critical appraisal
and content criteria
(95% noise reduction)
McMaster PLUS Project
Clinical Relevancy Filter (MORE)
~2,500 articles/y
meet critical appraisal
and content criteria
(95% noise reduction)
~20 articles/yr for
clinicians (99.96%
noise reduction)
~5-50 articles/y for
authors of evidencebased clinical topic
reviews
Dear Dr. Jones,
We want to alert you to NEW articles in the PLUS system.
These articles that have received very high relevancy and newsworthiness scores:
1. Bohlius
J, et al. Erythropoietin for patients with
malignant disease. Cochrane Database Syst Rev
2004;(3):CD003407.
Rated by: IM/General (patients
referred from Primary Care)
Relevance: 6 of
7
Newsworthiness: 6
of 7
2. Gourlay S, et al. Clonidine for smoking cessation.
Cochrane Database Syst Rev 2004;3:CD000058.
Rated by: IM/General (patients
referred from Primary Care)
Relevance: 6 of
7
Newsworthiness: 6
of 7
We hope that you will find these articles of value in your clinical practice.
Best wishes from the PLUS Team
PLUS Trial – Northern Ontario Physicians
134 nonrespondent
344 consent eligible
7 refused
consent
203 randomized: 10 communities
6 small clusters
Group 1 (3)
Group 2 (3)
4 large clusters
Group 1 (2)
Group 2 (2)
2 left study
Intervention
• Randomization to 2 different trial interfaces
Self Serve Version
Full Serve Version
• Ovid
• Ovid
• Stat!Ref
• Stat! Ref
• Pyramid of Evidence • Pyramid of Evidence
• PLUS Email Alerts
• PLUS Search Engine
PLUS Preliminary Findings:
% of Participants Using PLUS by Month
Full-Serve
70.0%
Percentage Using PLUS
70
60.0%
60
50
50.0%
40.0%
40
Con
30.0%
30
Inte
20
20.0%
Relative increase 58.7%, P=0.001
10
10.0%
Jun-05
May-05
Apr-05
Mar-05
Feb-05
Jan-05
Dec-04
Nov-04
Oct-04
Sep-04
Aug-04
Jul-04
Jun-04
May-04
Apr-04
Control cross-over begins
Mar-04
Feb-04
Jan-04
Dec-03
0
0.0%
RCT begins
Nov-03
Percentage of Participants Using PLUS
Percentage Self-serve
of PLUS Participants
PLUS by Month
Baseline (5 mo)
vsUsing
Full-serve
Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun
03
03
04 04 04 04
04 04 04 04 04 04 04 04
Month
05 05 05 05 05
Month
Self-serve
Full-Serve
05
Free EBM literature updating
service
http://www.bmjupdates.com
Free at www.bmjupdates.com!
(sponsored by BMJ Publishing Group)
Step 4. Applying evidence in practice
Barrier
• ineffectual continuing
education
Solution
• effective continuing
education and quality
improvement programs for
practitioners
Step 4. Applying evidence in clinical
decisions
Barrier
• ignorance about
barriers and their
solutions
Solution
• shift a portion of health
investment from services to
quality improvement
WHO estimates US$100B/yr for
health-related research
• not enough is for application research
• the balance is shifting slowly
• should there be a Nobel Prize for
applied research?
Step 5. Making better
clinical decisions
Barrier
Solutions
• not having the
• Computerized decision
right information at support
the right time
Effects of Computerized
Clinical Decision Support Systems
on Practitioner Performance and
Patient Outcomes
A Systematic Review
Amit Garg MD, Neill Adhikari
MD
, Heather McDonald MSc,
Patricia Rosas-Arellano MD,PhD, Phillip J. Devereaux
Justina Sam, R. Brian Haynes MD, PhD
, Joseph Beyene
MD,
Departments of Clinical Epidemiology and Biostatistics, McMaster University
Departments of Medicine, McMaster University, University of Toronto, and
University of Western Ontario
Department of Biostatistics and Epidemiology, University of Western Ontario
Ref: Garg et al. Effects of computerized clinical decision support systems on
practitioner performance and patient outcomes: a systematic review. JAMA
2005;293:1323-38.
,
PhD
Context – Computerized Clinical Decision Support Systems
 Software designed to directly aid in clinical decision making in
which characteristics of individual patients are matched to a
computerized knowledge base for the purpose of generating patient
specific assessments or recommendations.
Rules /
Algorithms
INPUT
Patient characteristics
• Automated through EMR
• By extra research staff
• By existing health care staff
• By the patient
• By the practitioner
Computer
OUTPUT
Recommendations
delivered to health
care provider
Outcomes
• Provider performance
• Patient outcomes
integrate into
workflow
• Directly by computer
• By pager
• By extra research staff
• By existing health care staff
Are CDSSs
clinically effective?
Did CDSS improve practitioner performance?
100 studies
“counting positive results on ≥ 50% outcomes measured”
Examined in 97 studies,
63 cited improvement (65%)
 In 16 of 21 (76%) reminder systems

 In 24 of 37 (65%) disease management systems


 In 19 of 29 (66%) drug dosing or prescribing systems
 In 4 of 10 (38%) diagnostic systems
Did CDSS improve patient outcome?
Update 100 studies
Examined in 52 studies,
7 cited improvement (13%)
 most had inadequate power to detect important difference
 none proven to improve definitive outcome such as mortality
 surrogate outcomes such as BP and HbA1C not meaningfully
improved in most studies
Reminder Systems
40 studies
Improved
Practitioner
Performance
- 76% -
Improved
Patient
Outcome
- 0% -
 Screening, counseling, vaccination, testing, medication use, or
the identification of at-risk behaviors
 CDSS successes were typically demonstrated in ambulatory care,
although one successful system was used in hospitalized patients
Disease Management Systems
37 studies
Improved
Practitioner
Performance
- 62% -
Most are RECOMMENDATIONS.
Range of problems, for example:
- diabetes care
- cardiovascular prevention
- incontinence in the elderly
- advanced directives
- ventilator support
- infertility
- corollary orders
- reduce unneeded health care utilization
Improved
Patient
Outcome
- 19% -
Step 5. Improving health care
decisions
Barrier
• low patient
adherence to
treatments
Solutions
• adoption of effective
strategies to assist
patients to follow
evidence-based health care
The weakest links
• Policy - especially at the local level
• Coordination - 4P
• Helping practitioners to recommend
effective treatments
• Helping patients to follow effective
treatments
The strongest link
• Organization of health care
knowledge according to the
hierarchy of evidence (evidencebased medicine)
The evolution of Evidence-Based
information systems
Systems
Summaries
Synopses
Syntheses
Studies
Examples
Computerized decision
support
Evidence-based textbooks
Evidence-based journal
abstracts
Systematic reviews
Original journal articles
KNOWLEDGE IS THE ENEMY OF
DISEASE
The application of what we know
will have a bigger impact on health
and disease than any single drug or
technology likely to be introduced
in the next decade.
Sir Muir Gray, UK National Library for Health