Transcript lecture 11,12

Evaluating Therapy
Articles
Lecture 11, 12
1
Randomized Trial
Not
Outcome
Outcome
Exposed
a
b
a+b
c
d
c+d
a+c
b+d
a b
c d
Not
Exposed
a+b+c+d
2
Results of randomized
controlled trial
• Mean difference (CI)
• ARR, RRR, NNT (CI)
3
• Most scientific papers are presented in a
standard format:
Abstract
Introduction (why the research was done),
Methods (how the study was done and what
analysis was used),
Results (what was found),
and Discussion (what the results mean).
4
5
Steps of critical
appraisal
A. Are the results of study valid
B. Are the valid results of this study
important ?
C. Can we apply this valid, important evidence
in caring for our patient?(usefulness in
clinical practice)
6
Example
• A 75-year-old man is seen in our office
after being discharged from hospital 2
weeks previously. During this admission
he underwent a carotid endarterectomy
after suffering a transient ischemic
attack (TIA) and being diagnosed with
significant carotid stenosis. His hospital
stay was uncomplicated and his discharge
medications included metoprolol 50 mg
twice daily for hypertension and aspirin
81 mg daily.
7
• Today, he has brought us an article from
the Internet describing the benefits of
statins for stroke prevention, and he
wonders what this medication is and if he
should take it. Our note from his last visit
showed that his total cholesterol was 200
mg/dl/L, HDL-cholesterol was 65 mg/dL,
and LDL-cholesterol was 80 mg/dL. His
examination was unremarkable.
8
• Based on this scenario, we posed the
following question: In a patient with
history of TIA, carotid endarterectomy,
hypertension, and normal lipid profile, does
therapy with a statin decrease risk of
stroke?
• Using PubMed Clinical Queries we
identified the MRC trial which might help
us answer this question.
• (We can also found the MRC trial in ACP
Journal Club Online.)
• MRC/BHF Heart Protection Study. Lancet
2002; 360: 7–22.
9
A. Are the Results of Study
Valid
• The issue of validity speaks to the
"truthfulness" of the information.
• The evidence that supports the validity or
truthfulness of the information is found
primarily in the study methodology. Here
is where the investigators address the
issue of bias, both conscious and
unconscious.
10
11
• If the study fails any of the criteria
discussed above, we need to decide if
the flaw is significant and threatens
the validity of the study.
12
1. Was the assignment of patients to
treatment randomized?
• Randomization balances the
treatment groups for prognostic
factors, even if we don’t yet know
enough about the target disorder to
know what they all are.
• We should insist on random allocation
to treatment because it comes closer
than any other research design to
creating groups of patients at the
start of the trial who are identical in
their risk of the event we are trying
to prevent.
13
2. Was the randomization concealed?
• We should look to see if randomization was
concealed from the clinicians and study personnel
who entered patients into the trial.
• If allocation was concealed, the clinicians would
be unaware of which treatment the next patient
would receive and thus unable, consciously or
unconsciously, to distort the balance between the
groups being compared.
• As with failure to use randomization, inadequate
concealment of allocation can distort the
apparent effect of treatment in either direction,
causing the effect to seem larger or smaller than
it really is.
• Often, articles don’t state whether the
randomization list was concealed, but if
randomization occurred by telephone or by some
system that was at a distance from where
patients were being entered into the trial, we can
be assured by this.
17
3. Were the groups similar at the start
of the trial?
• We should check to see if the groups were similar
in all prognostically important ways (except for
receiving the treatment) at the start of the trial.
• As noted above, the benefit of randomization is
the equal distribution of potential confounders
between the study groups.
• However, baseline differences between the study
groups may be present due to chance. If the
groups aren’t similar, we must determine if
adjustment for these potentially important
prognostic factors was carried out. It is
reassuring if the adjusted and unadjusted
analyses yield similar results.
18
19
20
4. Was follow-up of patients
sufficiently long and 5. complete?
• If, for example, patients receiving the
experimental treatment dropped out because
of adverse outcomes, their absence from the
analysis would lead to an over-estimation of
the efficacy of the treatment.
• Drop out should not be more than 20%
• worst case analysis
• We should also ensure that the follow-up of
patients was sufficiently long to see a
clinically important effect.
21
6. Were all patients analyzed in the
groups to which they were randomized?
• Because anything that happens after
randomization can affect the chance that
a study patient has the outcome of
interest, it’s important that all patients
(even those who fail to take their
medicine, or accidentally or intentionally
receive the wrong treatment) are analyzed
in the groups to which they were allocated;
•
22
24
25
7. Were patients, clinicians, and
study personnel kept blind to
treatment?
• Blinding is necessary to avoid patients’
reporting of symptoms or their adherence
to treatment being affected by hunches
about whether the treatment is effective.
• Similarly, blinding prevents the report or
interpretation of symptoms from being
affected by the clinician’s or outcomes
assessor’s suspicions about the
effectiveness of the study intervention.
26
• For example, in the North American
Symptomatic Carotid Endarterectomy
Trial (this study randomized patients with
symptomatic carotid stenosis to either
carotid endarterectomy or medical
therapy with aspirin),
• the patients in the surgical group could
obviously not be blinded to the treatment
they received.
• Outcome events were assessed by four
groups: the participating neurologist and
surgeon; the neurologist at the study
center; “blinded” members of the steering
committee; and “blinded” external
adjudicators.
28
8. Were groups treated equally, apart
from the experimental therapy?
• Blinding of patients, clinicians and study
personnel can prevent them from adding
any additional treatments (or “cointerventions”), apart from the
experimental treatment, to just one of the
groups. Usually, we can find information
about co-interventions in the methods
and/or results sections of an article.
29
• 9. Conclusion validity
• Sample size,
• Standardization and adherence to
the protocol
30
• 10 control of external factors
• (environment and constancy of
conditions)
31
•
•
•
•
11. outcome
- is it a valid outcome
Is it measured accurately
12. data analysis
32
34
B) ARE THE VALID RESULTS OF THIS
INDIVIDUAL STUDY IMPORTANT?
35
• Chloramphenicol treatment for acute infective
conjunctivitis in children in primary care: a
randomized double-blind placebo-controlled trial
• The mean difference in the time to cure was 0·3
days (log-rank test p=0.025).
• Clinical cure by day 7 occurred in 83% of children
with placebo compared with 86% of those with
chloramphenicol (risk difference 3·8%, 95% CI –4·1%
to 11·8%).
www.thelancet.com Vol 366 July 2, 2005
37
1. What is the magnitude of the
treatment effect?
38
42
• We can describe the adverse effects of therapy in an
analogous fashion, as the number needed to cause harm to
one more patient (NNH) from the therapy.
• The NNH is calculated as 1/ARI. In the statin study, 0.03%
of the control group experienced rhabdomyolysis compared
with 0.05% of patients who experienced this in the group
that received a statin. This absolute risk increase of
|0.03% − 0.05%|=0.02% generates an NNH over 5 years of
5000.
• This means that we’d need to treat 5000 patients with a
statin for 5 years to cause one additional patient to have
rhabdomyolysis. Thus, the NNT and NNH provide us with a
nice measure of the effort we and our patients have to
expend to prevent or cause one more bad outcome, and
their attractiveness as an effort:yield ratio (or “poor
clinicians’ cost-effectiveness analysis”) is easily recognized.
46
2. How precise is this estimate of the
treatment effect?
• Like any other clinical measure,
NNTs are estimates of the truth and
we should specify the limits within
which we can confidently state that
the true NNT lies.
49
52
53
C. ARE THE VALID, IMPORTANT RESULTS
OF THIS INDIVIDUAL STUDY APPLICABLE
TO OUR PATIENT?
• To evaluate the relevance of the
study’s benefit (=efficacy) to
clinical usefulness (=effectiveness),
examine carefully the study
population.
59
• Were the study participants sufficiently
different from my patient that this study doesn't
help me at all?
•
• Were all clinically important outcomes
considered?
•
• Are the likely treatment benefits worth the
potential harm and costs?
60
61
62
See suggested therapy
work sheet
64
• Further reading about individual
randomized trials
• Guyatt G, Rennie D, eds. Users’
Guides to the Medical Literature. A
Manual for Evidence-Based Clinical
Practice. AMA Press: Chicago, 2008.
65
• 1. write a clinical scenario
• 2. write an EBM question
• 3. Conduct a clinical search step by
step
• 4. explain in details how did you
search the pub med with Search
terms
66
Homework
• 6. select A RCT, Systematic review,
Cohort study or a case control study
• 7. summaries the study result in
appropriate table and write the author
conclusion
• 8. critically apprise it using the provided
format
• 9. what is your conclusion
67