Transcript Users Guides to Spin

• Design and Reporting Modifications in
Industry-Sponsored Comparative
• Psychopharmacology Trials
• DANIEL J. SAFER, M.D.1
• J nerve mental dis 2002
BMJ 2006;333;782-; originally
published online 6 Oct 2006;
Anders W Jørgensen, Jørgen
Hilden and Peter C Gøtzsche
review
meta-analyses of the same drugs:
systematic
supported meta-analyses and other
Cochrane reviews compared with
industry
Are you at risk?
falling victim to misleading
presentations as a result of conflict
of interest of investigators
Not at all
A little
Moderate risk
High risk
Spinning
• get in groups of 2 to 5
• write down 3 ways in which papers can
mislead
• 3 minutes
Spinning
• get in groups of 2 to 5
• write down three ways to avoid
being mislead
• 3 minutes
Conclusions and Funding,
Als-Nielsen, JAMA 2003
• 25 Cochrane reviews, 370 RCTs
– random selection of 167 of 1081 (issue 2, 2001)
– excluded
•
•
•
•
101 < 5 studies
6 no concealment variability
2 no binary outcome
16 non-pharmacological
• abstracted
–
–
–
–
sources of funding
effect on primary outcome
adverse effects
methodological quality (concealment, blinding)
Scale Used to Grade Conclusions in Trials
6 Points
Experimental intervention highly preferred and should now be considered
the standard intervention in all patients, or similar
5 Points
Experimental intervention preferred to control, but further trials still
indicated, experimental may be more costly, or similar
4 Points
Experimental and control intervention about equal, but the experimental
cheaper, easier to administer, or similar minor advantage
3 Points
Experimental and control intervention about equal, but the control may be
cheaper, easier to administer, or similar minor advantage
2 Points
Control intervention preferred to experimental intervention, but
experimental intervention might be promising under some
circumstances, or similar
1 Point
Control intervention highly preferred and should now be considered the
standard intervention in all patients, or similar
Relation Between Funding Source and
Conclusions in 370 Randomized Drug Trials
Funding
# of
Trials
Nonprofit organizations
67
4 (3-5)
11 (16.4)
Not reported
106
5 (3-6)
32 (30.1)
51
5 (4-6)
18 (35.2)
For-profit organizations
146
6 (5-6)
74 (50.6)
Total
370
5 (4-6)
135 (36.4)
Nonprofit and forprofit organizations
Median Score # (%) of Trials
Scoring 6 Points+
(IQR)+
Abbreviation: IQR, interquartile range.
*Conclusions in trials were assessed by a 1-6 point scale. If the conclusions
recommended the experimental drug as the treatment of choice without disclaimers,
6 points was assigned, and if not, 1-5 points was assigned.
+p<.001, using Kruskal-Wallis test (medians) or X2 test (proportions)
Als-Nielson, JAMA, 2003
• bigger effect size, more likely
stronger recommendation
• blinding, more likely stronger
recommendation
• after adjustment, industry funding,
odds ratio 5.3 (95% CI 2.0 to 14.4)
1
Read methods and results
bypass the discussion
ACP Journal Club?
• secondary journal since 1991
– provides structured summaries
– articles important to internists
– high methodologic quality
• therapy, randomization
• diagnosis, blind comparison to a gold standard
• may provide more information than original
article
– concealment of randomization
– who is blind
• patients, caregivers, outcome data collectors,
adjudictors of outcome, data analysts
• loss to follow-up
PROGRESS, Lancet 2001
PROGRESS, Lancet 2001
Methods
Results
Interpretation
perindopril
NO
R
placebo
Ask MD if
patient will
give two drugs
perindopril + indapamide
YES
R
placebo
PROGRESS Trial – ACP JC
PROGRESS
PROGRESS
ACP Journal Club
2
Read the abstract reported
in evidence-based secondary
publications (ACPJC)
Use of placebo when active comparator optimal
Do you manage patients with type II diabetes?
Is there any agent you would routinely suggest for
patients with diabetic nephropathy?
Angiotensin receptor blockers
for diabetic nephropathy
vs.
ACE inhibitors
Parving H-H et al. N Engl J Med 2001;345(12):870-878
Brenner BM N Engl J Med 2001;345(12):861-869
Lewis EJ et al. N Engl J Med 2001;345(12):851-60.
Hostetter TH. N Engl J Med 2001;345(12):910-912
Incomparable comparators
• 8 RCTS of 2nd generation neuroleptics
vs. 20 mg/d haloperidol
– fewer extrapyramidal symptoms
– standard dose haloperidol < 12 mg
• RCT paroxetine qd vs. amitriptyline bid
– less daytime somnolence
– standard amitriptyline qhs
Safer J Nerv Ment Dis 2002;190(9):583-92.
Geddes J et al BMJ 2000;321(7273):1371-6.
Christiansen PE, et al. Acta Psychiatr Scand 1996;93(3):158-63
• Neurology 2006;66;1294-1295
• Jacqueline A. French and Richard J.
Kryscio
• trials
• Active control trials for epilepsy :
Avoiding bias in head-to-head
Use of appropriate active comparator
instead of placebo
For profit agency
136 trials of new
treatments for
multiple myeloma
40%
Nonprofit agency
79%
Mann H et al. James Lind Library, 2003
• Goetschze in methods pharmaceutical
other than reporting bias (though has
reporting bias implications as well)
3
Beware faulty
comparators
Irbesartan vs amlodipine in
diabetic nephropathy
• in comparison to amlodipine, irbesartan
reduced the combined endpoint of all cause
mortality, progression to end stage renal
disease, and doubling of serum creatinine
(RRR 20%, 95% CI 7.5% to 32%)
• did irbesartan reduce all-cause mortality?
Risk reduction
with irbesartan (vs. amlodipine)
Doubling of creatinine concentration
RRR 33% (16-47%)
End-stage renal disease
RRR 23% (-0.5-41%)
All-cause mortality
RRR -3% (-35-22%)
Composite endpoint
-40
RRR 20% (7.5-32%)
-24
-8
8
24
40
56
RRR (95% CI)
RCT 1715 DM 2 nephropathy, HTN irbesartan vs amlodipine NEJM 2001; 345: 851
What has gone wrong?
• widely varying importance
• biggest effect on least important
– most important no effect
• criteria for use of composite
– similar patient-importance
– similar effect
UKPDS
Diabetes-related endpoints
After 10 years of treatment
Sudden death
with diet + SU or insulin: 35.3%
Death from hypoglycemia
with diet: 38.5%
Death from hyperglycemia
RD: 3.2%
Fatal MI
NNT for 10 years: 31
Nonfatal MI
Angina
Heart failure
SIMILAR PATIENT IMPORTANCE?
Stroke
SIMILAR TREATMENT EFFECT?
Renal failure
Amputation
Vitreous hemorrhage
Retinal photocoagulation
Blindness in one eye
Cataract extraction
2.7%
1 in 35 reviews note this dominance
4
Beware composite endpoints
5,269 patients with abnormal glucose tolerance test
randomized to lifestyle advice, or advice + rosiglitazone
followed for 3 to 5 years
What is the authors’ message?
• rosiglitazone to prevent diabetes:
• strong indication (for all)
• weak indication (for some)
• not indicated
Doctor, what do I gain by
taking rosiglitazone?
• Doc: less chance of diabetes
• Pt: what happens if I get diabetes
• Doc: you have to take a drug
• Pt: the same drug I’m taking to prevent diabetes?
• Doc: I could give you a drug with less problems
• Pt: I’ll take a drug every day for 3 years to lower
my risk of taking the same or a less toxic drug
from 25 to 10%???
drug
no drug
difference
30000
3650
26350
1060
2500
HR 0.38 (CI, 0.33-0.44)
Anxiety about diabetes
??
??
??
Costs inconvenience self-monitoring
??
??
??
Costs and inconvenience HbA1c,
lipoprotein testing, retinal exam, etc.
??
??
??
MI, stroke, CV death at 3 years
120
90
HR 1.39 (CI 0.81-2.37)
Heart failure, at 3 years
50
10
HR 7.03 (CI 1.6-30.9)
kidney, eyes, neuropathy
??
??
??
Peripheral edema, at 3 years
680
490
RR 1.4 (CI 1.1-1.8)
Weight gain (kg), at 3 years
+ 1.1
-1.1
+ 2.2
??
??
??
diabetes medication 3 years
new diagnosis of diabetes
Rare (fractures, macular edema)
What is your view?
• rosiglitazone to prevent diabetes:
• strong indication (for all)
• weak indication (for some)
• not indicated
• what has gone wrong here?
Other problematic surrogates
• tests of cognitive function instead of function
and behavior in Alzheimer’s
• bone density instead of fractures in
osteoporosis
• oxygenation instead of mortality in ARDS
• asymptomatic DVT instead of symptomatic
thromboembolism
• pulmonary function instead of exacerbations
and qol in respiratory disease
• lipids instead of CV events in atherosclerosis
Another surrogate
• patients with CV disease
• higher HDL associated with lower CV risk
– strongly agree
– agree
– disagree
• raising HDL should decrease CV risk
– strongly agree
– agree
– disagree
5
Beware substitute endpoints
Five vs Four Courses of
Therapy for Acute Myeloid
Leukemia
Wheatley K, Clayton D. Controlled Clinical Trials 2003;24:66-70
Five vs Four Courses of
Therapy for Acute Myeloid
Leukemia
Wheatley K, Clayton D. Controlled Clinical Trials 2003;24:66-70
Five vs Four Courses of
Therapy for Acute Myeloid
Leukemia
Wheatley K, Clayton D. Controlled Clinical Trials 2003;24:66-70
Five vs Four Courses of
Therapy for Acute Myeloid
Leukemia
Wheatley K, Clayton D. Controlled Clinical Trials 2003;24:66-70
Tifacogin [Recombinant Tissue
Factor Pathway Inhibitor (TFPI) in
Sepsis]
• Background:
– multicentre RCT (245 hospitals) comparing
TFPI to control in severe sepsis
– major endpoint: 28 day mortality
– 2 interim analyses
• Results:
– 1754 patients randomized over 18 months
– baseline characteristics similar
Abraham et al. JAMA 2003:290;238-247
Results: 3-Month Moving
Average Mortality
Interim analysis at 722 patients:
Mortality 38.9% vs 29.1% (p<.006)
Abraham et al. JAMA 2003:290;238-247
Results: 3-Month Moving
Average Mortality
Abraham et al. JAMA 2003:290;238-247
Stopping boundary
True beneficial effect
No effect
stop
Stopping boundary
True beneficial effect
No effect
Look after every patient or event
stop
stop
stop
Stopping boundary
True beneficial effect
No effect
Interim analyses every q patients or events
Poldermans, NEJM, 1999
• elective vascular surgery
– positive dobutamine stress echo
• compared bisoprolol to placebo
– unblinded
• primary endpoint death or nonfatal MI
Poldermans NEJM 1999
• planned sample size 266
• prior planned single look at 100 pts
– stop if exceeded O’Brien-Fleming boundary
• p < 0.001
• stopped after 112 patients
• primary endpoint
– 2 of 59 (3.4%) in bisoprolol group
– 18 of 53 (34%) in placebo
• RR 0.09, 95% CI 0.02 to 0.37, P< 0.001
Beta blockers in non-cardiac surgery
Summary of Findings
Quality Assessment
Quality
Relative
Effect
(95% CI)
Outcome
Number of
participants
(studies)
Risk of
Bias
Consistency
Directness
Precision
Publication
bias
Myocardial
infarction
10,125
(9)
No serious
limitations
No serious
imitations
No serious
limitations
No serious
limitations
Not
detected
High
0.71
(0.57 to 0.86)
Mortality
10,205
(7)
No serious
limitations
Possiblly
inconsistent
No serious
limitations
Serious
Imprecision
Not
detected
Moderate
or low
1.23
(0.98 – 1.55)
Stroke
10,889
(5)
No serious
limitaions
No serious
limitations
No serious
limitations
Possible
imprecision
Not
detected
Moderate
or High
2.21
(1.37 – 3.55)
Absolute risk
difference
15/1,000 fewer
(7 fewer to 21
fewer)
5/1,00 more
(1 fewer
to 13 more)
5/1,000 more
(2 more to
1.3 more0
6
Beware trials stopped early
Users Guides to Spin
1. read methods and results; bypass
the discussion section
2. read the abstract reported in
evidence-based secondary
publications
3. beware faulty comparators
4. beware composite endpoints
5. beware of substitute endpoints
6. beware trials stopped early
Wall St. Journal, November, 2006
Wall St. Journal, November, 2006
New York Times, May 9, 2007
Available at:
www.mhprofessional.com/jama