Transcript Publication bias in controlled trials

Publication bias in clinical
trials
Kamran Abbasi
Deputy editor, BMJ
Merhaba
I want to talk about . . .

What is publication bias?

Why does it matter?

What is the evidence for it?

What can be done about it?

How has the BMJ responded?
There are many types of bias





Selection bias: biased allocation to comparison
groups
Performance bias: unequal provision of care
except treatment being evaluated
Detection bias: biased assessment of outcome
Attrition bias: biased occurrence and handling
of deviations from protocol and loss to follow up
. . . and on and on
(From Egger et al BMJ 2001;323:42-46 (7 July)
What is publication bias (1)?

A definition:
“Publication bias refers to the greater
likelihood that studies with positive results
will be published”
JAMA 2002;287:2825-2828
What is publication bias (2)?

An alternative definition:
Publication bias is the selective or multiple
publication or suppression of trial results
so that the scientific record is distorted
Why does it matter?
Distorts the scientific record
 Hides the “truth”
 Influences doctors’ decision making
 Misleads policy makers
 Causes harm to patients
 Costly for the health service
 A form of scientific and research
misconduct

Who is to blame?

Wicked researchers?

Very wicked sponsors?

Editors: the wickedest of all?

(and let’s not forget reviewers)
What is the evidence for it (1)?
Stern and Simes BMJ 1997;315:640-645
 Question: To what extent is publication
influenced by study outcome?

Studies submitted to an Australian ethics
committee over 10 years
 Examined protocols
 Questionnaire to authors (70% response)

Stern and Simes: results
Positive>negative
Time to
publication
All studies
Clinical trials
(n=520)
(n=130)
2.32
3.13
(1.47 to 3.66)
(1.76 to 5.58)
4.8 vs 8.0 yrs
4.7 vs 8.0 yrs
Stern and Simes: conclusions
Positive trials are more likely to be
submitted for publication
 Positive trials are more likely to be
published
 Positive trials are more likely to be
published quickly

Implications for systematic reviews
 Important to register all trials

What is the evidence for it (2)?






Lexchin and Bero BMJ 2003;326:1167-70
Question: Does drug industry sponsorship
influence research quality and outcome?
Meta-meta-analysis
Industry research less likely to be published
(more likely in symposium proceedings)
No difference in methodological quality
More likely to have a positive finding (OR 4.05
95% CI 2.98 to 5.51)
Lexchin and Bero

A wide range of diseases
eg osteoarthiritis of the knee, multiple myeloma,
psychiatric problems, Alzheimer’s disease, venous
thromboembolism

A wide range of drugs
eg tacrine, clozapine, 3rd generation OCP,
erythropoietin, antidepressants, topical
glucocorticoids, treatment for HIV
Lexchin and Bero: conclusions

Published research from drug companies is more
likely to be favourable to the product

Do companies selectively fund trials?
Unlikely
Is it of poorer quality?
No
Are inappropriate comparators chosen?
Sometimes/often/a lot
Is it publication bias?
Yes



What is the evidence for it (3)?
Melander et al BMJ 2003;326:1171-3
 Question: Is there selective reporting of
sponsored studies by drug companies?

Trials submitted to the Swedish drug
regulatory authority (5 SSRIs, 42 trials)
 Multiple publication
 Selective publication
 Selective reporting

Melander et al: conclusion

“Any attempt to recommend a specific
selective serotonin reuptake inhibitor from
the publicly available data ONLY is likely
to be based on biased evidence.”
What is the evidence for it (4)?






Olson et al JAMA 2002;287:2825-2828
Question: Is there publication bias in editorial
decision making?
3 years, 745 manuscripts
Positive vs negative OR 1.30 (0.87 to 1.86)
Small effect of editorial decision making, much
less than researchers not submitting negative
studies
Will this be true for journals less grand than
JAMA?
What can be done about it (1)?







Better conduct and reporting of RCTs (CONSORT)
Better conduct and reporting of systematic
reviews (QUORUM)
“Publication” of unpublished trials
Enlightened sponsors (a code of good practice
Wager et al 2003 http://www.gpp-guidelines.org)
Better editorial policies
Vigilant editors and reviewers
Responsible authors
What can be done about it (2)?
Publication of original protocols and
deviations from protocol
 Declaration of competing (financial)
interests by authors, reviewers, and
editors
 Declaration of sponsorship/funding
 Registering all clinical trials

How has the BMJ responded?






A change in editorial thinking: Is it the question
that matters?
It is
Amnesty on unreported clinical trials
More transparency (CONSORT, QUORUM,
sponsorship, funding, competing interests)
Theme issue on doctors and the drug industry
?Protocols
?Registering clinical trials
Conclusions
Publication bias is an important problem
that impacts on patient care
 There is much evidence to support its
existence
 There are many players
 There are many ways to reduce its effect,
examples of good practice
 Ultimately there is a big responsibility on
sponsors of trials, authors, and editors
