Transcript Publication bias in controlled trials

Publication bias in
clinical trials
Kamran Abbasi
Deputy editor, BMJ
Merhaba
I want to talk about . . .

What is publication bias?

Why does it matter?

What is the evidence for it?

What can be done about it?

How has the BMJ responded?
There are many types of bias
Selection bias: biased allocation to
comparison groups
 Performance bias: unequal provision of
care except treatment being evaluated
 Detection bias: biased assessment of
outcome
 Attrition bias: biased occurrence and
handling of deviations from protocol and loss
to follow up
 . . . and on and on

(From Egger et al BMJ 2001;323:42-46 (7 July)
What is publication bias (1)?
A
definition:
“Publication bias refers to the
greater likelihood that studies
with positive results will be
published”
JAMA 2002;287:2825-2828
What is publication bias (2)?
 An
alternative definition:
Publication bias is the selective
or multiple publication or
suppression of trial results so
that the scientific record is
distorted
Why does it matter?
 Distorts
the scientific record
 Hides the “truth”
 Influences doctors’ decision
making
 Misleads policy makers
 Causes harm to patients
 Costly for the health service
 A form of scientific and research
misconduct
Who is to blame?
 Wicked
 Very
wicked sponsors?
 Editors:
 (and
researchers?
the wickedest of all?
let’s not forget reviewers)
What is the evidence for it (1)?
Stern and Simes BMJ 1997;315:640-645
 Question: To what extent is
publication influenced by study
outcome?

Studies submitted to an Australian
ethics committee over 10 years
 Examined protocols
 Questionnaire to authors (70%
response)

Stern and Simes: results
All studies
(n=520)
Clinical trials
(n=130)
Positive>
negative
(1.47 to 3.66) (1.76 to 5.58)
Time to
publication
4.8 vs
8.0 yrs
2.32
3.13
4.7 vs
8.0 yrs
Stern and Simes: conclusions
Positive trials are more likely to be
submitted for publication
 Positive trials are more likely to be
published
 Positive trials are more likely to be
published quickly

Implications for systematic reviews
 Important to register all trials

What is the evidence for it (2)?
Lexchin and Bero BMJ 2003;326:1167-70
 Question: Does drug industry
sponsorship influence research quality
and outcome?
 Meta-meta-analysis
 Industry research less likely to be
published (more likely in symposium
proceedings)
 No difference in methodological quality
 More likely to have a positive finding

(OR 4.05 95% CI 2.98 to 5.51)
Lexchin and Bero

A wide range of diseases
eg osteoarthiritis of the knee, multiple
myeloma, psychiatric problems,
Alzheimer’s disease, venous
thromboembolism

A wide range of drugs
eg tacrine, clozapine, 3rd generation OCP,
erythropoietin, antidepressants, topical
glucocorticoids, treatment for HIV
Lexchin and Bero: conclusions

Published research from drug
companies is more likely to be
favourable to the product
Do companies selectively fund trials?
Unlikely
 Is it of poorer quality?
No
 Are inappropriate comparators
chosen?
Sometimes/often/a lot
 Is it publication bias?
Yes

What is the evidence for it (3)?
Melander et al BMJ 2003;326:1171-3
 Question: Is there selective reporting
of sponsored studies by drug
companies?

Trials submitted to the Swedish drug
regulatory authority (5 SSRIs, 42
trials)
 Multiple publication
 Selective publication
 Selective reporting

Melander et al: conclusion
 “Any
attempt to recommend a
specific selective serotonin
reuptake inhibitor from the
publicly available data ONLY is
likely to be based on biased
evidence.”
What is the evidence for it (4)?
Olson et al JAMA 2002;287:2825-2828
 Question: Is there publication bias in
editorial decision making?

3 years, 745 manuscripts
 Positive vs negative OR 1.30 (0.87 to
1.86)
 Small effect of editorial decision
making, much less than researchers
not submitting negative studies


Will this be true for journals less
grand than JAMA?
What can be done about it (1)?
Better conduct and reporting of RCTs
(CONSORT)
 Better conduct and reporting of
systematic reviews (QUORUM)
 “Publication” of unpublished trials
 Enlightened sponsors (a code of good
practice Wager et al 2003
http://www.gpp-guidelines.org)
 Better editorial policies
 Vigilant editors and reviewers
 Responsible authors

What can be done about it (2)?
Publication of original protocols and
deviations from protocol
 Declaration of competing (financial)
interests by authors, reviewers, and
editors
 Declaration of sponsorship/funding
 Registering all clinical trials

How has the BMJ responded?
A change in editorial thinking: Is it the
question that matters?
It is
 Amnesty on unreported clinical trials
 More transparency (CONSORT,
QUORUM, sponsorship, funding,
competing interests)
 Theme issue on doctors and the drug
industry
 ?Protocols
 ?Registering clinical trials

Conclusions
Publication bias is an important
problem that impacts on patient
care
 There is much evidence to support
its existence
 There are many players
 There are many ways to reduce its
effect, examples of good practice
 Ultimately there is a big
responsibility on sponsors of trials,
authors, and editors
