Transcript Sense and Reproducibility: the problem of translating

Sense and Reproducibility:
the problem of translating academic
discovery to drug discovery
Panelists
•
Ira Mellman (chair): VP of Research Oncology,
Genentech; Prof of Biochemistry & Biophysics UCSF;
former JCB Editor in Chief
•
C. Glenn Begley: former VP of Global Oncology, Amgen
•
Elizabeth Iorns: CEO, Science Exchange (Reproducibility
Initiative)
The problem: biotech/pharma scientists
have found it difficult to reproduce
published work from academic groups
Prinz et al (2011) Believe it or not: how much can we rely on
published data on potential drug targets? Nat Rev Drug Discovery
10:712 [Bayer]
Begley & Ellis (2012) Drug development: raise standards for
preclinical cancer research. Nature 483:531 [Amgen]
Why was this discovered?
Industry and academia have different near-term goals:
Publication of interesting work that drives a field forward
vs
Verification of published observations to justify longterm, expensive drug discovery efforts
Questions to be addressed by the panel:
• What is the nature of the reproducibility problem?
• Poor scientific/analytic quality?
• Poor quality of the validation effort?
• Generalizability vs bad science?
• How widespread is it?
• Why has it occurred?
• Problems are complex and difficult to reproduce?
• Corners are cut in the rush to publish?
• Inaccurate data representation or analysis?
• What can we do about it?
• Nothing?
• Motivate higher standards?
• Vigilantism?
• Institutionalized data verification (Elizabeth Iorns)
• Journals set higher standards for editing/data display?
The JCB experience:
• Since 2002, figures for all accepted manuscripts screened for
inappropriate image manipulation (micrographs, gels)
• 10% of papers found to contain one or more examples
• 10% of these (1% overall) rejected after determination that
manipulation fraudulently altered a key conclusion
• Frequencies have not changed in 10 years
Issues:
Desire to make data look “optimal”?
Digital manipulation is easy to do?
Cultural acceptance of digital manipulation
Sense and Reproducibility:
the problem of translating academic
discovery to drug discovery
Panelists
•
Ira Mellman (chair): VP of Research Oncology,
Genentech; Prof of Biochemistry & Biophysics UCSF;
former JCB Editor in Chief
•
C. Glenn Begley: former VP of Global Oncology, Amgen
•
Elizabeth Iorns: CEO, Science Exchange (Reproducibility
Initiative)
Discussion questions:
• Is the reproducibility issue a new problem?
• Why is so much work apparently not reproducible?
• What should we do about it as a community?
• Will initiatives like Science Exchange have an impact?
• How can we guard against spurious claims?
• What is the role of journals and reviewers?
• What steps can we as individual scientists take to maximize
the chances that our work can be reproduced?