Transcript How the BMJ triages submitted manuscripts

How the BMJ triages
submitted manuscripts
Richard Smith
Editor, BMJ
www.bmj.com/talks
What I want to talk about
• Why should you triage studies?
• The aims of BMJ triage of
submitted manuscripts
• The “killer” triage question for
you
• The BMJ’s approach to triage
• The key questions for triaging
papers
Why should you triage studies
that you might come across
• You should pay attention to
studies that are valid and have a
relevant message for your
practice--that will make a
difference for your patients
• This is less than 1% of original
studies published in medical
journals
The aims of BMJ triage of
submitted manuscripts
• To identify “good” papers and
send them to reviewers as fast as
possible
• To identify papers we don’t want
to publish and give succinct but
good reasons for not publishing
them.
• To identify awful papers and
reject them immediately.
The aims of BMJ triage of
submitted manuscripts
• To identify those papers that
might or might not be sent for
review--and ask others
• To identify papers where we,
the editors, can make a
decision and which we might
want to publish
The first question for you
• If this study is “true” would it be
a POEM (Patient Oriented
Evidence that Matters)?
• If it wouldn’t, move on
• You will probably be able to
discard 70% of studies with this
question alone
BMJ approach I
•Read covering letter.
•Pay small attention.
This is a sell.
BMJ approach II
•Look for signs that this is a
totally unsuitable paper
•Written by hand
•Typed on an ancient typewriter.
•Full of spelling mistakes.
•Biblical quotations.
•A cure for schizophrenia or cancer
•The answer to SIDS.
•Incomprehensible first two
paragraphs.
BMJ approach III
•Read title page
•Is this an original study or some
other kind of contribution?
•Are these authors where the study
is likely to be sound?
•Don’t get too carried away by the
authors. “Unknown” authors
regularly produce great work.
“Known” authors sometimes
produce dreadful papers.
BMJ approach IV
•Do not read ‘TWIBS’ or
‘What this paper adds’
•These are what the
authors would like the
paper to say rather than
what it does say
• Remember that if we do publish this
study we need to sort this out
BMJ approach V
•Read structured abstract
•Have you got a clear fix on what
the paper is about and how it is
structured?
•If you haven’t, it’s looking bad
•Try to make sense of what the
paper is about from the
introduction
•If you can’t, reject it
BMJ approach VI
•Continuing with the structured
abstract
• Have the authors asked a question that
we want to know the answer to?
• We may not for the following reasons:
• Too specialist.
• Too inconsequential
• Too far removed from patient care or public
policy
• Too well known – but remember that lots of
things that are well known have no evidence to
support them.
BMJ approach VII
•Don’t reject papers that
ask an interesting
question but get a
“negative” answer
•The question is more
important than the result
Triage questions:
treatment papers
• Is it a randomised controlled trial or a
systematic review (see later)?
• If it is not an RCT, is it reasonable not to
have done one?
• Look for an answer to the question in
the paper. If you can’t find one, reject.
• If it is an RCT, was it really randomised?
• If it wasn’t, reject unless you can find a
good reason for not randomising
Triage questions:
diagnosis paper:
• Is the test compared prospectively and
blind with a gold standard?
• Does the test population include
patients with the condition, with related
conditions that could be confused with
the main condition, and people without
the condition?
• Does the paper include information on
sensitivity, specificity, etc?
• If the answer to any of these questions
is no, we probably don’t want it.
Triage questions:
prognosis studies
• Is there an cohort of patients
followed followed prospectively
from when they were first
identified with the disease?
• Are 80% of patients followed
up?
• If the answer to these questions
is no, we probably don’t want it.
Triage questions:
systematic reviews
•
•
•
•
Was a clear question asked?
Was a search described?
Were quality criteria set?
Were studies that didn’t meet them
discarded or, if included, done so
with a justification or discussion of
the effect of doing so?
• If not to any of these questions,
reject.
Triage questions:
qualitative research
• Were qualitative methods
appropriate for the question? Is it a
“why” or “how” study rather than a
“does it work” or “how often” study?
• Is there evidence that the data were
analysed by two people
independently?
• If the answer is no to either
question, you should probably reject
Triage questions:
Questionnaire survey
• We probably don’t want. This is
people saying what they do rather
than evidence on what they do
• But is it telling us something
important that we probably can’t get
information on in any other way?
• Or might it be a peg for an
educational article.
• If the response rate is below 55% we
almost certainly don’t want it.
Triage questions:
economic evaluation
• Is the underlying methodology
valid? For example, is an
evaluation of treatment based
on a randomised trial or a
systematic review?
• If the answer is no, reject
Triage questions:
case study
• Might it make a “lesson of the
week” or a “drug point”?
• If no, reject
• Lessons of the week must be:
– not so common that everybody
should know it
– nor so rare that it wouldn’t matter
if you didn’t
– a good read
Triage questions:
drug point
• Does the report simply say that a drug
was given and something happened to
the patient without any “extra
evidence” that there was a causative
link?
• If yes, reject
• Extra evidence includes
– rechallenge
– More than one case
– Physiological or pharmacological
explanation
– Seen with other similar drugs
Triage questions:
Quality improvement report
• Does the attempt at improvement
describe an initial assessment of the
problem, the introduction of a
change, and a further assessment?
• If the answer is no to any of these,
reject
• It doesn’t matter whether the change
led to improvement
• Remember we want to know the
broad context
Triage questions: two sorts of
studies we don’t want
• Prevalence study
• Boring
• Usually not possible to generalise
beyond the particular population
• Cost of illness study
• Boring
• Value is in the exactness, which is
usually of interest to only a few
• Again hard to generalise
Conclusion
• If your study would survive this triage or
if you are uncertain we will be pleased
to receive it
• Send it too if you are uncertain
• Don’t despair if your study wouldn’t
seem to survive--there are many other
journals
• Don’t be upset if you submit your article
and it is rejected. The process is
inevitably somewhat arbitrary. We often
see papers we have rejected in the
Lancet (and, I’m sure, vice versa