Transcript C. Out of 10 women with a positive mammogram, about 1 has breast

Biology seminar
“One of the most striking characteristics of modern
biology is the pervasiveness of statistical thinking.
However, discussions with colleagues suggest
that some teachers resist the idea that statistical
thinking is a necessary part of learning science.
As a result, many students, both majors and
nonmajors, remain ignorant of this fundamental
aspect of biological thought. Deprived of this
knowledge, they continue to view the living world
from a simplistic, deterministic perspective. They
also fail to master crucial intellectual skills that
should be used in everyday life.”
Kugler, Charles, Hagen, Joel, and Singer, Fred, “Teaching Statistical Thinking,” Journal
of College Science Teaching, Vol. 32 No. 7, p. 434-439, May 2003
“We argue that basic statistical concepts are
needed by educated people, but statistical
calculations should be kept simple to maintain
focus on the reasoning. Most importantly,
statistical thinking should be explicitly taught in
all parts of biology courses. Only by routinely
using statistical reasoning in lecture, laboratory,
and homework will students understand the
importance of biological variation and the basis
of many scientific conclusions.”
• Study involved 277 medical residents in 11 residency
programs
• instrument developed to reflect the statistical methods
and results most commonly represented in contemporary
research studies
• Authors reviewed all 239 original articles published from
January to March of 2005 in each issue of 6 general medical
journals American Journal of Medicine, Annals of Internal
Medicine, BMJ, JAMA, Lancet, and New England Journal of
Medicine) and summarized the frequency of statistical
methods described
• Questions developed based on this research
Biostatistics test of medical residents
To determine if fasting is associated with dengue fever,
data from 40 patients with dengue fever were
collected. These patients were matched for age, sex,
and race to 40 patients without dengue fever. The
hospital charts of these patients were then reviewed to
determine whether they also fasted prior to their
illness. This study type is known as:
A. cross-sectional study
B. concurrent cohort study
C. case-control study
D. retrospective cohort study
E. randomized clinical trial
Percentage with correct answer
39.4 (33.6-45.1)
Biostatistics test of medical residents
Any systematic error in the design, conduct, or
analysis of a study that results in a mistaken
estimate of an exposure’s effect on the risk of
disease is called:
A.Confounding
B.Bias
C.Interaction
D.Stratification
Percentage with correct answer
• 46.6 (40.7-52.4)
Biostatistics test of medical residents
Researchers designed a study looking at cardiovascular
deaths comparing a new drug to placebo. They
determined they would need 200 patients in each
group to detect a 15% difference in cardiovascular end
points given 90% power and a significance level of .01.
Which of the following changes would require the
members to increase their sample size?
A. Aim to detect a difference of 20%
B. Specify a power of 80%
C. Use a significance level of .05.
D. Aim to detect a difference of 10%.
Percentage with correct answer
• 30.3 (24.9-35.7)
Biostatistics test of medical residents
In a placebo-controlled study of the use of aspirin and
dipyridamole to prevent arterial restenosis after coronary
angioplasty, 38% of patients receiving the treatment had
restenosis and 39% of patients receiving the placebo had
restenosis. In reporting this finding, the authors stated that
P>.05. This means
A. The chances are greater than 1 in 20 that a difference
would be found again if the study were repeated.
B. The probability is less than 1 in 20 that a difference this
large could occur by chance alone.
C. The probability is greater than 1 in 20 that a difference this
large could occur by chance alone
D. The chance is 95% that the study is correct.
Percentage with correct answer
• 58.8 (53.0-64.6)
• Residents scored well on three questions
• 87% correctly understood the purpose of
double-blinded studies
• 82% correctly understood relative risk
• Residents with prior training in biostatistics,
but still averaged below 50% correct
http://www3.interscience.wiley.com/journal/121432447/abstract
“In a 2007 campaign advertisement, former New York
City mayor Rudy Giuliani said, ‘‘I had prostate cancer, 5,
6 years ago. My chance of surviving prostate cancer—
and thank God, I was cured of it—in the United States?
Eighty-two percent. My chance of surviving prostate
cancer in England? Only 44 percent under socialized
medicine’’ (Dobbs, 2007). For Giuliani, these health
statistics meant that he was lucky to be living in New
York and not in York, since his chances of surviving
prostate cancer appeared to be twice as high. This was
big news. As we will explain, it was also a big mistake.”
Survival rate:
Imagine a group of people all diagnosed with
cancer at the same time.
5-yr survival rate =
Number of patients diagnosed with cancer still
alive 5 years after diagnosis /
number of people diagnosed with cancer
Giuliani’s data
• British study found that the diagnosis rate was
49 men per 100,000, and of these 28 died
within 5 years. (Thus about 43% were still
alive after 5 years.)
Mortality rate:
Imagine a specific group of people (British men,
for example):
Annual mortality rate =
number of people who die from cancer over 1
year /
number of people in the group
Gigerenzer, et. al. p. 57
Absolute vs. relative risk
The contraceptive pill scare
“In October 1995, the U.K. Committee on Safety of
Medicines issued a warning that third-generation oral
contraceptive pills increased the risk of potentially lifethreatening blood clots in the legs or lungs twofold—
that is, by 100%. This information was passed on in
‘‘Dear Doctor’’ letters to 190,000 general practitioners,
pharmacists, and directors of public health and was
presented in an emergency announcement to the
media.”
Gigerenzer, et.al., p. 54
The data
• Second generation pill: 1 in 7000 instances of
thrombosis
• Third generation pill: 2 in 7000 instances of thrombosis
• Relative risk increased 100%
• Absolute risk of 3G pill: .00029
• Impact: for five years before pill scare, number of
abortions had steadily declined about 5000 per year.
Year after pill scare: abortions increased by 13000.
• Births by girls under 16 increased 800 that same year
Example of absolute risk reporting as
required by the FDA
http://www1.astrazeneca-us.com/pi/Seroquel.pdf
Mammography
Test conducted by one of the authors on 160 gynecologists at a
continuing education seminar:
Assume you conduct breast cancer screening using
mammography in a certain region. You know the following
information about the women in this region:
• The probability that a woman has breast cancer is 1%
(prevalence)
• If a woman has breast cancer, the probability that she tests
positive is 90% (sensitivity)
• If a woman does not have breast cancer, the probability that
she nevertheless tests positive is 9% (false-positive rate =
1-specificity)
A woman tests positive. She wants to know from you whether
that means that she has breast cancer for sure, or what the
chances are. What is the best answer?
A. The probability that she has breast cancer is about 81%.
B. Out of 10 women with a positive mammogram, about 9
have breast cancer.
C. Out of 10 women with a positive mammogram, about 1 has
breast cancer.
D. The probability that she has breast cancer is about 1%.
Not sure? Need help from the
audience?
Here are the responses from the doctors.
A. The probability that she has breast cancer is
about 81%. (47% of the doctors selected A.)
B. Out of 10 women with a positive mammogram,
about 9 have breast cancer. (13% selected B.)
C. Out of 10 women with a positive mammogram,
about 1 has breast cancer. (21% selected C.)
D. The probability that she has breast cancer is
about 1%. (19% selected D.)
Before you choose, suppose the data
are presented like this:
Assume you conduct breast cancer screening using
mammography in a certain region. You know the
following information about the women in this
region:
• Ten out of every 1,000 women have breast cancer
• Of these 10 women with breast cancer, 9 test
positive
• Of the 990 women without cancer, about 89
nevertheless test positive.
• What fraction of women who test positive
actually have breast cancer?
• Out of 1000 women, 9+89 = 98 test positive.
• Only 9 of these women actually have cancer.
• The fraction is 9/98 = .092
Answer C is best. Answers A, B, and D
are off by approximately an order of
magnitude
A. The probability that she has breast cancer is
about 81%.
B. Out of 10 women with a positive
mammogram, about 9 have breast cancer.
C. Out of 10 women with a positive
mammogram, about 1 has breast cancer.
D. The probability that she has breast cancer is
about 1%.
Gigenrenzer, et.al., p. 56
After training using natural
frequencies instead of conditional
probability, 87% of the doctors were
able to correctly answer the question.
• Gigerenzer, et.al., don’t just point out the
problem and measure it, but also provide
ideas for solutions as well
Basic points of statistical literacy in
health
• Learning to Live With Uncertainty
– Understand that there is no certainty and no zerorisk, but only risks that are more or less
acceptable.
• Questions to Ask About All Risks
– Risk of what?
– Time frame?
– How big?
– Does it apply to me?
Basic points of statistical literacy in
health
• Screening Tests
– Understand that screening tests may have benefits
and harms.
– Understand that screening tests can make two errors:
false positives and false negatives.
– Understand how to translate specificities, sensitivities,
and other conditional probabilities into natural
frequencies.
– Understand that the goal of screening is not simply
the early detection of disease; it is mortality reduction
or improvement of quality of life.
Basic points of statistical literacy in
health
Treatment
• Understand that treatments typically have
benefits and harms.
• Understand the size of the benefit and harm.
– What are the absolute risks with and without
treatment
Basic points of statistical literacy in
health
Questions About the Science Behind the
Numbers
• Quality of evidence?
• What conflicts of interest exist?
Guidelines for Assessment and Instruction in
Statistical Education (GAISE)
• Endorsed by the American Statistical Association
• Guidelines for K-12 and separate guidelines for
college
• https://www.amstat.org/education/gaise/index.cfm
Primary recommendations
• Emphasize statistical literacy and develop
statistical thinking;
• Use real data;
• Stress conceptual understanding rather than
mere knowledge of procedures;
• Foster active learning in the classroom;
• Use technology for developing conceptual
understanding and analyzing data;
• Use assessments to improve and evaluate
student learning
Students should believe and understand why:
• Data beat anecdotes.
• Variability is natural and is also predictable and quantifiable.
• Random sampling allows results of surveys and experiments to be
extended to the population from which the sample was taken.
• Random assignment in comparative experiments allows cause and
effect conclusions to be drawn.
• Association is not causation.
• Statistical significance does not necessarily imply practical
importance, especially for studies with large sample sizes.
• Finding no statistically significant difference or relationship does not
necessarily mean there is no difference or no relationship in the
population, especially for studies with small sample sizes.
Students should recognize:
• Common sources of bias in surveys and experiments.
• How to determine the population to which the results
of statistical inference can be extended, if any, based
on how the data were collected.
• How to determine when a cause and effect inference
can be drawn from an association, based on how the
data were collected (e.g., the design of the study)
• That words such as “normal”, “random” and
“correlation” have specific meanings in statistics that
may differ from common usage.
Students should understand the parts of the process
through which statistics works to answer questions,
namely:
• How to obtain or generate data.
• How to graph the data as a first step in analyzing data,
and how to know when that’s enough to answer the
question of interest.
• How to interpret numerical summaries and graphical
displays of data - both to answer questions and to
check conditions (in order to use statistical procedures
correctly).
• How to make appropriate use of statistical inference.
• How to communicate the results of a statistical
analysis.
WHAT MAKES REAL DATA REAL?
MEASURING QUALITY AND STRUCTURE OF DATA SETS
1. Length (# observations). Real data sets should be large enough to make the need for
computers, rather than calculators, obvious. They should be large enough that
graphical and numerical summary techniques reveal structure that is otherwise
not apparent.
2. Width (# variables). Real data sets should have enough variables that students have
room for exploration, for testing alternative hypotheses, and for performing
residual diagnostics.
3. Form. Real data include missing values, un-coded values, and sometimes misspelled
values; real data can include both numerical and character-valued variables.
4. Structure. Real data typically have complex structure. For example, linear relations
are rare and even then high correlations are unusual. Distributions can be highly
skewed, or multi-modal. Students should also see that some forms of data, for
example longitudinal, can be stored in different “shapes,” depending on whether
rows represent an observation made on a subject at a particular time, or contain
all observations for a particular subject
http://www.ime.usp.br/~abe/ICOTS7/Proceedings/PDFs/InvitedPapers/7A2_GOUL.pdf
Improving statistical skills of research scientists
in pharmaceutical discovery research
1) Time spent thinking on the conceptualization and design of an experiment
is time wisely spent;
2) The design of an experiment reflects the contributions from different
sources of variability;
3) The design of an experiment balances between its internal validity (proper
control of noise) and external validity (the experiment’s generalizability);
4) Good experimental practice provides the clue to bias minimization;
5) Good experimental design is the clue to the control of variability;
6) Experimental design integrates various disciplines;
7) A priori consideration of statistical power is an indispensable pillar of an
effective experiment.
Vandenbroeck, Philippe, Wouters, Luc, Molenberghs, Geert, Van Gestel, Jef and Bijnens,
Luc(2006)'Teaching Statistical Thinking to Life Scientists a Case-Based Approach',Journal of
Biopharmaceutical Statistics,16:1,61 — 75
To link to this Article: DOI: 10.1080/10543400500406520
URL: http://dx.doi.org/10.1080/10543400500406520
“Habits of mind” for students
• consideration of how to best obtain meaningful and relevant
data to answer the question at hand
• constant reflection on the variables involved and curiosity for
other ways of examining and thinking about the data and
problem at hand
• seeing the complete process with constant revision of each
component
• omnipresent skepticism about the data obtained
• constant relation of the data to the context of the problem
and interpretation of the conclusions in non-statistical terms
• thinking beyond the textbook
Chance, Beth L., Components of Statistical Thinking and Implications for
Instruction and Assessment, Journal of Statistics Education Volume 10,
Number 3 (2002)
www.amstat.org/publications/jse/v10n3/chance.html
Handy summaries of types of studies
• http://www.lib.uconn.edu/research/bysubject/nursi
ngtutorial/studies.htm
• http://www.hsrmethods.org/glossary.aspx
Hierarchy of Evidence for Intervention Studies
Systematic review
of randomized trials
Single randomized trial
Systematic review of observational
studies addressing patient-important outcomes
Single observational study addressing patientimportant outcomes
Physiologic studies
Unsystematic clinical observations
Adapted from: Guyatt et al. for the Evidence-Based Medicine Working Group. JAMA.
2000;284:1290-1296.
From: Freedman, Pisani, Purves and Adhikari, Statistics, Norton, 1991
Thank you.