Transcript T 2

The scientific method
F. Guesdon
MED610 DDP
March 2013
Is square A darker than B?
“Checker shadow illusion”, first described by Adelson, 1995
The “scientific method” model
o Describes “best practice” method for scientific
discovery
o Developed from observation of succesful scientists
History
10th Century
o Ibn al-Haytham (Alhacen): Pioneer in experimental
optics and psychology, use of scientific method.
13th and 14th Centuries
o Bacon (Collection of facts, induction)
o Occam (Parsimony)
17th century
o Descartes: Deductive method, predictions
o Galileo: Experimental approach
History - Modern
20th century
o Statistical criteria
o Popper: Falsification
o Kuhn: non-rational aspects
Our learning aims
o Reflect on what makes research scientifically
sound
o Understand what the “Scientific Method” is
o Ask if “Scientific Method” really accounts for all
scientists need to do
Session plan
Problem-solving strategies
Case study 1: The law of falling bodies
Case study 2: The bacterial origin of peptic ulcer
How useful is the “scientific method” model?
Thinking about probabilities
o 1% of women have breast cancer (p = 0.01)
o If a woman has breast cancer, the probability of a
mammogram detecting it is p = 0.8
o If a woman has no breast cancer, the probability
of the test being positive is p = 0.1
Estimate the probability that a woman whose
mammogram came up positive actually has cancer
From Gigerenzer, in The evolution of the mind, Dellarosa Cummins and Allen, Eds, 1988, chapter 1
Probabilistically correct answer
o For every 1,000 women tested, 10 will have breast
cancer and 990 won’t
o Of the 10 women with breast cancer, 8 will be
diagnosed correctly by the mammogram
o Of the 990 other women, 99 will have falsely positive
mammograms
o For every 1,000 women tested, 99 + 8 = 107
mammograms will be positive
o The probability that a positive mammogram indicates a
true breast cancer is 8 / 107 = 7.47 %
There are two different
ways of thinking
One way thoughts come to
mind:
Is happy
This way of thinking uses
perception and intution
Is angry
This way of thinking uses perception
and intution
Another way thoughts come
to mind
o Probability of having breast cancer is pc = 0.01 so of
1,000 women, we can expect 1,000 x 0.01 = 10 cases
o And therefore 1,000-10 = 990 healthy women
o The test has a rate of detection of pd = 0.8, so it should
pick up 0.8 x 10 = 8 cases from the sample of 1,000
o The test has a false positive rate of p+ = 0.1, so 990 x 0.1
= 99 healthy women will also have a positive result
o So, there will be in total 99 + 8 = 107 positive results.
o If I am one of those, the probablility I have cancer is p=
8/107 ≈ 0.075
Type 1 thinking = “natural”
o Automatic / intutive / effortless
o Uses perception, common sense, training
(skills)
o Jumps to conclusion
o “Heuristic”
Limitations of type 1 thinking
o Perception (sensory) biases
o We tend to misjudge numerical information
o We tend to confuse the most typical with the
most probable
o We seek solutions that conform with how we
perceive a problem rather than how it is
objectively (framing, economy of thought)
Type 2 thinking: organised
o Based on conscious processing
o Rational, analytical
o “Unatural”, difficult
o Technically accurate
o Slow or unconclusive when dealing with
complex problems (social / economy etc.)
Common sense relies mostly on
type 1 thinking
o Provides fast, practical answers
o Good for practical problems (hunting, farming,
buying and selling, stay safe etc.)
o Easy to commmunicate or convince – “Feels right”
o Influenced by and produces common knowledge
Contemporary common
knowledge
o In a given situation, people from different cultures
are likley to react differently
o In a given situation, people will react differently
depending on their personality traits
o Women have better verbal skills and more empathy
than men
Selecting candidates:
Common sense approach
o Candidates for PhD position selected by interview
o Staff believe they select the best candidates
o But they can only judge the performance of
students they took in
o So how do staff know that they select correctly?
o We think common sense works because it
“seems” to work
Science developped by
mistrusting common
sense and organising
knowledge
Hypothesis
Theory
o Focused
• Broad scope
o Generates specific
predictions
• Accommodates
alternative hypotheses
o Designed to be tested
rigorously
• Designed to be
inclusive: incorporates
as many facts and
explanations as possible
in a unified framework
o Will be rejected as
soon as it fails a single
test
• Will be abandonned if
cannot generate good
hypotheses, or when a
better theory is built
Problem-solving strategies:
Common sense v. rational thinking
Case study 1: The law of falling bodies
Case study 2: The bacterial origin of peptic ulcer
How useful is the “scientific method” model?
Aristotle’s description of the
motion of falling bodies
H
L
L
H
The speed of
falling objects is
proportional to
their weights.
What happens if a light object
(L) is tied to a heavy object (H)?
1. The falling speed of the tied objects should be
intermediate between those that they would
have individually.
2. When tied, the two objects (H+L) form a single
object heavier than H, so should fall faster
than H alone.
Aristotle’s description can lead
to contradictory predictions
• Limited predictive value
• Can lead to alternative contradictory predictions
• Does not explain what it tries to describe
Identifying the problem
1 – Galileo noted the logical inconsistency in Aristotle’s
description
2 - Observed that falling objects appear to start slowly and
then accelerate
3 – Looked for supporting evidence: dents in a cushion
4 – Seeked to measure how speed increased with time and
describe the relation in a manner fully consistent with
measurements
How Galileo may have
generated his hypothesis
He uses the most simple mathematical description of
accelerated motion:
The speed (V) increases in direct proportion to time
(T) since the object was dropped:
V=T
How to test this?
T=1
D=α
The equation
V=T
leads to a prediction about distance
fallen with time:
T=2
D=4xα
The distance (D) increases in
proportion to time squared (T2):
D =  T2
T=3
D=9xα
The rolling ball experiments (1603)
o Problem: Free fall is too fast
o Solution: Study balls rolling down an inclined
beam
D1
D2
D3
Gallileo assumed that this motion
followed the same law as free fall
Simulated Galileo data
D
1
4
9
16
Prediction
T
T2
1
1
2
4
3
9
4
16
Data
T
0.93
2.15
3.03
3.93
T2
0.87
4.62
9.18
15.4
The data does not fit perfectly the prediction
Does that means the hypothesis is wrong?
Replicating the experiment
Experimental T2
D
Predicted
T2
♯1
♯2
…
♯99
1
4
9
16
1
4
9
16
0.87
4.62
9.18
15.4
1.15
4.46
8.75
15.2
…
…
…
…
1.21
3.76
8.84
14.9
The data is never perfectly reproducible either
Does that mean the experiment is not reliable?
o The most important step when
interpreting data is ask if the data is good
enough to mean anything.
o Many experiments do not give “yes” or
“no” answers, just “maybe” answers
Simulated Galileo data
Experimental T2
D
Predicted
T2
♯1
♯2
…
♯99
1
4
9
16
1
4
9
16
0.87
4.62
9.18
15.4
1.15
4.46
8.75
15.2
…
…
…
…
1.21
3.76
8.84
14.9
The differences are
not significant, so the
data supports the
prediction that
D =  T2
Value judgments:
Interpreting data
o A researcher must interpret their data - decide
what it means.
o Interpretation is informed by controls
(standardisation), replication and statistical
analysis
o But not fully objective, depends on assumptions
o The interpretation can be contested by other
scientists (peers)
o The most important step when
interpreting data is ask if the data is good
enough to mean anything.
o Many experiments do not give “yes” or
“no” answers, just “maybe” answers
The Scientific Method
1. Observe phenomena
2. Develop a hypothesis (inductive thinking)
3. Derive predictions from the hypothesis
(deductive thinking).
4. Test one prediction (experiment)
5. Interpret the results: are they consistent with
the prediction?
• If yes, the model passes the test; test another
prediction
• If no, the hypothesis is proven wrong (falsified);
alter or discard hypothesis
Inductive reasoning
o Imagines possible causes or mechanisms to
explain the data
o Based on recognition of patterns or trends
o Can be intuitive, subjective
o Error-prone: risks confusing correlation with
causality
o Essential to make good hypotheses
Standard model of Scientific
Method
Hypothesis
Data
Prediction
Hypotheses are at the core of
the scientific method
o A hypothesis is an attempt at explaining
o Testing a hypothesis is testing our understanding
o understanding means being able to make
predictions!
o This distinguishes investigative science from
descriptive science (mapping, cataloguing,
sequencing)
Testing hypotheses: Falsification
o Experiments must be designed so as to reveal if the
hypothesis is wrong
o Experiments set up to confirm hypothesis are not
informative
Karl Popper (1902-1994)
Testing to faslsify…
How would you test the
following hypothesis?
“All cards that have a vowel on
one side have an even number
on the other side”
U
4
Testing the hypothesis
o You have a sample of 4 cards:
A
J
2
7
Which card(s) do you need to turn over to test
the hypothesis?
Write your choice(s) on a piece of paper
Prediction:
“All cards that have a vowel on one side
have an even number on the other side”
Available cards:
A
J
2
7
Would turning card A test the
hypothesis?
o What might we find if we turn over card A?
1. An even number
2. An odd number
o If it is and odd number, we will have learned
that the hypothesis is false
Apply this reasoning to all
available cards
A
J
2
7
“All cards that have a vowel on
one side have an even number
on the other side”
Correct choices:
Cards A and 7
o If you find an odd number on the other side of A, you
will know that the hypothesis is wrong
o If you find a vowel on the other side of 7, you will know
that the hypothesis is wrong
Were our initial choices wrong?
Why?
o Card 2 is not informative: whether there is a vowel or
consonant on the other side will tell you nothing about
the hypothesis - but many people choose it
o Most people choose card A but very few people choose
card 7 - this shows a natural bias towards seeking
confirmation, but ignores half the evidence available
Problem-solving strategies:
Common sense v. rational thinking
Case study 1: The law of falling bodies
2: The bacterial origin of peptic ulcer
How useful is the “scientific method” model?
Pre-1984 view of peptic ulcer
o Erosions of the lining of the stomach or duodenum
o Believed to be caused by overproduction of
stomach acids
o Thought to result from lifestyle factors (stress or
excess absorption of spicy food)
o Treatments were: avoiding lifestyle factors,
neutralising stomach acidity or preventing acid
secretion by severing nerves
o Alleviated symptoms, did not cure the disease
Observations of bacteria
o In the 1970s, fiber optic endoscopes made
possible stomach biopsies from live patients
o Until then, most samples of peptic ulcer tissues
had been obtained post mortem
o In the 1970s, researchers began to report
association of gram-bacillus in 80% of patients
with gastric ulcers.
Problems with the new
observations
o Medical texbooksk asserted that bacteria cannot
live in the stomach
o The bacteria could be grown in vitro after
isolation from the biopsies, preventing detailed
characterisation
o They were assumed to be Pseudomonas,
common contaminants of endoscopes
o All this suggested the bacteria seen in ulcer
samples were not genuine hosts of the stomach
Flaws in accepted knowledge
o Warren noticed that the presence of bacteria in
his biopsies strongly correlated gastritis –
suggesting an immune reaction against the
bacteria
o Also, the large numbers of bacteria, their
homogeneous distribution and their localisation
at the top of the cell layer were inconsistent with
accidental contamination
How could Warren’s
hypothesis be tested?
How would you test / prove the role of
bacteria in causing peptic ulcers?
Koch’s postulates
1. The microbe must be found in the bodies of the
patients or diseased animals
2. The microbe must be isolated from the
patients/ animals and grown outside the body
3. The innoculation of the microbe grown in pure
culture should produce the disease in an
experimental host
4. The same microbe shoud be re-isolated from
the experimental subject after the disease
develops
Read following sections of
hand-out:
o The pilot study (p.2)
o Isolating the bacteria (p. 3)
o The data (pp. 3-4 – Ignore Fig. 3)
o Presenting their results (p.5)
Think about questions 4, 1
and 5 (pp.6-7)
Pilot study (pp. 3-4): Design
Marshall and Warren recruited 100 patients or
healthy volunteers undergoing endoscopy.
Each participant had to complete a detailed
survey on:
o Their symptoms
o Their lifestyle histories:




Exposure to animals
Travels
Dental hygiene
Diet (Kentucky Fried Chicken?)
Why these questions?
Aim of the pilot study (1982)
o Are there bacteria in normal stomachs?
o Does the presence of bacteria correlate with
type and severity of pathology?
o Can the bacteria be cultured?
Koch’s postulates
1. The microbe must be found in the bodies of the
patients or diseased animals
2. The microbe must be isolated from the
patients/ animals and grown outside the body
3. The innoculation of the microbe grown in pure
culture should produce the disease in an
experimental host
4. The same microbe shoud be re-isolated from
the experimental subject after the disease
develops
Testing postulate 2 (Isolation)
o Attempts to grow the bacteria in vitro from 30
different biopsies failed repeatedly
o Until an accident happened: Due to an
emergency, technical staff once left the petri
dishes unattended for 5 days, and were then
able to see bacteria. The growth in vitro was
too slow for normal 2-days cultures.
Testing postulate 3
(Innoculation)
o In spite of repeated attempts, the bacteria
grown in vitro did not induce the disease in
model animals
o In desperation, Marshall subjected himself to
a self-experimentation and injested 30 ml of
aliquid culture of H. pylori
o Seven days later, he became ill
Postulate 4 – re-isolation
A silver stain of H. pylori on gastric mucus-secreting epithelial cells of Dr
Marshall’s stomach biopsy taken 8 days after he drank a culture of H. pylori.
Problem-solving strategies:
Common sense v. rational thinking
Case study 1: The law of falling bodies
Case study 2: The bacterial origin of
peptic ulcer
How useful is the “scientific method”
model?
Comparing the two case
studies
o IF Galileo’s study is taken as perfect example
of the scientific method, does the study of the
causes of pptic ulcer devaites from it?
Is the “Scientific Method” a
good model?
o Describes the rational element of scientific research
o Differentiates science from other disciplines
o Does not account for subjective or cultural aspects:
 How do scientists decide what to study?
 Who decides what to study?
 Qualitative (exploratory) research
 Role of chance discoveries
Pitfalls of the scientific method
Complex phenomena cannot always be
understood – predicted - through simple
hypotheses
Examples:
o Complex interactions between multiple factors
o Phenomena involving non-linear responses to
small changes
o Common chronic diseases, weather, climate
“Science is what scientists do, and there
are as many scientific methods as there
are individual scientists.“
P.W. Bridgman
(Nobel Prize in Physics 1923)
Alternatives to the scientific
method
1 - Exploratory research (mapping,
systematic fact finding) coupled with patternfinding approaches
• El niño
• Genome-wide associations with diseases,
identification of disease markers)
2 – Modelling
Any questions?