Transcript Tue, Sep 9 - Wharton Statistics Department

Lecture 2 Outline: Tue, Sep 9
• Chapter 1.2: Statistical Inference and Study
Design
– Types of Inference
– Observational Studies vs. Randomized
Experiments
– Confounding Variables
– Design of Experiments
– Inference to Populations: random sampling
studies vs. non-random sampling studies
Drawing Conclusions
• An inference is a conclusion from the data about
some broader context that the data represent.
– e.g., one egg in a container is rotten -- the rest are
rotten; when we flick on a light switch, the light turns
on -- flicking on the light switch causes the light to turn
on.
• A statistical inference is an inference justified by a
probability model linking the data to a broader
context. Statistical inferences include measures of
uncertainty about the conclusions.
Two “broader contexts” in statistics
• Population inference: an inference about
population characteristics, like the
difference between two population means
• Causal inference: an inference that a subject
would have received a different numerical
outcome had the subject belonged to a
different group.
Causal Questions
• Medicine: How effective is a new drug? What is the effect
of smoking on one’s chance of developing cancer?
• Psychology: What change in an individual’s normal
solitary performance and behavior occurs when people are
present? What changes in an individual’s moral behavior
occur when the individual is commanded by authority?
• Economics: What is the effect of a change in taxes on labor
supply and investment behavior? What is the effect of a
change in the minimum wage on employment?
• Education: What is the effect of smaller class sizes on
achievement?
Types of Causal Studies
• Observational study: Study in which group
status is observed, i.e., beyond the control
of the researcher.
• Controlled experiment: Study in which
group status is controlled by the researcher.
– Randomized experiment: Study in which group
status is assigned by a chance mechanism.
Examples of Causal Studies
• Motivation and creativity study (case study 1.1.1)
• Sex discrimination study (case study 1.1.2)
• Comparison of chromosomal aberrations of
Japanese atomic bomb survivors near blast and
those far from blast.
• Comparison of death rates in Navy and out of
Navy during Spanish American war.
• Comparison of heart attack rates of menopausal
women taking estrogen and women not taking
estrogen
Causal Inference
• Main lesson: statistical inferences of causation can
be made from randomized experiments, but not
from observational studies.
• In an observational study, one cannot rule out the
possibility that confounding variables are
responsible for group differences in the observed
outcome.
• In an observational study, one cannot rule out the
possibility of reverse causality or simultaneous
causality. Which came first – the chicken or the
egg? Beta-carotene intake and morbidity.
Confounding Variables
• A confounding variable is a variable that is related
to both group membership and the outcome. Its
presence makes it hard to establish the outcome as
being a direct consequence of group membership.
• Examples:
– Sex discrimination study
– Death rates in and out of Navy study
• Although it is possible to control for known
confounding variables (via multiple regression), in
an observational study we can never be sure that
there are not unknown confounding variables that
are responsible for group differences in outcome.
Association Is Not Causation
• There is a close relationship between the salaries
of Presbyterian ministers in Massachusetts and the
price of rum in Havana. Are the ministers
benefiting from the rum trade or supporting it?
• A study showed that cigarette smokers have lower
college grades than non-smokers. Does the road
to good grades lie in giving up smoking?
Do Observational Studies Have
Value – Yes!
• Establishing causation is not always the
goal.
• Establishing causation may be done in other
ways.
– Experiments not always practical or ethical
• Analysis of observational data may lend
evidence toward causal theories and suggest
the direction of future research.
Criteria for Establishing
Causation From Obs. Studies
• The association is strong.
• The association is consistent.
• Higher doses are associated with stronger
responses.
• The alleged cause precedes the effect in time.
• The alleged cause is plausible.
• Examples:
– Smoking and lung cancer
– Radiation from atomic bomb and cancer
Design of Experiments
• Principles of statistical design of experiments to
make causal inferences about different treatments
(policies)
– Control: Make sure that all other factors besides the
treatments are kept the same in the different groups
(e.g., use placebo, double blinding).
– Randomization: Use an impersonal chance mechanism
to assign units to treatments.
– Replication: Use many units to reduce chance variation
in results.
Logic of Controlled Randomized
Experiment
• Randomization produces groups that should be similar in
all respects before the treatment is applied.
• Control in design (i.e., use of the placebo, double blinding,
judges see poems in random order) ensures that influences
other than the treatment operate equally on the groups.
• Therefore differences between the treatment and control
groups must be due either to the treatment or to the play of
chance in the random assignment of units to the groups.
• Statistical inference provides a method for describing how
confident we can be that an observed difference between
the treatment and control group did not arise due to chance.
Statistical Inference in the
Motivation-Creativity Study
• The creativity scores tended to be larger in
the “intrinsic” than in the “extrinsic” group.
Either the intrinsic questionnaire caused a
higher score or else the more creative
writers happened to be placed in the
“intrinsic” group. The probability (p-value)
associated with this latter possibility is
0.011.
Inference to Populations
• Goal: Make conclusions about aspects of a
population (e.g., mean income in U.S.) based on a
sample.
• Two types of sampling designs.
– Random sampling study: Units are selected by the
investigator from a well-defined population through a
chance mechanism with each unit having a known (>0)
chance of being selected.
– Non-random sampling study: Units selected in way
other than through chance.
Statistical Inference for
Populations
• Statistical inference can be made for random sampling
studies (Ch. 1.4.1) by using the sampling design.
– Simple random sample of size n: Subset of population
of size n selected in such a way that every subset of n
has same chance of being selected.
– Sample might be nonrepresentative (e.g., have
markedly different characteristics than population) but
we can use probability to find the chance of it being
nonrepresentative.
• Statistical inference for populations cannot be made for
non-random sampling studies. A non-random sampling
study can be nonrepresentative (biased) in unknown ways.
The Literary Digest Poll
• The Literary Digest Poll. In the 1936 presidential
election, the Literary Digest predicted an
overwhelming victory Landon over Roosevelt.
• Roosevelt won the election by a landslide – 62%
to 38%. What went wrong?
• The sample was taken by mailing questionnaires
to 10 million people whose names and addresses
came from sources like telephone books and club
membership lists. 2.4 million peopled returned
the samples.
Biased Samples
• Selection Bias: When the procedure for selecting a
sample results in samples that are systematically
different from the population.
• When a selection procedure is biased, taking a
large sample does not help. This just repeats the
basic mistake on a large scale.
• Causes of selection bias:
– Voluntary Response Sample
– Undercoverage
– Nonresponse
Statistical Inferences Permitted
by Study Designs
• Examples:
– Motivation and Creativity study
– Sex Discrimination study
– Researchers measured the lead content in teeth and IQ
scores for all 3,229 children attending first and second
grade between 1975 and 1978 in Chelsea and
Somerville, Mass. IQ scores for those with low lead
concentrations found to be significantly higher than for
those with high lead concentrations.
• Conceptual Exercises 1-12 in Ch. 1 relate to
statistical inferences permitted by study designs.