Transcript Section 6-1 Statistical Significance

6.1 The Role of Probability in
Statistics: Statistical Significance
LEARNING GOAL
Understand the concept of statistical significance and
the essential role that probability plays in defining it.
Copyright © 2009 Pearson Education, Inc.
Definition
A set of measurements or observations in a
statistical study is said to be statistically
significant if it is unlikely to have occurred by
chance.
Copyright © 2009 Pearson Education, Inc.
Slide 6.1- 2
EXAMPLE 1 Likely Stories?
a. A detective in Detroit finds that 25 of the 62 guns
used in crimes during the past week were sold by
the same gun shop. This finding is statistically
significant.
Because there are many gun shops in the Detroit
area, having 25 out of 62 guns come from the same
shop seems unlikely to have occurred by chance.
Copyright © 2009 Pearson Education, Inc.
Slide 6.1- 3
EXAMPLE 1 Likely Stories?
b. In terms of the global average temperature, five of
the years between 1990 and 1999 were the five
hottest years in the 20th century. Having the five
hottest years in 1990–1999 is statistically significant.
By chance alone, any particular year in a century
would have a 5 in 100, or 1 in 20, chance of being
one of the five hottest years. Having five of those
years come in the same decade is very unlikely to
have occurred by chance alone.
This statistical significance suggests that the world
may be warming up.
Copyright © 2009 Pearson Education, Inc.
Slide 6.1- 4
EXAMPLE 1 Likely Stories?
c. The team with the worst win-loss record in
basketball wins one game against the defending
league champions.
This one win is not statistically significant because
although we expect a team with a poor win-loss
record to lose most of its games, we also expect it to
win occasionally, even against the defending league
champions.
Copyright © 2009 Pearson Education, Inc.
Slide 6.1- 5
From Sample to Population
EXAMPLE 2 Statistical Significance in Experiments
A researcher conducts a double-blind experiment that tests
whether a new herbal formula is effective in preventing colds.
During a three-month period, the 100 randomly selected people
in a treatment group take the herbal formula while the 100
randomly selected people in a control group take a placebo.
The results show that 30 people in the treatment group get colds,
compared to 32 people in the control group. Can we conclude
that the herbal formula is effective in preventing colds?
Copyright © 2009 Pearson Education, Inc.
Slide 6.1- 6
EXAMPLE 2 Statistical Significance in Experiments
Solution: Whether a person gets a cold during any three-month
period depends on many unpredictable factors. Therefore, we
should not expect the number of people with colds in any two
groups of 100 people to be exactly the same.
In this case, the difference between 30 people getting colds in
the treatment group and 32 people getting colds in the control
group seems small enough to be explainable by chance.
So the difference is not statistically significant, and we should
not conclude that the treatment is effective.
Copyright © 2009 Pearson Education, Inc.
Slide 6.1- 7
Quantifying Statistical Significance
The definition of statistical significance that we’ve
been using so far is too vague. We need a way to
quantify the idea of statistical significance.
In general, we determine statistical significance by
using probability to quantify the likelihood that a
result may have occurred by chance. We therefore
ask a question like this one:
Is the probability that the observed difference
occurred by chance less than or equal to 0.05 (or 1
in 20)?
Copyright © 2009 Pearson Education, Inc.
Slide 6.1- 8
Quantifying Statistical Significance
If the answer is yes (the probability is less than or
equal to 0.05), then we say that the difference is
statistically significant at the 0.05 level.
If the answer is no, the observed difference is
reasonably likely to have occurred by chance, so we
say that it is not statistically significant.
The choice of 0.05 is somewhat arbitrary, but it’s a
figure that statisticians frequently use. Nevertheless,
other probabilities are sometimes used, such as 0.1 or
0.01.
Copyright © 2009 Pearson Education, Inc.
Slide 6.1- 9
Quantifying Statistical Significance
• If the probability of an observed difference occurring
by chance is 0.05 (or 1 in 20) or less, the difference is
statistically significant at the 0.05 level.
• If the probability of an observed difference occurring
by chance is 0.01 (or 1 in 100) or less, the difference
is statistically significant at the 0.01 level.
Copyright © 2009 Pearson Education, Inc.
Slide 6.1- 10
TIME OUT TO THINK
Suppose an experiment finds that people taking a new
herbal remedy get fewer colds than people taking a
placebo, and the results are statistically significant at
the 0.01 level. Has the experiment proven that the
herbal remedy works? Explain.
Copyright © 2009 Pearson Education, Inc.
Slide 6.1- 11
EXAMPLE 2 Polio Vaccine Significance
In the test of the Salk polio vaccine (see Section 1.1),
33 of the 200,000 children in the treatment group got
paralytic polio, while 115 of the 200,000 in the control
group got paralytic polio. Calculations show that the
probability of this difference between the groups
occurring by chance is less than 0.01. Describe the
implications of this result.
Solution: The results of the polio vaccine test are
statistically significant at the 0.01 level, meaning that
there is a 0.01 chance (or less) that the difference
between the control and treatment groups occurred by
chance.
Copyright © 2009 Pearson Education, Inc.
Slide 6.1- 12
EXAMPLE 2 Polio Vaccine Significance
Solution: (cont.)
Therefore, we can be fairly confident that the vaccine
really was responsible for the fewer cases of polio in
the treatment group.
(In fact, the probability of the Salk results occurring by
chance is much less than 0.01, so researchers were
quite convinced that the vaccine worked; as we’ll
discuss in Chapter 9, this probability is called a “Pvalue.”)
Copyright © 2009 Pearson Education, Inc.
Slide 6.1- 13
The End
Copyright © 2009 Pearson Education, Inc.
Slide 6.1- 14