Transcript Section 10.2: Errors in Hypothesis Testing

Section 10.2: Errors in
Hypothesis Testing
• Test Procedure – the method we use to
determine whether H0 should be rejected.
• Type 1 Error: the error of rejecting H0
when H0 is true
• Type 2 Error: the error of failing to reject
H0 when H0 is false
Example
• The U.S. Department of Transportation reported
that during a recent period, 77% of all domestic
passenger flights arrived on time (meaning
within 15 minutes of the scheduled arrival).
Suppose that an airline with a poor on-time
record decides to offer its employees a bonus if,
in an upcoming month, the airline’s proportion of
on-time flights exceeds the overall industry rate
of 0.77. Let π be the true proportion of the
airline’s flights that are on time during the month
of interest.
• A random sample of flights might be selected
and used as a basis for choosing between
H0: π = .77 and Ha: π > .77
In this context, a Type I error (rejecting a true H0)
results in the airline rewarding its employees
when in fact their true proportion of on-time
flights did not exceed .77. A Type II error (not
rejecting a false H0) results in the airline
employees not receiving a reward that in fact
they deserved.
• The probability of a Type I error is denoted
by α and is called the level of significance
of the test. Thus, a test with α = .01 is said
to have a level of significance of .01 or to
be a level .01 test.
• The probability of a Type II error is
denoted by β.
Example
• Women with ovarian cancer are usually not
diagnosed until the disease is in an advanced
stage, when it is most difficult to treat. A new
blood test has been developed that appears to
be able to identify ovarian cancer at its earliest
stages. In a report issued by the National
Cancer Institute and the Food and Drug
Administration the following information from a
preliminary evaluation of the blood test was
given:
• The test was given to 50 women known to
have ovarian cancer, and it correctly
identified all of them as having cancer.
• The test was given to 66 women known
not to have ovarian cancer, and it correctly
identified 63 of these 66 as being cancer
free.
• We can think of using this blood test to
choose between two hypotheses:
H0: woman has ovarian cancer
Ha: woman does not have ovarian cancer
• In this situation, believing that a woman
with ovarian cancer is cancer free would
be a Type I error – rejecting the hypothesis
of ovarian cancer when it is, in fact, true.
• Believing that a woman who is actually
cancer free does have ovarian cancer is a
Type II error – not rejecting the null
hypothesis when it is, in fact, false.
• We can estimate the error of probabilities.
• The probability of a Type I error α is
approximately 0/50 = 0.
• The probability of a Type II error β is
approximately 3/66 = .046
• After assessing the consequences of Type
I and Type II errors, identify the largest α
that is tolerable for the problem. Then
employ a test procedure that uses this
maximum acceptable value – rather than
anything smaller – as the level of
significance (because using a smaller α
increases β). In other words, don’t make α
smaller than it needs to be.
Example
• The Associated Press reported that the
Environmental Protection Agency had
warned 819 communities that their tap
water contained too much lead. Drinking
water is considered unsafe if the mean
concentration of lead is 15 ppb (parts per
billion) or greater. The EPA requires the
cited communities to take corrective
actions and to monitor lead levels.
• With μ denoting the mean concentration of lead,
a cited community could test
H0: μ = 15 versus Ha: μ < 15
The null hypothesis states that the mean lead
concentration is excessive by EPA standards.
The alternative hypothesis states that the mean
lead concentration is at an acceptable level and
that the water system meets EPA standards for
lead.
• In this context, a Type I error leads to the
conclusion that a water source meets EPA
standards for lead when, in fact, it does not.
• Possible consequences of this type of error
include health risks associated with excessive
lead consumption (ex. Increased blood
pressure, hearing loss, and, in severe cases,
anemia and kidney damage)
• A Type II error is to conclude that the water does
not meet EPA standards for lead when, in fact, it
actually does.
• Possible consequences of a Type II error include
elimination of a community water source.
Because a Type I error might result in potentially
serious public health risks, a small value of α
such as .01 could be selected. This could
however, increase the risk of a Type II error.