Transcript Analyze - Intro to Hypothesis Testing

Analyze Phase
Introduction to Hypothesis Testing
Hypothesis Testing (ND)
Welcome to Analyze
“X” Sifting
Hypothesis Testing Purpose
Inferential Statistics
Tests for Central Tendency
Intro to Hypothesis Testing
Tests for Variance
Hypothesis Testing ND P1
ANOVA
Hypothesis Testing ND P2
Hypothesis Testing NND P1
Hypothesis Testing NND P2
Wrap Up & Action Items
OSSS LSS Black Belt v10.0 - Analyze Phase
2
© Open Source Six Sigma, LLC
Six Sigma Goals and Hypothesis Testing
Our goal is to improve our Process Capability, this translates to the need to move the
process Mean (or proportion) and reduce the Standard Deviation.
– Because it is too expensive or too impractical (not to mention theoretically
impossible) to collect population data, we will make decisions based on sample
data.
– Because we are dealing with sample data, there is some uncertainty about the
true population parameters.
Hypothesis Testing helps us make fact-based decisions about whether there are different
population parameters or that the differences are just due to expected sample variation.
Process Capability of Process Before
LSL
Process Capability of Process After
USL
P rocess Data
LS L
100.00000
Target
*
USL
120.00000
S ample M ean
108.65832
S ample N
150
S tD ev (Within)
2.35158
S tD ev (O v erall)
5.41996
LSL
Within
Ov erall
P otential (Within) C apability
Cp
1.42
C PL
1.23
C PU
1.61
C pk
1.23
C C pk 1.42
USL
P rocess Data
LS L
100.00000
Target
*
USL
120.00000
S ample M ean
109.86078
S ample N
100
S tD ev (Within)
1.55861
S tD ev (O v erall)
1.54407
Within
Ov erall
P otential (Within) C apability
Cp
2.14
C PL
2.11
C PU
2.17
C pk
2.11
C C pk 2.14
O v erall C apability
Pp
PPL
PPU
P pk
C pm
96
O bserv ed P erformance
P P M < LS L 6666.67
PPM > USL
0.00
P P M Total
6666.67
100
E xp. Within P erformance
P P M < LS L 115.74
PPM > USL
0.71
P P M Total
116.45
104
108
112
116
Pp
PPL
PPU
P pk
C pm
120
102
E xp. O v erall P erformance
P P M < LS L 55078.48
P P M > U S L 18193.49
P P M Total
73271.97
OSSS LSS Black Belt v10.0 - Analyze Phase
O v erall C apability
0.62
0.53
0.70
0.53
*
O bserv ed P erformance
P P M < LS L 0.00
P P M > U S L 0.00
P P M Total
0.00
3
105
E xp. Within P erformance
P P M < LS L 0.00
P P M > U S L 0.00
P P M Total
0.00
108
111
114
117
2.16
2.13
2.19
2.13
*
120
E xp. O v erall P erformance
P P M < LS L 0.00
P P M > U S L 0.00
P P M Total
0.00
© Open Source Six Sigma, LLC
Purpose of Hypothesis Testing
The purpose of appropriate Hypothesis Testing is to integrate the Voice of
the Process with the Voice of the Business to make data-based decisions
to resolve problems.
Hypothesis Testing can help avoid high costs of experimental efforts by
using existing data. This can be likened to:
– Local store costs versus mini bar expenses.
– There may be a need to eventually use experimentation, but careful
data analysis can indicate a direction for experimentation if
necessary.
The probability of occurrence is based on a pre-determined statistical
confidence.
Decisions are based on:
– Beliefs (past experience)
– Preferences (current needs)
– Evidence (statistical data)
– Risk (acceptable level of failure)
OSSS LSS Black Belt v10.0 - Analyze Phase
4
© Open Source Six Sigma, LLC
The Basic Concept for Hypothesis Tests
Recall from the discussion on classes and cause of distributions that
a data set may seem Normal, yet still be made up of multiple
distributions.
Hypothesis Testing can help establish a statistical difference
between factors from different distributions.
0.8
0.7
0.6
freq
0.5
0.4
0.3
0.2
0.1
0.0
-3
-2
-1
0
1
2
3
x
Did my sample come from this population? Or this? Or this?
OSSS LSS Black Belt v10.0 - Analyze Phase
5
© Open Source Six Sigma, LLC
Significant Difference
Are the two distributions “significantly” different from each
other? How sure are we of our decision?
How do the number of observations affect our confidence in
detecting population Mean?


Sample 2
Sample 1
OSSS LSS Black Belt v10.0 - Analyze Phase
6
© Open Source Six Sigma, LLC
Detecting Significance
Statistics provide a methodology to detect differences.
– Examples might include differences in suppliers, shifts or
equipment.
– Two types of significant differences occur and must be well
understood, practical and statistical.
– Failure to tie these two differences together is one of the most
common errors in statistics.
HO: The sky is not falling.
HA: The sky is falling.
OSSS LSS Black Belt v10.0 - Analyze Phase
7
© Open Source Six Sigma, LLC
Practical vs. Statistical
Practical Difference: The difference which results in an
improvement of practical or economic value to the company.
– Example, an improvement in yield from 96 to 99 percent.
Statistical Difference: A difference or change to the process that
probably (with some defined degree of confidence) did not happen
by chance.
– Examples might include differences in suppliers, markets or servers.
We will see that it is possible to realize a statistically
significant difference without realizing a practically
significant difference.
OSSS LSS Black Belt v10.0 - Analyze Phase
8
© Open Source Six Sigma, LLC
Detecting Significance
During the Measure Phase,
it is important that the nature of
the problem be well understood.
Mean Shift
In understanding the problem,
the practical difference to be
achieved must match the
statistical difference.
The difference can be either a
change in the Mean or in the
variance.
Variation Reduction
Detection of a difference is then
accomplished using statistical
Hypothesis Testing.
OSSS LSS Black Belt v10.0 - Analyze Phase
9
© Open Source Six Sigma, LLC
Hypothesis Testing
A Hypothesis Test is an a priori theory relating to differences
between variables.
A statistical test or Hypothesis Test is performed to prove or
disprove the theory.
A Hypothesis Test converts the practical problem into a statistical
problem.
– Since relatively small sample sizes are used to estimate
population parameters, there is always a chance of collecting
a non-representative sample.
– Inferential statistics allows us to estimate the probability of
getting a non-representative sample.
OSSS LSS Black Belt v10.0 - Analyze Phase
10
© Open Source Six Sigma, LLC
DICE Example
We could throw it a number of times and track how many each face
occurred. With a standard die, we would expect each face to occur 1/6 or
16.67% of the time.
If we threw the die 5 times and got 5 one’s, what would you conclude? How
sure can you be?
– Pr (1 one) = 0.1667 Pr (5 ones) = (0.1667)5 = 0.00013
There are approximately 1.3 chances out of 1000 that we could have gotten
5 ones with a standard die.
Therefore, we would say we are willing to take a 0.1% chance of being
wrong about our hypothesis that the die was “loaded” since the results do
not come close to our predicted outcome.
OSSS LSS Black Belt v10.0 - Analyze Phase
11
© Open Source Six Sigma, LLC
Hypothesis Testing
α
DECISIONS
β
OSSS LSS Black Belt v10.0 - Analyze Phase
n
12
© Open Source Six Sigma, LLC
Statistical Hypotheses
A hypothesis is a predetermined theory about the nature of, or
relationships between variables. Statistical tests can prove (with a
certain degree of confidence) that a relationship exists.
We have two alternatives for hypothesis.
– The “null hypothesis” Ho assumes that there are no differences or
relationships. This is the default assumption of all statistical
tests.
– The “alternative hypothesis” Ha states that there is a difference or
relationship.
P-value > 0.05
P-value < 0.05
Ho = no difference or relationship
Ha = is a difference or relationship
Making a decision does not FIX a problem,
taking action does.
OSSS LSS Black Belt v10.0 - Analyze Phase
13
© Open Source Six Sigma, LLC
Steps to Statistical Hypothesis Test
1. State the Practical Problem.
2. State the Statistical Problem.
a) HO: ___ = ___
b) HA: ___ ≠ ,>,< ___
3. Select the appropriate statistical test and risk levels.
a) α = .05
b) β = .10
4. Establish the sample size required to detect the difference.
5. State the Statistical Solution.
6. State the Practical Solution.
Noooot THAT practical
solution!
OSSS LSS Black Belt v10.0 - Analyze Phase
14
© Open Source Six Sigma, LLC
How Likely is Unlikely?
Any differences between observed data and claims made under H0
may be real or due to chance.
Hypothesis Tests determine the probabilities of these differences
occurring solely due to chance and call them P-values.
The a level of a test (level of significance) represents the yardstick
against which P-values are measured and H0 is rejected if the
P-value is less than the alpha level.
The most commonly used levels are 5%, 10% and 1%.
OSSS LSS Black Belt v10.0 - Analyze Phase
15
© Open Source Six Sigma, LLC
Hypothesis Testing Risk
The alpha risk or Type 1 Error (generally called the “Producer’s Risk”)
is the probability that we could be wrong in saying that something is
“different.” It is an assessment of the likelihood that the observed
difference could have occurred by random chance. Alpha is the
primary decision-making tool of most statistical tests.
Actual Conditions
Not Different
(Ho is True)
Not Different
(Fail to Reject Ho)
Statistical
Conclusions
(Ho is False)
Correct
Decision
Type II
Error
Type 1
Error
Correct
Decision
Different
(Reject Ho)
OSSS LSS Black Belt v10.0 - Analyze Phase
Different
16
© Open Source Six Sigma, LLC
Alpha Risk
Alpha ( ) risks are expressed relative to a reference
distribution.
Distributions include:
– t-distribution
The a-level is represented by
the clouded areas.
– z-distribution
–
 2-
Sample results in this area lead
to rejection of H0.
distribution
– F-distribution
Region of
DOUBT
Region of
DOUBT
Accept as chance differences
OSSS LSS Black Belt v10.0 - Analyze Phase
17
© Open Source Six Sigma, LLC
Hypothesis Testing Risk
The beta risk or Type 2 Error (also called the “Consumer’s Risk”) is
the probability that we could be wrong in saying that two or more
things are the same when, in fact, they are different.
Actual Conditions
Not Different
(Ho is True)
Not Different
(Fail to Reject Ho)
Statistical
Conclusions
Correct
Decision
Type II
Error
Type 1
Error
Correct
Decision
Different
(Reject Ho)
OSSS LSS Black Belt v10.0 - Analyze Phase
Different
(Ho is False)
18
© Open Source Six Sigma, LLC
Beta Risk
Beta Risk is the probability of failing to reject the null hypothesis
when a difference exists.
Distribution if H0 is true
Reject H0
 = Pr(Type 1 error)
 = 0.05
H0 value
Accept H0
= Pr(Type II error)
Distribution if Ha is true

Critical value of test
statistic
OSSS LSS Black Belt v10.0 - Analyze Phase
19
© Open Source Six Sigma, LLC
Distinguishing between Two Samples
Recall from the Central Limit
Theorem as the number of
individual observations
increase the Standard Error
decreases.
d
Theoretical Distribution
of Means
When n = 2
d=5
S=1
In this example when n=2 we
cannot distinguish the
difference between the Means
(> 5% overlap, P-value > 0.05).
When n=30, we can distinguish
between the Means (< 5%
overlap, P-value < 0.05) There
is a significant difference.
OSSS LSS Black Belt v10.0 - Analyze Phase
20
Theoretical Distribution
of Means
When n = 30
d=5
S=1
© Open Source Six Sigma, LLC
Delta Sigma—The Ratio between d and S
Delta (d) is the size of the
difference between two Means or
one Mean and a target value.
Sigma (S) is the sample Standard
Deviation of the distribution of
individuals of one or both of the
samples under question.
Large Delta
d
When d & S is large, we don’t
need statistics because the
differences are so large.
If the variance of the data is large,
it is difficult to establish
differences. We need larger
sample sizes to reduce
uncertainty.
Large S
We want to be 95% confident in all of our estimates!
OSSS LSS Black Belt v10.0 - Analyze Phase
21
© Open Source Six Sigma, LLC
Typical Questions on Sampling
Question:
“How many samples should we take?”
Answer:
“Well, that depends on the size of your delta and
Standard Deviation”.
Question:
Answer:
“How should we conduct the sampling?”
“Well, that depends on what you want to know”.
Question:
Answer:
“Was the sample we took large enough?”
“Well, that depends on the size of your delta and
Standard Deviation”.
Question:
Answer:
“Should we take some more samples just to be sure?”
“No, not if you took the correct number of samples the
first time!”
OSSS LSS Black Belt v10.0 - Analyze Phase
22
© Open Source Six Sigma, LLC
The Perfect Sample Size
The minimum sample
size required to
provide exactly 5%
overlap (risk). In
order to distinguish
the Delta.
Note: If you are
working with Nonnormal Data, multiply
your calculated
sample size by 1.1
40
60
70
60
70
Population
40
OSSS LSS Black Belt v10.0 - Analyze Phase
50
23
50
© Open Source Six Sigma, LLC
Hypothesis Testing Roadmap
Normal
Test of Equal Variance
1 Sample Variance
Variance Equal
2 Sample T
1 Sample t-test
Variance Not Equal
One Way ANOVA
OSSS LSS Black Belt v10.0 - Analyze Phase
2 Sample T
24
One Way ANOVA
© Open Source Six Sigma, LLC
Hypothesis Testing Roadmap
Non Normal
Test of Equal Variance
Mann-Whitney
OSSS LSS Black Belt v10.0 - Analyze Phase
Median Test
Several Median Tests
25
© Open Source Six Sigma, LLC
Hypothesis Testing Roadmap
Attribute Data
One Factor
Two
Samples
One Sample
One Sample
Proportion
Two Sample
Proportion
Minitab:
Stat - Basic Stats - 2 Proportions
If P-value < 0.05 the proportions
are different
Two Factors
Two or More
Samples
Chi Square Test
(Contingency Table)
Minitab:
Stat - Tables - Chi-Square Test
If P-value < 0.05 at least one
proportion is different
Chi Square Test
(Contingency Table)
Minitab:
Stat - Tables - Chi-Square Test
If P-value < 0.05 the factors are not
independent
OSSS LSS Black Belt v10.0 - Analyze Phase
26
© Open Source Six Sigma, LLC
Common Pitfalls to Avoid
While using Hypothesis Testing the following facts should be borne in
mind at the conclusion stage:
–
–
–
–
The decision is about Ho and NOT Ha.
The conclusion statement is whether the contention of Ha was upheld.
The null hypothesis (Ho) is on trial.
When a decision has been made:
• Nothing has been proved.
• It is just a decision.
• All decisions can lead to errors (Types I and II).
– If the decision is to “Reject Ho,” then the conclusion should read “There
is sufficient evidence at the α level of significance to show that “state
the alternative hypothesis Ha.”
– If the decision is to “Fail to Reject Ho,” then the conclusion should read
“There isn’t sufficient evidence at the α level of significance to show
that “state the alternative hypothesis.”
OSSS LSS Black Belt v10.0 - Analyze Phase
27
© Open Source Six Sigma, LLC
Summary
At this point, you should be able to:
• Articulate the purpose of Hypothesis Testing
• Explain the concepts of the Central Tendency
• Be familiar with the types of Hypothesis Tests
OSSS LSS Black Belt v10.0 - Analyze Phase
28
© Open Source Six Sigma, LLC