Transcript Type I and Type II error in SPC

The Importance of
Understanding Type I
and Type II Error in
Statistical Process
Control Charts
Phillip R. Rosenkrantz, Ed.D., P.E.
California State Polytechnic University
Pomona
ASQ Orange Empire Section
October 11, 2016
2
Goals

Provide a brief review of the concepts of process control
and process capability

Explain Type I and Type II error with colorful examples

Give examples of Type I and Type II error for common
decision rules

Illustrate how the improper use of decision rules creates
excessive Type I error and creates mistrust in the use of
SPC

Suggest simple approaches for reducing Type I error in SPC
3
Assignable vs. Common Cause
Variation
 Dr.
Walter Shewhart developed Statistical Process
Control (SPC) during the 1920s. Dr. W. Edwards
Deming promoted SPC during WWII and after.
 Premise
is that there are three types of variation
 Common Cause Variation
 Assignable (or Special Cause) variation
 Tampering (or over-adjusting)
 Each
of these types of variation require a different
approach or type of action.
4
Quinconx Demonstration
 Common
cause (natural) variation - Built-in random
variation in the system. Difficult to reduce without changing
the system or process. Responsibility of management
because they are responsible for the system.
 Assignable
or Special cause variation - Variation caused
by identifiable events usually under control of the work
group
 Tampering
- Over adjusting of the process resulting in
increased variation.
5
Common Cause vs. Assignable
Cause Variation
 According
to Dr. Deming’s research, more than 85% of
problems are the result of “common cause” variation.
Management is responsible for the system and it is
their responsibility to work on reducing this type of
variation. Later research puts the estimate at over
94%.
 The
work group is responsible for preventing and
reducing “assignable cause” variation.
 Management
needs to understand these concepts.
6
Tampering – The Third Type of
Variation
 Tampering
is over-adjusting the system caused by a
lack of understanding of variation.
Sometimes large built in variation is mistaken for a
process going “out of calibration” and needing
adjustment
 Over adjusting actually increases variation by adding
more variation each time the process is changed
 Tampering is a difficult habit to break because many
machine operators consider it their “job” to constantly
adjust their machine.


SPC reduces or eliminates unnecessary adjustments.
7
Major Concept #1: Process
Capability
 The
ability of a process to produce within
specification limits
to produce within specifications – process is
“capable”
 Not able to produce within specifications – “not
capable”
 Able
 Often
quantified with process capability
indices
Pp – Ability to stay within specs if centered
 Cpk, Ppk – Ability based on current distribution
 Cp,
8
Major Concept #2: Process Control
Process Control refers to how stable and
consistent the process is.
 “In-control”
- stable and only experiencing systematic or
“common cause” variation.
 “Not in-control” – Process is not stable. Mean and
variation are changing due to identifiable or “special”
causes (usually controllable by those running the
operation).
 Represents
<10% of the problems
Process Capability
What it is
Process Control
Note - no reference to
specs !
In Control
(Special Causes Eliminated)
Out of Control
(Special Causes Present)
Process Capability
Lower Spec Limit
Upper Spec Limit
In Control but not Capable
(Variation from Common Causes
Excessive)
In Control and Capable
(Variation from Common
Causes Reduced)
10
Control Charts
 Walter
Shewhart developed control charts that help
management and workers identify common cause and
special cause variation
Management’s responsibility to reduce common cause
variation
 The work group is primarily responsible for controlling special
or assignable cause variation

 Small
samples are taken periodically with statistics (e.g.,
average, range) plotted on charts and reveal the amount
and type of variation. Control limits are traditionally +/- 3
standard deviations from the process average.
11
Sample Statistical Process Control
(SPC) Chart
12
Use of Control Charts
When the process remains within control limits
with only a random pattern, process variation can
be attributed to common cause variation (random
variation in the system) and is deemed “in
control.” The process is stable and continues.
When the process goes beyond control limits or
is non-random, it is assumed that an assignable
cause is present and deemed “out of control.”
The process is not stable and predictable. Find
and eliminate the assignable cause.
13
Implementing SPC
 SPC
was designed to be a tool for first line workers
to monitor for the presence of assignable causes
 Requires
that management not to use results for
evaluating performance, but rather only for
improving processes--otherwise data will be biased
 Implies
that the work group and support personnel
take time from their other duties to permanently
eliminate assignable causes that reoccur
 Requires
a culture of trust to work effectively
14
Where to Use SPC
 Use
strategically on:
 Critical customer requirements
 Major problems
 Six Sigma project related processes
 Use
tactically on:
 Processes that are not “capable” and need to be
monitored closely
15
Managing SPC
 Any
Black Belt or Master Black Belt should be able
to set up the proper SPC Charts and monitor them.
 Issues
to address when designing SPC charts:
 Proper type of chart to use for the situation
 Sample size and sample frequency
 Sampling method
 Decision rules being used
 How assignable causes will be resolved
 Is the process capable or not capable
16
Decision or Sensitizing Rules
 Decision
Rules (a.k.a. Sensitizing rules) are used by
operators to determine if a pattern of points indicates a
process is no longer stable, that is: “out-of-control”.
 Some rules are designed to detect changes or shifts in
the process center (mean)
 Some rules are designed to detect changes in the
process variation (standard deviation)
 Some rules are designed to detect a non-normal
patterns (e.g. trends or cycles)
17
18
Types of error when you use
sampling
 Control
charts are based on sampling. Sampling is
subject to two kinds of error:
 Type I error (α): “False Alarm” – The sample
indicates the process is “out-of-control” but is not
 Type II error (β): “Failure to detect” – The sample
indicates the process is stable, but it really is “out-ofcontrol”
 In
most quality situations the larger concern is avoiding
Type II error: “Failure to detect”. However, with SPC
probably the larger concern is Type I error: “False
alarms”
19
Types of Error
Test Says
H0 True
State of
Reality
H0
True
H0
False
H0 False
Type I error: a
No error
False alarm,
producer’s risk
Type II error: b
Failure to detect,
consumer’s risk
No error
20
Examples
 Ho:
Part is good
Ha: Part is bad
 Ho:
Person did not commit the crime
Ha: Person did commit the crime
 Ho:
The appendix is good
Ha: The appendix is bad
 Ho:
The process is in control
Ha: The process in not in control
A look at two decision rules and
the probability of Type I and Type
II errors
22
23
The Central Limit Theorem is the basis for
assuming that a process “in control” follows a
Normal Distribution
24
Probability zones for the normal
distribution
Rule 1 – Any point outside the 3σ control limits
(probability shown for a sequence of 8 points)
False Alarm
Failure
To Detect
Failure
To Detect
Rule 4 – A run of 8 points on the same side of the
centerline but within the 3σ control limits
False Alarm
Failure
To Detect
Failure
To Detect
27
Overall Type I Error for both rules
28
Cumulative effect of Type I error on a sequence
of 8 points as decision rules are added
The probability of a False Alarm
Increases dramatically as decision rules
are added. It does not take too many
false alarms before operators begin
to lose faith in control charts and
start to ignore them.
29
Type I Error - A Common Problem
That Makes SPC Ineffective
 Too
much Type I error eventually renders SPC
ineffective. People get tired of chasing false alarms.
 Many
experts recommend using two decision rules
(three at the most) to minimize Type I error. Rules 1 and
4 are commonly used.
 Often,
upon set up, software installers toggle on all
decision rules thinking that is desirable.
 If
you use SPC software, ask to see which rules are in
effect.
30
Tactics for Managers
 Ask
to see SPC Charts
 Ask
how it was decided which type of chart to use.
 Ask
which decision rules are being used.
 Look
for out-of-control points on the chart and what the
response was in removing the causes.
 Ask
if the work group is having trouble resolving
assignable causes. Were Pareto Charts, Cause & Effect
Diagrams, or other tools used to prioritize efforts?