Transcript Conclusion - Northern Virginia Section 0511 ASQ

The Odds Are Against Auditing
Statistical Sampling Plans
Steven Walfish
Statistical Outsourcing Services
Olney, MD
301-325-3129
[email protected]
1
Topics of Discussion

The Paradox

Different types of sampling plans.

Types of Risk

Statistical Distribution
• Normal
• Binomial
• Poisson

When to Audit.
April 21, 2010
ASQ Section 511
2
The Paradox
 During
an audit you increase the sample
size if you have a finding…
 But,
no findings might be because your
sample size is too small to find errors.
April 21, 2010
ASQ Section 511
3
Common Sampling Strategies
 Simple
random sample.
 Stratified
sample.
 Systematic
sample.
 Haphazard
 Probability
April 21, 2010
proportional to size
ASQ Section 511
4
Types of Risk
Reality
Decision
April 21, 2010
Accept
Reject
Accept
Correct Decision
Type II Error (b)
Consumer Risk
Reject
Type I Error (a)
Producer Risk
Correct Decision
Power (1-b)
ASQ Section 511
5
Normal Distribution
D is tr ibutio n P lo t

Typical bell-shaped
curve.

Z-scores determine
how many standard
deviations a value is
from the mean.
No r m a l, M e a n= 0 , S tDe v= 1
0.4
De ns it y
0.3
0.2
0.1
0.0228
0.0
0 .0 2 2 8
-2
April 21, 2010
0
X
2
ASQ Section 511
6
Continuous Data Sample Size
n
 As
Z
a
 Zb

)
2
S
2
2
the effect size decreases, the sample size
increases.
 As
variability increases, sample size increases.
 Sample
size is proportional to risks taken.
April 21, 2010
ASQ Section 511
7
Binomial Distribution
 Binomial
Distribution
n
(n  x)
x
a     p  1 - p )
x
 where:
• n is the sample size
• x is the number of positives
• p is the probability
• a is the probability of the observing x in a
sample of n.
April 21, 2010
ASQ Section 511
8
Binomial Confidence Intervals
 Binomial
Distribution
n
(n  x)
x
a     p  1 - p )
x
 Solve
the equation for p given a, x and n.
 x=0,
n=11 and a=0.05 (95% confidence).
 x=2,
n=27 and a=0.01 (99% confidence).
• p=0.28 (table shows 0.30ucl)
• p=0.298 (table shows 0.30ucl)
April 21, 2010
ASQ Section 511
9
Poisson Distribution

Describes the number of times an event occurs
in a finite observation space.

For example, a Poisson distribution can
describe the number audit findings.

The Poisson distribution is defined by one
parameter: lambda. This parameter equals the
mean and variance. As lambda increases, the
Poisson distribution approaches a normal
distribution.
April 21, 2010
ASQ Section 511
10
Hypothesis Testing - Poisson

 e
x
P ( x) 

x!
April 21, 2010

P(x) = probability of
exactly x
occurrences.
 is the mean
number of
occurrences.
ASQ Section 511
11
Example of Poisson

If the average number () of audit findings is 5.5.

What is the probability of a sample with exactly 0
findings?
•

0.0041 (0.41%)
What is the probability of having 4 or less findings in a
sample
•
•
(x=0 + x=1 + x=2 + x=3 + x=4)
0.0041 + 0.0225 + 0.0618 + 0.1133 + 0.1558 = 0.358
(35.8%)
April 21, 2010
ASQ Section 511
12
Poisson Confidence Interval

The central confidence interval approach can
be approximated in two ways:

95% CI for x=6 would be (2.2,13.1)
1
2
 0 .9 7 5 ; 2 x   
2
 1 .9 6 5


2

April 21, 2010
1
2
 0 .0 2 5 ; 2 ( x  1)
2
2

 1 .9 6 5
x   

2


ASQ Section 511

x 1

2
13
Major Drawback
 What
is missing in ALL calculations for
the Poisson?
 No
reference to sample size.
 Assumes
April 21, 2010
a large population (np>5)
ASQ Section 511
14
Comparison
Sample Size
Poisson
Mean
5
50
100
500
1,000
April 21, 2010
0.1
0.05
0.01
0.005
UCL
10.51
Binomial
19.88
10.22
2.09
1.05
ASQ Section 511
9.94
10.22
10.45
10.47
15
N 1

was an unpublished report by the
AOAC in 1927.
N 1
 It
was intended to be a quick rule of
thumb for inspection of foods.
 Since
it was unpublished, there was not a
description of the statistical basis of it.
April 21, 2010
ASQ Section 511
16
N 1
 There
is no known statistical justification for the
use of the square root of n plus one’ sampling
plan.
 “Despite
the fact that there is no statistical basis
for a ‘square root of n plus one’ sampling plan,
most firms utilize this approach for incoming
raw materials.”
• Henson, E., A Pocket Guide to CGMP Sampling, IVT.
April 21, 2010
ASQ Section 511
17
Compare the Plans
ANSI/ASQ Z1.4
Square root N plus one

Lot Size N=1000

Lot Size N=1000

Sample size n=32

Sample size n=33

Acceptance Ac=0

Acceptance Ac=0

Rejection Re=1

Rejection Re=1

AQL=0.160%

AQL=0.153%

LQ = 6.94%

LQ = 6.63%
April 21, 2010
ASQ Section 511
18
Is it a Real Sampling Plan?
 Yes,
it meets the Z1.4 definition of a sampling
plan.
 It
is statistically valid in that it defines the lot
size, N, the sample size, n, the accept number,
Ac, and the reject number, Re.
 The
Operational Characteristic, OC, curve can
be calculated for any square root N plus one
plan.
April 21, 2010
ASQ Section 511
19
Sample Size Comparison
 It
is very common to use Z1.4 General Level I
as the plan for audits.
 The
sample sizes for square root N plus one
are very close to the sample sizes for Z1.4 GL I.
 Square
root N plus one can be used any where
that Z1.4 GL I is or could be used.
April 21, 2010
ASQ Section 511
20
Sample Size Comparison
Sqrt(N+1) versus Z1.4
Sample Size
1000
100
Sqrt (N+1)
Z1.4
10
1
1
10
100
1000
10000
100000
1000000
Lot Size
April 21, 2010
ASQ Section 511
21
Is it a Good Plan?
 Like
Z1.4 GL I it can be used for audits.
 Any
plan is justified by AQL and LQ
 It
is easy to use and calculate.
 Works
best with an Ac=0.
April 21, 2010
ASQ Section 511
22
Example
Lot Size Sample Size
April 21, 2010
Ac=0
Ac=1
AQL
LQ
AQL
LQ
4
3
1.69
54
13.50
80
10
4
1.27
44
9.78
68
25
6
0.85
32
6.30
51
50
8
0.64
25
4.60
41
100
11
0.46
19
3.30
31
250
17
0.30
13
2.10
21
500
23
0.22
9.5
1.57
16
1000
33
0.16
6.7
1.09
11
10000
101
0.05
2.3
0.35
3.8
ASQ Section 511
23
Using Statistics

How do you determine when you have too many
findings?

How do you determine the correct sample size for an
audit?

Would a confidence interval approach work?
•
As long as the observed number is lower than the upper
confidence interval, the system is in control.
April 21, 2010
ASQ Section 511
24
Deciding to Audit

Need to use risk or statistical probability
to determine when to audit:
• Critical components
• Low rank
• High Volume suppliers
• No third party data available
April 21, 2010
ASQ Section 511
25
Results of an Audit

The results of an audit can help to establish
acceptance controls.

Better audit results would have less risk, and
require smaller sample sizes for incoming
inspection.

Can use AQL or LTPD type of acceptance
plans based on audit results.
April 21, 2010
ASQ Section 511
26
Conclusion

Using the correct sampling strategy helps to
assure coverage during an audit.

Using confidence intervals to determine if a
system is in control.

More compliant systems require larger sample
sizes.
April 21, 2010
ASQ Section 511
27
Questions
Steven Walfish
[email protected]
301-325-3129 (Phone)
240-559-0989 (Fax)
April 21, 2010
ASQ Section 511
28