Transcript DESIGN OF EXPERIMENTS

DESIGN OF EXPERIMENTS
by
R. C. Baker
How to gain 20 years of experience in
one short week!
1
Role of DOE in Process Improvement
• DOE is a formal mathematical method for
systematically planning and conducting
scientific studies that change experimental
variables together in order to determine
their effect of a given response.
• DOE makes controlled changes to input
variables in order to gain maximum
amounts of information on cause and effect
relationships with a minimum sample size. 2
Role of DOE in Process Improvement
• DOE is more efficient that a standard
approach of changing “one variable at a
time” in order to observe the variable’s
impact on a given response.
• DOE generates information on the effect
various factors have on a response variable
and in some cases may be able to determine
optimal settings for those factors.
3
Role of DOE in Process Improvement
• DOE encourages “brainstorming” activities
associated with discussing key factors that
may affect a given response and allows the
experimenter to identify the “key” factors
for future studies.
• DOE is readily supported by numerous
statistical software packages available on
the market.
4
BASIC STEPS IN DOE
•
•
•
•
•
Four elements associated with DOE:
1. The design of the experiment,
2. The collection of the data,
3. The statistical analysis of the data, and
4. The conclusions reached and
recommendations made as a result of the
experiment.
5
TERMINOLOGY
• Replication – repetition of a basic
experiment without changing any factor
settings, allows the experimenter to estimate
the experimental error (noise) in the system
used to determine whether observed
differences in the data are “real” or “just
noise”, allows the experimenter to obtain
more statistical power (ability to identify
small effects)
6
TERMINOLOGY
• .Randomization – a statistical tool used to
minimize potential uncontrollable biases in
the experiment by randomly assigning
material, people, order that experimental
trials are conducted, or any other factor not
under the control of the experimenter.
Results in “averaging out” the effects of the
extraneous factors that may be present in
order to minimize the risk of these factors
affecting the experimental results.
7
TERMINOLOGY
• Blocking – technique used to increase the
precision of an experiment by breaking the
experiment into homogeneous segments
(blocks) in order to control any potential
block to block variability (multiple lots of
raw material, several shifts, several
machines, several inspectors). Any effects
on the experimental results as a result of the
blocking factor will be identified and
minimized.
8
TERMINOLOGY
• Confounding - A concept that basically
means that multiple effects are tied together
into one parent effect and cannot be
separated. For example,
• 1. Two people flipping two different coins
would result in the effect of the person and
the effect of the coin to be confounded
• 2. As experiments get large, higher order
interactions (discussed later) are
confounded with lower order interactions or
main effect.
9
TERMINOLOGY
• Factors – experimental factors or
independent variables (continuous or
discrete) an investigator manipulates to
capture any changes in the output of the
process. Other factors of concern are those
that are uncontrollable and those which are
controllable but held constant during the
experimental runs.
10
TERMINOLOGY
• Responses – dependent variable measured
to describe the output of the process.
• Treatment Combinations (run) –
experimental trial where all factors are set at
a specified level.
11
TERMINOLOGY
•
•
Fixed Effects Model - If the treatment
levels are specifically chosen by the
experimenter, then conclusions reached
will only apply to those levels.
Random Effects Model – If the treatment
levels are randomly chosen from a
population of many possible treatment
levels, then conclusions reached can be
extended to all treatment levels in the
population.
12
PLANNING A DOE
• Everyone involved in the experiment should
have a clear idea in advance of exactly what
is to be studied, the objectives of the
experiment, the questions one hopes to
answer and the results anticipated
13
PLANNING A DOE
• Select a response/dependent variable
(variables) that will provide information
about the problem under study and the
proposed measurement method for this
response variable, including an
understanding of the measurement system
variability
14
PLANNING A DOE
• Select the independent variables/factors
(quantitative or qualitative) to be
investigated in the experiment, the number
of levels for each factor, and the levels of
each factor chosen either specifically (fixed
effects model) or randomly (random effects
model).
15
PLANNING A DOE
• Choose an appropriate experimental design
(relatively simple design and analysis methods
are almost always best) that will allow your
experimental questions to be answered once the
data is collected and analyzed, keeping in mind
tradeoffs between statistical power and economic
efficiency. At this point in time it is generally
useful to simulate the study by generating and
analyzing artificial data to insure that
experimental questions can be answered as a
result of conducting your experiment
16
PLANNING A DOE
• Perform the experiment (collect data)
paying particular attention such things as
randomization and measurement system
accuracy, while maintaining as uniform an
experimental environment as possible.
How the data are to be collected is a critical
stage in DOE
17
PLANNING A DOE
• Analyze the data using the appropriate
statistical model insuring that attention is
paid to checking the model accuracy by
validating underlying assumptions
associated with the model. Be liberal in the
utilization of all tools, including graphical
techniques, available in the statistical
software package to insure that a maximum
amount of information is generated
18
PLANNING A DOE
• Based on the results of the analysis, draw
conclusions/inferences about the results,
interpret the physical meaning of these
results, determine the practical significance
of the findings, and make recommendations
for a course of action including further
experiments
19
SIMPLE COMPARATIVE EXPERIMENTS
• Single Mean Hypothesis Test
• Difference in Means Hypothesis Test with
Equal Variances
• Difference in Means Hypothesis Test with
Unequal Variances
• Difference in Variances Hypothesis Test
• Paired Difference in Mean Hypothesis Test
• One Way Analysis of Variance
20
CRITICAL ISSUES ASSOCIATED WITH
SIMPLE COMPARATIVE EXPERIMENTS
• How Large a Sample Should We Take?
• Why Does the Sample Size Matter
Anyway?
• What Kind of Protection Do We Have
Associated with Rejecting “Good” Stuff?
• What Kind of Protection Do We Have
Associated with Accepting “Bad” Stuff?
21
Single Mean Hypothesis Test
• After a production run of 12 oz. bottles,
concern is expressed about the possibility
that the average fill is too low.
• Ho: m = 12
• Ha: m <> 12
• level of significance = a = .05
• sample size = 9
• SPEC FOR THE MEAN: 12 + .1
22
Single Mean Hypothesis Test
•
•
•
•
•
•
Sample mean = 11.9
Sample standard deviation = 0.15
Sample size = 9
Computed t statistic = -2.0
P-Value = 0.0805162
CONCLUSION: Since P-Value > .05, you
fail to reject hypothesis and ship product.
23
Single Mean Hypothesis Test Power Curve
Power Curve
alpha = 0.05, sigma = 0.15
1
Power
0.8
0.6
0.4
0.2
0
11.8
11.9
12
12.1
12.2
True Mean
24
Single Mean Hypothesis Test Power Curve
25
Single Mean Hypothesis Test Power
Curve - Different Sample Sizes
26
DIFFERENCE IN MEANS - EQUAL
VARIANCES
•
•
•
•
•
Ho: m1 = m2
Ha: m1 <> m2
level of significance = a = .05
sample sizes both = 15
Assumption: s1 = s2
*****************************************************
• Sample means = 11.8 and 12.1
• Sample standard deviations = 0.1 and 0.2
• Sample sizes = 15 and 15
27
DIFFERENCE IN MEANS - EQUAL VARIANCES
Can you detect this difference?
28
DIFFERENCE IN MEANS - EQUAL
VARIANCES
29
DIFFERENCE IN MEANS - unEQUAL
VARIANCES
• Same as the “Equal Variance” case except
the variances are not assumed equal.
• How do you know if it is reasonable to
assume that variances are equal OR
unequal?
30
DIFFERENCE IN VARIANCE
HYPOTHESIS TEST
•
•
•
•
•
•
•
•
•
Same example as Difference in Mean:
Sample standard deviations = 0.1 and 0.2
Sample sizes = 15 and 15
**********************************
Null Hypothesis: ratio of variances = 1.0
Alternative: not equal
Computed F statistic = 0.25
P-Value = 0.0140071
Reject the null hypothesis for alpha = 0.05. 31
DIFFERENCE IN VARIANCE
HYPOTHESIS TEST
Can you detect this difference?
32
DIFFERENCE IN VARIANCE
HYPOTHESIS TEST -POWER CURVE
33
PAIRED DIFFERENCE IN MEANS
HYPOTHESIS TEST
• Two different inspectors each measure 10
parts on the same piece of test equipment.
• Null hypothesis: DIFFERENCE IN
MEANS = 0.0
• Alternative: not equal
• Computed t statistic = -1.22702
• P-Value = 0.250944
• Do not reject the null hypothesis for alpha =
0.05.
34
PAIRED DIFFERENCE IN MEANS
HYPOTHESIS TEST - POWER CURVE
Power Curve
alpha = 0.05, sigma = 3.866
1
Power
0.8
0.6
0.4
0.2
0
-5
-4
-3
-2
-1
0
1
2
3
4
5
Difference in Means
35
ONE WAY ANALYSIS OF VARIANCE
• Used to test hypothesis that the means of
several populations are equal.
• Example: Production line has 7 fill needles
and you wish to assess whether or not the
average fill is the same for all 7 needles.
• Experiment: sample 20 fills from each of
the 9 needles and test at 5% level of sign.
• Ho: m1 = m2 = m3= m4 = m5 = m6 = m7
36
RESULTS: ANALYSIS OF VARIANCE
TABLE
Analysis of Variance
---------------------------------------------------------------------------Source
Sum of Squares
Df Mean Square
F-Ratio
P-Valu
---------------------------------------------------------------------------Between groups
1.10019
6
0.183364
18.66
0.000
Within groups
1.30717
133
0.00982837
---------------------------------------------------------------------------Total (Corr.)
2.40736
139
37
SINCE NEEDLE MEANS ARE NOT ALL
EQUAL, WHICH ONES ARE DIFFERENT?
• Multiple Range Tests for 7 Needles
Method: 95.0 percent LSD
Col_2
Count
Mean
Homogeneous Groups
-------------------------------------------------------------------------------N7
20
11.786
X
N2
20
11.9811
X
N1
20
11.9827
X
N6
20
11.9873
X
N3
20
11.9951
X
N5
20
11.9953
X
N4
20
12.11
X
38
VISUAL COMPARISON OF 7
NEEDLES
Box-and-Whisker Plot
N1
Col_2
N2
N3
N4
N5
N6
N7
11.5
11.7
11.9
12.1
12.3
Col_1
39
FACTORIAL (2k) DESIGNS
• Experiments involving several factors ( k =
# of factors) where it is necessary to study
the joint effect of these factors on a specific
response.
• Each of the factors are set at two levels (a
“low” level and a “high” level) which may
be qualitative (machine A/machine B, fan
on/fan off) or quantitative (temperature
800/temperature 900, line speed 4000 per
hour/line speed 5000 per hour).
40
FACTORIAL (2k) DESIGNS
• Factors are assumed to be fixed (fixed
effects model)
• Designs are completely randomized
(experimental trials are run in a random
order, etc.)
• The usual normality assumptions are
satisfied.
41
FACTORIAL (2k) DESIGNS
• Particularly useful in the early stages of
experimental work when you are likely to
have many factors being investigated and
you want to minimize the number of
treatment combinations (sample size) but, at
the same time, study all k factors in a
complete factorial arrangement (the
experiment collects data at all possible
combinations of factor levels).
42
FACTORIAL (2k) DESIGNS
• As k gets large, the sample size will
increase exponentially. If experiment is
replicated, the # runs again increases.
k
2
3
4
5
6
7
8
9
10
# of runs
4
8
16
32
64
128
256
512
1024
43
FACTORIAL (2k) DESIGNS (k = 2)
• Two factors set at two levels (normally
referred to as low and high) would result in
the following design where each level of
factor A is paired with each level of factor
B.
Generalized Settings
RUN Factor A Factor B RESPONSE
Orthogonal Settings
RUN Factor A Factor B RESPONSE
1
low
low
y1
1
-1
-1
y1
2
high
low
y2
2
+1
-1
y2
3
low
high
y3
3
-1
+1
y3
4
high
high
y4
4
+1
+1
y4
44
FACTORIAL (2k) DESIGNS (k = 2)
• Estimating main effects associated with
changing the level of each factor from low
to high. This is the estimated effect on the
response variable associated with changing
factor A or B from their low to high values.
( y 2  y 4 ) ( y1  y3 )
Factor A Effect =

2
2
( y3  y4 ) ( y1  y2 )
Factor B Effect =

2
2
45
FACTORIAL (2k) DESIGNS (k = 2):
GRAPHICAL OUTPUT
• Neither factor A nor Factor B have an effect
on the response variable.
46
FACTORIAL (2k) DESIGNS (k = 2):
GRAPHICAL OUTPUT
• Factor A has an effect on the response
variable, but Factor B does not.
47
FACTORIAL (2k) DESIGNS (k = 2):
GRAPHICAL OUTPUT
• Factor A and Factor B have an effect on the
response variable.
48
FACTORIAL (2k) DESIGNS (k = 2):
GRAPHICAL OUTPUT
• Factor B has an effect on the response variable, but only if
factor A is set at the “High” level. This is called
interaction and it basically means that the effect one factor
has on a response is dependent on the level you set other
factors at. Interactions can be major problems in a DOE if
you fail to account for the interaction when designing your
experiment.
49
EXAMPLE:
FACTORIAL (2k) DESIGNS (k = 2)
• A microbiologist is interested in the effect
of two different culture mediums [medium 1
(low) and medium 2 (high)] and two
different times [10 hours (low) and 20 hours
(high)] on the growth rate of a particular
CFU.
50
EXAMPLE:
FACTORIAL (2k) DESIGNS (k = 2)
• Since two factors are of interest, k =2, and
we would need the following four runs
resulting in
Generalized Settings
RUN Medium
Time
Growth Rate
1
low
low
17
2
high
low
15
3
low
high
38
4
high
high
39
51
EXAMPLE:
FACTORIAL (2k) DESIGNS (k = 2)
• Estimates for the medium and time
effects are
• Medium effect = [(15+39)/2] – [(17 + 38)/2]
= -0.5
• Time effect = [(38+39)/2] – [(17 + 15)/2] =
22.5
52
EXAMPLE:
FACTORIAL (2k) DESIGNS (k = 2)
53
EXAMPLE:
FACTORIAL (2k) DESIGNS (k = 2)
• A statistical analysis using the appropriate
statistical model would result in the
following information. Factor A (medium)
and Factor B (time)
Type III Su ms of S quares
---- ------- ------- ------- ------ ------- ------- ------- ------- ------- ------- ------ ---Sour ce
Sum of Squa res
Df
M ean Squ are
F-Ratio
P- Valu
---- ------- ------- ------- ------ ------- ------- ------- ------- ------- ------- ------ ---FACT OR A
0 .25
1
0 .25
0.11
0 .795
FACT OR B
506 .25
1
506 .25
225.00
0 .042
Resi dual
2 .25
1
2 .25
---- ------- ------- ------- ------ ------- ------- ------- ------- ------- ------- ------ ---Tota l (corr ected)
508 .75
3
All F-ratio s are b ased on the r esidual mean s quare e rror.
54
EXAMPLE:
CONCLUSIONS
• In statistical language, one would conclude
that factor A (medium) is not statistically
significant at a 5% level of significance
since the p-value is greater than 5% (0.05),
but factor B (time) is statistically significant
at a 5 % level of significance since this pvalue is less than 5%.
55
EXAMPLE:
CONCLUSIONS
• In layman terms, this means that we have no
evidence that would allow us to conclude
that the medium used has an effect on the
growth rate, although it may well have an
effect (our conclusion was incorrect).
56
EXAMPLE:
CONCLUSIONS
• Additionally, we have evidence that would
allow us to conclude that time does have an
effect on the growth rate, although it may
well not have an effect (our conclusion was
incorrect).
57
EXAMPLE:
CONCLUSIONS
• In general we control the likelihood of
reaching these incorrect conclusions by the
selection of the level of significance for the
test and the amount of data collected
(sample size).
58
2k DESIGNS (k > 2)
• As the number of factors increase, the
number of runs needed to complete a
complete factorial experiment will increase
dramatically. The following 2k design
layout depict the number of runs needed for
values of k from 2 to 5. For example, when
k = 5, it will take 32 experimental runs for
the complete factorial experiment.
59
2k DESIGNS (k > 2)
k
2 Design Layouts (k = 2-5)
RUNS
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
k=5
k=4
k=3
k=2
-1
-1
-1
+1
-1
-1
-1
+1
-1
+1
+1
-1
-1
-1
+1
+1
-1
+1
-1
+1
+1
+1
+1
+1
-1
-1
-1
+1
-1
-1
-1
+1
-1
+1
+1
-1
-1
-1
+1
+1
-1
+1
-1
+1
+1
+1
+1
+1
-1
-1
-1
+1
-1
-1
-1
+1
-1
+1
+1
-1
-1
-1
+1
+1
-1
+1
-1
+1
+1
+1
+1
+1
-1
-1
-1
+1
-1
-1
-1
+1
-1
+1
+1
-1
-1
-1
+1
+1
-1
+1
-1
+1
+1
+1
+1
+1
-1
-1
-1
-1
-1
-1
-1
-1
+1
+1
+1
+1
+1
+1
+1
+1
-1
-1
-1
-1
-1
-1
-1
-1
+1
+1
+1
+1
+1
+1
+1
+1
-1
-1
-1
-1
-1
-1
-1
-1
-1
-1
-1
-1
-1
-1
-1
-1
+1
+1
+1
+1
+1
+1
+1
+1
+1
+1
+1
+1
+1
+1
+1
+1
60
Interactions for 2k Designs (k = 3)
•
Interactions between various factors can
be estimated for different designs above
by multiplying the appropriate columns
together and then subtracting the average
response for the lows from the average
response for the highs.
61
Interactions for 2k Designs (k = 3)
a
-1
+1
-1
+1
-1
+1
-1
+1
b
-1
-1
+1
+1
-1
-1
+1
+1
c
-1
-1
-1
-1
+!
+1
+1
+1
ab
1
-1
-1
1
1
-1
-1
1
ac
1
-1
1
-1
-1
1
-1
1
bc
1
1
-1
-1
-1
-1
1
1
abc
-1
1
1
-1
1
-1
-1
1
62
2k DESIGNS (k > 2)
• Once the effect for all factors and
interactions are determined, you are able to
develop a prediction model to estimate the
response for specific values of the factors.
In general, we will do this with statistical
software, but for these designs, you can do
it by hand calculations if you wish.
63
2k DESIGNS (k > 2)
• For example, if there are no significant interactions
present, you can estimate a response by the
following formula. (for quantitative factors only)
Y = (average of all responses) +
=Y (

factorEFFECT
[(
) * ( factorLEVEL)]
2
A

)* A ( B )*B
2
2
64
ONE FACTOR EXAMPLE
• Simple “one factor” example where the
factor is the number of hours a student
studies for an exam (LOW = 10 HRS,
HIGH = 20 HRS) and the response variable
is their grade. Estimate the model for
prediction a students grade as a function of
the number of hours they study.
65
ONE FACTOR EXAMPLE
Plot of Fitted Model
GRADE
95
85
75
65
55
10
12
14
16
#HRS STUDY
18
20
66
ONE FACTOR EXAMPLE
• The output shows the results of fitting a
general linear model to describe the
relationship between GRADE and #HRS
STUDY. The equation of the fitted general
model is
• GRADE = 29.3 + 3.1* (#HRS STUDY)
• The fitted orthogonal model is
• GRADE = 75 + 15 * (SCALED # HRS)
67
Two Level Screening Designs
• Suppose that your brainstorming session
resulted in 7 factors that various people
think “might” have an effect on a response.
A full factorial design would require 27 =
128 experimental runs without replication.
The purpose of screening designs is to
reduce (identify) the number of factors
down to the “major” role players with a
minimal number of experimental runs. One
way to do this is to use the 23 full factorial
design and use interaction columns for
factors.
68
Note that
* Any factor d effect is now confounded with the a*b
interaction
* Any factor e effect is now confounded with the a*c
interaction
* etc.
* What is the d*e interaction confounded with????????
a
-1
+1
-1
+1
-1
+1
-1
+1
b
-1
-1
+1
+1
-1
-1
+1
+1
c
-1
-1
-1
-1
+!
+1
+1
+1
d = ab
1
-1
-1
1
1
-1
-1
1
e = ac
1
-1
1
-1
-1
1
-1
1
f = bc
1
1
-1
-1
-1
-1
1
1
g = abc
-1
1
1
-1
1
-1
-1
1
69
Problems that Interactions Cause!
•
Interactions – If interactions exist and you fail to
account for this, you may reach erroneous
conclusions. Suppose that you plan an
experiment with four runs and three factors
resulting in the following data:
70
Problems that Interactions Cause!
•
•
•
•
Factor A Effect = 0
Factor B Effect = 0
Factor C Effect = 5
In this example, if you were assuming that
“larger is better” then you would set Factor
C at the “high level” and it appears to make
no difference where you set factors A and B.
In this case there is a factor A interaction
with factor B and this interaction is
confounded with the factor C effect.
71
Problems that Interactions Cause!
Interaction Plot
RESPONSE
10
FACT OR B
-1
1
9
8
7
6
5
-1
1
FACTOR A
72
Resolution of a Design
• The above design would be called a
resolution III design because main effects
are aliased (confounded) with two factor
interactions.
73
Resolution of a Design
•
•
•
Resolution III Designs – No main effects are
aliased with any other main effect BUT some (or
all) main effects are aliased with two way
interactions
Resolution IV Designs – No main effects are
aliased with any other main effect OR two factor
interaction, BUT two factor interactions may be
aliased with other two factor interactions
Resolution V Designs – No main effect OR two
factor interaction is aliased with any other main
effect or two factor interaction, BUT two factor
interactions are aliased with three factor
interactions.
74
Common Screening Designs
•
Fractional Factorial Designs – the total
number of experimental runs must be a
power of 2 (4, 8, 16, 32, 64, …). If you
believe first order interactions are small
compared to main effects, then you could
choose a resolution III design. Just
remember that if you have major
interactions, it can mess up your screening
experiment.
75
Common Screening Designs
•
Plackett-Burman Designs – Two level,
resolution III designs used to study up to
n-1 factors in n experimental runs, where
n is a multiple of 4 ( # of runs will be 4, 8,
12, 16, …). Since n may be quite large,
you can study a large number of factors
with moderately small sample sizes. (n =
100 means you can study 99 factors with
100 runs)
76
Other Design Issues
• May want to collect data at center points to
estimate non-linear responses
• More than two levels of a factor – no
problem (multi-level factorial)
• What do you do if you want to build a nonlinear model to “optimize” the response. (hit
a target, maximize, or minimize) – called
response surface modeling
77
Other Design Issues
• What do you do if the factors levels are
categorical and not quantitative, or some are
categorical and some are quantitative?
• What do you do if the structure of you
experiment is “nested”? These are called
heirarchical designs and will allow you to
partition the total variability among the
different levels of the design (called
variance components)
78
Response Surface Designs – Box-Behnken:
After screening designs identify major factors –Next step.
•
•
•
•
•
•
•
•
Design class: Response Surface
Design name: Box-Behnken design
Base Design
----------Number of experimental factors: 3 Number of blocks: 1
Number of responses: 1
Number of runs: 15
Error degrees of freedom: 5
Randomized: No
•
•
•
•
•
Factors
Low
High
Units
Continuous
-----------------------------------------------------------------------Factor_A
-1.0
1.0
Yes
Factor_B
-1.0
1.0
Yes
Factor_C
-1.0
1.0
Yes
79
Response Surface Designs – Box-Behnken
FACTOR A
FACTOR B
FACTOR C
0
0
0
-1
-1
0
1
-1
0
-1
1
0
1
1
0
-1
0
-1
1
0
-1
0
0
0
-1
0
1
1
0
1
0
-1
-1
0
1
-1
0
-1
1
0
1
1
0
0
0
80
Response Surface Designs – Central
Composite
•
•
•
•
•
•
Design class: Response Surface
Design name: Central composite blocked cube-star
Number of experimental factors: 3 Number of blocks: 2
Number of responses: 1
Number of runs: 16
Error degrees of freedom: 5
Randomized: No
• Factors
Low
High
Units
Continuous
• ----------------------------------------------------------------------• Factor_A
-1.0
1.0
Yes
• Factor_B
-1.0
1.0
Yes
• Factor_C
-1.0
1.0
Yes
81
Response Surface Designs – Central
Composite
FACTOR A
FACTOR B
FACTOR C
-1
-1
-1
1
-1
-1
-1
1
-1
1
1
-1
0
0
0
-1
-1
1
1
-1
1
-1
1
1
1
1
1
-1.76383
0
0
1.76383
0
0
0
-1.76383
0
0
0
0
0
1.76383
0
0
0
-1.76383
0
0
1.76383
82
Multilevel Factorial Designs
•
•
•
•
•
Design class: Multilevel Factorial
Number of experimental factors: 3 Number of blocks: 1
Number of responses: 1
Number of runs: 27
Error degrees of freedom: 17
Randomized: No
• Factors
Low
High
Levels Units
• -----------------------------------------------------------------------• Factor_A
-1.0
1.0
3
• Factor_B
-1.0
1.0
3
• Factor_C
-1.0
1.0
3
83
Multilevel Factorial Designs
FACTOR A FACTOR B FACTOR C
-1
-1
-1
0
-1
-1
1
-1
-1
-1
0
-1
0
0
-1
1
0
-1
-1
1
-1
0
1
-1
1
1
-1
-1
-1
0
0
-1
0
1
-1
0
-1
0
0
0
0
0
1
0
0
-1
1
0
0
1
0
1
1
0
-1
-1
1
0
-1
1
1
-1
1
-1
0
1
0
0
1
1
0
1
-1
1
1
0
1
1
1
1
1
84
Nested Design
• Design class: Variance Components
• Number of experimental factors: 3
• Number of responses: 1
• Number of runs: 27
• Randomized: No
Factors
Levels Units
• ----------------------------------------------• Factor_A
3
• Factor_B
3
• Factor_C
3
You have created a variance components design which will
estimate the contribution of 3 factors to overall process
variability. The design is hierarchical, with each factor
nested in the factor above it. A total of 27 measurements
are required.
85
Nested Design
FACTOR A FACTOR B FACTOR C
1
1
1
1
1
2
1
1
3
1
2
1
1
2
2
1
2
3
1
3
1
1
3
2
1
3
3
2
1
1
2
1
2
2
1
3
2
2
1
2
2
2
2
2
3
2
3
1
2
3
2
2
3
3
3
1
1
3
1
2
3
1
3
3
2
1
3
2
2
3
2
3
3
3
1
3
3
2
3
3
3
86
Response Surface Designs – Box-Behnken
EXAMPLE - RECAP
•
•
•
•
•
•
•
•
Design class: Response Surface
Design name: Box-Behnken design
Base Design
----------Number of experimental factors: 3 Number of blocks: 1
Number of responses: 1
Number of runs: 15
Error degrees of freedom: 5
Randomized: No
•
•
•
•
•
Factors
Low
High
Units
Continuous
-----------------------------------------------------------------------Factor_A
10
30
Yes
Factor_B
30
60
Yes
Factor_C
40
60
Yes
87
88
Response Surface Designs – Box-Behnken
REAL MODEL: Y = 40F1+200F2+100F3-10F1F2+9F1F1 + (NOISE)
RUN
F1
F2
F3
1
10
45
60
2
30
45
40
3
20
30
40
4
10
30
50
5
20
45
50
6
30
60
50
7
20
45
50
8
30
45
60
9
20
45
50
10
20
60
40
11
10
45
40
12
30
30
50
13
20
60
60
14
10
60
50
15
20
30
60
89
Response Surface Designs – Box-Behnken
REAL MODEL: Y = 40F1+200F2+100F3-10F1F2+9F1F1 + (NOISE)
Example: std. dev. of noise = 0
RUN
F1
F2
F3
Y0
1
10
45
60
11800
2
30
45
40
8800
3
20
30
40
8400
4
10
30
50
9300
5
20
45
50
9400
6
30
60
50
8300
7
20
45
50
9400
8
30
45
60
10800
9
20
45
50
9400
10
20
60
40
8400
11
10
45
40
9800
12
30
30
50
11300
13
20
60
60
10400
14
10
60
50
12300
15
20
30
60
10400
90
Response Surface Designs – BoxBehnken
REAL MODEL: Y = 40F1+200F2+100F3-10F1F2+9F1F1 + (NOISE)
Example: std. dev. of noise = 0
Regression coeffs. for Var_1
------------------------------------------------------------constant
= 0.0
A:Factor_A = 40.0
B:Factor_B = 200.0
C:Factor_C = 100.0
AA
= 9.0
AB
= -10.0
AC
= 0.0
BB
= 0.0
BC
= 0.0
CC
= 0.0
91
Response Surface Designs – Box-Behnken
REAL MODEL: Y = 40F1+200F2+100F3-10F1F2+9F1F1 + (NOISE)
Example: std. dev. of noise = 0
Optimize Response
----------------Goal: MAXIMIZE Var_1
Optimum value = 13261.4
Factor
Low
High
Optimum
----------------------------------------------------------------------Factor_A
10.0
30.0
10.1017
Factor_B
30.0
60.0
60.0
Factor_C
40.0
60.0
60.0
92
Response Surface Designs – Box-Behnken
REAL MODEL: Y = 40F1+200F2+100F3-10F1F2+9F1F1 + (NOISE)
Example: std. dev. of noise = 0
93
Response Surface Designs – Box-Behnken
REAL MODEL: Y = 40F1+200F2+100F3-10F1F2+9F1F1 + (NOISE)
Example: std. dev. of noise = 100
RUN
F1
F2
F3
Y100
1
10
45
60
11825
2
30
45
40
8781
3
20
30
40
8413
4
10
30
50
9216
5
20
45
50
9288
6
30
60
50
8261
7
20
45
50
9329
8
30
45
60
10855
9
20
45
50
9205
10
20
60
40
8538
11
10
45
40
9718
12
30
30
50
11308
13
20
60
60
10316
14
10
60
50
12056
15
20
30
60
10378
94
Response Surface Designs – Box-Behnken
REAL MODEL: Y = 40F1+200F2+100F3-10F1F2+9F1F1 + (NOISE)
Example: std. dev. of noise = 100
Regression coeffs. for Var_3
---------------------------------------------------------------------constant
= 2312.5
A:Factor_A = 36.575
B:Factor_B = 200.067
C:Factor_C = 3.85
AA
= 9.09875
AB
= -9.81167
AC
= -0.0825
BB
= 0.117222
BC
= -0.311667
CC
= 1.10875
95
Response Surface Designs – Box-Behnken
REAL MODEL: Y = 40F1+200F2+100F3-10F1F2+9F1F1 + (NOISE)
Example: std. dev. of noise = 100
Standardized Pareto Chart for Var_3
AB
C:Factor_C
AA
A:Factor_A
CC
BC
B:Factor_B
BB
AC
0
10
20
30
40
Standardized effect
96
Response Surface Designs – Box-Behnken
REAL MODEL: Y = 40F1+200F2+100F3-10F1F2+9F1F1 + (NOISE)
Example: std. dev. of noise = 100
Contours of Estimated Response Surface
Factor_C=60.0
60
Factor_B
55
50
45
40
35
30
10
14
18
22
Factor_A
26
30
Var_3
9300.0
9500.0
9700.0
9900.0
10100.0
10300.0
10500.0
10700.0
10900.0
11100.0
11300.0
11500.0
11700.0
97
Response Surface Designs – Box-Behnken
REAL MODEL: Y = 40F1+200F2+100F3-10F1F2+9F1F1 + (NOISE)
Example: std. dev. of noise = 100
• Optimize Response
• ----------------• Goal: maximize Y
• Optimum value = 13139.4
•
•
•
•
•
Factor
Low
High
Optimum
----------------------------------------------------------------------Factor_A
10.0
30.0
10.1036
Factor_B
30.0
60.0
60.0
Factor_C
40.0
60.0
60.0
98
Response Surface Designs – Three Level Factorial Design (33)
REAL MODEL: Y = 40F1+200F2+100F3-10F1F2+9F1F1 + (NOISE)
Example: std. dev. of noise = 100
RUN
F1
F2
F3
Y100
1
10
30
40
8270
2
20
30
40
8272
3
30
30
40
10324
4
10
45
40
9928
5
20
45
40
8520
6
30
45
40
8973
7
10
60
40
11082
8
20
60
40
8377
9
30
60
40
7410
10
10
30
50
9191
11
20
30
50
9331
12
30
30
50
11131
13
10
45
50
10615
99
Response Surface Designs – Three Level Factorial Design (33)
REAL MODEL: Y = 40F1+200F2+100F3-10F1F2+9F1F1 + (NOISE)
Example: std. dev. of noise = 100
RUN
F1
F2
F3
Y100
14
20
45
50
9302
15
30
45
50
9723
16
10
60
50
12088
17
20
60
50
9343
18
30
60
50
8260
19
10
30
60
10313
20
20
30
60
10363
21
30
30
60
12267
22
10
45
60
11763
23
20
45
60
10534
24
30
45
60
10791
25
10
60
60
13281
26
20
60
60
10349
27
30
60
60
9497
100
Response Surface Designs – Three Level Factorial Design (33)
REAL MODEL: Y = 40F1+200F2+100F3-10F1F2+9F1F1 + (NOISE)
Example: std. dev. of noise = 100
Regression coeffs. for Y
---------------------------------------------------------------------constant
= 2887.08
A:Factor_A = 36.7028
B:Factor_B = 212.806
C:Factor_C = -31.0306
AA
= 8.95833
AB
= -9.57333
AC
= -0.190833
BB
= -0.283704
BC
= 0.100556
CC
= 1.30333
101
Response Surface Designs – Three Level Factorial Design (33)
REAL MODEL: Y = 40F1+200F2+100F3-10F1F2+9F1F1 + (NOISE)
Example: std. dev. of noise = 100
• Optimize Response
• ----------------• Goal: maximize Y
• Optimum value = 13230.6
•
•
•
•
•
Factor
Low
High
Optimum
----------------------------------------------------------------------Factor_A
10.0
30.0
10.0
Factor_B
30.0
60.0
60.0
Factor_C
40.0
60.0
60.0
102
Response Surface Designs – Three Level Factorial Design (33)
REAL MODEL: Y = 40F1+200F2+100F3-10F1F2+9F1F1 + (NOISE)
Example: std. dev. of noise = 100
Estimated Response Surface
Factor_C=60.0
13400
Y
12400
11400
10400
9400
10
14
18
Factor_A
22
26
30
30
35 40
55 60
50
45
Factor_B
103
Response Surface Designs – Three Level Factorial Design (33)
REAL MODEL: Y = 40F1+200F2+100F3-10F1F2+9F1F1 + (NOISE)
Example: std. dev. of noise = 100
Contours of Estimated Response Surface
Factor_B=60.0
60
Y
Factor_C
56
52
48
44
40
10
14
18
22
Factor_A
26
30
8200.0
8600.0
9000.0
9400.0
9800.0
10200.0
10600.0
11000.0
11400.0
11800.0
12200.0
12600.0
104
CLASSROOM EXERCISE
•
•
•
•
STUDENT IN-CLASS EXPERIMENT:
Collect data for experiment to determine
factor settings (two factors) to hit a target
response (spot on wall).
Factor A – height of shaker (low and high)
Factor B – location of shaker (close to
hand and close to wall)
Design experiment – would suggest
several replications
105
CLASSROOM EXERCISE
• Conduct Experiment – student holds 3 foot
“pin the tail on the donkey” stick and
attempts to hit the target. An observer will
assist to mark the hit on the target.
• Collect data – students take data home for
week and come back with what you would
recommend AND why.
• YOU TELL THE CLASS HOW TO PLAY
THE GAME TO “WIN”.
106
CLASSROOM EXERCISE
107
CLASSROOM EXERCISE
MARKER VERTICAL
1ST OBS 2ND OBS 3RD OBS 4TH OBS
STICK
POLE
MEAN
STANDARD
DEVIATION
L
L
-2.750
-4.500
-4.750
-5.000
-4.250
1.021
H
L
-12.500
-6.750
-4.625
-4.000
-6.969
3.871
L
H
3.000
3.250
3.875
6.250
4.094
1.484
H
H
4.625
11.250
12.625
14.000
10.625
4.155
MARKER
STICK
L = VERTICAL POLE WAS CLOSE TO WALL (MARKER END OF STICK
H=VERTICAL POLE WAS CLOSE TO HAND
VERTICAL
POLE
L=SHAKING DEVICE LOCATED LOW ON VERTICAL POLE
H=SHAKING DEVICE LOCATED HIGH ON VERTICAL POLE
108
CLASSROOM EXERCISE
• HOMEWORK:
• .Determine the effects “marker stick” and “vertical
pole” have on the mean location of the hit.
• .Determine the effects “marker stick” and “vertical
pole” have on the standard deviation of the hit.
• .Which factor would you say affects the mean location
of the “hit”?
• .Which factor would you say affects the standard
deviation of the “hit”?
• OPTIMAL SETTINGS: Where would you
recommend we locate the “vertical pole” and the
“marker stick” IF we wish to (a) MINIMIZE THE
VARIABILITY OF THE HIT and (b) HIT THE
109
TARGET LOCATED AT “0”?
PIN THE TAIL DATA INPUT
110
ESTIMATE OF EFFECTS (MEAN HIT)
•
•
•
•
•
•
•
•
Estimated effects for MEAN
---------------------------------------------------------------------average
= 0.875
A:MARKER STICK = 1.906
B:VERTICAL POLE = 12.969
AB
= 4.625
---------------------------------------------------------------------No degrees of freedom left to estimate standard errors.
111
ESTIMATE OF EFFECTS (MEAN HIT)
Pareto Chart for MEAN
B:VERTICAL POLE
AB
A:MARKER STICK
0
3
6
9
12
15
Effect
112
ESTIMATE OF EFFECTS (MEAN HIT)
Main Effects Plot for MEAN
9
MEAN
6
3
0
-3
-6
-1.0
1.0
MARKER STICK
-1.0
1.0
VERTICAL POLE
113
INTERACTION PLOT (MEAN HIT)
Interaction Plot for MEAN
11
VERTICAL POLE=1.0
MEAN
8
5
VERTICAL POLE=1.0
2
-1
-4
VERTICAL POLE=-1.0
VERTICAL POLE=-1.0
-7
-1.0
1.0
MARKER STICK
114
3-D PLOT OF RESPONSE (MEAN HIT)
MEAN
Estimated Response Surface
11
8
5
2
-1
-4
-7
-1
-0.6
-0.2
0.2
0.6
1
-1
-0.6 -0.2
0.2
0.6
1
VERTICAL POLE
MARKER STICK
115
CONTOUR PLOT OF RESPONSE
(MEAN HIT)
VERTICAL POLE
Contours of Estimated Response Surface
1
MEAN
-4.0
-2.0
0.0
2.0
4.0
6.0
8.0
0.6
0.2
-0.2
-0.6
-1
-1
-0.6
-0.2
0.2
0.6
1
MARKER STICK
116
ANALYSIS OF VARIANCE TABLE
(MEAN HIT)
•
•
•
•
•
•
•
•
Analysis of Variance for MEAN
-------------------------------------------------------------------------------Source
Sum of Squares Df Mean Square F-Ratio P-Value
-------------------------------------------------------------------------------A:MARKER STICK
3.63284 1
3.63284
0.17 0.7511
B:VERTICAL POLE
168.195 1
168.195
7.86 0.2181
Total error
21.3906 1
21.3906
--------------------------------------------------------------------------------
117
ESTIMATED LINEAR RESPONSE MODEL
(MEAN HIT)
•
•
•
•
•
•
Regression coeffs. for MEAN
---------------------------------------------------------------------constant
= 0.875
A:MARKER STICK = 0.953
B:VERTICAL POLE = 6.4845
----------------------------------------------------------------------
• The StatAdvisor
• --------------•
This pane displays the regression equation which has been fitted to
• the data. The equation of the fitted model is
• MEAN = 0.875 + 0.953*MARKER STICK + 6.4845*VERTICAL POLE
118
OPTIMAL FACTOR SETTINGS
(MEAN HIT)
• Optimize Response
• ----------------• Goal: maintain MEAN at 0.0
• Optimum value = 0.0
•
•
•
•
Factor
Low
High
Optimum
----------------------------------------------------------------------MARKER STICK
-1.0
1.0
0.03311
VERTICAL POLE
-1.0
1.0
-0.139803
119
ESTIMATE OF EFFECTS (STD DEV HIT)
• Estimated effects for STD DEV
• --------------------------------------------------------------------• average
= 2.63275
• A:MARKER STICK = 2.7605
• B:VERTICAL POLE = 0.3735
• AB
= -0.0895
120
ESTIMATE OF EFFECTS (STD DEV HIT)
•
•
•
•
•
•
•
•
•
Analysis of Variance for STD DEV
-------------------------------------------------------------------------------Source
Sum of Squares Df Mean Square F-Ratio P-Value
-------------------------------------------------------------------------------A:MARKER STICK 7.62036 1
7.62036 951.33 0.0206
B:VERTICAL POLE 0.139502 1
0.139502 17.42 0.1497
Total error
0.00801025 1 0.00801025
-------------------------------------------------------------------------------Total (corr.)
7.76787 3
121
OPTIMAL FACTOR SETTINGS
(STD DEV HIT)
• Optimize Response
• ----------------• Goal: minimize STD DEV
• Optimum value = 1.06575
•
•
•
•
Factor
Low
High
Optimum
----------------------------------------------------------------------MARKER STICK
-1.0
1.0
-1.0
VERTICAL POLE
-1.0
1.0
-1.0
122
INTERACTION
(STD DEV HIT)
Interaction Plot for STD DEV
5
VERTICAL POLE=1.0
VERTICAL POLE=-1.0
STD DEV
4
3
2
1
VERTICAL POLE=1.0
VERTICAL POLE=-1.0
0
-1.0
1.0
MARKER STICK
123
CONTOUR PLOT OF RESPONSE
(STD DEV HIT)
VERTICAL POLE
Contours of Estimated Response Surface
1
STD DEV
1.0
1.5
2.0
2.5
3.0
3.5
4.0
4.5
0.6
0.2
-0.2
-0.6
-1
-1
-0.6
-0.2
0.2
0.6
1
MARKER STICK
124
SO, WHAT’S THE ANSWER?
• I WOULD:
• 1. SET THE “MARKER STICK” AT LOW
(CLOSE TO THE WALL)
• 2. SET THE “VERTICAL POLE” AT A
VALUE THAT WILL HIT THE TARGET.
125
SO, WHAT’S THE ANSWER?
• FROM REGRESSION FOR “MEAN HIT”,
SET MARKER STICK AT “-1”, HIT AT
“0”, AND SOLVE FOR VP
• HIT = .0875 +.953*MS + 6.4845*VP
•
0 = .875 + .953*(-1) + 6.4845*VP
• Resulting in
•
VP = .012 and MS = -1
126
Contour Plots for Mean and Std. Dev.
127