Transcript Lecture 10

Bayesian Model Selection in
Factorial Designs
Seminal work is by Box and Meyer
 Intuitive formulation and analytical
approach, but the devil is in the details!
 Look at simplifying assumptions as we
step through Box and Meyer’s approach
 One of the hottest areas in statistics for
several years

Bayesian Model Selection in
Factorial Designs
There are 2k-p-1=n-1 possible (fractional)
factorial models, denoted as a set {Ml}.
 To simplify later calculations, we usually
assume that the only active effects are
main effects, two-way effects or three-way
effects

– This assumption is already in place for lowresolution fractional factorials
Bayesian Model Selection in
Factorial Designs
Each Ml denotes a set of active effects
(both main effects and interactions) in a
hierarchical model.
 We will use Xik=1 for the high level of
effect k and Xik=-1 for the low level of
effect k.

Bayesian Model Selection in
Factorial Designs

We will assume that the response
variables have a linear model with normal
errors given model M

Xi and b are model-specific, but we will
use a saturated model in what follows
Bayesian Model Selection in
Factorial Designs

The likelihood for the data given the
parameters has the following form
2n
æ 1
2ö
L(b, s ,Y ) = Õ
exp ç - 2 (Yi - Xi¢b ) ÷
2
è 2s
ø
2ps
i=1
æ 1
ö
1
¢
=
exp ç - 2 (Y - X b ) (Y - X b ) ÷
n
2 2
è
ø
2
s
2ps
(
)
1
Bayesian Paradigm
Unlike in classical inference, we assume
the parameters, Q, are random variables
that have a prior distribution, fQ(q), rather
than being fixed unknown constants.
 In classical inference, we estimate q by
maximizing the likelihood L(q|y)

Bayesian Paradigm

Estimation using the Bayesian approach
relies on updating our prior distribution for
Q after collecting our data y. The posterior
density, by an application of Bayes rule, is
proportional to the product of the familiar
data density and the prior density:
fQ|Y (q | y ) µ f X |Q (x | q ) fQ (q )
Bayesian Paradigm
The Bayes estimate of Q minimizes Bayes
risk—the expected value (with respect to
the prior) of loss function L(q).
 Under squared error loss, the Bayes
estimate is the mean of the posterior
distribution:

qˆ ( y ) = EQ|Y Q
Bayesian Model Selection in
Factorial Designs

The Bayesian prior for models is quite
straightforward. The prior probability that r
effects are active in the model, given each
is active with prior probability p, is
L(p ) = C1p (1- p )
r
n-1-r
= C1 (1- p )
n-1
r
æ p ö
ç
÷
è 1- p ø
Bayesian Model Selection in
Factorial Designs

Since we’re using a Bayesian approach,
we need priors for b and s as well
b0 ~ N (0, s e ), e = 10
2
b j ~ N (0, g s
2
2
)
s ~ g(s ), g(s ) µ s
-a
-6
Bayesian Model Selection in
Factorial Designs

For non-orthogonal designs, it’s common
to use Zellner’s g-prior for b:
(
b j ~ N 0, g s ( X' X )

2
2
-1
)
Note that we did not assign priors to g or
p
Bayesian Model Selection in
Factorial Designs

We can combine f(b,s,M) and f(Y|b,s,M)
to obtain the full likelihood L(b,s,M,Y)
æ1ö æ 1 ö
÷
ç ÷ ç
è g ø è 2ps 2 ø
æ 1
ö
exp ç - 2 Q(b )÷
è 2s
ø
r
æ p ö
L(b, s , M,Y ) = C ç
÷
è 1- p ø
n-1
2n-1+a
2
´
Bayesian Model Selection in
Factorial Designs
Bayesian Model Selection in
Factorial Designs

Our goal is to derive the posterior
distribution of M given Y, which first
requires integrating out b and s.
L(M | Y ) µ L(M,Y ) =
¥
ò ò L(b, s , M,Y )d b ds =
0 Rn
r
æ p ö
Cç
÷
è 1- p ø
( X ¢X + G)
n-1
æ1ö
ç ÷
èg ø
(
-1 2
Q ( X ¢X + G) X ¢Y
-1
)
(n-1+a) 2
Bayesian Model Selection in
Factorial Designs
The first term is a penalty for model
complexity (smaller is better)
 The second term is a measure of model fit
(smaller is better)

Bayesian Model Selection in
Factorial Designs
p and g are still present. We will fix p; the
method is robust to the choice of p
 g is selected to minimize the probability of
no active factors

Bayesian Model Selection in
Factorial Designs
With L(M|Y) in hand, we can actually
evaluate the P(Mi|Y) for all Mi for any prior
choice of p, provided the number of Mi is
not burdensome
 This is in part why we assume eligible Mi
only include lower order effects.

Bayesian Model Selection in
Factorial Designs
Greedy search or MCMC algorithms are
used to select models when they cannot
be itemized
 Selection criteria include Bayes Factor,
Schwarz criterion, Bayesian Information
Criterion
 Refer to R package BMA and bic.glm for
fitting more general models.

Bayesian Model Selection in
Factorial Designs
For each effect, we sum the probabilities
for all Mi that contain that effect and obtain
a marginal posterior probability for that
effect.
 These marginal probabilities are relatively
robust to the choice of p.

Case Study
Violin data* (24 factorial design with n=11
replications)
 Response: Decibels
 Factors

–
–
–
–
A: Pressure (Low/High)
B: Placement (Near/Far)
C: Angle (Low/High)
D: Speed (Low/High)
*Carla Padgett, STAT 706 taught by Don Edwards
Case Study
Fractional Factorial
Design:
• A, B, and D significant
• AB marginal
Bayesian Model
Selection:
• A, B, D, AB, AD,
BD significant
• All others
negligible
*Carla Padgett, STAT 706 taught by Don Edwards