Transcript Econ 399 Chapter8b

8.4 Weighted Least Squares Estimation
Before the existence of heteroskedasticity-robust
statistics, one needed to know the form of
heteroskedasticity
-Het was then corrected using WEIGHTED
LEAST SQUARES (WLS)
-This method is still useful today, as if
heteroskedasticity can be correctly modeled, WLS
becomes more efficient than OLS
-ie: WLS becomes BLUE
8.4 Known Heteroskedasticity
-Assume first that the form of heteroskedasticity
is known and expressed as:
Var(u | X )   h( X )
2
-Where h(X) is some function of the independent
variables
-since variance must be positive, h(X)>0 for all
valid combinations of X
-given a random sample, we can write:
 i  Var(ui | X i )   hi
2
2
8.4 Known Het Example
-Assume that sanity is a function of econometrics
knowledge and other factors:
crazy   0  1econ  otherfactors  u
-However, by studying econometrics two things
happen: either one becomes more sane as one
understands the world, or one becomes more
crazy as one is pulled into a never-ending
vortex of causal relationships. Therefore:
Var (ui | X i )   econi
2
8.4 Known Heteroskedasticity
-Since h is a function of x, we know that:
E (ui / hi | X )  0 and
Var(u i | X )  E (u | X )   hi
2
i
2
-Therefore
E[(ui / hi | X ) ]  E (u ) | hi 
2
2
i
 hi
2
hi

2
-So inclusion of the h term in our model can solve
heteroskedasticity
8.4 Fixing Het – And Stay Down!
-We therefore have the modified equation:
yi
 0 1 xi1
 k xik ui


 ... 

hi
hi
hi
hi
hi
-Or alternately:
y   0   x  ...   k x  u
*
i
*
1 i1
*
ik
*
i
(8.26)
-Note that although our estimates for BJ will
change (and their standard errors become
valid), their interpretation is the same as the
straightforward OLS model (don’t try to bring h
into your interpretation)
8.4 Het Fixing – “I am the law”
-(8.26) is linear and satisfied MLR.1
-if the original sample was random, nothing
chances so MLR.2 is satisfied
-If no perfect collinearity existed before, MLR.3 is
still satisfied now
-E(ui*|Xi*)=0, so MLR.4 is satisfied
-Var(ui*|Xi*)=σ2, so MLR.5 is satisfied
-if ui has a normal distribution, so does ui*, so
MLR. 6 is satisfied
-Thus if the original model satisfies everything
but het, the new model satisfies MLR. 1 to 6
8.4 Het Fix – Control the Het Pop
-These BJ* estimates are different from typical
OLS estimates and are examples of
GENERALIZED LEAST SQUARES (GLS)
ESTIMATORS
-this GLS estimation provides standard errors, t
statistics and F statistics that are valid
-Since these estimates satisfy all 6 CLM
assumptions, and because they are BLUE, GLS is
always more efficient than OLS
-Note that OLS is a special case of GLS where
hi=1
8.4 Het Fix – Who broke it anyhow?
-Note that the R2 obtained from this regression is
useful for F statistics but is NOT useful for its
typical interpretation
-this is due to the fact that it explains how
much X* explains y*, not how much X explains y
-when GLS estimators are used to correct for
heteroskedasticity, they are called WEIGHTED
LEAST SQUARES (WLS) ESTIMATORS
-most econometric programs have commands to
minimize the weighted sum of squared residuals:
min  ( yi  0  1 xi1  ...   k xik )2 / hi
8.4 Incorrect Correcting?
What happens if h(x) is misspecified and WLS is
run (ie: if one expects x1 to cause het but x3
actually causes het)
1) WLS is still unbiased and consistent (similar to
OLS)
2) Standard Errors (thus t and F tests) are no
longer valid
-to avoid this, one can always apply a fully robust
inference for WLS (as we say for OLS in 8.2)
-this can be tedious
8.4 Incorrect Correcting?
WLS is often criticized as being better than OLS
ONLY IF the form of het is correctly chosen
-one may argue that making some correction for
het is better than none at all
-there is always the option of using robust WLS
estimation
-in cases of doubt, both robust WLS and robust
OLS results can be reported
8.4 Averages and Het
Heteroskedasticity will always exist when
AVERAGES are used
-when using averages, each observation is the
sum of all individual observations divided by
group size:
xi   x i / mi
-Therefore in our true regression, our error term
is the sum of all individual observations’ error
terms divided by group size:
ui   u i / mi
8.4 Averages and Het
If the individual model is homoskedastic, and no
correlation exists between groups, then the
average equation is heteroskedastic with a
weight of hi=1/mi
-In this way larger groups receive more weight in
the regression and is due to the fact that
Var (u i )   2 / mi
For example, assume that we run a regression on
how math knowledge impacts grades in econ
classes. Bigger classes (Econ 299) would be
weighted to give more information than
smaller classes (Econ 349.5 – Love and Econ.)
8.4 Feasible GLS
-In the previous section we assumed that we
knew the form of the heteroskedasticity, hi(x)
-often this is not the case an we need to use data
to estimate hihat
-this yields an estimator called FEASIBLE GLS
(FGLS) or ESTIMATED GLS (EGLS)
-Although h(x) can be measured many ways, we
assume that
2  0 1 x1 ...  k xk
Var (u | X )   e
 0 1 x1 ...  k xk
h( X )  e
(8.30)
8.4 Feasible GLS
-Note that while the BP test for Het assumed Het
was linear, here we allow for non-linear Het
-although testing for linear Het is effective,
correcting for Het has issues with linear models
as h(X) could be negative, making Var(u|X)
negative
-since delta is unknown, it must be estimated
-using (8.30),
2  0 1 x1 ...  k xk
u  e
2
v
-Where v, conditional on X, has a mean of unity
8.4 Feasible GLS
-If we assume v is independent of X,
log( u )   0  1 x1  ...   k xk  e
2
-Where e has zero mean and is independent of X
-note that the intercept changes, which is
unavoidable but not drastically important
-as usual, we only have residuals, not errors, so
we run the regression and obtain fitted values
lôg( uˆ )  ˆ 0  ˆ1 x1  ...  ˆk xk
2
-To obtain:
hˆi  e
2
lôg(u i )
To
1)
2)
3)
8.4 FGLS
use FGLS to correct for Heteroskedasticity,
Regress y on all x’s and obtain residuals uhat
Create log(uhat2)
Regress log(uhat2) on all x’s and obtain fitted
values ghat
4) Estimate hhat=exp(ghat)
5) Run WLS using weights 1/hhat
8.4 FGLS
If we used the actual h(X), our estimator would
be unbiased and BEST
-since h(X) is estimated using the same data as
FGLS, it is biased and therefore not BEST
-however, FGLS is consistent and asymptotically
more efficient than OLS
-therefore FGLS is a good alternative to OLS in
large samples
-note that FGLS estimates are interpreted the
same as OLS
-note also that heteroskedasticity-robust standard
errors can always be calculated in cases of doubt
8.4 FGLS Alternative
One alternative is to estimate ghat as:
2
ˆ
ˆ
lôg( uˆ )  ˆ 0  1 yˆ   2 yˆ
2
Using fitted y values from the OLS equation
-This changes step 3 above, but the remaining
steps are the same
-Note that the Park (1996) test is based on FGLS
but is inferior to our previous tests due to FGLS
only being consistent
8.4 F Tests and WLS
When conducting F tests using WLS,
1) First estimate the restricted and unrestricted
model using OLS
2) After determining weights, use these weights
on both the restricted and unrestricted model
3) Conduct F tests
Luckily most econometric programs have
commands for joint tests
8.4 WLS vs. OLS – Cage Match
In general, WLS and OLS estimates should
always differ due to sampling error
-However, some differences are problematic:
1) If significant variables change signs
2) If significant variables drastically change
magnitudes
-This usually indicates a violation of a GaussMarkov assumption, generally the zero
conditional mean assumption (MLR.4)
-this violation would cause bias
-the Hausman (1978) test exists to test for this,
but “eyeballing” is generally sufficient
8.5 Linear Probability Model
We’ve already seen that the Linear Probability
Model (LPM), where y is a Dummy Variable, is
subject to Heteroskedasticity
-the simplest way to deal with this Het is to use
OLS estimation with heteroskedastic-robust
standard errors
-since OLS estimators are generally inefficient in
LPM, we can use FGLS:
8.5 LPM and FGLS
We know that:
Var ( y | X )  p ( X )[1  p ( X )]
p ( X )   0  1 x1  ...   k xk
Where p(X) is the response probability;
probability that y=1
-OLS gives us fitted values and estimates
variance using
ˆ ˆ
ˆ
hi  yi [1  yi ]
Given that we now have hhat, we can apply
FGLS, except for one catch…
8.5 LPM and FGLS
If our fitted values, yhat, are outside our (0,1)
range, hhat becomes negative or zero
-if this happens WLS cannot be done as each
observation i is multiplied by 1 / ĥ
i
The easiest way to fix this is to use OLS and
heteroskedasticity-robust statistics
-One alternative is to modify yhat to fit in the
range, for example, let yhat=0.01 if yhat is too
low and yhat=0.99 if yhat is too high
-unfortunately this is very arbitrary and thus not
the same among estimations
8.5 LPM and FGLS
To estimate LPM using FGLS,
1) Estimate the model using OLS to obtain yhat
2) If some values of yhat are outside the unit
interval (0,1), adjust those yhat values
3) Estimate variance using:
hˆi  yˆi [1  yˆi ]
4) Perform WLS estimation using the weight hhat
8. Heteroskedasticic Review
1) Heteroskedasticity does not affect consistency
or biasedness, but does affect standard errors
and all tests
2) 2 ways to test for Het are:
a) Breuch-Pagan Test
b) White Test
3) If the form of Het is know, WLS is superior to
OLS
4) If the form of Het is unknown, FGLS can be
run and is asymptotically superior to OLS
5) Failing 3 or 4, heteroskedastic-robust standard
errors can be used in OLS