Transcript Applied Econometrics for Competition and Regulation

Properties of OLS
How Reliable is OLS?
Learning Objectives
1. Review of the idea that the OLS estimator
is a random variable
2. How do we judge the quality of an
estimator?
3. Show that OLS is unbiased
4. Show that OLS is Consistent
5. Show that OLS is efficient
6. The Gauss-Markov Theorem
1. OLS is a Random Variable
• An Estimator is any formula/algorithm that
produces an estimate of the “truth”
– An estimator is a function of the sample data e.g. OLS
– Others: “Draw a line”
– For individual consumption function its not so obvious
• Implications for accuracy of the estimates
• So how do we choose between different
estimators?
– What are the criteria
– What is so special about OLS?
– What does it take for OLS to go wrong?
consumpt
741.74
1.25
28
1043.21
income
Recall the Definition of OLS
• Review of where OLS comes from.
• Think of fitting a line to the data. This will never pass through every
point
C  b bY u
i
0
1 i
i
Cˆ i  b0  b1Yi
C  Cˆ  u
i
i
i
• Every choice of b0 and b1 will generate a new set of ui
• OLS chooses b0 and b1 to min sum of squared ui
• “Best fit”
– R2
– so what?
C= 01Y
C
C1
u1
C3
C2
u3
u2
1
Y2
Y1
Y3
Y
OLS Formulae
N
b1 
 (C  C )(Y  Y )
i
i 1
i
N
 (Y  Y )
i 1
2
i
b0  C  b1Y
• The key issue is that both are functions of the data so
the precise value of each estimate will depend on the
particular data points included in the sample
• This observation is basis of all statistical inference and
all judgments regarding the quality of estimates.
Distribution of Estimator
• Estimator is a random variable because sample is
random
– “Sampling error” or “Sampling distribution of the estimator”
• To see the impact of the sampling on estimates, try
different samples (see histograms over)
• Key point: even if we have the correct model we could
get an answer that is way off just because we are
unlucky in the sample.
• How do we know if we have been unlucky? How can we
minimise the chances of bad luck?
• This is basically how we assess the quality of one
estimation procedure compared to another
0
.5
Density
1
1.5
Comparing MAD and OLS
-1
0
beta_mad
1
2
0
.5
Density
1
1.5
-2
-2
-1
0
beta_ols
1
2
• Both estimators are
random variables
• The OLS estimator
has lower variance
than the MAD
distribution
• Both are centred
around the true value
of beta (0.75)
How Judge an Estimator?
• Comparing estimators amounts to comparing
distributions
• Estimators are judged on three criteria
– Unbiased
– Consistent
– Efficient
• These criteria are all different takes on the question of:
what is the probability that I will get a seriously wrong
answer from my regression?
• OLS is the Best Linear Unbiased Estimator (BLUE)
– Gauss-Markov Theorem
– This is why it is used
2. Bias
• Sampling distribution of the estimator is centered around the true
value
• E(bOLS)=
• Implication: With repeated attempts OLS will give correct answer on
average
– Does not imply that it will give the correct answer in any given
regression
• Consider the stylized distribution of two estimators below
– The OLS estimator is centered around the true value. The alternative is
not
– Is OLS better?
– Suppose your criteria were the avoidance of really small answers?
• Unbiasedness hinges on the model being correctly specified i.e.
correct variables
– Omitted relevant variables
• It doesn’t require a large number of observations: “small sample”
• Both MAD and OLS are unbiased
f(b?)
f(bOLS)
E(bOLS)= 
E(b?) 
• OLS is centered around the true value but has a
relatively high probability of returning a value
that is low
3. Consistency
• Consistency is a large sample property i.e. asymptotic
property
– As N  , the distribution of the estimator collapse to the true
value 
– The distribution gets narrower
• This is more useful than unbiasedness because it
implies that the probability of getting any wrong answer
falls as sample size increases
• Formalises the common intuition that more data is better
– “Law of Large Numbers”
• Note an estimator could be biased but still consistent
e.g. 2SLS
• Consistency requires a correctly specified model
Same estimator, Larger sample
TRUE
As sample size increases we get closer to truth e.g. prob of error falls

TRUE
As sample size increases we get closer to truth
4. Efficiency
• An efficient estimator has minimum variance
of all possible alternatives
– Squashed distribution
• Looks similar to consistency but is small
sample property
– Compares different estimators applied to the
same sample
• OLS is efficient i.e. “best”
– Reason why it is used where possible
– GLS, IV, WLS, 2SLS are inefficient
• OLS is more efficient in our example than
MAD
Same sample size, different estimator
TRUE
Prob of error is lower for efficient estimator at any sample size
5. Gauss-Markov Theorem
• A formal statement of what we have just
discussed
• Mathematical specification is required in order to
do hypothesis tests
• Standard model
– Observation = systematic component + random error:
– yi = 1 +2 xi + ui
– Sample regression line estimated using OLS
estimators:
– yi = b1 + b2 xi + ei
Assumptions
1. Linear Model ui = yi - 1- 2xi
2. Error terms have mean = 0
•
E(ui|x)=0 => E(y|x) = 1 + 2xi
3. Error terms have constant variance
(independent of x)
•
Var(ui|x) = 2=Var(yi|x) (homoscedastic errors)
4. Cov(ui, ui )= Cov(yi, yi )= 0. (no
autocorrelation)
5. X is not a constant and is fixed in repeated
samples.
6. Additional assumption:
•
ui~N(0, 2) => yi~N(1- 2xi, 2)
Summary BLUE
• Best Linear Unbiased Estimator
• Linear: Linear function of the data
• Unbiased: Expected Value of the estimator
equals true value
– Doesn’t mean always get the correct answer
– Algebraic proof in book
– Unbiasedness property hinges on the model
being correctly specified i.e. E(xi ui)=0
Some Comments On BLUE
• Aka Gauss-Markov Theorem:
• First 5 assumptions above must hold
• OLS estimators are the “best” among all linear and
unbiased estimators because
– they are efficient: i.e. they have smallest variance among all
other linear and unbiased estimators
• G-M result does not depend on normality of dependent
variable
– Normality comes in when we do hypothesis tests
• G-M refers to the estimators b1, b2, not to actual values
of b1, b2 calculated from a particular sample
• G-M applies only to linear and unbiased estimators,
there are other types of estimators which we can use
and these may be better in which case disregard G-M
– a biased estimator may be more efficient than an unbiased one
which fulfills G-M.
Conclusions
1. OLS estimator is a random variable:
– its precise value varies with the particular
sample used
2. OLS is unbiased:
–
the distribution is centred on the truth
3. OLS is Consistent:
– Probability of large error falls with sample
size
5. OLS is efficient:
– Probability of large error is smallest of all
possible estimators
6. The Gauss-Markov Theorem:
– formal statement that 3-5 hold when certain
assumptions are true
What’s Missing?
• What happens to OLS when the assumptions
of the GM theorem are violated
• There will be some other estimator that will
be better
• We will look at 4 violations:
–
–
–
–
Omitted variable bias
Multicolinearity
Heteroscedastcity
Autocorrelation