Transcript Making Sense/ Making Numbers/ Making Significance

GRA 6020
Multivariate Statistics; The Linear
Probability model and
The Logit Model (Probit)
Ulf H. Olsson
Professor of Statistics
Binary Response Models
y is a binary response var iable
x'  ( x1 , x2 ,......, xk ) is the full set of exp lanatory
var iables
Pr ob( y  1 | x)  G(  0  1 x1   2 x2  .....   k xk )
 G(  0  xβ)
•The Goal is to estimate the parameters
Ulf H. Olsson
The Logit Model
z
e
G( z) 
z
1 e
•The Logistic Function
•e ~ 2.71821828
ze
ln( z )
Ulf H. Olsson
The Logistic Curve G (The Cumulative
Normal Distribution)
Ulf H. Olsson
The Logit Model
G (  0  1 x1  .... k xk   )
 0  1 x1 ....  k xk 
e

 0  1 x1 ....  k xk 
1 e
1

(  (  0  1 x1 ....  k xk   ))
1 e
Ulf H. Olsson
Logit Model for Pi
y  1 or y  0;
Pi  Pr ob( yi  1)

1
(  (  0  1 x1 ....  k xk  ))

1 e
 Pi 
   0  1 x1  .... k xk  
ln 
 1  Pi 
Ulf H. Olsson
Simple Example
 Pi 
ln 
   0  1 x1  
 1  Pi 
Variables in the Equation
Step
a
1
addsc
Constant
B
,071
-5,898
S.E.
,017
1,018
Wald
17,883
33,588
df
1
1
Sig.
,000
,000
Exp(B)
1,074
,003
a. Variable(s) entered on step 1: addsc.

 Pi 
ln 
  5.898  0.071x1
 1  Pi 
Ulf H. Olsson
Simple Example

 Pi 
ln 
  5.898  0.071x1 
 1  Pi 
Pi
( 5.898  0.071 x1 )
e
1  Pi
x1  60  Pi  0.165
Ulf H. Olsson
The Logit Model
• Non-linear => Non-linear Estimation =>ML
• Model can be tested, but R-sq. does not work. Some pseudo
R.sq. have been proposed.
• Estimate a model to predict the probability
Ulf H. Olsson
Binary Response Models
• The magnitude of each effect  j is not especially useful since y*
rarely has a well-defined unit of measurement.
• But, it is possible to find the partial effects on the probabilities by
partial derivatives.
• We are interested in significance and directions (positive or
negative)
• To find the partial effects of roughly continuous variables on the
response probability:
p( x)
dG( z )
 g (  0  xβ)  j ; where g ( z ) 
x j
dz
Ulf H. Olsson
Introduction to the ML-estimator
Let  be the data matrix
  ( x1 , x2 ,......, xk ); where xi are vectors
The Likelihood function is as a function of the unknown
parameter vector  :
k
f ( x1 , x2 ,......, xk , )   f ( xi , )  L( | X )
i 1
Ulf H. Olsson
Introduction to the ML-estimator
• The value of the parameters that maximizes this function are the
maximum likelihood estimates
• Since the logarithm is a monotonic function, the values that
maximizes L are the same as those that minimizes ln L
The necessary conditions for max imiz in g L ( ) is
 ln L( )
0


We denote the ML  estimator  ML



L( )  L is the Likelihood function evaluated at 
Ulf H. Olsson
Goodness of Fit

2 ln L  2 log likelihood
•The lower the better (0 – perfect fit)
2(ln L2  ln L1 ) approximates a chi  square
df  q(no. of exp lanatory var iables )
•Some Pseudo R-sq.
•The Wald test for the individual
parameters
Ulf H. Olsson
The Wald Test
If x
N   ,  , then ( x   )'  ( x   ) is  (d )
1
2
H 0 : c( )  q,
then under H 0


W  (c( )  q) 'U (c( )  q)
1
is  (d )
2
Ulf H. Olsson
Example of the Wald test
• Consider a simple regression model
y   x 
H0 :    0 ,

we know

|  0 |

s ( )
 z or (t ) ;


W  (   0 ) 'Var (   0 ) (   0 )  z
1
2
is  2 (1)
Ulf H. Olsson