Transcript Introduction to matrix algebra

Welcome to the Matrix (Algebra) or:
Is this torture really necessary?!
• What for?
– Permits compact, intuitive depiction of regression
analysis
– Flexible, in that it can handle any number of
independent variables
– Generally used in statistical presentation, for OLS and
other techniques
• You need to be able to interpret it.
The Basics
• Matrix form:
5 8 
10
A  
,
where
a

0
23
12 1 0

Elements of a matrix can be indentified as
a11 a12 a13 
A  
a21 a22 a23 

:
Transpose (“prime”) of a Matrix
5 8 
10
A

12 1 0

10
A  5

8
12 
1 
0 

Vectors
• Vectors are essentially single rows or
columns:
6 
1
Y   
8


11
Y  6 1 8 11
Adding Matrices
Addition works only if matrices have the same dimension:
X1  X 2
4 3 8 1 
 

 
2 0  4 5
4  8 3  1  12 2 
 

 
2  4 0  (5) 6 5 
Multiplication of Matrices
Dimensions: A(r*q) * B(q*c) = C(r*c),
So the number of columns in the first matrix must match
the number of rows in the second matrix
2

1

6
5 
 4 2
0  
 5 7
2 
1

2
 (2  4)  (5  5)
(2  2)  (5  7)
(2  1)  (5  2) 


  (1  4)  (0  5)
(1 2)  (0  7)
(1  1)  (0  2) 


(6  4)  (2  5) (6  2)  (2  7) (6  1)  (2  2)
33 39 12


 4 2
1 


14
2
2


Rules for Matrix Multiplication
• Are matrices conformable?
A x B
=
(r x q) (q x c)
C
(r x c)
• Vector times a matrix:
A x B
=
C
(r x c) (c x 1)
(r x 1)
• Row and column vectors:
A x B
=
(r x 1) (1 x p)
C
(r x p)
A x B
(1 x r) (r x 1)
C
(1 x 1) a scalar
=
Identity Matrices
Square matrices with 1’s on diagonal and 0’s elsewhere:
1
0
I4  
0

0
0
1
0
0
0 0
0 0
1 0
0 1 

4 x 4 identity matrix
Identity matrices act like 1’s in familiar algebra:
I x B
=
(r x r) (r x c)
B
(r x c)
Matrix Inversion
Acts a bit like dividing any number by itself in algebra:
any matrix multiplied by its inverse is equal to the identity
matrix:
1
1
CC  C C  I
Here an example (multiply it out and check):
10 4 
0.112245 0.04082
1
C  
so C  

3 11
0.03061 0.102041

Inversion works only for square matrices
Finding the Identity Matrix:
An Example
3 1  1 a
C  

C


2 4 

b
c  1 0 

I
d 
 
0 1 

2a + 4b = 0 so 2a = -4b and a = -2b
3a + b = 1 so 3(-2b) + b = 1, and -5b=1 so b = -1/5
Therefore: a = -2(-1/5) so a = 2/5
3c + d = 0 so d = -3c
2c + 4d = 1 so 2c + 4(-3c) = 1 and -10c = 1 so c = -1/10
Therefore d = -3(-1/10) so d = 3/10
Example Continued
Now we can check and see the result of C x C-1:
1 
 2

3 1
1
5
10


C  1
3  C  

2
4




 5 10 
1
2
1
 2

(  3)  (  2) (  1)  (  4)
5
10
5
10

  1
(  3)  ( 3  2) ( 1  1)  ( 3  4)
 5

10
5
10
1 0
 
0 1

Regression in Matrix Form
• Assume a model using n observations, with K-1
Xi (independent) variables
Y (n 1) is a column vector of the observed dependent variable
ˆ (n 1) is a column vector of predicted Y values
Y
X (n  K) each column is of observations on an X, first column 1' s
B (K 1) a row vector of regression coefficients (first is b 0 )
U (n 1) is a column vector of n residual values
Regression in Matrix Form
Y  XB  U
Yˆ  XB
B  ( X X) 1 X Y
Note: we can’t uniquely define (X’X)-1 if any
column in the X matrix is a linear function of
any other column(s) in X. Why is that?
The X’X Matrix
 n
 X1
( X X)  
X2


 X3
X X
 X 
 X  X X  X X 

X
X
X
X
X


 
 X X  X X  X 
1
2
1
2
3
2
1
2
2
2
1
1
3
3
1
3
2
3
2
Note that you can obtain the basis for all the necessary
means, variances and covariances among the Xs from the
(X’X) matrix
2
3
An Example of Matrix Regression
Using a sample of 7 observations, where X has
Elements {X0, X1, X2, X3}
6
1 4 5 4
 6.48 
 0.48
11
1 7 2 3
10.02
 0.98 
 






4
1 2 6 4
 4.41 
  0.41
 






Y   3  X  1 1 9 6 Ŷ =  2.51  U   0.49  B = 3.96 1.06 0.04  0.49
5
1 3 4 5
 4.89 
 0.11 
 






9
1 7 3 4
 9.58 
 0.58
10
1 8 2 5
10.11
  0.11
 






Example:
Analysis of Test Scores
Source |
SS
df
MS
Number of obs =
420
-------------+-----------------------------F( 3,
416) = 107.45
Model | 66409.8837
3 22136.6279
Prob > F
= 0.0000
Residual | 85699.7099
416 206.008918
R-squared
= 0.4366
-------------+-----------------------------Adj R-squared = 0.4325
Total | 152109.594
419 363.030056
Root MSE
= 14.353
-----------------------------------------------------------------------------testscr |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------str | -.2863992
.4805232
-0.60
0.551
-1.230955
.658157
expn_stu |
.0038679
.0014121
2.74
0.006
.0010921
.0066437
el_pct | -.6560227
.0391059
-16.78
0.000
-.7328924
-.5791529
_cons |
649.5779
15.20572
42.72
0.000
619.6883
679.4676
------------------------------------------------------------------------------
Application of Multivariate
Regression Analysis
• Use the .do file posted to replicate the
preceding model using matrix algebra
• Evaluate the Output
• Draw Initial Conclusions
• Some Hints!
– Enlarge your matrix size (set mat command)
– Drop unnecessary variables
Break for analysis...
• Feel free to work in
groups
• Discuss Analyses
• Take 20 minutes