Transcript Chapter 4

Chapter 4
Linear Models and Matrix Algebra
Matrix algebra
• Provides a compact way of writing an
equation system.
• Leads to a way of testing the existence of
a solution by evaluation of a determinant.
• Gives a method of finding solution (if it
exists)
Matrices and Vectors
Matrices as Arrays:
a x  a x  ..... a x  d
11
1
12
2
1n
n
1
a x  a x  ..... a x  d
12
1
22
2
2n
n
2
..........
..........
..........
..........
.........
a x  a x  ..... a x  d m
m1
1
m2
2
mn
n
There are three types of ingredients in the equation system shown above:
1. The set of coefficients aij.
2. The set of variables x1, ….xn
3. The set of constant terms d1, ….dm
 a11 a12

a21 a22

A
 ...
...

am1 am2
... a1n 

... a2n 
... ... 

... amn 
 x1 
 
 x2 
.
x 
.
.
 
 xn 
d 
d 
 
.
d  
.
.
 
d m 
1
2
Numerical Example:
The following equation can be written in matrix form as follows:
2x1 + 3x2 + 5x3 = 10
x1 + 3x2 + 2x3 = 15
6x1 + x2 + 8x3 = 20
2 3 5
A  1 3 2
6 1 8 
 x1 
 
x   x2 
 x3 
10 
d  15 
20
This can also be written as Ax=d
A matrix is defined as a rectangular array of numbers, parameters
or variables. The members of the array are referred to as the
elements of the matrix.
Vectors as Special Matrices
The number of rows and columns in a matrix define the dimension of a
matrix. If there are m rows and n columns, the it is said to be a m x n matrix
(read as: “m by n”).
If m = n, then the matrix is called a square matrix.
 2 1 4
 6 2 2


8 12 1


3 0 7 
4 1
 3 2


This is a 4 x 3 matrix (“4 by 3”) where m = 4 and
n=3
This is 2 x 2 matrix (“2 by 2”) where m = 2 and n = 2. This is a square
matrix
Vectors---Column vectors:
These are matrices with dimensions n x 1. For example,
 2
 4
C 
6
 
8 
Dimension: 4 x 1
Row Vectors:
These are matrices with dimensions 1 x n. For example,
B  4 3 1 0 Dimensions: 1 x 4
Matrix Operations
Addition of matrices:
Two matrices can be added only when they have the same dimensions. When this
condition is met, the matrices are said to be conformable for addition.
General formula:
aij  bij   cij 
Numerical Example:
3 2 1   2 4 7   3  2 2  4 1  7   5 6 8
2 4  3  11  2 3  2  11 4  2  3  3  13 2 0

 
 
 

8 10 2   8 1 4  8  8 10  1 2  4  16 11 6
3x3
3x3
3x3
Matrix Operations..
Subtraction of matrices:
Two matrices can be subtracted only when they have the same dimensions. When this
condition is met, the matrices are said to be conformable for addition.
General formula:
aij  bij   dij 
Numerical Example:
 2 0 3   1 6 8   2  1 0  6 3  8    3  6  5
 8 5  3  10 1  3   8  10 5  1  3  3   2 4

0

 
 
 

 8 3 2   4 9 1   8  4 3  9 2  1   4  6 1 
3x3
3x3
3x3
Scalar Multiplication:
To multiply a matrix by a number – or in matrix algebra terminology,
by a scalar – is to multiply every element by the given scalar. The scalar
number can be positive or negative.
8 4 
2  4 2  3 3 

 Here the fraction 2/3 is a scalar


2
10
3 1 5  
 number.
3 3 
Multiplication of Matrices:
The matrix multiplication requires that the number of columns of the first
factor A be the same as the number of rows of the second factor B in
order to form the product AB. If this condition is not satisfied, then the
product is undefined.
General Form:
Numerical Example:
Matrix Operations..
Division of matrices:
While matrices can undergo addition, subtraction, and multiplication, subject
to the comfortability conditions, it is not possible to divide a matrix by
another. That is, we cannot have A/B.
The ∑ notation:
The summation shorthand makes use of Greek letter ∑ (sigma, for sum).
For instance,
4
x1  x2  x3  x4   xi
i 1
3
a1x0  a2 x1  a3 x2   a j x j 1
j 1
Notes on Vector Operations
Multiplication of vectors:
5 
Given p  3 and
 
q  3 2 1
5(3) 5(2) 5(1)  15 10 5
then pq  
   9 6 3
3
(
3
)
3
(
2
)
3
(
1
)

 


3x3
8 
Similarly, given a  2 4 and b    then ab  2(8)  4(2)  24
2

1x1
It is important to distinguish between the meaning of matrix pq (a matrix
larger than 1x1) and matrix ab (a 1x1 matrix, or a scalar).
Geometric Interpretation of Vector Operations:In this diagram, consider
two vectors v and w. The
sum v + w can be directly
plotted as the arrow shown.
If we construct a
parallelogram with the two
vectors v and w as two of its
sides, however, the
diagonal of the
parallelogram will turn out
exactly to be the arrow
representing the vector sum
v + w.
If we have vector u and
then the multiplication of
u by k will produce an
overlapping arrow, but the
arrowhead will be
relocated, unless k =1. If
k > 1, then the arrow will
be extended out, if 0<k<1,
the arrow will be
shortened, and if k = 0,
the arrow will shrink into
the point of origin. A
negative multiplier will
reverse the direction of
the arrow.
Linear Dependence:
A set of vectors v1, v2, …,vn is said to be linearly dependent if (and
only if) any one of them can be expressed as a linear combination of
the remaining vectors; otherwise they are linearly independent.
For instance,
2
v1   
5 
,
1
v2   
 3
, and
6
v3   
14
then
2 1  6 
4v1 – 2v2 = v3 because 4    2      = v3
5 3 14
Commutative, Associative, and
Distributive Laws
Commutative law:
A+B=B+A
Given that
2 4
1
A
 and B  
3
2


3
2
4
3 6
 A B  B  A  

6 6 
Associative law:
(A + B) + C = A + (B +C)
1
A 
3
,
2
B 
5 
3
and C  8
 
6
( A  B)  C   
16
6
A  (B  C)   
16
In matrix multiplication, AB

BA. To show this, lets consider:
2 3 1 5 11 16 
4 6 3 2  22 32  AB


 

1 5 2 3 22 33
3 2 4 5  14 21  BA


 

This proves that matrix multiplication is not commutative, i.e. AB
 BA
Associative law:
(AB)C = A(BC) = ABC
If A is m x n matrix, and if C is p x q matrix, then conformability
requires that B be n x p matrix.
Distributive law:
A(B+C) = AB + AC [premultiplication by A]
(B+C)A = BA + CA [postmultiplication by A]
Identity Matrices and Null Matrices
Identity Matrices:
A square matrix with 1s in its diagonal and 0s everywhere else. It is denoted by I or
In where n serves to indicate its row (as well as column) dimension.
Thus,
1
0

I 5  0

0
0
0 0 0 0
1 0 0 0
0 1 0 0

0 0 1 0
0 0 0 1 
3 2
3
Also, let A  
 AI  

4 1 
4
1
 IA  
0
2 1 0 3




1 0 1 4
0  3 2  3




1 4 1  4
2
A

1
2
A

1
Null Matrices
Null Matrices:
A null matrix is simply a matrix whose all elements are zero. Unlike I, the
null matrix is not restricted to a square matrix. It can have any dimensions.
Null matrices follow the following rules:
 2  0   2 
A0           A
 4  0   4 
0 
4 2  1   0
A0  
0     0


 0 1 0  0  0 
 
Transposes and Inverses
Transposes:
When the rows and columns of a matrix A are interchanged, we obtain a
transpose of A. It is either denoted by A’ or AT.
2 4
If A  
 A' 

3 2
2 3
 4 2 Here the dimension of the transpose matrix is

 identical to the original matrix.
Properties of transpose:
1. (A’)’ = A
2. (A+B)’ = A’ + B’
3. (AB)’ = B’A’
Inverses:
The inverse of a matrix A, denoted by A-1, is defined only if it’s a
square matrix, in which case the inverse is the matrix that satisfies
the condition: AA-1 = A-1A=I
Inverse Properties:
1. Not every square matrix has an inverse. If a square matrix A has an
inverse, then we call it nonsingular; if A has no inverse, then it is singular
matrix.
2. If A-1 does exist, then matrix A can be regarded as the inverse of A-1.
3. If A is n x n matrix, then A-1 must also be n x n.
4. If an inverse exists, then it is unique.
5. The two parts of AA-1 = A-1A=I actually imply each other.
Inverse Matrix and Solution Linear Equation System:
We earlier referred to an equation Ax = d to solve a system of linear equations.
The concept of inverse matrix can be applied to this:
Example:
Note: The method of testing the existence of inverse and its calculation will
be discussed in Chapter 5.
Finite Markov Chains
A common application of matrix algebra is found in what is known as
Markov processes or Markov Chains.
Markov processes are used to measure or estimate movements over time.
This involves the use of a Markov transition matrix, where each value in
the transition matrix is a probability of moving from one state to another.
There is also a vector containing the initial distribution across various states.
Example:
Let : PAA = probability that a current A remains in A.
PAB = probability that a current A moves to B.
PBB = probability that a current B remains in B.
PBA = probability that a current B moves to A.
At and Bt denote the populations of Abbotsford and Burnaby,
respectively, at some t.
If we denote the distribution of employees across locations at time t as a vector,
'
xt  At Bt 
and the transitional probabilities in matrix form.
P
M   AA
 PBA
PAB 
PBB 
Then the distribution of employees in the next period (t+1) is,
'
'
xt M  xt 1  At 1 Bt 1
In general, for n periods the distribution of employees will be,
At
n
 PAA PAB 
Bt 
 At  n

 PBA PBB 
Bt  n 