Transcript What is a Matrix - Texas A&M University

Review of Matrix Algebra
Department of Statistics
Texas A&M University
presented by Curtis Alexander
What is a Matrix ?
•
A matrix may also contain
2 4 7 
9 0 1 


 
 
 
  
– fractions
AND/OR
– decimals
1 5 0.71
3


8 4.9 
5

A matrix is a rectangular table,
or array, composed of either
– numbers
OR
– variables
•
Matrix Notation I
• Matrices (the plural of
matrix) are enclosed in
– brackets (as we have seen)
OR
– parenthesis (less commonly
used notation)
* Note: We will use the more
conventional bracket notation.
2 3
 5 1


 4 2


1 9
Matrix Notation II
• A matrix is typically denoted
by a capital letter.
A=
• When a matrix appears
inline with text, it is often
written in bold.
bold
2 3
 5 1


In simple linear regression, we call
H the hat matrix because it
transforms the vector of observed
responses into the vector of fitted
responses.
* Note: Different textbooks will use
different notation for matrices –
so check the notation!
Elements of a Matrix
element
• The individual
numbers (or
variables) in a matrix
are called elements.
* Each possible location
in a matrix must
contain an element.
Q : Is this a matrix ?
A : NO!!
missing element
4 0
8 5 


element
4 2 1
7

9


 2 5 
missing element
Rows and Columns
Column 3
Column 1
Q : What other parts of
a matrix are named?
Rows – run horizontally
AND
Columns – run vertically
Row 2
2
5

1

2

6
0
5
0
6
8
7
7
4

4
7

3
Row 4
Location of Elements
• We refer to a specific
element in a matrix by
referencing its location
using rows and columns.
• When referencing an
element within a matrix,
we use the lower case
letter of the matrix.
• We list the row first,
followed by the column.
• Thus we say that the
element t23 = 5 .
t23
T=
3
1

3

4
6
4
9
7
9
5
0
5
2

8
2

8
Elements of a Matrix Practice
Q
A
Q
A
Q
: w12 = ?
: w12 = 5
: 6=?
W
: 6 = w24
: The element with value
16 is located in which
row?
A : Row 3
Q : How many columns
(total) does this matrix
contain?
A : 4 columns
=
1 5 8 9
3 4 2 6


 0 11 16 18 


 22 7 10 21
Size of a Matrix
• The size of a matrix is
expressed in the form r
rows by p columns.
• Matrix A has 4 rows and
3 columns, so we list its
size as 4 x 3 – which we
read “four by three” and
which we write as A4x3.
* Note: Dimension is
another word used for
size of a matrix.
Q : What is the size (or
dimensions) of matrix X?
A :2x2
3
8
A=

8

1
X=
2
9
4
0
1

3
7

5
1 9 
0 3


Special Matrices I
• Vector – a matrix that has
only one row or one
column
– Column vector – matrix with
one column
– Row vector – matrix with one
row
row vector
* Note: Vectors are denoted
by lower case letters.
• Scalar – a matrix that has
only one row and one
column, or alternatively a
matrix that only has one
element
– Scalars are usually written
without brackets.
– Think of scalars as merely
constants.
2
3
d= 
 1 
column vector
g = 1 8 6
5
scalars
β
11
2
Special Matrices II
• Square matrix – the
number of rows and
diagonal
columns are equal
• Diagonal – the elements
that run from the upper
left element to the lower
right element in a square
matrix – also called the
main diagonal
• Diagonal matrix – a
square matrix with all
non-diagonal elements
diagonal
equal to zero
matrix
3 3 8
9 1 2 


 7 5 6 
2 0 0
0 7 0


 0 0 6 
Special Matrices III
• Zero matrix – a matrix
with all elements equal to
zero matrix
zero
• Identity matrix – a
diagonal matrix with all
elements along the
diagonal equal to one
1
0
I= 
0

0
– The identity matrix is
denoted by the capital
letter I.
– The identity matrix will
become important later
when we discuss the
inverse of a matrix.
identity matrix
0 0 
0 0 


0
1
0
0
0
0
1
0
0

0
0

1
Matrix Addition
• Matrices may only be
added if they are the same
size.
A=
3 2 1 
5 6 7 


B=
8 9 0 
 2 3 4


– A and B are both 2x3 so
they may be added.
• The result of A+B, which
we call C, is also size 2x3.
* Note: Matrix addition IS
commutative in general –
A+B = B+A
C=
• Take each matching
element of A and B and
add them together, placing
the sum in the same
=
elemental position of C.
3  8 2  9 1  0 
5  2 6  3 7  4 


11 11 1
7 9 11 


Matrix Subtraction
• Like matrix addition, matrices
may only be added if they are
the same size.
– D and E are both 2x3 so they
may be subtracted.
• The result of D - E, which we
call F, is also size 2x3.
* Note: Matrix subtraction is
NOT commutative in general –
D-E≠E-D
• Take the matching element of F =
D and E and subtract the
element from matrix E from the
element from matrix D, placing
the difference in the same
=
elemental position of F.
5 4 1 
D=
9 7 3


2 1 0 
E= 

8 8 5
5  2 4  1 1  0 
9  8 7  8 3  5


3 3 1 
1  1  2


Matrix Add/Sub Practice
Q:
A :
Q:
A :
A–B=?
 2 2
7 2 


C+A=?
9 6
9 7


Q: B+D=?
A : Sorry – trick question!
We cannot sum because
their sizes are different!
4 3
A= 

 7 5
 2 1
B= 

0
3


5 3
C= 

2
2


2 8 9 


D= 1 3 4


7 4 2
Matrix Multiplication Introduction
• To multiply matrices, the # of
columns of the first matrix
MUST equal the # of rows of
the second matrix.
• How do we easily go about
determining if M can be
multiplied by N?
– Write the dimensions of M
adjacent to the dimensions of
N. If the inner dimensions
match, then they may be
multiplied.
– The outer dimensions
determine the dimensions of
the product P.
* Note: Matrix multiplication is
NOT commutative in general -M*N ≠ N*M
3 3
M= 

1
0


 4 9 2
N=
7 5 1 


M
2x2
N
2x3
outer dimensions
inner dimensions
P=
33 42 9
7 5 1 


Matrix Multiplication Properties
• If A, B, and C are matrices and if the
multiplicative combinations below are
assumed to have the correct size then in
general
– (AB)C = A(BC)
– (A+B)C = AC + BC
– C(A+B) = CA + CB
associativity
left distributivity
right distributivity
• However, commutativity is does NOT hold
in the general case
– AB ≠ BA
Multiplication – Scalar * Matrix
• This is the easiest
form of matrix
multiplication – a
scalar, or constant,
multiplied by a matrix.
• Simply multiply each
element in the matrix
by the scalar.
2 9 0


3*  8 5 8 
 6 6 1 
=
 2 *3 9 *3 0 *3
 8*3 5*3 8*3 


 6 *3 6 *3 1*3 
=
 6 27 0 
 24 15 24 


18 18 3 
Multiplication – Vector * Vector
• The only vectors that can be
multiplied together are
– row vector * column vector
– column vector * row vector
• Check to ensure that the #
columns (or rows) of the vector
equals the # rows (or columns)
of the vector.
• To evaluate j*k, multiply the
first element of j by the first
element of k. The result is
placed in l11. Next multiply the
first element of j by the second
element of k and place it in l12.
• This is continued where the
element location of j becomes
the row location of the result
and the element location of k
becomes the column location
of the result.
j
k
2x1
1x2
j=
equal
k=
1 5
column vector
row vector
l=
2
3
 
 2 10
 3 15


Multiplication – Vector * Matrix I
• Only row vectors can be
multiplied by a matrix.
Check to ensure that the #
columns of the vector
equals the # rows of the
matrix.
• To evaluate a*B = c, first
multiply the elements of
vector a that correspond
with the elements of the
first column of B. Second,
sum all three of these
intermediate product results
to get the final result. This
result becomes the first
element of c (c11).
– The result of any vector
multiplied by a matrix is a
vector.
a
B
1x3
3x3
equal
a=
1
c=
1*2  2*4  3*8
=
2 3
2 3 1
4 5 2
B= 

 8 3 7 
34
? ?
? ?
Multiplication – Vector * Matrix II
• To further evaluate a*B
= c, multiply the
a=
elements of vector a
that correspond with
the elements of the
second column of B.
Sum all three of these
c=
intermediate product
results to get the final
result. This result
=
becomes the second
element of c (c12).
1
2 3
2 3 1
4 5 2
B= 

 8 3 7 
34
1*3  2*5  3*3 ?
34
22 ?
Multiplication – Vector * Matrix III
• To finish evaluating
a*B = c, multiply the
elements of vector a
that correspond with
the elements of the
third column of B.
Sum all three of these
intermediate product
results to get the final
result. This result
becomes the third
element of c (c13).
a=
c=
1
34
2 3
2 3 1
4 5 2
B= 

 8 3 7 
22 1*1  2*2  3*7
=
34
22 26
=
34
22 26
Multiplication – Matrix * Matrix I
• To multiply matrices, the
# columns of the first
matrix must equal the #
rows of the second
matrix.
• We proceed similar as to
when multiplying a
vector*matrix in order to
multiply X*Y = Z.
– First, multiply the first row
of matrix X by the first
column of matrix Y
(matching the elements as
we have done previously).
Sum these products and
the result goes in z11.
X
Y
2x2
2x3
equal
X=
 2 3
 4 5


Y=
5 6 2 
1 0 3 


2*5+3*1 = 13
Z=
13 ? ?
? ? ? 


Multiplication – Matrix * Matrix II
• To find z21, multiply the
second row of matrix X by
the first column of matrix
Y=
Y (matching the elements X =
as we have done
previously). Sum these
products and the result
goes in z21.
4*5+5*1 = 25
* Note: For a specific
Z=
element in Z, say z21, we
know this means multiply
the second row of matrix
z21
X by the first column of
matrix Y.
 2 3
 4 5


5 6 2 
1 0 3 


13 ? ? 
25 ? ?


2nd row of X
1st column of Y
Multiplication – Matrix * Matrix III
• For all subsequent
elements of Z, multiply
the appropriate row of
matrix X by the
appropriate column of
X=
matrix Y (matching the
elements as we have
done previously). Sum
these products and the
result is placed in the
appropriate location in Z.
• As a final example, to find
z13 multiply the first row of
matrix X by the third row
of matrix Y and then sum
the products of these
rows and columns.
 2 3
 4 5


Z=
Y=
5 6 2 
1 0 3 


13 12 13 
25 24 23


2*2+3*3 = 13
Matrix Multiplication Practice
Q:
A :
b*A = ?
Q:
A :
C*E = ?
75
54
5 19 22 
 45 56 53


Q : Which of the following
products exist?
–
A*b; A*D; b*C; E*C;
A*C; D*D; b*b
A : A*D; b*C; A*C; D*D
4 3
A= 

 7 5
b=
3 9
2 1 0 
C=
D=

3 4 5 
5 3
2 2


2 8 9 


E= 1 3 4


7 4 2
Trace
• To find the trace of a
square matrix, simply
sum all the elements
that lie along the main
diagonal of the matrix.
sum
these
elements
4 3
U= 

 7 5
sum
these
elements
tr (U)  4  5  9
2 8 9 


V= 1 3 4


7 4 2
tr (V)  2  3  2  7
Transpose
• To take the transpose of a
matrix, either
– write the rows as columns
OR
– write the columns as rows
• The transpose of a
column vector is a row
vector and vice versa.
* Note: Transpose may
either be written with a
capital T or using an
apostrophe.
2 4 2


A = 1 7 5 
8 6 0 
2 1 8
AT =  4 7 6 


 2 5 0 
1 
 
h = 3
4
 
4
h ́ = 1 3 4 4
Inverse Introduction I
• We have seen addition,
subtraction, and
multiplication of matrices.
There does not exist
division of matrices per se –
instead we shall use the
inverse of a matrix.
• For example, in order to
solve for x at right, you
simply divide both sides of
the equation by three and
find that x=4.
• Another way we say this is
accomplished is by
multiplying both sides of the
equation by 1/3.
3x  12
3x 12

3
3
x4
3x  12
1
1
3x  12
3
3
x4
Inverse Introduction II
• If A is a square matrix and the inverse of A
exists, then
A*A-1 = A-1*A = I (which is similar to
the algebraic expression ⅓ * 3 = 1).
• We say that A is invertible if A*A-1= I.
* Note: Not every square matrix has an
inverse. If a matrix, say A, does not have an
inverse then we say that the inverse of A
does not exist.
Determinant of a 2x2 Matrix
• Before we can get to
calculating the
inverse of a matrix,
we need to know how
to calculate the
determinant.
• To calculate the
determinant, simply
multiply the opposite
corner elements and
subtract the product
results.
a b 
X= 

c d 
det(X)  a * d  b * c
Y=
1 4 
5 6 


det(Y)  1*6  4*5
 14
Inverse of a 2x2 Matrix I
3 6
A = 2 5


det(A)  3 * 5  6 * 2  3
• First, calculate the
determinant of the
matrix you would like
 5  6
sign change
to invert.
 2 3 


• Next, invert the signs
of element a12 and a21 sign change
values exchanged
and then exchange
1  5  6
-1
the values of a11 and A = * 
3  2 3 
a22.
1
 5  6  5


2
• Finally, multiply det(A)
 3

3  3
=  2 3   2

by this new matrix.

 
1
 3
3 
 3

Inverse of a 2x2 Matrix II
a b 
X = c d 


• For a generic matrix
X, the formula for
 d  b
1
1
-1
calculating the
X = det(X)  a * d  b * c *  c a 


inverse is at right.
• When does the
division by 0 –
therefore
inverse not exist?
inverse does
3 4
• Whenever the
not exist
Z= 

6 8

determinant is 0
because we would be
dividing by 0 in the Z-1 = 1  1 *  8  4  1 *  8
det(Z ) 3 * 8  4 * 6  6 3  0  6
inverse equation.
 4
3 
Determinant and Inverse Practice
Q:
A:
Q:
A:
det(A)= ?
4*5 – 3*7=-1
D-1=?
 1
 2

 1
 2
3
 
4

5 
4 
Q : det(B)=?
A : 1*2 – 3*2=-4
Q : C-1=?
A : The inverse does not
exist because the
determinant is 0 -det(C)=4*5-2*10=0
4 3
A= 

 7 5
1 3 
B= 

2
2


 4 2
C= 

10
5


5 3
D= 

2
2


Writing a System of Linear
Equations in Matrix Form
• What if we had the following
system of linear equations.
• How could we write this in
matrix notation?
• First, write the coefficients in a
2x2 matrix (or an nxn matrix
depending upon the number of
equations).
• Next, create a column vector
containing the variables and
place it to the right of the
coefficient matrix.
• Finally create another column
vector containing the values
on the right of the equal sign.
2 x  3 y  9

4 x  7 y  10
9
2 3   x 
4  7   y   10
 

  
Solving Simple Matrix Equations
• To solve equations
like A*X=Y for X, we
do not divide Y by A,
but multiply the
inverse of A (which is
written as A-1) by Y to
find X.
identity
matrix
A*X = Y
A 1 * A * X  A 1 * Y
I * X = A -1 * Y
X = A -1 * Y
multiply on left
 2 3
A= 

4
5


 x1 
X = x 
 2
1 
Y=  
6
2
4

1
0

3  x1  1 
 



5  x2  6 
0   x1   2.5 1.5 1 




1   x2   2
1  6 
 x1  6.5
 x    4 
inverse
 2  
of A
Example from Multiple Linear
Regression I
• Where do we see
Yi  0  1x1i  2i x2i       p x pi  ei
matrices in statistics?
• One such place is in
 Y1 
multiple linear regression,
 0 
Y 
 
which is seen at right.
1
Y   2
This is a multiple linear
β 

  
regression model for n
 
 
independent observations
Y
  p 
 n
generated from p
predictor variables.
1 x11  x1 p 
 e1 
1 x

• Instead of writing out n
e 

x
21
2
p

 2
X
equations, this may be
e

    

written using matrices as


 
1 xn1  xnp 
Y  Xβ  e
e n 
Example from Multiple Linear
Regression II
• As a final example of
where matrices occur in
statistics, we look again
at multiple linear
regression.
• The residual sum of
squares of β is seen at
right.
• Using RSS(β), we
calculate the least
squares estimate of β. In
this form, we obtain the
least squares estimate of
each βi with just a few
matrix computations.
Yi  0  1x1i  2i x2i       p x pi  ei
Y  Xβ  e
RSS(β)  (Y  Xβ)T (Y  Xβ)
ˆβ  (XT X)1 XT Y
Matrix Calculator
• Web based Matrix Calculator
http://people.hofstra.edu/stefan_waner/Realworld/matrixalgebra/fancymatrixalg2.html
• Note that there are other matrix calculators
online, but this is among the easiest to use.
• Now I will demonstrate some of its
capabilities.