Transcript Lecture 9

Numerical Computation
Lecture 9: Vector Norms and
Matrix Condition Numbers
United International College
Review
• During our Last Class we covered:
– Operation count for Gaussian Elimination, LU
Factorization
– Accuracy of Matrix Methods
– Readings:
• Pav, section 3.4.1
• Moler, section 2.8
Today
• We will cover:
– Vector and Matrix Norms
– Matrix Condition Numbers
– Readings:
• Pav, section 1.3.2, 1.3.3, 1.4.1
• Moler, section 2.9
Vector Norms
• A vector norm is a quantity that measures how large
a vector is (the magnitude of the vector).
• For a number x, we have |x| as a measurement of
the magnitude of x.
• For a vector x, it is not clear what the “best”
measurement of size should be.
• Note: we will use bold-face type to denote a vector.
(x)
Vector Norms
• Example: x = ( 4 -1 )
–
4 2  (1) 2 is the standard Pythagorean length of x. This
is one possible measurement of the size of x.
x
Vector Norms
• Example: x = ( 4 -1 )
– |4| + |-1| is the “Taxicab” length of x. This is another
possible measurement of the size of x.
x
Vector Norms
• Example: x = ( 4 -1 )
– max(|4|,|-1|) is yet another possible measurement of
the size of x.
x
Vector Norms
• A vector norm is a quantity that measures how large
a vector is (the magnitude of the vector).
• Definition: A vector norm is a function that takes a
vector and returns a non-zero number. We denote
the norm of a vector x by
The norm must satisfy:
– Triangle Inequality:
– Scalar:
– Positive:
,and = 0 only when x is the zero vector.
Vector Norms
•
•
•
•
Our previous examples for vectors in Rn :
Manhattan
Euclidean
Chebyshev
• All of these satisfy the three properties for a norm.
Vector Norms Example
Vector Norms
• Definition: The Lp norm generalizes these three
norms. For p > 0, it is defined on Rn by:
• p=1
• p=2
• p= ∞
L1 norm
L2 norm
L∞ norm
Distance
Distance
• Class Practice:
– Find the L2 distance between the vectors x = (1, 2,
3) and y = (4, 0, 1).
– Find the L ∞ distance between the vectors x = (1,
2, 3) and y = (4, 0, 1).
Which norm is best?
• The answer depends on the application.
• The 1-norm and ∞-norm are good whenever
one is analyzing sensitivity of solutions.
• The 2-norm is good for comparing distances
of vectors.
• There is no one best vector norm!
Matlab Vector Norms
• In Matlab, the norm function computes the Lp norms
of vectors. Syntax: norm(x, p)
>> x = [ 3 4 -1 ];
>> n = norm(x,2)
n = 5.0990
>> n = norm(x,1)
n=8
>> n = norm(x, inf)
n=4
Matrix Norms
• Definition: Given a vector norm
the matrix
norm defined by the vector norm is given by:
• Example:
Matrix Norms
• Example:
• What does a matrix norm represent?
• It represents the maximum “stretching” that A
does to a vector x -> (Ax).
Matrix Norm Properties
•
•
•
•
•
|| A || > 0 if A ≠ O
|| c A || = | c| * ||A || if A ≠ O
|| A + B || ≤ || A || + || B ||
|| A B || ≤ || A || * ||B ||
|| A x || ≤ || A || * ||x ||
Matrix
• Multiplication of a vector x by a matrix A
results in a new vector Ax that can have a very
different norm from x.
• The range of the possible change can be
expressed by two numbers,
•
=||A||
• Here the max, min are over all non-zero vectors x.
Matrix Condition Number
• Definition: The condition number of a
nonsingular matrix A is given by:
κ(A) = M/m
by convention if A is singular (m=0) then κ(A) = ∞.
• Note: If we let Ax = y, then x = A-1 y and
m  min
Ax
x
 min
y
1
1

1
A y
A y
max
y

1
A1
Matrix Condition Number
• Theorem: The condition number of a
nonsingular matrix A can also be given as:
κ(A) = || A || * || A-1||
• Proof: κ(A) = M/m. Also, M = ||A|| and by
the previous slide m = 1 / (||A-1 ||). QED
Properties of the Matrix Condition
Number
•
•
•
•
For any matrix A, κ(A) ≥ 1.
For the identity matrix, κ(I) = 1.
For any permutation matrix P, κ(P) =1.
For any matrix A and nonzero scalar c ,
κ(c A) = κ(A).
• For any diagonal matrix D = diag(di),
κ(D) = (max|di|)/( min | di| )
What does the condition
number tell us?
• The condition number is a good indicator of
how close is a matrix to be singular. The
larger the condition number the closer we
are to singularity.
• It is also very useful in assessing the accuracy
of solutions to linear systems.
• In practice we don’t really calculate the
condition number, it is merely estimated , to
perhaps within an order of magnitude.
Condition Number And Accuracy
• Consider the problem of solving Ax = b. Suppose b
has some error, say b + δb. Then, when we solve the
equation, we will not get x but instead some value
near x, say x + δx.
A(x + δx) = b + δb
• Then, A(x + δx) = b + δb
Condition Number And Accuracy
• Class Practice: Show:
Condition Number And Accuracy
• The quantity ||δb||/||b|| is the relative
change in the right-hand side, and the
quantity ||δx||/||x|| is the relative error
caused by this change.
• This shows that the condition number is a
relative error magnification factor. That is,
changes in the right-hand side of Ax=b can
cause changes κ(A) times as large in the
solution.