Transcript How do I know the answer if I`m not sure of the question?

How do I know the answer if
I’m not sure of the question?
Putting robustness into estimation
K. E. Schubert
11/7/00
Familiar Picture?
Basic Problem
Picture of something
that has been blurred
 If I know how it was
blurred then I should
be able to clean it up
 If system is invertible
then I can get the
original
A

x
b
†
A
x
b
Familiar Picture
Encountering Resistance

Consider a simpler problem.
–
–
–
–

Unknown resistor.
Take current and voltage measurements.
Plot them out.
Want to fit a line to the points.
No measurement is perfect.
–
–
No exact fit to all the points.
Want “best” fit.
Measured Values
Unknown Resistor
12
10
8
6
4
2
0
0
2
4
6
8
10
voltage [V]
12
14
16
18
Gauss’ Stellar Problem
Orbit of Ceres.
 Errors were in people’s measurements
 Consider distance from the measurements to
the equation to fit
 minimize the square of this distance

–

2
min ||Ax-b||
T
-1 T
†
x=(A A) A b=A b
Understanding Solution
In our problem A, b are vectors
 Finding nearest scaled A to b
 Projection

b
Ax-b
A
Ax
Resistor Solved
Want to find slope, 1/R
 i=(1/R)v
 Ax=b
 A vector of voltages
 b vector of currents
 x is slope
†
 1/R=v i

Best line
Unknown Resistor
12
10
8
6
4
2
0
0
2
4
6
8
10
voltage [V]
12
14
16
18
Reasonable Question
What if I considered v=iR?
 Errors assumed in v now!
†
 R=i v
 How do the measured resistances compare?

Comparison of Methods
Unknown Resistor
12
10
8
6
4
2
0
0
2
4
6
8
10
voltage [V]
12
14
16
18
Errors in Both
A has errors (actual is A+dA)
 Want to minimize distance

–
2
min ||(A+dA)x-b||
Need to know something about dA
 Worst dA in bounded region
 Best dA in bounded region
 The dA that makes Ax=b consistent

Worst in a Bounded Region

Keep worst case ok, rest will be fine
 ||dA||< (bounded region)

Projection to farthest A+dA
b
(A+dA)x-b
dA
A
(A+dA)x
Best in a Bounded Region

Pick best dA but limit options
 ||dA||< (bounded region)

Projection to nearest A+dA
b
(A+dA)x-b
dA
(A+dA)x
A
Consistent Equation (TLS)
Called Total Least Squares
 Projection nearest to A and b in new space
 No bound on dA, as big as need!

b
(A+dA)x
A
General Regression Problems
All of the techniques mentioned so far fall
into the general category of regression
(including least squares)
 Find a solution for most by taking the
gradient and setting it equal to zero
T
-1 T
 x=(A A+I) A b
 Equation for , which is solved by finding the
roots of the equation (Newton’s or bisection)

Resistor by TLS
Unknown Resistor
12
10
8
6
4
2
0
0
2
4
6
8
10
voltage [V]
12
14
16
18
Simple Picture

Consider a city skyline.
–
–

Only consider outline of buildings.
Height is a function of horizontal distance.
Nice one dimensional picture.
Hazy Day

Smog and haze blur the image.
Rounds the corners off.
Want to get the corners back.
–
–
3.5
Actual
Measured
3
2.5
2
1.5
1
0.5
0
0
10
20
30
40
50
60
70
80
90
100
Least Squares Fails!
Blurring works like a Gaussian distribution
 Don’t know the exact blur

2000
Measured
Least S quares
1500
1000
500
0
-500
-1000
-1500
-2000
0
10
20
30
40
50
60
Least S quares Solution
70
80
90
100
TLS Too Optimistic!
TLS assumes things are consistent
 Allows dA to be large

60
Measured
TLS
40
20
0
-20
-40
-60
0
10
20
30
40
50
60
70
80
90
100
More Robust Solutions
Picking a solution with some restrictions
yields good results.
3.5
Actual
Measured
MinMin
MinBE
3
2.5
2
Signal

1.5
1
0.5
0
-0.5
0
10
20
30
40
50
Sample
60
70
80
90
100
Conclusions
Least Squares has nice properties and
generally works well.
 Problems can arise in simple problems.

–
Fundamental errors
Must account for errors in basic system.
 Robust ~ works well for all nearby systems

–
Can’t do as well or as bad (compromise)