Transcript CS 223-B Lecture 1 - Stanford University

AQUEOUS
HUMOR
CORNEA
CS 223-B
Lecture 1
Sebastian Thrun
Gary Bradski
http://robots.stanford.edu/cs223b/index.html
1
Readings
• Computer Vision, Forsyth and Ponce
– Chapter 1
• Introductory Techniques for 3D Computer
Vision, Trucco and Verri
– Chapter 2
2
Lenses and Cameras*
-- Brunelleschi, XVth Century
3
* Slides, where possible, stolen with abandon, many this lecture from Marc Pollefeys comp256, Lect 2
Distant objects appear smaller
A “similar triangle’s” approach to vision. Notes 1.1
4
Marc Pollefeys
Consequences: Parallel lines meet
• There exist vanishing points
5
Marc Pollefeys
Vanishing points
H VPL
VPR
VP2
VP1
Different directions correspond
to different vanishing points
6
VP3
Marc Pollefeys
Implications For Perception*
Same size things get smaller, we hardly notice…
Parallel lines meet at a point…
* A Cartoon Epistemology: http://cns-alumni.bu.edu/~slehar/cartoonepist/cartoonepist.html 7
Implications For Perception 2
Perception must be mapped to a space variant grid
Logrithmic in nature
8
Steve Lehar
The Effect of Perspective
9
Different Projections: Affine projection models:
Weak perspective projection
Smoosh everything flat onto a parallel plane at distance z0
 x'  mx where
 y '  my

f'
m
z0
is the magnification.
When the scene relief is small compared its distance from the
Camera, m can be taken constant: weak perspective projection.10
Marc Pollefeys
Affine projection models:
Orthographic projection
 x'  x

 y'  y
When the camera is at a
(roughly constant) distance
from the scene, take m=1.
11
Marc Pollefeys
Limits for pinhole cameras
12
Marc Pollefeys
On to Thin Lenses …
Snell’s law
n1 sin a1 = n2 sin a2
a 1 a1 a
q1
z2
d
b
F
q2
e
a2
Notes 1.2
13
Paraxial (or first-order) optics
α1  β1  γ 
α2  γ  β2 
h h

d1 R
h h

R d2
Snell’s law:
Small angles:
n1 sin a1 = n2 sin a2
n1 a1  n2a2
Sin a  a = y/r
Tan b  b = y/x
h h
h h 
n1     n2   
 d1 R 
 R d2 
n1 n2 n2  n1


d1 d 2
R
14
Marc Pollefeys
Thin Lenses
n1 n2 n2  n1


d1 d 2
R
spherical lens surfaces; incoming light  parallel to axis;
thickness << radii; same refractive index on both sides
1 n n 1


Z Z*
R
n 1 1 n
 
Z* Z'
R
n n 1 1


Z*
R
Z
n 1 n 1


Z*
R
Z'
R
and f 
2(n  1)
Notes 1.3 z->
15
8
n 1 1 n 1 1

 
R
R
Z Z'
1 1 1
 
z' z f
Marc Pollefeys
Thin Lenses summary
x

 x'  z ' z

y
 y'  z'
z

wher e
1 1 1
 
z' z f
Marc Pollefeys
R
and f 
2(n  1)
16
http://www.phy.ntnu.edu.tw/java/Lens/lens_e.html
The depth-of-field

17
Marc Pollefeys
The depth-of-field
yields

Z
1
1 i 1
Zo  f  
Zo
ZiZ i  ff
f Zo
Zi 
Zo  f
/ (dZbi)
Zii  ZZi id 
d Zo

Zo  f


Z


Z
b
b Z 0 Zf Zi (d
  b) i
i

Z

i
i
d b
b
d
Z o (Z o  f )


Zo  Zo  Zo 
Z0  f d / b  f
Similar formula for

Zo  Zo  Zo
18
Marc Pollefeys
The depth-of-field
Z 0 (Z 0  f )
Z  Z 0  Z 
Z0  f d / b  f

0


0
decreases with d, increases with Z0
strike a balance between incoming light and
sharp depth range. Notes 1.4
19
Marc Pollefeys
Deviations from the lens model
3 assumptions :
1. all rays from a point are focused onto 1 image point
• Remember thin lens small angle assumption
2. all image points in a single plane
f'
3. magnification m 
is constant
z0
Deviations from this ideal are aberrations

20
Marc Pollefeys
Aberrations
2 types :
1. geometrical
2. chromatic
geometrical : small for paraxial rays
study through 3rd order optics

chromatic : refractive index function of
wavelength
21
Marc Pollefeys
Geometrical aberrations
 spherical aberration
 astigmatism
 distortion
 coma
aberrations are reduced by combining lenses

22
Marc Pollefeys
Spherical aberration
rays parallel to the axis do not converge
outer portions of the lens yield smaller
focal lenghts

23
Marc Pollefeys
Astigmatism
Different focal length for inclined rays
24
Marc Pollefeys
Distortion
magnification/focal length different
for different angles of inclination
pincushion
(tele-photo)
barrel
(wide-angle)
Can be corrected! (if parameters are know)
25
Marc Pollefeys
Coma
point off the axis depicted as comet shaped blob
26
Marc Pollefeys
Chromatic aberration
rays of different wavelengths focused
in different planes
cannot be removed completely
sometimes achromatization is achieved for
more than 2 wavelengths

27
Marc Pollefeys
Vignetting
28
Marc Pollefeys
Calibration Gist:
Invert the image formation process
External
coordinate
system
(the camera will get several (K) views
of this grid in rotation)
Y
kth collection of points i
X
Z
pik   ( Pik , Rk , Tk , f , c,a , k ) where is theprojectionoperator
Pik
Note that rotation matrix R has constraints: determinant is 1, inverse
is equal to transpose, optimization routine should make use of this.
Image plane
Rk,Tk
Extrinsic Params
Rotation &
Translation
to image frame
coord. system
Camera
pik
y
[ Rk , Tk , f , c, a , k ]  Argmin  pik  pik
k
2
i
• This is typically solved through a gradient decent
optimization since the problem is manifestly convex.
z
0
Then we want the actual projection to be as close as possible to
The point given by the projection operator: p  p over all i points
ik
ik
and over all k images of grids:
x
f, c, a, k
Intrinsic Params
focus
center of image
Skew a = 0
k radial and tangential distortion
• Note that we need a good starting guess for the initial
“correct” projection points p’I the optimization then iterates to
solution.
• Stereo would then just double the parameters adding left l
and right r subscripts and additional summations over r & l.
29
Assumed Perspective Projection
32
Assumed Perspective Projection
33
Cameras
we consider 2 types :
1. CCD
2. CMOS

34
Marc Pollefeys
CCD
separate photo sensor at regular positions
no scanning
charge-coupled devices (CCDs)
area CCDs and linear CCDs
2 area architectures :
interline transfer and frame transfer
photosensitive
storage

35
Marc Pollefeys
The CCD camera
36
Marc Pollefeys
CMOS
Same sensor elements as CCD
Each photo sensor has its own amplifier
More noise (reduced by subtracting ‘black’ image)
Lower sensitivity (lower fill rate)
Uses standard CMOS technology
Allows to put other components on chip
‘Smart’ pixels
Foveon
4k x 4k sensor
0.18 process
70M transistors
37
Marc Pollefeys
CCD vs. CMOS
•
•
•
•
•
•
•
Mature technology
Specific technology
High production cost
High power consumption
Higher fill rate
Blooming
Sequential readout
•
•
•
•
•
•
•
•
•
Recent technology
Standard IC technology
Cheap
Low power
Less sensitive
Per pixel amplification
Random pixel access
Smart pixels
On chip integration
with other components
38
Marc Pollefeys
Colour cameras
We consider 3 concepts:
1. Prism (with 3 sensors)
2. Filter mosaic
3. Filter wheel
… and X3
39
Marc Pollefeys
Prism colour camera
Separate light in 3 beams using dichroic prism
Requires 3 sensors & precise alignment
Good color separation
40
Marc Pollefeys
Prism colour camera
41
Marc Pollefeys
Filter mosaic
Coat filter directly on sensor
Demosaicing (obtain full colour & full resolution image)
42
Marc Pollefeys
Filter wheel
Rotate multiple filters in front of lens
Allows more than 3 colour bands
Only suitable for static scenes
43
Marc Pollefeys
Prism vs. mosaic vs. wheel
approach
# sensors
Separation
Cost
Frame rate
Artifacts
Bands
Prism
3
High
High
High
Low
3
Mosaic
1
Average
Low
High
Aliasing
3
High-end
cameras
Low-end
cameras
Wheel
1
Good
Average
Low
Motion
3 or
Scientific
more
applications
44
Marc Pollefeys
new color CMOS sensor
Foveon’s X3
better image quality
smarter pixels
45
Marc Pollefeys
The Human Eye
Reproduced by permission, the American Society of Photogrammetry and
Remote Sensing. A.L. Nowicki, “Stereoscopy.” Manual of Photogrammetry,
Thompson, Radlinski, and Speert (eds.), third edition, 1966.
Cross section of the eye
Looking down the
optical axis of the
eye
46
Sensors and image processing
RGB + B/W happens here
Question: Which way does the light enter?
Light
Light
47
Eye cross section
48
The distribution of
rods and cones
across the retina
Reprinted from Foundations of Vision, by B. Wandell, Sinauer
Associates, Inc., (1995).  1995 Sinauer Associates, Inc.
Cones in the
fovea
Rods and cones in
the periphery
Reprinted from Foundations of Vision, by B. Wandell, Sinauer
Associates, Inc., (1995).  1995 Sinauer Associates, Inc.
49
There’s a lot more going on in Vision
…i.e. Light and Surfaces
50
Real vision includes invisible
inference
51
Real vision includes invisible
inference
52
Real vision includes invisible
inference
53