Transcript Class 2 - UNC Computer Science

Computer
Vision
Cameras, lenses and sensors
Marc Pollefeys
COMP 256
Computer
Vision
Cameras, lenses and sensors
• Camera Models
– Pinhole Perspective Projection
– Affine Projection
• Camera with Lenses
• Sensing
• The Human Eye
Reading: Chapter 1.
Computer Images are two-dimensional patterns of brightness values.
Vision
Figure from US Navy Manual of Basic Optics and Optical Instruments, prepared by Bureau of
Naval Personnel. Reprinted by Dover Publications, Inc., 1969.
They are formed by the projection of 3D objects.
Computer
Vision
Animal eye:
a looonnng time ago.
Photographic camera:
Niepce, 1816.
Pinhole perspective projection: Brunelleschi, XVth Century.
Camera obscura: XVIth Century.
Computer
Vision
Distant objects appear smaller
Computer
Vision
Parallel lines meet
• vanishing point
Computer
Vision
Vanishing points
H VPL
VPR
VP2
VP1
To different directions
correspond different vanishing points
VP3
Computer
Vision
Geometric properties of projection
• Points go to points
• Lines go to lines
• Planes go to whole image
or half-plane
• Polygons go to polygons
• Degenerate cases:
– line through focal point yields point
– plane through focal point yields line
Computer
Vision
Pinhole Perspective Equation
x

 x'  f ' z

 y'  f ' y

z
Computer
Vision
Affine projection models:
Weak perspective projection
 x'  mx where
 y '  my

f'
m
z0
is the magnification.
When the scene relief is small compared its distance from the
Camera, m can be taken constant: weak perspective projection.
Computer
Vision
Affine projection models:
Orthographic projection
 x'  x

 y'  y
When the camera is at a
(roughly constant) distance
from the scene, take m=1.
Computer
Vision
Planar pinhole
perspective
Orthographic
projection
Spherical pinhole
perspective
Computer
Vision
Limits for pinhole cameras
Computer
Vision

Camera obscura + lens
Computer
Vision
Lenses
Snell’s law
n1 sin a1 = n2 sin a2
Descartes’ law
Computer
Vision
Paraxial (or first-order) optics
α1  β1  γ 
α2  γ  β2 
h h

d1 R
h h

R d2
Snell’s law:
Small angles:
n1 sin a1 = n2 sin a2
n1 a1  n2a2
 h h
h h
n1     n2   
 d1 R 
 R d2 
n1 n2 n2  n1


d1 d 2
R
Computer
Vision
n1 n2 n2  n1
 
d1 d 2
R
Thin Lenses
spherical lens surfaces; incoming light  parallel to axis;
thickness << radii; same refractive index on both sides
1 n n 1


Z Z*
R
n 1 1 n
 
Z* Z'
R
n n 1 1


Z*
R
Z
n 1 n 1


Z*
R
Z'
n 1 1  n 1 1

 
R
R
Z Z'
1 1 1
 
z' z f
R
and f 
2(n  1)
Computer
Vision
Thin Lenses
x

 x'  z ' z

y
 y'  z'
z

wher e
1 1 1
 
z' z f
R
and f 
2(n  1)
http://www.phy.ntnu.edu.tw/java/Lens/lens_e.html
Computer
Vision
Thick Lens
Computer
Vision

The depth-of-field
Computer
Vision
The depth-of-field
yields

Z
1
1
i 1
Zo  f  
Zo
ZiZ i  ff
f Zo
Zi 
Zo  f
/ (
dZ
 ib)
Zii  ZZi id 
d Zo

Zo  f


Z


Z
b
b Z 0 Zf i (d
  b) i
Z 
Zi

d  db
Z o (Z o  f )


Zo  Zo  Zo 
Z0  f d / b  f
b
i
Similar formula for Zo  Zo  Zo

i
Computer
Vision
The depth-of-field
Z 0 (Z 0  f )
Z  Z 0  Z 
Z0  f d / b  f

0


0
decreases with d, increases with Z0
strike a balance between incoming light and
sharp depth range
Computer
Vision
Deviations from the lens model
3 assumptions :
1. all rays from a point are focused onto 1 image point
2. all image points in a single plane
3. magnification is constant
deviations from this ideal are aberrations

Computer
Vision
Aberrations
2 types :
1. geometrical
2. chromatic
geometrical : small for paraxial rays
study through 3rd order optics

chromatic : refractive index function of
wavelength
Computer
Vision
Geometrical aberrations
 spherical aberration
 astigmatism
 distortion
 coma
aberrations are reduced by combining lenses

Computer
Vision
Spherical aberration
rays parallel to the axis do not converge
outer portions of the lens yield smaller
focal lenghts

Computer
Vision
Astigmatism
Different focal length for inclined rays
Computer
Vision
Distortion
magnification/focal length different
for different angles of inclination
pincushion
(tele-photo)
barrel
(wide-angle)
Can be corrected! (if parameters are know)
Computer
Vision
Coma
point off the axis depicted as comet shaped blob
Computer
Vision
Chromatic aberration
rays of different wavelengths focused
in different planes
cannot be removed completely
sometimes achromatization is achieved for
more than 2 wavelengths

Computer
Vision
Lens materials
reference wavelengths :
F = 486.13nm
d = 587.56nm
C = 656.28nm
lens characteristics :
1. refractive index nd
2. Abbe number Vd= (nd - 1) / (nF - nC)
typically, both should be high
allows small components with sufficient refraction
notation : e.g. glass BK7(517642)
nd = 1.517 and Vd= 64.2

Computer
Vision
Lens materials
Crown Glass
Fused Quartz & Fused Silica
Calcium Fluoride
9000
Germanium 14000
Zinc Selenide
Saphire
18000
6000
Plastic (PMMA)
100
200
400
600
800
1000 1200 1400 1600 1800 2000 2200 2400
WAVELENGTH (nm)
additional considerations :
humidity and temperature resistance, weight, price,...

Computer
Vision
Vignetting
Figure from http://www.vanwalree.com/optics/vignetting.html
Computer
Vision
Photographs
(Niepce,
“La Table Servie,” 1822)
Collection Harlingue-Viollet.
Milestones:
Daguerreotypes (1839)
Photographic Film (Eastman,1889)
Cinema (Lumière Brothers,1895)
Color Photography
(Lumière Brothers, 1908)
Television
(Baird, Farnsworth, Zworykin, 1920s)
CCD Devices (1970)
more recently CMOS
Computer
Vision
Cameras
we consider 2 types :
1. CCD
2. CMOS

Computer
Vision
CCD
separate photo sensor at regular positions
no scanning
charge-coupled devices (CCDs)
area CCDs and linear CCDs
2 area architectures :
interline transfer and frame transfer
photosensitive
storage

Computer
Vision
The CCD camera
Computer
Vision
CMOS
Same sensor elements as CCD
Each photo sensor has its own amplifier
More noise (reduced by subtracting ‘black’ image)
Lower sensitivity (lower fill rate)
Uses standard CMOS technology
Allows to put other components on chip
‘Smart’ pixels
Foveon
4k x 4k sensor
0.18 process
70M transistors
Computer
Vision
CCD vs. CMOS
•
•
•
•
Mature technology
Specific technology
High production cost
High power
consumption
• Higher fill rate
• Blooming
• Sequential readout
•
•
•
•
•
•
•
•
•
Recent technology
Standard IC technology
Cheap
Low power
Less sensitive
Per pixel amplification
Random pixel access
Smart pixels
On chip integration
with other components
Computer
Vision
Color cameras
We consider 3 concepts:
1. Prism (with 3 sensors)
2. Filter mosaic
3. Filter wheel
… and X3
Computer
Vision
Prism color camera
Separate light in 3 beams using dichroic prism
Requires 3 sensors & precise alignment
Good color separation
Computer
Vision
Prism color camera
Computer
Vision
Filter mosaic
Coat filter directly on sensor
Demosaicing (obtain full colour & full resolution image)
Computer
Vision
Filter wheel
Rotate multiple filters in front of lens
Allows more than 3 colour bands
Only suitable for static scenes
Computer
Vision
Prism vs. mosaic vs. wheel
approach
# sensors
Separation
Cost
Framerate
Artefacts
Bands
Prism
3
High
High
High
Low
3
Mosaic
1
Average
Low
High
Aliasing
3
Wheel
1
Good
Average
Low
Motion
3 or more
High-end
cameras
Low-end
cameras
Scientific
applications
Computer
Vision
new color CMOS sensor
Foveon’s X3
better image quality
smarter pixels
Computer
Vision
Reproduced by permission, the American Society of Photogrammetry and
Remote Sensing. A.L. Nowicki, “Stereoscopy.” Manual of Photogrammetry,
Thompson, Radlinski, and Speert (eds.), third edition, 1966.
The Human Eye
Helmoltz’s
Schematic
Eye
Computer
Vision
The distribution of
rods and cones
across the retina
Reprinted from Foundations of Vision, by B. Wandell, Sinauer
Associates, Inc., (1995).  1995 Sinauer Associates, Inc.
Cones in the
fovea
Rods and cones in
the periphery
Reprinted from Foundations of Vision, by B. Wandell, Sinauer
Associates, Inc., (1995).  1995 Sinauer Associates, Inc.
Computer
Vision
Next class
Radiometry: lights and surfaces