Transcript Physics of Image Formation

Basic Principles of Imaging and Lenses
Light
Electromagnetic
Radiation
Photons
Light
These three are the same…
• Light
* pure energy
• Electromagnetic Waves
* energy-carrying waves emitted by vibrating electrons
• Photons
* particles of light
EM Radiation Travels as a Wave
c = 3 x 108 m/s
EM Radiation Carries Energy
• Quantum mechanics tells us that for photons E = hf
where E is energy and h is Planck’s constant.
• But f = c/l
• Putting these equations together, we see that
E = hc/l
Electromagnetic Wave Velocity
•
The speed of light is the same for all seven forms of light.
•
It is 300,000,000 meters per second or 186,000 miles per second.
The Electromagnetic Spectrum
•
•
•
•
•
•
•
Radio Waves - communication
Microwaves - used to cook
Infrared - “heat waves”
Visible Light - detected by your eyes
Ultraviolet - causes sunburns
X-rays - penetrates tissue
Gamma Rays - most energetic
The Multi-Wavelength Sun
X-Ray
UV
Composite
Infrared
Visible
Radio
EM Spectrum Relative Sizes
The Visible Spectrum
Light waves extend in wavelength from about 400 to 700 nanometers.
A Brief History of Images
Camera Obscura, Gemma Frisius, 1558
1544
Camera Obscura
"When images of illuminated objects ... penetrate through a small hole into a
very dark room ... you will see [on the opposite wall] these objects in their
proper form and color, reduced in size ... in a reversed position, owing to the
intersection of the rays". Leonardo da Vinci
http://www.acmi.net.au/AIC/CAMERA_OBSCURA.html (Russell Naughton)
Slide credit: David Jacobs
A Brief History of Images
Lens Based Camera Obscura, 1568
1558
1568
Jetty at Margate England,
1898.
http://brightbytes.com/cosite/collection2.html (Jack and Beverly Wilgus)
Slide credit: David Jacobs
A Brief History of Images
1558
1568
1837
Still Life, Louis Jaques Mande Daguerre, 1837
A Brief History of Images
1558
1568
1840?
Abraham Lincoln?
A Brief History of Images
1558
1568
1837
Silicon Image Detector, 1970
1970
A Brief History of Images
1558
1568
1837
Digital Cameras
1970
1995
A Brief History of Images
1558
1568
1837
Hasselblad HD2-39
1970
1995
2006
Geometric Optics and Image Formation
Pinhole Cameras
•
•
•
Pinhole camera - box with a small hole in it
Image is upside down, but not mirrored left-to-right
Question: Why does a mirror reverse left-to-right but not top-to-bottom?
Pinhole and the Perspective Projection
Is an image being formed
on the screen?
(x,y)
YES! But, not a “clear” one.
screen
scene
image plane
r  ( x, y , z )
y
optical
axis
effective focal length, f’
z
pinhole
x
r '  ( x' , y ' , f ' )
r' r

f' z
x' x

f' z
y' y

f' z
Problems with Pinholes
•
Pinhole size (aperture) must be “very small” to obtain a clear image.
•
However, as pinhole size is made smaller, less light is received by image plane.
•
If pinhole is comparable to wavelength of incoming light, DIFFRACTION
effects blur the image!
•
Sharpest image is obtained when:
pinhole diameter d  2
f 'l
Example: If f’ = 50mm,
l
= 600nm (red),
d = 0.36mm
The Reason for Lenses
Image Formation using (Thin) Lenses
•
Lenses are used to avoid problems with pinholes.
•
Ideal Lens: Same projection as pinhole but gathers more light!
o
i
P
P’
f
Gaussian Lens Formula:
1 1 1
 
i o f
• f is the focal length of the lens – determines the lens’s ability to bend (refract) light
• f different from the effective focal length f’ discussed before!
Focus and Defocus
aperture
Blur Circle,
aperture
diameter
b
d
o
i
i'
Gaussian Law:
1 1 1
 
i o f
o'
1 1 1
 
i ' o' f
Blur Circle Diameter :
(i 'i) 
b
f
f
(o  o' )
(o' f ) (o  f )
d
(i '  i )
i'
Depth of Field: Range of object distances over which image is sufficiently well focused,
i.e., range for which blur circle is less than the resolution of the imaging sensor.
Problems with Lenses
Compound (Thick) Lens
Vignetting
B
L3 L2 L1
principal planes


A
nodal points
thickness
Chromatic Abberation
more light from A than B !
Radial and Tangential Distortion
ideal
FB FG
FR
actual
ideal
actual
image plane
Lens has different refractive indices
for different wavelengths.
Spherical Aberration
Spherical lenses are the only easy shape to manufacture, but are not correct for perfect focus.
Two Lens System
d
object
final
image
f2
i2
o2
i1
f1
o1
image
plane
intermediate
virtual image
lens 2
lens 1
• Rule : Image formed by first lens is the object for the second lens.
• Main Rays : Ray passing through focus emerges parallel to optical axis.
Ray through optical center passes un-deviated.
• Magnification:
m
i2 i1
o2 o1
Exercises: What is the combined focal length of the system?
What is the combined focal length if d = 0?
Lens systems
• A good camera lens may
contain 15 elements and cost
a many thousand dollars
• The best modern lenses may
contain aspherical elements
Human Eye
•
The eye has an iris like a
camera
•
Focusing is done by changing
shape of lens
•
Retina contains cones (mostly
used) and rods (for low light)
•
The fovea is small region of
high resolution containing
mostly cones
•
Optic nerve: 1 million flexible
fibers
http://www.cas.vanderbilt.edu/bsci111b/eye/human-eye.jpg
Slide credit: David Jacobs
The Eye
• The human eye is a camera!
– Iris - colored annulus with radial muscles
– Pupil - the hole (aperture) whose size is controlled by the iris
– What’s the “film”?
• photoreceptor cells (rods and cones) in the retina
Human Eye vs. the Camera
• We make cameras that act “similar” to the human eye
Image Formation
Digital Camera
Film
The Eye
Insect Eye
We make cameras that act “similar” to the human eye
Fly
Mosquito
The Retina
Cross-section of eye
Cross section of retina
Pigmented
epithelium
Ganglion axons
Ganglion cell layer
Bipolar cell layer
Receptor layer
Retina up-close
Light
Two types of light-sensitive receptors
Cones
cone-shaped
less sensitive
operate in high light
color vision
Rods
rod-shaped
highly sensitive
operate at night
gray-scale vision
© Stephen E. Palmer, 2002
Rod / Cone sensitivity
The famous sock-matching problem…
Human Eye
• Rods
– Intensity only
– Essentially night vision and peripheral vision only
– Since we are trying to fool the center of field of view of human
eye (under well lit conditions) we ignore rods
Human Eye
• Cones
– Three types perceive different portions of the visible light
spectrum
Human Eye
• Because there are only 3 types of cones in human eyes,
we only need 3 stimulus values to fool the human eye
– Note: Chickens have 4 types of cones
Distribution of Rods and Cones
# Receptors/mm2
.
Fovea
150,000
Rods
Blind
Spot
Rods
100,000
50,000
0
Cones
Cones
80 60 40 20 0
20 40 60 80
Visual Angle (degrees from fovea)
Night Sky: why are there more stars off-center?
© Stephen E. Palmer, 2002
The Physics of Light
Some examples of the spectra of light sources
.
B. Gallium Phosphide Crystal
# Photons
# Photons
A. Ruby Laser
400 500
600
700
400 500
Wavelength (nm.)
700
Wavelength (nm.)
D. Normal Daylight
# Photons
C. Tungsten Lightbulb
# Photons
600
400 500
600
700
400 500
600
700
© Stephen E. Palmer, 2002
More Spectra
metamers
The Physics of Light
% Photons Reflected
Some examples of the reflectance spectra of surfaces
Red
400
Yellow
700 400
Blue
700 400
Wavelength (nm)
Purple
700 400
700
© Stephen E. Palmer, 2002
The Psychophysical Correspondence
There is no simple functional description for the perceived
color of all lights under all viewing conditions, but …...
A helpful constraint:
Consider only physical spectra with normal distributions
mean
area
# Photons
400
500
variance
600
700
Wavelength (nm.)
© Stephen E. Palmer, 2002
The Psychophysical Correspondence
# Photons
Mean
blue
Hue
green yellow
Wavelength
© Stephen E. Palmer, 2002
The Psychophysical Correspondence
# Photons
Variance
Saturation
hi. high
med. medium
low
low
Wavelength
© Stephen E. Palmer, 2002
The Psychophysical Correspondence
Area
Brightness
# Photons
B. Area
Lightness
bright
dark
Wavelength
© Stephen E. Palmer, 2002
Digital camera
• A digital camera replaces retina with a sensor array
– Each cell in the array is light-sensitive diode that converts photons to
electrons
– Two common types
• Charge Coupled Device (CCD)
• CMOS
– http://electronics.howstuffworks.com/digital-camera.htm
CCD Cameras
http://huizen.ddsw.nl/bewoners/maan/imaging/camera/ccd1.gif
Slide credit: David Jacobs
Sensor Array
CMOS sensor
Sampling and Quantization
Interlace vs. progressive scan
http://www.axis.com/products/video/camera/progressive_scan.htm
Progressive scan
http://www.axis.com/products/video/camera/progressive_scan.htm
Interlace
http://www.axis.com/products/video/camera/progressive_scan.htm
Color Sensing in Camera (RGB)
• 3-chip vs. 1-chip: quality vs. cost
• Why more green?
Why 3 colors?
http://www.cooldictionary.com/words/Bayer-filter.wikipedia
Practical Color Sensing: Bayer Grid
• Estimate RGB
at ‘G’ cels from
neighboring
values
http://www.cooldictionary.com/
words/Bayer-filter.wikipedia
Image Formation
f(x,y) = reflectance(x,y) * illumination(x,y)
Reflectance in [0,1], illumination in [0,inf]
White Balance
White World / Gray World assumptions
Problem: Dynamic Range
The real world has
High dynamic range
1
1500
25,000
400,000
2,000,000,000
Is Camera a photometer?
Image
pixel (312, 284) = 42
42 photos?
Long Exposure
10-6
Real world
High dynamic range
10-6
106
106
Picture
0 to 255
Short Exposure
10-6
Real world
High dynamic range
10-6
106
106
Picture
0 to 255
Image Acquisition Pipeline
Lens
scene
radiance
Shutter
sensor
irradiance
2
(W/sr/m )

sensor
exposure
Dt
CCD
ADC
analog
voltages
Remapping
digital
values
Camera is NOT a photometer!
pixel
values
Varying Exposure
What does the eye sees?
The eye has a huge dynamic range
Do we see a true radiance map?