Transcript Image Processing

• Light and the EM spectrum
• The H.V.S. and Color Perception
1
What is an Image ?
• An image is a projection of a 3D scene into a 2D
projection plane.
• An image can be defined as a 2 variable function I(x,y) ,
where for each position (x,y) in the projection plane,
I(x,y) defines the light intensity at this point.
2
Camera trial #1
scene
film
Put a piece of film in front of an object.
3
source: Yung-Yu Chuang
Pinhole camera
pinhole camera
scene
4
barrier
film
Add a barrier to block off most of the rays.
• It reduces blurring
• The pinhole is known as the aperture
• The image is inverted
source: Yung-Yu Chuang
5
The Pinhole Camera Model (where)
(x,y)
Y
d
X
(x,y,z)
center of
projection
(pinhole)
d – focal length
6
0
 x  1 0
  
0
 y   0 1
 w  0 0 1/ d
  
X
0  
 Y 
0  
Z

0  
1
d
Z
The Shading Model (what)
Shading Model: Given the illumination incident at a point
on a surface, what is reflected?
7
Shading Model Parameters
• The factors determining the shading effects are:
– The light source properties:
• Positions, Electromagnetic Spectrum, Shape.
– The surface properties:
• Position, orientation, Reflectance properties.
– The eye (camera) properties:
• Position, orientation, Sensor spectrum sensitivities.
8
The Light Properties
9
Newton’s Experiment, 1665 Cambridge.
Discovering the fundamental spectral components of light.
(from Foundations of Vision: Brian Wandell, 1995.
A prism
10
Electromagnetic Radiation - Spectrum
11
Wavelength in nanometers (nm)
Electromagnetic Wave
12
Monochromators
Monochromators measure the power or energy
at different wavelengths
13
Spectral Power Distribution (SPD)
The Spectral Power Distribution (SPD) of a light is a
function e() which defines the relative energy at each
wavelength.
Relative Power
1
0.5
0
400
500
600
Wavelength ()
14
700
Examples of Spectral Power Distributions
1
1
0.5
0.5
0
400
500
600
700
0
Blue Skylight
1
0.5
0.5
15
400
500
600
700
Red monitor phosphor
500
600
700
Tungsten bulb
1
0
400
0
400
500
600
700
Monochromatic light
The Surface Properties
Reflected
Light
Incoming
Light
Transmitted Light
• Interactions between light and matter depends on the
physical characteristics of light as well as the matter.
• Three types of interactions:
– Reflection
– Absorption
– Transmittance
16
The Bidirectional Reflectance
Distribution Function (BRDF)
• A BRDF describes how much light is reflected
when light makes contact with a certain material
Spectral radiance: quantity of
light reflected in direction (e,e)
L( e , e ,  )
BRDF 
E (i , i ,  )
Spectral irradiance: quantity of
light arriving from direction (i,i)
17
Simplified Model
Incident light normal

Specular reflection
Diffuse reflection
Diffuse (lambertian) reflection
Reflected randomly between color particles
reflection is equal in all directions.
Specular reflection
mirror like reflection at the surface
18
Different Types of Surfaces
19
Simplified rendering models: reflectance
Often are more interested in relative spectral
composition than in overall intensity, so the
spectral BRDF computation simplifies a
wavelength-by-wavelength multiplication of
relative energies.
.*
B. Freeman, and Foundations of Vision, by Brian Wandell,
=
Spectral Property of Lambertian Surfaces
Yellow
Red
1
1
0.8
0.8
0.6
0.6
0.4
0.4
0.2
0.2
400
500
600
700
400
500
Blue
1
0.8
0.8
0.6
0.6
0.4
0.4
0.2
0.2
500
600
700
Gray
1
400
600
700
400
500
600
700
Wavelength (nm)
21
Surface Body Reflectances (albedo)
Some reflectance spectra
Forsyth, 2002
The Eye Properties
Lens
Cornea
Pupil
Iris
Fovea
Optic Nerve
Vitreous
Humor
Optic Disc
Retina
Ocular Muscle
 Cornea - ‫קרנית‬
 Pupil - ‫אישון‬
 Iris - ‫קשתית‬
23 Retina - ‫רשתית‬
24
The Visual Pathway
Retina
Optic Nerve
Optic Chiasm
Lateral
Geniculate
Nucleus (LGN)
Visual Cortex
25
Eye v.s. Camera
26
Yaho Wang’s slides
The Human Retina
cones
rods
horizontal
bipolar
amacrine
ganglion
light
27
• Retina contains 2 types of photo-receptors
– Cones:
• Day vision, can perceive color tone
– Rods:
• Night vision, perceive brightness only
28
Cones:
• High illumination levels (Photopic vision)
• Sensitive to color (there are three cone types: L,M,S)
• Produces high-resolution vision
• 6-7 million cone receptors, located primarily in the central
portion of the retina
Relative sensitivity
Cone Spectral Sensitivity
1
L
M
M
S
0.75
0.5
0.25
0
29
400
500
600
Wavelength (nm)
700
A side note:
• Humans and some monkeys have three
types of cones (trichromatic vision); most
other mammals have two types of cones
(dichromatic vision).
• Marine mammals have one type of
cone.
• Most birds and fish have four types.
•Lacking one or more type of cones
result in color blindness.
Rods:
• Low illumination levels (Scotopic vision).
•
•
•
•
Highly sensitive (respond to a single photon).
Produces lower-resolution vision
100 million rods in each eye.
No rods in fovea.
Relative sensitivity
Rod Spectral Sensitivity
1
0.75
0.5
0.25
0
30
400
500
600
Wavelength (nm)
700
Photoreceptor Distribution
Foveal Periphery photoreceptors
31
rods
S - Cones
L/M - Cones
Cone Receptor Mosaic
(Roorda and Williams, 1999)
32
L-cones
M-cones
S-cones
Cone’s Distribution:
• L-cones (Red) occur at about ~65% of the cones throughout the retina.
• M-cones (green) occur at about ~30% of the cones.
• S-cones (blue) occur at about ~2-5% of the cones (Why so few?).
Receptors per square mm
18
x 10
4
rods
cones
14
Distribution of rod
and cone
photoreceptors
10
6
2
-60
33
-40
-20
0
fovea
20
40
60
Degrees of Visual Angle
The Cone Responses
Assuming Lambertian Surfaces
Sensors
Illuminant
L   l ( )e( ) k ( )
Output
M   m ( )e( ) k ( )
S   s ( )e( ) k ( )
34
e() – Fixed, point source illuminant
k() –surface’s reflectance
l(),m(),s() – Cone responsivities
Surface
Metamer - two lights that appear the same visually.
They might have different SPDs (spectral power distributions).
Tungsten light
Monitor emission
800
Power
200
400
100
0
400
500
600
700
0
400
500
600
700
Wavelength (nm)
35
The phosphors of the monitor were set to match
the tungsten light.
The Trichromatic Color Theory
Trichromatic: “tri”=three “chroma”=color
color vision is based on three primaries (i.e., it is 3D).
Thomas Young (1773-1829) A few different retinal receptors operating with different
wavelength sensitivities will allow humans to perceive
the number of colors that they do.
Suggested 3 receptors.
Helmholtz & Maxwell (1850) Color matching with 3 primaries.
36
Color Matching Experiment
• Given a set of 3 primaries, one can determine for every spectral
distribution, the intensity of the guns required to match the color of
that spectral distribution.
•
The 3 numbers can serve as a color representation.
test
match
T()
Primaries
37
+
-
R()
+
-
G()
+
-
B()
T    rR   gG   bB 
Color matching experiment 1
38
from: Bill Freeman
Color matching experiment 1
p1 p2
p
39 3
from: Bill Freeman
Color matching experiment 1
p1 p2
p
40 3
from: Bill Freeman
Color matching experiment 1
The primary color
amounts needed
for a match
p1 p2
p
41 3
from: Bill Freeman
Color matching experiment 2
42
from: Bill Freeman
Color matching experiment 2
p1 p2
p
43 3
from: Bill Freeman
Color matching experiment 2
p1 p2
p
44 3
from: Bill Freeman
Color matching experiment 2
We say a
“negative”
amount of p2
was needed to
make the match,
because we
added it to the
test color’s side.
p1 p2
p3
The primary color
amounts needed
for a match:
p1 p2
p3
p1 p2
45 3
p
from: Bill Freeman
Color matching experiment for Monochromatic lights
1
1
0.5
0
0.5
400 500 600 700
Primary Intensities
46
1
0
0.5
400 500 600 700
0
400 500 600 700
The Color Matching Functions (CMF)
Primary Intensity
3
r()
2
1
b()
g()
0
400
500
600
700
Wavelength (nm)
Stiles & Burch (1959) Color matching functions. Primaries are: 444.4
525.3 and 645.2
Problems: Some perceived colors cannot be generated. This is true for
47 any choice of visible primaries.
The superposition principle
48
Foundations of Vision, by Brian Wandell, Sinauer Assoc., 1995
from: Bill Freeman
• Observation - Color matching is linear:
– if (SP) then (S+NP+N)
– if (SP) then ( S  P)
• Let T()=c(-0)+d(-1) a double chromatic
color: How should we adjust the 3 primaries?
d
c
0
1
r  c r 0   dr 1  ; g  c g 0   dg 1  ; b  c b 0   db 1 
49
• Outcome 1: Any T() can be matched:
r   T   r   d ; g   T   g   d ; b   T  b   d
• Outcome 2: CMF can be calculated for any chosen
primaries U(), V(), W():
 u   a1 a2
  
 v    b1 b2
 w  c c
2
   1
50
a3  r 
 
b3  g 
c3  b 
The CIE Color Standard
• The CIE (Commission Internationale
d’Eclairage) defined in 1931 three hypothetical
lights X, Y, and Z whose matching functions are
positive everywhere:
51
Tristimulus
• Let X, Y, and Z be the tri-stimulus values.
• A color can be specified by its trichromatic
coefficients, defined as
x
X
X Y  Z
X ratio
y
Y
X Y  Z
Y ratio
z
Z
X Y  Z
Z ratio
Two trichromatic coefficients are enough to specify
a color (x + y + z = 1).
From: Bahadir Gunturk
52
CIE Chromaticity Diagram
Input light spectrum
y
x
From: Bahadir Gunturk
53
CIE Chromaticity Diagram
Input light spectrum
y
x
From: Bahadir Gunturk
54
CIE Chromaticity Diagram
Input light spectrum
y
x
From: Bahadir Gunturk
55
CIE Chromaticity Diagram
Input light spectrum
y
700nm
Boundary
380nm
x
From: Bahadir Gunturk
56
CIE Chromaticity Diagram
Input light spectrum
Boundary
From: Bahadir Gunturk
57
CIE Chromaticity Diagram
Light composition
From: Bahadir Gunturk
58
CIE Chromaticity Diagram
Light composition
Light composition
From: Bahadir Gunturk
59
The sRGB Color Standard
• The sRGB is a device-independent color space. It was created
in 1996 by HP and Microsoft for use on monitors and printers.
• It is the most commonly used color space.
• It is defined by a transformation from the xyz color space.
60
Color matching predicts matches,
not appearance
61
Color Appearance
62
Color Appearance
63
Color Appearance
64
Color Spaces
65
RGB Color Space (additive)
• Define colors with (r, g, b) ; amounts of red,
green, and blue
66
CMY Color Space (subtractive)
• Cyan, magenta, and yellow are the complements of
red, green, and blue
– We can use them as filters to subtract from white
– The space is the same as RGB except the origin is white
instead of black
67
500
600
700 nm
400
500
600
700 nm
400
500
600
700 nm
400
500
600
700 nm
400
500
600
700 nm
400
500
600
700 nm
blue
yellow
green
400
magenta
red
cyan
Color names for cartoon spectra
From: B. Freeman
red
Additive color mixing
500
600
700 nm
400
500
600
700 nm
yellow
green
400
When colors combine by
adding the color spectra.
Example color displays that
follow this mixing rule: CRT
phosphors, multiple projectors
aimed at a screen, Polachrome
slide film.
Red and green make…
Yellow!
400
500
600
700 nm
cyan
Subtractive color mixing
500
600
700 nm
yellow
400
500
600
700 nm
green
400
When colors combine by
multiplying the color spectra.
Examples that follow this
mixing rule: most photographic
films, paint, cascaded optical
filters, crayons.
Cyan and yellow (in crayons,
called “blue” and yellow)
make…
Green!
400
500
600
700 nm
71
Red
Yellow
Magenta
Green
72
Blue
Cyan
HSV color space
• Hue - the chroma we see (red, green, purple).
• Saturation - how pure is the color (how far the color from
gray ).
• Value (brightness) - how bright is the color.
73
HSV color space
Saturation
Value
Hue
74
THE END
75