Transcript Photo = Illusion

Light Field = Array of (virtual) Cameras
Sub-aperture
Virtual Camera =
Sub-aperture View

Marc Levoy
MERL
Mask-Enhanced Cameras: Heterodyned Light Fields & Coded Aperture
Veeraraghavan, Raskar, Agrawal,
Mohan & Tumblin
Sensor
Sensor
Microlens
array
Mask
Plenoptic Camera
Heterodyne Camera
•
Samples individual rays
•
Samples coded combination of rays
•
•
Predefined spectrum for lenses
Chromatic abberration
•
Supports any wavelength
•
High alignment precision
•
Reconfigurable f/#, Easier alignment
•
Peripheral pixels wasted pixels
•
•
No wastage
High resolution image for parts
of scene in focus
•
Negligible Light Loss
•
50 % Light Loss due to mask
x1
x2
x1’ = x1 + θi*z
θi
θj
θj
Shear of Light Field
θ
θi
θ
θj
x2
l(x,θ)
x1
x
l(x,θ)
x
x1
x'1
Light Propagation (Defocus Blur)
θ
l(x,θ)
2-D FFT
L(fx,fθ)
fθ
x
fx
Central Slice
Line Integral
1-D FFT
Captured
Photo
FFT of
Captured
Photo
Space of LF representations
Time-frequency representations
Phase space representations
Quasi light field
Other LF
representations
Other LF
representations
Rihaczek
Distribution
Function
Observable LF
Augmented
LF
Traditional
light field
incoherent
coherent
WDF
Quasi light fields
the utility of light fields, the versatility of Maxwell
We form coherent images by
Other LF
representatio
ns
Other LF
representatio
ns
Rihaczek
Distribution
Function
Observable LF
Augmente
d LF
WDF
Traditiona
l light field
incoherent
formulating,
coherent
capturing,
and integrating
quasi light fields.
(i) Observable Light Field
•
•
•
move aperture
across plane
look at
directional
spread
continuous
form of
plenoptic
camera
scene
aperture
position s
direction u
(ii) Augmented Light Field with
LF Transformer
light field
transformer
WDF
Augmented
LF
LF
LF
LF
LF
negative
radiance
(diffractive)
optical
element
Light
Field
LF
propagation
LF
propagation
Interaction at the optical elements
8
Virtual light projector with real valued
(possibly negative radiance) along a ray
real projector
first null
(OPD = λ/2)
virtual light projector
real projector
9
(ii) ALF with LF Transformer
1
(iii) Rihaczek Distribution Function
Tradeoff between cross-interference terms
and localization
u
y
(i) Spectrogram
non-negative
localization
(ii) Wigner
localization
cross terms
(iii) Rihaczek
localization
complex
3m
u
0m
0m
y
3m
0m
y
3m
0m
y
3m
Property of the Representation
Constant
along rays
Non-negativity
Coherence
Wavelength
Interference
Cross term
Traditional LF
always
constant
always
positive
only
incoherent
zero
no
Observable
LF
nearly
constant
always
positive
any
coherence
state
any
yes
Augmented
LF
only in the
paraxial
region
positive and
negative
any
any
yes
WDF
only in the
paraxial
region
positive and
negative
any
any
yes
complex
any
any
reduced
Rihaczek DF no; linear drift
Benefits & Limitations of the
Representation
Simplicity of Adaptability
Ability to
Modeling
computatio to current Near Field
propagate wave optics
n
pipe line
Traditional
LF
no
very simple
high
no
yes
Observable
not x-shear
LF
yes
modest
low
yes
yes
Augmented
LF
x-shear
yes
modest
high
no
yes
WDF
x-shear
yes
modest
low
yes
yes
yes
better than
WDF, not
as simple
as LF
low
no
yes
Rihaczek
DF
x-shear
Far Field
x-shear
Motivation
• What is the difference between a hologram
and a lenticular screen?
• How they capture ‘phase’ of a wavefront for
telescope applications?
• What is ‘wavefront coding’ lens for extended
depth of field imaging?
Application - Wavefront Coding
Dowski and Cathey 1995
point
in scene
cubic
phase plate
small change
in blur shape
same aberrant blur regardless of depth of focus
Can they be part of Computer Vision?
Moving away from 2D images or 4D lightfields?
Wavefront coding:
WLC mobile phone cameras
Holography:
Reference targets
Rendering:
New perspective
projection methods
QuickTime™ and a
BMP decompressor
are needed to see this picture.
Gaussian beam lasers:
Modern active illumination
Rotating PSF:
Depth from defocus
Raskar, Camera Culture, MIT Media Lab
Computational Photography
http://computationalphotography.org
1.
Epsilon Photography
–
Low-level Vision: Pixels
– Multiphotos by bracketing (HDR, panorama)
–
‘Ultimate camera’
2.
Coded Photography
–
Mid-Level Cues:
• Regions, Edges, Motion, Direct/global
–
Single/few snapshot
• Reversible encoding of data, Lightfield
–
Additional sensors/optics/illum
–
‘Smart Camera’
3.
Essence Photography
–
–
–
Not mimic human eye
Beyond single view/illum
‘New artform’
Resources
Rihaczek
Distribution
Function
Observable LF
Augmented
LF
Traditional
light field
• Website
– http://scripts.mit.edu/~raskar/lightfields/
– Or follow http://cvpr2009.org tutorial pages
• Key new papers
•
•
•
Wigner Distributions and How They Relate to the Light Field
Zhengyun Zhang and Marc Levoy, ICCP 2009 (best paper)
Augmenting Light Field to Model Wave Optics Effects ,
Se Baek Oh, Barbastathis, Raskar (in Preparation)
Quasi light fields: extending the light field to coherent radiation ,
Anthony Accardi, Wornell (in Preparation)
WDF
Acknowledgements
• Darthmuth
– Marcus Testorf,
• MIT
– Ankit Mohan, Ahmed Kirmani, Jaewon Kim
– George Barbastathis
• Stanford
– Marc Levoy, Ren Ng, Andrew Adams
• Adobe
– Todor Georgiev,
• MERL
– Ashok Veeraraghavan, Amit Agrawal
MIT Media Lab
Light Fields___
Camera Culture
Ramesh Raskar
MIT Media Lab
http:// CameraCulture . info/
Light Fields in Ray and Wave Optics
Introduction to Light Fields:
Ramesh Raskar
Wigner Distribution Function to explain Light Fields:
Zhengyun Zhang
Augmenting LF to explain Wigner Distribution Function:
Se Baek Oh
Q&A
Break
Light Fields with Coherent Light:
Anthony Accardi
New Opportunities and Applications:
Raskar and Oh
Q&A:
All