Transcript PowerPoint Slides (2.1 Mb ppt)

A Stereo Image
The space/geometry of
The space of all stereo images
all stereo/epipolar images/cameras
•S. Seitz, J. Kim, The Space of All Stereo Images, IJCV 2002 / ICCV 2001.
•T. Pajdla, Epipolar Geometry of Some Non-Classical Cameras, Slovenian
Computer Vision Winter Workshop, 2001.
or
The space (geometry) of ray sets (cameras)
that allow row-based stereo analysis
Outline
1) Regular stereo
2) Generalized / non-classical
camera
3) Stereo with generalized
cameras (an example)
4) When can we compute stereo
from generalized cameras?
A Stereo Image
Stereoscopic illusion
Real world
Stereo Illusions
Stereoscopic illusion
Stereoscopic Imaging
Key property: horizontal parallax
• Enables stereoscopic viewing and stereo algorithms
Thank you, Steve Seitz
Capturing stereo pairs
Left
camera
Right
camera
Display different image for each eye
1) Separating with vertical paper
2) Color glasses
3) Polarized glasses
4) Temporally synchronized
screen and glasses
0
p
Parallax and disparity
Original images
Aligned images
L
R
1
row
Z=0
Z=1
Alignment  Measure depth with respect
to this plane
Z=2
Z=1
Z=2
Z=0
L
R
Z
Parallax (3D)
L
R
Disparity (1D)
Displaying disparity
1
disparity 
depth
Stereo Algorithms
Photogrammetry (generating maps)
Stereoscopic Imaging
Key property: horizontal parallax
• Enables stereoscopic viewing and stereo algorithms
Thank you, Steve Seitz
Rectification
or
Why we can focus on 1-row
Line corresponds to line
(“epipolar lines”)
Rectified images: epipolar lines are image rows
Homography
?
?
Rectification (cont.)
Outline
1) Regular stereo
2) Generalized / non-classical
camera
3) Stereo with generalized
cameras (an example)
4) When can we compute stereo
from generalized cameras?
Geometric camera model
We model camera as
Mapping: world points  image points (pixels)

We model only the “projection” operation

We do not model: light, color, lens blurring, etc.
Example: pinhole camera model
(usual camera)
Important property:
every ray from the ray set of the camera
projects to one point
This projection operation is commonly described
by a 3x4 projective matrix.
For our purposes the following is more convenient:
A pinhole camera is defined by:
• set of rays (starting from the camera center)
• mapping from this rays to the image plane
Generalization of the camera model
Classical camera

One center of projection

Image surface (film) is planar
Generalized (ray-projective) camera

Multiple centers of projection
(origins of rays)

Image surface is arbitrary
A generalized camera maps rays to image points
For our purposes
camera  set of rays
Xerox machine
(non-classical cameras, example 1)
As a multi-prospective camera:
Existing Immersive Technology: Imax®
Can Not Combine Full FOV and Stereopsis
• Imax 3D
•
Incredibly realistic threedimensional images are projected
onto the giant IMAX screen with
such realism that you can hear the
audience gasp as they reach out to
grab the almost touchable images.
Imax Dome •
•
Experience wraps the audience in
images of unsurpassed size and
impact, providing an amazing sense
of involvement.
thanks to Shmuel Peleg, Hebrew University of Jerusalem
Pushbroom camera
(non-classical cameras, example 2)
•1D projective sensor
•… translating
Advantage: large field of
view in one dimension
Pushbroom camera
(non-classical cameras, example 2)
The generalized camera model:
y
x
X direction - parallel projection
Y direction - perspective projection
Notion Generator – the set of all ray origins
For other cameras, generator can be a 2D surface
Pushbroom camera
(non-classical cameras, example 2)
Imaging process
Images from usual camera
t
t+1
Image from generalized camera
Virtual generalized camera = device (usual camera) + software
thanks to Shmuel Peleg, Hebrew University of Jerusalem
t+2
Non-classical cameras
• Implementation through cuts of
3D video arrays
•Take images while moving a usual camera
•Stack them into 3D array
•Take a cut along the “time” dimension
t
Circular projective camera
(non-classical cameras, example 3)
Move “1D sensor” along a circle
record on a cylinder
Advantage: complete 360° horizontal view
thanks to Shmuel Peleg, Hebrew University of Jerusalem
Circular projective camera
(non-classical cameras, example 3)
The generalized camera model:
Note: Generator is a circle
Outline
1) Regular stereo
2) Generalized / non-classical
camera
3) Stereo with generalized
cameras (an example)
4) When can we compute stereo
from generalized cameras?
Generalized stereo: an example
Inward-facing camera, moving around an object
thanks to Steve Seitz, University of Washington
Images for both eyes
• Input: video sequence
• Output: 2 symmetric cuts of 3D video array
thanks to Steve Seitz, University of Washington
Results: red-blue stereo image
thanks to Steve Seitz, University of Washington
Results: 3D reconstruction
Using usual algorithm (built for usual camera)
with non-classical images
thanks to Steve Seitz, University of Washington
Ray geometry
How does the set of rays look?
Pixel = ray
There is a “blind” area in the
center of the scene
Ray geometry
What rays go through a point in the scene?
How disparity depends on depth?
Outline
1) Regular stereo
2) Generalized / non-classical
camera
3) Stereo with generalized
cameras (an example)
4) When can we compute stereo
from generalized cameras?
Parallax and disparity in Cyclographs
(review)
Choose “ground” plane Z=0
L
R
PARRALAX / DISPARITY
L
R
Z=0
Z=1
Z=2
The generalized camera model

D - image surface; P – ray space

A view (camera) is a function V: DP
D
Does not include:
• Multiple rays to 1-image point
• Curved light paths (mirror, lens)
A row (row-continuity)
A row is the set of points in one view image that
corresponds to a ray of the other view.
For rectified images: rows are image rows
Infinitesimality:
 Row has width  0
 Row has no holes
Stereo: basic constraints
Rays V(u1,v1) and V(u2,v2) intersect
v1=v2
D1
D2
(u1,v1)
(u2,v2)
Basic stereo constraints + row-continuity
Surfaces V(*,v1) and V(*,v2) , where v1, and v2 are
corresponding rows,
intersect in a surface(not a curve)
D1
D2
(u1,v1)
(u2,v2)
Example: The intersection of epipolar planes
The red plane ”intersects” the blue plane
Ruled surfaces:
Ruled surfaces: examples
• Generalized cylinder

Generalized cone
The most important slide
• Left camera “ruling” the scene
• Right camera “ruling” the scene
Doubly ruled surface
Doubly Ruled surfaces: examples
A plane
A hyperboloid
Theorem (D. Hilbert ):
The only doubly ruled surfaces are:
hyperboloid
hyperbolic paraboloid
plane
The Doubly Ruled Surfaces
of cyclograph
SUMMARY
The space (geometry)
of ray sets (cameras)
that allow row-based
stereo analysis
are doubly ruled surfaces
OmniStereo with Mirrors
Dynamic Scenes
mirror
viewing circle
optical center
Stereo Image
Spiral mirror acquiring
Spiral mirror acquiring
right eye panorama
left eye panorama
viewing circle
Optical center
Stereo Image