Transcript MultipleViews1

CREDITS
Rasmussen, UBC (Jim Little), Seitz (U. of Wash.), Camps
(Penn. State), UC, UMD (Jacobs), UNC, CUNY
Computer Vision : CISC 4/689
Multi-View Geometry
Relates
• 3D World Points
• Camera Centers
• Camera Orientations
Computer Vision : CISC 4/689
Multi-View Geometry
Relates
• 3D World Points
• Camera Centers
• Camera Orientations
• Camera Intrinsic Parameters
• Image Points
Computer Vision : CISC 4/689
Stereo
scene point
image plane
optical center
Computer Vision : CISC 4/689
Stereo
• Basic Principle: Triangulation
– Gives reconstruction as intersection of two rays
– Requires
• calibration
• point correspondence
Computer Vision : CISC 4/689
Stereo Constraints
p’ ?
p
Given p in left image, where can the corresponding point p’
in right image be?
Computer Vision : CISC 4/689
Stereo Constraints
M
Image plane
Y1
Epipolar Line
p
p’
Y2
Z1
O1
X2
X1
Focal plane
O2
Epipole
Computer Vision : CISC 4/689
Z2
Stereo
• The geometric information that relates two different
viewpoints of the same scene is entirely contained in a
mathematical construct known as fundamental matrix.
• The geometry of two different images of the same scene is
called the epipolar geometry.
Computer Vision : CISC 4/689
Stereo/Two-View Geometry
•
•
The relationship of two views of a
scene taken from different camera
positions to one another
Interpretations
– “Stereo vision” generally means
two synchronized cameras or eyes
capturing images
– Could also be two sequential views
from the same camera in motion
• Assuming a static scene
http://www-sop.inria.fr/robotvis/personnel/sbougnou/Meta3DViewer/EpipolarGeo
Computer Vision : CISC 4/689
3D from two-views
There are two ways of extracting 3D from a pair of images.
• Classical method, called Calibrated route, we need to calibrate both
cameras (or viewpoints) w.r.t some world coordinate system. i.e,
calculate the so-called epipolar geometry by extracting the essential
matrix of the system.
• Second method, called uncalibrated route, a quantity known as
fundamental matrix is calculated from image correspondences, and this
is then used to determine the 3D.
Either way, principle of binocular vision is triangulation. Given a single
image, the 3D location of any visible object point must lie on the
straight line that passes through COP and image point (see fig.).
Intersection of two such lines from two views is triangulation.
Computer Vision : CISC 4/689
Mapping Points between Images
• What is the relationship between the images x, x’ of the
scene point X in two views?
• Intuitively, it depends on:
– The rigid transformation between cameras (derivable from the
camera matrices P, P’)
– The scene structure (i.e., the depth of X)
• Parallax: Closer points appear to move more
Computer Vision : CISC 4/689
Example: Two-View Geometry
x3
x2
x’3
x1
x’2
x’1
courtesy of F. Dellaert
Is there a transformation relating the points
Computer Vision : CISC 4/689
xi to x’i ?
Epipolar Geometry
• Baseline: Line joining camera centers C, C’
• Epipolar plane ¦: Defined by baseline and scene point X
Computerbaseline
Vision : CISC 4/689
from Hartley
& Zisserman
Epipolar Lines
•
Epipolar lines l, l’: Intersection of epipolar plane ¦ with image planes
•
Epipoles e, e’: Where baseline intersects image planes
– Equivalently, the image in one view of the other camera center.
C’
C
Computer Vision : CISC 4/689
from Hartley
& Zisserman
Epipolar Pencil
•
As position of X varies, epipolar planes “rotate” about the baseline (like a book
with pages)
– This set of planes is called the epipolar pencil
•
Epipolar lines “radiate” from epipole—this is the pencil of epipolar lines
Computer Vision : CISC 4/689
from Hartley
& Zisserman
Epipolar Constraint
•
•
•
Camera center C and image point define ray in 3-D space that projects to epipolar line l’ in other
view (since it’s on the epipolar plane)
3-D point X is on this ray, so image of X in other view x’ must be on l’
In other words, the epipolar geometry defines a mapping
in the other
x ! l’, of points in one image to lines
x’
C’
C
from Hartley
& Zisserman
Computer Vision : CISC 4/689
Example: Epipolar Lines for Converging
Cameras
Left view
Right view
Intersection of epipolar lines = Epipole !
Indicates direction of other camera
Computer Vision : CISC 4/689
from Hartley
& Zisserman
Special Case:
Translation Parallel to Image Plane
Note that epipolar lines are parallel and corresponding points lie on correspondVision : CISC 4/689
ing epipolar lines (the latter is Computer
true for
all kinds of camera motions)
From Geometry to Algebra
P
p
p’
O’
O
Computer Vision : CISC 4/689
From Geometry to Algebra
P
p
p’
O’
O
Computer Vision : CISC 4/689
Rotation arrow should be at the other end, to get p in p’ coordinates
Linear Constraint:
Should be able to express as matrix
multiplication.
Computer Vision : CISC 4/689
Review: Matrix Form of Cross Product
Computer Vision : CISC 4/689
Review: Matrix Form of Cross Product
Computer Vision : CISC 4/689
Matrix Form
Computer Vision : CISC 4/689
The Essential Matrix
If calibrated, p gets multiplied by Intrisic matrix, K
Computer Vision : CISC 4/689
The Fundamental Matrix F
• Mapping of point in one image to epipolar line in other image x
! l’ is
expressed algebraically by the fundamental matrix F
line
point
= Fx
Since x’ is on l’, by the point-on-line definition we know that
x’T l’ = 0
Substitute l’ = Fx, we can thus relate corresponding points in the
camera pair (P, P’) to each other with the following:
x’T Fx = 0
• Write this as l’
•
•
Computer Vision : CISC 4/689