Transcript 3D Computer Vision

3D Computer Vision
Omnidirectional Vision
Spring 2006
Lecture 3 - Part 2
Omnidirectional Cameras
Zhang Aiwu
3D Computer Vision

Lecture Outline
Applications




Robot navigation, Surveillance, Smart rooms
Video-conferencing/ Tele-presence
Multimedia/Visualization
GIS

Page of Omnidirectional Vision (Many universities and companies….)
 http://www.cis.upenn.edu/~kostas/omni.html

Design Requirements





Several Important Designs




360 degree FOV, or semi-sphere or full sphere in one snapshot
Single effective viewpoint
Image Resolutions – one or more cameras?
Image Sharpness – optics as well as geometry
Catadioptric imaging : mirror (reflection) + lens ( refraction)
Mirrors: Planar, Conic, Spherical, Hyperboloidal, Ellipsoidal, Paraboloidal
Systematic design ( S. Nayar’s group)
Calibrations
 Harder or simpler?
3D Computer Vision
Which one?
From the Page of Omnidirectional Vision
http://www.cis.upenn.edu/~kostas/omni.html
3D Computer Vision
(Poly-)Dioptric solutions
One to two fish-eye cameras or many synchornized cameras
Homebrewed polydioptric cameras are cheaper,
but require calibrating and synchronizing;
commercial designs tend to be expensive
3D Computer Vision
Catadioptric solutions
Usually single camera combined with convex mirror
Pros:
- Single image
Cons:
- Blindspot
- Low resolution
3D Computer Vision

Catadioptric imaging :
 mirror (reflection) + lens ( refraction)

Theory of Catadioptric Image Formation ( S. Nayar’s group)

"A Theory of Single-Viewpoint Catadioptric Image Formation" , Simon Baker
and Shree K. Nayar ,International Journal of Computer Vision, 1999.

Mirrors
 Planar
 Conic, Spherical
 Hyperboloidal, Ellipsoidal
 Paraboloidal

Cameras (Lens)



Sensor Design
Perspective (pinhole) or orthogonal (tele-centric lens) projection
One or more?
Implementations


Compactness - size, support, and installation
Optics – Image sharpness, reflection, etc.
3D Computer Vision

Planar Mirror
Panoramic camera system using a pyramid with four (or
more) planar mirrors and four (or more) cameras
(Nalwa96) has a single effective viewpoint
Mirror
pyramid
6 cameras
4 camera design and 6 camera prototype:
FullView - Lucent Technology http://www.fullview.com/
Planar Mirror
3D Computer Vision

Panoramic camera system using a pyramid with four (or
more) planar mirrors and four (or more) cameras
(Nalwa96) has a single effective viewpoint
Viewpoint of the
Virtual camera
P1
P2
Geometry of 4 camera approach: four separate cameras in 4
viewpoints can generate images with a single effective viewpoint
3D Computer Vision
Planar Mirror Approach

A single effective viewpoint

More than one cameras
High image resolution

3D Computer Vision
Planar Mirror Approach

A single effective viewpoint

More than one cameras
High image resolution

3D Computer Vision




Conic Mirror
Viewpoints on a circle
semispherical view except occlusion
Perspective projection in each direction
Robot Navigation (Yagi90, Zhu96/98)
viewpoint
pinhole
3D Computer Vision
Spherical Mirror

Viewpoints on a spherical-like surface

Easy to construct (Hong91 -UMass )
Locus of
viewpoints
Intersection of incoming rays
are along this line
3D Computer Vision

Single Viewpoint


Hyperboloidal Mirror
if the pinhole of the real camera and the virtual
viewpoint are located at the two loci of the
hyperboloid
Semi-spherical view except the self occlusion
viewpoint
Rotation of the
hyperbolic curve
generates a
hyperboloid
P1
P2
pinhole
3D Computer Vision

ACCOWLE Co., LTD, A Spin-off at Kyoto University



Hyperboloidal Mirror
http://www.accowle.com/english/
Spherical Mirror
Hyperbolic Mirror
Image: High res. in the top
3D Computer Vision

Single Viewpoint


Ellipsoidal Mirror
if the pinhole of the real camera and the virtual
viewpoint are located at the two loci of the ellipsoid
Semi-spherical view except the self occlusion
pinhole
P2
P1
viewpoint
3D Computer Vision
Panoramic Annular Lens
- geometric mathematical model
for image transform & calibration
P1
panoramic annular lens (PAL)
- invented by Pal Greguss
* 40 mm in diameter, C-mount
P
B
Hyperboloidal mirror
O
pinhole C
* view: H: 360, V: -15 ~ +20
* single view point (O)
p p1
Ellipsoidal mirror
3D Computer Vision
panoramic annular lens (PAL)
- invented by P. Greguss
* 40 mm in diameter, C-mount
* view: H: 360, V: -15 ~ +20
•single view point (O)
•C-Mount to CCD Cameras
Panoramic Annular Lens
Image: High res. In the bottom
3D Computer Vision
Cylindrical panoramic un-warping
Two Steps:
(1). Center determination
(2) Distortion rectification
2-order polynomial approximation
Circular to cylindrical transformation
after eliminating radial distortion
Paraboloidal Mirror
3D Computer Vision



Semi-spherical view except the self occlusion
Single Viewpoint at the locus of the paraboloid, if
 Tele-lens - orthographic projection is used
Mapping between image, mirror and the world invariant to
translation of the mirror. This greatly simplifies calibration and
the computation of perspective images from paraboloidal images
viewpoint
P1
P2
tele-lens
3D Computer Vision


Paraboloidal Mirror
Remote Reality – A Spin-off at Columbia University
http://www.remotereality.com/
Camcorder
Web Camera
Back to Back : Full
Spherical View
3D Computer Vision


Paraboloidal Mirror
Remote Reality – A Spin-off at Columbia University
http://www.remotereality.com/
3D Computer Vision
Confused?
Confused?
Q: What kind of sensor should one use?
A: Depends on your application.
1. If you are primarily concerned with:
– resolution
– surveillance (coverage)
and can afford the bandwidth & expense,
you might stick with polydioptric solutions
2. If you are concerned with
– bandwidth
–servoing, SFM
investigate catadioptric or single wide
FOV dioptric solutions
3D Computer Vision
Application
3D Computer Vision
Application
3D Computer Vision
Application
3D Computer Vision
Application
3D Computer Vision
Application
3D Computer Vision
用GPS和全景相机 日本开发出绘城市地图新技术




城市的街道和建筑物日新月异，使得汽车导航系统必须不断更
新地图。日本的一种新技术利用全球卫星定位系统（GPS）和360
度全景照相机，可以及时发现城市变化，并修改地图。
据《日经产业新闻》日前报道，这种技术由日本名古屋大学教
授村濑洋的研究小组开发，能够将GPS提供的位置信息和来自全
景照相机的图像信息组合起来，自动检索出建筑物所处位置。
这项技术首先利用搭载有全景照相机的公交和出租车收集图像
。这些车辆一边穿街走巷一边拍摄周围的情况，然后将拍摄到的
图像汇总到计算机。根据车上GPS提供的位置信息，就可以迅速
判断出所收集到的图像是在何时、何地拍摄的。
GPS的位置信息存在10米左右的误差，但是计算机能够通过
比对多辆汽车传来的图像进行自动调整，从而将误差控制在40厘
米以内。应用这种图像识别技术，就可以在街道情况出现变化时
迅速更新城市地图。
3D Computer Vision
Image
Properties of Paraboloid System
(Assuming aspect ratio = 1)

The Image of a Line



Dual Vanishing Points


There are two VPs for each set of parallel lines, which are
the intersections of the corresponding circles
Collinear Centers


is a circular arc if the line is not parallel to the optical axis
Is projected on a (radial) line otherwise
The center of the circles for a set of parallel lines are
collinear
Vanishing Circle

The vanishing points of lines with coplanar directions* lie
on a circle ( all the lines parallel to a common plane)
3D Computer Vision
Image
Properties of Paraboloid System
(with aspect ratio)

The Image Center



Projection of a Line with unknown aspect ratio


Is an elliptical arc in the general case
The Aspect Ratio


Is on the (“vanishing”) line connecting the dual vanishing
points of each set of parallel lines
Can be determined by two sets of parallel lines
Is determined by the ratio of the lone-short axes of the
ellipse corresponding to a line
Intrinsic Calibration


Estimate aspect ratio by the ratio of ellipse
Estimate the image center by the intersection of vanishing
lines of two sets of parallel lines in 3-D space
3D Computer Vision

Calibration of Paraboloid System
The Image Center


Is on the (“vanishing”) line connecting the dual vanishing
points of each set of parallel lines
Can be determined by two sets of parallel lines
3D Computer Vision


Calibration of Paraboloid System
The Image Center
 Yellow “vanishing” line of horizontal set of parallel lines
 Pink “vanishing” line of vertical set of parallel lines
The Vanishing Circle (Red dotted)
 The vanishing points of lines with coplanar directions ( on a
plane in this example)
Projected to the plane of
the calibration pattern
Next
3D Computer Vision

Turn in your projects and schedule meetings with me
END