Transcript slides - Alexandru Dancu

Underwater reconstruction using
depth sensors
Alexandru Dancu1
Mickaël Fourgeaud1
Zlatko Franjcic2,1
Razmik Avetisyan3
1 Chalmers University of Technology
2 Qualisys AB
3 Rostock University
Contents
•
•
•
•
•
Related Work
Background
Experiment
Discussion
Conclusion
Related Work: Seafloor visualization
• advances in sonar technology, positioning
capabilities, and computational power
revolutionized seafloor imaging
• 1970’s, multibeam sonars were deepwater (12 kHz) systems, 16 beams
• now, more than 100 beams, the angular Multibeam sonar-derived backscatter data draped
sector covered > 120 degrees limited by over 20 m pixel bathymetry off Oahu, Hawaii
attenuation in deeper water
• shading, and 3-D rendering was used with
digital bathymetric data (DTM's) to form
natural looking and easily interpretable,
yet quantitative, landscapes.
Mayer, Larry A., et al. "Interactive 3-D Visualization: A tool for seafloor navigation, exploration and
engineering." OCEANS 2000 MTS/IEEE Conference and Exhibition. Vol. 2. IEEE, 2000.
Related Work: opti-acoustic fusion
• Acoustic cameras produce an image by
recording the reflected sound, once the
scene is insonified by acoustic pulse(s). In
a 3-D sonar, e.g., Echoscope [18], the
back-scattered signals are collected by a 2D array of transducers, and the image is
Circles depict matching points in optical (top) and
formed from “beam signals”
sonar (bottom)
Negahdaripour, Shahriar, Hicham Sekkati, and Hamed Pirsiavash. "Opti-acoustic stereo imaging: On system
calibration and 3-D target reconstruction." Image Processing, IEEE Transactions on 18.6 (2009): 1203-1214.
Related Work: Clear underwater vision
• Computer vision algorithm which inverts the
image formation process, to recover a good
visibility image of the object
• doubles the underwater
visibility range
• based on modelling the
image formation
• model is relevent here
Raw image
Recovered image
Based Schechner, Yoav Y., and Nir Karpel. "Clear underwater vision" Computer Vision and Pattern
Recognition, 2004. CVPR 2004. Proceedings of the 2004 IEEE Computer Society Conference on. Vol. 1.
Signal
Related Work: Clear underwater vision
Point spread function
empirical constants
Based Schechner, Yoav Y., and Nir Karpel. "Clear underwater vision" Computer Vision and Pattern
Recognition, 2004. CVPR 2004. Proceedings of the 2004 IEEE Computer Society Conference on. Vol. 1.
Related Work
• Voxelhashing
Background
Dinast sensors
Plastic box
Background: depth sensors
Kinect Sensor Components and Specifications
http://msdn.microsoft.com/enus/library/jj131033.aspx
ASUS Xtion on Hardware.info
http://us.hardware.info/productinfo/127001
/asus-xtion-pro
Background: Kinect IR emitter
Infrared image of a child playing a Kinect game http://youtu.be/0Yr5_0imcbY
Background: Kinect IR emitter
Infrared image of a child playing a Kinect game http://youtu.be/0Yr5_0imcbY
Water absorption of infrared
Water absorption of visible and near-infrared spectrum (blue curve); Kinect IR emitter wavelength
superimposed (red line), Based on data from GM Hale, MR Querry, Optical constants of water in the 200nm
to 200 mm wavelength, Applied Optics 12, 3 (1973), 555--563.
Water refraction
Refraction occurs when waves travel from a
medium with a given refractive index to a medium
with another at an oblique angle.
v1,v2 are phase velocities of wave
n1,n2 are refractive indices of medium
Objects look bent because of refraction
http://en.wikipedia.org/wiki/File:Pencil_in_a_bowl_of_water.svg
http://upload.wikimedia.org/wikipedia/commons/0/04/Refraction.jpg
http://upload.wikimedia.org/wikipedia/commons/a/ac/Angle_of_incidence_Refraction_example.jpg
Water refraction
Refraction occurs when waves travel from a
medium with a given refractive index to a medium
with another at an oblique angle.
v1,v2 are phase velocities of wave
n1,n2 are refractive indices of medium
Misleading picture,
the projection of Y and X
on water line is the same,
that is why it appears
higher
Objects look bent because of refraction
Contracted
http://en.wikipedia.org/wiki/File:Pencil_in_a_bowl_of_water.svg
http://upload.wikimedia.org/wikipedia/commons/0/04/Refraction.jpg
http://upload.wikimedia.org/wikipedia/commons/a/ac/Angle_of_incidence_Refraction_example.jpg
viewer observes smaller
distance to the pencil
particles because of
smaller angle in water
medium appears closer
Water refraction
Refraction occurs when waves travel from a
medium with a given refractive index to a
medium with another at an oblique angle.
viewer observes
smaller distance
to the pencil
particles because
of smaller angle
in water medium
appears closer
From water to air
n1= nwater = 1.3
n2= nair = 1.0
n1 > n2, v1 < v2,
so θ1 < θ2
From air to water
n1= nair = 1.0
n2= nwater = 1.3
n1 < n2, v1 > v2,
so θ1 > θ2
the ray in the higherindex medium is closer
to the normal.
Refraction of light at the
interface between two media
http://upload.wikimedia.org/wikipedia/c
ommons/thumb/3/3f/Snells_law2.svg/5
70px-Snells_law2.svg
Sensor inside water
Sensor above water
Sensor above water, reconstruction
Water refraction, reconstruction
Water refraction, reconstruction
Above water
• Mesh deforms, flattens
Underwater
From water to air
n1= nwater = 1.3
n2= nair = 1.0
n1 > n2, v1 < v2,
so θ1 < θ2
Water refraction, reconstruction
Refraction
Above water
• Mesh deforms, flattens
• Slope of mesh changes
• Assumption:
– Refraction responsible for
mesh deformation
Underwater
From water to air
n1= nwater = 1.3
n2= nair = 1.0
n1 > n2, v1 < v2,
so θ1 < θ2
Water absorption, reconstruction
Water absorption, reconstruction
Absorption
Above water
Underwater
Based on data from GM Hale, MR Querry, Optical constants of water in the 200nm
to 200 mm wavelength, Applied Optics 12, 3 (1973), 555--563.
Water absorption, reconstruction
Absorption
Above water
Underwater
Based on data from GM Hale, MR Querry, Optical constants of water in the 200nm
to 200 mm wavelength, Applied Optics 12, 3 (1973), 555--563.
Heatmap of mesh difference
Based on the refraction and absorption assumption we created an error heatmap
http://www.danielgm.net/cc
Discussion
• Discussion
– Kinect it works and the other sensors do not
because of infrared emitter power??
• Angle don’t mention it!!
• Drawing with kinect - 70 cm box idea
• DIVA sponsor slide!!!
Future work
• Zlatko
Ulf’s recommendations
• Kinect operation range = 0.8m - 3.5m
• 20 min maximum, max. 2 questions, should be
shorter and at point, 15min presentation + 5
min questions
• Talk while showing the video, pause, stretch
the video, parts should be proportional to
what is said.. Moving object should be fast.
• Q1 – can u get this from white papers?
• Emitter intensity (openkinect) (white paper)