Transcript 3D Localization

3D Vision
Andrew Fitzgibbon
Engineering
Andrew Glennerster
Physiology
The University of Oxford
Vision
A fundamental biological system
An attractive engineering problem
How are these related?
A 3D Visual Task
TOC
• Computer vision state of the art
• Biology state of the art
– an implausible model?
• Open questions
Artificial 3D Vision
Measurements
3D Structure
Application: 3D Video Editing
Application: 3D Video Editing
“Computer Vision” Summary
• Single representation suits all
– pointing
– navigation
• Built on
– algebra
– geometry
– statistics
• Really works on real data
Biological systems can perform 3D tasks
using visual information
• Pointing to remembered objects
• Ants homing
• Spiders turning to view prey
Computer vision is an unlikely model
for animal vision
• A “likely” model needs to
 predict experimental data
 be neurally plausible
• We don’t know how to multiply matrices in
neurons
• CV is open loop
– animate systems characterized by feedback
• 3D not necessary for many tasks
What is the incumbent model?
Eye-centred
coordinates
Head-centred
coordinates
Body-centred
coordinates
World-centred
coordinates
Posterior parietal
cortex
Eye
position
Neck
proprioception
Audition
Vestibular
information
Adapted from figure 3 from Andersen et al (1997)
Visual
information
An Alternative
• In simpler animals, the requirement for a map
is questioned.
• Many tasks can be performed by
local, low-level, reactive rules
“Biomimetic” Navigation
(Franz and Mallot, 2000)
“Biomimetic” Navigation
(Franz and Mallot, 2000)
“Biomimetic” Navigation
• Neurally plausible
• Real robot implementations
• Cannot model 3D pointing yet
• No complete model of ant, for example
“computer vision hitting a computational wall”
• Need infinite-time, infinite-space operation
• Computers are getting faster...
• When will O(2lt) beat O(n3)?
– n is number of features
– t is years from now
Dynamic Scenes
Open Problems
• No complete model of ant navigation
– Photons to pointing
• No plausible model of 3D vision in higher
animals
– Coordinate system: Is there one?
– One representation or many?
• Computer vision’s computational wall
– One solution will be better engineering
– Another may be “parallel slow neurons”
• Dynamic scenes
Thanks To
Andrew Glennerster
Andrew Parker
Andrew Blake
Andrew Zisserman
Andrew Davison
Andrew Stoddart
...and a few non-andrews
Questions
• How do biological systems perform such
tasks?
– We want an explanation “from
photons to pointing”
• How might artificial systems perform such
tasks?
– Artificial systems are “easy” to analyse
Status
• We have models for some of the biology
• We have artificial systems which work in
useful but limited domains