Transcript Slide 1 - COFES.com

VideoTrace: Interactive
3d modelling for all
Anton van den Hengel
Director, Australian Centre for Visual technologies
Associate Professor, Adelaide University, South Australia
Director, PunchCard Visual technologies
Input
Modelling
Results
Interactive 3D modelling

3D modelling is critical to all sorts of application
 Special
effects, but also mining, architecture, defence,
urban planning, …


People are getting more visually sophisticated
More 3D data is being generated
 More

cameras, but also scanners etc
The interfaces of modelling programs are
usually very hard to fathom
Why?
Insert your own objects into a game
 Put your couch into second life
 Model your house for Google Earth
 Video editing

 Cut
and paste between sequences
 Remove someone from your home videos
Put your truck into a game
Put your truck into a game
Modelling for animation
Video editing requires models
Dense surface reconstruction
Video editing requires models
Structure from motion
The process

Capture and import the video
 Perform
structure and motion analysis
Interact with the system to generate and
edit the model
 Export to your application

The approach

Pre-compute where possible
 Structure
from motion (camera tracking)
 Superpixels

Then interact
 Interactions
results
allow user to exploit precomputed
Structure from motion
Camera tracking
 Calculates

 Reconstructed
point cloud
 Camera parameters
Location
 Orientation
 Intrinsics (eg. Focal length)


Informs interaction interpretation process
Interactive modelling from video
Interactions

Straight lines
 Closed

sets of lines define planar polygons
Curves
 For
planar shapes with curved edges
 For NURBS surfaces

Mirroring
 Duplicates

Extrusion
existing geometry
Fitting planar faces
User specifies boundary
 Boundary specifies infinitely many planes
 Similar to pre-emptive RANSAC

 Generate
bounded plane hypotheses from
point cloud
 Eliminate hypotheses that fail a series of tests

Run simplest / most robust tests first

Generally 3d tests before 2d tests
Fitting planar faces
Line of sight
Object points
Image plane
Hierarchical RANSAC
Generate bounded plane hypotheses
 Tests

 Support
from point cloud
 Reprojects within new image boundaries
 Constraints on relative edge length and face
size
 Colour histogram matching on faces
 Colour matching on edge projections
 Reprojection is not self-occluding
Curves
Mirroring
Extrusion
Dense surface reconstruction



Needs to be at interactive speed
Calculated as a max-flow graph-cut over a
Markov Random Field
Link cost based on photoconsitency
Modelling without features
Modelling without features
Recent model
Future work





Other interactions
Other data sources
Occluding contours
Interactive SFM
De / Re-lighting