Transcript Insitute of Computer Graphics and Algorithms, Vienna University of

Practical Spectral Photography
Ralf Habel1
Michael Kudenov2
Michael Wimmer1
Institute of Computer Graphics and Algorithms
Vienna University of Technology1
Optical Detection Lab
University of Arizona2
Motivation
Spectroscopy is most important
analysis tool in all natural sciences
Astrophysics, chemical/material
sciences, biomedicine, geophysics,…
Industry applications:
Mining, airborne sensing, QA,…
In computer graphics:
Colors
Material reflectance
Spectral/predictive rendering
…
Ralf Habel
1
Spectral Imaging
Records image at narrow wavelength bands
In visible range not only RGB (3 channels)
but many more (6-400 channels)
Result: 3D data cube
2 spatial image axis
1 wavelength axis
Ralf Habel
2
Spectral Imaging
Usually done with highly specialized devices
Many methods to build devices
Scanning slits, rotating mirrors,
special sensor, filters, prisms, …
Usually scan along one of the data cube axis
All very costly due to opto-mechanical components
“Simplest” spectral imager:
Camera + band filters
Requires switching of filters
Limited in number of bands
Ralf Habel
3
Motivation
Why not use consumer cameras and
equipment for spectral imaging?
High quality, very sensitive
Highly accurate lenses
Practical Constraints:
No camera modification
No lab/desktop/optical bench setup
No expensive components
Ralf Habel
4
CTIS Principle
Computed Tomography Image Spectrometer
Diffraction grating parallel-projects 3D data cube in
different directions on image plane (sensor):
Ralf Habel
5
CTIS Principle
Computed Tomography Image Spectrometer
Diffraction grating parallel-projects 3D data cube in
different directions on image plane (sensor):
Ralf Habel
6
CTIS Principle
Sensor records projections of 3D data cube
All information needed is recorded in one image
“Snapshot” spectrometry
Challenge is to reconstruct 3D
data cube from projections
Tomographic rec. with Expectation
Maximization
More details in paper
Ralf Habel
7
CTIS Optical Path
Imaging lens + square/slit aperture creates virtual image
Ralf Habel
8
CTIS Optical Path
Imaging lens + square/slit aperture creates virtual image
Collimating lens makes light parallel
Ralf Habel
9
CTIS Optical Path
Imaging lens + square/slit aperture creates virtual image
Collimating lens makes light parallel
Diffraction grating creates projections
Ralf Habel
10
CTIS Optical Path
Imaging lens + square/slit aperture creates virtual image
Collimating lens makes light parallel
Diffraction grating creates projections
Re-imaging lens focuses on sensor
Ralf Habel
11
CTIS Optical Path
Imaging lens + square/slit aperture creates virtual image
Collimating lens makes light parallel
Diffraction grating creates projections
Re-imaging lens focuses on sensor
Ralf Habel
12
CTIS Optical Path
Built with:
Drain pipe & duct tape
50mm, 17-40mm and macro lens
Diffraction gel ($2 per sheet) in gel holder
Ralf Habel
13
CTIS Camera Objective
Ralf Habel
14
CTIS Camera Objective
Ralf Habel
15
HDR Image Acquisition
No overexposed pixels allowed
Projections (diffractions) weaker than center image
Avoids noisy signal where camera response is weak
Ralf Habel
16
Spatial Wavelength Calibration
Mapping from 3D data cube into projections

Laser pointers (red, green and blue) with known
wavelengths shot through a diffusor and pinhole
Monochromatic point light source
Pictures of pinhole give mapping of one voxel in 3D
data cube
All other projections values
interpolated/extrapolated
Ralf Habel
17
CTIS Principle
Ralf Habel
18
Spatial Wavelength Calibration
Ralf Habel
19
Spectral Response Calibration
Spectral response of the diffraction grating +
RGB sensor for red, green and blue
Picture of light source with continuous known spectrum
We use calibrated halogen lamp
Ralf Habel
20
Spectral Photography Results
Take HDR picture with CTIS camera objective
Reconstruct 3D data cube for red, green and blue
image color channels
Mapping from spatial calibration
Combine RGB spectral response of each pixel to
true spectrum with spectral de-mosaicking
Mapping from spectral response calibration
Ralf Habel
21
Spectral Photography Results
Protoype data cube resolutions:
120x120 pixels
4.59 nm (54 channels)
Accuracy reduced in high blue and low reds due
to color filters
Slight Expectation Maximization reconstruction
artifacts
Nowhere near possible optimum!
Ralf Habel
22
Spectral Photography Results
Ralf Habel
23
Spectral Photography Results
Ralf Habel
24
Future
Better CTIS objective
Drain pipes and duct tape have their limits…
Optimized optical path and components
More compact/integrated device
Increase data cube resolution/accuracy:
Structured aperture
Digital holography –
form diffraction/projections in any way
Better solutions to tomographic
reconstruction
Is active research in optics
No vision based approach yet!
Ralf Habel
25
Future
Turning mobile devices into spectrometers consumer spectroscopy?
8 MP high sensitivity sensors
HDR capabilities
Very low cost!
“Snapshot” capability:
Spectral movies with consumer cameras?
Not only good for computer graphics:
Blood sample analysis
Water contamination analysis
As part of a TricorderTM
Ralf Habel
26
Practical Spectral Photography
Thank You!
Ralf Habel
27