Transcript Logarithmic CMOS image sensors - Home

Logarithmic CMOS
image sensors
Dr. Dileepan Joseph
Dept. of Engineering Science
University of Oxford, UK
Outline
Motivation
Background
Method
Conclusions
Future work
Motivation: social
Society has invested over many millennia in
developing technology to record observed
scenes on an independent medium
Artistic license aside, the aim has been to render
images with a maximum of perceptual accuracy
using a minimum of effort
The digital camera is a culmination of the above
but its development is far from complete…
Although digital cameras have in many ways
surpassed film cameras, human vision routinely
outperforms the best cameras
Motivation: economic
A digital camera consists of many components
(optics, housing, battery, memory etc.), of which
the image sensor is considered principal
With market revenues of $1.7 billion in 2003,
there is widespread research and development
in a variety of image sensor designs
Modern designs may be either charge coupled
device (CCD) sensors or complementary metaloxide-semiconductor (CMOS) sensors
Motivation: technological
Criterion
Human eye
Film photo
CCD sensor
CMOS sensor
Pixel pitch
2–3 μm
10–20 μm
5–10 μm
5–10 μm
Image pitch
3 cm
Film size
1 mm–11 cm
1 mm–2 cm
Dynamic range
2–5 decades
1–4 decades
4 decades
3–5 decades
Max. frame rate
≈ 15 Hz
1 shot only
10 kHz
>> 10 kHz
Pre-processing
Extensive
None
None
Possible
Unit price
Invaluable
€ 0.1
€ 100
€ 10
Background: CCD image sensor
Marches photo generated
charge systematically
from an array of pixels to
an output amplifier
Established technology
High resolution, high
sensitivity, low noise
Fabrication process is
optimised for imaging
Market share of 93% in
1999 (49% in 2004?)
Background: CMOS image sensor
Works like memory array
with photosensitive pixels
instead of memory cells
Signal processing may be
included on the same die
High yield and good
video performance
May be fabricated by the
makers of microchips
Market share of 7% in
1999 (51% in 2004?)
Background: linear pixels
Linear pixels (CCD or
CMOS) integrate
photons over discrete
periods of time
They produce a
voltage directly
proportional to the
light intensity
The response may
saturate white or
black easily
Background: logarithmic pixels
Logarithmic pixels
(CMOS only) can
measure photon flux
continuously
They produce a
voltage proportional
to the logarithm of
light intensity
The response is
similar to that of
human vision
Background: image quality
Images are noisy with
logarithmic sensors
Colour is worse than
with linear sensors
Quality improves with
digital processing
No comprehensive
treatment of either
problem or solution
Method: theory
Model logarithmic CMOS image sensors
using optical & integrated circuit theory
Use the model to hypothesize the cause
and solution of image quality problems
Calibrate the model and test hypotheses
using constrained regression theory
Optimise digital image processing using
multilinear (or array) algebra
Method: simulation
Simulation of integrated circuits is more
accurate than a theoretical analysis
Cost of simulation in time and money is
small compared to that of experiment
Integrated circuits may be studied under
controlled and well-defined conditions
Internal states and variables may be
observed without specialised equipment,
circuit disruption and/or foresight
Method: experiment
Experiments were performed using a Fuga
15RGB camera from C-Cam Technologies
The camera was operated from a portable
PC via a custom Windows application
The image sensor had 512 × 512 pixels
and a full frame rate of about 8 Hz
Until recently, it was the most successful
commercial logarithmic image sensor
Conclusions: fixed pattern noise
y = a + b ln (c + x) + ε
for illuminance x and
response y of a pixel
Variation of offset a,
gain b, bias c or a
combination thereof
causes FPN
Calibration possible
within limits of the
stochastic error ε
Conclusions: fixed pattern noise
Left to right: FPN
correction for single,
double and triple
variation models
Top to bottom: two
decade attenuation of
illuminance in half
decade steps
Inter-scene plus intrascene dynamic range
of 3.5 decades
Conclusions: transient response
The transient response of
a pixel is fast enough for
most applications
Greater demands are
placed on the row and
column readouts
Premature digitization
results in a predictable
non-uniformity or FPN
Affects only a few rows
due to slow scanning
Conclusions: transient response
Premature digitization is
more serious for column
readout due to speed
For example, columns
need scanning at 100
MHz for HDTV video
Column-to-column gain
variation is caused by
transient response
Resolve with careful
timing and design
Conclusions: temperature
dependence
Unlike with humans,
digital cameras do not
regulate temperature
Hence, responses to
illuminance depend
on temperature
When temperature
dependence varies
from pixel to pixel,
FPN occurs
Conclusions: temperature
dependence
The dark response of
a pixel depends only
on temperature
It may be used to
correct FPN due to
temperature in the
light response
Experiments support
this conclusion but
simulation results are
shown for clarity
Conclusions: colour rendition
Combine the theories of
colour linear sensors and
b/w logarithmic sensors
Calibrate FPN, using
images of uniform stimuli,
by a relative analysis
Calibrate colour, using
images of a colour chart,
by an absolute analysis
Fuga 15RGB competes
with conventional digital
cameras (which have a
perceptual error of 15)
Conclusions: colour rendition
Image of a colour chart,
in 11 lux of illuminance,
was rendered using
calibrated models
Single, double and triple
variation results and ideal
colours are shown
As with vision, rendition
improves in brighter
lighting and worsens in
dimmer lighting
Future work
Digital cameras aim to render images with
a maximum of perceptual accuracy using
a minimum of effort
By modelling and calibrating logarithmic
CMOS image sensors, problems with
image quality may be solved
Past work has focused on maximising
perceptual accuracy but future work will
focus on minimising effort
Future work
Shrinking feature
sizes may be used to
improve imaging
There are challenges
with deep submicron
CMOS processes that
need overcoming
What about industrial
and biomedical uses
of the technology?
Acknowledgements
This work was funded thanks to the engineering
research councils of both Canada and the UK
Thanks also to colleagues at the Microelectronic
Circuits and Analogue Devices research group