Transcript ppt

Interactive
Environmental Sensing:
Signal and Image
Processing Challenges
Michael Allen, Eric Graham, Shaun Ahmadian, Teresa Ko,
Eric Yuen, Lewis Girod, Michael Hamilton, Deborah Estrin
Centre for Embedded Networked Sensing, UCLA
Environmental sensing

Has long been a motivator for embedded networked
sensing research


Observe at high sampling rates and multiple scales


Use of acoustic sensors and imagers creates large data sets
Information must be extracted from these data sets


Help answer scientific questions
Manual information extraction is impractical
The scientist must be included in the processing

Interactive environmental sensing
Interactive Environmental Sensing

Scientist’s confidence in processing



Automation is desirable


Treat levels of processing as black boxes
Interaction comes at points where scientist lacks confidence


Lower level: sensor reliability, data quality
Higher level: confidence in accuracy of detectors, classifiers
Phenomena are uncharacterised, hard to predict
Three motivating signal and image processing applications



Bioacoustics
Plant Phenology
Avian biology
Acoustic array

Research motivated by bioacoustics



Traditional approach: deploy wired microphones


Record audio, offline data analysis
Acoustic ENSBox networked platform




Detect, classify and localize animal/bird vocalizations
Gives insight into behaviour, census
Deploy instead of wired mics
4 microphone compact array (12cm spacing)
400MHz ARM CPU, 64MB RAM, 48KHz
Allow interaction through on-line operation


Change parameters, react to events
Enable new interactions (e.g. photos)
V2 (2007)
Processing goals/challenges

Example: on-line acoustic source localization

Monitoring animals in their natural habitat
Event detection

Position estimation
Challenges


DoA processing
Perform localization on-line, allow interaction for scientist
Environment affects signal attenuation, signal coherency


Cannot use time or amplitude differences easily
Implies using DoA methods for environment
Fast event detector

Find events of interest in continuous
stream of audio (1 channel)





Monitor energy in frequency bands of
interest (band pass filter)
Assume ambient noise can be modelled as
Gaussian distribution
Noise floor estimated on-line
(smoothed µ,σ)
Threshold is determined as β standard
deviations from mean
Adapting for different animals/birds


Has worked well for marmots in-situ
Change β, band pass filter range,
smoothing parameters
Position estimation


Direction of arrival estimation at
each compact array
Use Approximated Maximum
Likelihood (AML) algorithm



Compact array dimensions and signal
characteristics cause spatial aliasing
Can cause DoA ambiguity
Fuse local estimates


‘Pseudo-likelihood’ map of likelihood
crossings
Eliminate individual DoA ambiguities
Plant phenology


Phenology: the study of times of recurring phenomena
Plant phenology: yearly patterns of bud burst, flower
bloom, numbers and sizes of leaves


Traditional approach: Manual observation/logging of
phenological data


Indicators of environmental conditions
Time-consuming, labour intensive
Use of actuated imagers as biological sensors



Determine budburst using image streams
Higher frequency of observations
Use in remote locations for longer periods
Processing goals/challenges

Images taken using PTZ camera towers


Too many images for manual
inspection


1,200 images/day
Automated detection is desirable
Challenges:




Lighting conditions change
Flower/leaf colours may not
contrast well with environment
Obstructions/occlusions
Small flowers/leaves
James Reserve,
Idyllwild, CA
Interactive processing loop

Reduce images in data stream to event candidates


Scientist verifies reduced data set
Interaction – use reduced data set to work back through
previous days
Area of interest
Image capture
Interactive visualisation
and image processing
Scientific results
Tools for interaction

Processing tool for data set
reduction


Colour filtering



Mask areas of image based on hue
(HSL colour space)
Hue is not affected by changes in
ambient light
Image enhancement




Filter quickly over data set
Contrast stretching
Histogram equalisation
Gaussian blur
Image tagging

Creates ground truth for comparison
Automated processing

Automated leaf area
estimation



Mask image by hue, count
pixels
Indicates leaf growth
Automated flower counter




Blooming event prediction
Blob extraction, counting
Exact counts are not
necessarily important
Shows peaks in line with
ground truth
Avian Biology

Understand relationship between behaviour and
reproductive success in birds


Traditional approach:


Scientist manually observes nest boxes, logs data
Use of static imagers as biological sensors




Presence/absence, egg hatching success
Deploy wired/wireless imagers inside nest boxes
Minimise disturbance of birds
Increase sampling scale, density, frequency
Biologist can use tools to interact

Target visual investigation, anomalies
James
Reserve,
Idyllwild,
CA
Processing goals/challenges

Bird detection in image stream


Egg counting in image stream


Use infra-red – restricts images to greyscale
Different bird species build nests differently


Count eggs laid, infer number hatched, infer stage
Lack of uniform lighting through day/night


presence/absence changes over cycle (nesting, egg laying, …)
Affects generality of processing algorithms
Counting eggs in single images is hard to do


Even manually
Image quality issues, occlusions
Bird detection, presence/absence
Low/high density of interest points

Use ‘corners’ as interest points in
images:




Areas where gradients are large in 2
directions
Harris Stephens detector
Make use of temporal nature of
image stream
Presence/absence


Midpoint between min/max
interest points over a smoothed
time window
Gives reasonable threshold for
determining presence/absence
Bird
No bird
Egg detection

Scale Invariant Feature Transform
(SIFT) for blob detection:



Each image is treated as a weak
classifier




Scale from characterised egg sizes
Mean, variance of intensity in blob
area
Weak classifiers used as input into a
Hidden Markov Model
HMM states are true egg counts,
observations are estimates
Trained on ground truth data sets
Simple filtering can be performed

Based on distance of features from
decision boundary
Progression

Acoustic array



Plant phenology



Automatic classification and census in addition to localization
Other event detection approaches
More complex detection for more complex flowers
SIFT shows promising results
Avian Biology


Different processing techniques work better at different
stages
Estimate stage, adjust processing accordingly
Conclusion

Event detection in signal stream is common theme


New challenges naturally follow




accuracy improvement
self-adaptation
context aware processing
Flexibility and iteration are key to addressing challenges



Automation for well characterised events in stream
Characterisation comes with application experience
Keeps human interactively in the loop
Interaction comes where scientist lacks confidence in
processing loop
Backup: Localization performance
RMS error 0.78 m