Transcript ppt
Interactive
Environmental Sensing:
Signal and Image
Processing Challenges
Michael Allen, Eric Graham, Shaun Ahmadian, Teresa Ko,
Eric Yuen, Lewis Girod, Michael Hamilton, Deborah Estrin
Centre for Embedded Networked Sensing, UCLA
Environmental sensing
Has long been a motivator for embedded networked
sensing research
Observe at high sampling rates and multiple scales
Use of acoustic sensors and imagers creates large data sets
Information must be extracted from these data sets
Help answer scientific questions
Manual information extraction is impractical
The scientist must be included in the processing
Interactive environmental sensing
Interactive Environmental Sensing
Scientist’s confidence in processing
Automation is desirable
Treat levels of processing as black boxes
Interaction comes at points where scientist lacks confidence
Lower level: sensor reliability, data quality
Higher level: confidence in accuracy of detectors, classifiers
Phenomena are uncharacterised, hard to predict
Three motivating signal and image processing applications
Bioacoustics
Plant Phenology
Avian biology
Acoustic array
Research motivated by bioacoustics
Traditional approach: deploy wired microphones
Record audio, offline data analysis
Acoustic ENSBox networked platform
Detect, classify and localize animal/bird vocalizations
Gives insight into behaviour, census
Deploy instead of wired mics
4 microphone compact array (12cm spacing)
400MHz ARM CPU, 64MB RAM, 48KHz
Allow interaction through on-line operation
Change parameters, react to events
Enable new interactions (e.g. photos)
V2 (2007)
Processing goals/challenges
Example: on-line acoustic source localization
Monitoring animals in their natural habitat
Event detection
Position estimation
Challenges
DoA processing
Perform localization on-line, allow interaction for scientist
Environment affects signal attenuation, signal coherency
Cannot use time or amplitude differences easily
Implies using DoA methods for environment
Fast event detector
Find events of interest in continuous
stream of audio (1 channel)
Monitor energy in frequency bands of
interest (band pass filter)
Assume ambient noise can be modelled as
Gaussian distribution
Noise floor estimated on-line
(smoothed µ,σ)
Threshold is determined as β standard
deviations from mean
Adapting for different animals/birds
Has worked well for marmots in-situ
Change β, band pass filter range,
smoothing parameters
Position estimation
Direction of arrival estimation at
each compact array
Use Approximated Maximum
Likelihood (AML) algorithm
Compact array dimensions and signal
characteristics cause spatial aliasing
Can cause DoA ambiguity
Fuse local estimates
‘Pseudo-likelihood’ map of likelihood
crossings
Eliminate individual DoA ambiguities
Plant phenology
Phenology: the study of times of recurring phenomena
Plant phenology: yearly patterns of bud burst, flower
bloom, numbers and sizes of leaves
Traditional approach: Manual observation/logging of
phenological data
Indicators of environmental conditions
Time-consuming, labour intensive
Use of actuated imagers as biological sensors
Determine budburst using image streams
Higher frequency of observations
Use in remote locations for longer periods
Processing goals/challenges
Images taken using PTZ camera towers
Too many images for manual
inspection
1,200 images/day
Automated detection is desirable
Challenges:
Lighting conditions change
Flower/leaf colours may not
contrast well with environment
Obstructions/occlusions
Small flowers/leaves
James Reserve,
Idyllwild, CA
Interactive processing loop
Reduce images in data stream to event candidates
Scientist verifies reduced data set
Interaction – use reduced data set to work back through
previous days
Area of interest
Image capture
Interactive visualisation
and image processing
Scientific results
Tools for interaction
Processing tool for data set
reduction
Colour filtering
Mask areas of image based on hue
(HSL colour space)
Hue is not affected by changes in
ambient light
Image enhancement
Filter quickly over data set
Contrast stretching
Histogram equalisation
Gaussian blur
Image tagging
Creates ground truth for comparison
Automated processing
Automated leaf area
estimation
Mask image by hue, count
pixels
Indicates leaf growth
Automated flower counter
Blooming event prediction
Blob extraction, counting
Exact counts are not
necessarily important
Shows peaks in line with
ground truth
Avian Biology
Understand relationship between behaviour and
reproductive success in birds
Traditional approach:
Scientist manually observes nest boxes, logs data
Use of static imagers as biological sensors
Presence/absence, egg hatching success
Deploy wired/wireless imagers inside nest boxes
Minimise disturbance of birds
Increase sampling scale, density, frequency
Biologist can use tools to interact
Target visual investigation, anomalies
James
Reserve,
Idyllwild,
CA
Processing goals/challenges
Bird detection in image stream
Egg counting in image stream
Use infra-red – restricts images to greyscale
Different bird species build nests differently
Count eggs laid, infer number hatched, infer stage
Lack of uniform lighting through day/night
presence/absence changes over cycle (nesting, egg laying, …)
Affects generality of processing algorithms
Counting eggs in single images is hard to do
Even manually
Image quality issues, occlusions
Bird detection, presence/absence
Low/high density of interest points
Use ‘corners’ as interest points in
images:
Areas where gradients are large in 2
directions
Harris Stephens detector
Make use of temporal nature of
image stream
Presence/absence
Midpoint between min/max
interest points over a smoothed
time window
Gives reasonable threshold for
determining presence/absence
Bird
No bird
Egg detection
Scale Invariant Feature Transform
(SIFT) for blob detection:
Each image is treated as a weak
classifier
Scale from characterised egg sizes
Mean, variance of intensity in blob
area
Weak classifiers used as input into a
Hidden Markov Model
HMM states are true egg counts,
observations are estimates
Trained on ground truth data sets
Simple filtering can be performed
Based on distance of features from
decision boundary
Progression
Acoustic array
Plant phenology
Automatic classification and census in addition to localization
Other event detection approaches
More complex detection for more complex flowers
SIFT shows promising results
Avian Biology
Different processing techniques work better at different
stages
Estimate stage, adjust processing accordingly
Conclusion
Event detection in signal stream is common theme
New challenges naturally follow
accuracy improvement
self-adaptation
context aware processing
Flexibility and iteration are key to addressing challenges
Automation for well characterised events in stream
Characterisation comes with application experience
Keeps human interactively in the loop
Interaction comes where scientist lacks confidence in
processing loop
Backup: Localization performance
RMS error 0.78 m