Transcript Analysing Edges to Classify Leaves

Leaf Classification from
Boundary Analysis
Anne Jorstad
AMSC 663 Project Proposal
Fall 2007
Advisor: Dr. David Jacobs, Computer Science
1
Background
Electronic Field Guide for Plants




University of Maryland
Columbia University
National Museum of Natural History
Smithsonian Institution
Project in development over 4 years
2
Background

Current System:




Inputs photo of leaf on plain
background
Segments leaf from background
Compares leaf to all leaves in database,
using global shape information
Returns images of closest matches to
the user
3
Background
Sean White, Dominic Marino, Steven Feiner. Designing a Mobile User Interface for Automated Species
Identification. Columbia University, 2007.
4
Background



All leaves assumed to be from
woody plants the BaltimoreWashington, DC area
245 species, 8000 images
The proof of concept has been
implemented successfully
5
Proposal

Current System:


All shape information is compared at a
global level, no specific consideration of
edge types
My Project:

Incorporate local boundary information
to complement existing system
6
Proposal
Leaf edges:
serrated,
finer teeth
“double-toothed”
serrated
smooth
lobed and
serrated
wavy
7
Proposal Specifics



Start with boundary curves as discrete
points (already have this data with good
accuracy)
Represent as
techniques
x(t )  iy (t ) , to use 1-D
Classify!
8
Method 1: Harmonic Analysis

Harmonic Analysis



Decompose boundary into wavelet
basis
Different families of species have
distinct serration patterns in the
frequency domain
What wavelet basis to choose?
9
Aside: What is a wavelet?


Fourier Transform: decomposes a
function into frequency components
Wavelet Transform: similar to
Fourier, but with quickly decaying or
compactly supported basis functions
 good for feature detection
10
Method 1: Harmonic Analysis



Think of the boundary as a texture
Several Computer Vision algorithms
exist for classifying textures
Example:
Describe texture in terms
of a set of fundamental
features or patterns
(sound like a wavelet
basis?), search for them
throughout the image
11
Method 2: Inner-Distance

“Inner-Distance” on multiple scales


Measures the shortest distance between
two points on a path contained entirely
within a figure
Good for detecting similarities between
deformable structures
12
Method 2: Inner-Distance



The inner-distance has been successfully
applied in several situations
Used already as part of the global
classification
New: sample points on several scales
and look for shape discrepancies not
previously measured
13
Method 2: Inner-Distance

Examining inner-distances over a
hierarchy of scales will capture new
local information
Large scale:
similar inner-distances
Small scale:
distinct inner-distances
14
Method 3: Convexity


A serrated leaf is much less convex
than a smooth one; use convexity
measure as a pre-processing
classification tool
May not prove useful, but might be
worth exploring
15
Method 3: Convexity

Several ways to assign a convexity
number to a shape:

Area(object)
Convexity
Area(ConvexHull(object))

Convexity
Perim eter(ConvexHull(object))
Perim eter(object)
etc.
object
ConvexHull(object)
16
Algorithm Verification


Create artificial “leaves” with known
properties
Prove algorithm correctness on
these simple known cases
17
Algorithm Verification

Run new algorithm on current
data sets


Demonstrate “reasonable”
classification accuracy for relevant
examples
Global information not considered,
so expect that not all distinguishing
features will be recognized
18
Algorithm Verification

Incorporate into existing system

Ideally:

Provide classification results
independent from current results, so
together a better overall classification
is achieved
19
Specifications

Current system: MATLAB and C

My contribution: mostly MATLAB


Image Processing Toolbox
Wavelet Toolbox
20
Specifications

End product to run on portable
computer


Code must run quickly on a small
processor
Development and testing from PC
21
References





“A New Convexity Measure for Polygons”. Jovisa Zunic, Paul L. Rosin.
IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.
26, no. 7, July 2004.
“Contour and Texture Analysis for Image Segmentation”. Jitendra
Malik, Serge Belongie, thomas Leung, Jainbo Shi. International
Journal of Computer Vision, vol. 34, no. 1, July 2001.
“Designing a Mobile User Interface for Automated Species
Identification”. Sean White, Dominic Marino, Steven Feiner.
Proceedings of the SIGCHI, April 2007.
“First Steps Toward an Electronic Field Guide for Plants”. Gaurav
Agarwal, Haibin Ling, David Jacobs, Sameer Shirdhonkar, W. John
Kress, Rusty Russell, Peter Belhumeur, Nandan Dixit, Steve Feiner,
Dhruv Mahajan, Kalyan Sunkavalli, Ravi Ramamoorthi, Sean White.
Taxon, vol. 55, no. 3, Aug. 2006.
“Using the Inner-Distance for Classification of Articulated Shapes”.
Haibin Ling, David W. Jacobs. IEEE Conference on Computer Vision
and Pattern Recognition, vol. II, June 2005.
22
Questions? Comments?
23