Transcript Interactive Visual System

Interactive Visual System
By
Arthur Evans, John Sikorski, and Patricia Thomas
The Client

Graduate student and professor at
Rensselaer Polytechnic Institute

Implemented CAVIAR on a desktop PC

Interested in a mobile version
CAVIAR
CAVIAR George Nagy, DocLab
RPI, Troy, NY 12180
13
The Idea




Image recognition is difficult
Humans excel at segmentation
Computers excel at large scale memory and
calculations
Combine the two for optimal performance

Faster than a layperson and more accurate than a
fully automatic system
Requirements
Demonstrate user interaction with an image on a
handheld computer.
Allow for three methods of user interaction







Pointing to image
Outline part of image
User response to queries
Top three matches must be displayed with target
image.
Create a log file to log user actions and time.
Hardware
Our Handheld Criteria

Must run java

Must take a camera attachment

Must take a wireless LAN card
Personal Digital Assistants (PDA’s)

PDA’s run Personal Information Management
(PIM) software.




Address books
Calendars
Limited computational power.
Examples:


Palm Pilot
Handspring Visor
Handheld Computers




Extensive computational power.
Designed as portable computers.
Fill the gap between desktop/laptop PC’s and
PDA’s.
Examples:


Compaq Ipaq series
Sharp Zaurus
Hardware Selection

Sharp Zaurus SL-5500




Sharp CE-AG06 camera attachment


200 MHz Intel StrongArm processor
64 MB memory
1 Compact Flash port and 1 SD port
Plugs into Compact Flash port
Linksys wireless LAN card (WCF11)


802.11b
Plugs into Compact Flash port
Design
Data Flow Diagram
Photo of flower
Handheld
Camera
Store image
as local file
FlowerData
object
Server/Pattern
Recognition Software
Transmit flowerData
Object to Remote
Server
Load user
selected photo
Transmit classification
decision to user
PDA Stylus Pad
User commands
and data
Transmit receive
confirmation back to
user
Encapsulate Flower Image
& results of user
interaction into flowerData
object
PDA screen display
Package Structure



GUI – implemented with AWT
 MainGUI class provides layout for panels
 Each ‘action’ implemented as a class that extends java.awt.Panel
 Additional supporting classes
Data
 FlowerData class encapsulates image and data to send to server.
 FlowerMatch class encapsulates image matches from recognition
engine.
 Additional supporting classes
Remote - access recognition engine via RMI
 Well defined protocol
 Modular design
 Consists of several server and client side classes
Java 2 Micro Edition (J2ME)

Recent specification

Provides Java functionality to a broad range
of devices

Uses concept of Configurations and Profiles
Basic J2ME Architecture
Profile
Configuration
Java Virtual Machine
Operating System
Configurations



Tied to JVM on a low level
Consider memory, processor, display and
connectivity
Currently 2 are defined


Connected, Limited Device Protocol (CLDC)
Connected Device Protocol (CDC)
Profiles



Sits on top of a Configuration
Provides implementation API
Currently 2 are defined:

Mobile Information Device Protocol (MIDP)


Typically implemented on PDA’s and cellular phones
Personal Profile


Targets high end handheld
Reference implementation available for the Zaurus
Personal Java

Implementation by Insignia Systems – Jeode

Not a J2ME specification

Based on JDK 1.1.8

Implements most Personal Profile
specifications
Summary



Successfully demonstrated user interaction
on a handheld
Wireless connection to server
Created structure for IVS application



Easily extendible
Identify different objects by changing database
and data points
Exceeds client expectations
Next Semester

Add pattern recognition



Call C++ code with JNI?
Rewrite in java?
Add segmentation


Automatic
User interaction