Transcript Project presentation slides 2.

SAVI
Shopping assistant for the visually impaired
Andrew Ebaugh
Saurav Chatterjee
Basic concept of SAVI

What does it do?


SAVI allows blind people to locate and identify
items via RFID technology.
What is it good for?



Blind people can shop by themselves.
Facilitates in identifying items once they get
home.
It can substitute for Braille signs and tags.
Why is SAVI needed?
Currently, blind individuals cannot
navigate a grocery store independently.
• Have to depend on others to help with
shopping (Braille stickers on products)
• Little choice on prices and brand of
items being purchased
• Have to depend on others when
checking out at the register
• Organizing their items once they return
home is a problem
Product Demo


We will display three active pieces: Personal Server,
ear piece, & scanning glove
Simple demo idea: Identify common grocery items
with our system without using sight.




Plan to set up a display of several tagged items on shelves
Generate a grocery list from among the several possible items
Locate the correct items by handling the objects and scanning
the tags
As our audio board also transmits over FM, we’ll have a
small stereo set up to broadcast what the device user will
be hearing
Product Demo, cont.

Our original scenario was pretty close to this



it was for an actual user in a grocery store
It showed a user receiving audio feedback at the
register, which we will not demo
It also showed the user putting away groceries at
home, which we will not demo
Implementation – Overall diagram
RFID
Input
Output
Audio
m
ot
e
Radio
Personal
Server
Linux
JD
BC
SQL
ODBS
dB
Processing
MP3
Implementation, cont.
• RFID scanner – mote running TinyOS transmits RFID tag
scan to a mote in a CF card on personal server
• Serial forwarder service – dumps whatever is received on
the serial port to listeners on a network port (4444 TCP)
• ODBS listener service – Connects to network port, and
registers as a listening client, accepting forwarded packets.
ot
e
Radio
m
RFID
• CF card mote – accessible as a serial port on personal
server, /dev/tts/3
Personal
Server
Linux
ODBS
Implementation, cont.
• ODBS – Multi-threaded: port listening
thread, database query thread, output
thread
• Product information is stored in SQL
database
• Upon receiving a RFID scan, the
service queries the product database
Personal
Server
• Additionally queries for brand, price,
sale information
Linux
JD
BC
• Each piece of information returns an
associated audio clip; from all of these,
a presentation “playlist” is generated
SQL
dB
ODBS
Implementation, cont.
dB
• Database is provided by sqlite and accessed by sqlite
JDBC-connector
• Contains 7 tables: product, brand, measure, location,
sale, alphanum
Product table pseudo-SQL :
CREATE TABLE product (rfid, upc, brand_id, name, price, price_unit_id,
price_quantity, sale_type, sale_quantity, sale_price, quantity,
quantity_unit_id, location_id, section_id, clip)
Ex:
INSERT INTO product VALUES ('E00781BCC1957A75', '04482008', 9,
'Oreo Cookies', '3.69', NULL, NULL, 2, NULL, NULL, '1.13', 0, 9, 18,
'product-11');
Implementation, cont.
Personal
Server
MP3
Ru
nt
im
e
Linux
ODBS
Audio
• Presentation clip – Once the clip has been generated, ODBS makes a call to
Runtime.exec() to execute an mp3 audio player
• Execution call takes the form of:
madplay /mp3/brand-04.mp3 /mp3/product-02.mp3 /mp3/alphanum-39.mp3
• MP3 player talks to audio board as /dev/dsp
Implementation, cont.
Audio
• Slappy board – Audio is provided by board that plugs into personal server
• Board drivers are modular and with dependent modules (soundcore,
ac97_codec) provide standard linux audio interface to be accessed by
applications
• In addition to standard stereo output and line input, provides FM
transmission on configureable frequency.
• All this together gives a range of options, from enhanced ODBS access of
audio board, to a multitude of output devices
Implementation, cont.

What is done?



All the pieces described so far are working
together
System works end-to-end, from tag scan to audio
What isn’t?



Mainly finishing things up and work on aesthetics
Glove and personal server aren’t currently
something one would want to wear
Last minute hardware glitches
Evaluation

How well does it work?

Once the kinks are worked out, the system works
pretty well. The key metrics:




Distance glove needs to be from tag
Length of time glove needs to be reading tag
Delay between successful scan and start of audio
(i.e. processing time)
Length and relevance of audio clip
Evaluation, cont. – metrics

Distance glove needs to be from tag



Depends on how charged the batteries are, and how
sensitive the antenna is
For our glove reader with a Mylar antenna in palm and
full batteries, the distance is 3-4 inches.
Length of time glove needs to be reading tag


We have toyed with different reading frequencies to tune
this metric
Currently check for a tag 4 times per second. At this
speed, the reader can detect a tag when moved by it at
an acceptable speed (i.e. as if you were feeling around
the object)
Evaluation, cont. – metrics

Delay between successful scan and start of audio





This was our main concern, that doing the processing
and database queries would take time
Actually, this time is minimal
From successful scan, audio will start in under a second
The longest portion of this is starting up the mp3 player,
suggesting a possible point for improvement
Length and relevance of audio clip



Probably needs the most work
The overall clip is built on subclips. Currently output is:
brand, name, price, any sale information
It would be best if this was configurable, because
playing this information can take some time
Future Work

Lots to be done, lots of possibilities







Database contains a lot of information not currently used: UPC
code, location, quantity
There should really be an audio output component always
loaded, rather than making an external call
The product database could be a central store database
accessed over wi-fi or similar
Could integrate a shopping list or store navigation aid
Would like to have a scanner at the checkout register that
relates product name, price, subtotal as rung up
The glove we have is definitely a prototype: should have a
lightweight, chargeable battery, and several buttons
Possible text-to-speech for audio generation, and linking with
existing UPC-based product databases