Transcript Translation of Sign Language to Speech

Translation of Sign
Language to Speech
Sensor Glove Development
Name:
Garry Rank
Student Number: 12046339
Supervisor:
Mr Iain Murray
Design Requirements






Fast
Accurate and sensitive.
Robust
Low cost
Low profile – the glove should not impede
signing.
Be able to detect everything necessary for
translation.
These are all problems that make most virtual reality
gloves unsuitable for the task!
Required Data




Amount of bend of base and first knuckle of each
digit.
Pitch of the wrist.
Detection of finger splaying and fingers crossed.
General area contacts.
Top of hand
Side and palm of hand
General hand locations important to ASL:
Bend Detection

Capacitors lend a natural solution.
 Sliding
motion to change surface area.
 Capacitance is changed.
 Detectable by a micro-controller’s analogue to
digital converter (ADC).
Contact Sensors


Primary and Secondary contact locations.
Primary
 Finger splaying.
 Touching the palm.
 Fingers crossed.

Secondary
 Fingers
touching elsewhere.
 Chest, head, arms or elsewhere on the body.

Different types of contact sensors for primary
and secondary locations.
Primary Contact





Improve speed of sign recognition. Amount of
finger bend is redundant data if it is known a
finger is touching the palm.
Anode on palm and tops of fingers.
Cathode on finger tips.
When the fingers touch the palm a circuit is
closed.
Same reasoning can be used for finger splaying.
Primary Contact Locations
Anode
Cathode
Palm of Hand
Top of Hand
Note that finger splaying contacts are not shown here.
Secondary Contact
Used to supplement overall sign
recognition of the system.
 A simple button used to detect when a
finger is touching something not
detectable by the glove.
 Not as sensitive to soft contact

 Some
signs could be missed
Secondary Contact Locations
Palm of Hand
Top of Hand
Testing Bend Sensors




m16c development board linked to a PC via
serial cable.
8bit ADC resolution mode used for its increased
speed.
Results showed 10° accuracy possible, but
interference may produce erroneous values.
10bit resolution on the ADC offered a
dramatically increased reliability and accuracy.
Software Flowchart
Start Here
Set output pin
high (5V)
Transmit value
to PC
Use ADC for
voltage reading
Wait for
capacitor charge
Set output pin
low (0V)
Wait some
time T
Timing






A real time system – at least 10 full glove
readings per second.
Including one for the wrist, three for the thumb
and two for each finger, 24 bend sensors may
be required. Conversion timing is important!
Approximate 6us conversion time.
Worst case 40us additional wait time.
With up to 24 bend sensors, this is means a
1104us bend sensor clock time.
Well within acceptable limits!!!
ADC Input Multiplexing
m16c micro-controller doesn’t have
enough ADC inputs for all the required
bend-sensors!
 Inputs must be multiplexed.
 Diodes can be used to prevent all
capacitors multiplexed from being
charged.

Multiplexed Input
If only A, only B or only C is
turned on, then only one
capacitor will affect the
value as read from the
voltage probe (ADC input).
Two Gloves



The sensors so far assume only one glove. Two
handed interaction should also be detected.
More contact sensors seems like a good idea, or
is it?
Pins on microcontrollers are I/O.
 They
can be input or output.
 By controlling this, much more data can be obtained
from the same number of sensors.
 Incurs a time penalty.
Problems

The glove needs to be specially made.




Primary and secondary contacts become more sensitive
with small, hard locations to attach to.




Material moves too much with the hand.
Capacitors sewn onto the glove move with the material. Bending
of the wrist causes the most problems here.
Wrist bending is calculated and so these errors can be
algorithmically reduced
Thin plastic can be used as an exoskeleton on which to mount
sensors.
Straps to keep the glove fixed on the hand.
Electrical interference can induce currents that produce
errors in results.
These are all manufacturing problems, not theoretical
ones!
Thanks for Listening!
Feel free to ask some questions.