Transcript Presentation

SP13 ECE 445: Senior Design
Sign Language Teaching
Glove
Project #29:
Reebbhaa Mehta, Daniel Fong, Mayapati Tiwari
TA: Igor Fedorov
Introduction
• Motivation
- No available portable devices to teach sign language
• Objective
- Sensing unit to detect gestures accurately
- Kalman Filter to reduce noise from data
- Program to check each gesture
- Provide feedback to user
American Sign Language
Overview
System Components
• Three Main Units
- Sensing unit (MPU 6050 & Flex Sensors)
- Software Component
- Feedback unit (LED’s)
System Components
• Hardware
- Power Supply – 9V
- Microcontroller Unit – Arduino Uno
- 5 flex Sensors & 5 MPU 6050
- Bluetooth
- 5 Sensors
- LED Driver
• Software
- Arduino Programming Environment
- Kalman Filter
- Perceptron Learning Algorithm
Power Supply – Actual Choice
• First Choice – Lithium Backpack
- Supplies 5V to power Arduino Uno
- Supplies 3.3V to power all other components
- Matches Arduino size and fits on the back
- Rechargeable via USB
- Shorted !!
Power Supply – Alternate
• Alternate – Energizer 9V
- Supplies 9V
- Arduino on-board regulator generate 5V to power
Arduino Uno
- Arduino on-board regulator generate 3.3V to power all
other components
- Not very useful for effective space
utilization
- Not rechargeable
Sensing Unit
Consists of:
- 5 accelerometers and gyroscopes (MPU 6050)
- 5 flex sensors (FLX-03)
- I2C Multiplexer (TCA9548A)
Accelerometer & Gyroscope
MPU-6050
- Accelerometer & gyroscope in one chip
- Helps to detect gestures
- Accelerometers detect tilt
- Gyroscopes detect angular velocity
- Better space utilization
- Placed near fingertips – PCB’s
need to be small
MPU-6050
MPU-6050 Data
• Orientation from gravity
• Roll  tan 1(
• Pitch  tan 1(


ay
)
az
ax
(ay)  (az)
2
2
)
• Gyroscope for change in orientation
MPU 6050 Schematic
MPU 6050 PCB Pic
Flex Sensors
• Uni-directional flex sensors (FLX03) used
• Help to provide more accurate data
• Range: 10kΩ to 40kΩ
- For unflexed hand: 10kΩ
- For completely flexed hand: 40kΩ
Flex Sensor Circuit
Flex Sensor – Test & Data
0.5
1.5
Voltage (V)
2.5
V vs R
9
14
19
24
Resistance (Kohm)
R (Flex Sensor)
Vout
9.48 kΩ
1.78 V
15.3 kΩ
1.69 V
17.0 kΩ
1.59 V
21.2 kΩ
1.36 V
22.7 kΩ
1.32 V
Flex Sensor - Problems
• Problem
- Broke due to heat
- Used copper tape to fix the
problem without success
- Only one working flex sensor
• Consequences
- Less data from sensing unit
- Reduced accuracy to differentiate between gestures
- Failed requirement
I2C Multiplexer (TCA9548A)
• 8 bi-directional translating switches
• I2C bus compatible
• Channel selection via I2C bus
• 8-channel I2C switch communicate
with up to 8 I2C devices which have
the same address
Feedback Circuit
• Controls 10 LEDs
using 8 bit shift
register
• 3 arduino output
pins.
Microcontroller
• Arduino Uno
- Serial Communication
- Easily programmable
- Attaches to Lithium Backpack
(better space utilization)
- Voltage regulators to provide
3.3V and 5V
- Works well with external battery
- I2C protocol MPU 6050 communication library
Bluetooth
• Bluetooth Shield
- Arduino Uno compatible
- UART communication
- Up to 10m communication
- Fits the back of Arduino Uno
Requirement Failed: No
connection with computer.
Reason: Broken antenna or
chip not programmable
Kalman Filtering
F = state
• u = gyroscope data
• Var(w) = Q
• Var(v) = R
• H = [1;0]
• B = delta (T)
•
Kalman Filtering
• Estimation:
Kalman Filtering
• Update:
Kalman Filtering
Test A Sensor 1 – Little Finger
Kalman Filtering
Kalman Filtering
Kalman Filtering
Perceptron Learning Algorithm
•On-line, mistake driven algorithm.
•Linear classifier that updates the weight vector
incrementally when mistakes are made.
•Decision rule checks whether the dot product of the
weight vector with an input vector is greater that some
threshold
.
•
Perceptron Learning Algorithm
Failed Verification
Problem
Reason
Bluetooth
Initialization code
stopped working
after a while
Defective piece
(maybe)
Flex Sensors
Not within the
specified range
Broke due to heat –
resistive strip came
off
Perceptron
Wanted – 98%
accuracy
Achieved – 75%
No data from flex
sensor.
Adding more
variables.
Feedback Unit
Not integrated
No real time data
processing
Accomplishments
• Optimum data from MPU-6050
• Kalman Filter
• Perceptron to check gestures
• Effectively differentiate between 5 gestures (A, B, L, V
and Y)
• LED for feedback
• Soldering very small components like MPU-6050
Future Steps
• Replacing broken flex sensors to get more
accurate data for each gesture
• Increase the accuracy of perceptron by adding more
features
• Real time implementation
• Better feedback unit with haptic feedback and LED’s
• With more accuracy and better detection words can be
added to the library and thus progress can be made
Acknowledgement
Prof. Scott Carney
Igor Fedorov
Mark Smart
Skot Wiedmann
Waltham Smith
Daniel Mast
Aadhar Jain
Joseph Shim
Questions?