Transcript final presentation - Faculty Server Contact

By
Carl Tenenbaum
David Haynes
Philip Pham
Rachel Wakim
History of Driver Safety








1930s- Seat Belt first introduced
1949- Safety Cage and Padded Dashboard
1966- National Transportation Safety Board
1978- Child’s Booster Seat
1979- Car Crash Testing
1981- Airbag Introduced
1984- NY Enforced Seat Belt Use
2004- Rollover Risk Test
Causes of Car Accidents
Distracted Drivers (12% was Driver
Fatigue)
2. Driver Fatigue
3. Drunk Driving
4. Speeding
5. Aggressive Driving
6. Weather
1.
* According to Sixwise.com
Driver Fatigue Results
The National Highway Traffic Safety
Administration Yearly Statistics
 100,000 police-reported crashes
 1,550 deaths
 71,000 injuries
 $12.5 billion in monetary losses.

It is difficult to attribute crashes to sleepiness
To be attractive, a vehicle sensor
system should be:
Fairly inexpensive,
 Accurate, with a quick response time,
 Integrated with the car design, or at
least “plug and play”,
 Noninvasive,
 Discreet, and non-distracting,
 Adaptable to different user conditions:
i.e., sunglasses, gloves.

Head Position Detection



Sense changes in Head Position Tilt
Gives off a warning if the Head Tilt is facing a downward
angle. Does Not detect head backwards or turned.
Head Position Down is the Last Stage of Sleep Onset.
Usually too late and no warning to Driver.
Detect
Head
Angle
Is Head
Tilted?
Audio
Alarm
Reed Switch Device
Reed
Switch
Speaker/
Buzzer
Battery
Voice Detection



Sense changes in Discrete Voice Parameters such as pitch,
frequency, latency and amplitude.
A complex detection algorithm compares normal voice to
sample of potential fatigued voice
Can be integrated in GPS or command oriented car
systems
Voice Channel
Types of Voice Sounds
 Voiced
 Nasal
 Fricative
 Plosive
(Easiest to
detect
Fatigue)
Behavioral Detection
Sense Erratic Driving Behavior
 Stores Profile of Person’s Driving
Behavior
 Compares Profile such as Driver’s
Steering and Braking Reaction Time

Behaviors Detected
 Steering
Wheel Angle
 Steadiness of Wheel
 Lane Departure Proximity
 Braking Reaction
 Acceleration Reaction
Steering Angle Sensors
Use Mechanical (potentiometers) or
Optical (contact-free) technologies to
collect data or apply correction
 Mount on steering shafts
 Cover up to 1080o (3x steering wheel
rotations)
 Angle resolution of 0.1o

Lane Departure Warning
Use video, laser, and infrared to monitor
the lane markings
 Activate Vehicle Stability Control
(Infiniti), Electric Power Steering
(Lexus), etc. to maintain lane position

Driving Behavior (Steering Angle)
Driving Behavior (Gas Pedal)
Driving Behavior (Center Lane Distance)
Current Behavioral Sensors
Mercedes E-Class, Volvo, Lexus,
Nissan, Infiniti, Volkswagen
 Aftermarket- 3Q(2011)

 AudioVox ($600)
*Daimler Chrysler Website
Optical Detection



A camera or system of cameras monitor the driver’s facial
features for signs of drowsiness.
Computer algorithms analyze blink rate and duration.
Infrared LEDs are used to enhance pupil detection.
Yawning and sudden head nods are also detected.
Head/eye Camera
Measure head tilting/eye closing/yawning
as signs of fatigue or drowsiness.
 Non-invasive, no need for user
interface.
 Can be thwarted by sunglasses or hats.
Driver movement may confuse the camera.
 1/5 people do not show eye closure as a
warning sign. [US Dept. of Transportation]

Pupil Detection on Grayscale Image
Facial Feature Detection
Possible Camera Locations
Current Optical Systems
 Nap Alarm
 DD850
(LS888)
Driver Fatigue Monitor
Biometric Detection
EKG and EEG
 Blood pressure
 Skin conductivity (“GSR” – Galvanic
Skin Response)
 Skin temperature
 Breathing rate
 Grip force


All shown with correlation to relative
drowsiness
Electrocardiogram (EKG)



Get information about user’s heart rhythm from at
least two electrical contacts on skin.
By removing common mode noise and amplifying
the signal, a system can “read” the user’s heart
rate, the distance between successive “R” peaks
Drowsiness has been shown to be linked to
decreasing heart activity and changes in heart rate
variability (HRV)
Minimum EKG System
As long as there are at least two contact points,
sensor should be able to extract and isolate the
signal
 Can put these on wheel, seat, or both

Wheel sensor
Use sensors on steering wheel to measure
skin temperature and conductivity, pulse, etc.
 Estimate heart rate variability – can detect
drowsiness.
 Combines many different
metrics to get an overall
assessment of the user’s
state.
 Requires use of both hands,
without gloves.

Seat sensor

Two pieces of conductive fabric on the
driver’s seat (backrest) can take an ECG
- measurement.
•Or on bottom of seat,
with wheel as ground
(only needs one hand)
•Needs impedance
compensation for the
driver’s shirt/coat, etc.
Electroencephalogram (EEG)
Use multiple electrodes on scalp
to read brain waves
 Can very accurately determine
sleep/drowsiness stage this way
by measuring
amplitude/frequency variation of
signal
 BUT, very invasive

Other Possible Sensor
Locations
Blood pressure finger cuff on front seat
 EKG contacts on left or right armrests
 EKG sensors on shifter
 Etc.

Or any combination of these.
 Theory: the more bio-signs, the better!

Wireless wrist monitor





Wristwatch capable of detecting heart rate,
skin temperature and conductance.
Example: “Exmovere Empath Watch”:
Transmits via Bluetooth to phone which can
signal out; easily extended to cars, many of
which already are Bluetooth compatible.
Current design is 3.3” long, 1.7” wide, and 1.3”
tall.
Can be bulky, and may
not be appealing enough;
currently being remodeled
[http://www.exmovere.com/healthcare.html]
Current Biometric Detection
Systems
Currently, there are no systems of these
types in commercial use
 They all display a high level of accuracy,
but their weak point is their invasiveness
and unattractiveness
 With future work, some of these can be
integrated in a behind-the-scenes
manner during manufacturing

Fuzzy Logic Detection
More Uncorrelated Sensors Detecting Driver
Fatigue Will Increase Detection Probability
Corrective and Prevention Actions
1.
2.
Elevated Alarms
a) Provide Visual Alarm (lights, signs, etc.)
b) Provide Audio Alarm (warning tone or voice)
c) Recommend short nap (prevent car to start; studies
show 15-minute nap increases alertness to 4-5 hours
more)
Mechanical and Electronic Stimulations
a) Counteract to the effects (steering wheel turn, lane
drifting, speed change, etc.)
b) Apply brake to slow down to safety
c) Dispatch for help if no response
Corrective Flowchart Actions
Current Driver Fatigue Products
Non-
Overall
Products
Price
Accurate
Invasive Effective
Score
Company
Detection Type
Driver Nap Zapper
25
50%
3
5
No Nap
Motion
3
Leisure
Nap Alarm (LS888)
Auto
500
80%
5
6
6
Security
Optical
500
80%
5
6
6
Eye Alert
Optical
WristWatch
1000
90%
6
5
6
Exmovere
Biometric
Driver Assist Package
3000
90%
7
7
7
Mercedes
Behavioral
DD850
Driver
Fatigue
Monitor
Exmovere
Empath
Undeveloped Market. US Consumer Car GPS Market is $5.1 Billion Market in
2010.
Limitations and Future Work

Limitations
 Probability of Detection
 Lack of Effective and Timely Alerts
 Integration of Sensors

Future Work
 Increase Probability of Detection
 Use of Multiple Sensors to Increase
Probability
 Develop Effective and Timely Alerts
References











[1] “The 6 Most Common Causes of Automobile Crashes(2010)”. Retrieved February 9th 2011, from
http://www.sixwise.com/newsletters/05/07/20/the_6_most_common_causes_of_automobile_crashes.h
tm
[2] K. Strohl, J. Blatt, F. Council, K. Georges, J. Kiley, R. Kurrus, A. McCartt, S. Merritt, R.N, A. Pack,
S. Rogus, T. Roth, J. Stutts, P. Waller, and D. Willis, “Drowsy Driving and Automobile Crashes” (2010),
Retrieved February 21st 2011, from
http://www.nhtsa.gov/people/injury/drowsy_driving1/drowsy.html#NCSDR/NHTSA
[3] What causes Fatigue (2010), Retrieved February 21st 2011, from
http://unsafetrucks.org/driver_fatigue.htm
[4] H. Greeley, E. Friets,, J. Wilson, S. Raghavan and J. Berg, “Detecting Fatigue From Voice Using
Speech Recognition”, 2006 IEEE International Symposium on Signal Processing and Information
Technology
[5] D. Hu, G. Gong, C. Han, Z. Mu, and X. Zhao, “Modeling research on Driver Fatigue”, 2010
International Conference on Computer Application and System Modeling (ICCASM 2010)
[6]L. Bergasa, J. Nuevo, M. Sotelo, R. Barea, and M. Lopez, “Real-Time System for Monitoring Driver
Vigilance”, IEEE Transactions on Intelligent Transportation Systems, Vol. 7, no. 1, March 2006
[7] Z. Zhu, Q. Ji, K. Fujimura, and K. Lee, “Combining Kalman Filtering and Mean Shift for Real Time
Eye Tracking Under Active IR Illumination”, International Conference on Pattern Recognition,
Quebec, Canada, 2002
[8] US Department of Transportation, “An Evaluation of Emerging Driver Fatigue Detection Measures
and Technologies”, June 2009
[9] Haisong Gu, Qiang Ji, and Zhiwei Zhu, “Active Facial Tracking for Fatigue Detection” IEEE
Workshop on Applications of Computer Vision, Orlando, Florida, 2002.
[10]Y. Jie, Y. DaQuan, W. WeiNa, X. XiaoXia, and W. Hui, “Real-Time Detecting System of the
Driver’s Fatigue”, 2006
[11]L. Bergasa, J. Nuevo, M. Sotelo, R. Barea, and M. Lopez, “Real-Time System for Monitoring
Driver Vigilance”, IEEE Transactions on Intelligent Transportation Systems, vol. 7, no. 1, March, 2006
References (Continued)











[12] S. Deshmukh, D. Radake, K. Hande , “Driver Fatigue Detection Using Sensor Network”,
International Journal of Engineering Science and Technology, NCICT Conference Special Issue, pp
89-92, February 2011
[13] Y. Tanida, H. Hagiwara, “Simple Estimation of the Falling Asleep Period using the Lorenz Plot for
Heart Rate Interval”, JSMBE vol. 44, no. 1, pp. 156-162, Nov. 2005.
[14] S. Kar, M. Bhagat, and A. Routray, “EEG signal analysis for the assessment and quantification of
driver’s fatigue”, June 2010
[15] L. Servera, M. Fernandez-Chimeno, and M. González, “Study of Sleep Stages By Controlled
Inducement and Measurement of Drowsiness Related Biomedical Signals”, 4th International IEEE
EMBS Conference on Neural Engineering, April 2009
[16]P. Kithil, R. Jones, and J. MacCuish, “Development of Driver Alertness Detection System Using
Overhead Capacitive Sensor Array”, International Driving Symposium on Human Factors in Driver
Assessment, Training and Vehicle Design, Aspen, CO, 2001.
[17]X. Yu, “Real-time Nonintrusive Detection of Driver Drowsiness”, May 2009
[18] G. Yang, Y. Lin, and P. Bhattacharya , "A driver fatigue recognition model using fusion of multiple
features" Systems, Man and Cybernetics, 2005 IEEE International Conference on , vol.2, no., pp.
1777- 1784 Vol. 2, 10-12 Oct. 2005
[19]The John Hopkins university Applied Physics Laboratory “Technologies: Drowsy Driver Detection
System” http://www.jhuapl.edu/ott/technologies/featuredtech/DDDS/
[20]T. Matsuda and M.Makikawa, “ ECG Monitoring of a Car Driver Using Capacitively-Coupled
Electrodes”, 30th Annual International IEEE EMBS Conference ,Vancouver, British Columbia,
Canada, August 2008
[21]Y. Lin, H. Leng, G. Yang, and H. Cai, “An intelligent noninvasive sensor for driver pulse wave
measurement,” IEEE Sensors J., vol. 7, no. 5, pp. 790–799, May 2007.
[22] M. Bundele, and R. Banerjee, “Design of Early Fatigue Detection Elements of a Wearable
Computing System for the Prevention of Road Accidents”, IEEE,International Society of Automation,
Vol 1 , pp 136-139, 2010
References (Continued)



[23]I. Jeong, S. Jun, D. Lee and H. Yoon, “Development of Bio Signal Measurement System for
Vehicles”, 2007 International Conference on Convergence Information Technology
[24]Exmovere Holdings Inc, “The New Biotechnological Frontier: The Empath Watch”. Feb. 2011
http://www.exmovere.com/pdf/Exmovere_Wearable_Sensor_Research.pdf
[25] Frost & Sullivan’s, North American GPS Equipment Markets, 2010 (Report A601-22)