Transcript ğitim - Drexel University

Training and Workload Assessment
for Safe Piloting
‘Brain in the Loop’
Learning & Training
Systems
December 13, 2010
CONQUER CollabOrative
School of Biomedical Engineering, Science and Health Systems
Drexel University, Philadelphia, PA
Scope
…Merging two major advancements…
Increase in digital
computation and
storage
Dynamic, complex human
automation networks
Understanding of
human brain
function
Scope
Portable brain imaging technology
enables such convergence by
“bringing more of brain on
task”*
Brain
Activity
Crew can manage &
maintain high levels of
cognitive informationprocessing
Air Traffic
Controller
Cognitive
Workload Monitor
Safe & Effective
UAV Ground
Operator
Piloting
Training and
Workload Monitor
Monitor
Safe & effective
performance
*DARPA: Dylan Schmorrow, Ph.D. LCDR, MSC, US Navy, Program
Manager, ITO
Pilot
Expertise
Development
Monitor
Brain Activity Monitor
Functional Near-Infrared Monitor [fNIR]
>> Cortical activation-related to hemodynamic changes in natural
environments and diverse field conditions
Physiological Principles of fNIR
Neural Activity & Hemodynamic Response
Neurons consume energy (glucose)
when activated
Oxygen is required to metabolize the
glucose
As clusters of neurons are activated,
there is an increased need for oxygen
in that area
Oxygen is transported to neural tissue
via oxy-hemoglobin in the blood
The oxygen exchange occurs in the capillary beds
As oxy-hemoglobin gives up oxygen to the neural tissue, it is
transformed into deoxygenated hemoglobin
Oxy-Hb and deoxy-Hb are correlates of brain activity
through oxygen consumption by neurons
David J. Heeger & David Ress. (2002)What does fMRI tell us about neuronal activity? Nature Reviews Neuroscience 3, 142-151
Physical Principles of fNIR I
Photon Migration in Tissue
Source
Detector
Photons that enter the tissue undergoes two
types of interaction:
 Scattering
(cell membranes)
 Absorption
(Hb,HbO2water)
H Obrig , R Wenzel, M Kohl, S Horst, P Wobst, J Steinbrink, F Thomas, A Villringer. Near-infrared spectroscopy: does it function in functional activation studies
of the adult brain? International Journal of Psychophysiology 35(2-3):125-142, 2000
Physical Principles of fNIR II
Optical Window in Tissue
Stimuli
Presentation
Near IR
Peak
Oxygenation
Initial Dip
Post stimulus
undershoot
 (

C HB HBO
C HBO2 )L
 (HB
L
HBCHB
HBO22 CHBO
2

I I 0010
)
.G
IIBB




OD
 log
 ((
HB
C HB 

HBO
C
)L
OD 
log10
CHBO
10
HB CHB
HBO22
HBO22 ) L
II
Modified
Beer-Lambert
Law
 C Hb
Oxygenation  C HbO
HbO22
Hb
 CHb
BloodVolume  CHbO
HbO22
Hb
Io
.G
I
Air Traffic Controller
FAA Next Generation Air Transportation System
(NextGen) Study
Similar to SESAR (Single European Sky ATM Research) Program
• FAA NextGen – legacy versus new?
– Communication between air traffic controller (ATC) and the flight deck
– Transition from auditory-based communications (Voice Comm) to text-based
messaging (Data Comm) systems.
Allow ATC to manage more traffic with fewer cognitive
resources, thereby increasing both traffic capacity and
the safety of passengers
Need: objective, brain-based
measures of cognitive workload
Air Traffic Controller
FAA Next Generation Air Transportation System
(NextGen) Study
Similar to SESAR (Single European Sky ATM Research) Program
ATC part-task
Communication type: Voice Comm and
Data Comm
Workload manipulated by number of
aircrafts in the sector: n= 6,12,18
16-channel fNIR sensor
Air Traffic Controller
FAA Next Generation Air Transportation System (NextGen)
Similar to SESAR (Single European Sky ATM Research) Program
Results:
ATC Working Memory:
fNIR Measurements
ATC Task: fNIR Measurements
 Changes in blood
oxygenation in prefrontal
cortex were shown to be
associated with increasing
cognitive workload.
 Data-based required less
brain activation of the
operator than legacy Voicebased communication.
(F3,69= 4.37, p < 0.05)
Aircraft: (F2,42 = 6.39, p < 0.05) Comm: (F1,21 = 10.57, p < 0.05)
UAV Piloting
US Army Medical Research and Materiel Command
(USAMRMC)
The Telemedicine & Advanced Technology Research
Center (TATRC)
 UAV ground controller’s expertise
level, cognitive workload and
situational awareness are critical.
operation failures due to human error
(60% - 80% )
 ground flight crew’s mental
workload increase
 inadequate training
UAV Piloting
Integration of the functional
brain monitoring system with
 training procedures
 ground control systems
objective measure of expertise level to validate adequate
training
 real-time monitor of mental workload

safe operation of UAVs…
UAV Piloting
UAV Operators Cognitive Workload Assessment Study
Comparison of Day 2 (first task) and Day 9 (last task) yields significant drop on average oxy-Hb.
Landing Task (day2 vs. day9)
Approach Task (day2 vs. day9)
fNIR Measures
(day2 vs. day9)
Self-reported Measures
Menda J, Hing JT, Ayaz H, Shewokis PA, Izzetoglu K, Oh, P, Onaral B (in print). Optical Brain Imaging to Enhance UAV Operator Training, Evaluation, and Interface
Development. Journal of Intelligent and Robotic Systems.
National and International
Partnership
University of Maryland (Dr. José L. Contreras-Vidal)
University of Pittsburgh (Dr. Joseph Ricker)
Albert Einstein College of Medicine (Dr. Roee Holtzer)
Italy – Polytechnic University of Milan (Dr. Sergio Cerutti)
Spain – University of Seville ((Dr. Jose Leon-Carrion)
– Hospital Nacional de Paraplejicos Toledo & Dr. Guglielmo Foffani)
Israel – Haifa University (Dr. Zvia Breznitz)
Turkey – Middle East Technical University (Dr. Veysi Isler)
– Dokuz Eylul University (Dr. Murat Ozgeren )
Japan – Bukkyo University (Dr. Yasufumi Kuroda)
China – Wuhan National Laboratory for Optoelectronics (Dr. Britton Chance & Dr. Shoko Nioka)
United Kingdom – The Defence Science and Technology Laboratory (DSTL)
Thank you!
CONQUER Collaborative
3508 Market Street
Philadelphia, PA, USA 19104
WWW.BIOMED.DREXEL.EDU
http://www.biomed.drexel.edu/fNIR/CONQUER/Welcome.html