Transcript No Slide Title - University of Connecticut

Brain Computer Interface in BMI
CSE
5810
Ioannis Papavasileiou
Computer Science & Engineering Department
The University of Connecticut
371 Fairfield Road, Unit 4155
Storrs, CT 06269-2155
[email protected]
Papavasileiou-1
What is BCI?

CSE
5810

BCI is:
 System that allows direct communication pathway
between human brain and computer
 It consists of data acquisition devices, and
appropriate algorithms
How is it used in BMI:
 Clinical research
 Disease-condition detection and treatment
 Human computer interfaces for
 Control
 Emotions detection
 Text input - communication
Papavasileiou-2
Research areas involved

CSE
5810



Computer science
 Data mining
 Machine learning
 Human computer interaction
Neuroscience
Cognitive science
Engineering
Papavasileiou-3
Key challenges

CSE
5810

Technology-related:
 Sensor quality – low SNR
 Supervised learning – “curse of dimensionality”
 System usability
 Real-time constraints
 Non-invasive EEG information transfer rate is
approx. 1 order of magn. lower
People-related
 People are not always familiar with technology
 Preparation – training phases are not fun!
 Concentration, attention consciousness levels
 Task difficulty
Papavasileiou-4
BCI components

CSE
5810
Data acquisition
 Electroencephalography (EEG)
 Electrical activity recording
 Invasive or not

Functional Near Infrared Spectroscopy (fNIRS)
 Recording of infrared light reflections of the brain

Functional magnetic resonance imaging (FMRI)
 Detection of changes in blood flow



Data Analysis
 Data mining & machine learning
Decision making
Output & Control
 HCI
Papavasileiou-5
Typical BCI architecture
CSE
5810
Papavasileiou-6
Electroencephalography (EEG)

CSE
5810
What is it:


Types:



Recoding of the electrical activity
of the brain
Invasive
Non-invasive
Properties:



High temporal resolution
Low spatial resolution
Scalp acts as filter!
Papavasileiou-7
International 10-20 standard

CSE
5810

Electrodes located at the scalp at predefined
positions
Number of electrodes can vary
Papavasileiou-8
The EEG waves

CSE
5810



Alpha –
occipitally
Beta – frontally
and parietally
Theta – children,
sleeping adults
Delta – infants,
sleeping adults
Papavasileiou-9
fMRI

CSE
5810






Functional magnetic resonance imaging
Fact:
 Cerebral blood flow and neuronal activation
coupled
Detection of blood flow changes
Use of magnetic fields
High spatial resolution
Low temporal resolution
Clinical use:


Assess risky brain surgery
Study brain functions
 Normal
 Diseased
 Injured

Map functional areas of the brain
Papavasileiou-10
fNIRS

CSE
5810




Functional Near Infrared Spectroscopy
Project near infrared light into the brain from the scalp
Measure changes in the reflection of the light due to
 Oxygen levels associated with brain activity
 Result absorption and scattering of the light
photons
 Used to build maps of brain activity
High spatial resolution
 <1 cm
Lower temporal resolution
 >2-5 seconds
Papavasileiou-11
BMI & clinical applications

CSE
5810


Diagnose:
 Epilepsy – seizures
 Brain-death
 Alzheimer’s disease
 Physical or mental problems
Study of:
 Problems with loss of consciousness
 Schizophrenia (reduced
Delta waves during sleep)
Find location of:
 Tumor
 Infection
 bleeding
Source: http://www.webmd.com/,
http://www.nlm.nih.gov
Papavasileiou-12
Sleep disorders & mental tasks

CSE
5810

Sleep disorders study
 Insomnia
 Hypersomnia
 Circadian rhythm disorders
 Parasomnia (disruptions in slow sleep waves)
Mental tasks monitoring
 Mathematical operations
 Counting
 Etc.
Papavasileiou-13
Neurofeedback

CSE
5810
Applications in
 Autistic Spectrum Disorder (ASD)
 Anxiety
 Depression
 Personality
 Mood
 Nervous system
 Self control
Papavasileiou-14
Feedback EEG-BCI architecture
CSE
5810
Papavasileiou-15
Typical data analysis process

CSE
5810
Data acquisition and segmentation
 Preprocessing
 Removal of artifacts
 Facial muscle activity
 External sources, like power lines

Feature extraction
 Typically sliding window
 Time-frequency features
 Latency introduced
Papavasileiou-16
Feature extraction

CSE
5810



Model-based methods
 Require selection of the model order
FFT (Fast Fourier Transform) – based methods
 Apply a smoothing window
Features used:
 Specific frequency band power
 Band-pass filtering and squaring
 Autoregressive spectral analysis
Many times a feature selection or projection is done to
reduce the huge feature vectors
Papavasileiou-17
Data Classification

CSE
5810

Typical classifiers used
 Artificial Neural Networks (ANN)
 Linear Discriminant analysis (LDA)
 Support Vector Machines (SVM)
 Bayesian classifier
 Hidden Markov Models (HMM)
 K-nearest neighbor (KNN)
Parameters for each classifier can affect the
performance
 # of hidden units in ANN
 # of supporting vectors for SVMs
 Etc.
Papavasileiou-18
Human computer interaction

CSE
5810

BCIs are considered to be means of communication
and control for their users
HCI community defines three types:
 Active BCIs
 Consciously controlled by the user
 E.g. sensorimotor imagery (multi-valued control signal)

Reactive BCIs
 Output derived from reaction to external stimulation
 Like P300 spellers

Passive BCIs
 Output is related to arbitrary brain activity
 E.g. memory load, emotional state, surprise, etc.

Used in assistive technologies and rehabilitation
therapies
Papavasileiou-19
BCI & Assistive Technologies

CSE
5810

Communication systems
 Basic yes/no
 Character spellers
 Virtual keyboards
Control
 Movement imagination
 Cursor
 Wheelchairs
 Artificial limbs & prosthesis
Automation in smart environments
Current BCI systems have at most 10-25 bits/minute
maximum information transfer rates
 It can be valuable for those with severe disabilities


Papavasileiou-20
P300 spellers

CSE
5810

Most typical reactive BCI
3-4 characters / min with 95% success
Papavasileiou-21
P300 wave

CSE
5810



Event related potential (ERP)
Elicited in the process of decision making
Occurs when person reacts to stimulus
Characteristics:
 Positive deflection in voltage
 Latency 250 to 500 ms
 Typically 300 ms


Close to the parietal lobe in the brain
Averaging over multiple records required
Papavasileiou-22
Other ERP uses

CSE
5810


Lie detection
 Increased legal permissibility
 Compared to other methods
ERP abnormalities related to conditions such as:
 Parkinson’s
 Stroke
 Head injuries
 And others
Typical ERP paradigms
 Event related synchronization (ERS)
 Event related de-synchronization (ERD)
Papavasileiou-23
Other Control BCI paradigms

CSE
5810




Lateralized readiness potential
 Game control
 1~2 seconds latency
 Negative shift in EEG develops before actual
movement onset
Steady-state visually evoked potentials (SSVEPs)
Slow cortical potential (SCP)
 Imaged movements affect mu-rhythms
 They shift polarity (+ or -) of SCP
Sensorimotor cortex rhythms (SMR)
EMG
Papavasileiou-24
SCP & SMR vs P300

CSE
5810


Typically SCP and SMR BCIs require significant
training to gain sufficient control
In contrast P300 BCIs require less as they record
response to stimuli
 However, they require some sort of stimuli like
visual (monitor always in place) or audio
Also SCP BCIs have longer response times
Papavasileiou-25
Binary speller control

CSE
5810

User imagines movement of cursor
 Typically hand movement
The goal is to select a character
Papavasileiou-26
Wheel chair control

CSE
5810



All the mentioned BCI
paradigms have been
applied to wheelchair
control
Either using a monitor for
feedback
Or active paradigms as
sensorimotor imagery
(SMR)
Similar approaches have
been applied to robotics
 Artificial limbs
 etc
Papavasileiou-27
Environment control

CSE
5810


BCIs used by disabled to improve quality of life
Operation of devices like
 Lights
 TV
 Stereo sets
 Motorized beds
 Doors
 Etc
Typically use of P300, SMR and EMG related BCIs
Papavasileiou-28
EMG-based human-robot interface example

CSE
5810


Motion prediction based on hand position
EMG pattern classification as control command
Combination of both yields motion command to
prosthetic hand
Papavasileiou-29
Emotions detection

CSE
5810


Use of facial expressions to imply user emotions
ERD/ERS based BCIs
 Emotional state can change the asymmetry of the
frontal alpha
P300 - SSVEP
 Emotional state can change the amplitude of the
signal from 200ms after stimulus presentation
Papavasileiou-30
BCIs for recreation

CSE
5810


Games
 EPOC headset
 Mindset
Virtual reality
 Outputs of a BCI are
Shown virtual environment
Creative Expression
 Music
 Generated form EEG signals

Visual art
 Painting for artists who are locked in as a result of ALS
– amyotrophic lateral sclerosis
Papavasileiou-31
Security and EEG

CSE
5810




EEG has been used in user authentication
Every brain is different
Different characteristics of EEG waves are used in
user authentication
Pros
 User has nothing to remember
 Harmless
 Automatically applied
Cons
 User has to wear an EEG headset
 Accuracy is still not 100%
 Still not used in practice
Papavasileiou-32
CSE
5810
Thank you!
Papavasileiou-33