Transcript BCI Engine : software - Luigi Bianchi Home Page

ICBEM 2000
An Object Oriented
Approach To Bio-Feedback
Applications For Disabled
People
Luigi Bianchi1,5, Fabio Babiloni2, Febo Cincotti3, Serenella Salinari4, Maria Grazia Marciani
Dip. Neuroscienze, Università “Tor Vergata”, Rome, ITALY
Fisiologia Umana e Farmacologia, Università “La Sapienza”, Rome, ITALY
3 IRCCS, Fondazione “S. Lucia”, Rome, ITALY
4 Dip. Informatica e Sistemistica, Università “La Sapienza”, Rome, ITALY
5 Brainware, Rome, ITALY
1
2 Dip.
E-mail: [email protected]
1,3
ICBEM 2000
Introduction (1)
One of the main problems encountered in the development of
computer-based systems for handicapped people is that it is very
difficult to optimize them in a wide range of situations. This
happens mainly because every potential user has residual
capabilities that are specific to his condition and that make him in
some way unique. It should be extremely useful to use all of the
voluntarily controlled activities (VCA) such as eye movements,
muscle contractions, EEG activity, etc. as inputs for a processing
unit that could take care of recognizing them and then translating
them into desired tasks like opening doors, playing-back some prerecorded phrases, using the telephone and so on. Unfortunately,
building a system for even a well-defined pathology might be
difficult: optimal use of residual capabilities in different subjects
could require modifying it at a level that is incompatible with
practical needs.
ICBEM 2000
Introduction (2)
However, while the number and type of the voluntarily controlled
biological signals is specific to every patient, the number of tasks that
one wants to execute is quite homogeneous. Furthermore, even if the
nature of the utilized signals may vary among different pathological
situations, the way in which a biofeedback system works is quite
stereotyped: after the data have been acquired, there is a DSP preprocessing stage that performs some basic operations on the input
signals, then a classification stage in which some features are
extracted (FE), manifested in some way to the user and eventually
attributed to one of the subject VCA (discriminator), and finally, a
stage in which a task can be executed.
Ref 1
Ref 2
...
VCA 1
Ref m
VCA 2
...
ACQ
VCA m
patient
state
DSP
F.E.
Discr.
classifier
HW
biofeedback kernel
Task
driver
task
manager
Task 1
Task 2
...
Task n
environment
ICBEM 2000
Introduction (3)
Finally it is very frequent that these systems provide different
operating modalities such as training, testing, setup, and running.
This is a situation in which an object oriented programming (OOP)
approach reaches his best results: inheritance, virtual functions and
templates are all instruments created to reuse code and to allow
generic programming. In our case it is possible to describe the
operative flow that is common to all biofeedback applications leaving
to be defined only those aspects that are specific to the single
implementation, such as the algorithms and the classification rules.
Then, in a separate step, several systems can be realized just
defining the points that are intentionally left unspecified.
Here we describe an object oriented software framework that can
be used as a skeleton for the practical implementation of a wide
range of Bio-feedback systems.
It is important to notice that the proposed solution does not make
any assumption on both the operating system and the hardware used.
ICBEM 2000
Bio-Feedback System
A Bio-Feedback system is usually composed
of various hardware and software
modules that interact among themself
and that implement different tasks. The
main sub-systems are:
1) A Data Acquisition Board that collects
biological signals;
2) A hardware that receives collected data
(e.g. a Personal Computer) and that
interacts with the environment, driving
special devices;
3) Some algorithms that extract the
desired features from the biological
signals;
4) Different operating modalities, such as
training, exercise, running.
5) Several supporting utilities for
configuring the system in all the aspects.
Data
Acquisition
Hardware
Algorithm
Modalities
Utilities
ICBEM 2000
Modalities (1)
Another aspect that is always encountered in different systems is the
fact that they must operate in more modalities such as training, testing,
running and setup. The training modality is relative to a period in which
some parameters relative to the user are extracted in order to tune
the system to him. During this phase, one asks to the subject to
perform an action several times in order to collect enough reference
data that will be used later on by the classifier. This procedure is
repeated for all the VCAs that the patient intends to use.
a) training
The running modality
is devoted to the
identification of a VCA
and to the triggering
of an event that might
start an action in the
case of a successful
classification.
Ref 1
Ref 2
VCA 1
...
Ref m
VCA 2
...
ACQ
DSP
F.E.
VCA m
b) running
Ref 1
Ref 2
...
VCA 1
Ref m
VCA 2
...
VCA m
state
ACQ
DSP
F.E.
Discr.
Task
driver
Task 1
Task 2
...
Task n
ICBEM 2000
Modalities (2)
The testing modality is somehow in the middle between the
previous two: the system asks to the subject to perform one of
the VCAs, and then try to classify it. After that, an internal
score is updated and an event is triggered in order to signal a
success or a failure. In this way it is possible to evaluate the
performance of the whole system for every single performed
task.
The setup modality is relative to the hardware configuration
and to the loading and saving of the references and other
parameters computed during previous training sessions.
ICBEM 2000
Algorithms
Even if Bio-Feedback systems are mainly characterized by the
classification rules and the feature extraction algorithms,
most of them share the same DSP engines at least at a preprocessing stage. This includes FFT computation and digital
filtering. Furthermore, advanced matrix computation routines
are commonly required for the complete implementation of a
particular tool such as the ones based on Signal Space
Projection or on the Fisher Discriminant Linear Classifiers.
Other algorithms which are based on Neural Networks, still
need a pre-processing stage for DSP. Anyway, the workflow of
all these systems are still the one illustrated in the introduction
section, or at least it is easily re-conducible to it.
ICBEM 2000
Hardware-Data Acquisition
Actually there are several implementations for the realization of
the various systems. They are mainly characterized by:
• Platform used (PC, Macintosh, Palm, proprietary, etc…)
• Operating System (Win32, WinCE, MacOS, Linux, Unix, BeOS,
proprietary, etc…)
• A/D and D/A converter (proprietary, National Instruments,
Analog Devices, etc…)
• Biological signal treated (EMG, EEG, Voice, Breath, etc…)
• Amplifiers
• I/O protocols and peripheral (RS232, microphones and speakers,
TTL triggers, digital cameras, USB, IEEE488, FireWire, etc…)
ICBEM 2000
A modern approach
A different way to realize a versatile and easily configurable
Bio-Feedback system is to make few assumptions about it. In
particular it must:
• be an open architecture;
• operate with a high hardware abstraction level;
• be easily configurable;
• be easily customizable;
• maximize the software source code reuse;
• maximize the software binary code reuse;
• interact with the tools and devices already available;
• facilitate the diffusion of new algorithms.
ICBEM 2000
A modern Answer (1)
Object Oriented Programming can give a lot of answers to
the previously listed key-points. Out implementation uses C++
for the following reasons:
C compatibility, in which almost all the operating system are
written;
Speed, because c++ generated binary code is generally faster
than the one generated by other languages ones such as Java;
Diffusion, because it is the most widely used;
Hardware control, because the vast majority of peripherals
and IO board are easily driven in C;
Language features, such as virtual functions, inheritance,
template, that allows generic programming;
Cross-platform portability, because it is possible to recompile an ANSI C++ program under almost every platform.
ICBEM 2000
A modern Answer (2)
Another key issue is that it is fundamental to keep separate
the hardware IO aspects from the implementation of the
different algorithms. This goal can be achieved by making a
simple assumption: the acquired data must be stored on a
buffer whose size is large enough to hold the data required for
the real-time processing. It is not important how the data are
collected: it is sufficient to notify in some way when and where
new biological acquired data are available. Other relevant
information are related to every single acquired channel, such
as gain, sampling rate, and coordinate position. This last can be
used by special algorithms such as those based on Laplacian
estimation for example in some Brain Computer Interface
(BCI) systems.
ICBEM 2000
A modern Answer (3)
To solve all these problems, a hierarchy of classes have been
implemented, in order to keep into account different
requirements. The first class is the BFEnv, which implements
the operative flow of an Bio-Feedback application as described
in the introduction. All the timing aspects are treated here, and
special virtual functions such as DecisionRule() and
ComputeAlgorithm() must be defined to characterize a specific
system. Other virtual functions such as OnSuccess() and
OnFailure() are automatically triggered every time a recognition
based on the specific DecisionRule and ComputeAlgorhitm
implementations occurs.
Another object derived from BFEnv, called BFSpeEnv, adds
some FFT capabilities, while BFAdvEnv gives also matrix
computation routines. Note that all the source described here is
cross-platform and to be considered freeware for any non
commercial use and can be obtained at www.luigibianchi.com.
ICBEM 2000
SSP implementation (1)
MS Windows
SSP
Linux
Bio-Feedback
Software Development Kit
(with matrix computation and basic DSP routines)
Mac OS
BeOS
At this point we have decided to implement a Signal Space
Projection (SSP) algorithm. In order to do that, we have decided
to adopt MS Windows as operating system, and the EEG signal as
biological input.
ICBEM 2000
SSP implementation (2)
Anyway, the implementation was performed into two separate
levels: the first one for all the necessary processing, including
the DecisionRule and ComputeAlgorithm implementation (and
this still save the cross-platform compatibility); the second one
to add all aspects that are dependent on the operating system
used such as the Graphic User Interface (GUI) and other
external tool support such as Text to Speech, Voice Command
capabilities (SAPI 4.0) and Dual Monitor support. This last
feature was necessary to give to different views to the user
(cursor movement) and to the technician (full EEG signal display
with real-time 3D spectral maps)
In an immediate future step, accessibility options integrated
in all the Win32 platform will be also used.
All the implementations were done using Borland C++Builder 4
compiler.
ICBEM 2000
SSP implementation (3)
It is important to note that all the implementation without the
GUI and some other OS specific aspects, required less that 200
lines of C++ code, so divided:
20 lines for pseudo-inverse matrix computation;
30 lines to implement the DecisionRule and the
ComputeAlgorithm functions;
60 lines for constructors, destructors, allocation and deallocation;
80 lines for other things, including dual monitor support, in
order to give to different views to the user (cursor movement)
and to the technician (full EEG signal display with real-time 3D
spectral maps)
ICBEM 2000
SPP Screenshot
ICBEM 2000
Conclusions
A model for a generic biofeedback system was
described. Its main advantages are the net
distinction of the functional blocks that are relative
to the hardware, the input signals, the algorithms
and the type and degree of disabilities. In this way
it is possible to reuse a huge amount of code, thus
minimizing the time required in the development and
adaptation of a system to a wide number of
patients.