Transcript VideoPresentation

University of Milan
ORESTEIA -MODULAR HYBRID ARTEFACTS WITH ADAPTIVE FUNCTIONALITY
http://www.image.ntua.gr/oresteia/
PARTNERS
Project Technical Committee
Technical Manager: John Taylor (KCL)
ALTEC SA
Sotiris Pavlopoulos, George Malinescos
IVML-NTUA
Stefanos Kollias, Nicolas Tsapatsoulis
University of Milan
Bruno Apolloni, Dario Malchiodi
Kings College of London
Stathis Kasderidis
Imperial College
Eric Yeatman, Tim Green
Contents
 Executive Summary
 Problems to be Solved
– Attention
– Data Fusion
– Emergence
 Architecture and partners' tasks
– Diagram
– Module Explanation
 Feature Extraction through Signal Modelling
 Methods for State mappings
– CAM/SPM
– PAC Meditation / Fuzzy Relaxation
 Demo 1
 Demo 2
 Data Collection
– Laboratory Studies
– Car Driving Simulator
– Linguistic Rules
 MicroPower Generation – Wireless Communication
Executive Summary
Scope/Aims
 Create a guidance system for humans, for
more efficient and less hazardous living and
interacting with their environment, through a
set of decision-making facilities embedded
in the environment and suitably adapted to
the particular user
 Investigate enabling technologies for DC in
the form of energy harvesting and low power
wireless communications
Inputs
 Low level sensorial data
– Physiological class of sensors
– Environmental class of sensors
– Other
 Symbolic knowledge, a priori available (linguistic
rules)
 Subsymbolic knowledge, constructed based on
numerical data (Input/Output pairs)
 Attention-based functionality, inspired from brain
operation
Outputs
 Decisions
 Actions on behalf of the user for:
– Managing repetitive and trivial jobs,
– Providing indication of abnormal user activity
and state,
– Providing planning facilities,
– Providing information filtering facilities,
 Maintaining good user state (physiological,
psychological, etc)
Key Properties
 Autonomy
 Responsiveness
 Robustness
Approach
 Develop a multi-level attention-based agent architecture
adapted to solve decision /guidance problems arising from
sensors of various types, some worn by humans, others in
devices (such as cars) being used by the humans. The
decision/ guidance response of an agent is as to what is
the state of the human, given the sensor data, or what is
optimal continued use of the device on the basis of joint
sensor data arriving at the agent for a given user from all
sources
 Develop multi-agent systems that handle data available,
also, from a set of agents (from interacting users),
providing for decision/guidance as to overall optimal (best)
use, and ranking of the users as to which needs further
analysis
Problems to be solved
 Attention
 Data Fusion
 Emergence
Problems to be solved:
Attention
I shouldn’t
produce
these
outputs, with
these
inputs…
• Self-evaluation error
• Irregular inputs
• Hardware failure
Artefact
The input
signals
have
irregular
patterns…
Self
Evaluator
Attention
Controller
INPUTS
Attention
Signal
Data
History
My
batter
y is
low!
OUTPUTS
Problems to be solved:
Data Fusion
You can make use
of the two bottom
Artefacts
Which of
these data
shall I need?
“WORLD MODEL”
Artefact
data
Artefact
data
data
data
data
data
data
data
data
data
data
Artefact
“Clever Space”
data data
Intelligent
Artefact
data
data
data
Artefact
data
data
data
data
AGENT
data
Artefact
Problems to be solved:
Emergence
“Clever Space”
How much are you
willing to pay for the
services?
WHY NOT
Artefact
BE MINE?
Artefact
I need to
use these!
Artefact
Artefact
AGENT
Artefact
AGENT
Artefact
Artefact
THESE
ARE
MINE!
Artefact
Artefact
And I need
to use
these!
AGENT
HEY,I SAW
THEM FIRST!
Artefact
I’ll take
these!
Artefact
Architecture of a single Agent
IMC: Inverse Attention/modulation Controller
Response/
Decision
Monitor
IMC
Rules
Action
Reinforcement
Fused
Goals
Level 4
Fuzzy/
Statistical
Response/
Decision
UNIMODAL
Monitor
IMC
Rules
Action
Level 3
Goals
Historic Mean
State
ANN
Hourglass
NeuroFuzzy
Reinforcement
Histogram Analysis
Level 2
Data Smoothing
Sequential
Data
Level 1
Sequential
Data
Sensors
Sequential
Data
Overall
View
ANN Fused
State
Other User/
Environment
Symbolic Inputs
USER
Database
Historic Mean
State
Reinforcement
Level 1: Sensors
 Data Content: Classes of signals used by
higher levels (Level 2-4)
– Data collection (KCL-QUB, ALTEC)
– Synthetic data generators (NTUA, UM, KCL)
 Sensor autonomy
– Efficient energy harvesting (ICSTM)
 Communication links
– Low power consumption (ICSTM)
Level 2: Preprocessing
 Signal preprocessing (NTUA, KCL)
– Noise Reduction
– Buffering
– Transforms
 Feature extraction
– Which features? (ALTEC, KCL-QUB)
– How? (NTUA, KCL)
 Modeling signals
– Extraction of hidden parameters (UM)
Architecture of a single Agent
Level 3: Domain Experts – State
Representation
State Mappings
 ANN Hourglass
– Subsymbolic state representation (UM, NTUA)
 Neurofuzzy
– Symbolic state representation (NTUA, UM)
Action Module
 Stores the ‘response’ of the system. Three
levels of sophistication:
– No real action.
– Simple suggestive actions/messages.
– Simple action sequences.
 Responsible Partners: KCL, NTUA, UM
Rules Module
 Consists of three components:
– World Model. Contains all the information
needed for forming useful functionalities and
maintaining a set of artefacts.
– Autonomic. Maintains rules that are necessary
for the robust run-time behaviour of the system.
– Other. Aids the implementation of alternative (to
the ones implemented in the State module)
decision-making systems.
 Responsible Partners: KCL, NTUA, UM
Goals Module
 This module includes three parts:
– Values. Is closely associated with the World
Model (in the Rules module) to provide default
(universal) values for the various thresholds and
triggers present in the architecture.
– User Profile. Provides specific user deltas (i.e.
deviations from the default values defined in the
Values part).
– Services. Includes a catalogue of services that
are offered by the artefact.
 Responsible Partners: KCL, NTUA, UM
Monitor Module
 Creates an error signal level after comparing
the current State with a Historic State. It
fulfils two basic requirements:
– Universal definition of an error function.
Independent of the output of State Module (UM,
NTUA, KCL)
– Standard definition of an error function. Context
sensitive, seamless knowledge of state
representation (UM, NTUA, KCL)
Attention Controller
 This module is inspired from motor control
systems in the brain as well as from
engineering control ideas. It operates in two
modes:
– Feedforward mode. The controller sends a
signal, governed directly by the Goal module, to
produce a desired response from the action
module (KCL)
– Feedback mode. Feedback information from the
Monitor module is used as a feedback
component (KCL)
Level 4: Agent Construction






Combination of conceptual blocks
Agent Formation
Data Fusion
Overall system training
Reinforcement signal production/handling
Attention Control
Responsible partner: KCL
Feature Extraction through Signal
Modelling
HYBRID TRAINING
DYNAMICS
E(w) = e(y1,..., yT)
yVI  5 yV  11y IV  25 y III  34 y II  20 y I  24 y  0
Jy 1 (w )
T
=   y t e(y 1 ,..., y T)Jy t (w )
...
t =1
Jy T(w )
Jz t (w ) =JF(z t-1 ,v t-1 ;w ,t)
Integrated by a fourth order Runge-Kutta method
Jz t (w ) =JF(z t-1 ,v t-1 ;w ,t)
Jz t-1 (w )
= Jz t-1F(z t-1 ,v t-1 ;w ,t-1)Jz t-1 (w ) + Jw F(z t-1 ,v t-1 ;w ,t-1)
0
I
Jz t-1 (w )
= Jz t-1F(z t-1 ,v t-1 ;w ,t-1)Jz t-1 (w ) + Jw F(z t-1 ,v t-1 ;w ,t-1)
0
I
SIGNALS FROM BODY
8
6
4
CH1
2
CH4
CH5
CH6
95,35
90,05
84,75
79,45
74,15
68,85
63,55
58,25
52,95
47,65
42,35
37,05
31,75
26,45
21,15
15,85
10,55
sec
-2
5,25
0
-4
-6
0.4
0.4
0.2
0.2
20
40
60
80
100
120
0.4
ECG
20
140
40
60
0.2
80
100
120
20
140
-0.2
-0.2
-0.2
-0.4
-0.4
-0.4
-0.6
-0.6
-0.6
Symbolic
health diagnosys
40
60
80
100
120
140
Methods for State Mappings:
CAM/SPM Module
Overall View
output
SYMBOLIC
PROCESSING
MODULE
evaluated symbolic
predicates
CONNECTIONIST
ASSOCIATION
MODULE
features
CAM Module
Scope
 The Connectionist Association Module
(CAM) provides the system with the ability of
grounding the symbolic predicates
 Using the CAM, the set of features is
associated with the set of evaluated
symbolic predicates (partitioning the input
space)
Why Neural Network?
 Generally the internal state defined by the
neural network output is not so simple to be
considered as a simple fuzzy partitioning;
 Instead the neural network performs the
appropriate data clustering to provide the
evaluation of the required symbolic
predicates based on numerical data
Diastolic Pressure Example
Attention Signal Handling
 To which input elements have to be
concentrated on?
SPM Module
Scope
 It implements a semantically rich reasoning
process. It takes as inputs a set of features
and gives a set of recognised situations.
 It performs the conceptual reasoning process
that finally results to the degree of which the
output situations are recognised
Why Neurofuzzy?
 Fuzzy relational systems represent symbolic
knowledge in a formal, numerical
framework.
 On the other hand, neural networks are
typical learning systems that work in a
numeric framework.
Rule Insertion
 Rules describing situations are based on
linguistic terms and are generally of the form
If fuzzy_predicate(1) and fuzzy_predicate(2)
then output(3)”
 Each rule consists of an antecedent (the if
part of the rule) and a consequence (the
then part of the rule)
Rule Insertion
Rule1
 The antecedent part of the rule
is used to create the weight
matrix of the first layer
 The consequence part of the
rule is used to create the rule
matrix of the second layer
 The antecedent of all the rules
existed is the set of the fuzzy
predicates describing the
system
 The consequence of all the
rules is the set of the recognised
situations of the system
Predicates Activ ation
Rule2
1
0.8
In1
0
Predicates Activ ation
0
0
Rule3
0
0
Predicates Activ ation
Rule4
1
Layer1Out
Predicates Activ ation
Rule5
Predicates Activ ation
Rule6
Predicates Activ ation
Rule Insertion (Example)
Layer 1
Layer 2
Methods for State Mappings:
PAC Meditation / Fuzzy
Relaxation
PAC Meditation
mapping
formula
fitness
ok?
formula
n
fitness
y
fuzzy relaxation
formula
ok?
y
prejudice
n
final formula
end
PAC Meditation
0-level inner border
0-level outer border
1-level inner border
1-level outer border
Fuzzy relaxation
(dk)
m dk   m dk  
-
d
1
2
3=s
   d
d
k
 m dk   m dk   1
d

d
s
 i    m (d k )
k 1
m
m
m
i1
i1
i1
O f ,    1  Li   2    i   3    i   4 0

SA Algorithm
CurrentState := InitialState
CurrentTemperature := InitialTemperature
Repeat
GetTemperature(CoolingSchedule)
ProposedState := SelectNeighborState
ProposedCost := EvaluateCost(ProposedState)
If (Accepted(ProposedState, ProposedCost))
Then CurrentState := ProposedState
Until StoppingRule
Return(CurrentState)
d
k

   d 
   d
k

   d 
DEMO 1: Health Monitoring
DEMO 2: Car Hazard Avoidance
Description
 This demo includes the ability to generate a
set of events in the environment, driver, or
car. Environmental effects include shocks in
the visibility (fog) and the temperature. In
this category also the appearance of ice
patches in road segments and existence of
other cars in the same or opposite lane is
included as well
Aims
 To validate the ORESTEIA Architecture
 To test the integration of the work of the
partners
 To offer another context for testing the
adequacy of the State mapping methods
 To search for the existence of common
design principles in the various contexts
Data Collection
 Laboratory Studies
 Car Driving Simulator
 Linguistic Rules
Data Collection:
Laboratory Studies
Aim
 Experiment design for collecting ECG,
Respiration Rate (RSP), Galvanic Skin
Response (GSR), and Skin Temperature
(SKT) in laboratory conditions for abnormal
physiological and psychological state
prediction
ECG Signal
RSP Signal
GSR Signal
SKT Signal
Data Collection:
Car Driving Simulator
Aim
 Experiment design for collecting ECG,
Respiration Rate (RSP), Galvanic Skin
Response (GSR), and Skin Temperature
(SKT) while driving a car simulator (for
abnormal physiological and psychological
state prediction)
Car Driving Simulator
Data Collection:
Linguistic Rules
Physiological Features
 The system takes as inputs a set of
medical features and gives a set of
recognised situations.
 The input features are the values of
RSP, BT, HR, PS, PD, and the derived
features from the ECG
Inputs






RSP: Respiration
BT: Body Temperature
HR: Heart Rate
PS: Systolic Blood Pressure
PD: Diastolic Blood Pressure
ECG: Electrocardiogram
Outputs




Normal: some features are not perfect
Warning: stop and rest until you get normal
Urgent: stop and call medical centre
Emergency: very urgent
Definition of Symbolic
Predicates
 The term predicate refers to the partition of
the input features
 Predicates are characterised as Very Low
(VL), Low (L), Medium Low (ML), Medium
High (MH) High (H), and Very High (VH)
 Some features are characterised as Normal
(N) and Abnormal (A)
Rule Extraction
 Rules describing situations are based on
linguistic terms and are generally of the form
“if predicate(1) and predicate(2) then
output(3)”
 In order to detect the recognised situations,
we must first define the rules that describe
these situations
A Subset of the Rules
Nr
Rule
Output
1
HR(VH) + ECG-5(A) + BT(VH)
Urgent
2
PS(VH) + HR(VH) + ECG-5(A) + BT(VH)
Emergency
3
ECG-6(A) + PS(VL)
Emergency
4
PS(L) + HR(H) + BT(H)
Warning
5
HR(M) + BT(M) + PS(M) + RSP(M) + ECG-5(N)
Normal
6
HR(L) + BT(L) + PS(M) + RSP(M) + ECG-5(N)
Normal
7
PS(M) + HR(L) + RSP(M) + ECG-1(H)
Warning
MicroPower Generation and
Wireless Communication
Aims
 Development of artefacts that
– Are autonomous
– Have a long lifetime without maintenance
 Development of sources that can scavenge
energy from the local environment
 Electromechanical energy conversion using
MEMS
Analysis of micro-power generator
topologies
 Detailed investigation of the
operation of various different
inertial generator topologies
 The parametric generator is
optimal when the input
movement is greater than the
dimensions of the device by a
factor of ~10 or more
Fabrication and test of an initial
prototype generator
 Cross-section of
prototype generator
 Photograph of
fabricated device
Fabrication and test of an initial
prototype generator
 Experimental setup for
charge transfer
experiments
amplifier output voltage (V)
 Typical discharge
transient
0.5
0
-0.5
-1
-1.5
-2
15
17
19
21
time (microseconds)
23
25
Analysis of wireless communication
schemes
100
Power Transfer Comparison /dB
 Comparison of nearfield transmission (at
50 MHz) with far-field
transmission (at 470
MHz) for a distance of
50cm, varying the near
and far-field antenna
dimensions (shortened
to NF and FF
respectively on the
axis labels)
50
0
-50
0.08
-100
0.06
-150
0.08
0.04
0.07
0.06
0.02
0.05
0.04
0.03
FF Antenna Radius/m
0.02
0.01
0
0
NF Radius/m