Transcript Machine Intelligence

UNESCO Workshop on Integrated
Modeling Approaches to Support
Water Resource Decision Making:
Crossing the Chasm
Motivated Machine Learning for
Water Resource Management
Janusz Starzyk
School of Electrical Engineering and
Computer Science, Ohio University, USA
www.ent.ohiou.edu/~starzyk
UNESCO Crossing the Chasm
Outline








Challenges in Water Management
Embodied Intelligence (EI)
Embodiment of Mind
EI Interaction with Environment
How to Motivate a Machine
Goal Creation Hierarchy
GCS Experiment
Promises of EI
 To economy
 To society
UNESCO Crossing the Chasm
Challenges in Water
Management
Water management is challenging for various reasons:




Strategies in water management are developed
mostly on national level
There is a growing competition between countries
for water
Water policy making effects environment and
society, health and development, and economy
Growing demands of countries’ populations for
water
 Leads to hydrological nationalism
 Creates a need to integrate water sciences and policy
making


There is an acute need for legitimate scientific data
Decision making in water-related health, food and
energy systems are critical to economical
development and security
UNESCO Crossing the Chasm
Challenges in Water
Management
Decision makers must answer important questions:







How do we make water use sustainable?
Who owns the water?
What policies, institutional and legal framework
can promote sustainable use of water?
How to protect water resources from overuse
and contamination?
Water problems became too complex,
interconnected and large to be handled by any
one institution or by one group of professionals
It is a challenge to evolve strategies and
institutional frameworks for quick policy
changes towards an acceptable water use
It is necessary to create assessment and
modeling tools to improve policy making and
facilitate interaction.
UNESCO Crossing the Chasm
Challenges in Water
Management
Why development of integrated modeling to
support decision making is important ?




Computerized models were used for many
years to support water related decisions.
Models often simplify dynamics of economic,
social and environmental interactions and lead
to inappropriate policy making and
management decisions.
This note proposes models to emerge from
interaction with real dynamically changing
environments with all of their complexities and
societal dependencies.
The main objective is to suggest an integrated
modeling framework that may assist with the
tasks of water related decision making.
UNESCO Crossing the Chasm
Challenges in Water
Management
What are deficiencies of machine created models?



Artificial neural networks, fuzzy logic, and genetic
algorithms have all been used to model the hydrological
cycle
However, it is still difficult to apply these tools in making
real-life water decisions as the related parameters are not
explicitly known
What may be needed is a motivated machine learning for
characterizing the data and making predictions about their
dynamic changes




It could support intelligent decision making in dynamically
changing environment
It could be used to observe impacts of alternative water
management policies
It may consider social, cultural, political, economic and
institutional elements that influence decision making
This strategic note presents a goal creation approach in
embodied intelligence (EI) that motivates machine to
develop into a useful research toll.
UNESCO Crossing the Chasm
Challenges in Water
Management
Embodied intelligence may support decision making:


EI mimics biological intelligent systems, extracting
general principles of intelligent behavior and applying
them to design intelligent agents
It uses emerging, self-organizing, goal creation (GC)
system that motivates embodied intelligence to learn
how to efficiently interact with the environment
 Knowledge is not entered into such systems, but rather
is a result of their successful use in a given environment.
 This knowledge is validated through active interaction
with the environment.


Motivated intelligent systems adapt to unpredictable
and dynamic situations in the environment by learning,
which gives them a high degree of autonomy
Learning in such systems is incremental, with
continuous prediction of the input associations based
on the emerging models - only new information is
registered in the memory
UNESCO Crossing the Chasm
Challenges in Water
Management
Use the motivated learning scheme to integrate
modelling and decision making:

It is suggested to apply ML approach to water
management in changing environments where the
existing methods fail or work with difficulty.
 For instance, using classical machine learning to
predict the future for physical processes works only
under the assumption that same processes will repeat.
 However, if a process changes beyond certain limits,
the prediction will not be correct.
 GC systems may overcome this difficulty and such
systems can be trained to advice humans.



Design concepts, computational mechanisms, and
architectural organization of embodied intelligence
are presented in this talk
The talk will explain internal motivation mechanism
that leads to effective goal oriented learning
In addition, a goal creation mechanism and goal
driven learning will be described.
UNESCO Crossing the Chasm
Intelligence
AI’s holy grail
From Pattie Maes MIT Media Lab

“…Perhaps the last frontier of science – its
ultimate challenge- is to understand the biological
basis of consciousness and the mental process by
which we perceive, act, learn and remember..”
from Principles of Neural Science by E. R. Kandel et al.
 E. R. Kandel won Nobel Price in 2000 for his work on physiological
basis of memory storage in neurons.
 “…The question of intelligence is the last great
terrestrial frontier of science...” from Jeff Hawkins On
Intelligence.
 Jeff Hawkins founded the Redwood Neuroscience Institute devoted
to brain research
UNESCO Crossing the Chasm
Traditional AI

Abstract intelligence
Embodied Intelligence

 attempt to simulate
“highest” human faculties:
– language, discursive
reason, mathematics,
abstract problem solving

Environment model
 Condition for problem
solving in abstract way
 “brain in a vat”
UNESCO Crossing the Chasm
Embodiment
 knowledge is implicit in the
fact that we have a body
– embodiment supports brain
development

Intelligence develops
through interaction with
environment
 Situated in environment
 Environment is its best model
Design principles of intelligent systems
from Rolf Pfeifer “Understanding of Intelligence”, 1999








Interaction with
complex environment
cheap design
ecological balance
redundancy principle
parallel, loosely
coupled processes
asynchronous
sensory-motor
coordination
value principle
UNESCO Crossing the Chasm
Agent
Drawing by Ciarán O’Leary- Dublin Institute of Technology
Embodied Intelligence
Definition
 Embodied Intelligence (EI) is a mechanism that learns
how to survive in a hostile environment
– Mechanism: biological, mechanical or virtual agent
with embodied sensors and actuators
– EI acts on environment and perceives its actions
– Environment hostility is persistent and stimulates EI to act
– Hostility: direct aggression, pain, scarce resources, etc
– EI learns so it must have associative self-organizing memory
– Knowledge is acquired by EI
UNESCO Crossing the Chasm
Embodiment of a Mind




Embodiment contains
intelligence core and
sensory motor interfaces
under its control to interact
with environment
Necessary for development
of intelligence
Not necessarily constant or
in the form of a physical
body
Boundary transforms
modifying brain’s selfdetermination
UNESCO Crossing the Chasm
Embodiment
Sensors
channel
Environment
Intelligence
core
Actuators
channel
Embodiment of a Mind




Brain learns own body’s dynamic
Self-awareness is a result of
identification with own embodiment
Embodiment can be extended by
using tools and machines
Successful operation is a function
of correct perception of
environment and own embodiment
UNESCO Crossing the Chasm
EI Interaction with Environment
Agent Architecture
Reason
Short-term Memory
Perceive
Act
RETRIEVAL
LEARNING
Long-term Memory
INPUT
OUTPUT
Task
Environment
Simulation or
Real-World System
UNESCO Crossing the Chasm
From Randolph M. Jones, P : www.soartech.com
How to Motivate a Machine ?
The fundamental question is how to
motivate a machine to do anything, in
particular to increase its “brain”
complexity?
How to motivate it to explore the
environment and learn how to
effectively work in this environment?
Can a machine that only implements
externally given goals be intelligent?
If not how these goals can be
created?
UNESCO Crossing the Chasm
How to Motivate a Machine ?

I suggest that hostility of environment motivates us.
 It is the pain that moves us.
 Our intelligence that tries to minimize this pain motivates our actions,
learning and development

We need both the environment hostility and the mechanism
that learns how to reduce inflicted by the environment pain
 In this work I propose, based on the
pain, mechanism that motivates the
machine to act, learn and develop.
So the pain is good.
Without the pain there will be no intelligence.
Without the pain there will be no motivation to
develop.
UNESCO Crossing the Chasm
Pain-center and Goal Creation
Dual pain level
Pain increase
Sensor
(-)





Simple Mechanism
Creates hierarchy of
values
Leads to formulation of
complex goals
Reinforcement :
• Pain increase
• Pain decrease
Forces exploration
+
(+)
Environment
(+)
(-)
Pain level
Wall-E’s goal is to keep
his plants from dying
UNESCO Crossing the Chasm
(-)
-
(+)
Motor
Pain decrease
Excitation
Primitive Goal Creation
faucet
refill
garbage
w. can
sit on
water
tank
Dual pain
Dry soil
UNESCO Crossing the Chasm
+
Pain
Primitive
level
open
Abstract Goal Creation
 The goal is to reduce
the primitive pain level
 Abstract goals are
created to reduce abstract
pains in order to satisfy the
primitive goals
 Abstract pain center
Sensory pathway Motor pathway
(perception, sense) (action, reaction)
faucet
“water can” –
sensory input
to abstract pain
w. can
center
Activation
Stimulation
Inhibition
Reinforcement
Echo
Need
Expectation
UNESCO Crossing the Chasm
open
-
Dry soil
+
Abstract pain
water
Dual pain
Level II
Level I
+
Pain
Primitive
Level
Abstract Goal Hierarchy
Sensory pathway
(perception, sense)
 A hierarchy of
abstract goals is
created - they satisfy
the lower level goals
Motor pathway
(action, reaction)
tank
refill
-
+
faucet
open
-
Activation
Stimulation
Inhibition
Reinforcement
Echo
Need
Expectation
Dry soil
UNESCO Crossing the Chasm
Level II
+
w. can
water
-
Level III
Level I
+
Primitive
Level
GCS vs. Reinforcement Learning
States
Policy
Desired
action
&state
Pain
States
Critic
Value
Function
action
GCS
Sensory
pathway
Action
decision
Motor
pathway
reward
Environment
Gate control
Environment
Action
RL Actor-critic design
Goal creation system
Case study: “How can Wall-E water his plants if
the water resources are limited and hard to find?”
UNESCO Crossing the Chasm
Goal Creation Experiment
SENSORY
MOTOR
INCREASES
DECREASES
1
water can
water the plant
moisture
water in can
8
faucet
open
water in can
water in tank
15
tank
refill
water in tank
reservoir water
22
pipe
open
reservoir water
lake water
29
rain
fall
lake water
-
PAIR #
Sensory-motor pairs and their effect on the environment
UNESCO Crossing the Chasm
Results from GCS scheme
Dry soil
pain
4
2
0
0
100
200
400
500
600
300
400
500
600
300
400
500
600
300
400
500
600
300
400
500
600
300
No water in can
pain
2
1
0
0
100
200
100
200
100
200
100
200
pain
2
1
0
0
1
pain
No water in tank
No water in reservoir
0.5
0
0
pain
4
No water in lake
2
0
0
UNESCO Crossing the Chasm
GCS vs. Reinforcement Learning
Averaged performance over 10 trials:
GCS:
Primitive pain
pain
1
0.5
0
0
100
200
400
500
600
400
500
600
400
500
600
Lack of food
RL:
1
pain
300
30
0.5
20
0
0
100
200
300
10
Lack of money
0.4
0
pain
0
100
200
300
0.2
Machine using GCS learns to control all abstract pains and
0
maintains
the primitive
pain
0
100
200
300 signal on400a low level
500 in
Lack of bank savingsconditions.
demanding environment
0.4
UNESCO Crossing the Chasm
600
Goal Creation Experiment
Goal Scatter Plot
40
35
30
Goal ID
25
20
15
10
5
0
0
100
200
300
400
Discrete time
500
600
Action scatters in 5 CGS simulations
UNESCO Crossing the Chasm
Goal Creation Experiment
Pain
Pain
Pain
Pain
Pain
Primitive pain – dry soil
0.5
0
0.2
0.1
0
0.2
0.1
0
0.2
0.1
0
0.1
0.05
0
0
100
200
300
400
Lack of water in can
500
600
0
100
200
300
400
Lack of water in tank
500
600
0
100
200
300
400
Lack of water in reservoir
500
600
0
100
200
300
400
Lack of water in lake
500
600
0
100
200
300
Discrete time
500
600
400
The average pain signals in 100 CGS simulations
UNESCO Crossing the Chasm
Compare RL (TDF) and GCS
Mean primitive pain
Pp value as a
function of the
number of
iterations.
Dashed lines indicate
moment when Pp is
getting stable
- green line for TDF
- blue line for GCS.
UNESCO Crossing the Chasm
Compare RL (TDF) and GCS

Comparison of
execution time on
log-log scale
 TD-Falcon green
 GCS blue

Combined
efficiency of GCS
1000 better than
TDF
Problem solved
Conclusion: embodied intelligence, with motivated learning based on
goal creation system, effectively integrates environment modeling
and decision making – thus it is poised to cross the chasm
UNESCO Crossing the Chasm
Promises of embodied intelligence

To society
 Advanced use of technology
– Robots
– Tutors
– Intelligent gadgets
 Intelligence age follows
– Industrial age
– Technological age
– Information age
 Society of minds
– Superhuman intelligence
– Progress in science
– Solution to societies’ ills

To industry
 Technological development
 New markets
 Economical growth
UNESCO Crossing the Chasm
ISAC, a Two-Armed Humanoid Robot
Vanderbilt University
Biomimetics and Bio-inspired Systems
Mission Complexity
Impact on Space Transportation, Space Science and Earth Science
2002
2010
2020
2030
Embryonics
Self Assembled Array
Space Transportation
Memristors
Biologically inspired
aero-space systems
Sensor Web
Extremophiles
Mars in situ
life detector
Skin and Bone
Self healing structure
and thermal protection
systems
Biological nanopore
low resolution
UNESCO Crossing the Chasm
Brain-like
computing
Artificial nanopore
high resolution
DNA
Computing
Biological Mimicking
Sounds like science fiction


UNESCO Crossing the Chasm
If you’re trying to look far
ahead, and what you see
seems like science fiction,
it might be wrong.
But if it doesn’t seem like
science fiction, it’s
definitely wrong.
From presentation by Foresight Institute
Questions?
UNESCO Crossing the Chasm