Transcript slides from lecture 8

ENACTIVE AI
Dr. Tom Froese
Investigations in minimal cognition
• We looked at 4 examples. The models basically question
the following assumptions:
1.
2.
3.
4.
No internal state = reactive system?
One attractor = one behavior?
No weight changes = no learning?
Behavior-switching = meta-cognition?
• These are existence proofs / proofs of concept. Their aim
is to challenge our presuppositions about what is and isn’t
possible when explaining animal behavior.
Pros and cons of minimal cognition
• The evolutionary robotics method is useful for creating
models of minimal cognition.
• Models of minimal cognition are only toy models, but they
have several advantages:
• They can be formally investigated as entire brain-body-environment
systems in fully controllable and analyzable settings.
• They probe our intuitions (so-called “intuition pumps”).
• Assumptions about mechanisms are made explicit in creating a model.
• Assumptions about behavior can be refuted by the model’s results.
• They allow the generation of new hypotheses.
• Yet they tend to ignore the physiological basis of behavior.
Internal robotics (2004)
• “Robotics can contribute significantly to our understanding of the
behaviour of organisms because of the emphasis on the role of the
body and its physical interactions with the external environment in
determining the organism’s behaviour.
• However, behaviour is the result of the interactions of an organism’s
nervous system with both the external environment and the internal
environment, i.e. with what lies within the organism’s body.
• While robotics has concentrated so far on the first type of interactions
(external robotics), to understand the behaviour of organisms more
adequately we also need to reproduce in robots the inside of the body
of organisms and to study the interactions of the robot’s control
system with what is inside the body (internal robotics).”
• Parisi (2004, p. 338)
Internal robotics (2004)
Internal robotics (2004)
• Seven differences between the two “environments”:
1. The nervous system’s interactions with the external environment
are predominantly physical, whereas those with the internal
environment are predominantly chemical.
2. Two kinds of influences on the nervous system (neurotransmissory versus neuro-modulatory)
3. The circuit ‘nervous system–rest of the body’ is entirely evolved,
whereas the circuit ‘nervous system–external environment’ is
evolved only for the part of the circuit that is in the nervous
system.
4. For the nervous system the rest of the body is something that is
always present and always more or less the same, whereas the
external environment can be present or absent and it can be
very different at different times.
Internal robotics (2004)
• Seven differences between the two “environments”:
5. The causal influences originating from within the body can result
in the emergence of a private world, whereas those originating in
the external environment define a public world.
6. The cognitive components of behaviour emerge from the
interactions of the nervous system with the external environment,
whereas its emotional or affective components emerge from the
interactions of the nervous system with the rest of the body.
7. The interactions of the organism’s nervous system with the
external environment tend to produce effects that the organism is
able to predict and are voluntary, whereas the interactions of the
nervous system with what is inside the body give rise to effects
that the organism is unable to predict and are involuntary.
Internal robotics (2004)
But: does “to rest” have any meaning for these robots?
Internal robotics (2004)
• Robots should stop moving when they are “ill”
• “Imagine that the neural network that controls the robot’s behaviour has an
additional set of input units (pain units) that are connected with the network’s
motor output units.
• If these units encode some particular activation pattern when the robot’s body
is damaged and a different activation pattern when the body is healthy, these
two different activation patterns can regulate the robot’s movements
appropriately.” (p. 335)
• Robots that are “hungry” and “thirsty”
• “The neural network that controls the robots’ behaviour has two sets of input
units that separately encode the position of the nearest food element and the
position of the nearest water element and two sets of ‘motivational’ units that,
respectively, encode the quantity of energy and of liquids that are currently
present in the robot’s body.” (p. 336)
• “In the simulation we do not simulate the evolution of the motivational state,
that is, the manner in which the body ‘learns’ to communicate to the nervous
system how much energy and how much liquid it currently contains.
• We hardwire in the two sets of motivational units the activation patterns that
reflect the current quantity of energy and of liquids that are present in the
robot’s body.” (p. 336)
Does the state of the body have any meaning for the robot?
Ultrastable robot (2000)
Di Paolo (2000)
Ultrastable robot (2000)
Di Paolo (2000)
Behavior and internal stability are disjoint
• “The robot is asked to meet two different requirements. Evolution may
come up with two possible classes of solutions to this problem:
• a) internal and behavioural stability require one another,
• b) internal and behavioural stability simply do not interfere with each other.
• In the first case, we shall observe instances of homeostatic
adaptation, in the second we shall not, as robots are capable of
regaining internal stability without altering the perturbed behaviour.
• It would be much better if we could design a scenario where a)
always holds.
• This is not something that was required in the Ashbyan framework,
but the use of random search allowed for eventual adaptation
whenever possible. In real organisms, however, the situation is
different as we have seen.”
Di Paolo (2003)
Improvements to the model (I)
• “Extended homeostatic adaptation: Improving the link between
internal and behavioural stability” (Iizuka and Di Paolo 2008)
• The use of intermittent plasticity in combination with this dual
selective pressure allows controllers to evolve where an
association is created between internal homeostasis and the
desired behaviour.
• This association is evolved to be positive: high homeostasis goes
together with good performance.
• If the situation changes, such as in an inversion of the visual
field or some other sensorimotor perturbation, this causes a
breakdown of coordination.
• Under these circumstances some evolved agents also show a
breakdown of internal homeostasis demonstrating that some agents
evolve at least one negative association: lack of phototaxis induces
lack of homeostasis.
Improvements to the model (I)
• As this happens, the local adaptive mechanism activates until it
finds a new synaptic configuration which can sustain the
activations within the homeostatic region.
• In these conditions, some evolved agents are also able to re-
form the behavioural coordination (even if they had not been
trained to adapt to the induced perturbation).
• These agents are then able to re-create a positive association:
regaining homeostasis induces a recovery of the original
behavioural performances.
• However, the original work has a problem in that these
necessary further associations between internal and behavioural
stability that allow adaptation to unseen perturbations are
contingent.
Iizuka and Di Paolo (2008)
Improvements to the model (I)
• We propose an extended homeostatic neural controller where
neurons
• are biased to have a strong resting membrane potential and
• an additional fitness condition rewarding not only a positive link
between homeostasis and a desired behaviour but also a negative one
between the breakdown of homeostasis and undesired behaviour.
• The agents evolved in the extended model are more adaptive
against unexperienced morphological disruptions and random
initial weight connections.
• Although it is not always guaranteed that such perturbations
can be adapted to, the extended model should be able to adapt
more reliably than the basic homeostatic controllers because it
decreases the chances of internal and behavioural stability
being independent of each other.
Iizuka and Di Paolo (2008)
Metabolic causation (2013)
• Iizuka, Ando and Maeda (2013)
• “In the homeostatic adaptation model, higher-level dynamics
internally self-organized from sensorimotor dynamics are
associated with desired behaviors.
• These dynamics are regenerated when drastic changes occur,
which might break the internal dynamics.
• Due to the weak link between desired behavior and internal
homeostasis in the original homeostatic adaptation model,
adaptivity is limited.
• In this paper, we improve on the homeostatic adaptation model
to create a stronger link between desired behavior and internal
homeostasis by introducing a metabolic causation in a plasticity
mechanism and show that it becomes more adaptive.” (p. 263)
Metabolic causation (2013)
• “the set of initial weights for the
neural controller is given randomly
at the beginning of the trial”
• “This modified setting makes the
task more difficult than the original
one. The agent is expected to adapt
to a suitable weight set using
plasticity and interaction with the
environment.” (p. 265)
• Iizuka et al. include “a simple model
of the agent’s metabolism, which in
this case depends on
photosynthesis.
• Accordingly, if the agent moves
away from its energy source,
homeostasis eventually breaks
down. This in turn has the effect of
making the agent adapt its
behavior.” (p. 266)
Metabolic causation (2013)
• “As before, the local homeostatic condition is fulfilled if a
node is firing within a specified range.
• In the absence of sensory activation, certain parameters that
control resting potentials, synaptic strengths and the size of
the homeostatic region should be chosen to enhance the
chances for a solution to evolve with the desired internal and
interactive associations.
• To achieve this, two parameters, α and β, are added to the
typical CTRNN equations and another parameter γ is added
to control the size of the homeostatic region.”
• (pp. 265-6)
Metabolic causation (2013)
• There is a selective
pressure to evolve
homeostatic dynamics and
phototaxis during a trial.
• The agents are evaluated
by measuring two factors:
• fs, the proportion of time that
the agent spends near the
light source, defined as
positions less than a distance
of 20 units from light source,
• and fh, the time-average of
the proportion of neurons that
have behaved
homeostatically. (p. 267)
Towards true robot (energy) autonomy
• Most mobile robots are not truly autonomous; most
operate in simplified environments.
• Almost all non-industrial robots still require a helping hand
from humans, e.g. battery charging, the odd push if they
get stuck, obstacles that are suited to their sensors, etc.
• Goal: building robots which will be self-sufficient in terms
of decision making and energy - robots which can be left
unsupervised to organize their work and nourishment.
Melhuish et al.
SlugBot (2001)
• The initial stage looked at the problems faced by a robot predator. It is
not only the energy transformation process but the necessary
behavior of the robot which we wish to study. This is an important
point - the two processes are tightly interlinked.
• In this case the robot 'hunted' slugs. The collected slugs would be
fermented to produce biogas in a separate off-board digester unit.
The gas would then be passed through methane fuel cell to generate
electricity. The electricity would be stored in batteries and could be
downloaded to a 'hungry' robot.
• We are currently working on the employment of a different type of
'digester' - the microbial fuel cell (MFC). In this type of cell microbes
are employed in a special container with a semi-permeable
membrane to extract electrons from the nutrient (such as carrot
peelings) and pass them onto an electrode. In this way a form of
'biological battery' can be made.
Melhuish et al.
SlugBot (2001)
Melhuish et al.
EcoBot III (2010)
• The main objective of this project was to develop a robot with
onboard fluid circulation, capable of collecting its energy from
the environment and getting rid of its own waste; all of these
functions are powered by the Microbial Fuel Cells (MFCs).
• This is the upper most component of the robot which consists
of the ingestion, artificial digestion and solid waste excretion
mechanism. The image shows sludge within the vessel and
solid waste sedimented into the egester.
• This is the middle section of the robot which consists of the
sludge distribution mechanism (white solid helical rings), and
the MFCs (24 in total) which are shown just below the
distribution mechanism. Underneath the MFCs there is an
overflow collection tray which feeds back into the ingestion
vessel above.
EcoBot III (2010)
• “EcoBot-III demonstrated
energy autonomy, when fed
with nutrient rich liquid
feedstocks and within the
boundaries of its
environment.
• To the best of the authors’
knowledge, this is the first
example of a robot, which
integrates real life and
machine in a symbiotic
manner (Symbot) for
digestion and autonomous
operation as an exemplar of
artificial life.
• Ieropoulos et al. (2010, p. 740)
Is it an autonomous system?
Di Paolo (2015)
Is it an autonomous system?
• No, if we consider the “body” of the robot (plastic, metal and
electronic components) as part of the system.
• Yes, if we focus on the network of relations between different
processes that robot enables.
• Movement and feeding behavior depends on energy.
• Energy depends on the digestion of decomposable matter.
• The provision of decomposable matter depends on movement and
feeding behavior.
• We have a operationally closed circle of processes.
• This operational closure takes place at the level of interactional and
internal processes.
Typography of AI systems
Froese and Ziemke (2009)
Typography of AI systems
Froese and Ziemke (2009)
References
• Di Paolo, E. A. (2000). Homeostatic adaptation to inversion of the
visual field and other sensorimotor disruptions. In J.-A. Meyer, A.
Berthoz, D. Floreano, H. L. Roitblat & S. W. Wilson (Eds.), From
Animals to Animats 6: Proceedings of the Sixth International
Conference on Simulation of Adaptive Behavior (pp. 440-449).
Cambridge, MA: MIT Press
• Di Paolo, E. A. (2003). Organismically-inspired robotics: Homeostatic
adaptation and teleology beyond the closed sensorimotor loop. In K.
Murase & T. Asakura (Eds.), Dynamical Systems Approach to
Embodiment and Sociality (pp. 19-42). Adelaide, Australia: Advanced
Knowledge International
• Di Paolo, E. A. (2015). El enactivismo y la naturalización de la mente.
In D. Pérez Chico & M. G. Bedia (Eds.), Nueva Ciencia Cognitiva:
Hacia una Teoría Integral de la Mente (in press). Zaragoza: PUZ
• Froese, T., & Ziemke, T. (2009). Enactive artificial intelligence:
Investigating the systemic organization of life and mind. Artificial
Intelligence, 173(3-4), 366-500
References
• Ieropoulos, I., Greenman, J., Melhuish, C., & Horseld, I. (2010).
EcoBot-III: A Robot with Guts. In H. Fellermann, M. Dörr, M. M.
Hanczyc, L. L. Laursen, S. Maurer, D. Merkle, P.-A. Monnard, K. Støy
& S. Rasmussen (Eds.), Artificial Life XII: Proceedings of the Twelfth
International Conference on the Synthesis and Simulation of Living
Systems (pp. 733-741). Cambridge, MA: The MIT Press
• Iizuka, H., Ando, H., & Maeda, T. (2013). Extended homeostatic
adaptation model with metabolic causation in plasticity mechanism toward constructing a dynamic neural network model for mental
imagery. Adaptive Behavior, 21(4), 263-273.
• Iizuka, H., & Di Paolo, E. A. (2008). Extended Homeostatic
Adaptation: Improving the Link between Internal and Behavioural
Stability. In M. Asada, J. C. T. Hallam, J.-A. Meyer & J. Tani (Eds.),
From Animals to Animats 10: 10th International Conference on
Simulation of Adaptive Behavior, SAB 2008 (pp. 1-11). Berlin,
Germany: Springer-Verlag
• Parisi, D. (2004). Internal robotics. Connection Science, 16(4), 325338
Homework
Please finish reading:
Froese, T., & Ziemke, T. (2009). Enactive artificial
intelligence: Investigating the systemic organization of life
and mind. Artificial Intelligence, 173(3-4), 366-500
Everyone must prepare a question for next class!
We will discuss the first assignment of the course.