Transcript Slide 1

Adaptive behavior in
autonomous individuals
Michael Vigorito
Department of Psychology
History of Mind
Phrenology – Franz Gall (1758 –
1828) suggested that bumps of the
skull represented mental abilities. His
theory, though incorrect, nevertheless
proposed that different mental abilities
were modular.
Studying the mind (intelligence) from
the perspective of information
processing (Cognitive Psychology) .
Studying intelligence from the
perspective of behavioral adaptation to
the environment
Neural Communication
Neuroscientists and psychologists understand that humans and
animals operate similarly when responding to environmental stimuli
and when processing information with some exceptions (e.g.,
language processing)
Today we know that much of animal/human behavior is a result of numerous
complex circuits that function in parallel.
• Behavior as an adaptation to the environment
• Adaptive Behavior does not always require information processing (cognition)
A simple nervous
system circuit
1. Sensory neuron
2. Motor neuron
3. Interneuron
•Behavior change (Learning) as an adaptation to environmental
change.
Rat in an operant Chamber
• Anthropomorphism is a common error when explaining animal behavior
• We sometimes overestimate the importance of our cognition (thinking) in the
control of our own behavior.
Unconditioned (innate) Behaviors
Simple =>
(e.g., reflexes)
Eliciting
Elicited
Stimulus
Response
Heat
Loud noise
Air puff to eye
Limb withdrawal
Startle
Eye blink
Food in mouth
Complex =>
[e.g., sign
stimuli &
FAP ]
Salivation
Long beak with red spot
Pecking response
Egg
Egg retrieval
Visual stimulus
Eye fixation
Smell of food
Approach and search (appetitive behavior)
Taste of Food
Chewing/swallowing (consummatory behavior)
Very Complex =>
Ethology
e.g. Courtship and reproductive behavior in the stickleback
Behavior e.g. General Search
Systems
Focal Search
Food Handling/Ingestion
Valentino Braitenberg’s Vehicles
• Valentino Braitenberg's booklet 'Vehicles' has received much
attention in the robotics community.
• Shows that complex behavior does not require a complex brain, but
emerges from the interactions between the vehicle and its
environment. The more complex its environment, the more complex
its behavior.
Law of Uphill Analysis and Downhill Invention: machines are easy
to understand if you’re creating them; much harder to understand
"from the outside".
Psychological consequence: if we don’t know the internal structure
of a machine, we tend to overestimate its complexity.
Vehicle 1: Alive
Components:
- One heat sensor
- One motor (rotates in forward direction only)
- One connection between sensor & motor
Principle: The more there is of the quality (e.g.,
heat) to which the sensor is tuned, the faster the
motor goes.
Description: alive, restless, doesn’t "like" heat
Vehicle 2a: Cowardly
Components: 2 light sensors, 2 motors, each
sensor connected to the motor on the same
side ("uncrossed")
Principle: The more there is of the quality to
which the sensor is tuned, the faster the
motors go ("excitatory").
Vehicle 2a: Cowardly
+
+
Description: dislikes source to which the sensor
is tuned; occasionally "attacks" it
Vehicle 2b: Aggressive
Components: 2 sensors, 2 motors,
each sensor connected to the motor on
the opposite side ("crossed")
Principle: The more there is of the
quality to which the sensor is tuned, the
faster the motors go ("excitatory").
Description: dislikes source to which
the sensor is tuned; "attacks" it
Vehicle 2b: Aggressive
Vehicle 2a:
Coward
Vehicles 3a & 3b Loving
Principle: The more there is of the quality to which the sensor
is tuned, the slower the motors go ("inhibitory").
Vehicle 3a :
Loving/Quietly
accepting
Components:
uncrossed
connections
Description: loves
the source, wants
to be near it, comes
to rest facing it
Vehicle 3b:
Loving/Exploring
Components: crossed
connections
Description: likes the
source, but easily
attracted away
Vehicles 3a & 3b Loving
Principle: The more there is of the quality to which the sensor
is tuned, the slower the motors go ("inhibitory").
Vehicle 3a :
Loving/Quietly
accepting
Components:
uncrossed
connections
Description: loves
the source, wants
to be near it, comes
to rest facing it
Vehicle 3b:
Loving/Exploring
Components: crossed
connections
Description: likes the
source, but easily
attracted away
Vehicle 3c: Knowing, Valuing
Components: 4 sensors, 4 motors, each tuned to
different properties of the environment.
V
U
U V
Excitatory
Inhibitory
Principle: one each of the four types so far:
• uncrossed/excitatory: tuned to temperature
• crossed/excitatory: tuned to light
• uncrossed/inhibitory: tuned to organic material
• crossed/inhibitory: tuned to oxygen level
Description: Cowardly toward areas of high
temperature;
Aggressive toward light sources;
Loves organic material; leaves and seeks new
source if environment is depleted;
Restlessly seeks best source of oxygen.
Vehicle 3c appears to know a great deal and
appears to have a system of Values
Vehicle 4a: Displaying Instincts, Specialization
Monotonic relationship
Inhibitory
Motor Speed
Motor Speed
Excitatory
Intensity of Stimulation
Intensity of Stimulation
Excitatory
Intensity of Stimulation
Motor Speed
Motor Speed
Non-Monotonic relationship
Inhibitory
Intensity of Stimulation
Vehicle 4a: Displaying Instincts, Specialization
Components: sensors and motors
Principle: connections both excitatory and inhibitory
but non-monotonic
Description: does everything 3c vehicles do, but with
much less predictability.
Vehicle 4b: Making Decisions
Components: sensors and motors, and threshold devices
Principle: connections both excitatory and inhibitory but nonmonotonic
Motor Speed
Description: does everything 3c vehicles do, but with much less
predictability, and appears to ponder over its "decisions";
appears to will.
Intensity of Stimulation
Vehicle 5: Reasoning Logically, Counting, Recognizing Individuals
Components: sensors, motors, threshold devices, some networked, so
they give output according to some numerical formula (e.g., one out for
every three in).
Principle: connections are both excitatory and inhibitory, monotonic and
non-monotonic. Like a computer.
Description: apparently recognizes individuals, counts, does logic,
math calculations.
Learning
• Change in Behavior as a result of experience
• Existing reflexes, FAP, and complex behavioral
systems are modified as a result of experience
• Learning = behavioral adaptation to changing
environments
– mobility results in a changing environment
– the presence of autonomous others results in a
changing environment
Environmental Stimuli
Emitted
Response
Eliciting
Elicited
Stimulus
Response
Internal Stimuli
• motor feedback
• motivation (extrinsic; intrinsic)
• emotion
• cognition (e.g. expectation; intention)
*Environmental Stimuli
Emitted
Response
*Eliciting
Elicited
Stimulus
Response
Types of learning (& Laboratory Procedures for investigating them):
1. Habituation/Sensitization
2. Classical (Pavlovian) conditioning
3. Operant (Instrumental) conditioning
4. Cognitive Learning
Habituation
*Environmental Stimuli
Emitted
Response
*Eliciting
Elicited
Stimulus
Response
Not Required
In an habituation experiment the eliciting stimulus is presented more
than once or repeatedly.
Habituation
Gill Withdrawal Reflex
Habituation in a rat
Sensitization
Role of Habituation & Novelty in
Exploratory Behavior
• Given a choice between an object that was previously encountered
and a second novel item, rats and monkeys prefer the novel item.
• This increases exploration of objects in the environment
Part 1: exposure
to single object
Part 2: exposure
to both objects
Rat spends more time
exploring novel object
Classical Conditioning – associating two stimuli
*Environmental Stimuli
Emitted
Response
Not Required
*Eliciting
Elicited
Stimulus
Response
Classical Conditioning – associating two stimuli
*Environmental Stimuli
*Neutral
Stimulus
(Light)
Emitted
Response
+
*Eliciting
Elicited
Stimulus
Response
Not Required
Restrained dog
Food
Mobile pigeon
Food
Salivation
Approach & consumption
Sign Tracking revealed through a
classical conditioning procedure
Sign Tracking as Adaptive Behavior
Sign tracking Video
The form of the pecking behavior Video
Sign Tracking as Maladaptive Behavior
Sign tracking long box experiment Video
Socially Learned Food
Preferences in rodents
Part 1: Demonstrator rat eats a novel food alone
Novel Food A
Part 2: Observer rat socially interacts
with demonstrator after the demonstrator
is finished eating
Demonstrator
Observer rat
Part 3: Observer rat prefers the food
eaten by the demonstrator rat
Novel Food A
vs.
Novel Food B
How do they learn?
Lingering smell of novel food is associated with Carbon Disulfide smell on Demonstrator’s breadth
(Stimulus 1)
(Stimulus 2)
Operant Conditioning
– Associate responses and reinforcement (eliciting stimulus)
– Organism is active (responses are required)
– Responses that emerge or are strengthened are said
to be emitted
*Environmental Stimuli
Emitted
Response
*Eliciting
Elicited
Stimulus
Response
Eliciting Stimulus if effective is a reinforcer.
The emitted response (Operant) is increased in
probability when followed by a reinforcer.
Operant Conditioning
Shaping Video
Skinner explains operant conditioning
http://www.youtube.com/watch?v=I_ctJqjlrHA
Cognitive Learning
(Learning in the absence of eliciting stimuli)
• Latent learning
– Tolman’s rats: cognitive maps
Learning Information that may (or may not) be useful in the future
Copyright © 2005 Allyn & Bacon
Spatial Learning in the
Water maze Task
First exposure to the water maze
After several trials