Transcript Syllabus P140C (68530) Cognitive Science

COMPUTATIONAL COGNITIVE
SCIENCE
Cognitive Revolution
• Development of computer led to rise of cognitive
psychology and artificial intelligence
BINAC: the Binary Automatic Computer,
developed in 1949
Artificial Intelligence
• Constructing artificial computer-based systems
that produce intelligent outcomes
• Examples
– Game playing programs
• Deep blue
– Intelligent robots
• Mars rovers
• Darpa’s urban challenge
– Netflix competition
– Conversational agents
Weak vs. Strong AI
• Weak AI — using AI as a tool to understand
human cognition
• Strong AI — a properly programmed computer
has a “mind” capable of understanding
Turing Test
• Can artificial intelligence be as good as
human intelligence? How can we test
this?
• Turing test (1950)
– designed to test whether humans can distinguish
between humans and computers based on
conversations
– A human interrogator could ask a respondent
(either a computer or a human, whose identity was
hidden) any question he or she wished, and based
on either the computer’s or the human’s response,
the interrogator had to decide if the answer was
given by the computer or by the human.
Alan Turing (1912-1954)
A classic conversational agent
• The Turing Test inspired an early, satirical, attempt to
create a computerized Rogerian therapist, “Eliza”:
Eliza’s program is a set of “If…Then” rules:
– “IF person mentions ‘father’ THEN say ‘Who else in
your family comes to mind when you think about this?’
– “IF person writes ‘I remember X’ THEN say ‘Does it
make you feel happy to recall X?’
ELIZA has no intelligence itself. Intelligence
comes from people interpreting its
statements.
Example of modern conversational agents
http://testing.turinghub.com/
[following is a fragment where A=your instructor; B = very good artificial agent]
A: are you human?
B: What do you think? I am a Supreme Ruler.
A: seriously
B: I am always serious.
A: who was Turing?
B: A brilliant and tragic figure in the history of computer science.
A: Do you think the Turing test is a good one?
B: Some of them can.
A: are you having a good day?
B: I am having fun.
A: How do you know you are having fun?
B: By introspection.
Future of AI
• Computer chip capacity and
processing speed are
increasing exponentially
• Some theorists (e.g. Ray
Kurzweil) believe this will
lead to a technological
singularity along with
dramatic improvements in
AI
Computational Modeling
• Most modeling in cognitive science targets
natural intelligence
• Goal is to develop model or mimic some aspects
of human cognitive functioning
– produce the same errors as humans
 Simulations of aspects of human behaviour
Why do we need computational models?
• Makes vague verbal terms specific
– Provides precision needed to specify complex
theories.
• Provides explanations
• Obtain quantitative predictions
– just as meteorologists use computer models to predict
tomorrow’s weather, the goal of modeling human
behavior is to predict performance in novel settings
Neural Networks
Neural Networks
• Alternative to traditional information processing
models
• Also known as:
– PDP (parallel distributed processing approach)
– Connectionist models
David Rumelhart
Jay McClelland
Neural Networks
• Neural networks are networks of simple processors
that operate simultaneously
• Some biological plausibility
Idealized neurons (units)
Inputs
Processor
S
Output
Abstract, simplified description of a neuron
Neural Networks
• Units
– Activation = Activity of unit
– Weight = Strength of the connection between two units
• Learning = changing strength of connections
between units
• Excitatory and inhibitory connections
– correspond to positive and negative weights
respectively
An example calculation for a single (artificial) neuron
Diagram showing how the
inputs from a number of
units are combined to
determine the overall input
to unit-i.
final output
Unit-i has a threshold of 1;
so if its net input exceeds 1
then it will respond with +1,
but if the net input is less
than 1 then it will respond
with –1
What would happen if we
change the input J3 from
+1 to -1?
a) output changes to -1
b) output stays at +1
c) do not know
What would happen if we
change the input J4 from
+1 to -1?
a) output changes to -1
b) output stays at +1
c) do not know
final output
If we want a positive
correlation between the
output and input J3, how
should we change the
weight for J3?
final output
a) make it negative
b) make it positive
c) do not know
Multi-layered Networks
output units
• Activation flows from a layer of
input units through a set of
hidden units to output units
• Weights determine how input
patterns are mapped to output
patterns
hidden units
input units
Multi-layered Networks
output units
• Network can learn to associate
output patterns with input
patterns by adjusting weights
• Hidden units tend to develop
internal representations of the
input-output associations
hidden units
• Backpropagation is a common
weight-adjustment algorithm
input units
A classic neural network: NETtalk
network learns to pronounce English words: i.e., learns spelling to sound
relationships. Listen to this audio demo.
teacher
target output
/k/
26 output units
80 hidden units
7 groups of
29 input units
_
a
_
c
target letter
(after Hinton, 1989)
a
t
_
7 letters of
text input
Different ways to represent information with
neural networks: localist representation
concept 1
1 0 0 0 0 0
concept 2
0 0 0 1 0 0
concept 3
0 1 0 0 0 0
(activations of units; 0=off 1=on)
Each unit represents just one item  “grandmother” cells
Distributed Representations
(aka Coarse Coding)
concept 1
1 1 1 0 0 0
concept 2
1 0 1 1 0 1
concept 3
0 1 0 1 0 1
(activations of units; 0=off 1=on)
Each unit is involved in the representation of multiple items
Suppose we lost unit 6
concept 1
1 1 1 0 0 0
concept 2
1 0 1 1 0 1
concept 3
0 1 0 1 0 1
(activations of units; 0=off 1=on)
Can the three concepts
still be discriminated?
a) NO
b) YES
c) do not know
Representation A
Unit
1
W 1
X 1
Y 1
Z 1
Unit
2
0
0
0
0
Unit
3
0
0
0
0
Unit
4
0
0
0
0
Representation B
W
X
Y
Z
Unit
1
1
0
0
1
Unit
2
0
1
1
0
Unit
3
0
1
0
1
Unit
4
1
0
1
0
Which representation is a good example of
distributed representation?
a) representation A
b) representation B
c) neither
Advantage of Distributed Representations
• Efficiency
– Solve the combinatorial explosion problem: With n
binary units, 2n different representations possible.
(e.g.) How many English words from a combination of
26 alphabet letters?
• Damage resistance
– Even if some units do not work, information is still
preserved – because information is distributed across
a network, performance degrades gradually as
function of damage
– (aka: robustness, fault-tolerance, graceful
degradation)
Neural Network Models
• Inspired by real neurons and brain organization but are
highly idealized
• Can spontaneously generalize beyond information
explicitly given to network
• Retrieve information even when network is damaged
(graceful degradation)
• Networks can be taught: learning is possible by changing
weighted connections between nodes
Recent Neural Network Research
(since 2006)
• “Deep neural networks” by Geoff Hinton
– Demos of learning digits
– Demos of learned movements
• What is new about these networks?
– they can stack many hidden layers
– can capture more regularities in data and
generalize better
– activity can flow from input to output
and vice-versa
In case you want to see more details: YouTube video
Geoff Hinton
Samples generated by network by propagation activation from label
nodes downwards to input nodes (e.g. pixels)
Graphic in this slide from Geoff Hinton
Examples of correctly recognized handwritten digits
that the neural network had never seen before
Graphic in this slide from Geoff Hinton
Other Demos & Tools
If you are interested, here are tools to create your own neural networks and train it on data:
Hopfield network
http://www.cbu.edu/~pong/ai/hopfield/hopfieldapplet.html
Backpropagation algorithm and competitive learning:
http://www.psychology.mcmaster.ca/4i03/demos/demos.html
Competitive learning:
http://www.neuroinformatik.ruhr-uni-bochum.de/ini/VDM/research/gsn/DemoGNG/GNG.html
Various networks:
http://diwww.epfl.ch/mantra/tutorial/english/
Optical character recognition:
http://sund.de/netze/applets/BPN/bpn2/ochre.html
Brain-wave simulator
http://www.itee.uq.edu.au/%7Ecogs2010/cmc/home.html