Transcript Lecture 7

Connectionism
&
Connectionism and LOTH
Language of Thought
Quizzes
Some common problems:
LOTH does not require mentalese.
Face recognition is innate?
Homuncular fallacy vs. homuncular functionalism.
Folk psychology originated in Ancient Greece?
Grading
70
60-69
50-59
40-49
Below 40
A
B
C
D
F
Remember:
This quiz is only 25% of your final grade.
1st paper: 25%
2nd paper: 40%
Tutorial performance: 10%
So, if you did badly, don’t be too discouraged.
But, if you did well, don’t be too cocky!
How is connectionism an
alternative to LOTH?
• LOT usually represented as implemented by
“classical AI.” (Also known as GOFAI: “good, oldfashioned AI”.)
• Semantic symbols and syntactic rules are easy
to represent in classic AI architecture.
• Connectionism does not require symbols, but
representations can be symbolic.
Types of Connection Representations
1) Local representation.
Meows
Fur
Pointed
ears
Whiskers
Output: “it’s a cat”
This node is a local representation of “cat”.
Local representations:
• Individual nodes are symbols, and can be
components of a language of thought.
• Not typical of connectionist networks.
• Not so biologically plausible
-- “Grandmother neurons”
2) Distributed representations.
e.g.
Cat
Tiger
Leopard
Lion
See also:
www.mind.ilstu.edu/curriculum/nature_of_computers/comp
uter_types.php
Distributed representations:
• Connectionist networks are typically
distributed representations.
• Distributed representations are not
necessarily symbolic.
• Distributed representations are more
robust to damage than local
representations.
3) No representation.
More controversially, connectionist networks can have no
representational properties.
Note:
• Output of connectionist network may be recognition of a
concept, e.g. cat, mine, man, etc. but…
• Output of connectionist network may be action, e.g.
moving through space, reading aloud
• Rather than representing content, networks can just act.
Comparison
What goes on in your mind when you
decide to drink a glass of water that
is in front of you?
LOTH: the action is the conclusion of a practical
syllogism conducted through symbol
manipulation
Connectionism: the action is output of a neural net
responding to a certain set of inputs
LOTH approach:
I am thirsty.
There is a cup of water in front of me.
I believe that drinking the water will relieve my thirst.
(There are is no reason not to drink the water)
Conclusion: I drink the water.
The conclusion is reached after manipulating the
semantic symbols representing beliefs and desires
in accordance with syntactic laws.
Beliefs and desires give rise to action.
Connectionist approach:
Inputs from body
Inputs from environment
Output: I drink the water.
There are no symbols involved.
Connectionism makes eliminativism possible.
Note: in the connectionist/eliminativist approach, the mind
concocts the belief-desire explanation, “I drank the water
because I was thirsty” to explain its behavior.
But the desire (thirst) and beliefs (“the water is in front of
me”, “the water is safe to drink”, “the water will relieve my
thirst”) are not literally part of the process whereby the
mind decides to drink.
In other words, the mind only uses symbolic
representation when translating/explaining its thoughts in
language (talking to oneself or talking to others).
But how can “thirst” not play a role in deciding to
drink? Isn’t it part of the input from the body?
“Thirst” is a feeling. What plays the functional role of
“thirst” may be a mechanism to detect that the body
is low on water, or is somewhat overheated, but this
may not be recognized by you as a desire, until you
try to explain your own behavior.
Note: imagine reaching unconsciously for a glass of
water, and when someone asks, “why are you
drinking that?”, you say, “I guess I was thirsty.”
The explanation could be rather different than the
cause.
Advantages of Connectionism
1) Biological plausibility
Connectionist networks are deliberately analogous to
neural processes in the brain
Units ~ neurons
Connections ~ synapses
Activations ~ neural signals
Neuron
Connectionist unit
2) Fast Parallel Processing
• Neurons are slow.
Neurons change state very slowly compared with
computer computations. Neurons can only process 100
changes a second, whereas computers can process a
million. But the brain can solve many complex problems in
less than a second, e.g. recognizing a face.
• 100 Steps Rule
To mimic brain operations, computer programs should
solve similar problems in less than 100 steps.
• Connectionist programs are conducted through parallel
processing, thus more can be done in 100 steps.
3) Performance of connectionist networks
resembles performance of human brains
Connectionist networks are good at:
• Pattern recognition: networks can learn through
examples
• Content-addressable memory: items can be
retrieved based on their meanings or properties
• Generalizations: networks can generalize
connections between characteristics or
properties
Connectionist networks exhibit:
Graceful degradation
When a connectionist network has some
incorrect input -- “noisy input” -- or is partially
damaged, it stills performs more poorly, but
doesn’t completely break down.
4) Connectionism provides a naturalistic
mechanism for creating concepts.
No need to posit inborn concepts.
Concepts can precede language without being
inborn.
Fodor once claimed that mentalese was “the only
game in town”.
Connectionism is a new game!
Criticisms of connectionism
The advantages of connectionism revisited:
1) Biological plausibility
2) 100-steps rule
3) Pattern recognition and concept formation:
yes, but very slow
1) Biological plausibility
Networks aren’t really like neurons.
•
No reverse connections (necessary for
backward propagation) in the brain.
•
Neurons only fire or not: they cannot be both
inhibitory and excitatory.
•
Connectionist units are too fast, neurons are
quite slow.
Biological plausibility (cont.)
• There are many different types
of neurons in the brain, but
connectionist units are meant
to represent all neurons.
• In addition, role of
neurotransmitters and
hormones in thinking is ignored
in connectionist models.
Different types of neurons
Note: most people admit that connectionist
networks are still more biologically plausible
than classical AI architectures.
2) The 100-steps rule
Problem: what is a step?
Is, recognizing a color one step? Or does it break down into
numerous steps?
The 100-steps rule only works if each unit of a
connectionist network corresponds to one neuron.
If one unit corresponds to several neurons working
together, the 100-steps constraint may be greatly
exceeded.
Also, the 100-steps argument assumes only connectionist
architectures are parallel processors, and all classical
architectures are serial. But it is possible to build parallel
classical architectures.
3) Network learning is extremely slow
Connectionist networks need a huge amount of
explicit feedback to learn.
The brain often can learn a new concept or pattern
in one shot.
One-shot learning is especially easy when
information is gathered through language.
Example: think of teaching an intelligent chimp vs.
a five-year-old child, to push the red button for
food.
Another weakness of Connectionism
Systematicity and productivity: very difficult (impossible?) to
implement in connectionist architecture.
Connectionist responses:
1)
Deny systematicity and productivity of the mind:
Is human thinking really systematic/productive?
Do animals think systematically/productively?
2)
Maintain the ability of connectionist nets to generate
systematicity and productivity
The Relationship between
Connectionism and LOTH
Three possibilities:
1) Connectionism implements LOTH
2) Connectionism replaces LOTH
3) Hybrid theory. Some mental processes
are connectionist, some are conducted
through LOT.
1) Connectionism implements LOT
Connectionist nets can be regarded
as a lower-level implementation
of LOT.
Neural nets can represent semantic
symbols which are then manipulated in accordance
with language-like laws (also implemented by neural
nets).
Criticism: if connectionist nets only implement LOT, many
of the advantages of connectionism are lost.
2) Connectionism replaces LOT.
Consequence: all the advantages (e.g.
systematicity and productivity) of LOT are
lost.
Can we do without them?
3)Hybrid theory.
Some mental processes are connectionist,
some are conducted through LOT.
e.g.
Perception, pattern recognition and motor
control handled by connectionist nets.
Reasoning and language handled by LOT
(and implemented by connectionist nets).
Connectionism and Modularity
Connectionist networks can do simple, small tasks.
In more complicated tasks, they are overwhelmed by the
complexity (because the connections increase
exponentially).
Mind must be organized into simple units, connected up in
an efficient way.
“Connectoplasm”: the mind an unorganized mess of
connections. Not a viable idea.
Mental modules: some connections preset, some learned.
A way to contain the complexity.
Readings for next week
Required:
Thomas Nagel (1974), “What is it like to be a bat?”, The Philosophical Review, LXXXIII, 4
(October 1974), 435-50
at: members.aol.com/NeoNoetics/Nagel_Bat.html
Block (2002), “Some Concepts of Consciousness”, in David Chalmers (Ed.). Philosophy
of Mind: Classical and Contemporary Readings Oxford University Press
at: www.nyu.edu/gsas/dept/philo/faculty/block/papers/Abridged%20BBS.htm
Optional:
Gallup, Jr., Povinelli (1998). Can Animals Empathize? Scientific American Exploring Intelligence (a debate), available on reserve at Philosophy
Department