Transcript Slide 1

Connectionism
&
Connectionism and LOTH
Language of Thought
How is connectionism an
alternative to LOTH?
• LOT usually represented as implemented by
“classical AI.” (Also known as GOFAI: “good, oldfashioned AI”.)
• Semantic symbols and syntactic rules are easy
to represent in classic AI architecture.
• Connectionism does not require symbols, but
representations can be symbolic.
Types of Connectionist Representations
1) Local representation.
Meows
Fur
Pointed
ears
Whiskers
Output: “it’s a cat”
This node is a local representation of “cat”.
Characterizing local representations:
• Individual nodes are symbols, and can be
components of a language of thought.
• Not typical of connectionist networks.
• One neuron per symbol does not seem
biologically plausible. Cell assemblies have
thus been proposed for neuro-symbols:
http://publik.tuwien.ac.at/files/PubDat_166316.pdf
2) Distributed representations.
E.g.:
Cat
Tiger
Leopard
Lion
See also:
David L. Anderson. Computer Types: Classical vs. Non-classical
http://www.mind.ilstu.edu/curriculum/nature_of_computers/computer_types.php
(cf. SmartKitchen: http://smartkitchen.ict.tuwien.ac.at/project/project.html)
Characterizing distributed representations:
• Connectionist networks are typically
distributed representations.
• Distributed representations are not
necessarily symbolic.
• Distributed representations are more
robust to damage than local
representations.
3) No representation.
More controversially, connectionist networks might have no
representational properties.
Note:
• Output of connectionist network may be recognition of a
concept, e.g. Cat, Tiger, Man, etc. but…
• Output of connectionist network may also be action, e.g.
moving through space, reading aloud
• Rather than representing content, networks can just act.
Comparison
What goes on in your mind when you
decide to drink a glass of water that
is in front of you?
LOTH: the action is the conclusion of a practical
syllogism conducted through symbol
manipulation
Connectionism: the action is output of a neural
net responding to a certain set of inputs
LOTH approach:
I am thirsty.
There is a cup of water in front of me.
I believe that drinking the water will relieve my thirst.
(There is no reason not to drink the water)
Conclusion: I drink the water.
The conclusion is reached after manipulating the
semantic symbols representing beliefs and desires
in accordance with syntactic laws.
Beliefs and desires give rise to action.
Connectionist approach:
Inputs from body
Inputs from environment
Output: I drink the water.
There are no symbols involved.
Connectionism makes eliminativism possible.
Note: in the connectionist/eliminativist approach, the mind
concocts the belief-desire explanation, “I drank the water
because I was thirsty” to explain its behavior.
But the desire (thirst) and beliefs (“the water is in front of
me”, “the water is safe to drink”, “the water will relieve my
thirst”) are not literally part of the process whereby the
mind decides to drink.
In other words, the mind only uses symbolic
representation when translating/explaining its thoughts in
language (talking to oneself or talking to others).
But how can “thirst” not play a role in deciding to
drink? Isn’t it part of the input from the body?
“Thirst” is a feeling. What plays the functional role of
“thirst” may be a mechanism to detect that the body
is low on water, or is somewhat overheated, but this
may not be recognized by you as a desire, until you
try to explain your own behavior.
Note: imagine reaching unconsciously for a glass of
water, and when someone asks, “why are you
drinking that?”, you say, “I guess I was thirsty.”
The explanation could be rather different than the
cause (cf. Freud’s concept of rationalization).
Advantages of Connectionism
1) Biological plausibility
Connectionist networks are deliberately analogous to
neural processes in the brain
Units ~ neurons
Connections ~ synapses
Activations ~ neural signals
Neuron
Connectionist unit
2) Fast processing via parallelism
• “100 Step” argument.
Neurons change state very slowly compared with computer
computations. Neurons can only process 100 steps a second
(whereas computers can process a million). But the brain can
solve many complex problems in less than 1 second, e.g.
face recognition  for these, it can use maximally 100 steps.
• Conventional computer programs do mostly serial processing
and usually require considerably more than 100 processing
steps for problems where brains need less than a second.
So, such computers cannot provide a good model of cognition.
• Connectionist computations are done by parallel processing,
thus much more can be achieved in 100 steps.
Cf.: http://www.ucs.louisiana.edu/~isb9112/dept/phil341/myths/myths.html
3) Performance of connectionist networks
resembles performance of human brains
Connectionist networks are good at:
• Pattern recognition: networks can learn through
examples
• Content-addressable memory: items can be
retrieved based on their meanings or properties
• Generalizations: networks can generalize
connections between characteristics or
properties
Connectionist networks exhibit:
Graceful degradation
When a connectionist network has some
incorrect input -- “noisy input” -- or is itself
partially damaged, it still performs, more poorly,
but doesn’t completely break down.
4) Connectionism provides a naturalistic
mechanism for creating concepts.
No need to posit inborn concepts.
Concepts can precede language without being
inborn.
Fodor once claimed that mentalese was
“the only game in town”.
Connectionism is a new game!
Criticisms of connectionism
The advantages of connectionism revisited:
1) Biological plausibility
2) 100-steps argument
3) Pattern recognition and concept formation:
yes, but can be slow
1) Biological plausibility
Networks aren’t really like neurons.
•
No reverse connections (necessary for
backward propagation) in the brain.
•
Neurons only fire or not: they cannot be both
inhibitory and excitatory.
•
Connectionist units are too fast, neurons are
quite slow.
Biological plausibility (cont.)
• There are many different types
of neurons in the brain, but
connectionist units are meant
to represent all neurons.
• In addition, role of
neurotransmitters and
hormones in thinking is ignored
in connectionist models.
Different types of neurons
Note: most people admit that connectionist
networks are still more biologically plausible
than classical AI architectures.
2) The 100 step argument
Problem: what is a step?
Is, recognizing a color one step? Or does it break down into
numerous steps?
The 100 step argument only works if each unit of a
connectionist network corresponds to one neuron.
If one unit corresponds to several neurons working
together, the 100 step constraint may be greatly
exceeded.
Also, the 100 step argument assumes only connectionist
architectures are parallel processors, while all nonconnectionist architectures are serial. But it is possible to
build parallel non-connectionist architectures. Cf. review:
http://www.icsr.agh.edu.pl/publications/html/ppam97prof/ppam97prof.html
3) Network learning can be slow
Many connectionist networks need a large amount
of explicit feedback to learn. Others, e.g. selforganizing maps, use unsupervised learning :
http://www.willamette.edu/~gorr/classes/cs449/Unsupervised/SOM.html
The brain often seems to learn a new concept or
pattern in one shot.
One-shot learning is especially easy when
information is gathered through language.
Example: think of teaching an intelligent chimp vs.
a five-year-old child, to push the red button for
food.
Another weakness of Connectionism
Systematicity and productivity: very difficult (impossible?) to
implement in connectionist architecture.
Connectionist responses:
1)
Deny systematicity and productivity of the mind:
Is human thinking really systematic/productive?
Do animals think systematically/productively?
2)
Maintain the ability of connectionist nets to generate
systematicity and productivity
The Relationship between
Connectionism and LOTH
Three possibilities:
1) Connectionism implements LOTH
2) Connectionism replaces LOTH
3) Hybrid theory. Some mental processes
are connectionist, others are conducted
through LOT.
1) Connectionism implements LOT
Connectionist nets can be regarded
as a lower-level implementation
of LOT.
Neural nets can represent semantic
symbols which are then manipulated in accordance
with language-like laws (also implemented by neural
nets).
Criticism: if connectionist nets only implement LOT, many
of the advantages of connectionism are lost.
2) Connectionism replaces LOT
Consequence: all the advantages (e.g.
systematicity and productivity) of LOT are
lost.
Can we do without them?
3) Hybrid theory
Some mental processes are connectionist,
others are conducted through LOT.
E.g.:
Perception and motor control handled by
connectionist nets.
Reasoning and language handled by LOT,
and implemented by connectionist nets.
Class-level logic of hybrid theory presented as a graph
subClassOf
Mental Process
Peripheral Process
Perception
Motor Control
Central Process
Reasoning
handle
handle
Connectionist Net
LOT
implement
Language
Rule composing the implement and handle relations
Z
handle
Y
realize
X
implement
Class-level logic extended by inferred relation
subClassOf
Mental Process
Peripheral Process
Perception
Motor Control
Central Process
Reasoning
Language
handle
handle
LOT
realize
Connectionist Net
implement
Connectionism and Modularity
Connectionist networks can do simple, small tasks.
In more complicated tasks, they are overwhelmed by the
complexity (because the connections increase
exponentially).
Mind must be organized into simple units, connected up in
an efficient way.
“Connectoplasm”: the mind an unorganized mess of
connections. Not a viable idea.
Mental modules: some connections preset, others learned.
A way to contain the complexity (maybe even recursively:
modules of modules).
Readings for next week
Focus:
Thomas Nagel (1974), “What is it like to be a bat?”, The Philosophical Review, LXXXIII, 4
(October 1974), 435-50
http://www.clarku.edu/students/philosophyclub/docs/nagel.pdf
Block (2002), “Some Concepts of Consciousness”, in David Chalmers (Ed.). Philosophy
of Mind: Classical and Contemporary Readings Oxford University Press
http://www.nyu.edu/gsas/dept/philo/faculty/block/papers/Abridged%20BBS.htm
Extra:
Gallup, Jr., Povinelli (1998). Can Animals Empathize? Yes. Scientific American Exploring Intelligence (a debate).
http://www.sciamdigital.com/index.cfm?fa=Products.ViewIssuePreview&ARTICLEID_
CHAR=9123A7A5-59B3-4355-8946-C0E31A72A09