Transcript Semantics Without Categorization

Emergence of Semantics from
Experience
Jay McClelland
Department of Psychology and
Center for Mind, Brain, and Computation
Stanford University
1
The Parallel Distributed Processing
Approach to Semantic Cognition
• Representation is a pattern of
activation distributed over
neurons within and across
brain areas.
language
• Bidirectional propagation of
activation underlies the ability
to bring these representations
to mind from given inputs.
• The knowledge underlying
propagation of activation is in
the connections.
• Experience affects our
knowledge representations
through a gradual connection
adjustment process
2
Distributed Representations:
and Overlapping Patterns for Related
Concepts
dog
goat
hammer
dog goat hammer
3
Emergence of Meaning in Learned
Distributed Representations
• Learned distributed representations that
capture important aspects of meaning emerge
through a gradual learning process in simple
connectionist networks
• The progression of learning captures several
aspects of cognitive development:
– Differentiation of Concepts
– Illusory Correlations
– Overgeneralization
– And many other things
4
5
The Rumelhart Model
6
The Training Data:
All propositions true of
items at the bottom level
of the tree, e.g.:
Robin can {grow, move, fly}
7
Target output for ‘robin can’ input
8
Forward Propagation of Activation
aj
wij
neti=Sajwij
ai
wki
9
Back Propagation of Error (d)
aj
wij
di ~
Sdkwki
ai
wki
Error-correcting learning:
At the output layer:
At the prior layer:
…
dk ~ (tk-ak)
Dwki = edkai
Dwij = edjaj
10
11
12
Early
Later
Later
Still
E
x
p
e
r
i
e
n
c
e
13
Why Does the Model
Show Progressive
Differentiation?
•
Learning is sensitive to
patterns of coherent
covariation
•
Coherent Covariation:
– The tendency for
properties of objects
to co-vary in clusters
•
Figure shows attribute loadings on
the principal dimensions of
covariation. These capture:
–
–
–
1. Plants vs. animals
2. Birds vs. fish
3. Trees vs. flowers
•
Same color = features that covary
•
Diff color
= anti-covarying
features
14
Trajectories of Concept Representations During Differentiation
15
Illusory Correlations
16
A typical property that
a particular object lacks
e.g., pine has leaves
An infrequent,
atypical property
17
Overgeneralization of Frequent
Names to Similar Objects
“goat”
“tree”
“dog”
18
19
Other Applications of the Model
• Expertise effects
• Conceptual reorganization
• Effects of language and culture
• Effects of brain damage:
– Loss of differentiation
– Overgeneralization in
object naming
– Illusory correlations
camel
swan
20
Conclusion
• We represent objects using patterns of activity
over neuron-like processing units
• These patterns depend on connection weights
learned through experience
• Differences in experience lead to differences in
conceptual representations.
21