Transcript Representing Regularity: The English Past Tense

Representing Regularity:
The English Past Tense
Matt Davis
William Marslen-Wilson
Centre for Speech and Language
Birkbeck College
University of London
and
Mary Hare
Center for Research in Language
University of California
San Diego
 Abstract:
Evidence from priming experiments suggests
differences in the lexical representation of regular
and irregular forms of the English past tense. Such
results have been used to argue for a dual
mechanism account of English inflectional
morphology.
A single mechanism connectionist model is
described which learns an abstract version of the
task of recognising English inflected verbs. Analysis
of the networks internal representations show
differences between regular and irregular verbs that
could account for the priming data. This suggests
that behavioural and representational differences
need not be taken as evidence for two distinct
processing mechanisms.
1
The English past tense has been a popular case-study for investigating language
processing since it provides clear examples of both regular and irregular linguistic
processes. Psycholinguistic accounts of English inflection have focused on the process
or processes that map between stem and past tense forms. The debate between single
and dual mechanism accounts of language processing has been directed at the
psychological status of the rule that describes how a verb stem is inflected to produce a
regular past tense.
 Dual Mechanism Accounts:
(e.g. Pinker 1991)
•Regular verbs
Inflected by a symbolic rule-based
system
 Single Mechanism Accounts:
(eg. Rumelhart & McClelland 1986)
•Regular and Irregular verbs
Both regular and irregular verbs are
inflected by a distributed network
mapping from verb stems to past tenses
•Irregular verbs
Stored in an (associative) memory
system that blocks the application of the
rule-governed route
2
These accounts, focusing just on the phonological relationship between verb stems and
past tenses seem unsatisfactory as an account of comprehension or production, and
make the implicit assumption that accessing the lexical representation of an inflected
verb proceeds via a phonological representation of the verb stem.
Experiments using a repetition priming task have cast doubt on this assumption since
they suggest that the representations accessed in comprehending inflected words differ
according to the regularity of the inflection.
 Hare, Older, Ford and
Marslen-Wilson (1995)
•Cross-modal immediate repetition
priming:
– Subjects hear an auditory prime
– A visual target is presented on a
computer screen at the acoustic
offset of the prime
– Subjects make a lexical decision
response to the target word
•Compared lexical decision RTs to
verb stems preceded by:
– Past tense primes (reg/irreg)
– Present tense primes (all reg)
– Unrelated control primes
•Tested all the irregular verbs in
British English and matched regular
verbs
– Excluding homophones
(e.g. ate/eight)
– Excluding identity inflected verbs
(e.g. hit)
3
Results show that the past tense of regular verbs significantly prime their stems,
whereas irregular verbs do not. Such data is hard to explain in terms of semantic or
phonological priming and has been interpreted as evidence for differences in the lexical
representation of regular and irregular verbs; a dual mechanism account (Pinker 1991
citing Stanners et. al. 1979). Our purpose here is to investigate whether representational
differences between regular and irregular verbs can be accounted for by a single
mechanism, connectionist model.
Previous research has shown
that this cross-modal repetition
priming task is not susceptible
to form based priming (i.e.
whisky doesn’t prime whisk).
Marslen-Wilson et. al. (1994)
 Results:
Hare, Older, Ford and Marslen-WIlson (1995)
60
Regular
(jump - jumped)
Semiweak
Priming / ms
(control - test)
50
40
30
20
(bend - bent)
Vowelchange
(give - gave)
10
0
Present Tense
Prime
Past Tense
Prime
4
The network we report here was trained to map from a phonological input to a
distributed “semantic” vector and a tense output. This is the reverse of the mapping
investigated by Cottrell and Plunkett (1991) - and can be seen as analogous to the
comprehension of inflected verbs. The network was trained on 988 monosyllabic
English verbs, each presented as a stem and a past tense in proportion to their log
frequency of occurrence. An additional 110 regular verbs were presented in one form
only, to allow testing of the networks generalization abilities.
 Network trained to identify verb
 Training Set:
stems and past tenses
“Semantics”
(50 units)
Regular
Irregular
Total
No. Types
875
113
988
% Types
88.6%
11.4%
No. Tokens
1063
286
% Tokens
78.8%
21.2%
Tense
(2 Units)
Hidden Units
(200 Units)
A 50 bit random
vector that uniquely
identifies each verb
root.
Verb Type
Phonology
(58 Units)
A structured phonological representation developed
for models of reading aloud. It uses phonotactics
and sonority to minimise duplication of segments
within mono-syllables. Plaut et al. (1996)
1349
5
The network was trained for 2000 passes through the training set at which point the
training error curve had reached asymptote and training stopped. The performance of
the network was then evaluated using a nearest target criterion.
The hidden unit representations developed by the network to perform the mapping were
also evaluated. Measures of the Euclidean Distance between the representation of stem
and past tense forms of regular and irregular verbs were taken.
 Training set:
 Euclidean Distance between stem
Error rate < 3% (of 1768 items)
Most were homophone errors
build - billed
and past tense representations:
(65%)
Some tense errors
threw - identified as stem (35%)
dread - identified as past tense
 Test set:
The network was correct on 85% of
the novel forms of familiar verbs
(of 110
items)
E.D. =
n=r
2
(
s
p
)
 n n
n=0
r = total no. of units in group
sn = stem activation, unit n
pn = past activation, unit n
6
Distance measures in hidden unit space show that the representation of stems and past
tenses is more similar for regular than for irregular verbs. However we need to confirm
that this is an effect of regularity and not just differences in the amount of phonological
overlap. The same analysis was therefore carried out on the input representations.
Comparing distance measures in the input and hidden units shows that the representation of regular verbs is significantly more similar than would be predicted on the basis
of phonological overlap alone.
(jump - jumped)
Semiweak
Vowelchange
Unrelated
control
Phonological
Control
(bend - bent)
(give - gave)
(shake - halt)
(store - storm)
3
8
7
6
2
5
4
3
1
2
1
0
Euclidean Distance
The unequal scales in
the two graphs reflect
the different numbers
of units in the input and
hidden
unit
representations.
Regular
Euclidean Distance
ANOVA on ratio of
input/hidden distance
show significant differences between the
three sets of verbs.
F(2,961)=1434.4,
p<0.0001
0
Hidden Units
Input Units
7
The network appears to have learnt to use the consistent relationship between the final
segment of regular verbs and their tense. This can be seen in the networks
generalization performance, and the tense errors that it makes after training. Without
the inflectional ending on regular verbs, the network can then map an invariant
phonological form onto the semantics of the verb. Hence the very similar representation
of both forms of the regular verbs at the hidden units.
However for the irregular verbs, this process breaks down; either through changes in
the verb stem (semi-weak verbs such as sleep-slept), exceptions to the affix-tense
regularity (semi-weak verbs such as bend-bent) or a combination of the two (vowelchange verbs such as give-gave). In these cases there is no longer a consistent mapping
for both forms of the verb and the network must therefore develop more separate
representations at the hidden units.
Regular verbs:
Irregular verbs:
(turn - turned)
tn
tn -d
(guide - guided)
gaId
gaId -Id
(talk - talked)
t
t - t
(sleep-slept) slp
slEpt
tense
marking
changes to
verb root
(bend-bent) bEnd
bEnt
(give-gave)
gIv
geIv
exceptional
tense marking
Since the degree of overlap between two distributed representations is correlated with
the magnitude of priming observed in a network (Masson 1995), this finding provides
an account of the reduced priming observed for irregular verbs.
Moreover, since this is the result of a single mechanism, connectionist model trained on
a mixture of verbs, the network further suggests that behavioral and representation
differences between regular and irregular verbs need not imply different processing
mechanisms.
 References:
Cottrell, G. W. & Plunkett, K. (1991). Learning the past tense in a
recurrent network: Acquiring the mapping from meanings to sounds.
In Proceedings of the Thirteenth Annual Conference of the
Cognitive Science Society. Hillsdale NJ: Lawrence Erlbaum
Associates.
Hare, M., Older, L., Ford, M. & Marslen-Wilson, W. (1995)
Frequency, competition and lexical representation. In Proceedings
of the Seventeenth Annual Conference of the Cognitive Science
Society. Hillsdale NJ: Lawrence Erlbaum Associates.
Marslen-Wilson, W., Tyler, L., Waksler, R. & Older, L. (1994).
Morphology and meaning in the English mental lexicon.
Psychological Review. 101(1), 3-33
Masson, M. E. J. (1995). A distributed memory model of semantic
priming. Journal of Experimental Psychology: Learning, Memory
and Cognition. 2(1), 3-23.
Pinker, S. (1991). Rules of language. Science, 253, 530-535.
Plaut, D. C., McClelland, J. L., Seidenberg, M. S. & Patterson, K.
(1996) Understanding normal and impaired word reading Computational principles in quasi-regular domains. Psychological
Review. 103(1), 56-115.
Rumelhart, D. E. & McClelland, J. L. (1986). On learning the past
tense of English verbs. In J. L. McClelland, D. E. Rumelhart and
PDP Research Group (Eds), Parallel distributed processing:
Volume 2 (pp. 216-271). Cambridge, MA: MIT Press.
Stanners, R. F., Neiser, J. J., Hernon, W. P. & Hall, R. (1979).
Memory representation for morphologically related words. Journal
of Verbal Learning and Verbal Behaviour, 18, 399-412.
 Acknowledgements:
Thanks are due to David Plaut for providing his phonological
representation for use in the network.
Thanks also to John Bullinaria, Gareth Gaskell, Tom Loucas
Lianne Older, Bilii Randall and members of the Morphology and
Modelling group at the Centre for Speech and Language for useful
discussions.
9