Transcript document

On using eyetracking data to evaluate
theories of on-line sentence processing:
The case of reduced relative clauses
Charles Clifton, Jr.
grateful acknowledgments to
Mohamed Mohamed
Matt Traxler
Rihana Williams
Keith Rayner
Robin Morris
Sungryong Koh
Lyn Frazier
Eye movements and sentence
comprehension
• Existing measures not a transparent window
into cognitive processes
– Lexical processing: some good ideas about
mapping
– Comprehension: another story
– Consider one case of garden-path sentences…
Garden Paths in Reduced
Relative Clause Sentences
(the curse of Tom Bever)
• The horse raced past the barn fell.
• But sometimes reduced relative clauses are
easy to comprehend.
McKoon & Ratcliff, 2003
• The window broken by John couldn’t be
repaired
• The window examined by John needed
repairing
• Both cases: (easy) reduced relative
construction denotes an entity that
participates in an externally caused event
Trueswell, Tanenhaus, &
Garnsey, 1994
• The defendant examined by the lawyer proved to
be unreliable
– Produces clear reading disruption in disambiguating
region compared to full relative clause (The defendant
who was…)
• The evidence examined by the lawyer proved to be
unreliable
– Initial NP inanimate, and a good theme but a poor agent
– Claimed to eliminate difficulty of reduced relative
clause
• TTG 1994 actually a followup to Ferreira &
Clifton, 1986
Ferreira & Clifton 1987, Disambiguating Region
First Pass Time, Ms/char
50
ReducedRel
FullRel
40
30
20
0
Animate
Inanimate
Animacy
Animate: The defendant examined by the lawyer…
Inanimate: The evidence examined by the lawyer…
Criticism: Trueswell, Tanenhaus, &
Garnsey, 1994
• Some of the Ferreira & Clifton inanimate
sentences weren’t all that implausible as
main clause sentences
– The trash smelled…. (pretty bad)
– The car towed…(the trailer)
• TTG, better materials, first noun normed for
plausibility as agent and as patient of first
verb
Trueswell, Tanenhaus, & Garnsey
Disambiguating Region
First Pass Time, ms
600
RedRel
FullRel
500
400
0
Animate
Inanimate
Subject Noun
Why the differences?
• Maybe TTG were right – the F&C materials
were bad
• But maybe something else: parafoveal
preview of “by the…”
– Evidence from Burgess, Spivey, McRae, etc, in
self-paced reading
– Possible display limitations in F&C (42
character display, line break typically before PP
would have prevented effective preview)
Clifton, Traxler, Mohamed,
Williams, Morris, Rayner, in press
• Redo Ferreira & Clifton using Trueswell et
al. materials (thanks to John et al.)
• Add boundary change manipulation to
permit or deny parafoveal preview of “by
the noun”
• Perform additional analyses
Clifton et al. details
• Used materials from Trueswell, Tanenhaus, &
Garnsey
• 8 conditions
– Animate (“difficult”) vs inanimate (“easy”) initial noun
– Reduced vs full (control) relative clause
– Parafoveal preview vs. no preview of “by the” (reader
saw random 6-letter string instead of “by the”)
• Measured eye movements while reading for
comprehension (24 Ss)
Disambiguating Region, FP
800
First First
Pass Time,
ms ms
Pass Time,
Disambiguating Region, FP
800
700
700
600
RedRel
FullRel
RedRel
FullRel
600
500
500
400
400 0
0
Animate Inanimate
Preview
Animate Inanimate
No Preview
Animate Inanimate Animate Inanimate
Previeweffect for animate
No Previewnouns,
ambiguity
Overall: 66 ms
34 ms
ambiguity effect for inanimate nouns (sig. smaller, but > 0)
Regression Path Duration, ms
Disambiguating Region, RPD
800
RedRel
700
FullRel
600
500
400
0
Animate Inanimate Animate Inanimate
Preview
No Preview
Overall: 68 ms ambiguity effect for animate nouns, 83 ms
ambiguity effect for inanimate nouns (n.s. different)
Interim Conclusion and Question
• Looking only at first pass times gave an
incomplete picture of sentence
comprehension
– RPD measure indicated garden-pathing even
with inanimate initial noun
– Trueswell et al. experiment not an adequate
basis to reject modular serial parsing model
• But do the increased RPDs reflect the usual
behavior of the eyes?
Proportion First Pass Regressions Out
Disambiguating Region, Reg Out
25.0
20.0
RedRel
FullRel
15.0
10.0
5.0
0.0
Animate Inanimate Animate Inanimate
Preview
No Preview
More regressions for No Preview than Preview, and for Animate
than Inanimate, but apparent effect of Reduction and interactions
not significant.
20.0
RedRel
FullRel
15.0
10.0
5.0
0.0
Second Pass Time, ms., from Left
Animate Inanimate Animate Inanimate
Preview
No Preview
Regression Path Duration, ms., Outside Region
Proportion First Pass Regressions Out
Disambiguating Region, Reg Out
25.0
Disambiguating Region, RPD Outside
120
100
80
RedRel
FullRel
60
40
20
0
Animate Inanimate Animate Inanimate
Previe w
No Previe w
Disambiguating Region, Second Pass from Left
120
100
RedRel
FullRel
80
60
40
20
0
Animate Inanimate Animate Inanimate
Previe w
No Previe w
A similar pattern for:
regression path duration
(outside the region)
second pass on target
(coming from left)
Combined data: First pass regressions
• Combine data with parallel experiment
conducted at U South Carolina
– Always preview of “by the” phrase
– Vary high vs low span readers
• Total n = 68, more stable data
Proportion First Pass Regressions
Combined Data, Regressions Out
14.0
12.0
RedRel
10.0
FullRel
8.0
6.0
4.0
2.0
0.0
Anim
Inan
Condition
Significant effects of animacy, ambiguity, no interaction
Summary
• Eye movement measures show comparable
ambiguity effects for sentences with
animate and with inanimate subjects
– It is premature to conclude that inanimacy
blocks garden-pathing
– Other data are needed to evaluate models that
give logical priority to syntactic structure
A problem for understanding eye
movements
• Regression path durations are inflated by
processing difficulty
– This inflation can only come from trials where
there were regressions
• But regressions were very infrequent, <
10% of the trials
A similar effect in other
experiments
• Reduced relative clauses with various
classes of verbs (manner of motion, thetagrid changing, unaccusatives)
• No garden-pathing in first pass times, big
garden-pathing in RPD, but < 30%
regressions
Conclusions
• Sentence processing difficulty is only variably
mirrored in the ET record
– Doesn’t always show up in FPT
• One clear response to difficulty: a regression
– But only happens 5-30% of time
• A common response to difficulty: fuggettabottit
– Just get the words, `mam
I’d like to be shown wrong, but…
• ET response to comprehension difficulty is
variable: linger, regress, nothing
• Effects seen in means may actually occur on
only a minority of trials
• Eye movements may be controlled by word
recognition, and only exceptionally affected
by higher-level factors
Put on those rose-colored glasses
• Maybe all these problems will disappear if
people read texts they really care about, not
just single sentences
• But I wouldn’t bet on it
Stevenson & Merlo, 1997
• The troops marched across the fields all
day resented the general.
– Manner of movement verbs hard in reduced
relatives; transitive use of these verbs requires
an operation of “syntactic causativisation”
• But…The witch melted in the Wizard of Oz
was played by a famous actress.
– Seems easy. Suggestion: Unaccusative verbs
OK in reduced relatives
Eyetracking Experiment
• Sentences with Manner of Movement
(MOM) verbs
• Sentences with verbs whose thematic grid
changes (agent-theme to goal-theme) with
passivization
• Sentences with unaccusative verbs (theme
in both active and passive subject position)
MOM (manner of movement)
sentences
• The horse raced along the beach frightened the
onlookers
• The craft sailed to the island served as a makeshift
ferry
• Controls
– The horse that was raced to to the beach frightened the
onlookers. (full relative clause)
– The horse raced along the beach and frightened the
onlookers. (main clause)
– Disambiguating region indicated in boldface
Thematic Grid Change Sentences
• The secretary brought the forms filled them
out.
• The client asked the question received no
thanks.
– Main clause structure: Agent – verb – theme
– Relative clause structure: Goal – verb – theme
– Control: The client asked the question but
received no thanks
Unaccusative Verbs
• The leaves burned today smoked a lot.
• The potatoes baked in the oven smelled
terrific.
– Main clause structure: theme – verb
– Relative clause structure: theme – verb
– Control: The potatoes baked in the oven and
smelled terrific.
Predictions
• Manner of movement sentences: Hard, on
everybody’s story
• Thematic grid change sentences:
Comparably hard, if thematic grid, not
causal structure, the root
• Unaccusative sentences: Relatively easy, if
thematic grid change not syntactic structure
change the underlying problem
First Pass Times
First Pass Time, Disambig Reg,ms
Legend
600
RedRel
FullRel
MainCl
500
400
0
MOM
Thematic Unaccusative
Change
Regression Path Durations
Regression Path Duration, ms
Disambiguating Region
Legend
1400
RedRel
FullRel
MainCl
1200
1000
800
600
400
0
MOM ThemeChg Unaccus
Verb Type
Comprehension Accuracy
• Questions were asked following 2/3 of the items (half of these were
relevant to the RC)
– MOM questions: The boy marched to school looked unhappy.
• How did the boy get to school?
Someone marched him there/He marched there on his own
– Theta-grid changing questions: The client asked the question
received no thanks.
• Who asked the question?
The client / someone
– Unaccusative questions: The potatoes baked in the oven smelled
terrific
• What was happening with the potatoes?
Someone had baked them in the oven / They were sitting in the oven,
baking.
Proportion Correct
1.0
Question-answering Accuracy
RedRel
0.8
FullRel
MainCl
0.6
0.4
0.2
0.0
MOM
ThemeChg
Verb Type
Note: accuracy for Unaccusative verb sentences is not
shown. The questions were bad – only 67% correct for
main clause items.
Conclusions
• Intuitions of sentence processing difficulty
not transparently mirrored in the ET record
• Typical response to difficulty: a regression
– May be specific to this experiment; lots of hard,
tricky RC sentences
– But even here, only about 1/3 of the trials
• Another response to difficulty: fughetabottit
– Just get the words, `mam
For the future: I’d like to be
shown wrong, but…
• ET response to comprehension difficulty seems to
be variable: linger, regress, nothing
• Eye movements may generally be controlled by
word recognition, and only exceptionally affected
by higher-level factors
• Effects seen in means may actually occur on only
a minority of trials
• Maybe all these problems will disappear if people
read texts they really care about, not just single
sentences
• But I wouldn’t bet on it
Regression Path Duration, ms., Outside Region
Disambiguating Region, RPD Outside
120
100
80
RedRel
FullRel
60
40
20
0
Animate Inanimate Animate Inanimate
Previe w
No Previe w
Second Pass Time, ms., from Left
Disambiguating Region, Second Pass from Left
120
100
RedRel
FullRel
80
60
40
20
0
Animate Inanimate Animate Inanimate
Preview
No Preview
MacDonald, Pearlmutter,
Seidenberg, 1994
• In all cases, the examples cited here were
not the only reduced relatives in these
articles.
• The pattern of results obtained with global
reading times is reflected by differences
located on the critical disambiguating
phrase.
Ferreira & Clifton 1986 – An early
attempt to find easy RRCs
• Ambiguous animate: The defendant
examined by the lawyer was unreliable
• Ambiguous inanimate: The evidence
examined by the lawyer was unreliable
• Controls: The defendant/evidence who/that
was examined by the lawyer was unreliable.
– Disambiguating region indicated in boldface