Transcript PSY 369: Psycholinguistics - the Department of Psychology at

PSY 369: Psycholinguistics
Language Comprehension:
The role of memory
He sent a letter about it to Galileo, the great Italian
scientist.

Center embedded structures


The house burned down.
The house the handyman painted burned down.
The house the handyman the teacher hired painted
burned down.

Center
embedded
structures
This sentence
is syntactically
legal.






So
is it soburned
hard to comprehend?
Thewhy
house
down.
One possibility is that there are limitations as a result of our
The
house the handyman painted burned down.
cognitive systems
Today’s topic: the role of memory in sentence comprehension
Memory and comprehension

Center embedded structures

The house burned down.
Memory and comprehension

Center embedded structures


The house burned down.
The house the handyman painted burned down.
Memory and comprehension
This one may be legal, but

Center embedded structures



that doesn’t mean that it is
(easily) comprehensible
The house burned down.
The house the handyman painted burned down.
The house the handyman the teacher hired painted
burned down.

(the handyman that the teacher hired painted the house that burned
down)
Memory and comprehension

Brief summary so far:

What is the role of syntax in comprehension?

Syntax is important for getting the right
interpretation during on-line comprehension
 There is a lot of research examining what factors
influence the on-line construction of syntax

e.g., parsing principles, lexical semantics,
plausibility, discourse context, intonnational
information

Today: What is the role of Memory in language
comprehension?
Memory and comprehension
The man that the woman that the child hugged kissed laughed.
S

NP
RR
NP
C
VP

NP
VP
V
V
laughed
The man
that
RR
NP
the woman
NP
C
kissed
VP
NP
that
V
The child
hugged
Typically we build right
branching structures
But here there is a big
series of branches
down the center
Memory and comprehension
The man that the woman that the child hugged kissed laughed.


Most readers having trouble figuring out who did what to
whom (called thematic role assignment).
 Easier to assign thematic roles in the two sentences that
form it:
 The man that the woman kissed laughed.
 The woman that the child hugged kissed the man.
Possible explanation for the trouble:
 Insufficient working memory resources to retain the
intermediate products of computation made building the
complex syntactic structure
Measuring memory span

Daneman and Carpenter (1980) Technique:



This technique involves presenting sequences of 2 to 6
sentences, each of 12 to 17 words.
The participant has to read the sentences out loud, and
attempt to remember the last word of each.
Then asked to recall as many last words as possible (in
any order).
Measuring memory span
When at last his eyes opened, there was no gleam of triumph, no shade of anger.
Measuring memory span
The taxi turned up Michigan Avenue where they had a clear view of the lake.
Measuring memory span

Recall the last words
When at last his eyes opened, there was no gleam of triumph, no shade of anger.
The taxi turned up Michigan Avenue where they had a clear view of the lake.
Measuring memory span
I turned my memories over at random like pictures in a photograph album.
Measuring memory span
I will not shock my readers by describing the cold-blooded butchery that followed.
Measuring memory span
He had an odd elongated skull which sat on his shoulder like a pear on a dish.
Measuring memory span
You can check out the books that you need for this course at the local library.
Measuring memory span
The radio station was promoting the concert with free tickets and back stage passes.
Measuring memory span
The professor could be seen on weekends in the backyard garden pulling out weeds.
Measuring memory span

Recall the last words
I turned my memories over at random like pictures in a photograph album.
I will not shock my readers by describing the cold-blooded butchery that followed.
He had an odd elongated skull which sat on his shoulder like a pear on a dish.
You can check out the books that you need for this course at the local library.
The radio station was promoting the concert with free tickets and back stage passes.
The professor could be seen on weekends in the backyard garden pulling out weeds.

Ok for two sentences; Hard at 3 sentences; Very hard
for 4 or more.

Used to classify readers as high and low span
Memory and online comprehension
The Capacity Theory of Comprehension

(Just & Carpenter, 1992)

Proposed that individual differences in working memory
capacity should influence how readers comprehend
sentences
Memory and online comprehension
The Capacity Theory of Comprehension

(Just & Carpenter, 1992)
Proposed that individual differences in working memory
capacity should influence how readers comprehend
sentences

The animacy of the first noun may constrain the possible
interpretation of the sentence
Semantically Unconstrained:
The defendant examined by the lawyer shocked the jury.
The defendant that was examined by the lawyer shocked the jury.
Semantically Constrained (so should be faster if animacy can be used)
The evidence examined by the lawyer shocked the jury.
The evidence that was examined by the lawyer shocked the jury.
“that was” disambiguates these sentences

Memory and online comprehension
Just & Carpenter (1992)
600
The defendant examined by the
lawyer shocked the jury.
550
500
msec
Just the ambiguous sentences
animate NP
Inanimate NP
450
400
The evidence examined by the
lawyer shocked the jury.
350
300
Low-span
High-span
High span readers could use the semantic information
to resolve the ambiguity
Memory and online comprehension
King and Just (1991)

Verbs which could provide strong pragmatic cues as to which of
the two potential actors in the sentence was the agent:
Embedded clause
The robber that the fireman rescued stole the jewelry.


Two possible agents:


the robber
the fireman

Two verbs, which is the
main verb of the
sentence?:


rescued
stole
Memory and online comprehension
King and Just (1991)

Verbs which could provide strong pragmatic cues as to which of
the two potential actors in the sentence was the agent:
Strong bias  .The robber that the fireman rescued stole the jewelry.


No bias



The robber that the fireman rescued watched the program.
The robber that the fireman detested stole the jewelry.
The robber that the fireman detested watched the program.
Can bias which Noun goes with which Verb pragmatically (or not)
Questions:
 can speakers use this information?
 Does memory have an impact?
Memory and online comprehension
King and Just (1991)

Verbs which could provide strong pragmatic cues as to which of
the two potential actors in the sentence was the agent:
embedded verb Main verb




The robber that the fireman rescued stole the jewelry.
The robber that the fireman rescued watched the program.
The robber that the fireman detested stole the jewelry.
The robber that the fireman detested watched the program.
Memory and online comprehension
King and Just (1991)

Verbs which could provide strong pragmatic cues as to which of
the two potential actors in the sentence was the agent:
Results



Comprehension accuracy

90
H
80 L
70
H
L
H
H
L
L
60
both
emmV main
V
neither
Low-capacity subjects had lower comprehension overall
High-capacity subjects did NOT improve with pragmatic info
Low-capacity subjects did improve with pragmatic info
Memory and online comprehension
Garnsey, Pearlmutter, Pirog (2003)
The professor (who was) confronted by the student was
not ready for an argument.
The professor (had) confronted the student but was
not ready for an argument.
Question:
Do readers differ specifically in how quickly they can use
disambiguating words to rule out incorrect alternatives?
Memory and online comprehension
Garnsey, Pearlmutter, Pirog (2003)
Eye fixations were analyzed separately
- By whether preview of “by” while still fixating on verb likely
If last fix was here,
trial not used
The professor confronted by the student was not ready to …
If last fix was here,
trial coded as
Preview Unlikely
If last fix was here,
trial coded as
Preview Likely
Memory and online comprehension
Readers who score high on the Reading Span test
- Make better use of a peripherally visible disambiguating word to
quickly rule out a preferred but incorrect interpretation
Memory and online comprehension




Just & Carpenter (1992) - high span readers used
semantic information early, but low span readers didn’t
King & Just (1991) - high span readers did not use
pragmatic information to resolve ambiguity, but low span
readers did
Garnsey, Pearlmutter, Pirog (2003) - span differences may
also depend on where the eye lands (which determines
what kind of preview readers get)
What information is used to resolve syntactic
ambiguities depends on individuals working memory
capacity (but see Walters and Caplan (1996) for alternative
view)
Memory and comprehension

Brief summary so far:

What is the role of syntax in comprehension?

Syntax is important for getting the right
interpretation during on-line comprehension
 Memory capacity may play an important role in
determining what kinds of information we can use
to comprehend sentences

What about memory for language over the longer
term? What do we remember about sentences?
Memory for sentences
Fillenbaum (1966)
 Given:


The window is not closed
Conclusions: Meaning gets
preserved, surface structure
(and syntax) forgotten
Tested:




The window is not closed
The window is closed
<-- surface similar, meaning different
The window is not open <-- surface similar, meaning different
The window is open
<-- surface different, surface different
Most common error
Memory for sentences
Think back to the beginning of class. Which of the following
sentences did you read?

Galileo, the great Italian scientist, sent him a letter about it.

He sent Galileo, the great Italian scientist, a letter about it.

He sent a letter about it to Galileo, the great Italian scientist.

A letter about it was sent to Galileo, the great Italian scientist.
Memory for sentences
Sachs (1967, 1974)
 Heard (read):


Tested:





“He sent a letter about it to Galileo, the great Italian scientist.”
Same: He sent a letter about it to Galileo, the great Italian scientist.
Act/Pass: A letter about it was sent to Galileo, the great Italian
scientist.
Formal: He sent Galileo, the great Italian scientist, a letter about it.
Meaning: Galileo, the great Italian scientist, sent him a letter about
it.
Measured accuracy of detecting changes
Memory for sentences
Percent correct
Sachs (1967)
semantic
change
active/passive
change
formal change
100
90
80
70
60
50
identical
sentence
0
80
160
Amount of interpolated material
(number of syllables)
Conclusions: Meaning gets preserved, surface structure
(and syntax) forgotten
Just good enough representations

Ferreira and colleagues (Christianson et al 2001)

Garden-path sentence



While Anna dressed the baby played in the crib
While Anna dressed, the baby played in the crib
Did the baby play in the crib?
Did Anna dress the baby?
100% correct
40% correct
Comprehenders don’t always get all of the meaning
right, but get enough to get by
Memory and comprehension

Brief summary so far:

What is the role of syntax in comprehension?

Syntax is important for getting the right
interpretation during on-line comprehension
 Memory capacity may play an important role in
determining what kinds of information we can use
to comprehend sentences

What about memory for language over the longer
term? What do we remember about sentences?

Syntax may not be too important later, we
remember the meaning of sentences but not so
much the form (syntax) of the sentence
Propositions

Good memory for meaning but not for form

How do we represent sentence meaning?

Propositions

Two or more concepts with a relationship between them
A mouse bit a cat
bit (mouse, cat)
Propositions

Good memory for meaning but not for form

How do we represent sentence meaning?

Propositions

Two or more concepts with a relationship between them
A mouse bit a cat
bit (mouse, cat)

Can represent this within a network framework
Meaning as Propositions

Propositions

A mouse bit a cat
or

A cat was bitten by a
mouse
patient
relation

mouse
agent
A set of conceptual
nodes connected by
labeled pathways
that expresses the
meaning of a
sentence
bit
cat
Deriving Propositions

More complex example:

Children who are slow eat bread that is cold



Slow children
Children eat bread
Bread is cold
Past
Slow
Children
Eat
Bread
Cold
Evidence for Propositions


“The crowded passengers squirmed uncomfortably”




Memory better for sentences with fewer
propositions
passengers crowded
passengers squirmed
passengers uncomfortable
Three propositions
“The horse stumbled and broke a leg”
 horse stumbled
Two propositions
 horse broke leg
Evidence for Propositions

Bransford & Franks, 1971

Constructed four-fact sentences, and broke
them down into smaller sentences:




4 - The ants in the kitchen ate the sweet jelly that was
on the table.
3 - The ants in the kitchen ate the sweet jelly
2 - The ants in the kitchen ate the jelly.
1 - The jelly was sweet.
Evidence for Propositions

Bransford & Franks, 1971

Study: Heard 1-, 2-, and 3-fact sentences only

Test: Heard 1-, 2-, 3-, 4-fact sentences
(most of which were never presented)
Evidence for Propositions

Bransford & Franks, 1971

Results:



the more facts in the sentences, the more likely Ss
would judge them as “old” and with higher confidence
Even if they hadn’t actually seen the sentence
Constructive Model: we integrate info from
individual sentences in order to construct larger
ideas

emphasizes the active nature of our cognitive processes
Priming Propositions

Ratcliff and McKoon (1978)
“The mausoleum that enshrined the tsar overlooked the square.”

Involves two propositions:


P1 [OVERLOOK, MAUSOLEUM, SQUARE]
P2 [ENSHRINE, MAUSOLEUM, TSAR].
Priming Propositions

Ratcliff and McKoon (1978)

Results in a cued memory task (how long does it
take to verify “square” was in the sentence:
Condition
Examples
RT to Target
Priming Effects
Across sentences
squareclutch
square-Tsar
671 msec
None; baseline
571 msec
100 msec facilitation
squaremausoleum
551 msec
120 msed facilitation
Between two propositions in
the same sentence
Within a single proposition
Alternative Representations

Propositions are symbolic

Problems:





The referential problem
The implementation problem
The lack of scientific productivity
The lack of a biological foundation
Alternative

Embodied representations (e.g., Barsalou;
1999; Glenberg, 1999)
Embodiment in language

Embodied representations


Perceptual and motor systems play a central role in
language production and comprehension
Theoretical proposals




Linguistics: Lakoff, Langacker, Talmy
Neuroscience: Damasio, Edelman
Cognitive psychology: Barsalou, Gibbs, Glenberg,
MacWhinney
Computer science: Steels, Feldman
Embodiment in language

Embodied representations

Perceptual and motor systems play a central role in
language production and comprehension


Words and sentences are usually grounded to
perceptual, motoric, and emotional experiences.
In absence of inmediate sensory-motor referents, words
and sentences refer to mental models or simulations of
experience.
Embodiment in language

Embodied representations

Brain activity



Comprehension and images
Concrete words
Action words activate motor representations
Simulation hypothesis
We understand utterances by mentally simulating
their content.


Simulation exploits some of the
same neural structures activated during performance,
perception, imagining, memory…
Language gives us enough information to simulate
Inference in comprehension

Not all propositions come from the bottom-up

Elaboration - integration of new information with
information from long term memory


Memory for the new information improves as it is
integrated
Inferences - a proposition (or other representation)
drawn by the comprehender

From LTM, not directly from the input
Inference in comprehension

Bransford, and colleagues (1972, 73)


We draw inferences in the course of
understanding new events.
The inferences get encoded into our memory
of the events.

e.g., drawing inferences of instruments
Inference in comprehension

Bransford, and colleagues (1972, 73)
John was trying to fix the birdhouse. He was looking for the nail when
his father came out to watch him and to help him do the work.
John was using the hammer to fix the birdhouse when his father came
out to watch him and to help him do the work.
Inference in comprehension

Bransford, and colleagues (1972, 73)
John was trying to fix the birdhouse. He was looking for the nail when
his father came out to watch him and to help him do the work.
John was using the hammer to fix the birdhouse when his father came
out to watch him and to help him do the work.
was not mentioned in the
text, but was inferred
What does language do?
A sentence can evoke an imagined scene and resulting inferences:
“Harry walked to the cafe.”
CAFE
– Goal of action = at cafe
– Source = away from cafe
– cafe = point-like location
“Harry walked into the cafe.”
CAFE
– Goal of action = inside cafe
– Source = outside cafe
– cafe = containing location
Embodied inferences
WALL
The scientist walked into the wall.
The hobo drifted into the house.
The smoke drifted into the house.
Bonk!!
Summing up



The results of sentence comprehension are meaning
representations
Some debate over what these representations are
Whatever they are, they get integrated with existing knowledge
from LTM
Discourse Psycholinguistics

Traditional Psycholinguistics


Determining what happens when we understand
sentences
Broader View

How we resolve/understand sentences against the
current discourse representation

Sentence comprehension is a process that anchors the
interpretation of the sentence to the representation of the
prior text
Processing Discourse

What is discourse?

Units of analysis larger than a sentence


Applies to both spoken and written forms
Ways we process (i.e., comprehend and remember) units
of language larger than a sentence



lectures
personal narratives
expository discourse
Bill and Ted traveled through time and space.
Bill asked, “Dude, did you find him?”
“Yeah, Bach was in the music store, but I haven’t found Abe
yet. Dude, you should hear him play, he rocks!”
“Excellent! Man, we’ve got to get these dudes back to
school before we get there.”
Bill and Ted traveled through time and space.
Bill asked, “Dude, did you find him?”
“Yeah, Bach was in the music store, but I haven’t found Abe
yet. Dude, you should hear him play, he rocks!”
“Excellent! Man, we’ve got to get these dudes back to
school before we get there.”

To whom does “him” refer to?
Bill and Ted traveled through time and space.
Bill asked, “Dude, did you find him?”
“Yeah, Bach was in the music store, but I haven’t found Abe
yet. Dude, you should hear him play, he rocks!”
“Excellent! Man, we’ve got to get these dudes back to
school before we get there.”

To whom does “him” refer?
Bach
Bill and Ted traveled through time and space.
Bill asked, “Dude, did you find him?”
“Yeah, Bach was in the music store, but I haven’t found Abe
yet. Dude, you should hear him play, he rocks!”
“Excellent! Man, we’ve got to get these dudes back to
school before we get there.”

To whom does this “him” refer?
Bill and Ted traveled through time and space.
Bill asked, “Dude, did you find him?”
“Yeah, Bach was in the music store, but I haven’t found Abe
yet. Dude, you should hear him play, he rocks!”
“Excellent! Man, we’ve got to get these dudes back to
school before we get there.”

To whom does this “him” refer?
Bach again
Bill and Ted traveled through time and space.
Bill asked, “Dude, did you find him?”
“Yeah, Bach was in the music store, but I haven’t found Abe
yet. Dude, you should hear him play, he rocks!”
“Excellent! Man, we’ve got to get these dudes back to
school before we get there.”

To whom does this “him” refer?
Bach again
Why not Abe?
Bill and Ted traveled through time and space.
Bill asked, “Dude, did you find him?”
“Yeah, Bach was in the music store, but I haven’t found Abe
yet. Dude, you should hear him play, he rocks!”
“Excellent! Man, we’ve got to get these dudes back to
school before we get there.”

Huh!?
Bill and Ted traveled through time and space.
Bill asked, “Dude, did you find him?”
“Yeah, Bach was in the music store, but I haven’t found Abe
yet. Dude, you should hear him play, he rocks!”
“Excellent! Man, we’ve got to get these dudes back to
school before we get there.”

Huh!?
Oh yeah, they’re time travelers.
Characteristics of Discourse

Local Structure (microstructure):


The relationship between individual sentences
Global Structure (macrostructure):

The relationship between the sentences and our
knowledge of the world
Characteristics of Discourse

Local Structure:

Coherence:



Does the passage make sense
Logical consistency and semantic continuity
Cohesion:


Does the discourse “stick together”
Interpretation of one sentence depends on other
sentences
Characteristics of Discourse

Coherence: does it make sense?

Incoherence


When the meanings of individual sentences do not
hang together
Given/new distinction

Readers expect speakers to cues as to what
information is old (already known by the listener) and
what is new (not known)
Developing coherence
Haviland and Clark (1974)

Process of understanding a sentence in discourse
context involves 3 stages:



identifying the given and new info in the current sentence
finding an antecedent in memory for the given
information
attaching the new information to this spot in memory
Developing coherence
Haviland and Clark (1974)


Herb unpacked some beer.
The beer was warm.
Developing coherence
Haviland and Clark (1974)


Herb unpacked some beer.
The beer was warm.
Definite article “the” signals that “the beer” is
given information
Developing coherence
Haviland and Clark (1974)


Herb unpacked some beer.
The beer was warm.
Definite article “the” signals that “the beer” is
given information
Connect the new information “was warm” to
the appropriate discourse concept
Developing coherence
Haviland and Clark (1974)


Herb unpacked some beer.
The beer was warm.
Definite article “the” signals that “the beer” is
given information
Connect the new information “was warm” to
the appropriate discourse concept “some beer”
This process is called Direct Matching
Developing coherence
Haviland and Clark (1974)


Herb unpacked some picnic supplies.
The beer was warm.
Definite article “the” signals that “the beer” is
given information
So connect the new information “was warm”
to the appropriate discourse concept
Need a bridging inference to connect “the
warm beer” to “some picnic supplies”
Developing coherence

Direct Matching
“some beer”

“the beer”
Bridging Inference
“some picnic supplies”
“the beer”
World knowledge
Developing coherence
Singer, Halldorson, Lear, & Andrusiak (1992)
Requires
inference
“Murray poured water on the fire.”
“The fire went out.”
No
Required
inference
T/F “water extinguishes fire”
“Murray drank a glass of water.”
“He watched the fire go out.”
T/F “water extinguishes fire”
Suggests that the inference was made
Faster
Characteristics of Discourse

Cohesion: Interpretation of one sentence
depends on other sentences

Referential Cohesion


Substitution Cohesion


“We’ve got to get these dudes back to …”
And many more


“Dude, you should hear him play…”
See pg 155 of textbook for table of other categories of
cohesion
The relationship between the referring expression and the
antecedent create referential cohesion of discourse
Types of Referential Cohesion

Anaphoric Reference

Using an expression to refer back to something previously
mentioned in discourse
“…Bach was in the music store …”
“Dude, you should hear him play, he rocks.”

Cataphoric Reference

Using an expression to refer forward to something that is
coming up in discourse
Dude, did you find him?”
“Yeah, Bach was in the music store...”
Comprehending Anaphoric References
Daneman and Carpenter (1980)

Reading Span Test


Smaller reading spans = smaller working memory capacity
Comprehension task

Reading a passage and answer questions about the referents of
pronouns
Sitting with Richie, Archie, Walter and the rest of my gang in the Grill yesterday, I began to
feel uneasy. Robbie had put a dime in the juke box. It was blaring one of the latest
“Rock and Roll” favorites. I was studying, in horror, the reactions of my friends to the
music. I was especially perturbed by the expression on my best friend’s face. Wayne
looked intense and was pounding the table furiously to the beat. Now, I like most of
the things other teenage boys like. I like girls with soft blonde hair, girls with dark curly
hair, in fact all girls. I like milkshakes, football games and beach parties. I like denim
jeans, fancy T-shirs and sneakers. It is not that I dislike rock music but I think it is
supposed to be fun and not taken too seriously. And here he was, “all shook up” and
serious over the crazy music.
Question: Who was “all shook up” and serious over the music?
Comprehending Anaphoric References
Daneman and Carpenter (1980)

Reading Span Test


Smaller reading spans = smaller working memory capacity
Comprehension task

Reading a passage and answer questions about the referents of
pronouns
Sitting with Richie, Archie, Walter and the rest of my gang in the Grill yesterday, I began to
feel uneasy. Robbie had put a dime in the juke box. It was blaring one of the latest
“Rock and Roll” favorites. I was studying, in horror, the reactions of my friends to the
music. I was especially perturbed by the expression on my best friend’s face. Wayne
looked intense and was pounding the table furiously to the beat. Now, I like most of
the things other teenage boys like. I like girls with soft blonde hair, girls with dark curly
hair, in fact all girls. I like milkshakes, football games and beach parties. I like denim
jeans, fancy T-shirs and sneakers. It is not that I dislike rock music but I think it is
supposed to be fun and not taken too seriously. And here he was, “all shook up” and
serious over the crazy music.
Question: Who was “all shook up” and serious over the music?
Comprehending Anaphoric References
Daneman and Carpenter (1980)

Reading Span Test


Smaller reading spans = smaller working memory capacity
Comprehension task

Reading a passage and answer questions about the referents of
pronouns
Sitting with Richie, Archie, Walter and the rest of my gang in the Grill yesterday, I began to
feel uneasy. Robbie had put a dime in the juke box. It was blaring one of the latest
“Rock and Roll” favorites. I was studying, in horror, the reactions of my friends to the
music. I was especially perturbed by the expression on my best friend’s face. Wayne
looked intense and was pounding the table furiously to the beat. Now, I like most of
the things other teenage boys like. I like girls with soft blonde hair, girls with dark curly
hair, in fact all girls. I like milkshakes, football games and beach parties. I like denim
jeans, fancy T-shirs and sneakers. It is not that I dislike rock music but I think it is
supposed to be fun and not taken too seriously. And here he was, “all shook up” and
serious over the crazy music.
Question: Who was “all shook up” and serious over the music?
Comprehending Anaphoric References
Daneman and Carpenter (1980)
Manipulated how many sentences intervened between the
pronoun and the antecedent
Percentage correct

120
100
80
60
40
20
0
small (2-3)
High span
medium span
Low span
medium
(4-5)
Large (6-7)
Number of sentences
between pronoun and
antecedent