Transcript phillips_japan_1 - Department of Linguistics

Language Mind and Brain:
The Unification Problem
Colin Phillips
Cognitive Neuroscience of Language Laboratory
Department of Linguistics
University of Maryland
Unification Problem
• Objective is to bridge gap between linguistic models, realtime models of mental processes, and brain-level models,
i.e., focus is on the Unification Problem
• Cognitive Neuroscience of Language necessarily draws on
disparate areas - not just looking at pictures of brains!
Unification Problem
??
Unification Problem
• No longer concerned with questions of whether linguistics
is a natural science
• Language is clearly a remarkable specialization of human
neurobiology (e.g., a sophisticated symbolic, recursive
system, with fixed and parameterized aspects)
• What is it about human psychology & neurobiology that
allows it to support the things that we know about
language?
• In addressing this question, are we faced with problems or
with mysteries?
Unification Problem
• ...And if there are mismatches between theories at different
levels, whose problem is this?
Linguists?
Psychologists?
Neuroscientists?
Unification Problem
On mismatches between cognitive and neural models
“The more we learn about the brain,
the greater the disanalogy becomes.”
(A philosophy of (neuro-)science talk, October 2001)
Unification Problem
On mismatches between cognitive and neural models
“If language is unlike anything else in the biological world,
…then too bad for biology!”
(linguist, often accompanied by story about chemistry & physics)
Encoding & Computation
• Two main issues
– How are linguistic representations encoded?
– How are linguistic representations computed?
Sensory Maps
Internal representations of
the outside world. Cellular
neuroscience has discovered
a great deal in this area.
Vowel Space
Notions of sensory maps may be
applicable to human phonetic
representations…
…although attempts to find
them have had little success to
date.
Encoding of Symbols: Abstraction
• But most areas of linguistics (phonology, morphology,
syntax, semantics) are concerned with symbolic, abstract
representations,
...which do not involve internal representations of
dimensions of the outside world.
…hence, the notion of sensory maps does not get us very
far into language
Computation: Discrete Infinity
• In neuroscience there are many findings about long-term
storage of object representations (e.g., edges, faces,
grandmothers, toothbrushes, …).
• Such representations are always finite in number, which
can be retrieved from long-term memory
• BUT, much of what interests us in linguistics is infinite
• If representations are drawn from an infinite set, they
cannot be retrieved from long-term memory; they must be
constructed on-line, as needed - this poses a different kind
of challenge …
Overview of Talks
Overview of Talks
1. The Unification Problem
Overview of Talks
1. The Unification Problem
2. Building Syntactic Relations
In-situ
どの生徒に…
1100
Reading Time
1000
900
DeclC
QP
800
700
600
1
2
3
4
5
Region
6
7
8
Overview of Talks
1. The Unification Problem
3. Abstraction: Sounds to Symbols
2. Building Syntactic Relations
In-situ
どの生徒に…
1100
Reading Time
1000
900
DeclC
QP
800
700
600
1
2
3
4
5
Region
6
7
8
Overview of Talks
1. The Unification Problem
3. Abstraction: Sounds to Symbols
2. Building Syntactic Relations
4. Linguistics and Learning
In-situ
どの生徒に…
1100
Reading Time
1000
900
DeclC
QP
800
700
600
1
2
3
4
5
Region
6
7
8
with help from ...
University of Maryland
University of Delaware
Shani Abada
Sachiko Aoshima
Daniel Garcia-Pedrosa
Ana Gouvea
Nina Kazanina
Moti Lieberman
Leticia Pablos
David Poeppel
Beth Rabbin
Silke Urban
Carol Whitney
Evniki Edgar
Bowen Hui
Baris Kabak
Tom Pellathy
Dave Schneider
Kaia Wong
Alec Marantz, MIT
Elron Yellin, MIT
National Science Foundation
James S. McDonnell Foundation
Human Frontiers Science Program
Japan Science & Technology Program
Kanazawa Institute of Technology
Outline
•
•
•
•
•
•
•
The Challenge
Real-time Grammar
Accurate Parsing
Incremental Parsing
Mapping onto the Brain: Electrophysiology
Encoding
Outlook
Outline
•
•
•
•
•
•
•
The Challenge
Real-time Grammar
Accurate Parsing
Incremental Parsing
Mapping onto the Brain: Electrophysiology
Encoding
Outlook
Discrete Infinity
• Linguistic Creativity
Ability to make infinite use of finite means
• The finite and infinite aspects of this system
present rather different challenges for explicit
linking hypotheses
Discrete Infinity
• Linguistic Creativity
Ability to make infinite use of finite means
• Lexical entries, Argument Structure templates
• Widespread assumption: same representations
accessed in comprehension, production,
acceptability ratings, etc.
• Learner’s task is to construct a single lexical entry
that covers all of these tasks
Discrete Infinity
• Linguistic Creativity
Ability to make infinite use of finite means
• Sentence structures
• Structures must be assembled, cannot simply be
retrieved from memory
• Widespread assumption: multiple different systems
responsible for structure assembly in comprehension,
production, acceptability, etc.
• Learner must master a number of different systems
Discrete Infinity
• Linguistic Creativity
Ability to make infinite use of finite means
• Time-independent vs. Time-dependent systems
Standard View
324
697+
?
arithmetic
217 x 32 = ?
Standard View
specialized algorithm
324
697+
?
arithmetic
specialized algorithm
217 x 32 = ?
Standard View
specialized algorithm
specialized algorithm
324
697+
?
arithmetic
217 x 32 = ?
?
something deeper
Standard View
specialized algorithm
speaking
language
specialized algorithm
understanding
grammatical
knowledge,
competence
Standard View
specialized algorithm
speaking
language
specialized algorithm
understanding
grammatical
knowledge,
competence
precise
but ill-adapted to
real-time operation
Standard View
specialized algorithm
speaking
language
specialized algorithm
understanding
grammatical
knowledge,
competence
well-adapted to
real-time operation
but maybe inaccurate
If speaking and understanding involve different
systems, there must be an additional store of
knowledge that encodes what is shared between
speaking and understanding.
As soon as we assume constructs such as ‘parsing
strategies’, we are adopting task-specific
mechanisms, and endorsing something like the
standard architecture.
Standard View
specialized algorithm
speaking
language
specialized algorithm
understanding
grammatical
knowledge,
competence
Analysis-by-Synthesis
speaking,
understanding,
grammaticality
• Sentences are understood by internally generating a
representation that matches the input
• No separate time-independent system of knowledge
• We know that humans have a real-time system for
linguistic computation - issue is whether that’s all there is
But wait a minute...
• Wasn’t this all shown to be wrong long ago?
(Fodor, Bever & Garrett 1974; Levelt 1974; Fillenbaum 1971; etc.,
etc.)?
• And a recent commentary:
“In this desert of ignorance there have been attempts to resurrect
earlier claims that the grammar and the parser are one and the same
thing. … The enterprise is misconceived … probably incoherent.”
(Smith, 1999)
Motivation for Standard Architecture
•
•
•
•
How to constrain hypothesis generation
Grammars are not incremental real-time systems
Evidence for input/output-specific strategies
Analysis-by-synthesis implies active generation,
ahead of input
• Reputation of performance systems as fast but
inaccurate
• Debunking of ‘Derivational Theory of Complexity’
Motivation for Alternative Architecture
• Time-dependent system of computation makes it
feasible to generate testable linking hypotheses
Outline
•
•
•
•
•
•
•
The Challenge
Real-time Grammar
Accurate Parsing
Incremental Parsing
Mapping onto the Brain: Electrophysiology
Encoding
Outlook
Linear Order and Constituency
Linguistic Inquiry, 2003
Incremental Structure Building
• Evidence that sentence structures can only
be assembled in a left-to-right derivation.
S
NP
John
VP
V
said
S’
Comp
that
S
NP
he
VP
V
ate
NP
the entire pizza
S
NP
John
VP
V
said
S’
Comp
that
S
NP
he
VP
V
ate
Constituents
NP
the entire pizza
S
NP
John
VP
V
said
S’
Comp
that
S
NP
he
VP
V
ate
Constituents
NP
the entire pizza
S
NP
John
VP
V
said
S’
Comp
that
S
NP
he
VP
V
ate
Constituents
NP
the entire pizza
S
NP
John
VP
V
said
S’
Comp
that
S
NP
he
VP
V
ate
Constituents
NP
the entire pizza
• Many tools used to diagnose
groupings of words:
–
–
–
–
–
coordination
deletion
interpretation (coreference)
movement, focus, topicalization
etc.
• There are many cases where the tools
converge on the same result
• There are also many cases where the
tools yield conflicting results
Incremental Structure Building
A
(Phillips 2003)
Incremental Structure Building
A
B
(Phillips 2003)
Incremental Structure Building
A
B
C
(Phillips 2003)
Incremental Structure Building
A
B
C
D
(Phillips 2003)
Incremental Structure Building
A
B
C
D
E
(Phillips 2003)
Incremental Structure Building
A
B
(Phillips 2003)
Incremental Structure Building
A
B
constituent
(Phillips 2003)
Incremental Structure Building
A
B
C
constituent is destroyed by
addition of new material
(Phillips 2003)
Incremental Structure Building
A
B
C
(Phillips 2003)
Incremental Structure Building
A
B
C
constituent
(Phillips 2003)
Incremental Structure Building
A
B
C
D
constituent is destroyed by
addition of new material
(Phillips 2003)
Incremental Structure Building
the cat
(Phillips 2003)
Incremental Structure Building
the cat
sat
(Phillips 2003)
Incremental Structure Building
the cat
sat
on
(Phillips 2003)
Incremental Structure Building
the cat
sat
on
the rug
(Phillips 2003)
Incremental Structure Building
the cat
sat
on
(Phillips 2003)
Incremental Structure Building
the cat
sat
on
the rug
(Phillips 2003)
Incremental Structure Building
the cat
sat
on
the rug
[sat on] is a temporary
constituent, which is
destroyed as soon as the
NP [the rug] is added.
(Phillips 2003)
Incremental Structure Building
Conflicting Constituency Tests
Verb + Preposition sequences can undergo coordination…
(1) The cat sat on and slept under the rug.
…but cannot undergo pseudogapping (Baltin & Postal, 1996)
(2) *The cat sat on the rug and the dog did the chair.
(Phillips 2003)
Incremental Structure Building
the cat
sat
on
(Phillips 2003)
Incremental Structure Building
the cat
and
sat
on
slept
under
(Phillips 2003)
Incremental Structure Building
the cat
coordination applies
early, before the V+P
constituent is destroyed.
and
sat
on
slept
under
(Phillips 2003)
Incremental Structure Building
the cat
sat
on
(Phillips 2003)
Incremental Structure Building
the cat
sat
on
the rug
(Phillips 2003)
Incremental Structure Building
the cat
and the dog
did
sat
on
the rug
(Phillips 2003)
Incremental Structure Building
the cat
and the dog
did
sat
on
the rug
pseudogapping applies
too late, after the V+P
constituent is destroyed.
(Phillips 2003)
Incremental Structure Building
• Constituency Problem
Different diagnostics of constituency frequently yield
conflicting results
• Incrementality Hypothesis
(a) Syntactic processes see a ‘snapshot’ of a
derivation - they target constituents that are present
when the process applies
(b) Conflicts reflect the fact that different processes
have different linear properties
start
Ellipsis blocks Scope/Binding
Bill read all the books in a week (ambiguous: collective/distributive scope)
…and Sue did in a month (unambiguous: collective scope only)
Bill read as many books as Sue did in a week. (ambiguous)
Bill read as many books in a week as Sue did in a month. (unambiguous)
Ellipsis blocks Scope/Binding
John gave books to the children on Tuesday
…and Mary did on Thursday
John gave books to the children on each other’s birthdays
*…and Mary did on each other’s first day of school
Japanese
John
IP
IP
I’
John
+fin
VP
gave
I’
John
+fin
read
VP
books
V
VP
all the books
V
VP
to the children
V
VP
on each other’s birthdays
PP
in a week
IP
IP
I’
John
+fin
VP
gave
I’
John
+fin
read
VP
books
V
VP
all the books
V
VP
to the children
V
VP
on each other’s birthdays
PP
in a week
IP
and
IP
I’
John
+fin
I’
Bill
VP
gave
IP
did
VP
VP
books
V
VP
to the children
V
on each other’s birthdays
IP
and
IP
I’
John
+fin
I’
Bill
VP
gave
IP
did
VP
VP
books
V
VP
to the children
V
on each other’s birthdays
IP
I’
John
+fin
VP
gave
VP
books
V
to the children
IP
I’
John
VP
+fin
on each other’s birthdays
VP
gave
VP
books
V
to the children
IP
and
IP
I’
John
did
on each other’s birthdays
VP
gave
I’
Bill
VP
+fin
IP
VP
books
V
to the children
VP
IP
and
IP
I’
John
did
on each other’s birthdays
VP
gave
I’
Bill
VP
+fin
IP
VP
books
V
to the children
VP
IP
and
IP
I’
John
did
on each other’s birthdays
VP
gave
I’
Bill
VP
+fin
IP
VP
books
V
VP
gave
VP
books
V
to the children
to the children
IP
and
IP
I’
John
did
on each other’s birthdays
VP
gave
I’
Bill
VP
+fin
IP
books
V
to the children
on each other’s first day of school
VP
gave
VP
VP
VP
books
V
to the children
Movement & Binding
a.
John gave books to them on each other’s birthdays.
Movement & Binding
a.
John gave books to them on each other’s birthdays.
VP
gave
VP
books
V
VP
to them
V
(Pesetsky 1995)
on each other’s birthdays
Movement & Binding
a.
John gave books to them on each other’s birthdays.
VP
gave
VP
books
V
VP
to them
V
(Pesetsky 1995)
on each other’s birthdays
Movement & Binding
b. …and [give books to them] he did ___ on each other’s birthdays
(Pesetsky 1995)
VP
give
VP
books
V
to them
IP
VP
give
IP
VP
books
V
he
to them
did
IP
VP
give
IP
VP
books
V
he
to them
did
IP
VP
give
IP
VP
books
V
I’
he
did
to them
constituent
movement
VP
give
VP
books
V
to them
IP
VP
give
IP
VP
books
V
I’
he
did
to them
constituent
movement
VP
give
VP
books
V
VP
to them
V
on each other’s birthdays
IP
VP
give
IP
VP
books
V
I’
he
did
to them
constituent
movement
VP
give
VP
books
V
VP
to them
V
binding under
c-command
on each other’s birthdays
Interim Conclusion
•
By building syntactic structures from left-to-right we can explain a number of
otherwise mysterious constituency phenomena (see Phillips 1996, 2003 for
more examples; see Richards 1999, 2002 for some applications to Japanese)
•
We knew independently that humans have a left-to-right structure-building
system (i.e. parser, producer)
Possibility arises that the incremental left-to-right system is the only structurebuilding system that humans have
•
•
Other arguments leading to related conclusions about grammar, in widely
varying formalisms: Kempson et al. (2001), Steedman (2000), Kempen (1999),
Milward (1992, 1994)
Outline
•
•
•
•
•
•
•
The Challenge
Real-time Grammar
Accurate Parsing
Incremental Parsing
Mapping onto the Brain: Electrophysiology
Encoding
Outlook
Grammatical Accuracy
• It is not enough to show that syntactic structure-building
looks like a real-time process
• If the real-time system is the only system, then it should
also show the syntactic sophistication normally associated
with the grammar - the parser cannot be ‘dumb’
• Question: does the parser access only grammatically legal
structural analyses?
Beyond Ambiguity
• Much of parsing literature focuses on issues of ambiguity,
i.e. when 2 structures are grammatically possible, how to
choose the right one?
• More basic question: grammatical search
i.e. how do we figure out if a sequence of words has any
grammatical analyses?
Example: Argument Structure
Dative and Double-Object Constructions
Alternator Verb: give
The millionaire gave the painting to the museum.
The millionaire gave the museum the painting.
Non-alternator Verb: donate
The millionaire donated the painting to the museum.
*The millionaire donated the museum the painting.
(Phillips, Edgar & Kabak, 2000)
Example: Argument Structure
A Severe Garden-Path Sentence
Alternator Verb: give
The man gave the boy the dog bit a cookie
Example: Argument Structure
A Severe Garden-Path Sentence
Alternator Verb: give
The man gave [the boy [the dog bit]] a cookie
Example: Argument Structure
A Severe Garden-Path Sentence
Alternator Verb: give
The man gave [the boy [the dog bit]] a cookie
availability of double-object parse
leads to difficulty at embedded verb
Example: Argument Structure
A Severe Garden-Path Sentence
Alternator Verb: give
The man gave [the boy [the dog bit]] a cookie
availability of double-object parse
leads to difficulty at embedded verb
Non-Alternator Verb: donate
The man donated [the boy [the dog bit …
difficulty should arise
earlier in the sentence
Results (partial)
Alternators (e.g. give)
start
Non-Alternators (e.g. donate)
Relative to unambiguous control sentence, readers get into
difficulty at V with alternator verbs, at NP2 with non-alternators.
--> Argument structure constraint immediately active on-line
Example: Movement Constraints
Grammatical Accuracy in Parsing
Wh-Questions
Englishmen cook wonderful dinners.
Grammatical Accuracy in Parsing
Wh-Questions
Englishmen cook wonderful dinners.
Grammatical Accuracy in Parsing
Wh-Questions
Englishmen cook
what
Grammatical Accuracy in Parsing
Wh-Questions
Englishmen cook
what
Grammatical Accuracy in Parsing
Wh-Questions
What do
Englishmen cook
Grammatical Accuracy in Parsing
Wh-Questions
What do
Englishmen cook
gap
Grammatical Accuracy in Parsing
Wh-Questions

What do
Englishmen cook
gap
Grammatical Accuracy in Parsing
Long-distance Wh-Questions
Few people think that anybody realizes
that Englishmen cook wonderful dinners
Grammatical Accuracy in Parsing
Long-distance Wh-Questions
Few people think that anybody realizes
that Englishmen cook
what
Grammatical Accuracy in Parsing
Long-distance Wh-Questions
What do few people think that anybody realizes
that Englishmen cook
gap

Grammatical Accuracy in Parsing
The plan to remove the equipment ultimately destroyed the building.
Grammatical Accuracy in Parsing
The plan to remove the equipment ultimately destroyed the building.
Direct Object NP
Direct Object NP
Grammatical Accuracy in Parsing
The plan to remove the equipment ultimately destroyed the building.
Direct Object NP
Direct Object NP
Main Clause
Grammatical Accuracy in Parsing
Subject NP
The plan to remove the equipment ultimately destroyed the building.
Direct Object NP
Embedded Clause
Direct Object NP
Main Clause
Grammatical Accuracy in Parsing
What did the plan to remove the equipment ultimately destroy
Grammatical Accuracy in Parsing

What did the plan to remove the equipment ultimately destroy gap
Grammatical Accuracy in Parsing
What did the plan to remove
ultimately destroy the building
Grammatical Accuracy in Parsing

What did the plan to remove
gap
ultimately destroy the building
Grammatical Accuracy in Parsing
Subject

What did the plan to remove
gap
ultimately destroy the building
Island Constraint
A wh-phrase cannot be moved out of a subject.
Question…
• Do people respect island constraints on
movement immediately on-line?
(tomorrow’s talk)
Outline
•
•
•
•
•
•
•
The Challenge
Real-time Grammar
Accurate Parsing
Incremental Parsing
Mapping onto the Brain: Electrophysiology
Encoding
Outlook
Incrementality
• Question: is structure building immediate?
Does it operate on a word-by-word level?
…Even in a language where this may be hard?
Incremental Application of
Binding Constraints in Japanese
Sachiko Aoshima
Colin Phillips
Amy Weinberg
Structure-building in Japanese
• NP-wa NP-ni [NP-ga NP-o V] V
• Head-driven Parsing (e.g. Pritchett 1991, Mulders 2002)
Structure-building is delayed until verbs are processed
– explains how parsing is possible in Japanese
– accounts for flexibility, limited garden-paths in Japanese
• Incremental Parsing
Structure-building occurs immediately
– accounts for native-speaker intuition of continuous comprehension
– hard to demonstrate experimentally
John-ga …
(Mazuka & Itoh 1995)
John-ga Mary-ni …
(Mazuka & Itoh 1995)
John-ga Mary-ni ringo-o …
(Mazuka & Itoh 1995)
John-ga Mary-ni ringo-o tabeta …
(Mazuka & Itoh 1995)
John-ga Mary-ni ringo-o tabeta inu-o ageta
(Mazuka & Itoh 1995)
Verb Surprise Effects
• Surprise effect appears at verb
Can be explained by both head-driven and incremental
theories (Schneider 1999, Mulders 2002)
• Better: evidence of structure-building that precedes the
verb
e.g. immediate application of grammatical constraints
English
To which of his children did the man give a gift
(Aoshima, Phillips & Weinberg 2002)
English
To which of his children did the man give a gift
Which of his children gave the man a gift?
(Aoshima, Phillips & Weinberg 2002)
Japanese
which of his children (DAT) the man (NOM) …
which of his children (NOM) the man (DAT) …
(Aoshima, Phillips & Weinberg 2002)
Japanese pronoun and its
antecedent
which
which of
of his
his children
children (DAT)
(DAT) the man (NOM) …

which of his children (NOM) the man (DAT) …
*?
his
Gender Mismatch
which of his children (DAT) the man (NOM) …
the woman
which of his children (NOM) the man (DAT) …
the woman
Gender Mismatch
which
which of
of his
his children
children (DAT)
(DAT) the man (NOM) …

the woman
Gender mismatch
which of his children (NOM) the man (DAT) …
*?
the woman
Gender mismatch irrelevant
his
Conditions
a. Scrambled - Gender Mismatch
Adverb / [his / which NP]-dat / Adverb / NP FEMALE-nom / Adverb / NP-acc /
verb-Q / NPMALE-top / verb
b. Scrambled - Gender Match
Adverb / [his / which NP]-dat / Adverb / NP MALE-nom / Adverb / NP-acc /
verb-Q / NPFEMALE-top / verb
c. Non-scrambled - Gender Mismatch
Adverb / [his / which NP]-nom / Adverb / NP FEMALE-dat / Adverb / NP-acc /
verb-Q / NPMALE-top / verb
d. Non-scrambled - Gender Match
Adverb / [his / which NP]-nom / Adverb / NP MALE-dat / Adverb / NP-acc /
verb-Q / NPMALE-top / verb.
Examples
a. 台所で 彼の どの子供に 朝食後 叔母が 急いで お弁当を 渡したか
父親は 覚えていた。
b. 台所で 彼の どの子供に 朝食後 叔父が 急いで お弁当を 渡したか
叔母は 覚えていた。
c. 台所で 彼の どの子供が 朝食後 叔母に 急いで お弁当を 渡したか
父親は 覚えていた。
d. 台所で 彼の どの子供が 朝食後 叔父に 急いで お弁当を 渡したか
父親は 覚えていた。
Design & Procedure
• 2 X 2 factorial design
• 4 lists were created by distributing 24 items in a
Latin Square design
• 56 filler sentences
• Comprehension questions: matching a subject
with a predicate
• Self-paced reading task -Moving Window • 40 native speakers of Japanese
Self-paced reading task
----- --- --- ---- ---- --- ------ -------
Self-paced reading task
どの子供に --- --- ---- ---- --- ------ -------
Self-paced reading task
----- 叔母は --- ---- ---- --- ------ -------
Self-paced reading task
----- --- 母親が ---- ---- --- ------ -------
Self-paced reading task
----- --- --- ケーキを ---- --- ------ -------
Self-paced reading task
----- --- --- ---- 焼いたと --- ------ -------
Self-paced reading task
----- --- --- ---- ---- 台所で ------ -------
Self-paced reading task
----- --- --- ---- ---- --- お手伝いさんに -------
Self-paced reading task
----- --- --- ---- ---- --- ------ 知らせましたか。
Results: Scrambled conditions
1100
± Match
F1(1, 39) = 8.6, p<.01;
F2(1,23)=7.4, p<.01
1000
900
scramble,match
800
scramble,mismatch
700
his/her
600
500
1
2
3
4
5
6
7
8
9
10
Slowdown at mismatching NP is observed.
Results: Non-scrambled
conditions
1100
± Match
1000
FS<1
900
unscr,match
800
unscr,mismatch
700
his/her
600
500
1
2
3
4
5
6
7
8
9
10
Slowdown at mismatching NP only when NP is possible antecedent.
Summary: Experiment 3
 Binding constraint
application takes place in
advance of the verb.
 Wh-gap is posited in a
simple clause.
HIS-WH
NP-nom
gap
Verb
start
Experiment 3 (off-line):
Grammatical judgment test
which of his children (DAT) the man (NOM) …

which of his children (NOM) the man (DAT) …
*?
Experiment 3 (off-line):
Stimuli
a. Non-wh, Non-scrambled
[His children]-nom Adv the man-dat
b. Non-wh, Scrambled
[His children]- dat Adv the man-nom
c. Wh, Non-scrambled
[Which of his children]-nom Adv the man-dat
d. Wh, Scrambled
[Which of his children]-dat Adv the man-nom
Experiment 3 (off-line):
Design & Procedure
• 4 lists were created by distributing 32 items in a
Latin Square design
• 16 items: same materials from online test, and
16 items: different from those in online test.
• 32 filler sentences
• Anaphoric relation judgment task
• 40 native speakers of Japanese, same individuals
as the online test
Experiment 3 (off-line):
Results
5
• Backwards anaphora
are more allowed in
scrambled conditions.
4.5
4
3.5
3
• It confirms that the
binding facts
underlying in the
online test are correct.
2.5
2
1.5
1
Non-w h, Unscr
Non-w h, Scr
Wh, Unscr
Wh, Scr
start
Grammatical Search and Reanalysis
David Schneider
Colin Phillips
Journal of Memory & Language, 2001
• Previous study showed incremental application of
grammatical constraints,
i.e. derivations operate on a word-by-word time-scale
• Next study: do real-time derivations show consistency,
i.e does structure-building keep to a single derivation?
how does is grammar searched to find possible analyses?
Grammatical Search
S
NP
the man
V
knows
VP
NP
the woman
(Schneider & Phillips, 2001)
Grammatical Search
S
NP
the man
V
knows
VP
NP
the woman
V
likes
(Schneider & Phillips, 2001)
Grammatical Search
S
NP
the man
VP
V
knows
S
NP
the woman
V
likes
(Schneider & Phillips, 2001)
Grammatical Search
S
NP
the man
It’s clear that this is the
right conclusion, but
it’s less clear how the
system reaches this
conclusion.
VP
V
knows
S
NP
the woman
V
likes
(Schneider & Phillips, 2001)
Grammatical Search
Option 1: combine with a
local NP, ignoring existing
status of the NP.
S
NP
the man
V
knows
VP
NP
the woman
V
likes
(Schneider & Phillips, 2001)
Grammatical Search
Option 1: combine with a
local NP, ignoring existing
status of the NP.
S
NP
the man
V
knows
VP
NP
the woman
V
likes
(Schneider & Phillips, 2001)
Grammatical Search
Option 2: search the
structure for an NP subject
that currently lacks a q-role,
i.e., focused search.
S
NP
the man
V
knows
VP
NP
the woman
V
likes
(Schneider & Phillips, 2001)
Grammatical Search
Option 2: search the
structure for an NP subject
that currently lacks a q-role,
i.e., focused search.
S
NP
the man
V
knows
VP
NP
the woman
V
likes
(Schneider & Phillips, 2001)
Grammatical Search
Option 2: search the
structure for an NP subject
that currently lacks a q-role,
i.e., focused search.
S
NP
the man
V
knows
VP
NP
the woman
V
likes
This fails, so reanalysis is
needed, … but only as a last
resort operation.
Test Case
If there is a higher NP,
currently lacking a q-role, a
focused search will find it.
S
NP
the man
V
knows
VP
NP
the woman
V
likes
(Schneider & Phillips, 2001)
Test Case
If there is a higher NP,
currently lacking a q-role, a
focused search will find it.
NP
NP
the man
S’
who
S
t
VP
V
knows
NP
the woman
V
likes
(Schneider & Phillips, 2001)
S
V
likes
NP
NP
the man
Test Case
If there is a higher NP,
currently lacking a q-role, a
focused search will find it.
S’
who
S
t
VP
V
knows
NP
the woman
(Schneider & Phillips, 2001)
Test Case
If there is a higher NP,
currently lacking a q-role, a
focused search will find it.
NP
NP
the man
S’
who
An unconstrained search
will not be affected by the
presence of the higher NP.
S
t
VP
V
knows
NP
the woman
V
likes
(Schneider & Phillips, 2001)
Test Case
If there is a higher NP,
currently lacking a q-role, a
focused search will find it.
NP
NP
the man
S’
who
An unconstrained search
will not be affected by the
presence of the higher NP.
S
t
VP
V
knows
S
NP
the woman
V
likes
(Schneider & Phillips, 2001)
Test Case
If there is a higher NP,
currently lacking a q-role, a
focused search will find it.
NP
NP
the man
S’
who
An unconstrained search
will not be affected by the
presence of the higher NP.
S
t
Probe
Antecedents for reflexives.
VP
V
knows
S
NP
the woman
V
likes
(Schneider & Phillips, 2001)
Test Case
If there is a higher NP,
currently lacking a q-role, a
focused search will find it.
NP
NP
the man
S’
who
An unconstrained search
will not be affected by the
presence of the higher NP.
S
t
Probe
Antecedents for reflexives.
VP
V
knows
...the recipe herself
...the recipe himself
S
NP
the woman
V
likes
(Schneider & Phillips, 2001)
S
V
likes
NP
NP
the man
Test Case
If there is a higher NP,
currently lacking a q-role, a
focused search will find it.
S’
who
An unconstrained search
will not be affected by the
presence of the higher NP.
S
t
VP
V
knows
NP
the woman
Probe
Antecedents for reflexives.
...the recipe herself
...the recipe himself
(Schneider & Phillips, 2001)
Grammatical Search
Relative to its
unambiguous control,
high attached reflexives
pose no difficulty.
(Schneider & Phillips, 2001)
Grammatical Search
Relative to its
unambiguous control,
high attached reflexives
pose no difficulty.
…but low attached
reflexives present great
difficulty.
(Schneider & Phillips, 2001)
S
V
likes
NP
NP
the man
Therefore...
High attachment is chosen
…despite well-known localattachment biases.
S’
who
Grammatical search is
focused, constrained by
existing commitments
S
t
VP
V
knows
NP
the woman
(Schneider & Phillips, 2001)
Therefore...
High attachment is chosen
…despite well-known localattachment biases.
Grammatical search is
focused, constrained by
existing commitments
S
NP
the man
V
knows
VP
NP
the woman
V
likes
This helps us to understand
how grammatical search
proceeds in simple cases.
Conclusions
• Local attachment is easy
• The local reanalysis is easy (cf. Sturt et al. 2000)
• So why is local attachment avoided?
Reanalysis is a Last Resort
(even if it’s easy)
Reversal of results…
NP-biased
hear, warn,
understand
S-biased
claim, believe,
suspect
Outline
•
•
•
•
•
•
•
The Challenge
Real-time Grammar
Accurate Parsing
Incremental Parsing
Mapping onto the Brain: Electrophysiology
Encoding
Outlook
Time Resolution
• Syntax: phrase-by-phrase time scale
• Reading-time studies: word-by-word time scale
• Brain recordings: millisecond time scale
Event-Related Potentials (ERPs)
John
is
laughing.
s1
s2
s3
Evolving understanding of ERP components
associated with language…
Semantically unexpected input
• She spread the warm bread with socks.
(Kutas & Hillyard, 1980)
• She was stung by a fly.
(Kutas, et al., 1984;
Federmeier & Kutas, 1999)
(Slide from Kaan (2001)
N400
• Negative polarity
• peaking at around 400 ms
• central scalp distribution
(Slide from Kaan (2001)
ERP Sentence Processing
N400
• Developing understanding of
N400 is informative
I drink my coffee with cream and sugar
I drink my coffee with cream and socks
• Response to ‘violations’
Kutas & Hillyard (1980)
Vervet Monkeys
• Many predators: leopard, lion
hyena, jackal, eagle, etc. etc.
• Distinct alarm calls for different predators
ERP Sentence Processing
N400
• Developing understanding of
N400 is informative
Fully Congruent
Most new drugs are tested on
white lab rats.
• Response to normal
sentences
Van Petten & Kutas (1991)
ERP Sentence Processing
N400
• N400 to semantic anomalies is a special
case of a much more general phenomenon
• All words elicit N400-like response, timing
and amplitude proportional to congruency,
frequency, etc.
• More detailed understanding is contingent
on more detailed models of semantic
interpretation
Morpho-Syntactic violations
Every Monday he mows the lawn.
Every Monday he *mow the lawn.
The plane brought us to paradise.
The plane brought *we to paradise.
(Coulson et al., 1998)
(Slide from Kaan (2001)
P600
he mows
he *mow
(Slide from Kaan (2001)
Left Anterior Negativity
(LAN)
P600
he mows
he *mow
(Slide from Kaan (2001)
ERP Sentence Processing
LAN, P600
Sie bereist dasneuter Landneuter …
Sie bereist denmasculine Landneuter …
she travels the
land
...
Gunter et al. (2000)
ERP Sentence Processing
P600
Emily wondered who the performer in the concert
had imitated for the audience’s amusement.
Emily wondered whether the performer in the concert
had imitated a pop star for the audience’s amusement.
• P600 reflects
normal structurebuilding processes.
Kaan et al. (2000)
Electrophysiology of
Wh-movement
Colin Phillips, Nina Kazanina,
Shani Abada, Daniel Garcia-Pedrosa
[revision of work done at UDel in 2000]
Experiment Design
Materials
a. The actress wished that the producers knew that the witty
host would tell the jokes during the party.
b. The actress wished that the producers knew which jokes
the witty host would tell __ during the party.
c. The producers knew that the actress wished that the witty
host would tell the jokes during the party.
d. The producers knew which jokes the actress wished that
the witty host would tell __ during the party.
Experiment Design
Materials
a. The actress wished that the producers knew that the witty
host would tell the jokes during the party.
b. The actress wished that the producers knew which jokes
the witty host would tell
during the party.
c. The producers knew that the actress wished that the witty
host would tell the jokes during the party.
d. The producers knew which jokes the actress wished that
the witty host would tell
during the party.
Short wh-dependency
Experiment Design
Materials
a. The actress wished that the producers knew that the witty
host would tell the jokes during the party.
b. The actress wished that the producers knew which jokes
the witty host would tell
during the party.
c. The producers knew that the actress wished that the witty
host would tell the jokes during the party.
d. The producers knew which jokes the actress wished that
the witty host would tell
during the party.
Long wh-dependency
Electrode PZ
n=20
Short Conditions
-5
-4
-3
-2
-wh
-1
Verb
0
1
2
3
+wh
4
5
-100
0
100
200
300
400
500
600
700
800
900
1000
Effect of wh-movement significant (p<.01) from 300-400ms onwards
start
Electrode PZ
n=20
Long Conditions
-5
-4
-3
-2
-wh
-1
Verb
0
1
2
3
+wh
4
5
-100
0
100
200
300
400
500
600
700
800
900
1000
Effect of wh-movement significant (p<.01) from 300-400ms onwards
How fast is Structural Computation?
Silke Urban
Colin Phillips
Background
Early Left Anterior Negativity
(Angela Friederici, Anja Hahne, et al.)
Neville et al., 1991
The scientist criticized a proof of the theorem.
The scientist criticized Max’s of proof the theorem.
500ms/word
500ms/word
Hahne & Friederici, 1999
 Das Baby wurde gefüttert
The baby was fed
 Das Baby wurde im gefüttert
The baby was in-the fed
Question: are the brain responses to
violations automatic?
Hahne & Friederici, 1999
P600
P600
Hahne & Friederici, 1999
ELAN
ELAN
ELAN
•
•
•
•
Very fast: 150-250ms
Automatic
Left anterior (= frontal) scalp distribution
Elicited by a subclass of syntactic violations
“Phase 1 (100–300 ms) represents the time window in
which the initial syntactic structure is formed on the basis
of information about the word category.”
(Friederici 2002)
Questions about ELAN
• How plausible is it that ELAN reflects syntactic
structure building?
– Speed: 150ms is faster than lexical access!
– Generality: ELAN is not elicited by most violations;
almost all studies on ELAN involve one construction
(for each of German, English)
– Localization…
Brodmann Areas
Magnetoencephalography (MEG)
pickup coil & SQUID
assembly
160 SQUID
whole-head
array
(Friederici et al. 2000)
Two regions
of interest
(Friederici et al. 2000)
(Friederici et al. 2000)
Anterior Temporal Lobe?
• Why is anterior temporal lobe so important in ELAN?
• How does it differ from Broca’s area (BA 44 etc.) that are
implicated so often in other studies?
• Friederici: both responsible for ‘structure building’; BA44
also responsible for ‘syntactic working memory’; ‘the
inferior portion of BA44 is selectively activated when
syntactic processes are in focus.’
• Anterior temporal lobe associated with
– lexical information
– activated in fMRI by comparisons of sentences with word lists
Alternative Interpretation
• ELAN reflects violation/suppression of automatic
lexical prediction
–
–
–
–
accounts for localization to anterior temporal lobe
accounts for very early timing
might account for automaticity
accounts for very limited distribution
Neville et al., 1991
The scientist criticized a proof of the theorem.
The scientist criticized Max’s of proof the theorem.
NP
Max’s
N
Hahne & Friederici, 1999
 Das Baby wurde gefüttert
The baby was fed
 Das Baby wurde im gefüttert
The baby was in-the fed
PP
in
NP
the
N
Prediction
• If ELAN reflects violation of lexical prediction,
rather than syntactic structure-building, then…
– change lexical predictions
– keep syntactic violation the same
– should ‘turn off’ ELAN brain response
Neville et al., 1991
The scientist criticized a proof of the theorem.
The scientist criticized Max’s of proof the theorem.
Possible to block the automatic prediction
of an N following a possessor: ellipsis
NP
Max’s
N
Ellipsis
• Possessors may appear alone in ellipsis contexts:
Although I like Mary’s theory, I don’t like John’s.
Experimental Conditions
Although Erica kissed Mary’s mother, she did not kiss the daughter of the bride.
Although Erica kissed Mary’s mother, she did not kiss Dana’s of the bride.
Although the bridesmaid kissed Mary, she did not kiss the daughter of the bride.
Although the bridesmaid kissed Mary, she did not kiss Dana’s of the bride.
Experimental Conditions
Although Erica kissed Mary’s mother, she did not kiss the daughter of the bride.
Although Erica kissed Mary’s mother, she did not kiss Dana’s of the bride.
ellipsis possible
Although the bridesmaid kissed Mary, she did not kiss the daughter of the bride.
Although the bridesmaid kissed Mary, she did not kiss Dana’s of the bride.

ellipsis impossible
Experimental Design
• 384 sentences per session
– 128 targets (drawn from 128 sets of 4 conditions)
– 64 items designed to elicit ‘agreement violation’ LAN
– 192 filler items, designed to hide violations and promote ellipsis
• Procedure
– RSVP (Rapid Serial Visual Presentation), 500ms/word
– Grammaticality judgment task
• Recording: 32-electrode montage
• 22 subjects (so far)
+
Although
Erica
kissed
Mary’s
mother,
she
did
not
kiss
Dana’s
of
the
bride.
???
good
bad
Preliminary Results
b
a
a. Although … Mary’s mother … Dana’s of …
b. Although … Mary
… Dana’s of …
Interim Conclusion
• Preliminary results lend support to our
interpretation of the (E)LAN - the anterior
negativity is reduced in an ellipsis context
– structural violation is identical in both conditions
– obligatory lexical prediction of N following possessor
(e.g. Mary’s…) is absent in ellipsis context
• Structure-building may begin ~250-300ms after a
word is presented
Outline
•
•
•
•
•
•
•
The Challenge
Real-time Grammar
Accurate Parsing
Incremental Parsing
Mapping onto the Brain: Electrophysiology
Encoding
Outlook
The Binding Problem
• Discrete infinity
Individual neurons (or groups of neurons) can store finite
information about objects, words, etc.
But sentences are infinite in number!
• Representing structure: can’t just activate all words
e.g. THE + MAN + ATE + PIZZA
• Must create & discard structures quickly: 100s of msec
Temporal Binding
the
phase
locked
man
~25ms, 40Hz
ate
pizza
phase
locked
Evidence from Animals & Humans
• Direct recordings (cat)
(Singer 1999)
• EEG recordings (human)
(Tallon-Baudry & Bertrand 1999)
Limitations…
• No evidence yet of role in human syntax
• Limited capacity - ~7 bindings
• 1-level of hierarchy only
• Interesting hypothesis (Whitney & Weinberg 2002)
– temporal binding is the neural representation used for the
‘syntactic workspace’
– additional neural encoding mechanism used for long-term
representation and storage
Outline
•
•
•
•
•
•
•
The Challenge
Real-time Grammar
Accurate Parsing
Incremental Parsing
Mapping onto the Brain: Electrophysiology
Encoding
Outlook
Outlook
• Overview
– challenge for unification: real-time hypotheses
• Real-time Grammar
– syntactic derivations look like real-time derivations
• Accurate Parsing
– real-time derivations have the sophistication that is needed
• Incremental Parsing
– real-time interpretation is time-locked to incoming words
• Mapping onto the Brain: Electrophysiology
– extreme time-precision of EEG/MEG can be linked to detailed
linguistic constructs
• Encoding
– plausible models of neural encoding of structure are emerging
• Unification: Problem or Mystery…
www.ling.umd.edu/colin
[email protected]