Transcript Seminar Cognitive Neuropsychology

Situation Models
and Embodied Language Processes
Franz Schmalhofer
University of Osnabrück / Germany
1)
Memory and Situation Models
2)
Computational Modeling of Inferences
3)
What Memory and Language are for
4)
Neural Correlates
5)
Integration of Behavioral Experiments and Neural Correlates (ERP;
fMRI) by Formal Models
Cognition and Knowledge
• Traditional:
• Embodiment of Meaning
– Cognition = Computation
– Cognition is serving perception
and actions
– Representation by propositions
– Representation = Patterns of
possible bodily interactions
with the world (lawfully related
to the world)
– Propositions are abstract
relations
– What an object, event, sentence
means for you, is what you can
do with the object, event,
sentence.
Embodiment of Memory (Glenberg, 1997)
• Projectable properties: information available through
the senses
• Non-projectable properties: information available
through other sources (e.g. memory)
• Conceptualization: Combination (mesh) of projectable
and non-projectable properties
 Primary function of memory is to mesh the embodied
conceptualizations of projectable properties of the
environment with embodied experiences that provide
non-projectable properties
Embodiment of Memory
• Evidence for embodiment...
• ...and affect:
– Forced frowning or smiling influences affective judgments
(Berkowitz & Trocolli 1990, Berkowitz et. al 1993)
• ...and imagery:
– Actually rotating facilitates orientation opposed to imagining
rotating(Montello & Presson 1993, Rieser etl al. 1994).
• ...and memory:
– Retrieval of memorized spatial layouts depends on position
on body axis (Bryant 1992).
Embodied Memory
• Combination of patterns of possible action
= meshing.
• Meshing of patterns of action derived from:
– projectable properties of the environment
– non-projectable properties
• as (spatial-functional) constraint satisfaction.
Function of Memory
• Meshing projectable & non-projectable properties:
• Meshing is important for:
– perception
– imagination
– Comprehension
• Projectable properties as well as non-projectable
properties can be meshed with each-other.
Interaction of perceptions and memory
• Environment has to be primary
• Clamping projectable properties keeps the system
reality-oriented
• Experiences stay individuated
Updating memory
• Experiences are shifts between conceptualizations
• Trajectories from one pattern of action to another
• Trajectories more often used become reinforced
Prediction and Planning
• Hypothetical conceptualizations by using trajectories
from memory
• But:
– Simulating action does not change the environment
– Clamped projectable properties provide wrong constraints for
prediction
– Environment has to be suppressed (effortfull process)
• Suppression loosens tie to reality
Two modes of memory
(like the distinction between implicit/explicit)
• Automatic
– Meshing of projectable and
nonprojectable properties
– Causes conceptual priming:
based on non-projectable
properties, (therefore semantic)
• Effortful
– Suppression of projectable
properties, conceptualization by
trajectories from memory
Cognitive Meshing
• Imagine a ball
• Now Imagine that it has
yellow and white stripes
• Now Imagine that it is
deflated
• Mutual modification of
mashed pattern: not only the
ball but also the stripes
become deformed when the
ball is deflated.
• Patterns of actions to the
same spatio-functional
constraints.
What memory and language are for
• The primary function of memory “is to mesh
the embodied conceptualizations of projectable
properties of the environment with embodied
experiences that provide nonprojectable
properties…This meshed conceptualization,
the meaning, is in the service of control of
action in a three-dimensional environment“
(Glenberg 1997)
• “Language is a surrogate for experience”
(Taylor and Tversky, 1992):
Summary: Embodied Representations & Symbol
Grounding
• Embodied Meaning – action based coding of objects
and situations
• Embodied representations are lawfully and analogically
related to properties of the world (Harnard, 1990, 1993)
• Notion of “mesh”: mutual modification of patterns of
action (Glenberg, 1997)
• Meaning of a situation is a meshed pattern of possible
actions = embodied conceptualization.
Overview of the methods of cognitive
neuropsychology/science (I)
• Advances in science by technology:
– Invention of the telescope in 1608 changed astronomers‘
observational methods
– If well-formulated questions are not asked, even the most
powerful tools will not provide sensible answers
• Cognitive Psychology / Computer Modeling
• Neuroanatomy
– Gross Neuroanatomy (general structures and connections)
– Fine Neuroanatomy (components of individual neurons)
Overview of the methods of cognitive
neuropsychology/science (II)
• Neurophysiology (experimental methods used with
animals)
–
–
–
–
Electrical stimulation
Single-cell recording
Lesions
Genetic manipulations
• Neurology
– Structural imaging and neurological damage
– Causes of neurological disorders (vascular disorders,tumors,
degenerous and infectious disorders, traumata, epilepsies)
– Functional neuro-surgery
Overview of the methods of cognitive
neuropsychology/science (III)
• Converging methods
– Cognitive deficits following brain damage
– Virtual lesions: Transcranial magnetic stimulation (TMS)
– Functional imaging
• Electrical and magnetic signals in the brain (EEG, MEG)
• Metabolic signals
– Positron emission tomography (PET): regional cerebral blood flow;
– fMRI: blood oxygenation level dependent effect or BOLD effect
Brain functions (1810-1819)
• Do parts of the brain working
independently enable the
mind?
(componential hypothesis)
• Franz Joseph Gall and J. G.
Spurzheim
– 35 specific brain functions
– Language, color perception,
hope, self-esteem
– With practice, areas grow,
causing a bump in the
overlying skull
– Anatomical personology
– phrenology
• Does the whole brain work in
concert?
(wholistic hypothesis)
• Pierre Flourens (1794-1867)
– All sensations, all perceptions
and all volitions occupy the
same seat in these cerebral
organs.
– The faculty of sensation,
percept and volition is then
esssentially one faculty.
– Empirical evidence: no matter
where he leasoned a bird brain,
the bird recovered
Language Areas
• Broca
• Wernicke
Neuroanatomist Korbinian Brodman (1909)
• Analyzed cellular
organization of the cortex
• Tissue stains to visualize
different brain regions
• To a large extent
cytoarchitecturally described
brain areas do indeed
represent functionally distinct
brain regions
Componential or wholistic?
Again the question
• Camillo Golgi
– Developed stain that
impregnated individual neurons
– Believed the whole brain to be
a continuous mass of tissue that
shares a common cytoplasm
• Cajal
– Used Golgi stains
– Identified the unitary nature of
neurons
– Transmittion of information by
electricity
How does the nervous system work
(20-th century)
• Understand how single
neurons behave and interact
• Knowing all the elements,
can we figure out the system?
• Billions of neurons
• Brain-damaged humans show
lack of typical symptoms
• Impossible to localize
„higher cognitive functions“
Jackson: Lesion might well affect other structures in the
brain because the lesion might have damaged neurons
connected to other regions; diaschisis: damage of one part
can create problems for another.
Gestaltist view: The whole is different from the sum of its
parts
Summary
• Localists
– Wrong, in that they tried to map behaviors and perceptions into single
locations in the cortex
– Any behavior is produced by many areas
– Complex functions
• Globalists
– A function can be achieved in numerous different ways (in this sense
the globalists were right)
• But
– simple processes that are recruited to exercise an ability are localized
Event-Related Brain Potentials
What are event-related brain potentials
(ERPs)
• Like EEG, but related to an event (a task, e.g. making a
decision, reading a word, etc.)
• The ERP (a few µV is small in relation to the EEG
(about 50µV)
• The international 10-20 system (Jasper 1958) allows
for between-laboratory and between-experiment
comparisons
EEG profiles
obtained during
various states of
consciousness
After Penfield and
Jasper (1954)
How ERPs are obtained from EEG-data
Schematic representation of ERP-Procedure
ERP-components
ERP-Components
• Usually labeled by polarity and latency, P300, P3
(ordinal latency of the component)
• Scalp locations, e.g. frontal P300
• Psychological or experimental conditions
– Novelty P3
– Readiness potential
– Mismatch negativity, MMN
• Sensory or exogeneous
• Interaction subject – response (task requirements)
endogenous
From the brain to the scalp
• Distant manifestations of activations of populations of
neurons (recorded on surface of skull)
• Requirements
–
–
–
–
Neurons must act synchronously
Electric fields must be oriented so that they cumulate
Therefore only a subset of neural activity is visible
Open field organizations (dentritic trees are ordered), neurons
are organized in layers, most of cortex, parts of thalamus,
cerebellum and others
• Presynaptic potentials (spikes) high frequency
• Postsynaptic potentials (slower), summation thereof
Forward and backward solutions – inverse dipole
modeling
Conclusions from ERPs
• Just a sample of neuronal activity
– If you find the same effect in different experimental
conditions
– If you do not find an effect
From the scalp to the brain: Inferring the
sources of ERPs
• Observations are: voltages differences between scalp electrodes
and a reference electrode.
• Identify neural generators of ERPs:
–
–
–
–
indefinite number of unknown parameters
No unique solution
Head is not a homogeneous medium
Difficult to compute
• Non-invasive and invasive techniques
– Dense electrode arrays and source monitoring
– Neurophysiological knowledge, other imaging techniques
• Invasive techniques
– Implanting electrodes, lesion studies with animals
The concept of components
• Voltage x time x location function.
• Segments of the ERP waveform to covary in response
to a specific experimental condition
– Positive, negative
– Aspects of the ERP waveform
– In terms of neural structures that generate them
• But a peak may be the sum of several functionally and
structurally distinct components
Quantification of ERP components
• Artifacts: eyeballs, eyelids, muscles of the head.
• Signal to noise ratio
– ERPs are constant over trials
– Noise is random
– ERPs are independent of the background noise
• Peak measurements
• Covariation measures (e.g. covary with condition)
• Source-activity measures (algorithms for dipole, Loreta,
Baillet & Garnero, 1997) spatio-temporal dipole model;
distributed source models
Problems in Component Measurements
•
•
•
•
Are components identical.
Revisions of component classification
Components overlap
Principle component analysis (application statistics,
linear algebra)
• Subtraction procedure, only amplitude, not latency vary
across conditions
Experimental logic
• Discovery: establishing functional significance
• Components antecedents (I.e. Experimental
manipulations)
• Consequences of variation
• Speculations about the psychological or
neuropsychological function it manifests
Psycho physiological inference
• Conditions are different
• Conditions differ at a particular time
• Conditions differ with respect to the latency of some
process
• Conditions differ with respect to the degree to which
some process occurs
Some ERP-findings
• Movement-related potentials
– Lateralized readiness potential (LRP)
– Contingent negative variation (CNV)
– Error-related negativity (ERN)
• Sensory components
– The early negatives (ERPs and locus of selective attention)
– The middle latency cognitive components (mismatch negativity of MMN)
– N200s (or N2)
• The late cognitive ERPs:
– P300, elicited by deviant stimuli
– The “frontal” P3, elicited by novel stimuli, novelty P3 (no memory
template is available
Some ERP-findings (continued)
• ERP effects associated with subsequent memory
– Distinctive word (van Restoff, character change, large P300s; recalled
ones show larger P300s as compared to not recalled ones.
– Same-different task (Sanquist et al., 1980) larger amplitude P300s were
better recognized in subsequent recognition test.
– Two-process model of recognition (large P300 when explicit recollection
as opposed to “just know”
• N 400 (language-related)
–
–
–
–
–
More prolonged over the right rather than the left hemisphere
N400 may be generated by the parahippocampal anterior fusiform gyrus
A distinctively semantic process
Inversely related to the subject’s expectancy (cloze probability)
Semantically related to sentence completion produce smaller N 400. “The
pizza was too hot to drink / cry”.
Brain Imaging
Brain Imaging
• Magnetic resonance imaging (MRI)
• Angelo Mosso (1846-1910): correlation blood flow –
neuronal activity
• Seymour Kety (1915-2000), Lou Sokoloff quantified
relation (middle of the 20-th century)
• Position Emission tomography (PET)
• 1980s: Michael Posner, Steve Peterson: Study human
cognition by PET; Marcus Raichle
• Donders (1868) method of subtraction
Subtraction Method
Brain imaging (fMRI)
• Behavior of hydrogen atoms or protons in a magnetic
field
• Paul Lauterbur: MRI (Nobel-prize, 2003)
• Seiji Ogawa: functional states of the brain (fMRI);
– Amount of oxygen carried by hemoglobin chances the degree
to which hemoglobin disturbs a magnetic field
– Tracking blood flow
– BOLD-signal: blood oxygen level dependent
Steps in analysis: cortex-segmentation
Computational Modeling and fMRI
A prominent
cognitive
architecture
ACT-R
(Anderson &
Lebiere, 1998)
How to map time predictions to the BOLD-signal
Experimental
Predictions by the
ACT-R model
Comparison of
model
predictions and
observations by
a measure of
proportionality
Summary
• Embodied memory representations
• New methods of cognitive science
• ERP
• fMRI
• Modeling