Transcript Document

Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience
Edited by Bernard J. Baars and Nicole M. Gage
2007 Academic Press
Chapter 7 Hearing and Speech
“it was very fortunate that, even in Helmholtz’ time, the great anatomical discoveries by Corti (and others)
had already made it clear that the vibrating tissue most important for hearing is the basilar membrane of the
inner ear, because the cells on which the nerve endings terminate are seated on this membrane … the
problem of how we hear was reduced largely to a mechanical question: how does the basilar membrane
vibrate when the eardrum is exposed to a sinusoidal sound pressure?”
Bekesy, Nobel Prize Lecture (online)
Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience
Edited by Bernard J. Baars and Nicole M. Gage
2007 Academic Press
Chapter Outline
1.0 Introduction
2.0 The central auditory system
3.0 Functional mapping of auditory processing
4.0 Speech Perception
5.0 Music Perception
6.0 Learning and plasticity
7.0 Auditory awareness and imagery
8.0 Summary
Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience
Edited by Bernard J. Baars and Nicole M. Gage
2007 Academic Press
1.0 Introduction
A model for sound processing
Sound inputs enter the system
and there is a very brief
storage or buffer called echoic
memory.
There may be many competing
sounds occurring at the same
time -- such as the sounds in a
noisy lecture hall
Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience
Edited by Bernard J. Baars and Nicole M. Gage
2007 Academic Press
1.0 Introduction
A model for sound processing
Selective attention allows the
system to direct resources to a
subset of the sensory inputs
that are competing for attention
-- such as attending to a
lecturer and ignoring the
sounds of someone whispering
behind you
Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience
Edited by Bernard J. Baars and Nicole M. Gage
2007 Academic Press
1.0 Introduction
A model for sound processing
Complex interactions occur
between the new inputs and
existing memories and
experiences, as well with other
sensory systems.
Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience
Edited by Bernard J. Baars and Nicole M. Gage
2007 Academic Press
1.0 Introduction
A model for sound processing
The ultimate goal or ‘action’ to
be performed is important as
well, and will affect how
information is encoded and
stored.
These processes are not oneway: there are interactions that
occur through the encoding of
sounds, both within the auditory
system and across other
sensory, cognitive, memory,
and motor systems.
Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience
Edited by Bernard J. Baars and Nicole M. Gage
2007 Academic Press
1.0 Introduction
It’s about time …
Time is a critical aspect of auditory
processing: the auditory system differs
from the visual system in that all sound
processing occurs over time.
A spectrogram provides a ‘picture’ of
speech, with time on the x-axis and
frequency on the y-axis. Speech
contains harmonics -- formants - at
specific regions on the spectrogram.
Darker bands indicate more energy at
that region in the speech signal.
A spectrogram showing the energy at
different frequencies in the speech signal
Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience
Edited by Bernard J. Baars and Nicole M. Gage
2007 Academic Press
1.0 Introduction
It’s about time …
Three time scales are critical for decoding
speech -- 20, 200, 2000 milliseconds (ms):
Formants in consonants such as ‘b’ and ‘d’
change on a time scale of about 20/1000 of
a second, or 20 ms
A syllable lasts about 200 ms
A typical sentence lasts around 2000 ms
A spectrogram showing the energy at
different frequencies in the speech signal
Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience
Edited by Bernard J. Baars and Nicole M. Gage
2007 Academic Press
1.0 Introduction
It’s about time …
The auditory system must be able to
decode information at each of these
timescales in order to accurately
perceive speech.
A spectrogram showing the energy at
different frequencies in the speech signal
Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience
Edited by Bernard J. Baars and Nicole M. Gage
2007 Academic Press
1.0 Introduction
Sound and hearing basics
spectral-domain
time-domain
Two ways to represent the physical features of a sound pictorially:
In the time-domain: time is on the x-axis and pressure (or amplitude) is on the y-axis
In the spectral-domain: time is on the x-axis and frequency (in Hz) is on the y-axis
Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience
Edited by Bernard J. Baars and Nicole M. Gage
2007 Academic Press
1.0 Introduction
Sound and hearing basics
Physical features of sounds: Frequency, intensity, and time
The frequency of a sound is the rate of sound wave vibration and is calculated in
terms of cycles completed per second, or hertz (Hz). A sinusoid with 1000 cycles per
second has a frequency of 1000 Hz
Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience
Edited by Bernard J. Baars and Nicole M. Gage
2007 Academic Press
1.0 Introduction
Sound and hearing basics
The intensity of a sound reflects
the amount of amplitude (or
displacement) within its cycle and
over time.
Intensity is reported in units of
decibels (dB). Hearing thresholds
for humans and cats are shown,
along with the dB level for various
types of sounds. Sounds over 120
dB present a high risk for damage
to the hearing system.
Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience
Edited by Bernard J. Baars and Nicole M. Gage
2007 Academic Press
1.0 Introduction
Sound and hearing basics
The peripheral hearing system:
external, middle and inner ear
The cochlea is located in the inner
ear. Within the cochlea lies the
basilar membrane
Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience
Edited by Bernard J. Baars and Nicole M. Gage
2007 Academic Press
1.0 Introduction
A cut-away view of the external, middle, and inner ear showing the
transmission of sound (red arrows). Note the basilar membrane curving within
the cochlea
Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience
Edited by Bernard J. Baars and Nicole M. Gage
2007 Academic Press
1.0 Introduction
Innervation patterns of afferent (ascending pathway) and efferent (descending
pathway) neurons in the Organ of Corti
Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience
Edited by Bernard J. Baars and Nicole M. Gage
2007 Academic Press
2.0 The Central Auditory System
Auditory pathways
The auditory system is complex, with
many stages and pathways on the
way to auditory cortex.
The two main flow of information are
in the ascending and descending
pathways.
The ascending (afferent) pathways
transmit information about sounds
from the periphery to cortex. This is
not a simple delivery system, but
entails many stages where
information is encoded and recoded
Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience
Edited by Bernard J. Baars and Nicole M. Gage
2007 Academic Press
2.0 The Central Auditory System
Auditory pathways
The descending (efferent) pathways
extend from regions in the cortical
and subcortical auditory system to
the periphery. These pathways are
under direct or indirect cortical
control.
An important function of the
descending pathways is to provide
‘top-down’ information that aids in
selective attention processes.
Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience
Edited by Bernard J. Baars and Nicole M. Gage
2007 Academic Press
2.0 The Central Auditory System
Auditory cortex
Auditory cortex is the region within the
cortex that is specialized for sound
processing. It is located within the
Sylvian fissure on the surface of the
supratemporal plan and the upper
banks of the superior temporal gyrus,
so it is difficult to see from a lateral view
of the brain.
Brodmann areas for auditory and
receptive language processing include
Brodmann 22, 41, 42, and 52.
Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience
Edited by Bernard J. Baars and Nicole M. Gage
2007 Academic Press
2.0 The Central Auditory System
Auditory cortex
Much of what we know about
auditory cortex comes from
studies in non-human
primates. Here is a map of
auditory regions in the left
hemisphere of the macaque.
The core, belt, and parabelt
regions that form auditory
cortex are tucked inside the
Sylvian fissure and shown in
an opened cutaway.
Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience
Edited by Bernard J. Baars and Nicole M. Gage
2007 Academic Press
2.0 The Central Auditory System
Auditory cortex
Human brain, showing auditory cortical regions within
the Sylvian fissure in the left hemisphere.
Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience
Edited by Bernard J. Baars and Nicole M. Gage
2007 Academic Press
2.0 The Central Auditory System
An axial slice of the brain showing a schematic of regions within auditory cortex. Note
the hemispheric asymmetries for each region.
Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience
Edited by Bernard J. Baars and Nicole M. Gage
2007 Academic Press
2.0 The Central Auditory System
Auditory cortex
Top: MRI coronal slice showing
the location of the
supratemporal plane
Bottom: MRI axial slice
showing the transverse gyrus
of Heschl and the planum
temporale
Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience
Edited by Bernard J. Baars and Nicole M. Gage
2007 Academic Press
2.0 The Central Auditory System
Receptive field properties of neurons in auditory cortex
Here are receptive fields of two auditory
cortical neurons plotted as a function of
sound pressure level and azimuth (a ‘map’
of auditory space with respect to the two
ears).
Notice that the top map shows a cell that
is broadly tuned across much of the
azimuth on the contralateral side, while
the bottom map shows a cell with a much
narrower tuning.
Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience
Edited by Bernard J. Baars and Nicole M. Gage
2007 Academic Press
3.0 Functional Mapping of Auditory Processing
Auditory stream analysis
A core function of the auditory system is to localize
sound. Another central function is to recognize
auditory ‘objects’ such as a friend’s voice or the
sound of a car braking. An important role for the
auditory system in humans is the perception of
speech.
These differing functions have led to the suggestion
that there are multiple -- but highly interactive -auditory processing ‘streams’ that underlie these
functions and that interface with other sensory
modalities.
Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience
Edited by Bernard J. Baars and Nicole M. Gage
2007 Academic Press
3.0 Functional Mapping of Auditory Processing
Heschl’s gyrus and the Planum temporale
Heschl’s gyrus and the planum temporale
have been the focus of many fMRI studies
in order to understand their functional role
in sound processing and speech
perception.
Heschl’s gyrus is the site for primary
auditory cortex (A1). Far less is known
about the functional role of the planum
temporale. Recent studies have suggested
that it serves as a nexus or hub for auditory
processing streams.
Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience
Edited by Bernard J. Baars and Nicole M. Gage
2007 Academic Press
3.0 Functional Mapping of Auditory Processing
‘What’ and ‘where’ processing streams
‘What’ and ‘where’ processing
streams have been suggested based
on studies in non-human primate.
Similar to the pathways or streams
evidenced in the visual system, the
‘what’ pathway is engaged in object
recognition while the ‘where’ pathway
is engaged in sound localization
Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience
Edited by Bernard J. Baars and Nicole M. Gage
2007 Academic Press
3.0 Functional Mapping of Auditory Processing
‘What’ and ‘where’ processing streams
Human studies have provided evidence
that auditory streams may not simply
form two pathways: this topic remains
an open one, but recent studies have
provided evidence for a ‘how’ stream in
addition to the ‘what’ and ‘where’
streams. Other studies have provided
evidence for a ‘who’ stream for voice
recognition.
Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience
Edited by Bernard J. Baars and Nicole M. Gage
2007 Academic Press
4.0 Speech Perception
Speech perception
The decoding of human speech is a central and important function of the human
auditory system. Here is a spectrogram showing time on the x-axis, frequency on the yaxis, with darkened shadings showing increased energy at that time and frequency.
Speech perceptual systems must be able to decode information in short time windows
(20 ms) as well as longer ones (2000 ms).
Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience
Edited by Bernard J. Baars and Nicole M. Gage
2007 Academic Press
4.0 Speech Perception
Distinctive features
Early work using the spectrogram
to study the sound-based (acoustic
phonetics) aspects of speech
identified distinctive features of
consonants and vowels.
Here is a schematic illustration of
the formant patterns for types of
consonants in American English
Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience
Edited by Bernard J. Baars and Nicole M. Gage
2007 Academic Press
4.0 Speech Perception
The Lack of Invariance Problem
Speech scientists were searching for the basic ‘building blocks’ or primitives that
enabled us to decode speech. The distinctive features in speech sounds seemed like
a promising candidate for speech primitives -- however the lack of invariant formant
patterns for consonants, differing depending on the following vowel, made it obvious
that encoding distinctive feature information could not be the only way speech
decoded.
Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience
Edited by Bernard J. Baars and Nicole M. Gage
2007 Academic Press
4.0 Speech Perception
A process model
Other models for understanding
how speech perception was
accomplished by the brain took into
account other processes -- such as
semantic decoding -- that were
closely association with speech
perception
Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience
Edited by Bernard J. Baars and Nicole M. Gage
2007 Academic Press
4.0 Speech Perception
Speech perception in the brain
The advent of functional neuroimaging techniques allow us to investigate brain
regions involved both in speech perception and production
Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience
Edited by Bernard J. Baars and Nicole M. Gage
2007 Academic Press
4.0 Speech Perception
19th and 20th Century models
for speech and language
The groundbreaking work of 19th
century scientists such as Broca and
Wernicke identified regions that
appeared critical for speech
perception (temporal lobe, ‘Wernicke’s
area’) and production (frontal lobe,
‘Broca’s area’).
Functional imaging studies have
confirmed these general findings,
although new discoveries are still
occurring to clarify the roles of these
brain regions in speech.
Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience
Edited by Bernard J. Baars and Nicole M. Gage
2007 Academic Press
4.0 Speech Perception
Beyond Broca …
A recent model extends the earlier
views of receptive and productive
language regions beyond Broca’s and
Wernicke’s areas. This is an active
area of study and new models are
constantly being advanced.
Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience
Edited by Bernard J. Baars and Nicole M. Gage
2007 Academic Press
5.0 Music Perception
Music perception
The perception of music is complex and likely involves many interactive stages
Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience
Edited by Bernard J. Baars and Nicole M. Gage
2007 Academic Press
5.0 Music Perception
Music perception
Music and language are unique
to human beings
Recent studies have
investigated if same or similar
brain areas are involved in
encoding music and language.
Music perception in the brain is
a relatively new field of study -like speech, there are no simple
‘building blocks’ or primitives.
Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience
Edited by Bernard J. Baars and Nicole M. Gage
2007 Academic Press
6.0 Learning and Plasticity
Sensory memory formation
We are continually learning to
recognize new auditory ‘objects’ in
our environment: a new friend’s
voice, a new cellphone ringtone …
how, and where, does the auditory
system accomplish this?
Sensory memory formation: a large
body of evidence indicates that
sensory memories are both formed
and stored in auditory cortex.
Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience
Edited by Bernard J. Baars and Nicole M. Gage
2007 Academic Press
7.0 Auditory Awareness and Imagery
The hearing system is the last to fall
asleep and the first to awaken.
Why is it that the best way to wake
someone up is to call their name? Is
there a ‘wake-up’ area in the brain?
A neuroimaging study investigated
that question by playing simple
sounds (beeps) and one’s own name
in two conditions: while the subject
was awake and while the subject
was asleep
Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience
Edited by Bernard J. Baars and Nicole M. Gage
2007 Academic Press
7.0 Auditory Awareness and Imagery
A brain wake-up call?
Results: areas in the middle
temporal and frontal lobes were
more active for hearing one’s
name vs. beeps in the sleep
condition.
It might be that these regions
monitor sound during sleep and
activate other brain regions upon
hearing one’s name.
Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience
Edited by Bernard J. Baars and Nicole M. Gage
2007 Academic Press
7.0 Auditory Awareness and Imagery
Are the same brain areas active when you imagine a sound vs. hearing it?
Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience
Edited by Bernard J. Baars and Nicole M. Gage
2007 Academic Press
7.0 Auditory Awareness and Imagery
An fMRI study investigated that question: primary auditory cortex (A1) was active
during the perception phase but not during the imagery phase of the study.
Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience
Edited by Bernard J. Baars and Nicole M. Gage
2007 Academic Press
8.0 Summary
•While much work has been accomplished to understand how the brain
decodes complex sound patterns such as those in human speech and
music, much more work needs to be done to understand how and
where this happens in the brain
•There are likely multiple auditory streams that interact in encoding
auditory sensory inputs, that are also highly interactive with other
sensory, memory, and action systems
•The advent of neuroimaging techniques have made it possible to
investigate aspects of auditory function that were previously difficult or
impossible to study, such as auditory imagery and auditory awareness
during sleep