Transcript No Slide Title

Tension in Music :
Cognition, Emotion, Brain, Movement
Carol L. Krumhansl
Department of Psychology
Cornell University
Ithaca, NY 14853
USA
Tension: Linking Cognition, Emotion, Brain, and Motion
WIPPPPP
Work in Past Published in Press Present Planned
"…expectation is always ahead of the music, creating a
background of diffuse tension against which particular delays
articulate the affective curve and create meaning.”
“Not only does music use no linguistic signs but, on one level at least,
it operates as a closed system, that is, it employs no signs or symbols
referring to the non-musical world of objects, concepts, and human
desires. …This puzzling combination of abstractness with concrete
emotional and aesthetic experience can, if understood correctly,
perhaps yield useful insights into more general problems of meaning
and communication.”
L. B. Meyer, Emotion and Meaning in Music, 1956
Tension and cognition
Mozart, Piano Sonata, K. 282
Krumhansl, Music Perception, 1996
Duration of each beat in the music as performed
Judgments of sections ends
Duration of each beat in the music as performed
Judgments of new musical ideas
Duration of each beat in the music as performed
Krumhansl, Music Perception, 1996
Lerdahl, Tonal Pitch Space, 2001
(a) octave (roo t) level:
(b) fifth leve l:
(c) triadic leve l:
(d) diatoni c level:
(e) ch roma tic level:
0
(0)
0
7
(0)
0
4
7
(0)
0 2 45 7 9
11 (0)
0 1 2 3 4 5 6 7 8 9 10 11 (0)
Diatonic basic space, se t to I/C (C = 0, C# = 1, …B = 11) .
Computing Distance from d minor (vi) chord in F major key
to C major (I) chord in C major key
Region Distance
Chord Distance
Basic Space Differences
F major to C major
d minor to C major
New entries in Basic Space
Tensing
Relaxing
Slide 40
Krumhansl, Music Perception, 1996
Chopin Prelude
125
Tension
100
75
50
25
0
1
3
5
7
9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47
Event
Judged
Predicted
Messiaen, Quartet for the end of time
Messiaen Quartet
125
Tension
100
75
50
25
0
1
3
5
7
9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39
Event
J udged
P redic ted
Bach Chorale
Cristus, der ist mein Liebe
Bach Chorale
125
Tension
100
75
50
25
0
1
3
5
7
9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41
Event
J udged
P redic ted
W agner Diat onic
Hierarchical Right -Branching +
125
100
Tension
Wagner
Parsifal
Grail
Theme
75
50
25
0
1
Diatonic
2
3
4
5
6
7
8
9
8
9
E vent
J udged
P redic ted
W agner Chromat ic
Shif ting Diatonic
Sequent ial to Hierarchical
125
Tension
100
75
50
25
Chromatic
0
1
2
3
4
5
6
7
Event
Judged
Lerdahl & Krumhansl, Music Perception.in press
Predicted
Tipping the (Fourier) balances:
A geometric approach to representing pitch structure
in non-tonal music
Fourier Balance One
FB1
FB 1
Quic kT ime™ and a
T IFF (Uncompres sed) decompres sor
are needed to s ee this picture.
Stable for minor second, major seventh
Unstable for fourth, fifth, and tritone
Stable for a diatonic scale
Fourier Balance Two
FB2
FB 2
Quick Time™ a nd a
TIFF ( Un compr ess ed ) d eco mp res so r
ar e n eed ed to s ee this pi ctur e.
Stable for tritone, minor second, major seventh, fourth and fifth
Unstable for minor third, major sixth
Stable for an octatonic scale
FB 1
FB 4
FB 2
FB 5
FB 3
FB 6
Compare Fourier Balance Model to Tonal Pitch Space Model
Predicting Judged Tension with FB model
Fourier Balance Model
R-squared (2,5) = .999
Predicting Judged Tension with TPS Model
R-squared (2,5) = .942
100
75
Y
50
25
0
-25
1.0
Y
Judged T ension
2.0
3.0
4.0
5.0
6.0
Predict ed from
EventFB model
7.0
8.0
Predict ed from T PS model
Messiaen
Quartet for the End of Time
Predicting Judged Tension with FB Model
R-squared (18,21) = .823
Y
Predicting Judged Tension with TPS Model
R-squared (2,37) = .758
100
90
80
70
60
50
40
30
20
10
0
0
10
20
30
40
Rows
Y
Judged T ension
Predict ed from FB model
Predict ed from T PS model
Summary:
Fourier Balance model fits judged tension better than
Tonal Pitch Space model
but:
The two models are closely connected
International Congress on Music Perception and Cognition
Bologna, August. 2006
A Proto-Music Theory
from Unsupervised Learning
Carol L. Krumhansl
David G. Rand
Background
Experiments demonstrate listeners are sensitive to
statistically frequent patterns
“Statistical learning”
adults -- cross-cultural (Krumhansl, et al., 1999, 2000)
adults -- melodies on artificial tone-sets (Oram & Cuddy, 1995)
infants -- tone and syllable sequences (Saffran, et al. 1999)
Limitations
Low-order statistics for sequences
P(c1), P(c2 | c1), P(c3 | c1 c2)
c1  c2  c3  …  ck
Results show effects of acculturation only in higher-order statistics
(Krumhansl et al., 1999, 2000)
Automatic discrimination of musical styles only with higher-order
statistics (Krumhansl, 2000)
Coding of music ignores durations of tones
Questions
Can a statistical learning model distill musically interpretable patterns
from a musical corpus?
What does such a model show about the relevance of rhythm to melodic
structure?
ADIOS (Automatic DIstillation Of Structure)
Zach Solan, David Horn, Eytan Ruppin (Tel-Aviv University),
Shimon Edelman (Cornell University)
Two classes of syntax models
Language-specific theory of syntax (generative theory)
General-purpose statistical or distributional learning models
ADIOS has features of both
Distillation of rule-like regularities out of the acquired knowledge
Knowledge acquired only from “raw” distributional information
Representational Data Structure (RDS)
Directed Graph Input: raw unlabeled corpus data (not tagged for part of speech,
only BEGIN and END of each sentence)
Node = one constituent (word)
Directed edge is inserted if transition between constituents exists in corpus
Pattern Acquisition (PA) Algorithm
A Pattern is a similarly structured sequence of constituents that recurs in
the corpus.
An Equivalence Class is a set of constituents from different paths that
occur in the same position in a pattern
This is the syntax that ADIOS is distilling
Bundles are formed when two or more paths run in parallel
and dissolved when more paths leave the bundle than stay in
Criterion for judging pattern significance
For path c1  c2  c3  …  ck:
S = e-(L/k)2 P(c1,c2,…,ck) log( P(k)(path) / P(2)(path)), where:
L = typical context length
k = length of the candidate path
P(k) (path) = P(c1)P(c2|c1)P(c3|c1  c2)…P(ck|c1  c2  c3  …  ck)
“k-gram”
P(2) (path) = P(c1) P(c2|c1) P(c3|c2)… P(ck|ck-1)
“random walk”
Bootstrapping
The identification of new equivalence classes is done using acquired equivalence classes
- “bootstrapping”
Musical Corpus
Musical Themes from Classical Corpus in Themefinder
300 themes in C major, 300 themes in A minor
Monotone (no harmony)
C major scale tones: C D E F G A B
A minor natural scale tones: A B C D E F G
Kern format:
• 2aa 2ee 4dd 8cc 8b 4a 4a 2ee 2b 4a 8g 8f 4e 4e #
2=half note, 4=quarter note, 8=eighth note, etc
G = G below middle C, g = G above middle C, gg= G an octave and a fifth above middle C, etc.
* = begin
# = end
Each duration-pitch pair is treated as an independent constituent
There is NO information given to ADIOS to indicate that:
16a and 8a are the same pitch (with different durations)
16a and 16b are the same duration (with different pitches)
16G and 16g are the same pitch in different octaves (octave equivalent)
Comparison with Musicians Judgments
43 Major, Minor, Major Shuffled, Minor Shuffled, Major Scrambled, Minor Scrambled
11.1 years instruction on musical instruments, 3 or more music theory courses
QuickTime™ and a
TIFF (LZW) decompressor
are needed to see this picture.
Overall Summary
ADIOS distills interpretable structure from musical corpus
Musically interpretable Patterns and Equivalence Classes
Tonal hierarchy is evident in Patterns and Equivalence Classes
When trained on Major, can correctly discriminate Major vs Minor
When trained on Major and Minor, can discriminate Original vs Shuffled
(duration-pitch pairs moved)
ADIOS develops “Proto-Music Theory”
Not fully developed, but sufficient to determine:
Neighbors (pitch proximity)
Same pitch independent of duration
Same duration independent of pitch
Octave equivalent pitches
Harmonically related pitches
Major vs. Minor
Order of duration-pitch pairs
Tension and Emotion
Krumhansl, An exploratory study of musical emotions and psychophysiology
Canadian Journal of Psychology, 1997
Experimental Design
Dynamic Ratings
Sad
Fear
Happy
Tension
Sad Excerpts:
Tomaso Albinoni, Adagio in G minor for Strings and Orchestra
Samuel Barber, Adagio for Strings, Op. 11
Fear Excerpts:
Gustav Holst: Mars -- the Bringer of War from The Planets
Modest Mussorgsky, Night on Bare Mountain
Happy Excerpts:
Antonio Vivaldi, The Four Seasons, La Primavera (Spring), Danza pastorale
Hugo Alfven, Midsommarvaka
Physiological Measures Recorded at 1-second Intervals
1) cardiac interbeat interval (IBI), measured in milliseconds, with shorter IBIs
taken to indicate a higher level of cardiovascular arousal
2) pulse transmission time to the finger (FPTT), measured in milliseconds, with
shorter pulse transmission times indicative of greater autonomic (sympathetic)
activation
3) finger pulse amplitude (FPA), a measure of the amount of blood in the
periphery, with reduced amplitude indicating greater vasoconstriction and
associated with greater autonomic (sympathetic) activation
4) pulse transmission time to the ear (EPTT), another measure of blood flow
5) respiration intercycle interval (ICI), measuring the time between successive
inspirations in milliseconds
6) respiration depth (RD), which is the point of maximum inspiration minus the
point of maximum expiration
7) respiration-sinus asynchrony (RSA)
8) systolic blood pressure (SBP)
9) diastolic blood pressure (DBP)
10) mean arterial pressure (MAP)
11) skin conductance level (SCL), with increased skin conductance indicative of
greater autonomic (sympathetic) activation
12) temperature on the finger (TEM) measured in degrees Fahrenheit.
Correlations between Dynamic Emotion Ratings and Dynamic Physiology Ratings
Sad Ratings
Fear Ratings
Happy Ratings
-.01
-.15***
.16***
-.09**
-.31***
.24***
.15***
.03
-.12***
-.16***
.00
-.09*
-.11***
-.13***
-.23***
-.14***
-.24***
-.14***
-.25***
-.10**
.06
-.08*
-.20***
.21***
Interbeat-Interval
.14***
Finger Pulse Transmission Time
.10**
Finger Pulse Amplitude
-.14***
Ear Pulse Transmission Time
.07*
Respiration Intercycle-Interval
.05
Respiration Depth
.00
Respiration Sinus Asynchrony
-.02
Systolic Blood Pressure
.37***
Diastolic Blood Pressure
.41***
Mean Arterial Pressure
.37***
Skin Conductance Level
-.36***
Finger Temperature
-.35***
Factor analysis of correlations between physiological measures
Distinct groupings
Blood
Pressure
SCL
Temp
Respiration
Rate
Blood Flow Heart
Finger
Rate
Blood Flow
Ear
Tension and Brain
Functional Magnetic Resonance Imaging (fMRI)
The neural mechanisms of tonal and rhythmic expectations
were studied in two ways:
Stimulus
- introducing either tonal or rhythmic violations (or both)
Task
-judge either the tonal structure or the rhythmic structure
-(or passively listen to the melodies)
Musical Sequences: Melodies composed by Diego Vega,
Cornell University (6 sec), piano timbre. Sample sequences:
Original
Tonal Violations
Rhythmic Violations
Tonal and Rhythmic
Violations
Passive Listening: Tonal and Rhythmic Violations
This analysis contrasted
musical sequences containing both
Tonal and Rhythmic violations with
musical sequences with no
violations.
Even when the subjects were in not
performing a task, activations in:
superior temporal cortex
(especially on the right)
Active Judgments: Tonal and Rhythmic Violations
This analysis also contrasted
Musical Sequences containing both Tonal
and Rhythmic violations with musical
sequences with no violations (as before)
Judged whether a violation occurred
Superior temporal activation (as for
Passive Listening), in addition, when
making either Tonality or Rhythm
Judgments:
right dorsolateral frontal right
inferior frontal (bilateral)
Tension and motion
Tension and motion in dance
"The cognitive representation of an event unit involving human motion can be
described as some small set of relatively stable, preparatory motions, followed by
this relatively unstable, completing motion. Such a schematic structure enables
the perceiver to anticipate temporal relations within an unfolding event, and to fit
successive parts of the temporal sequence into this anticipated structure.”
M. Lasher, Cognitive Psychology, 1981.
"The interaction between movement and sound is the most fundamental element of
dance. The dance does not mimic the music -- there is not a particular part of the
music for every gesture and step -- but the basic "kinetic feel" or "energy shape"
of the music is expressed in the dance. The choreographer uses the music not only
for its rhythmic pulse, but also as a source of emotional and structural ideas. Thus,
elements of the music are often observed in the dance.”
K. Teck, Movement to Music, 1990.
Exp erimental Design
Musi c
W. A. Mozart, Div ertimento No. 15 Bb,
Minu etto
Dance
George Balan chin e, School of American
Ball et
Sub jects Musi c Lessons 9.2 years
Dance Lessons 7.3 years
Condi tions
Musi c Only
Dance Only
Both Musi c and Dance
Tasks
du ring
vid eotape
Section End (dis crete judgment)
Tension (continuous judg ment )
New Idea (dis crete judg ment)
Emotion Exp ressed (continuous judg ment)
Task after
vid eotape
Emotion Quali ty (overall judgment)
Intrinsic Relationships Between Music and Dance
Rhythmic
accent, meter, sounds produced by the dancers
Dynamic
volume of musical and choreographic
Textural
number of instruments/performers, homophony versus
polyphony, counterpoint
Structural
corresponding motives or figures, phrases, structures
Qualitative
choreomusical parallels of tessitura, timbre, articulation,
dissonance/consonance
Mimetic
choreography imitates a particular sound in the music
P. Hodges (1992) Relationships between score and choreography in 20th
century dance. London: Mellen.
"To see Balanchine's choreography … is to hear the music with ones eyes…
The choreography emphasizes relationships of which I had hardly been aware…
and the performance was like a tour of a building for which I had drawn the plans
but never explored the result.”
I. Stravinsky, quoted in S. Jordan, Dance Chronicle, 1993
Exploratory Study of Anticipating
Human Movement in Dance
Barycenter
Center of sub-region
including head, trunk,
legs
Stop Position
Target Position
Judged Position
Camurri, Krumhansl, Mazzarino, Volpe, 2004
Tension and motion in music performance
Marcello Wanderley, IRCAM thesis, 2002
Non-obvious performance gestures
(Not needed to produce tone)
Normal Expression, Exaggerated Expression,
Immobile
Experimental Design
Continuous Judgments of Tension
Continuous Judgments of Phrasing
while:
watching performance (no sound)
hearing performance (no image)
both watching and hearing performance
Stravinsky, Second of Five Pieces for Solo Clarinet
Tension judgments in experiment
Phrasing judgments in experiment
Analysis of data with
Functional Data Analysis
J. Ramsay
Audio, Visual, Audio and Visual conditions
similar for judgments of phrasing
But complex interactions between Visual and Audio
in judgments of tension
Vines, Wanderley, Levitin, Krumhansl, Cognition, 2006
Tension in facial and body gesture
Behavior
Cognition
Emotion
Brain
Motion
Music
Cognition
Emotion
Motion
Brain
Tension
L. B. Meyer
Emotion and Meaning in Music
1956
"…expectation is always ahead of the music, creating a
background of diffuse tension against which particular delays
articulate the affective curve and create meaning.”