Transcript QM CICI Slides

AGICR AGIRC AGCIR AGCRI AGRCI AGRIC ARIGC ARICG ARGIC ARGCI
ARCGI ARCIG
CAIGR CAIRG CAGIR CAGRI CARGI CARIG CIAGR CIARG CIGAR
CIGRA CIRGA CIRAG
CGIAR CGIRA CGCIA CGARI CGRAI CGRIA CRIGA CRIAG CRGIA
CRGAI CRAGI CRAIG
GCIAR GCIRA GCAIR GCARI GCRAI GCRIA GICAR GICRA GIACR
GIARC GIRAC GIRCA
GAICR GAIRC GACIR GACRI GARCI GARIC GRIAC GRICA GRGIA
GRACI GRCAI GRCIA
ICAGR ICARG ICGAR ICGRA ICRGA ICRAG IACGR IACRG IAGCR
IAGRC IARGC IARCG
IGACR IGARC IGCAR IGCRA IGRCA IGRAC IRAGC IRACG IRGAC
IRGCA IRCGA IRCAG
RCIGA RCIAG RCGIA RCGAI RCAGI RCAIG RICGA RICAG RIGCA
RIGAC RIAGC RIACG
RGICA RGIAC RGCIA RGCAI RGACI RGAIC RAIGC RAICG RAGIC
RAGCI RACGI RACIG
A Quantitative Measure of
Melodic Structure:
Computational Infrastructure
and Cognitive Implications
Craig Graci, Cognitive Science Program
State University of New York at Oswego, USA
What?
I will talk about a metric which is intended
to assess the degree to which structural
interpretations of melody are plausible with
respect to tonal theory.
Why?
• The problem of modeling melodic
structure in tonal music has relevance
to the study of listening (Berger, 2004),
performance (Clarke, 2005) and
composition (Marsella & Schmidt,
1999).
• Consequently, the problem of
measuring the degree to which a
model actually captures melodic
structure in tonal music should be of
interest.
Outline
• Introduction
• The Metric: Conception and Evaluation
• Computational Framework for the
Metric
• Cognitive Relevance of the Metric
• Conclusion
Prelude to a Talk
Generative Theory of Tonal Music ✗ Grouping
Well-Formedness Rules ✗ Grouping Preference
Rules ✗ Gestalt Principles of Organization ✗
Knowledge Representation ✗ Structural
Generality ✗ Lisp ✗ Java ✗ Correlation ✗
Analysis of Covariance ✗ Microworld ✗ Cognitive
Artifact
Introduction
The Grouping Problem
• A grouping structure for a melody is
what results from recursively
partitioning the sequence of notes
which constitute the melody into
subsequences of notes.
• The grouping problem for a tonal
melody is to determine a
psychologically plausible grouping
structure for a given melody.
Two Very Different
Grouping Structures
Two Rather Similar
Grouping Structures
GTTM Chapter 3
• Perception - The process of finding
meaningful patterns in sensory
information.
• Gestalt Principles - Ideas (e.g.,
proximetry, similarity, “good form”)
pertaining to how things are perceptually
grouped.
• Grouping Preference Rules - With
respect to grouping in tonal music, the
GTTM GPRs are a manifestation of
various Gestalt principles.
Sample GTTM Grouping
Preference Rule Applications
Proximity
Similarity
“Good Form”
Preference Rule Conflict
Similarity
Symmetry
GTTM GPR Summary
•
•
•
•
GPR 1 Singleton “Avoidance”
GPR 2 Proximity
GPR 3 Similarity
GPR 4 IntensificationGPR 5 SymmetryGPR
6 ParallelismGPR 7 Time-Span and
Prolongational Stability
Gamma
Gamma is a metric which computes the
degree to which a structural interpretation of a
melody is consistent with the Gestalt principles
of perceptual organization, as manifested in
the GTTM GPRs.
The Metric:
Conception and
Evaluation
Definition of Gamma
γ = ω1 γ1 + ω2 γ2 + ω3 γ3 + ω5 γ5 + ω6 γ6
where
• γ is the GPR factor - a function mapping a
i
i
structural interpretation of the melody onto
a real number between 0 and 1.
• ω are weights - real numbers which sum
i
to 1.0
Two Very Different
Grouping Structures
γ = 0.349
γ = 0.622
Two Rather Similar
Grouping Structures
γ = 0.592
γ = 0.560
Claims NOT Made
About Gamma
• Gamma is really good at doing what it is
intended to do (which is to measure the
quality of grouping structures for tonal
melody).
• There is such a thing as a fixed “one
size fits all” metric for judging the quality
of a grouping structure for all tonal
melodies.
Claims Made
About Gamma
• Gamma is a useful tool for investigating
phenomena surrounding the grouping
problem in tonal melody.
• Gamma is useful as an analytical tool for
helping to determine sound grouping
structures in tonal melody, and helping to
learn about structural interpretation.
Example Gamma
Computation
Little Tune (Kavalevsky)
γ = ω1 γ1 + ω2 γ2 + ω3 γ3 + ω5 γ5 + ω6 γ6
= 1.0*0.1 + 0.22*0.42 + 0.22*0.5 + 0.2*1.0 + 0.34*0.75
= 0.667
γ2 computation for a
Little Tune Interpretation
E
D E C
D C
D2 E
D
E C D2 D2 E
D
E C
D
γ2 = 5.41/13 = 0.416
C D2 E
D E
D C2 C2
γ3 computation for a
Little Tune Interpretation
E
D E C
D C
D2 E
D
E C D2 D2 E
D
E C
D
γ3 = 6.93/14 = 0.495
C D2 E
D E
D C2 C2
Computational
Framework
for the Metric
Clay?
• Clay is a simple symbolic language
which can be adapted to manipulate
different sorts of virtual objects.
• Clay has been adapted to manipulate
rectangles, coins and dice, number
sequences, and notes to obtain
Mondrian, Chance, Number Theory,
and Music “Worlds”.
Characteristics of Clay
As a music knowledge representation Clay
possesses a number of significant properties:
• Clay is executable
• Clay is procedural
...
• Clay is “structurally general” with
respect to grouping
Structural Generality
• According to Wiggins and Smaill (2000),
structural generality “measures the
amount of information about musical
structure which can be encoded
explicitly.”
• Clay facilitates study of grouping
structure in tonal melody by virtue of its
ability to explicitly encode melodic
structure.
Clay and the Note
• Clay, as a music knowledge
representation language, features a note.
• The note has lots of properties, including
a scale, pitch (degree within the scale),
duration (with respect to one beat),
amplitude, and timbre.
• Melodies are modeled by playing, resting,
and manipulating the state of the note.
Some “Lower Level”
Clay Primitives
• P - play the note
• R - rest the note
• X2 / X3 / X5 / X7 - expand the duration
• S2 / S3 / S5 / S7 - shrink the duration
• RP / LP - raise/lower the pitch a scale
degree
Lower Level Clay
Interaction Examples
? P P P X3 P S3
C C C C3
? P LP LP P RP P RP P
C\ A/B/C
Some “Higher Level”
Clay
Primitives
• PL - play the note for twice its duration
• PS - play the note for half its duration
• PD - play the note for 1.5 times its
duration
• RP2 / RP3 / RP4 ... - raise the pitch of
the note the number of scale degrees
specified
• LP2 / LP3 / LP4 ... - lower the pitch of
the note the number of scale degrees
specified
Higher Level Clay
Interaction Examples
? RP2 P LP P RP P LP2 P
/E\D\E\C
? RP P LP P RP PL LP
/ D \ C / D2
Clay Programming Example
? G1 = RP2 P LP P RP P LP2 P
? G2 = RP P LP P RP PL LP
? PH1 = G1 G2
? PH1
/ E \ D / E \ C / D \ C / D2
Clay Programming and
Structural Interpretation
? G1 = RP2 P LP P RP P LP2 P
? G2 = RP P LP P RP PL LP
? PH1 = G1 G2
As a rule, a nonprimitive Clay
command corresponds to a group.
MxM: Music Exploration
Machine
• MxM is the host computational
environment for Clay
• Lurking within MxM, right along side
Clay, are MetaClay commands for
displaying, sketching, scoring, and
analyzing Clay commands.
“Little Tune” in Clay
LT = P1 P2P1 = PH1 PH2P2 = PH1 PH3PH1 =
G1 G2PH2 = G1 G3PH3 = G4 G5G1 = RP2 P
LP P RP P LP2 PG2 = RP P LP P RP PL LPG3
= RP PL PL LPG4 = RP2 P LP P RP P LP P
LPG5 = PL PL
Text / Tree
The Score
Gamma as a MetaClay
Command
Gamma X: Gamma with
Explanation
Cognitive Relevance
of the Metric
Two Small Studies
• Study 1:
Correlational study comparing
Gamma and ratings of grouping
structure.
• Study 2:
Quasi-experiment
investigating the role that computational
modeling, informed by Gamma, may
play in developing structural grouping
knowledge and ability.
The Correlational Study
This study was designed to empirically
investigate the validity of Gamma as a
measure of grouping structure.
Method
• Three melodies.
Twenty-six structural
interpretations of each melody.
• Ratings from five musical people.
• Gamma values.
• The correlation between the average
ratings and the values was calculated.
The Three Melodies
German Folk Song
(GFS)
Dona Nobis Pacem (DNP)
Ecossaise (Beethoven)
(ECOS)
The Musical People
• Recording engineer / horn player / Music
Department faculty member
• HCI Graduate Student with MIR
experience
• School teacher / linguist who did chorus
and band throughout high school
• Network administrator who did chorus and
band throughout high school
• Psycholinguistics professor / banjo picker
The “Rating” Procedure
• Introduction / Instruction (30 min)
• Melody 1: Listen/Study (4 min) followed
by Evaluations (26 min)
• Melody 2:
Listen/Study (4 min) followed
by Evaluations (26 min)
• Melody 3:
Listen/Study (4 min) followed
by Evaluations (26 min)
“Marginally Bad”
Example
“Marginally Good”
Example
Little Tune (Kabalevsky)
“Extremely Good”
Example
Little Tune (Kabalevsky)
“Extremely Bad” Example
Little Tune (Kabalevsky)
“OK” Example
Little Tune (Kabalevsky)
Example (GFS)
Interrater Reliability
For DPN, Cronbach’s Alpha = 0.81 for the 5
raters
For GFS, Cronbach’s Alpha = 0.90 for the 5
raters
For ECOS, Cronbach’s Alpha = 0.94 for the 5
raters
Data for Gamma and
Humans
Correlation Values
For the DPN melody, r(24) = 0.816
For the GFS melody, r(24) = 0.832
For the ECOS melody, r(24) = 0.693
In each case, correlation is significant
at the 0.01 level.
The Quasi-experiment
The study was designed to investigate
the role that computational modeling,
informed by Gamma, may play in
developing structural grouping
knowledge and ability.
Participants
Computational Modeling Group: 26
undergraduate students, each enrolled in a
cognitive science course.
Control Group: 17 undergraduate students, each
enrolled in a semiotics course.
Procedure
Background survey - computing/music
Pretest - structure 3 melodies
Lecture - Gestalt principles / melodic grouping
structure
Training (experimental group only) - Computer
modeling with Gamma in mind
Posttest - structure 3 melodies
A Note on the Participants
Some of the students (more than half) have no
computer programming skills.
Some of the students (roughly a third) cannot
read music.
Consequently, computer programming skills and
music reading skills could not be presumed.
A Note on the Pre/Post
Tests
Sloboda (2005) suggests the need for more
‘indirect’ measures to probe structural
awareness in ordinary untrained listeners.
The pretest/posttest methodology in this study
appears to fall squarely into this category of
measure.
Pre/Post Test Questions
Subjects listen to a the melody.
Subjects provide phrase level grouping by boxing
7 to 12 note sequences, while listening to the
melody twice more.
Subjects provide lower level grouping by boxing 1
to 6 note sequences, while listening to the
melody twice more.
The First Pretest Melody
A Participant’s Lower Level
Grouping
The Participant’s Phrase
Level Grouping
STOR File for the
Participant’s Grouping
Structure
/
/
\
\
E E | / F0.5 \ E0.5 \ D0.5 \ C0.5 / D |
G G2 | || \ E E | / F0.5 \ E0.5 \ D0.5
C0.5 \ B | / D \ G2 | || / G G | / A0.5
G0.5 \ F0.5 \ E0.5 / A | A A2 | \ G \ E | ||
/ G0.5 \ F0.5 \ E0.5 \ D0.5 \ C2 | || |||
Reverse Compilation
STOR ➞ Clay
The Structured Score
The Training
• Training for the computer modeling group
consisted of a Clay tutorial phase and a
Clay modeling phase.
• The Clay tutorial phase consisted of 2
hours of prescribed Clay
interaction/programming in a lab setting.
• The Clay modeling phase consisted of
modeling four melodies in Clay as the
main part of a take home exam. The first
melody was “practice”. The others were
“real”.
Training Melodies
• The practice melody consisted of four
bars of Mozart. The real melodies were
Dona Nobis Pacem, the German Folk
Song, and Beethoven’s Eccossaise.
• Students were instructed to try to
maximize the Gamma value of their
model.
• The resulting 26 interpretations of each of
the three melodies were used as the
basis of the correlational study.
Posttest Scores
Results
• An ANCOVA was performed to determine
the effect of the computational modeling
activity on hierarchical structuring of
melodies, using performance on the
pretest measure of hierarchical
structuring knowledge and musical ability
as covariates.
• Participants in the computer modeling
group were better able to structure the
three posttest melodies than participants
in the control group.
• The result was significant at the 0.1 level.
Conclusion
• Clay and MxM support the claim that
computational systems which are
sensitive to representational issues in
music are viable environments for
studying musical phenomena from an
empirical perspective (e.g., Honing,
1993).
• Metrics, such as Gamma, grounded in
Gestalt principles, may have a useful role
to play, not only in the process of
determining sound grouping structure, but
also in the process of learning to
establish and evaluate grouping
structure.