Transcript ICMPC2006_time-based-retrieval - Music

A Time Based Approach
to Musical Pattern Discovery
in Polyphonic Music
Tamar Berman
Graduate School of Library and Information Science
University of Illinois at Urbana-Champaign
ICMPC 9, Bologna 2006
Musical Pattern Retrieval
• Method and system for musical pattern
discovery and retrieval
• Designed as a tool for music researchers,
scholars and students. Not designed as a
model of human perception
• Yet, analysis of the system’s outputs through
evaluation by humans yields interesting data
for music theory/perception/cognition
research
Musical Pattern Retrieval
• Question: Can we create a search engine
that receives a sung or played melody as
input, and searches for matches in a
music database?
• Answer: Yes
– String matching: McNab et al. (1996)
– N-grams: Downie and Nelson (2000)
– Markov models: Birmingham et al. (2001)
Musical Pattern Retrieval
• Question: Can we create a search engine
that receives a description of a musical
structure as input, and searches for
matches in a music database?
Musical Schemas / Style Structures
• Leonard Meyer describes archetypical
patterns and traditional schemata that are
the “classes” in terms of which particular
musical events are perceived and
comprehended.
• “No melody, however original and
inventive, is an exception to this principle”
(Meyer 1973)
Musical Schemas / Style Structures
• Eugene Narmour (1977) discusses style forms
and style structures, upon which a “stylistic
language” is constructed.
• Style forms are “parametric entities” which
achieve enough closure so we can understand
their functional coherence without reference to
the specific contexts from which they come.
• Style structures can be created from style forms
by arranging them in various contexts “according
to their statistically most common occurrences”
Example: The 1-7…4-3 schema
• Prevalent in 18th century music
• First noted by Meyer (1973) and studied further
by Gjerdingen (1988)
• Consists of two event pairs (*):
– 1-7: The melody descends from the 1st degree to the
7th. The harmony shifts from I to V
– 4-3: The melody descends from the 4th degree to the
3rd. The harmony shifts from V7 to I
• Examples:
– KV543.sib
– KV200.sib
(*) Simplified definition
System for Musical Pattern
Retrieval
• Distinguishing features:
– Support for the description and retrieval of
complex, polyphonic patterns
– Noise resilience: instances will be retrieved
even if embedded within other patterns or
interspersed with other events
– User-friendly interface for pattern specification
– Retrieved instances can be ranked according
to their likelihood of fit to the desired pattern
Retrieving the 1-7…4-3
Retrieving the 1-7…4-3
Sequence Retrieval Example
Mozart, Violin Concerto No. 6 in Eb
K268, Allegro moderato,
measures 24-29 (51.5’)
kv268-1.sib
Test Data
• 505 Midi files of music by W.A. Mozart, taken
from http://www.classicalarchives.com
• Includes symphonies, piano sonatas, piano
concertos, other concertos and piano trios
• Truncated to first 50 measures
• Normalized
• Converted into note objects and then into
time series
Time Series Representation
• A time series is a set of observations on
the value of one or more variables, taken
at successive points in time
• In the musical time series:
– Variables: 12 pitch classes
– Values: role played by pitch class at the time
of observation (top/bass/middle/absent)
• Result: a series of “harmonic windows”
representing each musical piece
Musical Time Series Parameters
• Window length: size (in seconds) of the
time interval described by each harmonic
window
• Onset interval: time (in seconds) between
window onsets (“sampling rate”)
Use of Absolute Time Units
• Motivation:
– Readily and reliably available in midi data
– Potential application to audio data
• Justification:
– For events that are close to each other in time,
seconds – rather than beats – are likely more
relevant
– For fast music, schema events could be further
apart (in beats/measures) than for slow music
System Evaluation
•
•
•
A selection of 115 retrieved candidate
instances were evaluated by 3 human judges
and by 12 queries
The queries differed from each other in
parameters such as window length, onset
interval and role specifications for pitch classes
within each event
Instances that were rated as correct by a
majority of the human judges were rated as
correct by a majority of the queries
=> 100% precision is attainable!
System Performance
Window Length
Onset Interval
Query Type
Precision
1.000
0.500
TV
0.632
1.000
0.500
CB
0.282
0.500
0.500
TV
0.875
0.500
0.500
CB
0.500
0.500
0.250
TV
0.733
0.500
0.250
CB
0.317
0.250
0.250
TV
0.778
0.250
0.250
CB
0.538
0.250
0.125
TV
0.538
0.250
0.125
CB
0.333
0.125
0.125
TV
0.857
0.125
0.125
CB
0.600
N/A
N/A
Majority vote
1.000
Optimal at 0.5 second windows
- Observed by Wundt (1874)
Question
• Do these excerpts sound similar?
– Mozart Clarinet Concerto in A, K622,
beginning of Allegro
– Mozart Piano Concerto No. 6 in Bb, K238,
beginning of Rondo
Similarity
They both contain sequences which satisfy the following
constrains:
1. The first event includes pitches C, E, G with G on top
2. The second event includes pitches C, E with E on top
3. The third event includes pitches F, A, C
4. The fourth event includes pitches C, E
5. The fifth event includes pitches D, F, A
6. The sixth event includes pitches D, F, A with F on top
7. The seventh event includes pitches C, G
8. The eighth event includes pitches G, B, D, F
9. The maximum duration of the sequence is 15 seconds
Conclusions
• Applying simple pitch constraints at
multiple time resolutions yields successful
retrieval
• The “top voice” requirement for melody is
effective
– Observed by Meek and Birmingham (2001)
• Creating a search tool for musical
structures is feasible!
• The technology could be used for similarity
retrieval or theme variations retrieval
Future Work
• Support for constraints on rhythm, contour
and metric placement
• Enabling multiple roles per pitch class
• Describing distance in beats and
measures
• Integration with alternative representations
• Application to audio data
Thank You!