Transcript 11 Applications of Machine Learning to Music Research: Empirical

11 Applications of Machine Learning to
Music Research: Empirical Investigations into
the Phenomenon of Musical Expression
99419-811
이인복
Introduction
• Expressive music performance
• Why music?
– A set of difficult learning task
– weak(imprecise, incomplete) domain
knowledge
– the notion of musical knowledge
Expressive music performance
• “Shaping” a piece of music
– not exactly as given in the written score
– continuously varying certain musical
parameters
• dynamics(variations of loudness)
• rubato, expressive timing(variations of local tempo)
• Input - melodies (sequences of notes)
– loudness(dynamics dimension), tempo
• Given new pieces, how loud? How fast?
The nature and importance of
background knowledge
• The task is to learn to ‘draw’ ‘correct’, ‘sensible’
curves above new melodies.
• One symbol alone does not uniquely determine the
numeric value.
• It is not at all clear what the relevant context is.
• Humans possess additional knowledge about the
meaning of the symbols.
• Expression is not arbitrary but highly correlated
with the structure of music.
Approach I : Learning at the note
level
• Learning proceeds at the level of notes.
• The goal is to learn rules that determine the
precise degrees of loudness and tempo to be
applied to each note in a piece.
• Distinguish two classes of notes : rise and
fall
– crescendo, decrescendo
– accelerando, ritardando
The Qualitative domain theory
• Knowledge about relevant musical structure is needed.
• Two major components
– model of structural hearing
• set of programs that perform a structural analysis of a given
melody and explicitly annotate the melody with various
musical structures that are perceived by human listeners.
– qualitative dependency network
• intuitions concerning possible relations between structural
aspects of the music and appropriate expressive performance
decisions
IBL-SMART(1/3)
• Two major component
– symbolic learning component
• learns to distinguish between the symbolic target
concepts(e.g. crescendo and decrescendo)
• utilize domain knowledge in the form of a
quantitative model
– instance-based component
• stores the instances with their precise numeric
attributes
• predict the target value for some new note by
numeric interpolation over known instances
IBL-SMART(2/3)
• Each rule learned by the symbolic
component describes a subset of instances
– These are assumed to represent a subtype of the
target concept(e.g. some particular type of
crescendo situations)
• All the instances covered by a rule are given
to the instance-based learner to be stored
together in a separate instance space.
IBL-SMART(3/3)
• Predicting the target value for some new
note in a new piece involves matching the
note against the symbolic rules.
• Using only those numeric instance
spaces(interpolation tables) for prediction
whose associated rules are satisfied by note.
Experiment
• J S Bach’s Notenbuchlein fur Anna
Magdalena Bach
– Played on an electronic piano and recorded
through a MIDI interface.
• Two part
– learning with the second half
– tested with the first half
Learning at the structure level
• The note level is not really appropriate from
a musical point of view.
– Lacked a certain smoothness
– performers tend to comprehend music in terms
of higher-level abstract forms like phrase
• Alternative approaches are needed.
Learning at the structure level
• Tries to learn expression rules directly at the
level of musical structures.
– Transforms the training examples and the entire
learning problem to a musically plausible
abstraction level.
• Proceeds in two stages.
Learning at the structure level
• The system first performs a musical
analysis of the given melody.
– Analysis routines identify various structures in
the melody that might be heard as units or
‘chunks’ by a listener or musician.
• In the second step, the abstract target
concepts for the learner are identified.
– Tries to find prototypical shapes in the given
expression curves that can be associated with
these structures.
Learning at the structure level
• The results <musical structure, expressive
shape> are passed on to IBL-SMART.
An experiment
• experiments with waltzes by Chopin
• The results look and sound musically
convincing.
A machine learning analysis of
real artistic performances
• Real data - performances of a complete piece by
internationally famous pianists.
– tested with Schumann’s “Traumerei”
– by Claudio Arrau, Vladimir Ashkenazy, Alfred Brendel
– showed considerable agreement in the overall
• Different results
– Vladimir Horowitz’s performance decisions can’t be so
easily related to by obvious structural features of the
music.
Quantitative analysis
• A precise quantitative evaluation of the results is
not possible.
– Simply counting the number of matching decisions is
far too simplistic.
• Apply simple weighting scheme
Useful qualitative results for
musicology
• While abstraction to the structure level generally
provides better results for various types of
classical music, for other styles like jazz the note
level is more adequate.
• Ritardando(Note, X) :- interval_prev(Note, I),
at_least(I, maj6), dir_prev(Note, up).
– Increase the duration(by a certaion amount X) of all
notes that terminate an upward melodic leap of at least
a major sixth
Conclusion
• Music is in many ways ‘softer’
– many aspects ar not quantifiable
– difficult to perform precise experiments
• Machine learning can make useful
qualitative contributions
– thorough analysis of the application domain