Transcript Document

Introduction to Music Informatics
Donald Byrd
Rev. 4 Nov. 2007
Copyright © 2006-07, Donald Byrd
Welcome
• This is a very exciting time for music!
• Music informatics is hard to teach
– Music and technology are both changing
quickly
– Few students have much background in both
– Solution: compromise
• Spend some time bringing each “up to speed”
• Either limit tech. demands in both areas, or…
• Work in teams
• How much background does everyone have?
30 Aug. 2006
2
Class Background: Results 1
• Music: everyone has good background;
several have degrees
• Programming: all over the map
– Few know the R language at all
• Experience w/ music recommenders,
iTunes, etc.: variable, but most don’t
have much
29 Aug. 2007
3
Class Background: Results 2
• Have 10 students, 1 auditor
• Music Theory
– Q1: 10; Q2: 10; Q3: 8 + 2 partial
• Best of any class I’ve ever had
• Audio & Computer Technology
– Q4: 6; Q5: 3 + 7 partial; Q6: 6 + 4 partial
– Q7: 5 + 4 partial; Q8: 1 + 4 partial; Q9: 3 + 4
partial
• Q8 is hard, esp. to say briefly: cf. my
Vocabulary!
rev. 9 Sep. 07
4
Class Plans & Procedures 1
• Communication
– Web site (has many helpful resources)
• …including “Miscellaneous Class Procedures”
– E-mail
– Announcements in class
– OnCourse? not for now
• Goals & competencies
• Assignments & grading: cf. class syllabus
–
–
–
–
Project & presentations most important
Preparation & participation are important
Participation => avoid “death by PowerPoint”
Importance/frequency of quizzes is TBD
rev. 9 Sep. 07
5
Class Plans & Procedures 2
• What about programming?
– More “reading knowledge” than writing
– …but will write some, in R
• Midterm presentation
• Final project: presentation & paper
– List of possibilities to appear “soon”
• Some with programming, some without
– Proposal due in a few weeks
– Presentations last weeks of class
– Paper due last week of classes
• Will have specific grading rubrics
rev. 27 Aug. 07
6
What Is Music Informatics?
• Some definitions
1. Like music information technology, but
more research-oriented
2. Music information retrieval & related areas
• ISMIR = “International Conference
(Symposium) on Music Information Retrieval”,
but… Change name?
3. Discipline #1 plus a curriculum (SoI def.)
• Tim Bell’s overview (2006)
– Moving music from one “form” or “place” to
another (my words)
26 Aug. 2007
7
Visual
display
Digital
image
Human
memory
Digital
semantic
Audio
30 Aug. 2006
8
Visual
display
Digital
image
Weak links
Human
memory
Digital
semantic
Audio
30 Aug. 2006
9
We’ll Consider All Kinds of Music
• Existing music covers a huge range
• I follow Schickele/Ellington philosophy
– Peter Schickele: All musics are created equal
– Duke Ellington: If it sounds good, it is good
• Great, but makes many problems much
harder for computer
– If pop only, can rely on existence of lyrics
– If classical only, can rely on dynamic range
– If pop or folk, can assume texture is melody
& accompaniment
– If many genres, can assume 12 notes/octave
rev. 30 Aug. 07
10
We’ll Consider All Representations of
Music (Preview)
Digital Audio
Audio (e.g., CD, MP3):
like speech
Time-stamped
Time-stamped
Events Events
(e.g., MIDI file): like
unformatted text
Musiclike
Notation
Music Notation:
text with complex
formatting
27 Jan.
11
Music & Research 1
• Different approaches to research
– Highly controlled environments
• More objective, easier to quantify results
– Less controlled
• Better for exploration
– Sim. to “naturalists 1st, scientists later” idea
• Music isn’t a science! But…
– What music “scholars” do similar to what
“scientists” (esp. social scientists) do
– In a way, physical scientists have a much
easier job!
30 Aug. 2006
12
Music & Research 2
• Different approaches to research
– What “scholars” do is similar to what “scientists” (esp.
social scientists) do
– Good reason: music is subtle & subjective
– …but it’s not magic!
• David Huron: Explanatory Goals of Music
Analysis
– Learned as a music student: “methodology is fetish;
rigour is a form of self-deception”
– …as a music scholar: methodology is “simply a way of
internalizing the lessons learned from past scholarly
mistakes”
• The “Scholarly Method”
30 Aug. 2006
13
Music & Research 3
• Example: Gladwell’s article “The Formula”, on
predicting hit songs or movies by computer
–
–
–
–
Not a scholarly article, but is it plausible?
Physical aspects of audio are easy for computers
…but what we care about is almost always perceptual
What are concepts like melody & harmony?
• Core Competency
– “Understand the difference between physical
(objective) and perceptual (subjective) parameters of
musical sound, and why computers can deal much
more easily with the former, while people can deal
much more easily with the latter”
31 Aug. 2007
14
Classification: Logician General’s Warning
• Classification is dangerous to your understanding
–
–
–
–
Almost everything in the real world is messy
Absolute correlations between characteristics are rare
Example: some mammals lay eggs; some are “naked”
Example: is the piano a keyboard, a string, or a percussion
instrument?
• People say “an X has characteristics A, B, C…”
• Nearly always mean “an X has A, & usually B, C…”
• Leads to:
– People who know better claiming absolute correlations
– Arguments among experts over which characteristic is most
fundamental
– Don changing his mind
• But lack of classification is also dangerous to your
understanding!
• Cf. version of this on my Teaching page
rev. 30 Aug. 07
15
Examples of Music Informatics
Research/Technology in Action
• Listen Game
– Nice example of “games with a purpose”
– …but doesn’t seem to be usable anymore!
• sCrAmBlEd?HaCkZ!
– Audio mosaicing
• http://www.popmodernism.org/scrambledhackz/
• "Together" Listening Experiments
– PhD dissertation research by Matt Wright
– on perception of musical rhythm
• “Perceptual Attack Time”
• http://ccrma.stanford.edu/~matt/together/
rev. 12 Sep. 07
16
Our Own Music
• “Music We're Interested In, and What's
Special About It?”
– Everyone contributes something
– Will play & briefly discuss (from standpoint
of music informatics) in class
– Audio files will be on Web site
– Will use for something
• Maybe test lossy compression, or play with
audio-recognition programs?
• Definitely, test audio segmentation program!
– Thanks to Nina Fales for the idea
30 Aug. 2007
17
Materials for Studying Audio 1
• Waveforms & sounds
– Fourier Series Applet (www.falstad.com/fourier/)
– Simple “artificial” waveforms
• Sine, square, triangle, etc.
• Standard on old analog synths (Moog, etc.)
• Sine wave (tuning fork) is the simplest
• Sine function from trigonometry
– Fourier’s theorem
• Any periodic function (repeating waveform) = sum
of harmonically related sine waves
• Add up harmonically related sine ways to make
(approximation to) square wave, etc.
31 Aug. 2007
18
Materials for Studying Audio 2
• What are interesting sounds really like?
– Static waveforms are simple, boring
• Not just sine, square, etc.: cf. the “random
waveform generator” (R demo)
– Acoustic instrument sounds are never static
– …a big reason they’re interesting
• Musical instrument samples
• Audacity audio editor
– For Windows, Mac OS 9 and X, Linux
– Download from
http://audacity.sourceforge.net/
31 Aug. 2007
19
Programming in R: No Problem!
•
•
•
•
R is very interactive: can use as powerful calculator
Assignments will be simple
Much help available: from Don & other students
Why R?
– NOT because it's great for statistics!
– easy to do simple things with it, including graphs and
handling audio files
• probably not good for complex programs
– free, and available for all popular operating systems
– very interactive => easy to experiment
– has good documentation
– Prof. Raphael is using it, and he thinks it's good for music
informatics
– Prof. Raphael is using it, and standardizing is a good thing
30 Aug. 2006
20
Rudiments of R
• Originally for statistics; good for far more
• How to get R
– Web site: http://cran.us.r-project.org/
– Versions for Linux, Mac OS X, Windows
– Already on STC computers & in M373
• Tutorial (kind of math-heavy):
• http://xavier.informatics.indiana.edu/~craphael/teach/sy
mbolic_music/
• Can use R interactively as a powerful graphing,
musicing, etc. calculator
• …but it’s not perfect: sometimes very cryptic
9 Sep. 07
21
Representations of Music & Audio 1
• Three basic forms (representations) of music
– Audio: most important for most people (general public)
– MIDI files: often best/essential for some musicians,
especially for pop, rock, film/TV
– Notation: often best/ essential for musicians (even
amateurs) & music scholars
– Essential difference: how much explicit structure
• Music holdings of Library of Congress: over 10M
items
– Includes over 6M pieces of sheet music and 100K’s of
scores of operas, symphonies, etc.: all notation!
• Differences are profound
rev. 8 Sep. 2006
22
Representations of Music & Audio 2
Digital Audio
Audio (e.g., CD, MP3):
like speech
Time-stamped
Time-stamped
Events Events
(e.g., MIDI file): like
unformatted text
Musiclike
Notation
Music Notation:
text with complex
formatting
1 Sep. 2006
23
Rudiments of Musical Acoustics
• Need some musical acoustics for almost
anything in digital audio
• Need a bit now (use with R), more later
• Acoustics: part of physics
– Concepts like frequency & amplitude
• Psychoacoustics: part of psychology
– Concepts like pitch & loudness (perceptual)
rev. 9 Sep. 07
24
Rudiments of Digital Audio
• Sampling rate => maximum frequency
– Human hearing goes up to ca. 15-20K Hz
– Need 2 samples per cycle
– CDs: 44,100
• Sample width (in bits per sample) =>
Signal-to-Quantization Noise ratio
– About 6 dB per bit
– CDs: 16 bits = ca. 96 dB SQNR
rev. 9 Sep. 07
25
Research in Music Informatics 1
• Scientific method vs. scholarly “method”
– Wide variation in use from field to field
– Music theory doesn’t use scientific method (EJI says)
– …but lots of music informatics does
• Debugging as an example of scientific method
– Hypothesis: program X has no bugs
– Methodology: look for bugs
– …but no amount of testing can prove the absence of
bugs, just their presence
– Cf. Einstein: “No experiments can prove me right; one
experiment can prove me wrong.”
– …& Dijkstra: “Testing can only show the presence of
bugs, never their absence.”
– A theorem (in math) can be proven; a theory (like a
program) can’t!
rev. 4 Nov. 07
26
Research in Music Informatics 2
• Evaluating reliability of info sources
– Especially difficult on the Web: cf. www.dhmo.org
• It’s easy to jump to wrong conclusion: why?
– Backus on why musicians’ explanations in acoustics
are almost always wrong
– Cf. Logician General’s Classification Warning
– Almost everything in the world is messy
• Def. of “trombone” may be more clearcut than of
“piano”
• …but it’s still not well-defined!
rev. 4 Nov. 07
27
Research in Music Informatics 3
• How do you study a question in the most
objective possible way?
– If it’s a matter of perception, maybe by asking people
what they think or hear!
– Example: determining “Perceptual Attack Time” of a
sound
• Cf. D. Huron on what he learned as a music
student vs. as a music scholar
– Methodology as way to avoid repeating mistakes
• Check out my “Information Sources for Music
Informatics Students”
rev. 19 Sep. 07
28