Music Information Retrieval
Download
Report
Transcript Music Information Retrieval
Music Information Retrieval
-orhow to search for (and maybe
find) music and do away with incipits
Michael Fingerhut
Multimedia Library and
Engineering Bureau
IRCAM – Centre Pompidou
IAML - IASA 2004 Congress, Oslo
IRCAM - Institut de Recherche et Coordination Acoustique/Musique
IAML- International Association of Music Libraries
IASA – International association of sound archives
06/28/2009
Presented by: Shailesh Deshpande ([email protected])
Agenda
Introduction
Why MIR?
Take 1: multi-disciplinary domain
Take 2: schematic
Take 3: typology
Challenges
IRCAM cataloging tool
Introduction
Music information retrieval (MIR) is the interdisciplinary
science of retrieving information from music
Paper presents three views of this domain
Challenges
What is an incipit?
First few words or opening line of a book. In music – first few
notes of a composition.
Why MIR?
Storage => increased availability of musical content
in digital form (locally)
CD’s, DVD’s, iPods
Computing power => faster processing of large
volumes of digitized content
Networks => increased availability of musical
content in digital form (remotely)
Pandora,
Yahoo Music, iTunes
Technological advances + demand from consumers =
attention of research and industry
Take 1: multi-disciplinary domain
General
Philosophy and Psychology
Computer Science, Data Processing, AI, Pattern Recognition, Library &
Information Sciences
Sensory Perception, Emotions & feelings, Mental processes & intelligence
Social Sciences
Sociology & Anthropology, Culture & Institutions, Law, Commerce
Natural Science & Mathematics
General Technology
Electric, Electronic, Magnetic, Communications & Computer Engineering
The Arts
Music, Aesthetics, Composition
Take 2: schematic representation of MIR
Take 3: a typology of MIR
Preprocessing
OCR, digitization, compression
Encoding, notation
Feature extraction
Segmentation
Instrument recognition
Voice recognition
Indexing
Identification
Clustering
Classification
Extraction
Melody, Key, Harmony, Rhythm
Structural analysis
Polyphony
Repetition
Similarity
Summarization
Organization
Databases, systems, networks
Compression
Search
Objective criteria
Metadata indices (name, title, period, genre,
instrumentation)
Full-text (with or without semantic tags)
Query by example (audio excerpt, melody,
contour, rhythm, tonality, harmony)
Similarity
Acoustical characteristics
Subjective criteria
Mood
Taste
Retrieve, deliver, use
Browsing
Playlists
Using and reusing (annotate, combine,
transform)
Rights management (recognition, watermarking)
Usability
Evaluation
User studies
Music terms used in MIR
Pitch – perceived fundamental frequency of a sound. Maybe different from actual
frequency because of harmonics.
Timbre – the quality of a musical note that distinguishes different types of sound
production, such as voices or musical instruments (saxophone vs. trumpet – with same
pitch and loudness)
Rhythm (aka beat) - the variation of the length and accentuation of a series of
sounds
Tempo – the speed or pace of a musical piece. Usually affects the Mood of a song.
Melody – a linear succession of musical tones which is perceived as a single entity
(‘horizontal’ aspect of music)
Harmony – simultaneous use of different pitches (‘vertical’ aspect of music)
Monophony – musical texture consisting of melody without accompanying harmony
Polyphony - is a texture consisting of two or more independent melodic voices
Common Methods
Modeling: start from a theory, look for patterns
Look
for melodies, harmonic progressions
Attempt to find elements in data that correspond to such
entities
Statistical methods: look for patterns, build a theory
Perform
statistical analysis on data, find common patterns
and group them in clusters
Attempt to interpret their occurrence in musical pieces
MIR Challenges
The integration of audiovisual, symbolic and textual data
Fingerprinting - unique small set of features excerpted from a
sound file, allowing to discriminate it from any other sound file
Music Summarization- how to select a representative excerpt
that gives a good idea of the work (similar to thumbnails for
image files)
Computing Similarity – no unique way in which two pieces may
be similar
Melodic, Rhythmic, Timbre, Genre, Style similarities
Indexing a musical piece by melody – to allow QBH interface
MIR Challenges contd..
Encoding of music – at acoustic, structural and semantic levels
Query-by-example – search for music by singing, humming,
whistling or playing an audio excerpt
Watermarking – adding identification information to digital
audio for DRM
Benchmarking - limited number of standardized test collections
available for evaluation of MIR systems
A tool to catalog and extract audio
CD contents for online distribution
Automatic identification of CDs
Compute
CDDB of the CD
CDDB - a binary number reflecting the offsets (start
time) and lengths of the tracks of the CD
Metadata retrieval and correction
Query
Internet CDDB for metadata
Allow correction
Extraction and compression
Transfer to a Web server
IRCAM tool interface
When a CD is inserted in the
computer:
-The tool computes its CDDB
- Retrieves the metadata if
available (freedb.org,
cddb.com, allmusic.com)
- Allows the librarian to
correct errors, structure the
tracks into works and select
names from authority lists.
- When done, it adds the
metadata to the catalog, and
extracts the tracks,
compresses them and sends
them to the audio server.
Information sources
The International Society for Music Information Retrieval
(http://www.ismir.net/)
University of Illinois’ Graduate School of Library and
Information Science (http://www.music-ir.org/)
IRCAM (http://www.ircam.fr/)
http://articles.ircam.fr/textes/Fingerhut04b/
The Listen Game — UCSD Computer Audition Lab MIR music
ranking game (Herd It on Facebook)
Multi-player game where you listen to music with lots of other people
(aka the Herd). You are asked to describe the music (genre, mood,
singer etc.) and get points when the Herd agrees with you.
Innovative way to harness the power of social networking and collect
metadata for MIR