Transcript MUSE Music Transcription through statistical analysis

Music Transcription
through Statistical Analysis
Design Phase Presentation
Group 3
Austin Assavavallop, William Feater, Greg Heim,
Philipp Pfieffenberger, Wamba Yves
Music Transcription
• Challenge: Decode note information
from a monophonic music signal
• Overview of chosen approach
–
–
–
–
Noise vs. Signal
Pitch Detection
Timing Analysis
Note Detection
• Development and Implementation
Sound Waves to Tablature
•
•
•
•
Sample stream into frames
Identify events frame-by-frame
Control a state machine with frame events
State machine reports each note’s onset,
velocity, pitch, and pauses.
What’s music, anyway?
• Notes
–
–
–
–
–
Time limited
Periodic
Audible frequencies
Random phase offset
Considerable* signal energy
• Rests
– Not periodic
– Relatively little* signal energy (noise)
*Comparatively
Noise Characterization
• Notes vs. Rests
– High signal energy: Note
– Low signal energy: Rest
• How much energy is due to noise?
– Assume system noise at start of operation
– Calculate Mean, Standard Deviation
• Assume Gaussian noise
– Mean ~ 0
– Expected Noise Amplitude =~ Std. Dev.
• Set threshold for pause
– P(High Signal Energy | only noise) =~ 0
– Threshold ~= 3 * Std. Dev. of noise
Pitch Detection
Sample Frame s(n)
Pitch Detector
Period Np
Pitch Detection
Fundamental Period
Pitch Detection: A4
Pitch Detection
• Assume western scale
f (Octave, Note)  ( f A1 * 21Octave ) * 21 Note/12
– fA1=55Hz, fA2 =110Hz, .. fAN =55*2N
– Determine frequency presence in signal
•
•
•
•
Take advantage of period multiples
Probe signal for all notes of first octave
Probe multiples of closest note
Fundamental = lowest harmonic period
Pitch Detection: Correlation
s(n) : Sampled frame, Np = attempted period
 N /2

min   | s(n)  s(n  N p ) |   N p
 n 0

Caveat: Like all frequency detection techniques, correlation leads to a
tradeoff between the lowest detectable frequency and timing accuracy.
Timing Analysis
Variance V
Sample Frame s(n)
Timing Analysis
Pitch Np
Amplitude A
Pitch Np
Timing Analysis
• Detect frame’s fundamental frequency
• Divide frame into one period blocks
• Measure energy of each block
Timing Analysis
• Variance: Variance of block energies
• Frame Amplitude: Average block energy
System Overview
Frame s(n) = [0..N]
Pitch Detector
Pitch Np
Envelope Analysis
Pitch Np
Amplitude A
Variance V
Start, End time
Controller
Velocity
Pitch
Controller
Pitch Np
Amplitude A
Variance V
Start, End time
Controller
Velocity
Pitch
Controller Logic
ΔNp && ΔV
ΔNp
ΔA
Initialization
ΔA
Stable Note
ΔNp
ΔNp || ΔV
New Note
ΔA && ΔV
Report Event
Only accessible
before the first
Stable Note
ΔA
Rest
ΔA
Implementation
•Design Approach
•Modularization
•Time domain – Timing Analysis, Controller
•Frequency domain – Pitch Detection
•Testing and Integration
•Implementation issues
•Processing within 45.3ms
•Optimization using libraries
•GUI for note interpretation
Questions?