Transcript Document

Interesting Sounds & Musical
Acoustics & Psychoacoustics
Donald Byrd
29 Feb. 2008
Copyright © 2006-08, Donald Byrd
Rudiments of Musical Acoustics
• Need some musical acoustics for almost
anything in digital audio
• Acoustics: branch of physics that studies sound
(of any kind)
– Concepts like frequency & amplitude
– Besides musical, architectural, ultrasonic (for medicine,
underwater, etc.), other
• Psychoacoustics: study of how sound is
perceived; mostly psychology
– Concepts like pitch, loudness, timbre
• Relationship to physical concepts often roughly
logarithmic
• …but only roughly: always more complex than that
rev. 20 Feb. 08
2
Materials for Studying Audio
• What are interesting sounds really like?
– Sine waves, etc. are boring (cf. addsynenv)
– Sounds of acoustic instruments are “rich”
– Vary in every way: with pitch, loudness, time
• Musical instrument samples
• Audacity audio editor
– For Windows, Mac OS 9 and X, Linux
– Download from
http://audacity.sourceforge.net/
• Programs in (e.g.) R
• addsynenv (additive synthesis demo)
rev. 20 Feb. 08
3
Review: Parameters of Musical Sound
• Four basic parameters of a definite-pitched
musical note
1. pitch: how high or low the sound is: perceptual analog
of frequency
2. duration: how long the note lasts
3. loudness: perceptual analog of amplitude
4. timbre or tone quality
• Above is decreasing order of importance for
most Western music
• …and decreasing order of explicitness in CMN
(Conventional Music Notation)!
10 Sept. 2006
4
Musical Acoustics (1)
• Acoustics involves physics
• Musical (opposed to architectural, etc.) acoustics
–
–
–
–
Frequency (=> pitch)
Amplitude (=> loudness)
Spectrum, envelope, & “other” characteristics => timbre
Partials vs. harmonics
• Psychoacoustics involves psychology/perception
– Perceptual coding (for “lossy” compression: MP3, etc.)
• Timbre
– Old idea (thru ca. 1960’s?): timbre produced by static
relationships of partials, plus envelope
– …but attack often more distinctive than “steady state”!
– Rich (interesting) sounds are complex; nothing is static
– Time domain (waveform) vs. frequency domain (spectrum,
spectrogram) views
rev. 11 Sept. 06
5
Materials for Studying Acoustics
• Most useful
–
–
–
–
–
–
–
Piano (preferably a grand)
String instruments (preferably large & bowed)
Strobe light
Drinking glasses filled to different levels
Simple percussion: sleighbell, “rainegg”, etc.
Recording equipment
Audio editor, spectral-analysis program
• Also helpful
– Other musical instruments
– Tuning fork
– Echoic & anechoic chambers
rev. 20 Feb. 08
6
Acoustic Phenomena
•
•
•
•
Resonance & sympathetic vibration
The harmonic series
Standing waves in a solid (e.g., string)
Interference & beats
– Chorus effect
• Harmonics (on bowed strings) show:
– Modes of vibration
– Nodes
– Relationship to sul ponticello?
• Architectural acoustics
– Resonance (as in I Am Sitting in a Room)
– Standing waves in air
20 Feb. 08
7
Modes of Vibration &
Simple Vibrating Systems
• Waves can be transverse or longitudinal
– Which is water? Which is sound?
• Simple vibrating systems & waveforms
– Tuning fork
• Not many other examples!
– What’s the waveform?
4 April 07
8
Complex Vibrations (1)
• Modes of vibration
–
–
–
–
–
String clamped at both ends vibrates at f, 2f, 3f…
Air column closed at both ends: likewise
…where f = frequency determined by length
Air column closed at one end, open at other: f, 3f, 5f…
…where f = half the frequency of previous case
20 Feb. 08
9
Complex Vibrations (2)
• Resonance
– “When a system that can vibrate at a certain frequency is
acted on by periodic disturbance at same frequency,
vibrations of large amplitude can be produced.” —Backus
– Ex: swing in playground
– Ex: string & body of musical instrument
– Ex: sympathetic vibration
– Def. of resonance: the tendency of a system to oscillate at
maximum amplitude at a certain frequency.
20 Feb. 08
10
How String Instruments
Make Sounds
• Vibrating strings
• Bowing
– Rosin increases friction: bow pulls string, snaps back
– …so motion of string is almost a sawtooth
– Waveform is somewhat like sawtooth => lots of harmonics!
– Special effects involving bow: sul ponticello, col legno, etc.
• Plucking
• Special effects not involving bow
– Harmonics, “natural” & “artificial”
• Player touches string lightly => forces node at that point
• NB: not the usual acoustics sense of harmonics
– Muting
rev. 20 Feb. 08
11
How Woodwinds Make
Sounds
• Vibrating air columns
• Without reeds: flute
– Acts like open column
• With reeds: oboe, clarinet, bassoon
– Clarinet acts like half-open column; others like open
column
– Cause: impedance mismatch w/ outside air(?)
– Result: tube is about same length as flute & oboe, but
it sounds an octave lower
• Overblowing
• What about brass instruments?
20 Feb. 08
12
Psychocoustic Phenomena
• A matter of psychology (and, to some extent,
music theory)
• Basis for auditory illusions
– Shepard tones (a.k.a. endless glissando)
– “Mystery melody”
• Practical application: perceptual coding used in
lossy compression (MP3, WMA, etc.)
– Masking
– Threshhold of audibility
4 April 07
13
Scholars (and others) Beware! (1)
• Plausible (at the time!) assumptions
– Men have more teeth than women (ancient)
• Aristotle, The History of Animals, Book II
– Diseases can’t be transmitted by invisible
organisms (19th century)
– Stomach ulcers can’t be caused by organisms
(20th century)
• What you expect vs. what you see/hear
– Standard use of sponges on kitchen counters
increases harmful bacteria (early 21st cent.)
– New York Times headline: “Dinosaurs Cavort
in Film for Doyle” (1923)
rev. 29 Feb. 08
14
Scholars (and others) Beware! (2)
• What you expect vs. what you hear
– Don & the Kurzweil 250 flute sound
• “Obvious” something was wrong w/ the
sound—until compared to a real flute
– Hammond organ vs. a real pipe organ
• Experts & laypeople couldn’t distinguish
– Don, a famous musician, & K250 handclaps
– Huron on what he “knew” & learned
• R. Moog at Kurzweil & piano touch
rev. 29 Feb. 08
15
Creating Interesting Sounds (1)
• Periodic, nearly-, & non-periodic signals
• Approaches to creating sounds
–
–
–
–
Additive synthesis: like painting
Subtractive synthesis: like sculpture
Sampling: like collage
Others: modulation, physical modeling
• Most possible with analog or digital
hardware, but analog is limited
11 Sept. 2006
16
Creating Interesting Sounds (2)
• Cf. “Electronic Music Tutorial” (Ishkur)
– In Ishkur’s Electronic Music Guide website
• Additive Synthesis
• Fourier's Theorem
– Any periodic signal => sum of harmonically-related
sine waves
• Envelopes: continuous & piecewise linear
– Important special case: “ADSR”
– Attack, Decay, Sustain, Release
• Phase, interference, & beats
– Phase by itself rarely important, but relationships are
– Diagrams & demo in CECM Acoustics Primer, Sec. 8
– Interference between channels or very close partials
rev. 13 Sept. 2006
17
Additive Synthesis & Envelopes
• addsynenv does additive synthesis of up to six partials
–
–
–
–
–
Each has arbitrary partial no., starting phase, "ADSR" envelope
Partial no. can be non-integer => not harmonic
ADSR = Attack/Decay/Sustain/Release (3 breakpoints)
…but addsynenv allows much more complex envelopes
Plays one note with waveform specified by partial nos. & their
envelopes (maybe also phases)
– Simultaneously displays “spectrogram” or “sonogram”
– …but not waveform
– Phase in real world normally has little effect, but can be critical in
recording & digital worlds (e.g., cancellation)
•
•
•
•
Cf. Roads, Computer Music Tutorial
Really additive Fourier synthesis: alternatives are grains
(Wavelet Transform) & Walsh functions
Additive synthesis can’t create aperiodic (non-definite pitch)
sounds
• ...or many realistic attacks
rev. 29 Feb. 08
18
Creating Interesting Sounds (3)
• Early CCRMA (Stanford) studies of acoustic
instrument sounds
– Envelope for each partial with a few segments
– Similar to addsynenv
• Subtractive Synthesis
• Early (analog: Moog, etc.) synthesizer model
• Signal source: sine/square/sawtooth (for
additive)
• …or noise & filters (for subtractive)
• LFO to add vibrato, tremolo, glissando, etc.
• ADSR envelope for entire sound
11 Sept. 2006
19
Periodicity & Definite Pitch
Periodic waveform: clearcut
definite pitch
With sharp corners: much
high-freq. energy
square wave; loud trumpet
Without sharp corners: little sine wave; low, soft flute
high-freq. energy
Almost-periodic waveform:
definite pitch
piano
Somewhat periodic
waveform: complex or
multiple pitches
bells
Aperiodic waveform: noisy,
no definite pitch
cymbal, bass drum
31 Oct. 07
20
Real-World Musical Sounds (1)
• The “Attack/Sustain/Release” (ASR) model for notes
– Attack, Sustain, Release modified from recordings
– Attack includes Decay from ADSR model
• Used in the Kurzweil 250 (1984), etc.
–
–
–
–
Original version had only 2 MB for all samples
Piano had diff. samples for 2 loudness levels
…and diff. sound for every 4-6 semitones
1-2 sec. per sample for A+S+R
• How good did the K250 really sound?
– COUNTDOWN, by Christopher Yavelow
– “An opera for the nuclear age”
• “the ‘orchestral accompaniment’ is in reality a Kurzweil250 digital sampler, synchronized to the baton of the
conductor…”
– http://www.yavelow.com/docs/countdown.html
12 Feb. 07
21
Real-World Musical Sounds (2)
• Nowadays, can afford “unlimited” sustain
• …but also need diff. sounds for many (8?) diff.
loudness levels (multisampling)
– “All Together Now”, Electronic Musician, Jan. 2007
• …and diff. sound for every semitone or two
• W/ unlimited sustain, takes gigabytes just for
piano!
19 Feb. 07
22
Timbre Space & Envelopes (1)
• Timbre space
– Includes all possible timbres
– Not well understood; may be 3-dimensional
• “FourierRandomTimbre” covers only a tiny part
• Some ways to enlarge it:
– Overall envelopes
– Allow (inharmonic) partials
– Envelopes for partials
• Formants show timbre involves context: not
really a property of a note!
• Timbre as "the psychoacoustician's
multidimensional wastebasket category"
29 Oct. 07
23
Timbre Space & Envelopes (2)
• Envelope
– Cf. Wikipedia article on timbre
– Amplitude structure of a sound, so called because sound just
"fits" inside its envelope
• Piecewise-linear envelope
– Envelope that can be drawn with small number of straight
lines & no curves
– Important in music synthesis because can approximate many
complex real-world sounds well with very little data
– Special case: "ADSR" (Attack - Decay - Sustain - Release)
envelopes; common on synthesizers, etc.
– Samplers may combine A & D
29 Oct. 07
24
Uncompressed Audio Files are Big
• 1 byte = 8 bits (nearly always)
• How much data on a CD?
– CD audio is 44,100 samples/channel/sec. * 2
bytes/sample * 2 channels = 176,400
bytes/sec., or 10.5 MByte/min.
– CD can store up to 74 min. (or 80) of music
– 10.5 MByte/min. * 74 min. = 777 MBytes
– Actually more: also index, error correction
data, etc.
22 Sep. 2006
25
Compressed Audio: Lossless & Lossy
• Don’t confuse data compression and dynamicrange compression (a.k.a. audio level
compression, limiting)
• Codec = COmpressor/DECompressor
• Lossless compression
– Standard methods (LZW: .zip, etc.) don’t do much for
audio
– Audio specific methods
• MLP used for DVD-Audio
• Apple & Microsoft Lossless
• Lossy compression
– Depends on psychoacoustics (“perceptual coding”)
16 Feb. 06
26
Specs for Some Common Audio Formats
Format
Encoding Type
Details
Fidelity
“Red Book” (CD)
Uncompressed,
linear
44.1KHz, 16 bit s/sample,
st ereo
Very high
Bandwidth
(Kbps)
ca. 1400
Early game audio
Uncompressed,
linear
22.05KHz, 8 bit s/sample,
mono
Low
176
MLP, Apple
Lossless
Compression, etc.
MP2 (Variat ions1)
Lossless comp.
compression ca. 2:1
Very high
ca. 700
Lossy comp.
compression ca. 3:1
High
ca. 400
MP3 (Variations2),
AAC, WMA
Lossy comp.
compression ca. 7:1 to
over 10:1
High t o
very high
ca. 128-192
compression more than
20:1
Medium
ca. 28-64
AAC (Variations2), Lossy comp.
WMA
13 Feb. 06
27
Psychoacoustics & Perceptual Coding
• Pohlmann, Ken (2005). Principles of Digital
Audio, 5th ed., Chapter 10: Perceptual Coding
• Rationale: much better data compression
• Physiology of ear and critical bands
– Not fixed frequency: any sound creates one or more
critical bands
• Masking
– Depends on relative loudness & frequency
– Noise is much better than pitched sounds
• Threshhold of hearing
– Depends greatly on frequency
22 Sep. 2006
28
Compressed Audio: Lossy Compression
• General method
1. Divide signal into sub-bands by frequency
2. Take advantage of:
• Masking (“shadows”), via amplitude within
critical bands
• Threshhold of audibility (varies w/ frequency)
• Redundancy among channels
• MPEG-1 layers I thru III (MP-1, 2, 3), AAC get better &
better compression via more & more complex techniques
– “There is probably no limit to the complexity of
psychoacoustics.” --Pohlmann, 5th ed.
– However, there probably is an “asymptotic” limit to
compression!
• Implemented in hardware or software codecs
22 Feb. 06
29
Another “Analog vs. Digital” Battle
• Merton, Orren (2006, February). The Sum of All
Tracks. Electronic Musician 22,2, pp. 57-63
• About summing (mixing) with analog vs. digital
hardware
• Tested with excellent hardware, panel of experts
• Methodology not really scientific
• But is this fair? Does it really matter? How can
you do better with subjective phenomena?
– Electronic Musician isn’t a scholarly journal
– Critical: blind study => results probably meaningful
– Cf. http://createdigitalmusic.com/2006/09/06/analogsumming-pm8-for-people-who-dont-trust-softwaremixing/
rev. 25 Sep. 2006
30
Scientific Evaluation of Subjective
Phenomena: Lossy Compression (1)
• Pohlmann, Ken (2005). Principles of Digital
Audio, 5th ed., pp. 403-413
• Discusses perceptual coding performance
evaluation
• Want to evaluate as objectively as possible!
• One idea: measure artifacts
– Difference between original & compressed =>
distortion
– Psychoacoustic model can estimate NMR (Noise-toMasking Ratio)
25 Sep. 2006
31
Scientific Evaluation of Subjective
Phenomena: Lossy Compression (2)
• Pohlmann: “best way to evaluate is to
exhaustively listen”
• Tests with excellent hardware, panel of experts
• …but serious scientific methodology
• Double-blind, ITU-R methods, CCIR 5-pt scale,
>=50 subjects, etc.
• Similar methods used in validation of MP3, AAC,
etc.
• …and ff123 et al’s evaluation of Ogg Vorbis
• Our listening test: at
http://www.informatics.indiana.edu/donbyrd/N
560Site-Fall06/Worst/
25 Sep. 2006
32
Auditory Illusions 1
• Wikipedia article “auditory illusion”
• Deutsch's Musical Illusions
– E.g., Mysterious melody, Chromatic illusion
– http://psy.ucsd.edu/~ddeutsch/psychology/d
eutsch_research1.html
• Visual and Auditory Illusions (from UBC)
– Often analogous illusions
– E.g., Shepard's Tones, Tritone Paradox
– http://www.cs.ubc.ca/nest/imager/contributi
ons/flinn/Illusions/Illusions.html
25 Sep. 2006
33
Auditory Illusions 2: Note, Chord, or
Scale?
• addsynenv whole-tone preset
• Issue is partials each with its own envelope
– Clue: relative frequencies of partials
– Clue: shapes of envelopes
• Try to simplify the sound…
• Do you experience fusion?
– Probably not, but…
rev. 29 Feb. 08
34
Auditory Illusions: Shepard’s Tones
• Also called (less accurately) endless glissando
• Demo on Web
– http://www.cs.ubc.ca/nest/imager/contributions/flinn/Illusions/Illu
sions.html
– Wikipedia version has a more interesting timbre
• Again, it’s partials & envelopes
• Clue: relative freqs. of partials are important,
unusual
• Clue: envelopes of partials are important
• Used by J.C. Risset in computer-synthesized
music (Little Boy, etc.)
– in one case, with rhythm equivalent
rev. 5 Nov. 2007
35
Visual Analogue of Shepard’s Tones
rev. 25 Jan. 2007
36
Auditory Illusions: Mysterious Melody
• Hard to recognize, unless you already
know what it is!
• Clue: probably easier for Americans
rev. 5 Nov. 2007
37