Transcript Audio
Audio Sound Audio synthesis Nov 30, Fall 2006 IAT 410 1 Audio Perception Sound: Pressure waves in frequencies between 50Herz - 22,000Herz Lower frequencies more felt by the whole body than heard Sounds can be perceived as coming from a location – Not terribly accurate – Cone of confusion Nov 30, Fall 2006 IAT 410 2 3D Audio Perception Cone of confusion – Cone-shaped zones in front of and behind head 3D Audio cues: – Interaural Time Difference – Interaural Intensity Difference – Pinnae filtering – Body filtering Nov 30, Fall 2006 IAT 410 3 3D Audio Perception Goal for 3D sound is “Spatialization” The sense that the – Sound originates outside your head – Sound has a direction Interaural Time Difference – The more extremely left or right, the greater the difference – Time difference < 5ms Nov 30, Fall 2006 IAT 410 4 3D Audio Perception Interaural Intensity Difference – Head absorbs and reflects sound energy – The first ear to get sound gets loudest sound – “Head Shadow” Cone of confusion: – Time difference too small to detect – Intensity is similar in both ears Nov 30, Fall 2006 IAT 410 5 Pinnae Filtering Outer ear (Pinna) shape filters sound based on its direction Childhood learning trains brain to associate filtering effects with direction Unique per person Record directional white noise – Microphone in ear canal – Sounds from speakers located about head Nov 30, Fall 2006 IAT 410 6 Pinnae Filtering A “Generic” Pinna can be simulated Record directional white noise received by dummy head Body filtering – Reflection and absorption – Included in Pinna model Nov 30, Fall 2006 IAT 410 7 Head-Related Transfer Function HRTF is the general term Transformation of “real” sound to spatialized sound Best delivered by earphones Nov 30, Fall 2006 IAT 410 8 Environmental Effects Sound exists in an environment – Bounces off objects – Is absorbed by objects Simple effects – Reverb: Simulate the environmental echo • Echo is the attenuated signal • Gives a richer room-like feeling • Larger room has longer time delay Nov 30, Fall 2006 IAT 410 9 Audio signals Nyquist limit: – Must sample signal at least twice as frequently as highest reproducible frequency – Audio: 44.1KHz (CD) – 22KHz – 11KHz (Analog AM Radio) – 8KHz (Telephone) Nov 30, Fall 2006 IAT 410 10 Audio - Digital Implications 44,100 Hz – 44,100 Samples/sec – 16-bit samples – Stereo – 172KBytes/sec Specialized Nov 30, Fall 2006 hardware - Sound card IAT 410 11 Reproduction Sampling – Record sounds by whatever means Synthesis – Analog Synthesis – FM Synthesis – Wavetable Synthesis Nov 30, Fall 2006 IAT 410 12 Control MIDI - Musical Instrument Digital Interface Developed to control music synthesizers – Details of synthesis are controlled by synthesizer MIDI data – Sets synthesis parameters – Sets music sequence Nov 30, Fall 2006 IAT 410 13 Synthesis Analog Synthesis – Simple sum of frequencies – Select from a palette of source frequencies – Sum of frequencies is filtered FM – One frequency is controlled by another Wavetable – Digitize audio signals Nov 30, Fall 2006 IAT 410 14 Analog Synthesis Fourier’s observation – Any signal can be created as the sum of sine waves – Square wave: Infinite sum – f + 2f + 4f… Synthesizer: – Collection of oscillators Nov 30, Fall 2006 IAT 410 15 Frequency Domain A perfectly periodic signal plotted in the frequency domain (Time domain plot) Nov 30, Fall 2006 IAT 410 16 Spectrum Spectrum represents the set of frequencies present in the signal Nov 30, Fall 2006 IAT 410 17 Filters Eliminate part of the signal by removing certain frequencies Analog filters don’t have these “square” response shapes Band pass – Bandwidth Nov 30, Fall 2006 IAT 410 18 FM Synthesis Modulate the frequency of a wave Carrier frequency is modulated by Modulator signal Nov 30, Fall 2006 IAT 410 19 FM Synthesis for Synthesizers The greater the Modulator amplitude, the greater the Carrier frequency variation – Higher Carrier bandwidth DX: Carrier and Modulator are “musicallytuned frequency” – Depends on the note you are playing – Controls the harmonic content of a note Nov 30, Fall 2006 IAT 410 20 Wavetable Synthesis Collect a sample of the real sound Issues: – Reduce memory load by looping sample – Shift pitch instead of sampling each individual note – Apply interpolation techniques to make pitch shifting work right Nov 30, Fall 2006 IAT 410 21 Audio Path MIDI Note Sequence Envelope Loudness Control Nov 30, Fall 2006 Raw Sound (Sample, FM, etc) Tuned Mixing/ Combination IAT 410 Resampling Reverb, Environment. Spatialization 22 Wavetable Synthesis Example Leyanda (Guitar) Leyanda (CDShaw) Nov 30, Fall 2006 IAT 410 23 Interactive Sound Goal – Want to enhance the interactive experience – Give the user a sense of presence – Add to the emotional content of the game – Make it more fun Nov 30, Fall 2006 IAT 410 24 Interactive Sound Music Sound effects – Noises – Commentary - Sports Narrative Conversations Nov 30, Fall 2006 IAT 410 25 Interactive Problems Regular music composition has – Beginning – Middle – End Interactive user control makes this difficult – Some genres have this structure Nov 30, Fall 2006 IAT 410 26 Interactive Music Game genres with order – Sports – Racing – Fighting Semi-Ordered – Puzzle – Adventure Nov 30, Fall 2006 IAT 410 27 Music Genres The Infinite Loop – Theme and variation plays forever – Pomp & Circumstance – Diablo Problems: – 30 second piece repeated over 6 hours! – 720 repetitions! – Diablo example: 12 Repetitions/hour Nov 30, Fall 2006 IAT 410 28 Repetition Solutions Make the Dominant theme hard to find – No catchy theme! – Create a variety of textures – Make only transitions stand out Where repetition is small – Don’t repeat musical phrases Nov 30, Fall 2006 IAT 410 29 Music Strategies Play Win or Lose music – Music must be long enough to be meaningful – Music may be so long that the game situation changes before completion – Very short music makes little sense Interrupt current music – Sounds jarring Nov 30, Fall 2006 IAT 410 30 Modules Modular chunks – Each segment of the game plays independently of others – Some thematic relation – Disjointed Nov 30, Fall 2006 IAT 410 31 Music Strategies Compose many themes in parallel Switch between themes Connect modular components together Nov 30, Fall 2006 IAT 410 32 Analogy: Parallel Trains N trains of music running in parallel Each train serves an emotional purpose – Train – Train – Train – Train A: Calm B: Rising Excitement C: Climactic moments D: Falling Excitement Generally, Train A would be most commonly played Nov 30, Fall 2006 IAT 410 33 Within Each Musical Train Each “Car” contains a few bars of music Switch between trains when a “Car” is complete – Don’t switch in the middle of a “Car” Simple version: – Each musical phrase ends on last bar of “Car” Complex: – Notes at end are carried over to next Nov 30, Fall 2006 IAT 410 34 Parallel Trains Bars 1-8 Bars 9-16 Bars 17-24 Bars 25-32 Bars 1-8 Bars 9-16 Bars 17-24 Bars 25-32 Bars 1-8 Bars 9-16 Bars 17-24 Bars 25-32 Bars 1-8 Bars 9-16 Bars 17-24 Bars 25-32 Nov 30, Fall 2006 IAT 410 35 Parallel Trains: Shuffle Cars Shuffle cars – Instead of playing cars in order Problem: Random cars sound like random radio tuning Must determine – Appropriate car pairings – Reasonable paths Nov 30, Fall 2006 IAT 410 36 Repetition In Trains Use repeated phrases carefully Maybe use a statistical tool to analyze paths – Bayesian nets Endings: – Use transitions as an opportunity to “End” – Use next Car to “Begin” new series Nov 30, Fall 2006 IAT 410 37 Composing a Train Create a piece with all layers – Piece can probably survive a layer or two removed – Variation = piece with layer removed Be careful with prominent instruments – Fallback: Use instruments with similar acoustical properties • Piano, Organ, Woodwinds • No Trumpets, Drums or Screaming guitar! Nov 30, Fall 2006 IAT 410 38