Transcript soundhistory

Sound History
A History of Game Sound
•
•
•
•
•
•
A simple sound generator
PC sound (1980)
Amiga/Sega/Nintendo Chip Music
Midi
Wave
MP3
A History of Game Sound
• Why are we looking at history?
• Because it repeats itself.
• We thought we’d seen the last of PC
sound generators when PCs got sound
cards. Then we got mobile phones with
PC-style ring tones. Now mobile phones
have gone to mp3... but what comes next?
A Simple Sound Generator
Parallel Data From PC
Output
Register
LSB
MSB
Gnd
A Simple Sound Generator
• The CPU changes the data on the output
port to alternate the output LSB (least
significant bit) between one and zero.
• This puts a square wave onto the
loudspeaker.
• This technique is so CPU-intensive that it
is seldom used (even though the hardware
is minimal)
PC Sound
• Can play a square wave (sounds a bit like
a violin)
• PC programmed frequency (pitch) and
whether the sound was on or off
• Could only play one note at a time
• Referred to as PC beeper or PC squeaker
• Introduced in original IBM PC in 1980
• Still there on every motherboard.
PC Sound
Parallel Data From PC
System
Clock (415 MHz)
Intel 8253
Programmable Divider
Chip
Square
wave output
Gnd
PC Sound
• One line of music is not as restrictive as it
looks:
• (star1.mid)
PC Sound
• We can add an accompaniment:
• (star2.mid)
Chip Music
• In an attempt to brighten up the
background music for games, a music chip
was added to early games computers,
such as Nintendo, Amiga, Sega Megadrive
etc.
• Typical chips were: General instruments
AY-3-8910 or 8912,Yamaha YM2612 or
YM3812
• They were programmed by the CPU.
Chip Music
• Most chip sounds are synthesised by
simply dividing a clock square wave to get
a square wave of desired frequency
• LFOs (low frequency oscillators) are used
to control or influence a sound parameter
such as pitch or filters in a repeating cycle
to give tremolo or vibrato.
Chip Music
• There were three different sound sources on the
same chip, each of which could be programmed
to give a note (for melody or accompaniment) or
filtered white noise (for percussion)
• Generally chip tunes consist of basic waveforms,
such as sine waves, square waves and sawtooth
or triangle waves, and basic percussion, often
generated from white noise going through an
envelope–controlled synthesizer.
Chip Music
• The white noise for percussion sounds is
generated on-chip using a random number
generator,
• The random number generator is
programmed into the hardware.
• A popular design for a random number
generator is an m-sequence generator.
(see lecture on random numbers)
Chip Music
• The classic chiptune 8-bit sound can be
recognised from:
– Its synthesised square or pulse wave
instruments,
– Simple white noise percussion
– Heavy use of ultra-fast arpeggios to emulate
chords of three or four notes on a single
channel (due to hardware limitations, several
notes must be placed on the same channel).
Chip Music
• There is now a nostalgic music genre
called “Chiptune”.
• Composers programme music chips of the
sort found in a Gameboy.
• (Sabrepulse - Gare de Europe)
– Vibrato using LFOs
– Filtered noise as percussion
– Intricate music hides that only 3 generators
MIDI
• Musical Instrument Digital Interface
• (Also used to control theatre lighting)
• It’s a little like the music rolls you can get
for a steam-organ, or the cylinder of a
musical box.
• It gives instructions “switch this note on at
this volume” and “switch this note off”.
MIDI
• One minute of:
– WAV: ~10.2 Megabytes
– mp3: ~1.1 Megabytes
– Midi: ~ 17 kilobytes
• In other words, a midi file is less than
1/500 of the size of the equivalent WAV
(an mp3 is about 1/10 of the size of a
WAV.)
MIDI
• There are 16 “Channels”. Each channel
can be programmed to correspond to a
different musical instrument.
• There are 127 instruments to choose from
• How these instruments are rendered
(played) depends entirely on the computer
that is playing the file. It might be an
accurate rendition. It might be awful.
MIDI
• To change the instrument of a channel
takes 2 bytes:
• eg: “#C0” means “change channel 1”
• eg: “#0D” means “Xylophone”
• So “change channel 1 to xylophone is
“#C00D” = 1100 0000 0000 1101
MIDI
• There is a byte to turn on a note
• There are 127 notes available, over 9
octaves
• Each note is given a volume of 0 - 127 (0 #7F)
• Instead of volume, midi people talk about
velocity - how fast the key on the keyboard
was pressed. Fast means loud.
MIDI
• “Switch on a note on Channel 5” = “#94”
• Let it be middle C = “#3C” (
• Let it be of medium volume = “#40”
• So you send “#943C40”
MIDI
• ...so we’ve got a “C” running on channel 5.
It will run until we stop it. This is how we
stop it:
• “Stop note #3C on channel 5”: “#843C”
• (there is also a command for “stop
everything”)
MIDI
• There is also a midi wiring standard.
• Midi connectors are 5-pin DIN connectors.
• It is quite common to run a midi cable
between a piano accordion and a midi
synthesiser box.
• The accordion has sensors and a
rudimentary serial interface. The box has
all the fancy electronics.
MIDI
• In olden days, midi synthesisers converted
music by imitating instruments. They
would produce a square or triangular wave
and frequency modulate it with another
wave. (Like they did with chip music.)
• The result - FM synthesis - sounds LIKE
the instrument it was trying to imitate, but it
wouldn’t fool anybody.
MIDI
• Nowadays, MIDI synthesisers use wave
synthesis.
• In wave synthesis, someone would record
a real piano - perhaps even the actual
piano at Abbey Road Studios that was
used to make so many tracks
• This sample would be looped to give an
authentic instrument sound.
MIDI
• The point is that the composition is
divorced from the rendition
• The midi file is like sheet music - it says
what to play, but it gives no guarantee of
quality
• There is a rich variety of midi playback
software and hardware. Quality of
playback ranges from breathtakingly good
to spectacularly awful.
MIDI
• Q: Why am I saying so much about MIDI?
• A: Because the Java platform you are
working with will cheerfully handle MIDI
tunes.
• You don’t need to write your own (unless
you want to - I do.) You can find them all
over the internet.
• Hint: look for rags.
WAV Sound Files
• We hear things because sound pressure
waves move our eardrums backwards and
forwards
• Microphones can convert these pressure
waves into voltages
• The voltages fluctuate at frequencies
between 0 ... 16000 Hz. (Hz used to be
called cycles per second, which made far
more sense.)
WAV Sound Files
• We convert the voltage from the
microphone into digital form by repeatedly
measuring the voltage height.
WAV Sound Files
• A WAV file is simply a list of the heights of
the waves.
• Nyquist determined that if you want to
accurately record audio to a bandwidth of
16kHz (hi-fi), then you need to take
measurements at 32kHz.
• Some applications only need a bandwidth
of 8kHz, needing a sampling rate of
16kHz.
WAV Sound Files
• Why am I telling you all this?
• Well the Java platform for mobile games
will play back WAV files.
• Wav files are good for sound effects. (Or
to put it another way, MIDI is useless for
sound effects.)
• However, one minute of Wav file needs
about 10 Megabytes of memory, so you
can’t store much in a mobile phone.
MP3
• MPEG-1 Audio Layer 3, more commonly
referred to as MP3
• Digital audio encoding format using a form
of lossy data compression.
• Human ears take in sound vibrations as a
time-domain waveform...
MP3
• ... and the Cochlea in the ear turns it into a
frequency representation
MP3
• (voice signal and its spectrum)
MP3
• Nerves from the cochlea carry the signal
frequency information in massively parallel
form to the brain.
• The brain hears frequencies, not
waveforms. The frequencies change
slowly compared to the actual waveform.
• Storing a sound as frequency information
reduces the data needed for storage.
MP3
• There is also the masking effect: quiet
sounds close in frequency to a loud sound
are simply not heard... so you don’t have
to store them.
MP3
• The upshot is that by storing sound as
frequency information, and only storing
what will be heard, the data is cut to 1/10.
Granular Synthesis
• Usually background sound is pre-recorded
and looped.
• This works for music as long as you
provide new music every level - otherwise
it gets repetitive and irritating.
• Soundscapes and sound effects are
trickier - after a while you know what
comes next. Repeated effects become
irritating.
Granular Synthesis
Solution:
• Short samples of effects and music
• Assembled randomly or according to an
algorithm
• Changed in pitch or duration using signal
processing techniques (usually something
to do with a Fourier transform)
• Granular Synthesis
Granular Synthesis
Dialogue
• You’ve got an effect of a guy shouting:
“Hey, over here”
• After a while you and everybody else
notices that it is shouted in exactly the
same way every time
• You can randomly change the speed and
pitch of delivery
Granular Synthesis
Crowd Scenes
• You could make up a crowd from a
number of individuals
• A less computationally expensive way is to
take samples from a crowd and play them
in a random order
• You can also randomise the stereo
panning to distribute them around the
environment
Granular Synthesis
Ambiences: eg Jungle
• River, wind and insect background is built
from a handful of similar samples, mixed in
random order
• Birds and monkeys are distinctive - they
are played at random and with randomised
parameters
Granular Synthesis
Ambiences: eg Jungle
• Soundscape can be made responsive to
game events
• After you fire your rifle, everything goes
quieter for a while, then builds back up.
• Prior to the Unexpected Challenge, the
sound can quieten down for a while to
build the suspense
Granular Synthesis
Music
• Matching sections of music are reordered by an
algorithm or at random.
• In “Diner Dash” there are four major themes.
Each theme consists of a small number of
smaller pieces (2, 4 or 8 bars long), which go
together no matter how they are arranged. What
piece to play is decided by the game-state!!
• (dinerdash.mpg)