Transcript Sound in Games

4.3.SOUND
Use of sound within games
In lecture exploration of answers to frequently asked student questions
Overview of the different uses of sound within games
Use of sound in game
The use of sound within games
includes:
• Music - establishing scene mood,
e.g. uplifting, urgent, peaceful, etc.
• Sound effects - dependent sounds
tied to in-game events, e.g.
collecting power-ups, jumping,
etc.
• Dialog – recorded/generated
speech used within cut-scenes
(furthering plot) or in-game
(improving realism, emersion,
etc.).
The importance of sound
Sound plays a very important role in
setting the mood for the game as well as,
increasingly, how engaging the game is.
For example:
A pinball game without any sound (ball
hits, accumulating score) will appear dull
and lifeless (the player will feel more
detached).
In a 3D game, character movement
without footfall sounds will reduce the
believability/emersion (consciously, or not,
the player will feel movement is not
‘right’).
Basic principles behind managing sound playback
Output channels
Sound hardware typically supports a
maximum number of audio output
channels (simultaneous sounds).
Additionally, sound APIs may support a
maximum number of output channels.
The number of channels defines the
maximum number of simultaneously
playing sounds.
Aside: “The Java Sound API provides
support for the capture, processing,
and rendering of audio and MIDI data.
Java Sound … is a 16-bit, 32-channel
audio rendering and MIDI controlled
sound synthesis engine.”
Output channels (a limited resource? Example 1)
Constraint: You
cannot have more
individual sounds
playing than you have
channels.
Actually, 32+
channels is ample
for most games…
Background music
People talking
Water
sounds
Foot falls
Output channels (a limited resource? Example 2)
What happens if every projectile from a ship is to
make a sound when it hits something?
This could be a problem…
Dealing with a large number of concurrent sounds…
Question: Given a finite number of channels,
how can we deal with a potentially large
number of simultaneous sounds?
Start
10
9
8
7
6
5
4
3
2
Finished
1mins
30
min
mins
sec
Dealing with a large number of concurrent sounds…
Approach 1: Allocate a maximum
number of sound channels that can
be used for a particular type of
sound.
Total 32 channels
Develop a manager that
dynamically monitors channel usage
(imposing sound type channel
restriction if all channels are in use)
Issues: When the playback limit is
reached, how are new requests
handled? Are they ignored? Is
playback of the longest playing
sound in that class cut short? If other
classes have exceeded their
maximum, which one will contract?
Projectile hits
(max 10 channels)
Explosions
(max 6 channels)
Dealing with a large number of concurrent sounds…
Approach 2: As sound waveforms
are additive, software/hardware
can be used to combine separate
waveforms to be then played via a
single output channel.
This effectively provides an
unlimited number of channels, but
with the additional cost of
combining the individual sounds.
Aside: Approach 1 and 2 can be
successfully combined together.
Overview of sound usage in Java
Sound in Java
Sound playback is achieved
using the Java sound API, using:
• SourceDataLine – referring to a
source of data for a mixer (i.e.
output from a program)
• TargetDataLine – referring to a
source of output from a mixer
• Clip – a loaded (not streamed)
sound file.
• LineListener –attached to lines to
monitor LineEvents (i.e. audio
opened, closed, started, or
stopped)
Sound in Java
See the following classes
within the core code
repository:
● SoundAsset
● SoundAssetClip
● SoundAssetAssembly
Asset
GraphicalAsset
SoundAsset
SoundAssetClip
SoundAssetClip
ImageAsset
ImageAssetRibbon
ImageAssetSequence
SoundAssetAssembly
SoundAssetAssembly
ImageAssetTile
DrawnAssetMessage
Overview of sound usage in XNA
Sound in XNA (XAct)
XNA audio is created using the CrossPlatform Audio Creation Tool (XACT).
XACT supports WAV, AIFF, XMA
formats and stero or 5.1 output
XACT provides excellent audio
integration features and supports
streaming support and audio event
notification
Aside: To play MP3s (and similar
formats) it will be necessary to convert
them into a WAV format). This applies
to XNA and Java
Sound in XNA (XAct)
Audio within XACT is structured as follows:
• Wave Banks – a collection of wave files
• Sound Banks – instructions and cues for
the playback of sounds in a wave bank
• Cues – triggering audio playback from a
sound bank. A cue can contain a sequence
of ordered sounds to play, or a list from
which to select a random sound to play.
audioEngine = new AudioEngine(audioParameters);
waveBank = new WaveBank(audioEngine, @“Assets\WaveBank.xwb");
soundBank = new SoundBank(audioEngine, @“Assets\SoundBank.xsb");
soundBank.PlayCue(Name);
Cue cue = soundBank.GetCue(Name); cue.Play();
Summary
Today we
explored:
 Overview of
the use of
sound within
games and
how sounds
can be
managed
 Overview of
Java/XNA
sound classes