Physical Modelling for Sound Synthesis - Game Sound Technology and Player Interaction: Concepts and Developments - page 342

become expensive in terms of both the memory
and processing power required. Cook (2002) goes
on to say: “To have truly realistic continuously
interactive sound synthesis, however, essentially
infinite memory would be required to store all pos-
sible samples” (pp. xi-xii) and “the equivalent of
a truly exhaustive physical model would have to
be computed in order to determine which subset
of the samples and parameters would be required
to generate the correct output sound” (p. xii).
This means that, while using samples can provide
excellent audio in pre-determined situations, in a
fully interactive environment it is not capable of
producing realistic sound for every eventuality
that may occur.
Even if one were to ignore, as game develop-
ers have been doing, the limitation of sample
playback described, there are further drawbacks
with the technique. Firstly, it costs time and
hence money to record a library of samples,
something which is described by Doel, Kry, and
Pai (2001) as “a slow, labor intensive process”
(p. 537). As some sounds will not be specific to
just one game, for example, footsteps or doors
opening and closing, it may be possible to reuse
samples from a previously compiled library but
this leads to another disadvantage of the practice,
which is that repeatedly using the same samples
as often as is necessary can become repetitive to
the listener/gamer. A further disadvantage is that
sound effects can only be included for objects
that are conceived during a game's development.
There is no scope for allowing the user to create,
modify, and then sound his or her own custom
objects. This is becoming more restrictive with
the increasing popularity of games such as Me-
dia Molecule's LittleBigPlanet (Sony Computer
Entertainment Europe, 2008) that feature User
Generated Content (UGC) and developments like
LucasArts' Digital Molecular Matter technology,
which can realistically simulate objects being
broken, hence creating a different combination
of shattered pieces each time.

The solution is to defer some of the audio
processing until runtime when information on the
specific sound producing event is known. This
information is used to drive more realistic audio
production. Director of Audio at Microsoft Game
Studios, Guy Whitmore (2009), explains this in
a statement written on his then upcoming Audio
Keynote speech to the 2009 Develop Conference
in Brighton:

fully-mixed wave files, common in the majority
of games today, are like 2D art; you can only see
one side. They're big and clunky and typically
pre-processed. Yet there's a desire and need to
hear sound and music from all perspectives, from
many directions, in various surroundings, and in
different contexts. In order to effectively react and
adapt to a non-fixed game timeline, audio content
must be authored and kept in smaller component
parts until runtime, when they are assembled and
mixed. So take your entire music studio and sound
design suite, including mixers, synths, samplers,
DSP, mics, speakers; virtualise everything and
make it adequately efficient, then drop it all into
your game authoring environment. Stir in a non-
linear DAW [Digital Audio Workstation], the
right game ties, and some mixing intelligence,
and there you have it; a complete runtime studio,
ready with the flexibility to turn on a dime to meet
the needs of the game.

This technique improves on simply playing
back samples, but researchers wish to take the
idea further. “It's about sound as a process rather
than sound as data” (Farnell, 2008, p. 1). Farnell
goes on to explain the concept of procedural audio:

The sample based data model requires that most
of the work is done in advance, prior to execu-
tion on the platform. Many decisions are made
in advance and cast in stone. Procedural audio
on the other hand is highly dynamic and flexible,
it defers many decisions until runtime. (p. 301)