Emotion, Content, and Context in Sound and Music - Game Sound Technology and Player Interaction: Concepts and Developments - page 245

tion would be found attached to each of
the media components present. For ex-
ample, the publishing house, year of pro-
duction, copyright information, and name
of the game in which a multimedia object
(a sound or otherwise) appears will always
be the same. This information is generally
that which is exclusively available in the
form of meta-data and requires little data
mining to extract
• Exclusive content information. This is
information about the content that can
only be found in a given type of media.
Although the same content information
may appear in multiple instances of that
media, it will generally be exclusive to that
type of media. For example, if we consid-
er the music present in a computer game,
the exclusive content information would
include the tempo, amplitude range, time
signature, spectral representation, self-
similarity measurement, and so on.

As Zhang and Jay Kuo (2001) demonstrated,
it is quite possible to extract and classify a range
of different sound content types from multime-
dia data, especially the kind of mixes found in
traditional entertainment like television shows
and movies. Though their work is focused on the
traditional media of multimedia communication,
the computer game environment is simply a natural
extension of this, with the major difference being
the integration of an element of interactivity.
It is these principles that we hope content analy-
sis allows us to build upon and utilise in the field
of electronic media processing and development.
In particular, we hope that game sound content
can be analysed to provide an enhanced gaming
experience. As a good starting point for consider-
ation, we began to explore the relationship between
visual information and music in electronic media,
to provide an augmented experience when viewing
the visual data. In another of our works (Davies,
Cunningham, & Grout, 2007) we attempted to
generate musical sequences based upon analysis
of digital images: in that particular case, those
of photographs and traditional works of art. The
underlying thoughts and questions that motivated
that research revolved around suggestions such
as: What would the Mona Lisa sound like? We felt
this would also provide additional information for
people who were, for example, visually impaired,
and it could be used to provide added description
and emotional information relating to a particular
still image. It became a logical ethos that the only
way in which this could be achieved would be
to analyse the content of the image, as it is this
that contains the information and components
required to relay the same information but in an
alternative format.
A tool that we have found very effective in
analysing musical content is that of the Audio Simi-
larity Matrix (ASM), based upon ideas initially
proposed and demonstrated by Foote (1999). This
allows a visual indication of the self-similarity,
and therefore structure, of a musical piece. We
suggest further reading into Foote's work as a

The relationship between sound and visual
elements has been a mainstay of the media field
since its inception. Consider the music video and
Hollywood movie. Careful correlation occurs
in these areas between the content presented to
the user in these fields. Prime examples of these
include the synchronisation between actions and
transitions appearing in the visual field and the
sound content. An illustration of this that the
authors find particularly effective is in the open-
ing sequence of the 1977 movie Saturday Night
Fever . This particular scene sees the watcher
treated to shots of John Travolta's feet, pounding
the streets of New York in time to the Bee Gee's
classic Stayin' Alive : a classic in its own right
and an almost ridiculously simple example of the
sound content being combined in the production
of the visual content to produce something that
has a much greater impact than either of the two
individual components.