Uncanny Speech - Game Sound Technology and Player Interaction: Concepts and Developments - page 225

and television and the corresponding implications
for characters in computer games.

with a smaller window of acceptable asynchrony
for when sound lagged behind video at 220ms.
Standards set by the television broadcasting in-
dustry require that the audio stream should not
precede the video stream by more than 45ms and
that the audio stream should not lag behind the
video stream by more than 125ms (ITU-R, 1998).
An asynchrony for speech with lip movement
can lead to one misinterpreting what has been said:
the McGurk Effect (1976). As a viewer, one can
interpret what has been heard by what has been
seen. Depending on which modality one's atten-
tion may be drawn to for audio-visual speech (and
depending on which syllable is used), the pronun-
ciation of a visual syllable can take precedence
over the auditory syllable. Conversely a sound
syllable can take precedence over the visual syl-
lable. Alternatively, as one comprehends the visual
articulatory process of speech both automatically
and subconsciously, one can combine the sound
and visual syllable information to create a new
syllable. For example, a visual “ga” coinciding
with the sound “ba” can be interpreted as a “da”
sound. (This type of effect was observed by Mac-
Dorman (2006) for the character Mary Smith's
speech, who was criticized for being uncanny.)
A viewer's overall enjoyment of a television
programme can be disrupted if delays occur be-
tween transmission devices for video and audio
signals. To prevent confusion or irritation for the
viewer, sub-titles are often preferred to dubbing
of speech for foreign works. (Hassanpour, 2009).
Errors in the synchronization of lip move-
ments with voice for figures onscreen (lip sync
error) can result in different responses from the
viewer depending upon the context within which
the errors are portrayed. A study by Reeves and
Voelker (1993) found that not only is lip sync error
potentially stressful for the television viewer, but it
can also lead to a dislike for a particular program
and viewers evaluating the people displayed on the
screen more negatively and as “less interesting,
more unpleasant, less influential, more agitated,
more confusing, and less successful” (p. 4). On the

Lip syncing for television and Film

The process of a viewer accepting that sound
and image occur simultaneously from one given
source is referred to as synchresis (Chion, 1994)
or synchrony (Anderson, 1996). 1 For early sound
cinema, various methods of sound recording and
post production techniques were applied before
a viewer no longer doubted that a voice actually
belonged to a figure onscreen. A perceived lack
of synchronization between image and sound has
been equated with much of the uncanny sensation
evoked by films within the horror genre in early
sound cinema (Spadoni, 2000, pp. 58-60). Errors
in synchrony evoked the uncanny for a scene in
Browning's Dracula (1931). As a figure's lips
remained still, human laughter resonated within
the scene. With no given body or source, the laugh-
ter is regarded as an eerie, disembodied sound.
Whilst technology allows for some improvement
with cinema speakers, televisions and personal
computers, most sound is still delivered through
some mechanism that is physically disjunct from
the onscreen image (for example, via headphones
or separate speakers). Tinwell and Grimshaw
(2010) note that future technologies may overcome
issues with asynchrony within the broadcasting
industry: “Presumably, there will be no need for
such perceptual deceit once flat-panel speakers
with accurate point-source technology provide
simultaneously a visual display” (p. 7).
For human figures in television and film,
viewers are more sensitive to an asynchrony of lip
movement with speech than for visual information
presented with music (Vatakis & Spence, 2005).
Viewers are also more sensitive to asynchrony
when sound precedes video and less so when sound
lags behind video (Grant et al., 2004). Grant et al.
found that for continuous streams of audio-visual
speech presented onscreen, detectable asynchrony
occurred at 50ms when sound preceded video,