Uncanny Speech - Game Sound Technology and Player Interaction: Concepts and Developments - page 214

of synchronization of speech with lip movement
were factors that reduced the overall believability
for this character (Tinwell & Grimshaw, 2010). A
mismatch between the conveyed emotion of Mary
Smith's voice with her gestures and posture exac-
erbated how unnatural and odd the character was
perceived to be. MacDorman (quoted in Gouskos,
2006), observed that a perceived asynchrony of
lip movement with speech was one of the factors
that people found disturbing about Mary Smith:

visually-based, excluding sound as a factor. As
a way towards building a conceptual framework
for the uncanny and virtual characters in immer-
sive 3D environments, this chapter defines how
characteristics for a character's speech may exag-
gerate the uncanny by considering aspects such
as synchronization of audio and video streams,
articulation, and qualities of speech.
The first section provides an exposition of the
Uncanny Valley describing how the theory came
about, previous investigation into the theory and
potential limitations of the theory in relation to
virtual characters.
Previous authors (such as Bailenson et al.,
2005; Brenton, Gillies, Ballin, & Chatting, 2005;
and Vinayagamoorthy, Steed, & Slater, 2005)
have suggested that uncanniness is increased
when the behavioural fidelity for a realistic,
human-like character does not match up with
that character's realistic, human-like appearance.
The second section discusses how a cross-modal
mismatch between a character's appearance and
speech may exaggerate the uncanny. For instance
whether a character's speech may be perceived
as belonging to a character or not, based on that
character's appearance.
The third section discusses how particular
qualities of speech such as slowness of speech,
intonation and pitch and how monotone the voice
sounds, may influence perceived uncanniness and
how such qualities might work to the advantage
of those characters intended to elicit an eerie
sensation.
The results from the UM study (Tinwell &
Grimshaw, 2010) revealed a strong relationship
between how strange a character is perceived
to be and the lack of synchronization of speech
and lip movement. (Characters rated as close to
perfect synchronization for lip movement and
speech were perceived as less strange than those
with disparities in synchronization.) The fourth
section reviews the findings from this study and
also puts forward future experiments that may

In addition, there is sometimes a lack of synchro-
nization with her speech and lip movements, which
is very disturbing to people. People 'hear' with
their eyes as well as their ears. By this, I mean
that if you play an identical sound while looking
at a person's lips, the lip movements can cause
you to hear the sound differently.

Since Mary Smith was revealed in 2006,
increasing technological sophistication for com-
puter games has allowed for heightened realism
of human-like characters. Cinematic animation
is achieved not only for cut scenes and trailers
containing full motion video (FMV) but also for
animation during in-game play. For example,
the phoneme extractor and facial expression tool
Faceposer designed by Valve for titles such as Left
4 Dead (2008) and Half Life 2 (2008). However
it would seem that speech, as a factor integral to
the uncanny phenomenon, is often overlooked
when compared to the aesthetic visual qualities of
behaviour of a human-like character. So far there
have been limited studies to ascertain which factors
contribute to the uncanny for virtual characters.
In response to the hearsay in mass media raised
by characters such as Mary Smith, Tinwell and
Grimshaw (2010) conducted a study to investigate
how the cross-modality of image and sound might
exaggerate the uncanny. The results from this study
are referred to throughout all sections within this
chapter as the Uncanny Modality (UM) study, un-
less otherwise stated from another study. Prior to
this, much of the work on the uncanny had been