Co-speech Gesture Generation for Embodied Agents and its Effects on User Evaluation - Coverbal Synchrony in Human-Machine Interaction - page 224

development (Goldin-Meadow and Butcher, 2003). Even congenitally
blind people, who have never seen anybody gesturing, spontaneously
produce gestures while talking (Iverson and Goldin-Meadow, 1998).
The important role of gestures in communication is further supported
by the fact that “to date there is no report of a culture that lacks co-
speech gestures” (Kita, 2009, p. 146). The role of gestures is of such
significance that speech-accompanying gestures do not disappear
when visual contact between speaker and listener is absent, e.g. on
the telephone (Cohen, 1977; Bavelas et al., 2008).
As the use of speech-accompanying iconic gestures is such a
ubiquitous characteristic of human-human communication, it is
therefore desirable to endow embodied agents with similar gestural
expressiveness and flexibility to improve the interaction between
humans and machines with regard to naturalness and intuitiveness.
Thus, research is faced with two major problems: first, how to master
the technical challenge to generate flexible conversational behavior
automatically in embodied conversational agents and, second, how
to ensure that the produced synthetic behavior improves the human-
agent conversation valued by human users.
This chapter aims to cover these two issues. First, it will be
addressed how gestural behavior can be automatically generated and
put in combination with speech to be realized in embodied agents,
and second, it will be explored if and how automatically generated
communicative behavior for embodied agents might be benefi cial for
human-machine interaction. Section 2 covers a phenomenological view
on communicative gestures and collects evidence for a number of factors
modulating gesture use which should be considered in computational
models. Section 3 provides an overview of existing computational
simulation accounts and Section 4 presents the Generation Network for
Iconic Gestures (GNetIc; Bergmann and Kopp, 2009a). Finally, Section
5 deals with the question how humans judge automatically generated
gestural behavior.
2. What Shapes Iconic Gestures?—Individual and
Common Patterns in Gesture Use
In Figure 1, examples are given from three speakers who are describing
the same stimulus, a round church window. The speaker on the left-
hand side uses a two-handed gesture in which the shape of the hands
statically depicts the shape of the window. The gesture of the speaker
in the middle is similar such that the hand is shaped in a way that
bears a resemblance with the shape of the window. The difference is
that the one in the middle is performed with only one hand. Finally,