Co-speech Gesture Generation for Embodied Agents and its Effects on User Evaluation - Coverbal Synchrony in Human-Machine Interaction - page 233

i.e. taking the test case data for previous decisions as a basis. This
way, the quality of decision-making rules was evaluated directly to
avoid dealing with the above-mentioned problem. The mean accuracy
for rule-based choices made in all network's decision nodes is 57.8%
(SD = 15.5). Altogether, given the large potential variability for each
of the variables, the results are quite satisfying. For example, the
mean deviation of the predicted BoH orientation (the direction of the
vector running along the back of hand) is 37.4 degrees, with the worst,
opposite rating corresponding to a deviation of 180 degrees. Note also
that we are using a very strict measure for accuracy as every decision
made with GNetIc is assessed against what has been exactly annotated.
Even minor variation (e.g. 'right' instead of 'right/up') leads to a
mismatch classification. However, even gestures whose features are
partly classified as mismatches may very well communicate adequate
semantic features.
5. How Humans Judge Embodied Agents
Using Co-Speech Gestures
There is increasing evidence that endowing embodied agents with
human-like, nonverbal behavior may lead to enhancements of the
likeability of the agent, trust in the agent, satisfaction with the
interaction, naturalness of interaction, ease of use, and efficiency of
task completion (Bickmore and Cassell, 2005; Heylen et al., 2002). With
regard to effects of co-speech gestures, Krämer et al. (2003) found no
effect on agent perception when comparing a gesturing agent with a
non-gesturing one. The agent displaying gestures was perceived just
as likable, competent, and relaxed as the agent that did not produce
gestures. In contrast, Cassell and Thórisson (1999) reported that
nonverbal behavior (including beat gestures) resulted in an increase
of perceived language ability and life-likeness of the agent, as well
as smoothness of interaction. A study by Rehm and André (2007)
revealed that the perception of an agent's politeness depended on
the graphical quality of the employed gestures. Moreover, Buisine
and Martin (2007) found effects of different types of speech-gesture
cooperation in agent's behavior. They found that redundant gestures
increased ratings of explanation quality, expressiveness of the agent,
likeability and a more positive perception of the agent's personality.
In an evaluation of speaker-specific gesture style simulation, Neff
et al. (2008) reported that the proportion of subjects who correctly
recognized a speaker from generated gestures was significantly above
chance. Recently, Niewiadomski et al. (2010) considered the role of
verbal vs. nonverbal vs. multimodal emotional displays on warmth,