Modeling Human Communication Dynamics for Virtual Human - Coverbal Synchrony in Human-Machine Interaction - page 371

the speaker 's unfolding utterance. This requires a natural language
system that can incrementally understand the human speaker 's
utterance. Conversely, the virtual human as it is speaking should
be aware of a human listener 's behavior, responding to nonverbal
signals such as confusion. Together such dynamic interaction suggests
capabilities such as interruption of behavior as well as incremental
understanding and generation of verbal and nonverbal behavior.
3.1 An approach to nonverbal behavior generation
A range of systems have tackled various aspects of these challenges
(refs). Here we discuss one of the approaches: The Nonverbal Behavior
Generator (NVBG) (Lee and Marsella, 2006; Wang et al., 2011). NVBG
automates the generation of physical behaviors for virtual humans,
including nonverbal behaviors accompanying the virtual humans
dialog, responses to perceptual events as well as listening behaviors.
It takes input from the virtual human's knowledge of its task, dialog,
emotional reactions to events and perceptual processes. Modular
processing pipelines transform the input into behavior schedules,
written in the Behavior Markup Language (BML, Kopp et al., 2006)
and then passed to a character animation system (SmartBody, Thiebaux
et al., 2008).
Figure 1 depicts three pipelines used to generate behavior for
the virtual human including coverbal nonverbal behavior, reaction to
events and listening behavior. The sections below discuss these three
Figure 4. Overview architecture for verbal and nonverbal behavior generation in a virtual
human.