TTS-driven Synthetic Behavior Generation Model for Embodied Conversational Agents - Coverbal Synchrony in Human-Machine Interaction - page 329

the animation (based on the movements' trajectories, and other motion
functions). ECAs generated by the concepts presented in this work
rely on key-poses, and an automatic key-frame-based generation of
interpolated motion between successive motion-segments. Each group
of such independent motion-segments is regarded as an independent
animation stream, which can be reconfigured (paused, stopped,
remodeled, etc.) at any given frame of the animation.
The Behavior Expression Animation Toolkit (BEAT) (Cassell et
al., 2001) is a rule-based system that can, based on the input text,
generate non-verbal behavior. It is a text-driven behavior model that
enables visual speech synthesis and coverbal behavior generation.
For example, it allows for the animators to input typed text that
they wish to be spoken by an animated human figure. The output is
obtained as appropriate and synchronized non-verbal behaviors and
synthesized speech, both provided in a form that can be sent to a
number of different animation systems (realization engines). In terms
of the SAIBA behavior model, the BEAT toolkit performs the intent
planning and behavior planning stages. The system uses linguistic
and contextual information contained within the text, to identify
the communicative intent and to plan its physical realization, e.g. to
control the movements of the hands, arms, and face, and the intonation
of the voice. The mapping between text to facial, intonational and
body gestures is contained in sets of rules derived from the state of
the art in non-verbal conversational behavioral research. The intent
planning phase is implemented by using Language tagging and Behavior
suggestion modules. The Language tagging module annotates the input
text with the linguistic and contextual information. The automatic
language-tagging process proposed by the toolkit is quite similar to
the TTS-based tagging processes that allow TTS systems to produce
appropriate word phrases along with the prosodic features (e.g.
intonation, accents, etc.).
Similarly, Poggi et al. (2005) and Hartmann et al. (2005) suggest a
concept for the expressive gesture synthesis of an ECA named GRETA.
Their gesture production model is based on the concept of gestural
dictionary that stores different shapes obtained during the annotation
process. These gestural affiliates are then accessible online by using
APML behavior specification language (De Carolis et al., 2004), and
semantically described input text sequences.
Most of the related research, therefore, separates speech and
gesture production, either externally or internally. The co-alignment
of speech and coverbal movement is regarded as an implicit process
of semantically matching words (word patterns) and shapes based
on rules or different context-processing techniques performed by