TTS-driven Synthetic Behavior Generation Model for Embodied Conversational Agents - Coverbal Synchrony in Human-Machine Interaction - page 343

functional role), and the physical manifestation the meaning may be
represented by. The symbolic synchronization of meaning, therefore,
only indicates a sequence of shapes that adequately illustrate the
meaning of the inputted text. For instance, in the Slovenian phrase ' zelo
velika prednost' ( a very big advantage ), the word ' velika' ( big ) indicates the
meaning; however, in the context of the movement-phases, the stroke
phase would be co-aligned with the word ' zelo' ( very ), and not the word
that is indicated as meaningful ' velika '. The inner fluidity of coverbal
expressions in form of movement-phases is within the PLATTOS
system driven by the symbolic and acoustic prosodic deques (on the
syllable level). The next section describes in detail how movement
structure is then co-aligned with words.
5.2 Inner fluidity deque
The inner fluidity deque performs the temporal alignment of
movement-phases regarding the coverbal gesture, and the input-text
on the level of syllables. It extracts and processes the information
in these HRG layers: Content unit relation, Phrase breaks, Word
relation, and Syllable relation. Additionally, it creates a new layer,
named Movement Phases, for describing the relation between the
propagation of coverbal movement, and input text on the syllable
level. The functioning of this deque and corresponding HRG structure
are presented in Figure 6. Firstly, the starting and ending points of
Content units are compared against prosodic word phrases (logical
content segments), as predicted by the TTS engine's symbolic prosody
deque. In the symbolic prosody deque, the CART-based phrase-break
prediction model uses a B3 label for labeling major phrase breaks, and
a B2 label for minor phrase breaks. A meaningful phrase inducing
coverbal gesture is indicated by prosodic word phrase. Further,
sentences with more than one prosodic word phrase indicate either the
existence of multiple sequential gestures, or additional explanation,
emphasis, or even negation of the preceding meaning.
The fi rst process ' Search for word phrase breaks' adjusts (extends,
or even removes) the starting and ending points of the Content units
based on phrase-break patterns. It aligns Content units with their
predicted prosodic counterparts. A rule applied in this process is that
each prosodic word phrase can contain only one (or none) coverbal
gesture. The movement-propagation of each coverbal expression must
also be maintained within the indicated prosodic word phrase. As seen
in Figure 6, the inner fl uidity deque processes a Slovenian text sequence
' vedno iste in samo iste obraze ' ( always the same, and only the same faces ).
The starting and ending points of both Content units, CU-1 and CU-