TTS-driven Synthetic Behavior Generation Model for Embodied Conversational Agents - Coverbal Synchrony in Human-Machine Interaction - page 346

Figure 7. The generated movement structure, and sequence of shapes for conversational
expressions.
propagating, two shapes will be manifested. The first shape CU1-P acts
as an initial pose and will manifest itself over the syllables ' ve ' and
' dno '. The preparation movement phase is followed by a hold, indicated
by the sil unit. The stroke shape CU1-S will manifest itself over the
syllable ' is '. The residual syllable ' te ' indicates the post-stroke hold
phase (H). An additional post-stroke hold phase (H) is inserted due to
the presence of the predicted sil unit after the word ' iste '. Similarly, we
can structurally describe the conversational expression CE-2 aligned
with the prosodic boundaries (phrase breaks) of the word phrase WP2.
The resulting movement-phases layer stores the movement structure
of those gestures performed within the observed conversational act.
However, this module does not specify any temporal information for
the coverbal gestures.
5.3 Temporal-sync deque
Finally, the temporal-sync deque temporally aligns speech fragments and
coverbal gestures. As presented in Figure 8, the temporal-sync deque
benefits from these HRG's layers: Content unit relation layer, Segments
relation layer, and Movement phases relation layer. The deque creates
new units, named Phase units stored within the Movement Phases
relation layer of the HRG structure, and enriches the Content units,
stored within Content unit relation layer, with additional attributes.
The first process in the temporal-sync deque is the 'filter process' that
searches for those sil units that have predicted durations 0 ms (can
be sil units following words, and/or phrase-breaks). In order to filter
unnecessary holds, the vertical HRG's relations between the sil unit
(Segments relation layer) and the hold movement-phase are suspended,
and the corresponding Phase unit removed from the Movement
Phases relation layer. In the case shown in Figure 8, the first sil unit
positioned between syllables 'dno' and 'is' has a predicted duration
of 0 ms. Therefore, the connected hold movement phase unit has to
be removed from the Movement-Phase sequence, and a new relation
between neighboring preparation and stroke movement-phases has