TTS-driven Synthetic Behavior Generation Model for Embodied Conversational Agents - Coverbal Synchrony in Human-Machine Interaction - page 349

persistence = 0). The duration attribute (equation 3) represents the
overall duration of the coverbal gesture ( n represents the number of
movement phases). The duration-down attribute (equation 4) represents
the duration of the retraction phase ( n represents the number of
segments, k represents the first segment). The Phase item (PI), stored
within the Movement Phases relation layer, in addition to shape
manifestation, also stores the temporal values of the preparation/
stroke movement-phase and the temporal values of the holds that
follow the observed preparation/stroke movement-phase. Attribute
duration-up (equation 5) represents the duration of the preparation/
stroke movement-phase; that is how long the transformation between
shapes has to last ( n represents the number of segments, k represents
the first segment). As can be seen, the attribute's value is calculated
as a sum of the predicted temporal values on those segment units
(phonemes/visemes) contained within the specific preparation/stroke
movement-phase. The attribute persistence (equation 6) is determined
by the hold phase that follows the observed preparation/stroke
movement-phase. This attribute's value is calculated as the sum of the
temporal values for those segments contained within the hold phase.
Figure 9 represents the creation of conversational expressions, and the
temporal alignment of movement on the level of phonemes/visemes.
Figure 9 presents the complex relation that can be efficiently and
flexibly established between Segment relation layer, and the Movement
phases relation layer. The Left-hand side in Figure 9 presents the
relations between coverbal gestures, words, movement-phases, and
phase-items. As can be observed, the conversational expression CE-1
consists of two phase-items, marked as PI1 and PI2. The execution of
CE-1 is finished by a hold (H) movement-phase. The CE-1 is temporally
described by equations 1-3. Since the CE-1 in this case starts with
the first segment of the input text sequence, the CE-1.delay attribute
is set to 0. The CE. persistence attribute's value is determined by the
duration of its last hold movement-phase (equation 2). In this case
the value for CE-1. persistence is 0.068 s. The CE-1.duration attribute's
value (equation 3) is determined by the sum of the PI-1 and PI-2
durations. The PI duration is determined by the predicted duration
of segments it encapsulates. In the case of CE-1, the PI-1.duration
attribute is determined as follows: t('v') + t('e') + t('d') + t('n') +
Figure 9. Temporal alignment of movement on phoneme/viseme level.