TTS-driven Synthetic Behavior Generation Model for Embodied Conversational Agents - Coverbal Synchrony in Human-Machine Interaction - page 341

(p.pos), and the adverb (pp.pos)), and by matching them against
semiotic patterns, a semiotic pattern of an iconic gesture is indicated.
The general patterns, such as adverb-adjective-noun, represent an
emphasis on the physical dimension of the object (a noun). For
instance, the Slovenian verbal phrase 'zelo velika prednost' ( very great
advantage ) emphasizes how huge the advantage was. At this level,
the words and word phrases are also matched against implicit rules.
These rules can override/replace the decisions based on semiotics.
A semiotic pattern verb-verb-verb will result in three isolated verbs,
indicating three 'pointing' gestures (e.g. thinking, watching, and
writing in the text sequence 'I was thinking, watching, and writing').
However, an implicit rule for enumeration will also indicate that such
a sequence represents an enumerative expression. Instead of 'pointing'
gestures, the system will indicate the context of enumeration. The
final processing step of the deque represents the semantic matching
process. In this step, lexical affiliates are chosen for those words/
word phrases identified as carriers of meaning. This process performs
the look-up in the gesture dictionary, and selects an appropriate
physical representation of the word(s). The FSM-based engine hosting
the relations firstly filters the available expressions for the semiotic
type. Secondly, the engine filters the remaining expressions based on
the meaningful word and morphological patterns. If more than one
conversational expression is available at the end, the matching process
selects such a conversational expression that matches the implicit rule,
or semiotic patterns of the phrase, or represents the closest matching
pattern (e.g. at least adjective-noun pattern). Further, the initial pose,
within the preparation phase of the gesture, is chosen based on the
most probable initial state of the meaningful manifestation. The initial
pose is selected based on the following rules: (a) the pose that has
been observed to precede the meaningful manifestation, and (b) the
pose that has been observed to precede the meaningful manifestations
co-align-induced by similar semiotic patterns. The properties of the
coverbal movement are then stored as units within the Word relation
layer. As presented in Figure 5, the deque is able to identify the
left- and the right-word-boundaries of the coverbal expression (e.g.
words 'vedno' ( always ) and 'iste' ( the same )). The new information
within the Content units is stored in the form of attribute-name/
values specification. Each attribute has a predefined set of values that
subsequent processes and deques can select from. For instance, the set
of the predefined values for the attribute semi is based on the semiotic
types the system is able to classify (e.g. iconic, adaptor, symbolic,
etc.). The attribute mwords holds those key-words that may, in the
observed sequence, represent meaning (e.g. word 'iste' ( the same )).