TTS-driven Synthetic Behavior Generation Model for Embodied Conversational Agents - Coverbal Synchrony in Human-Machine Interaction - page 337

phrase and, in general, signal the existence of a pause at the end of
the conversational expression. Adaptors (manipulators) are gestures that
address/outline speech-flow and interruptions in the flow. The more
common interruptions are: speakers searching for words in the lexical
dictionary, instances of thinking or re-thinking, etc. These gestures are
usually triggered by fillers (phrases), such as hmm…, oh…, well…,
etc. They are co-aligned with the triggering filler. Additionally, these
gestures also maintain bodily manifestation during the silent period
(if it exists), following the trigger until the next conversational
expression is produced. The word phrases, such as 'ohh…', 'right…',
etc., are usually accompanied by a gestural movement. Mappings
between words (word-types) and adaptors are stored in the form of
FSM-based gestural dictionary relations. Iconic expressions are gestures
illustrating objects, processes, or qualitative meaning. Gestures that
indicate (outline) concrete objects are usually triggered by preposition-
noun (PN), adjective-noun (AN), adverb-adjective-noun (RAN), and
other combinations involving a noun as primary indicator. Gestures
illustrating processes are triggered by similar compositions as object
gestures; however, the noun is replaced by a verb as the primary
indicator. Mappings between words, word-types and word-type-orders
(semiotic patterns), and iconic expressions are also stored in the form
of a gestural dictionary. Most languages require paring of at least
a noun (subject, object, topic, etc.) and verb (predicate, operation,
process, comment, etc.) in order to constitute a well-formed sentence.
In languages, e.g. English and Slovenian, pronouns can also replace
nouns. Further, the communicative dialog also contains words/word
phrases that address/outline speech flow, interruptions in the flow
(e.g. speakers searching for words in the lexical dictionary, instance
of thinking or re-thinking, etc.), and outlining directionality. The most
probable word indicators of such adaptive movement are linguistic
fillers (phrases), such as hmm…, oh…, well…, etc., and determiners
(my, this, your, etc.). Therefore, for the semiotic grammar used in the
PLATTOS system, nouns, verbs, pronouns, and linguistic fillers are
those word-types that indicate a high probability of movement. In
addition to semiotic grammar, also rules indicating implicit (semantic)
relations between words (word phrases) are established. For instance, a
Slovenian word phrase 'in tako naprej' ( and so forth ) is represented by
a circular gesture reaching its maximum over the stressed fragment of
the word 'tako' ( so ) and propagating over the whole phrase (not only
over the word 'tako'). Similarly, a concept of enumeration is better
explained by implicit rules than directly by semiotics. For instance,
when a speaker is enumerating activities, e.g. phrase 'Mislim, gledam
in pišem.' ( I think, watch and write ), the coverbal sequence represents