TTS-driven Synthetic Behavior Generation Model for Embodied Conversational Agents - Coverbal Synchrony in Human-Machine Interaction - page 338

an enumerative gesture composition rather than direct representation
of words in the form of a sequence of indexical (or iconic) expressions,
as it would be suggested directly by the semiotic rules.
5. Algorithms for Behavior Planning and
Coverbal Synchronization
This section describes in detail the TTS-driven architecture (as
proposed in Figure 1), based on the PLATTOS TTS system. The
TTS system is a corpus-based speech synthesis system, using a
concatenative approach and TD-PSOLA speech synthesis algorithm
(Rojc and Ka`´i`´, 2007). All deques are tied together into a common
engine, using the heterogeneous relation graph structure (HRG) for
representation of all linguistic information (Figure 3). All modules
contribute to the linguistic information used for generating both the
speech signal and the coverbal gestures. One common HRG structure
is used, and is made accessible by all modules in the system. In this
way, processes in given modules are able to access, change, store or
enrich linguistic information when appropriate. The HRG structure
gathers all the linguistic information extracted from the input sentence.
In the final system's modules it is used for generation of the speech
signal (acoustic processing deque), and for generating EVA-SCRIPT-
based behavior descriptions (non-verbal generator deque).
The HRG structure demonstrates the use of both types of relation
structures, in the form of linear lists and trees. Those relation
Figure 3. A complete queuing mechanism with an HRG structure.