A Distributed Architecture for Real-time Dialogue and On-task Learning of Efficient Co-operative Turn-taking - Coverbal Synchrony in Human-Machine Interaction - page 304

way. 7 As an indication of the architecture's present scope, Figure 1
presents a full (flat) view of the system's gross architecture. (Note that
while a flat view is informative of the architecture's scope, it belies
its naturally hierarchical nature—an important feature of our system
that allows us to build a complex architecture with a very small team
of developers.) We will discuss the architecture's various components
below. A complete introduction to the architecture is beyond the scope
of this chapter; the main focus of this chapter will be on the parts of
turn-taking needed to support learning of efficient turn-taking.
Following the Ymir architecture (Thórisson, 1996), our system's
modules are categorized based on their functionality; perception-,
decider-, and action modules, at the coarsest granularity (see Figure
2). We will now describe the modules of these types that relate to the
turn-taking system.
3.1 Perception
As already mentioned, although the architecture is inherently a
multimodal system (as shown in prior work (c.f. Bonaiuto and
Thórisson, 2008; Ng-Thow-Hing et al., 2007; Thórisson, 2002b, 1996)),
the current system's input is limited to audio input. There are two
main perception modules that deal with prosodic features in the
system, the Prosody Tracker and the Prosody Analyzer. The Prosody
Tracker is a low-level perception module whose input is a raw audio
signal (Nivel and Thorisson, 2008). It computes speech signal levels
and determines information about speech activity, producing time-
stamped Speech-On and Speech-Off messages. It also analyzes the
speech pitch incrementally (in steps of 16 msecs) and produces pitch
values, in the form of a continuous stream of pitch message updates.
Figure 2. System components, each component consists of one or more modules.
7 This is important because it not only allows the architecture to be more easily
expanded in the future, but also the learning mechanisms, which we will show in
future papers.