An Assessment of the Evolution of Research and Systems - Man-Machine Dialogue: Design and Challenges - page 21

natural language, have explored different types of logics: propositional logics,
modal logics, temporal logics, description logics and hybrid logics.
1.2.3. System reaction types and their manifestation
Once the system has decided what action to carry out as a reaction to the
user's utterance, it still needs to materialize this action. In the case of a system
which is only written or only oral, it must generate an utterance in natural
language. Natural language generation is a research field in itself [REI 00] that
includes various aspects such as sentence construction and the determination
of referential expressions. We find here the same issues as those involved in
automatic understanding, but in an inverted manner. Even if they have some
linguistic resources in common, the generation methods and algorithms are
specific and are not a simple overturn of their understanding equivalents. In the
case of an oral system, the last step carried out by the system is text to speech,
that is pronouncing the chosen utterance. We can also find here the concerns
of prosody: to look real, the utterance must be pronounced with intonation,
rhythm and even focalization, and all these must be perfectly in keeping with
the system's communication intent.
In the case of a multimodal system, for example when an avatar
graphically represents the machine, the issue includes the gesture generation
and their temporal synchronization with the words of the generated verbal
message. When gesture is possible, the issues of generation, and especially
that of determining referential expressions, reach a new dimension: speech
and gesture complement each other, and the system must choose which part
of the message to allocate to each aspect. Moreover, the graphical design of
the avatar itself is a field that generates important questions about the realistic
aspect of the avatar's physical appearance, its gaze and its movements - eye,
eyebrow, lip movements when an utterance is verbalized - of the head, if
nodding, and more generally of the body. The indications given to the user
through these movements play a role in the man-machine communication,
and it is essential for the various movements that have the same purpose, for
example those of the eyes and the eyebrows that indicate the avatar's attention
level, to be correctly synchronized. The emotions are also transmitted through
gestures and are also a field of research on to themselves, which requires
studies on the typologies of emotions, their relevance in MMD and the way
they should be rendered, not only visually, but also in speech. Finally, in the
case of a system including an MMI and manipulating a vast quantity of data,