Multimodal Output Management - Man-Machine Dialogue: Design and Challenges - page 146

To achieve a satisfactory level of realism, the system's behavior can also
integrate certain aspects of spontaneous communication described in Chapters
3 and 5. It can, for example, generate hesitations and repetitions like a human
speaker would [ROS 08, p. 84], or even manage the cognitive load of the user,
as we will see with human factors that are the focus of this chapter, first with
a few general principles to design modules in charge of the system's output
(section 9.1), especially at the pragmatic level of dialogue acts (section 9.2),
then the description of several important processes in multimodal dialogue
(section 9.3).
9.1. Output management methodology
9.1.1. General principles of output multimodality
A multimedia information presenter is meant to translate messages coming
from the dialogue manager taking into account, in the best possible way, the
specific characteristics of the information to be presented (and thus displayed
or verbalized), of the terminal on which the dialogue is carried out, the
physical environment (dialogue in a noisy environment, in a plane, on a
field of operation) and the user. When the information is to be allocated to
several communication modalities, we speak of multimodal, the process with
an opposite goal from that of multimodal fusion described in Chapter 6. The
term information covers natural language or multimodal utterances as well
as data stemming from the application model such as the characteristics of
a set of trains. This information can be given labels describing its status when
taking the task in progress into account: urgency or importance aspect (e.g.
critical).Othercharacteristicscanalsobelabeledorcalculatedbythepresenter
to test the presentation possibilities: discrete or continuous aspect, volume,
complexity number of elements (section 9.1.2). This, in particular, allows
management of totally different natural language utterances and data such as
geographical maps and timetable databases. Rather simplistically, the dialogue
manager decides:
- who: to whom the information has to be presented;
- what: what is the information to present;
- which: which part of the information has to be emphasized;
- where: where can the information be displayed, that is on which devices;
- when: when and for how long must the information be presented.