Multimodal Output Management - Man-Machine Dialogue: Design and Challenges - page 155

- the message content: choice of the channel best able to render the
information or simultaneous use of several channels if the information is rather
complex;
- the communication act: a single channel for a simple act, two channels
for a composite act;
- interaction history: prioritize the use of a channel that has already been
used;
- user preferences: use a single channel if the user has expressed a
preference in that way;
- human factors: allocate in a long display time a very large piece of
information for which knowledge acquisition requires maintained attention;
take into account the user attention if the system can detect it (in case of
sustained attention, the system does not need to make any specific effort to
explicitly allocate the information over communication channels).
The third stage, which consists of clarifying a link between the modalities,
comes back to the notion of deixis (see section 5.1.4), with terms such as
the demonstrative adverb “here”. When part of the information is displayed
and the other part is verbalized, it can be that the user does not draw a link
between them spontaneously. The danger then is that he/she might consider
that the system is providing him/her with two distinct messages: the two parts
of the information can be of different natures and not depend on each other,
contrary, for example, to a video in which the soundtrack and the image are
immediately seen as two pieces of the same information due to their temporal
synchronization. It can thus be helpful for the system to add an indication
to the semantic content to be transmitted, which helps the user make a link.
This indication is carried by a modality and reminds the user of the existence
of the other piece of information. It can be a visual icon indicating that the
system is speaking. It can be a deictic referring expression that relies on visual
context or an utterance such as “on the geographical map you can see the trains
going to Paris” or “the train changing at Meudon is flashing”. Generating such
utterances requires several processing stages, such as:
- the dialogue manager generates the presentation query “make the
Meudon-Paris itinerary clear to the user”;