Multimodal Output Management - Man-Machine Dialogue: Design and Challenges - page 154

multimedia presenter must thus be aware that it is highlighting and ordering,
especially when this order was not clarified by the dialogue manager (this is
the case when information is simply displayed in a left to right manner), in
the same way as it has to be aware when it aggregates information and can
thus lead to group references rather than to individual elements. The point is
to warn not only the speech recognition module to adapt the speech model,
but also the semantic analysis module and the reference resolution module, to
let them know of the logic underlying the aggregation's order. When the
system is about to reach the point of understanding the user's utterance, it
then has all the elements to identify the right referent.
9.3. Processes
9.3.1. Allocation of the information over communication channels
We can see with the spread of human factors and the complexity of
pragmatic aspects at the generation level that the presentation of multimedia
information, if it is implemented according to a deep model, involves
numerous parameters and complex, often interdependent processes. Among
them, a key process for multimodal dialogue is the allocation of information
to be transmitted on the communication channels. The first step in this
process consists of taking into account the constraints, the second is taking
into account the preferences and the third, which can potentially lead to new
decisions, consists of linking the components of the information thus
allocated, that is to clarify the links between the modalities.
The first step is prioritizing the constraints. It is a question of taking into
account the constraints that are inherent to the information (visual modality
for a geography map), the constraints linked to the terminal (if it does not have
any speaker, all the utterances in natural language will have to be displayed
in writing, with the morphological and spelling constraints this involves), the
constraints linked to the presentation environment (a noisy atmosphere can
lead the system to forego a vocal modality) as well as the constraints linked to
the user's abilities and roles.
The next step consists of taking into account a set of rules on:
- the urgency or criticality of the message: if one of the parameters is at a
high level, it might be required to use all the communication channels;