Semantic Analyses and Representations - Man-Machine Dialogue: Design and Challenges - page 91

expressions and the user's general attitude as it is recorded and identified by
the emotion tracking modules and conversational gesture tracking modules.
As an example, a gesture of discouragement, a facial expression close to a
frown of distaste or even a set of vocal and gestural hints that translate the
user's clear irritation (speed of the movements, speech rate and speech
intensity) is a semantic hint which can help clarify the semantic content of the
oral utterance. Two main cases present themselves: either the indication given
by the other modalities are compatible with oral utterances, for example with
the irritation markers compatible with “this will not do, I need a single for
Paris and not Lyon”, and in this case the indications allow us to specify the
mental states of the user (which helps determine the system's reaction), or
these indications do not match the oral utterance, and we are probably in the
presence of an ironic behavior, an understatement or at least a strong allusion
that has to be deciphered and clarified, possibly by asking the user an explicit
question.
Thisconfrontationofsemanticcontentsstemmingfromdifferentmodalities
is called multimodal information fusion and is an aspect that we will recall in
Chapter 7 for the confrontation of dialogue acts, and right now in Chapter 6
for reference resolution.
5.4. Conclusion
In a spontaneous oral dialogue, the user's utterance at the system's input is
characterized by its prosodic, lexical, syntactic and semantic properties. All
these linguistic characteristics create identification and automatic processing
issues and lead to implementing devoted techniques depending on the panel
of phenomena which has been chosen. This chapter summarizes the input
processing and shows how to achieve operational system internal
representations. The emphasis is put on the reconstruction of the implicit and
explicit meaning of the utterance, so that the system reasons on an enriched
semantic representation which is as close as possible to that matching the
user's intention.