Dialogue Acts Recognition - Man-Machine Dialogue: Design and Challenges - page 119

- An index of dialogue conventions, which includes the examples of
composite acts with a set of possible reactions for each act: for example, we
react to an act that has a question and a comment by answering the question,
and maybe by confirming or denying the comment (especially if it is wrong).
- The speaker's preferences, which can be paraphrased as “when he/she
says this it is to do this and that”.
- The task model so as to determine the order of importance of the different
acts present.
- Hypotheses on the user's mental states, so as to favor one act whose
satisfaction will have the greater incidence on these mental states. We here
clearly fall back into the relevance theory with the notion of contextual effect
[SPE 95].
In both cases, the process implemented in the current systems once again
relies on machine learning. For these indirect and composite acts which are
both explicit and implicit, there is an adapted technique, which is supervised
classification, with act labels as hidden classes to be detected [JUR 09].
7.3. Multimodal dialogue act processing
We have seen in section 5.1.4 that certain gestures can carry a dialogue act.
In the case of an MMD system with camera tracking, wide-opened eyes can
be equivalent to a question such as “what did you say?”. If they accompany
a pointing gesture, the communication intent might be something like “what
is it?”. A pointing gesture can also match an order, like an X-shaped gesture
on an object can mean the order to delete this object in the case of a touch
screen system. Finally, and this is the case of most of the expressive gestures, a
gesture can carry out an assertion whose content is added to the simultaneous
linguistic utterance's content according to the multimodal fusion process at a
semantic level (see section 6.1.4). Obviously, if there is no linguistic utterance
accompanying the gesture, its interpretation finishes with the determination of
its single dialogue act. However, in multimodal dialogue, we are faced with
examples in which linguistic utterance is given a speech act, as is the gesture,
whether it is a “telling to”, an “asking” or a “saying that” (we will remain once
againhere withinthe frameworkof relevanttheory)andin whichthe automatic
understanding goes through the processing of these multimodal dialogue acts.