Information Technology Reference
In-Depth Information
number of incidences and the reasons of a rupture can also be questioned.
These aspects are independent of the modalities and so there is nothing to add
and we finally have a full-fledged multimodal DQR.
As we have mentioned in section 10.1.3, the other materialization of this
methodology is the DCR paradigm [ANT 99], which replaces the assessing
question by a control C, thus minimizing the issue of the system's ability to
answer a question that is at times metalinguistic. The control consists of a
simplification or a reformulation of the initial user request. Thus, taking up
some of the previous examples, we can have multimodal controls intervene in
the following: “put 'submis.tex' in (x
1
, y
1
)”; “move obj
4353
by (x
1
, y
1
)to
(x
2
,y
2
)”; “move obj
4353
according to the points of passage (x
3
, y
3
), (x
4
, y
4
),
(x
5
, y
5
).” Going from a multimodal DQR to a multimodal DCR requires the
paraphrase to be done in a simple and non-ambiguous manner for the
multimodal references, with the description of the spatial coordinates in
natural language. The other aspects do not present any specific problem, or at
least no more a problem than the passage from DQR to DCR.
10.3.4. Towards other assessment methods
The principles of Peace, presented in the article by L. Devillers et al.in
[GAR 02], are the reformulation of the dialogue history into a single sentence,
the use of this sentence for a contextual assessment of the current utterance
and the use of reference semantic representations. We have already mentioned
the difficulty in applying this principle to multimodal dialogue, and thus the
idea of reformulating the history is what we are focusing on here.
Dialogue history modeling is a recurring issue in MMD and is particularly
complex in multimodal dialogue [LAN 04]. As we mentioned in Chapter 6,
the dialogue history must keep both the referent identifiers (to use them when
interpreting an anaphora) and the mentions used to refer to it (to interpret
mentioning references or metalinguistic expressions as well as to interpret
ellipses, and especially noun ellipses). In a multimodal system, it is the same
thing and multimodal referring forms have to be kept, as well as the state of
the visual scene at each stage, thus leading to at least a linguistic history, a
gesture history and a visual history. A reference calling on the modalities used
and the user's memories can then be interpreted, for example “the object
which I have just pointed at”, “the two objects grouped together a bit further
Search WWH ::
Custom Search