Information Technology Reference
In-Depth Information
- Both references can be generated at the same time as a single gesture,
which might describe a (movement) trajectory or could be solely interpreted
with the two extremities (disappearance and reappearance). The ambiguity
could be important, for example when the movement action leaves a visible
trackonthescreen, atrackthatitselfbecomesapartofthedrawinginprogress.
In the movement hypothesis, additional ambiguity can happen if the task
implements two types of movement: one with no effect on the objects found
on the path it describes, and the other leading to pushing any obstacle away.
These examples show how important action modeling and natural language
underspecification are. To solve them, it is useful to implement a multistage
reference resolution process that contains:
- visual content analysis (analysis of the perceptive groups and the
differences between “that” and the objects already “there”);
- analysis of the gesture trajectories (presence of an avoidance phase, for
example);
- linguistic analyses (verbal semantics, thematic roles and temporal
aspects);
- confrontation of the three analyses thus carried out to solve, in a parallel
manner, the object references and action references (multimodal fusion that
takes the constraints of each modality into account);
- confrontation of the pragmatic analyses, which will be the focus of
Chapter 7.
6.3. Anaphora and coreference processing
We mentioned one of the roles of dialogue history, which is to remember
the mentioned objects as well as the expressions used to point these objects
out, as the dialogue progresses. Thus, it is possible to solve the anaphora and
identify the coreferences. Solving anaphora is a process that is much studied
in linguistics and NLP [MIT 02], which consists of creating a link between
an anaphoric expression and its antecedent. Thus, in “take a cube and put it
in the box”, “it” is an anaphoric expression, i.e. it cannot be interpreted in the
immediate visual context but requires the system to explore the linguistic co-
text so as to find a referent that has already been mentioned and is taken up
Search WWH ::




Custom Search