Reference Resolution - Man-Machine Dialogue: Design and Challenges - page 102

linguistically referred to as “that” (i.e. one of the vaguest referring
expressions there is in English) but also concerns a location, with “there”.
This second reference is necessarily accompanied by a gesture and is thus a
case of multimodal reference. The resolution of this reference creates other
issues than those we have seen until now. The nature of the referent is
indicated by “there”, but the exact determination of the referent depends on
several parameters: nature of the action, here a positioning; nature of the
object to be placed, especially its size (putting a nail in a particular place is
not remotely similar to putting a carpet in a particular place, according to the
example given by L. Romary in the context of an interior decoration task);
and nature of the objects already present in the indicated location (see
following section for the setting linked to the action). Moreover, the reference
can be applied to abstract objects, which can be concepts known to the task,
for example “delay” or “cost” as in “delays are unacceptable” or “what is the
cost of this journey?”, or states, actions or processes, as in “being late is
unacceptable” or “how much does this journey cost?”. In general, any full
word can have a reference. In any case, the reference resolution process can
take inspiration from the detailed process for concrete objects, or the one we
will now see for actions.
6.2. Action reference resolution
The example “put that there”, beyond the fact that it has two multimodal
references, refers to an action carried by the verb in the imperative mood.
Depending on the task and the possibilities it presents, a link between the
word “put” and one of the actions that can be carried out by the application is
not necessarily easy or self-evident. For this is the center of the issue: given
an utterance, what function of the application is it referring to? Using the
example of speech-based drawing software, “put that there” can trigger an
action of object movement, an action which is more linked to the verb “move”
than the verb “put”, the latter also being able to trigger the creation of an
object (“put a block there”). Solving the reference thus involves the semantics
of the verb used, its valency (section 6.2.1), as well as the objects in question,
and, in general, the task in progress (section 6.2.2).
6.2.1. Action reference and verbal semantics
The task model groups the list of actions that the application can run.
Solving a reference to an action thus consists of bringing it back to one of the