An Assessment of the Evolution of Research and Systems - Man-Machine Dialogue: Design and Challenges - page 16

the question should instead be to know for which tasks it would be useful to
implement a dialogue rather than any other kind of interaction technique: the
major hindrance is the low interest of users in using an MMD system.
At the level of research prototypes, the natural dialogue in natural
language becomes feasible, at least within the framework of a targeted task.
This is the case for the closed-domain dialogue and also for some
open-domain demonstrators such as IBM Watson. One should however note
that the recent endeavors have focused on broadening the systems' abilities
rather than developing NLP aspects. We will see this in the three parts
matching the three characteristics of a cognitive system: input processing
(section 1.2.1), the system's internal analyses (section 1.2.2) and output
management (section 1.2.3).
1.2.1. Recording devices and their use
Chapter 2 of [LÓP 05] draws an exhaustive list of the multimodal MMD
systems with processes carried out on inputs. Without drawing up such a list
again, let us quickly mention the following recordings: speech recording; lip-
readingtheusertohelporevenreplacespeechrecognition(noisyenvironment,
disableduser, whispering); userrecognition; facelocationandtracking, aswell
as mouth or eye tracking, and thus eye direction (both to monitor attention in
relation to the dialogue and to help resolve a reference to an object in the
scene); facial emotion recording; pointing gesture recording, especially those
of the hand, and more general kinds of gesture made with the hands or the
body. Moreover, we have already mentioned the force-feedback gesture in the
case of a haptic interaction: this is a device that manages both the recording of
the hand's position and the generation of a potential resistance toward the user.
The point is to couple this device to an immersion in a virtual environment, the
user seeing a graphical representation of his/her hand manipulating objects in
the virtual scene. In this context, the force feedback makes complete sense: it
simulates a touch perception that completes the visual perception.
There is no system that can carry all this out simultaneously and in real
time, but it is an interesting challenge for the more technophile members of
the MMD research community. We can see there are many possibilities and
the computer challenges are vast: the processes matching these types of
recording include many issues falling within the scope of artificial vision,