Multimodal Dialogue System Assessment - Man-Machine Dialogue: Design and Challenges - page 166

graphical user interfaces, sometimes, the worse is yet to come: these
specialists have their habits and try and find them again or test them on the
new multimodal system. The smallest error, uncertainty, discrepancy between
what they define to be normal and what they are faced with is intolerable.
They are all the more susceptible to question the point of voice command or
multimodal command. In other words, these are the most demanding users
you can imagine, and getting assessment elements from them is a risk. This is
the field, however, in which the main challenge of MMD is a domain of
applied research: with the aim to bring multimodal communication
spontaneity to society, we have to go beyond the research prototypes and aim
to build systems for the general public or transform existing graphical
systems into multimodal systems. In other words, we need to aim for
scalability, with all the qualitative and quantitative challenges that it involves
(see section 3.2.6).
Taking such aspects into account when building multimodal dialogue
systems raises many challenges, and a perfectly trustworthy build is not in the
immediate future. Yet, a true assessment is only relevant in genuine operating
conditions. This might be the reason why the MMD domain is so different
from most of the other NLP domains: the scalability of systems that rely on
the written language and implement fewer sensible software components is
not as delicate as a dialogue system. Some systems, for example seeking and
indexing engines, are even designed, very early on, to process a huge amount
of data in genuine use conditions. Their scalability is thus made easier, and
assessment can be carried out with stability.
10.1.2. Man-machine interface methodologies
Scalability is also mastered in the MMI domain, which can inspire the
MMD since the interaction between the human being and the machine is at
the heart of the interface. As [KOL 93] and [GRI 96] show, an assessment
consists of checking and validating the system. The system is checked to see
if it matches the specifications arising from the definition of needs and
validated if it matches the needs and respects the application domain
constraints. The ergonomic assessment of an MMI consists of ensuring that
the user is able to carry out his task due to the system that is offered to him.
The notion of usefulness determines if the MMI allows the user to achieve his
work goals. This usability accounts for the man-machine interaction quality