Graphics Reference
In-Depth Information
CHAPTER 15
Multimodal Fusion in
Human-Agent Dialogue
Elisabeth André, Jean-Claude Martin,
Florian Lingenfelser and Johannes Wagner
1. Introduction
Sophisticated fusion techniques are an essential component of any
multimodal system. Historically, systems aimed at analyzing the
semantics of multimodal commands and typically investigated a
combination of pointing and drawing gestures and speech. The
most prominent example includes the “Put-that-there” system (Bolt,
1980) that analyzes speech in combination with 3D pointing gestures
referring to objects on a graphical display. Since this groundbreaking
work, numerous researchers have investigated mechanisms for
multimodal input interpretation mainly working on speech, gestures
and gaze while the trend is moving towards intuitive interactions
in everyday environments. Since interaction occurs more and more
in mobile and tangible environments, modern multimodal interfaces
require a greater amount of context-awareness (Johnston et al.,
2011).
At the same time, we can observe a shift from pure task-based
dialogue to more human-like dialogues that aim to create social
experiences. Usually, such dialogue systems rely on a personification
of the user interface by means of embodied conversational agents or
social robots. The driving force behind this work is the insight that a
user interface is more likely to be accepted by the user if the machine
Search WWH ::
Custom Search