Graphics Reference
In-Depth Information
performed offline, i.e. after the interaction between the human and
the agent. Such an approach may, however, be problematic because
the participants of an experiment might have forgotten what they
experienced at a particular instance of time. As an alternative, Gilroy
et al. (2011) present an evaluation approach which compares the results
of the fusion process with the users' physiological response during
the interaction with the system.
4.7 Social signal fusion in human-agent interaction
Starting the recent years, various attempts have been made to explore
the potential of social signal processing in human interaction with
embodied conversational agents and social robots.
Sanghvi et al. (2011) analyzed body postures and gestures as an
indicator of the emotional engagement of children playing chess with
the iCat robot. They came to the conclusion that the accuracy of the
detection methods was high enough to integrate the approach into
an affect recognition system for a game companion. Even though
the approach above addressed an attractive scenario for multimodal
social signal fusion, it was only tested in offline mode. An integration
of the approach into an interactive human-robot system scenario did
not take place.
Increasing effort has been made on the multimodal analysis of
verbal and nonverbal backchannel behaviors during the interaction
with a virtual agent. An example includes the previously mentioned
artificial listener by Gratch et al. (2007) that aims to create rapport with
a human interlocutor through simple contingent nonverbal behaviors.
A more recent example is the virtual health care provider recently
presented by Scherer et al. (2012). This agent is able to detect and
respond multimodal behaviors related to stress and post-traumatic
stress disorder. For example, when the patient pauses a lot in the
conversation, the agent tries to encourage her to continue speaking.
Even though both systems are able to analyze multiple modalities, they
do not seem to employ a fusion engine, but rather directly respond to
cues conveyed in a particular modality, such as a head nod.
An exemplary application that is based on a fusion algorithm
adapted to the specific needs of online processing is the Affective
Listener “Alfred” developed by Wagner et al. (2011b). Alfred is a butler-
like virtual character that is aware of the user and reacts to his or her
affective expressions. The user interacts with Alfred via acoustics of
speech and facial expressions (see Figure 2). As a response, Alfred simply
mirrors the user's emotional state by appropriate facial expressions.
This behavior can be interpreted as a simple form of showing empathy.
Search WWH ::
Custom Search