A Survey of Listener Behavior and Listener Models for Embodied Conversational Agents - Coverbal Synchrony in Human-Machine Interaction - page 251

virtual agent. Most backchannel signals conveyed either a positive or
a negative connotation. All signals containing the actions head nod
and smile were associated to positive meanings, whereas negative
meanings were mostly attributed to signals containing head shake
and frown. Only three signals ( raise of the left eyebrow , head tilt on the
right+raise eyebrows and eyes wide open ) were hard to interpret for the
users in terms of positive and negative. The authors supposed that
some signals are particularly hard to interpret when showed context-
free since their meaning can strongly depend on the context and
to the listener's personality. With regard to the second hypothesis,
backchannel signals polysemy and synonymy, results confirmed
that participants associated different meanings to the same signals
and that they attributed the same meaning to different signals or
a combination of signals. For example, head nod can convey the
meanings of agreement, acceptance and understanding. The meaning
of refusal can be associated to the signals head shake , head shake+frown ,
frown+tension of the lips , head shake+frown+tension of the lips .
In order to generalize these findings, the pilot experiment needed
to be performed with more subjects. Moreover, as the authors have
tested combinations of signals, they decided to prepare a second
version of the experiment in order to assess the meaning of each single
action. In particular, the authors aimed to get a better understanding
about how different actions contribute to the interpretation of a facial
expression. A first question to explore was: is it possible to identify a
signal (or a combination of signals) for each meaning? For example,
is there a signal more relevant than others for a specific meaning
or can a single meaning be expressed through different signals or a
combination of signals? The researchers hypothesized that for each
meaning they could find a prototypical signal. A second question
was: does a combination of signals alter the meaning of backchannel
single signals? The hypothesis was that in some cases, adding a signal
to another could significantly change the perceived meaning. In that
case, the independent variable was the combination of signals and the
dependent variable was the meaning attributed to each signal by the
subjects. Sixty French subjects were involved in that experiment. The
3D agent Greta displayed 21 video clips (see Table 1) showing signals
chosen among those proposed by Allwood and Cerrato (2003) and
Poggi (2007). After each video clip and before moving on, participants
could select one meaning according to their opinion about which
meaning fitted that particular backchannel signal best.
The same list of meanings proposed in the pilot test was used in
this evaluation too. It was possible to select several meanings for one
signal and when none of the meanings seems to fit, participants could