A Survey of Listener Behavior and Listener Models for Embodied Conversational Agents - Coverbal Synchrony in Human-Machine Interaction - page 252

Table 1. Backchannel signals performed by the 3D agent Greta. The action tension means

tension of the lips.

1. nod 8. raise eyebrows 15. nod and raise eyebrows

2. smile 9. shake and frown 16. shake, frown and tension

3. shake 10. tilt and frown 17. tilt and raise eyebrows

4. frown 11. sad eyebrows 18. tilt and gaze right down

5. tension 12. frown and tension 19. eyes wide open

6. tilt 13. gaze right down 20. raise left eyebrows

7. nod and smile 14. eyes roll up 21. tilt and sad eyebrows

just select either “I don't know” or “none” (if they thought that there
was a meaning but different from the ones proposed). Through this
test, prototypical signals for most of our meanings were determined.
For the positive meanings, agreement is meant by a nod, as well
as acceptance. To mean liking a smile appears as the most appropriate
signal. A nod associated to a raise of the eyebrows seems to convey
understanding; however, only 17 subjects out of 30 thought so. As for
interest and belief, no significant results were found. A combination
of smile+raise eyebrows could be a possibility for interest. For the
negative meanings, disagreement and refusal are meant by a shake ,
whereas not liking is represented by frown and tension of the lips . A
tilt+frown as well as a raise of the left eyebrow mean disbelief for most
of the subjects. The best signal to mean not understanding seems
to be a frown . And tilt and gaze right down as well as eyes roll up are
more relevant for the meaning not interest. It also appeared that a
combination of signals could significantly alter the perceived meaning.
For instance, tension alone and frown alone do not mean not liking,
but the combination frown+tension does. The combination tilt+frown
means disbelief, whereas tilt alone and frown alone do not convey
this meaning. Tilt alone and gaze right down alone do not mean not
interest as significantly as the combination tilt+gaze . Conversely the
signal frown means not understanding but when the signal shake is
added, frown+shake significantly loses this meaning. The resulting
set of interpretable signals has been used to define a library of
prototypical backchannel signals, called backchannel lexicon . Such a
library of behaviors has been implemented for the virtual agent Greta
and it has been used to determine the appropriate animation the agent
should display while listening. For example, if the agent intends
to communicate its agreement toward what the user is saying, the
system will find in the backchannel lexicon the appropriate behavior
to display, for instance a head nod.