Speech Technology and Conversational Activity in Human-Machine Interaction - Coverbal Synchrony in Human-Machine Interaction - page 14

This empathic nature of conversational activity needs to be included
in our future technology.
We have made a start, as shown in Figure 1, simply by observing
who is speaking when, and how the other members of the group
respond to each utterance in terms of both speech (simply presented
as noise) and motion (from observing head movements), and we
thereby form an image of the progress of the conversation and of
the participant roles of each of the interlocutors. Perhaps this is the
best that young babies and dogs can do, as they certainly cannot be
expected to follow the text of any conversations they hear, and it is this
level of processing that inspires our latest technology developments
for dialogue tracking and processing.
This chapter has presented results from work related to multimodal
recordings of multiparty conversational interactions, showing that
participants typically engage positively throughout a discourse,
synchronizing their speech and movements to a very high degree.
It has presented the concept of niblets to account for the small and
often ungrammatical fragments of speech that are characteristic of
such informal conversations, and has argued that there is a strong
social component related to empathy in human spoken interactions
and that our present technologies for speech processing fail to take
this into account. The chapter concluded by describing a small robot
that has been developed for the gathering of socially relevant data in
a conversational setting.
Acknowledgement
This work is being carried out as part of the FastNet project at Trinity
College Dublin (The University of Dublin) with support from Science
Foundation Ireland (SFI Stokes Professorship Award 07/SK/I1218).
REFERENCES
Campbell, N. and P. Mokhtari. 2003. Voice quality: The 4th prosodic dimension,
in Proceedings of the 15th International Congress of Phonetic Sciences
(ICPhS'03), Barcelona, Spain, pp. 2417-2420.
Campbell, Nick. 2006. “A multimedia database of meetings and informal
interactions for tracking participant involvement and discourse flow”,
Proc. LREC 2006, Lisbon.
Campbell, Nick. 2007. “On the Use of Nonverbal Speech Sounds in Human
Communication”, pp. 117-128 in Verbal and Nonverbal Communication
Behaviors , LNAI, Vol. 4775.