The Situated Multimodal Facets of Human Communication - Coverbal Synchrony in Human-Machine Interaction - page 174

his wife and his little daughter all together …”. For someone that was not
seeing her while speaking, figuring out what she meant with this “tic,
tic, tic …”, can be a bit difficult, even though it would be possible to
appropriately guess what it was with a little imaginative effort and
by exploiting the context she indirectly provided in her sentence. For
me, speaking face to face (I must say body-to-body to be appropriate)
there was no need to guess or ask for clarifications, since through
her gestures, body movements, and facial expressions, I immediately
understood that she was referring to the noise of the wheelchair while
running through the corridor of the hospital.
There were several communicative signals that should be accounted
for while the abovementioned information exchange took place, all
contributing to my comprehension:
a) The context she was referring to;
b) Her vocal intonation and pauses in mimicking the wheelchair
noise;
c) Her hand gestures while mimicking it;
d) Her body movements;
e) Her facial expressions;
f) Her speech;
g) More.
I refer to these as the situated multimodal facets of human
communication and I want, in the discussion below, to consider
their contribution to human information exchanges. Therefore, in
the following, I will dedicate the fi rst section of this paper to the
importance of context, the second one to pausing strategies, the
third one to hand gestures and their role in communication. The
fourth section is dedicated to the perception of facial expressions, in
particular “facial emotional expressions”. I will neglect the speech
produced since it is language dependent and can be formulated
through different speech motor programs, and the “more” item,
since it includes my personal background and knowledge, as well
as, my cultural specificity and the social specific communication
rules adopted in my Mediterranean community. This is not to
say that these aspects are less important, and surely they deserve
to be accounted for, nevertheless they are not the focus of this paper.
Several themes have emerged during my own research on
human multimodal communication, and I highlight these here. These
are: the importance of context; pausing strategies when speaking;
gestures; and facial expressions. I will review each of these, with