Graphics Reference
In-Depth Information
CHAPTER 7
The Situated Multimodal Facets
of Human Communication
Anna Esposito
1. Introduction
Humans interact with each other through a gestalt of emotionally
cognitive actions which involve much more than the speech production
system. In particular, in human interaction, the verbal and nonverbal
communication modes seem to cooperate jointly in assigning semantic
and pragmatic contents to the conveyed message by unraveling
the participants' cognitive and emotional states and allowing the
exploitation of this information to tailor the interactional process.
These multimodal signals consist of visual and audio information
that singularly or combined may characterize relevant actions for
collaborative learning, shared understanding, decision making and
problem solving. This work will focus on the visual and audio
information including contextual instances, hand gestures, body
movements, facial expressions, and paralinguistic information such
as speech pauses, all grouped under the name of nonverbal data, and
on the role they are supposed to play, assisting humans in building
meanings from them.
Just to give an example, I am reporting a dialogue I had with a
friend of mine a few days ago. She asked me about a common friend
and at a certain point she said: “
I can never forget when his wife gave
birth to his last child. I was in the hospital because of my daughter's car
accident, and then I heard this tic, tic, tic … I turned round and saw him,
Search WWH ::
Custom Search