French Face-to-Face Interaction: Repetition as a Multimodal Resource - Coverbal Synchrony in Human-Machine Interaction - page 143

The use of such terms as adaptation as well as alignment (Garrod
and Pickering, 2004; Pickering and Garrod, 2006), accomodation (Giles
et al., 1987) or mimicry (Kimbara, 2006 among others) to quote but
a few studies, refers to convergence phenomena. In the Interactive-
alignment Model of Dialogue (Pickering and Garrod, 2006), the
alignment observed at one level is automatically extended to other
levels, resulting in a similarity at the level of discourse or gesture and
at the level of representations. For the authors, this alignment is the
basis of successful communication in dialog. Adaptation refers to the
fact that participants adapt their responses to the other interactant(s)'
productions. In the Communication Accomodation Theory (CAT, Giles
et al., 1987), adaptation and accomodation can be used indifferently.
Speakers are tailored to their partners ( adaptive behavior ) to affiliate
with their social status for example. Mimicry is a direct imitation of
what the other participant produces (exact match at prosodic level,
Couper-Kuhlen and Selting, 1996; exact or very close match at gesture
level, Jones, 2006). A discussion about the relevance of one term
or another is out of the scope here (Guardiola, in progress). 1 It can
nevertheless be noted that the choice of a term varies according to
the linguistic field (psycholinguistics, sociolinguistics, phonetics, …)
but also the modality considered in the type of study (see Section 3).
In this chapter, after presenting the corpus as well as some of the
annotations developed in the OTIM project, we then focus on the specifi c
phenomenon of repetition. After briefl y discussing this notion, we show
that different degrees of convergence can be achieved by speakers
depending on the multimodal complexity of the repetition and on the
timing in between the repeated element and the model. Although we
focus more specifi cally on the gestural level, we present a multimodal
analysis of gestural repetitions in which we met several issues linked to
multimodal annotations of any type. This gives an overview of crucial
issues in cross-level linguistic annotation, such as the defi nition of a
phenomenon including formal and/or functional categorization.
2. Corpus and Annotations
A multimodal analysis of interaction requires the encoding of many
different pieces of information, from different domains, with different
levels of granularity. All the information has to be connected and
synchronized (with the signal for example). Different steps in the
annotation were adopted in the OTIM project to achieve this goal.
Before presenting the annotation process and some of the annotations
1 For further details, see SPIM ANR-08-BLAN-0276-01: http://spim.risc.cnrs.fr/