Graphics Reference
In-Depth Information
annotation scheme. The theory describing which categories to use
and how to classify the data into these elements is also called a 'data
model', although this term is often used in a more restricted sense
as a formalized description of the data objects and their relations. In
practical work, annotation categories are usually defined based on
extensive literature, in which the phenomena are treated by extracting
the essential features for the purpose of the study. Annotation
schemes can thus vary due to different theoretical assumptions (i.e.
they have different data models), but they may also have different
granularity and completeness levels, e.g. gesture annotation may
include hand-gestures but not finger movement (granularity) or encode
the function rather than the form of the gesture (completeness). Whilst
the different data models are often incompatible with each other
and, consequently, cause incompatibilities in the annotation scheme,
different granularity or completeness levels do not necessarily render
annotations incompatible. Annotation schemes aim to be extensible, i.e.
more annotation levels and categories can be added, and the existing
categories can be further specified or combined under a more general
feature, and also incremental in that the extensions can be done at
different stages of the annotation process. Annotations also need to
be consistent, and to this end, annotation categories have particular
semantics and are applicable for the annotations they have been
designed for. The annotation results have to be validated by applying
the scheme to practical coding tasks and by calculating inter-coder
agreement by several coders.
In general, multimodal annotation has to address the following
issues:
What to annotate: identification of an element as communicatively
relevant
How to segment the data: defining the boundaries of the elements
How to deal with interdependence: relations between elements
and annotation levels.
The first two are relevant to all annotations, whilst the third is
especially characteristic of multimodal annotations. In the subsections
below, we discuss these issues in more detail.
2.1 What to annotate?
In multimodal annotation, different levels are identified and various
annotation features selected depending on the annotation level. In
general, annotations deal with the agent's identity, physiological state,
Search WWH ::
Custom Search