Inference of Co-occurring Classes: Multi-class and Multi-label Classification - Computational Intelligence Paradigms in Advanced Pattern Classification - page 188

Recognizing the entire set of 712 affective states is not necessarily desirable for
human-computer interfaces, in addition, getting into such a detailed description
level may reduce the accuracy of the annotation and reduces the number of labeled
samples per label. Therefore:

• A set of classes to be recognized should be defined to facilitate various applica-

tions of the classification system.

• The system should be easily adapted to new speakers

Instead, an extension of the prototype approach was used. Nine affective-state
groups were chosen for the described system. These were recognized as common
and relevant to the analysis of various human-computer interactions and in every-
day life. These included: confidence, concentration, disagreement, excitement, in-
terest, joy, stress, thinking and uncertainty .
Different applications may need different sets of affective states, therefore:

• The classification system should be built in a modular and adaptive manner.

The dynamic nature and co-occurrences of affective states are translated into two
demands of the classification system:

• Multi-label classification is required

• For tracking of dynamic changes, the classification output should present mul-

tiple levels of each of the inferred affective states

All the taxonomic methods, including the chosen Mind Reading taxonomy and
annotated database, associate each speech sample (sentence) with a single label,
therefore:

• The classification should be semi-blind multi-label classification, i.e. the input

is a single label for each sample, while the output can comprise of several la-
bels for each sample.

Initial observations, on a database of naturally evoked sustained interactions,
Doors, revealed two characteristics that affected the classification [58]:

• Different sets of features distinguish between different pairs of affective states.

• Transitions between affective states are gradual, i.e. there is a continuous tran-

sition along the features that distinguish them, and a threshold signifies the ac-
tual transition (when listeners recognize a different affective state).

All these issues define the requirements of the classification process. The clas-
sification process and its relations to the choice of taxonomy and to the require-
ments posed by the desired range of applications and by the data characteristics in
schematically described in Figure 4.