Inference of Co-occurring Classes: Multi-class and Multi-label Classification - Computational Intelligence Paradigms in Advanced Pattern Classification - page 191

Different features distinguish different pairs of affective-state groups. There-
fore, each binary classifier was independently trained and optimized in terms of
both feature set and classification algorithm. An iterative process of feature selec-
tion and training was repeated until a “good enough” classification, in terms of
cross-validation, precision and symmetry of the true-positive recognition between
the two compared classes, was achieved [60]. The two classification algorithms
that were finally used for the binary classifiers were linear SVM [65] and C4.5 de-
cision tree [48]. These yielded results which are comparable to many other me-
thods and are simple to implement. In addition, these two methods are based on
finding the border between classes rather than focusing on class centers, which is
in accordance with the observation that thresholds distinguish between classes.
The feature selection process confirmed the observation that different sets of fea-
tures distinguish different pairs of classes. On average, ten features were required
for the binary classifiers, but overall nearly all the features were used. The average
tenfold cross-validation for the 36 machines was 75%.
Fig. 6 A schematic description of the one-against-one classification process.
Because several affective states can co-occur, a threshold was defined as a se-
lection criteria, i.e. all the affective-state groups (labels) whose ranking was above
the threshold were associated with the expression in the examined sentence or ut-
terance. The threshold for selection was set over a standard deviation above the
mean number of machines, i.e. a label was selected by at least six of the eight bi-
nary classifiers. From a set of samples of 93 affective states which were chosen to
represent the nine affective-state groups, 60 percent were used for training and
testing of the binary classifiers, and 40 percent were used for the verification of
the combined classifier. The overall accuracy was 83 percent true-positive recog-
nition for the original class-labels. The combination of inferred labels was com-
pared to the lexical definition of the examined affective states. Combinations of
labels (multi-label classification) were found also for affective states and samples
that were used for training with a single label.