Inference of Co-occurring Classes: Multi-class and Multi-label Classification - Computational Intelligence Paradigms in Advanced Pattern Classification - page 183

belong to several (predefined) topics, such as government and health [42, 44, 63,
77]; in computational biology, such as in the area of functional genomics, each
gene may be associated with a set of functional classes, such as metabolism, tran-
scription and protein synthesis [5, 77]; in computer vision, in applications such as
scene classification, each scene image may belong to several semantic classes,
such as beach and urban [8, 68, 77]; in social computing, such as in the case of
affective computing, people simultaneously experience and express emotions,
mental states, attitudes and moods, such as thinking and experiencing uncertainty
at the same time. Such affective states can be expressed and analyzed in various
behavioral cues, such as facial expressions [17] and non-verbal speech [60]. In all
these cases, each sample in the training set is associated with a set of labels, and
the task is to output a label set. The size of the output label-set for each unseen
sample or instance is unknown in advance.
Multi-label classification techniques are mostly extensions and adaptations of
the multi-class classification techniques (and these were extensions of methods for
the binary problem). Some of these methods deal directly with the multi-label
problem. For example, variants of AdaBoost, such as AdaBoost.MH and Ada-
Boost.MR [4, 54]. Zhang [77] adapts the K-nearest Neighbor model by utilizing
the statistical information from the neighborhood of the examined sample. McCal-
lum [42] uses Bayesian mixture models to select the most probable classes. Wang
et al. [67] use a hierarchical generative probabilistic model. Others are variations
of the methods which are based on combinations of binary classifiers, and as in
multi-class classification, but with variations of the combination methods. The
methods of the first type are sometimes called adaptation methods, while the me-
thods which dissolve the classification problem into binary classifiers are also
called transformation methods [64].
The multi-label classification methods have to deal with issues such as how to
choose the most probable labels and how to use the inherent semantic or probabil-
istic (probability of co-occurrences) connections or relations between the probable
sets of labels. Semantic relations can be synonyms, hierarchical connections (for
example: vehicle , car , the car model; flora , flower , daisy , etc), ranking [13], or the
fact the probability of an appearance of both an office and a car in a single image
is relatively low. For example, Qi et al. [47] review three paradigms, of individual
concept annotation, in which each concept is detected independently of other con-
cepts; context-based conceptual fusion annotation, which explores the relations
between the results of the individual detectors, and propose an integrated multi-
label annotation paradigm for correlative multi-label video annotation, in which
the learning of the individual concepts and their correlation are modeled together.
Wang [67] present a model for correlated multi-labeling. Tsoumakas et al. [64],
presents label power-sets (LP) which represent frequent combinations of labels in
the training sets as new labels.
There are three types of approaches that can be recognized from review of
techniques for image annotation, and can be seen or extended to other modalities
and applications. The first is indeed multi-label classification on the entire sample,