Digital Signal Processing Reference
In-Depth Information
Table 12.3
Results of the 2-class and 5-class tasks of the HU-ASA database with various classifiers
and feature sets
Classifier
[
%
]
2-class
5-class
Features
UA
WA
UA
WA
SVM
IS09-func
69.0
72.0
46.4
57.2
SVM
MFCC-func
73.9
75.6
42.2
56.0
Left-right HMM
MFCC
79.0
79.8
47.3
63.4
cyclic HMM
MFCC
79.0
79.6
49.5
64.0
LSTM
MFCC
80.0
81.3
41.1
62.3
The training set was up-sampled for each fold for the LSTM-RNN and SVM classi-
fiers. This was done by copying training instances of minority classes until a near-
uniform class distribution is achieved. This step was not necessary in the case of
HMMs, as each class is learnt by an individual model, and classification is per-
formed with HMMs and the maximum likelihood criterion, i.e., class priors, were
not used in the decision rule. For classification with the LSTM RNN each sequence
in the test set was presented frame by frame to the input layer, and each frame was
assigned to the class with the highest probability as indicated by the output layer.
Then, a majority vote over the frame-level decisions was made to label the sequence.
Table
12.3
depicts results by UA and WA for the 2-class and 5-class tasks of the
HU-ASA database, as defined in Table
12.2
. Always deciding for the majority class
leads to WA and UA of 55.1 % and 20.0 % (5-class task), and 67.5 % and 50.0 %
(2-class task).
In SVM classification on the 2-class task, the MFCC-func feature set outperforms
the IS09-func set in terms of WA by 3.6 % absolute, being significant at the 5 % level
(one-tailed
z
-test). However, the IS09-func feature leads to a significantly higher UA
(4.4 % absolute improvement) for the 5-class task. Both types of HMMs outperform
static classification by SVM. Further, the cyclic HMM is superior to the left-right
HMM justifying the made assumption of partly quasi-periodic vocalisations. Yet,
this observation is not significant on the 5 % level. To explain this, the estimated
'cycle probability'
a
N
,
1
of the HMMs is shown for each class, on average across the
ten folds, in Table
12.4
. There, the cycle probabilities are around 28 % in the models
for songbirds (
Passeriformes
) and primates, but below 10 % for
Felidae
.
The additional LLDs from Table A.1 as input features for the HMMs could not
improve the above results. The impact of a varying number of Gaussian mixtures
Table 12.4
Cycle
probabilities
a
N
,
1
after
training of the cyclic HMMs
for comparison among each
other given for each class in
the 5-class task, averaged
over ten folds
Class
a
N
,
1
[%]
Passeriformes
28.1
Non-Passeriformes
17.2
Canidae
14.2
Felidae
9.9
Primates
28.0
