Graphics Reference
In-Depth Information
∗
and another with the results of the pairwise class noise. A star '
'nexttoap-value
indicates that the corresponding single algorithm obtains more ranks than the MCS
in Wilcoxon's test comparing the individual classifier and the MCS. Note that the
robustness can only bemeasured if the noise level is higher than 0%, so the robustness
results are presented from a noise level of 5% and higher.
From the raw results we can extract some interesting conclusions. If we consider
the performance results with uniform class noise we can observe that MCS3-
k
is
statistically better than SVM, but in the case of C4.5 statistical differences are only
found at the lowest noise level. For the rest of the noise levels, MCS3-1 is statistically
equivalent to C4.5. Statistical differences are found between MCS3-1 and 1-NN for
all the noise levels, indicating that MCS are specially suitable when taking noise
sensitive classifiers into account.
In the case of pairwise class noise the conclusions are very similar. MCS3-1
statistically outperforms its individual components when the noise level is below
45%, whereas it only performs statistically worse than SVM when the noise level
reaches 50% (regardless of the value of 1). MCS3-1 obtains more ranks than C4.5 in
most of the cases; moreover, it is statistically better than C4.5 when the noise level
is below 15%. Again MCS3-1 statistically outperforms 1-NN regardless of the level
of noise.
In uniform class noise MCS3-1 is significantly more robust than SVM up to a
noise level of 30%. Both are equivalent from 35% onwards—even though MCS3-1
obtains more ranks at 35-40% and SVM at 45-50%. The robustness of C4.5 excels
with respect to MCS3-1, observing the differences found. MCS3-1 is statistically
better than 1-NN. The Robustness results with pairwise class noise present some
differences with respect to uniformclass noise.MCS3-1 statistically overcomes SVM
up to a 20% noise level, they are equivalent up to 45% and MCS3-1 is outperformed
by SVM at 50%. C4.5 is statistically more robust than MCS3-1 (except in highly
affected data sets, 45-50%) and The superiority of MCS3-1 against 1-NN is notable,
as it is statistically better at all noise levels.
It is remarkable that the uniform scheme is the most disruptive class noise for
the majority of the classifiers. The higher disruptiveness of the uniform class noise
in MCSs built with heterogeneous classifiers can be attributed to two main reasons:
(i) this type of noise affects all the output domain, that is, all the classes, to the
same extent, whereas the pairwise scheme only affects the two majority classes; (ii)
a noise level x%with the uniform scheme implies that exactly x% of the examples in
the data sets contain noise, whereas with the pairwise scheme, the number of noisy
examples for the same noise level x% depends on the number of examples of the
majority class
N
maj
; as a consequence, the global noise level in the whole data set
is usually lower—more specifically, the number of noisy examples can be computed
as
100.
With the performance results in uniform class noise MCS3-1 generally outper-
forms its single classifier components. MCS3-1 is better than SVM and 1-NN,
whereas it only performs statistically better than C4.5 at the lowest noise levels.
In pairwise class noise MCS3-1 improves SVM up to a 40% noise level, it is better
than C4.5 at the lowest noise levels—these noise levels are lower than those of the
(
x
·
N
maj
)/
