Graphics Reference
In-Depth Information
On the other hand, analyzing the gaussian attribute noise results in the test accuracy
of the methods using OVO being better at all the noise levels. The low p-values show
that this advantage, also in favor of OVO, is statistically significant. With respect to
the RLA results the p-values show a clear decreasing tendency when the noise level
increases in all the algorithms. In the case of C4.5, OVO is statistically better from a
35% noise level onwards. RIPPER and 5-NN are statistically equivalent at all noise
levels—although 5-NN with OVO obtains higher Wilcoxon's ranks.
Hence, the OVO approach is also suitable considering the accuracy achieved with
this type of attribute noise. The robustness results are similar between the OVO and
non-OVO versions with RIPPER and 5-NN. However, for C4.5 there are statistical
differences in favor of OVO at the highest noise levels. It is important to note that in
some cases, particularly in the comparisons involving RIPPER, some RLA results
show that OVO is better than the non-OVO version in average but the latter obtains
more ranks in the statistical test—even though these differences are not significant.
This is due to the extreme results of some individual data sets, such as
led7digit
or
flare
, in which the RLA results of the non-OVO version are much worse than
those of the OVO version. Anyway, we should notice that average results themselves
are not meaningful and the corresponding non-parametric statistical analysis must
be carried out in order to extract meaningful conclusions, which reflects the real
differences between algorithms.
5.5.5.3 Conclusions
The results obtained have shown that the OVO decomposition improves the baseline
classifiers in terms of accuracy when data is corrupted by noise in all the noise
schemes shown in this chapter. The robustness results are particularly notable with
the more disruptive noise schemes—the uniform random class noise scheme and the
uniform random attribute noise scheme—where a larger amount of noisy examples
and with higher corruptions are available, which produce greater differences (with
statistical significance).
In conclusion, we must emphasize that one usually does not know the type and
level of noise present in the data of the problem that is going to be addressed.
Decomposing a problem suffering from noise with OVO has shown a better accuracy,
higher robustness and homogeneity in all the classification algorithms tested. For
this reason, the use of the OVO decomposition strategy in noisy environments can
be recommended as an easy-to-applicate, yet powerful tool to overcome the negative
effects of noise in multi-class problems.
References
1. Abellán, J., Masegosa, A.R.: Bagging decision trees on data sets with classification noise. In:
Link S., Prade H. (eds.) FoIKS, Lecture Notes in Computer Science, vol. 5956, pp. 248-265.
Springer, Heidelberg (2009)
