Graphics Reference
In-Depth Information
Table 2.8 Adjusted p -values for the post-hoc tests
i
Algorithm
Unadjusted p
p Bonf
p Holm
p Hochberg
1
RBFN-C
0.001745
0.005235
0.005235
0.005235
2
LVQ-C
0.00365
0.010951
0.007301
0.007301
3
MLP-CG-C
0.044171
0.132514
0.044171
0.044171
the readability. Table 2.8 contains all the adjusted p -values for Bonferroni-Dunn's,
Holm's and Hochberg's test from the unadjusted values.
Taking a significance level of
05 we can observe that the conclusions
obtained from Table 2.8 are the same than those obtained in Table 2.7 without the
need to establish the unadjusted p -value that acts as a threshold for the null-hypothesis
rejection. The user only needs to observe those adjusted p -values that fall under the
desired
α =
0
.
α
significance level.
References
1. Alpaydin, E.: Introduction to Machine Learning, 2nd edn. MIT Press, Cambridge (2010)
2. Bache, K., Lichman, M.: UCI machine learning repository (2013). http://archive.ics.uci.edu/
ml
3. Barandela, R., Sánchez, J.S., García, V., Rangel, E.: Strategies for learning in class imbalance
problems. Pattern Recognit. 36 (3), 849-851 (2003)
4. Ben-David, A.: A lot of randomness is hiding in accuracy. Eng. Appl. Artif. Intell. 20 (7),
875-885 (2007)
5. Demsar, J.: Statistical comparisons of classifiers over multiple data sets. J. Mach. Learn. Res.
7 , 1-30 (2006)
6. Efron, B., Gong, G.: A leisurely look at the bootstrap, the jackknife, and cross-validation. Am.
Stat. 37 (1), 36-48 (1983)
7. Friedman, M.: The use of ranks to avoid the assumption of normality implicit in the analysis
of variance. J. Am. Stat. Assoc. 32 (200), 675-701 (1937)
8. Friedman, M.: A comparison of alternative tests of significance for the problem of m rankings.
Ann. Math. Stat. 11 (1), 86-92 (1940)
9. García, S., Fernández, A., Luengo, J., Herrera, F.: Advanced nonparametric tests for multi-
ple comparisons in the design of experiments in computational intelligence and data mining:
Experimental analysis of power. Inf. Sci. 180 (10), 2044-2064 (2010)
10. García, S., Herrera, F.: An extension on “statistical comparisons of classifiers over multiple
data sets” for all pairwise comparisons. J. Mach. Learn. Res. 9 , 2677-2694 (2008)
11. Hochberg, Y.: A sharper bonferroni procedure for multiple tests of significance. Biometrika
75 (4), 800-802 (1988)
12. Hodges, J., Lehmann, E.: Rankmethods for combination of independent experiments in analysis
of variance. Ann. Math. Statist 33 , 482-497 (1962)
13. Holm, S.: A simple sequentially rejective multiple test procedure. Scand. J. Stat. 6 , 65-70
(1979)
14. Huang, J., Ling, C.X.: Using AUC and accuracy in evaluating learning algorithms. IEEE Trans.
Knowl. Data Eng. 17 (3), 299-310 (2005)
15. Iman, R., Davenport, J.: Approximations of the critical region of the Friedman statistic. Com-
mun. Stat. 9 , 571-595 (1980)
 
 
 
Search WWH ::




Custom Search