Information Technology Reference
In-Depth Information
Table 6.6.
Accuracies (%) on entire training datasets labeled with three different
distributions
Distribution I
J4.8 BPNN SVM CLR OneR Bagged J4.8 Boosted J4.8 Stacking CAMLET
anneal
74.7
71.6 47.4 56.8
55.8
87.4
100.0
27.4
77.9
audiology 47.0
51.7 40.3 45.6
52.3
87.2
47.0
21.5
63.1
autos
66.7
63.8 46.8 46.156.0
89.4
66.7
29.8
53.2
balance-
scale
58.0
59.4 39.5 43.4
53.0
83.3
58.0
39.5
39.5
breast-
cancer
55.7
61.5 40.2 50.8
59.0
88.5
70.5
23.8
41.0
breast-w 86.1
91.1 38.0 46.8
54.4
96.2
100.0
34.2
77.2
colic
91.8
82.0 42.6 60.7
55.7
88.5
100.0
29.5
67.2
57.4
48.7 35.7 39.154.8
9 .3
57.4
26.5
55.7
credit-a
Distribution II
J4.8 BPNN SVM CLR OneR Bagged J4.8 Boosted J4.8 Stacking CAMLET
anneal
68.4
66.3 56.8 60.0
56.8
85.3
87.4
49.5
67.4
audiology 60.4
61.1 43.6 55.0
56.4
87.2
69.8
50.3
autos
63.164.5 52.5 53.2
57.4
90.8
00.0
39.0
67.4
balance-
scale
61.6
57.7 49.8 55.2
58.0
80.4
61.6
45.6
41.9
breast-
cancer
68.0
70.5 47.5 58.2
59.8
77.9
96.7
33.6
64.8
breast-w 89.9
93.7 49.4 58.2
62.0
98.7
100.0
59.5
78.5
colic
77.0
78.7 57.4 62.3
67.2
85.2
100.0
29.5
88.5
credit-a
61.3
59.1 41.3 52.6
56.1
89.6
62.2
47.4
53.5
Distribution III
J4.8 BPNN SVM CLR OneR Bagged J4.8 Boosted J4.8 Stacking CAMLET
anneal
74.7
70.5 67.4 70.5
73.7
84.2
94.7
67.4
66.3
audiology 65.8
67.8 63.8 64.4
67.1
83.2
67.1
59.7
65.1
autos
85.173.8 68.170.2
73.8
87.9
00.0
66.7
67.4
balance-
scale
70.5
69.8 64.8 65.8
69.8
80.1
85.8
62.6
63.0
breast-
cancer
71.3
77.0 66.4 65.6
77.9
86.9
79.5
73.0
73.0
breast-w 74.7
86.173.4 68.4
74.7
87.3
00.0
63.3
70.9
colic
70.5
77.0 65.6 60.7
73.8
85.2
100.0
49.2
60.7
70.9
70.0 65.2 65.2
71.3
85.7
87.8
61.7
65.2
credit-a
SVM, CLR and Stacking affected the class distribution differences. Their per-
formances were sometimes lower the percentage for each majority class. Although
Stacking is a kind of selective meta-learning algorithm, it performed worse than
the other two selective meta-learning algorithms, because it included SVM and
CLR at the same time and failed to control the predictive results of these worse
learning algorithms.
Evaluation of Learning Curves.
Similar to the evaluations of the learning
curves on the meningitis rule set, we estimated the minimum training subsets
for a valid model, which works better than just predicting the majority class of
the datasets.
Table 6.7 shows the sizes of the minimum training subsets, which can help
construct more accurate rule evaluation models than the percentages of the
majority class formed by each learning algorithm. For datasets with balanced
