K Nearest Neighbor Edition to Guide Classification Tree Learning: Motivation and Experimental Results - Data Mining: Theory, Methodology, Techniques, and Applications - page 61

Table 3. K -NN-boosting vs. ID3 for every K. A ↑ sign means that k -NN-boosting

outperforms ID3 with a significance level of 95% (Wilcoxon test).

Database K=1 K=2 K=3 K=4 K=5 K=6 K=7 K=8 K=9 K=10

Diabetes = = = = = = = = = =

Australian = = = = = = = = = =

Heart ↑ = = = = = = = = =

Monk2 ↑ ↑ ↑ ↑ = = ↑ ↑ ↑ ↑

Wine = = = = = = = = = =

Zoo = = = = = = = = = =

Waveform-21 = = = = ↑ = = = ↑ =

Nettalk = = = = = = = = = =

Letter = = = = = = = = = =

Shuttle = = = = = = = = = =

Table 4. Sizes of the augmented databases

Database Original K=1 K=2 K=3 K=4 K=5 K=6 K=7 K=8 K=9 K=10

size

Diabetes 768 1014 990 1003 987 987 976 977 973 972 969

Australian 690 928 916 916 909 905 895 893 894 897 890

Heart 270 385 375 365 360 360 364 359 360 363 366

Monk2 432 552 580 580 588 604 590 575 565 564 565

Wine 178 219 236 227 238 232 234 238 236 229 237

Zoo 101 103 123 108 106 109 111 113 117 120 122

Wavef.-21 5000 6098 6129 5930 5964 5907 5891 5851 5848 5824 5824

Nettalk 14471 15318 15059 15103 15065 15085 15069 15077 15056 15059 15061

Letter 20000 20746 20993 20799 20889 20828 20857 20862 20920 20922 20991

Shuttle 58000 58098 58111 58096 58108 58111 58112 58111 58120 58129 58133

performance comparison is done between the ID3 paradigm and our proposed
one, as they work in a similar manner.

6 Conclusions and Further Work

In this paper a new hybrid classifier that combines Classification Trees (ID3)
with distance-based algorithms is presented. The main idea is to augment the
training test duplicating the badly classified cases according to k -NN algorithm.
The underlying idea is to test if one algorithm ( k -NN)couldbeusedtoboost
a different one (ID3), acting over the distribution of the training examples and
then causing two effects: the choice of a different variable to split at some node,
and the change in the decision about pruning or not a subtree.
The experimental results support the idea that such boosting is possible and
deserve further research. A more complete experimental work on more databases
as well as another weight changing schemas (let us remember that our approach