Mixing Independently Trained Classifiers - Design and Analysis of Learning Classifier Systems - page 115

Prediction of Heavisine function using different mixing models

0.6

InvVar
Conf
MaxConf
LS
IRLS
XCS
f(x)

0.4

0.2

0

-0.2

-0.4

-0.6

0 0.2 0.4 0.6 0.8 1

x

Fig. 6.4. Resulting predictions of a single run, using different mixing models for the
Heavisine function. See the text for an explanation of the experimental setup.

Table 6.2. The mean likelihoods of the different mixing models, as a fraction of the
mean likelihood of IRLS, averaged over 50 experimental runs per function. A lin added
to the function name indicates the use of classifiers that model straight lines rather
than averaging classifiers. For averaging classifiers, IRLS and IRLSf, and LS and LSf
are equivalent, and so their results are combined. The results written in bold indicate
that there is no significant difference to the best-performing mixing model for this
function. Those results that are significantly worse than the best mixing model but not
significantly worse than the best model in their group are written in italics. Statistical
significance was determined by Tukey's HSD post-hoc test at the 0.01 level.

Function Likelihood of Mixing Model as Fraction of IRLS
IRLS IRLSf LS LSf InvVar Conf MaxConf XCS
Blocks 1.00000 0.99473 0.99991 0.99988 0.99973 0.99877
Bumps 1.00000 0.94930 0.98442 0.97740 0.96367 0.94678
Doppler 1.00000 0.94930 0.98442 0.97740 0.96367 0.94678
Heavisine 1.00000 0.96289 0.96697 0.95123 0.95864 0.95807
Blocks lin 1.00000 1.00014 0.99141 0.99559 0.99955 0.99929 0.99956 0.99722
Bumps lin 1.00000 0.99720 0.94596 0.94870 0.98425 0.97494 0.97797 0.94107
Doppler lin 1.00000 0.99856 0.94827 0.98628 0.98723 0.97818 0.98172 0.94395
Heavisine lin 1.00000 0.99523 0.98480 0.96854 0.98448 0.97347 0.99005 0.95739

method are shown in Table 6.2. An ANOVA reveals that there is a significant
performance difference between the different methods ( F (7 , 2744) = 43 . 0688,
p =0 . 0). Comparing the means shows that the method that performs best is
IRLS, followed by IRLSf, InvVar, MaxConf, Conf, LSf, LS, and last, XCS. The