A Probabilistic Model for LCS - Design and Analysis of Learning Classifier Systems - page 63

4.6 Discussion and Summary

In this chapter, a probabilistic LCS model was introduced as a generalisation
of the MoE model, by adding matching as a form of forced localisation of the
experts. Additionally, training was simplified by handling the classifiers inde-
pendently of the gating network. The resulting probabilistic LCS model acts as
the basis for further development in this topic. In fact, solving (4.24) to train
the classifiers forms the basis of the next chapter. The chapter thereafter deals
with the mixing model by describing how the solution to (4.13) can be found
exactly and by approximation. Thus, in combination, the following two chapters
describe in detail how the model can be trained by maximum likelihood, both
by batch learning and incrementally.
Even though we have approached the LCS model from a different perspec-
tive, the resulting structure is very similar to a currently existing LCS:XCS
and its derivatives follow the same path of independently training the classifier
models and combining them by a mixing model. While in XCS it is not explicitly
identified that the classifiers are indeed trained independently, this fact becomes
apparent in the next chapter, where it is shown that the classifier parameter
update equations that result from independent classifier training resemble those
of XCS. The mixing model used by XCS does not conform to the generalised
softmax function but rather relies on heuristics, as is demonstrated in Chap. 6.
Independent classifier training moves LCS closer to ensemble learning. This
similarity has been exploited recently by Brown, Marshall and Kovacs [29, 162],
who have used knowledge from ensemble learning and other machine learning
methods to improve the performance of UCS [161]. Even though this direction
is very promising, the direct link between LCS and ensemble learning will not
be considered further in this topic.
In summary, amongst currently popular LCS, the presented model is most
similar to XCS(F). It combines independently trained classifiers by a mixing
model to provide a global model that aims at explaining the given observations.
This particular model type was chosen not to represent the “best” LCS model,
but as an example to demonstrate the model-based approach. Other LCS model
are equally amendable to this approach, but for the beginning, only a single
model type is fully considered. As in this model type the classifiers are trained
independently of each other, it is possible to concentrate on the training of a
single classifier, as is done in the following chapter.