A Probabilistic Model for LCS - Design and Analysis of Learning Classifier Systems - page 59

matched by only this classifier, the classifier models these observations in full,
independent of the other classifiers 5 .
Let us consider how observations are modelled that are matched by more than
one classifier: as a consequence of (4.12), the non-negative responsibilities of all
matching classifiers sum up to 1, and are therefore between 0 and 1. Hence,
by (4.14), each matching classifier assigns less weight to modelling the observa-
tion than if it would be the only classifier matching it. Intuitively, overlapping
classifiers “share” the observation when modelling it.
To summarise, i) a classifier only models observations that it matches, ii)
it assigns full weight to observations that no other classifier matches, and iii) it
assigns partial weight to observations that other classifiers match. Expressed dif-
ferently, a classifier fully models all observations that it matches alone, and par-
tially models observations that itself and other classifiers match. Consequently,
the local model provided by a classifier cannot be interpreted by their matching
function alone, but also requires knowledge of the gating network parameters.
Additionally, when changing the model structure as discussed in Sect. 3.2.6 by
adding, removing, or changing the localisation of classifiers, all other overlapping
classifiers need to be re-trained as their model is now incorrect due to changing
responsibilities. These problems can be avoided by training the classifiers inde-
pendently of each other, making the classifier model more transparent.

4.4.3 Introducing Independent Classifier Training

Classifiers are trained independently if we replace the responsibilities r nk in
(4.14) by the matching functions m k ( x n )toget

K K

m k ( x n )ln p ( y n |

x n , θ k ) .

max
θ

(4.24)

n =1 k =1

Hence, a classifier models all observations that it matches, independent of the
other classifiers. Thus, the first goal of simplifying the intuition about what a
single classifier models is reached. While this does not cause any change for ob-
servations that are matched by a single classifier, observations that are matched
by several classifiers are modelled by each of these classifiers independently rat-
her than shared between them. This independence is shown by the graphical
model in Fig. 4.6, which illustrates the model of a single classifier k .
With this change, the classifiers are independent of the responsibilities and
subsequently also of the gating network. Thus, they can be trained completely
independently, and the model structure can be modified by adding, removing,
or changing classifier locations without re-training the other classifiers that are
currently in the model, and thereby make searching the space of possible model
structures more ecient.

5 XCS has the tendency to evolve sets of classifiers with little overlap in the areas
that they match. In such cases, all classifiers model their assigned observations in
full, independent of if they are trained independently or in combination.