A Learning Classifier Systems Model - Design and Analysis of Learning Classifier Systems - page 38

Having underlined the importance of knowing the underlying model of a method,
the family of parametric models is introduced, in order to identify LCS as a
member of that family. The description is based on reflections on what classifiers
actually are and do, and how they cooperate to form a model. While a general
LCS model overview and its training is given, more details have to wait until
after the a formal probabilistic LCS model is introduced in the following chapter.

3.2.1 Parametric Models

The chosen hypothesis during model training is usually determined by a set of
adjustable parameters θ . Models for which the number of parameters is inde-
pendent of the training set and remains unchanged during model training are
commonly referred to as parametric models. In contrast, non-parametric models
are models for which the number of adjustable parameters either depends on the
training set, changes during training, or both.
Another property of a parametric model is its structure (often also referred

to as scale ). Given a model family, the choice of structure determines which mo-
del to use from this family. For example, considering the family of feed-forward
neural networks with a single hidden layer, the model structure is the number of
hidden neurons and the model parameters are the weights of the neural connec-
tions. Hence, the model structure is the adjustable part of the model that remain
unchanged during training but might determine the number of parameters.
With these definitions, our aims can be re-formulated: Firstly, and adequate
model structure

M

is to be found that provides the model hypotheses f M ( x ; θ ).
Secondly, the model parameter values θ need to be found such that the expected
risk for the chosen loss function is minimised.

M

3.2.2 An LCS Model

An LCS forms a global model by the combination of local models , represented by
the classifiers. The number of classifiers can change during the training process,
and so can the number of adjustable parameters by action of the GA. Hence, an
LCS is not a parametric model per se.
An LCS can be turned into a parametric model by assuming that the number
of classifiers is fixed, and that each classifier represents a parametric model.
While this choice seems arbitrary at first, it becomes useful for later development.
Its consequences are that both the number of classifiers and how they are located
in the input space are part of the model structure and are not modified while

adjusting the model parameters. The model parameters θ are the parameters of
the classifiers and those required to combine their local models.
Consequently, training an LCS is conceptually split into two parts: Finding
a good model structure

M

, that is, the adequate number of classifiers and their
location, and for that structure the values of the model parameters θ .This
interpretation justifies calling LCS adaptive models .

M