Information Technology Reference
In-Depth Information
8 An Algorithmic Description
In the previous chapter, the optimal set of classifiers given some data
D
was
defined as the one given by the model structure
).
In addition, a Bayesian LCS model for both regression and classification was
introduced, and it was shown how to apply variational Bayesian inference to
compute a lower bound on ln p (
M
that maximises p (
M|D
.
To demonstrate that the definition of the optimal classifier set leads to useful
results, a set of simple algorithms are introduced that demonstrate its use on a
set of regression tasks. This includes two possible approaches to search the mo-
del structure space in order to maximise p (
M|D
) for some given
M
and
D
), one based on a basic genetic
algorithm to create a simple Pittsburgh-style LCS, and the other on sampling
from the model posterior p (
M|D
) by Markov Chain Monte Carlo (MCMC) me-
thods. These approaches are by no means supposed to act as viable competitors
to current LCS, but rather as prototype implementations to demonstrate the
correctness and usefulness of the optimality definition. Additionally, the presen-
ted formulation of the algorithm seeks for readability rather than performance.
Thus, there might still be plenty of room for optimisation.
The core of both approaches is the evaluation of p (
M|D
) and its comparison
for different classifier sets in order to find the best set. The evaluation of p (
M|D
)
is approached by variational Bayesian inference, as introduced in the previous
chapter. Thus, the algorithmic description of how to find p (
M|D
)alsoprovides
a summary of the variational approach for regression classifier models and a
better understanding of how it can be implemented. Even though not described
here, the algorithm can easily be modified to handle classification rather than
regression. A general drawback of the algorithm as it is presented here is that it
does not scale well with the number of classifiers, and that it can currently only
operate in batch mode. The reader is reminded, however, that the algorithmic
description is only meant to show that the definition of the optimal set of clas-
sifiers is a viable one. Possible extensions to this work, as described later in this
chapter, allude on how this definition can be incorporated into current LCS or
can kindle the development of new LCS.
Firstly, a set of functions are introduced, that in combination compute a
measure of the quality of a classifier set given the data. As this measure can
M|D
 
Search WWH ::




Custom Search