Information Technology Reference
In-Depth Information
•
The
training phase
, where each individual expert (module) is trained until
it learns to solve its particular sub-problem.
•
The
decision integration
. Corresponds to combining the outputs of the
experts, producing an integrated network output. There are three ap-
proaches one can use to accomplish this task: gating network [110]; module
votes [14]; hierarchical integration (which can also use voting and/or gat-
ing networks) [123,111]. The above cited work [200] uses a gating network,
which works as an additional expert trained to recognize the region of the
input space where each of the experts have their regions of expertise, de-
fined in the task decomposition phase.
Once the learning process has finished and a new, unclassified, pattern is
presented to the network, the individual experts compute the class it might
belong, but the gate network selects only a particular output from the expert
it considers to be 'competent' to solve the problem, taking into account the
region of the input space to which the pattern belongs.
As mentioned earlier, task decomposition learning is done before training
the modules. This is usually accomplished by a clustering algorithm [231,5,4,
64]. There are several well-known algorithms to perform clustering, being the
most common ones those based on matrix theory and graph theory. However,
as mentioned in Sect. 6.4, it is also known that these and other kinds of
algorithms often have serious diculties in identifying clusters that reflect the
local structure of the data objects. In Sect. 6.4 we showed how the LEGClust
algorithm successfully partitions the data into meaningful clusters, reflecting
the local structuring of the data. These are the basic reasons why the MNN
classifier described in [200] uses the LEGClust algorithm — a MEE-flavored
clustering approach — for the task decomposition. Moreover, it uses MEE
MLPs as expert modules.
6.5.2 Experiments
A considerable number of classification experiments with several datasets
were performed and reported in [200], using modular neural networks with
task decomposition carried out by three different clustering algorithms:
k
-
means (K-MNN), Spectral Clustering (S-MNN) [166], and LEGClust (EC-
MNN). All neural networks used in the experiments, both the modules and
the gates of the MNN, were MLPs with one hidden-layer, trained with the
R
2
EE risk functional. The MLPs were trained with the backpropagation al-
gorithm and early stopping. The experimental validation was made with the
holdout method: two runs with half the dataset used for training and the
other half for testing, swapping the roles in the second run. Each module
was trained with the input data selected by LEGClust (each module learns
the data from each cluster). The gate network was trained with all the data
labeled by LEGClust.
