Information Technology Reference
In-Depth Information
Fig. 6.17. Two discriminant hyperspherical surfaces in dimension N =2.The
center of the sphere, indicated by a cross, may be outside the region occupied by
the examples
6.5.2 Constructive Heuristics
As mentioned above, if the discriminant surface is neither linear nor spherical,
the classification problem may be turned into a regression problem, and the
techniques of training and model selection of Chap. 2 may be applied. In that
case, the neurons must have differentiable activation functions. The number of
hidden units has to be postulated a priori, and is generally adjusted by com-
paring results obtained with different network sizes, at the cost of time and
resources. An alternative solution, presented in this chapter, is to determine
the discriminant surfaces by combining linear and spherical separations using
binary hidden neurons. Those are included in the network sequentially, follow-
ing constructive heuristics that use different criteria to associate binary inter-
nal representations to the input patterns in the training set. The hidden units
states corresponding to an input pattern constitute its internal representation .
Its dimension is the number of hidden units. If those internal representations
are linearly separable, an output perceptron connected to the hidden units
can learn the discrimination. The probability that the pattern belongs to the
class assigned by the classifier may be estimated using the results described in
the section “Probabilistic formulation of learning” of the present chapter. In
any case, whether the neurons used are binary or continuous, it is important
to use the techniques for model selection through cross validation, or statisti-
cal tests, explained in Chap. 2, to avoid overfitting: a classifier that classifies
correctly all training patterns may be unable to generalize satisfactorily.
Search WWH ::




Custom Search