NONLINEAR SEQUENTIAL STATE ESTIMATION FOR SOLVING PATTERN CLASSIFICATION PROBLEMS - Adaptive Signal Processing: Next Generation Solutions - page 340

replaced by the new inner product

T ( x i ) F ( x j )

k ( x i , x j ) ¼F (6 : 3)

which is called the Mercer kernel. This terminology follows from the fact that
the functions assigned to the hidden units of the RBF network must satisfy
the celebrated Mercer's theorem, which is indeed satisfied by the choice of
Gaussian functions. Equation (6.3) provides the basis for the kernel trick,
which states the following.

Insofar as pattern-classification in the output space of the RBF network is concerned, spe-
cification of the Mercer kernel k ( x i , x j ) is sufficient, the implication of which is that there
is no need to explicitly compute the adjustable parameters of the output layer.

For this reason, a network structure exemplified by the RBF network, is referred to as a
kernel machine and the SVM learning algorithm used to train it is referred to as a
kernel method.

6.3 SUPERVISED TRAINING FRAMEWORK OF MLPs USING
NONLINEAR SEQUENTIAL STATE ESTIMATION

To describe how a nonlinear sequential state estimator can be used to train a MLP in a
supervised manner, consider a MLP with s synaptic weights and p output nodes. With
n denoting a time-step in the supervised training of the MLP, let the vector w n denote
the entire set of synaptic weights in the MLP computed at time step n . For example, we
may construct w n by stacking the weights associated with neuron 1 in the first hidden
layer on top of each other, followed by those of neuron 2, carrying on in this manner
until we have accounted for all the neurons in the first hidden layer; then we do the
same for the second and any other hidden layer in the MLP, until all the weights in
the MLP have been accounted for in the vector w n in the orderly fashion just described.
With sequential state estimation in mind, the state-space model of the MLP under
training is defined by the following pair of models (see Fig. 6.4).

I

Figure 6.4 Nonlinear state-space model depicting the underlying dynamics of a MLP
undergoing supervised training.