NONLINEAR SEQUENTIAL STATE ESTIMATION FOR SOLVING PATTERN CLASSIFICATION PROBLEMS - Adaptive Signal Processing: Next Generation Solutions - page 342

Table 6.1 Summary of the EKF algorithm for supervised training of the MLP

Training sample: fu n , d n g , n ¼ 1, 2, ...N
where u n is the input vector applied to the MLP and d n is the corresponding desired response.
MLP and Kalman filter: Parameters and variables

b (.,.) : vector-valued measurement function
B : linearized measurement matrix
w n : weight vector at time step n
ˆ njn 2 1 : predicted estimate of the weight vector
ˆ njn : filtered estimate of the weight vector
y n : output vector of the MLP produced in response to u n
Q n : covariance matrix of dynamic noise v n
R n : covariance matrix of measurement noise v n
G n : Kalman gain
P njn 2 1 : prediction error covariance matrix
P njn : filtering error covariance matrix

Initialization : The initial weights w 1 j 0 are drawn from a uniform distribution with zero mean
and variance equal to the reciprocal of the number of synaptic connections feeding into a node
(fan-in). The associated covariance of the initial weight estimate is fixed at dI , where d can be
'some' multiples of 10, and I is the identity matrix.
Recursive computation :For n ¼ 1, 2, ... compute the following:

G n ¼ P njn 1 B n [ B n P njn 1 B n þR n ] 1
a n ¼ d n b n ( w njn 1 , u n )
ˆ njn ¼ w njn 1 þG n a n
P njn ¼ P njn 1 G n B n P njn 1
ˆ nþ 1 jn ¼ ˆ njn
P nþ 1 jn ¼ P njn þQ n

2. The weight (state) update , defined by

w njn ¼ w njn 1 þG n a n (6 : 7)

where w njn 1 is the predicted (old) state estimate of the MLP's weight vector w
at time n , given the desired response up to and including time ( n 2 1), and w njn
is the filtered (updated) estimate of w on the receipt of observable d n . The matrix
G n is the Kalman gain , which is an integral part of the EKF algorithm.

Examining the underlying operation of the MLP, we find that the term
b ( w njn 1 , u n ) is the actual output vector y n produced by the MLP with its old
weight vector w njn 1 in response to the input vector u n . We may therefore rewrite
the combination of (6.6) and (6.7) as a single equation:

w njn ¼ w njn 1 þG n ( d n y n ) :

(6 : 8)