Respiratory Motion Estimation with Hybrid Implementation - Prediction and Classification of Respiratory Motion - page 66

The recurrent neural network (RNN) can also be one of the estimation methods for
the predictive control in many application systems [ 20 - 32 ]. Here, RNN is a class
of neural network where connections between units exhibit dynamic temporal
behavior with their synaptic weights. Owing to this dynamic behavior, RNN can
implement dynamical nonlinear multivariable discrete-time systems of arbitrary
complexity [ 33 - 36 ].
A target-tracking estimation can be one of the applications for RNN because of
its adaptive learning, an ability to learn how to do tasks based on the data given for
training or initial experience [ 20 , 21 , 25 , 26 ]. For example, RNN can be used for
the respiratory motion prediction for real-time motion adaptation in the medical
application [ 37 - 42 ]. Because of the self-organized characteristic of neural net-
works, it can have a built-in capability to adapt their synaptic weights to change
based on the given samples in the specific circumstance; thus, it can provide the
better performance in comparison to the conventional methods of the respiratory
motion prediction [ 43 - 47 ]. Intrinsically, training algorithm for RNN became an
issue to improve the performance of dynamical systems with respect to the specific
environment [ 48 ].
There are several algorithms available for training the weights of recurrent
networks based on streams of input-output data. Basically, the most widely used
are the back-propagation-through-time (BPTT) algorithm [ 49 , 50 , 52 - 54 ] and the
real-time recurrent learning (RTRL) algorithm [ 51 - 54 ], which are both based on
computation of the gradient of an output error measure with respect to network
weights. However, the calculation of dynamic derivatives of a recurrent network's
outputs with respect to its weights by RTRL is computationally expensive, since
these derivatives cannot be computed by the same back-propagation mechanism
that was employed in the training of multilayer perceptron (MLP) networks [ 55 ].
As an alternative or improvement of the gradient descent-based methodology,
several authors have noted that the extended Kalman filter (EKF) can also be used
for the purpose of training networks to perform desired input-output mappings
[ 15 - 19 ]. Note that the predictor-corrector property is an intrinsic property of the
Kalman filter, its variants, and extensions. Thus, whereas in traditional applica-
tions of the Kalman filter for sequential state estimation, the roles of predictor and
corrector are embodied in the Kalman filter itself; in supervised-training appli-
cations these two roles are split between the RNN and the EKF. Here, the RRN in
which the input training samples are applied to the recurrent multilayer perceptron
(RMLP) as the excitation, performs the role of the predictor, and the EKF, in
which the training samples of desired response are applied to the EKF as the
observable to provide the supervision, performs the role of the corrector [ 55 ].
With comparison to the gradient descent algorithms, EKF-based algorithms for
recurrent networks do not require batch processing, making them more suitable for
on-line use. To improve the computational requirements of the EKF, Puskorius
et al. proposed decoupled extended Kalman filter (DEKF) as a practical remedy for
the proper management of computational resources [ 15 ]. The author in [ 15 ]
restricted to a DEKF algorithm for which the weights connecting inputs to a
node are grouped together. This approach, however, sacrifices computational