Geology Reference
In-Depth Information
Fig. 4.9 Simple LSTM arti
cial neural network
4.4 Training Algorithms
This section gives a brief description of the training algorithms adopted in this
book, namely the BFGS neural network training algorithm, CG training algorithms,
and LM. The purpose of training algorithms is to minimize some measure of error
on the training data by adjusting the model parameters and weights. The network
weights are adjusted until the error is at a minimum or a prede
ned limit has been
reached.
The minimization of the error can be expressed as an optimization in weight
space, if we de
ne the error function as differentiable function of the outputs and
thus function of the weights.
Consider the standard sum-of-squares error function
2
X
n
j¼1
ð
1
2
ð
w
Þ
¼
ð
z
;
t
Þ
¼
z
j
t
j
Þ
ð
4
:
19
Þ
E
E
where z is the network output, t is the target output, and n is the number of nodes in
the output layer. One can obtain the error at each output node, if we differentiate as
follows:
