Neural Networks Approach - Computational Intelligence in Time Series Forecasting - page 113

A more recent approach to solving the problems of appropriate training termination
departs from some stopping criteria. For instance, based on the automated stopping
criterion of Natarajan and Rhinehart (1997), Iyer and Rhinehart (2000) take as the
stopping criterion the performance-to-cost ratio of the network. Assuming that the
entire cost of a validation set consisting of N data points is Ȟ

, where C is
the cost of single data points, and assuming that the cost of training and test data
sets are CN t and

C N

Ȟ

CN respectively, then the corresponding performance-to-cost

c

ratio is

1

U

,

ECN ( N N )

ce t c Ȟ

where c E is the cumulative error on the test set for a trained network. Setting this
result in relation to the total costs for training termination has reached the
minimum RMS error without the validation cost will become

1

V ( ,

CN ( N )

T C

so their ratio

U NN

NN Q

[ -

,

T C

V

t c

with

(

- .

E

ce

However, even when using the predetermined number of training steps, there will
generally be no guarantee that the network parameters will be adequately tuned.
The optimal stopping strategy is to stop training after the network has learnt all
about the problem class it has to solve. This happens when the training stopping is
effected at the point where the network has reached the maximal generalization .
For the practising expert, this means that the stopping should be triggered exactly
at the point where the network output error has reached its minimal value, This is
known as early stopping . If the training is continued beyond this point, then the
result could be the network overtraining or network overfitting.