Neural computing in pharmaceutical products and process development - Computer-Aided Applications in Pharmaceutical Technology - page 109

should be stopped. If network overfi tting is likely to occur, it is advisable
to reduce the number of hidden layers or neurons in those hidden layers
(Sun et al., 2003). Validation data are completely kept aside during the
training process and are used only when the training process is complete.
Validation data contain samples that were not presented to the network
during the training process, but it is important to note that these samples
should be within the data space defi ned by the training data. Division of
data in these three sets (training, testing, and validation) is not an easy
task and there are no strict rules on this issue. Some authors propose that
the training set should be at least equal to the number of weights in the
model, multiplied by the inverse of the minimum target error (Baum and
Haussler, 1989); that the ratio of samples in the training set to number of
weight factors should be larger than 10 (Dowla and Rogers, 1995); or
that 65% of data should be used for training, 25% for testing, and 10%
for validation (Looney, 1996).
Developed ANNs are often tested using the cross-validation approach.
This means that the whole data set is divided into equal sized subsets.
The network is then trained the number of times that is equal to the
number of subsets. Each time, a different subset is left out and used for
validation, whereas the rest of the subsets are used for training and testing
of the network. In the case of 'leave-one-out' cross-validation, subsets are
actually data samples, that is, each subset is one sample. Cross-validation
is markedly superior for small data sets (Sun et al., 2003).
In the process of validation, data set values predicted by the network
are compared to those experimentally obtained, and usually correlation
coeffi cient R is calculated to check the appropriateness of the prediction:

￿

￿ ￿

[5.17]

where experimentally obtained x values are compared to predicted (by
the network) y values.
ANNs are, in general, used to predict outputs for the data not input
into the network during its training and testing. Therefore, it is an
interpolation method. The prediction ability of the ANN is restricted to
space limit of input/output data presented to the model for training
(Sun et al., 2003). Extrapolation outside this data space should not
be performed. This should be borne in mind when experiments are
planned.