Databases Reference
In-Depth Information
Previous
movement
Current computed
movement
Momentum Adjusted Movement
Momentum = 0
(a)
Momentum = 1
(b)
Momentum = 0.5
(c)
Figure 4.9
Momentum Adjustments
The addition of learning rate and momentum parameters has improved the
ability of ANNs to train toward more globally optimal locations. The question
then becomes, “What should the learning rate and momentum be set to, in order
to optimize the training process?” The answer varies from dataset to dataset.
There is no single best combination of parameter settings. VisMiner's method
for setting these two parameters, while training, is presented in Chapter 5.
ANN Advantages and limitations
As previously pointed out, the primary advantage of artificial neural networks is
the ability to fit almost any dataset. Yet this is also a weakness - if not monitored
carefully during training, it is very easy to overfit the model to the training data.
The VisMiner ANN monitoring process, which will assist in avoiding an overfit
model, is presented in Chapter 5.
Other issues with respect to ANN models are:
Reproducibility of results - the initial neuron weights of the ANN are
random values. Therefore, depending on the initial starting position, you are
not guaranteed to arrive at the same final solution from run to run and, as
previously stated, that solution may not be an optimal solution. With most
datasets, this is not usually an issue if the learning rate and momentum
parameters are monitored and adjusted as training progresses.
Interpretability of the model - from a mathematical perspective, ANNs are
complex. You cannot look at a single coefficient value in the model formula
to determine the contribution to the overall prediction of a single input
attribute.
