Neural Networks Approach - Computational Intelligence in Time Series Forecasting - page 99

Therefore, the eigen-values and eigen-vectors of G are ^

`

and ^ i

O i P

z

respectively. G can be made positive definite by increasing P until

O ! for

P 0

i

all i .
Therefore, the Levenberg-Marquardt modification to Gauss-Newton method is

1

T T

ª J P I º e (3.31)

WWJ ww J ww

¬ ¼

k 1 k k k k k

whereby the parameter µ is multiplied by some factor ȕ whenever a step would
result in an increased value of Vw . When a step reduces this value, µ is divided

by ȕ . Notice that when µ is large the algorithm becomes steepest descent with the
step size approximately 1/

( )

k

P On the other hand, for small µ the algorithm becomes

Gauss-Newtonian.
Obviously, the calculation of the Jacobian matrix is the key step in applying
this algorithm. At first, all the adjustable parameters of the network should be
arranged in one column vector

w For a neural network mapping problem the
terms in the Jacobian matrix can be computed by simple modification to the
backpropagation algorithm (Hagan and Menhaj, 1994). In the standard
backpropagation version, partial derivatives of the performance function with
respect to the adjustable parameters are needed, while in Levenberg-Marquardt
algorithm the derivative of the error is needed for the Jacobian matrix. This means
that the Jacobian matrix can be calculated using the sensitivity term of the
performance index derived in the standard backpropagation algorithm with one
modification at the final layer, i.e. by dropping the error term (Hagan and Menhaj,
1994). The Jacobian matrix computation for a neuro-fuzzy network is described in
Chapter 6.
The algorithm described above can easily be extended to train the multilayer
perceptron networks.

.

3.5 Forecasting Methodology

Forecasting methodology is generally understood as a collection of approaches,
methods, and tools for collection of time series data to be used for forecast or
prediction of future values of the time series, based on past values. The forecasting
methodology includes the following operational steps:

x data preparation for forecasting, i.e. acquisition, preprocessing,
normalization, and structuring of data, determination of training and test
data sets, and the like
x network architecture determination , i.e. selection of the type of network to
be used for forecasting, determination of number of network input and
output nodes, number of layers, the number of neurons within the layers,
determination of interconnections between the neurons, selection of neuron
activation functions, etc .