Neural Networks Approach - Computational Intelligence in Time Series Forecasting - page 80

3.3 Networks Used for Forecasting

Hu (1964) was the first to demonstrate - on a practical weather forecasting example
- the general forecasting capability of neural networks. Werbos (1974) later
experimented with the neural networks as tools for time series forecasting, based
on observational data. However, apart from some isolated attempts to solve the
forecasting problems using the then still poorly developed neural networks
technology, the research work in practical application of neural networks had
generally undergone a long period of stagnation. The stagnation was broken and
the work on neural network applications enthusiastically resumed after the
backpropagation training algorithm was formulated by Rumelhart et al. (1986).
Experimenting with the backpropagation-trained neural networks, Werbos (1989,
1990) also concluded that the networks even outperform the statistical forecasting
methods, such as regression analysis and the Box-Jenkins forecasting approach .
Lapedes and Farber (1988) also successfully used neural networks for modelling
and prediction of nonlinear time series.
In the following, typical neural networks used for forecasting and prediction
purposes will be described.

3.3.1 Multilayer Perceptron Networks

Although in the meantime the variety of proposed neural network structures has
grown, the multilayered perceptron has remained the prevailing one and also the
most widespread network structure. This particularly holds for the three-layer
network structure in which the input layer and the output layer are directly
interconnected with the intermediate single hidden layer . The inherent capability
of the three-layer network structure to carry out any arbitrary input-output mapping
highly qualifies the multilayer perceptron networks for efficient time series
forecasting. When trained on examples of observation data, the networks can learn
the characteristic features “hidden” in the examples of the collected data and even
generalize the knowledge learnt, which will be discussed later in detail.
The multilayer perceptron, because of its cascaded structure, performs the
input-output mapping of nonlinearities. For instance, the input-output mapping of a
one hidden layer perceptron network can generally be written as

yf wf f

T
i

x .

¦¦ w

0 hh i

Relying on the Stone-Weierstrass theorem, which states that any arbitrary function
can be approximated with a given accuracy by a sufficiently large-order
polynomial, Cybenko (1989) and Hornik et al . (1989) proved that a single hidden
layer neural network is a universal approximator because it can approximate an
arbitrary continuous function with the desired accuracy provided that the number
of perceptrons in it is high enough. This network capability is general, i.e. it does
not depend on the shape of the perceptron activation function if it is nonlinear.