Neural Networks Approach - Computational Intelligence in Time Series Forecasting - page 75

3
Neural Networks Approach
3.1 Introduction
Neural networks are massively parallel, distributed processing systems
representing a new computational technology built on the analogy to the human
information processing system. That is how we know the neural networks today,
but the evolution of artificial neural networks, from the early idea of neuro-
physiologist Heb (1949) about the structure and the behaviour of a biological
neural system up to the recent model of artificial neural system, was very long. The
first cornerstones here were laid down by the neurologists McCulloch and Pitts
(1943) who, using formal logic, modelled neural networks using the neurons as
binary devices with fixed thresholds interconnected by synapses. Nevertheless, the
list of pioneer contributors in this field of work is long. It certainly includes the
names of distinguished researchers like Rosenblatt (1958), who extended the idea
of the computing neuron to the perceptron as an element of a self-organizing
computational network capable of learning by feedback and by structural
adaptation. Further pioneer work was also done by Widrow and Hoff (1960), who
created and implemented the analogue electronic devices known as ADALINE
(Adaptive Linear Element) and MADALINE (Multiple ADALINE) to mimic the
neurons, or perceptrons. They used the least mean squares algorithm, simply called
the delta rule , to train the devices to learn the pattern vectors presented to their
inputs. In 1969, Minsky and Papert (1969) portrayed perceptron history in an
excellent way but their view, that the multilayer perceptron (MLP) systems had
limited learning capabilities similar to the one-layer perceptron system, was later
disproved by Rumelhart and McClelland (1986). Rumelhart and McClelland in fact
showed that multilayer neural networks have outstanding nonlinear discriminating
capabilities and are capable of learning more complex patterns by
backpropagation learning . This essentially terminates the most fundamental
development phase of perceptron-based neural networks.
After a period of stagnation, the research interest was turned to the possible
alternative network variants that have been found in self-organizing networks