Evolving Neural and Fuzzy Systems - Computational Intelligence in Time Series Forecasting - page 299

In 1990s, the awareness was spread out among the experts that the evolutionary
algorithms could, in the future, become the most efficient tools for neural networks
training, so that since that time the evolutionary approaches have been very
successfully used in training of backpropagation neural networks (Johnson and
Frenzel, 1992; Porto et al., 1995; Schwefel, 1995), and later in training of recurrent
neural networks (Angeline et al. , 1994; McDonnell and Waagen, 1994).
Surprisingly, although the evolutionary approaches, while based on extensive
computations are generally slower than the gradient methods, it was reported by
some investigators that in network training the evolutionary algorithms have been
considerably faster than the gradient methods (Prados, 1992; Porto et al ., 1995;
Sexton et al ., 1998).
McInerney and Dhawan (1993) pursued an alternative way of network training
by combining two different search algorithms for network training, namely the
backpropagation and the genetic algorithms. They in this way created two
alternative hybrid training algorithm s:
x algorithms that use genetic programming to bring the search process close
to the global optimum and then the backpropagation algorithm has to locate
it more exactly
x an algorithm that first finds (based on backpropagation search) “all” local
minima and then leaves the task for the genetic algorithm to find the
smallest one as the global minimum.
In both algorithms the backpropagation algorithm is used because it is relatively
fast, but it suffers with the inherent troubles associated with gradient methods
being prematurely trapped in local minima. Genetic algorithms, although being
relatively slow, are used because they are robust in finding the global optimum.
Their combination, as expected, profits from the advantage of one algorithm and
from the possibility of counterbalancing the disadvantages of the other. In addition,
in the evolutionary algorithms, unlike in the gradient-based training algorithms, the
error function, i.e. the fitness function, does not require any differentiation and
even need not be continuous.
The joint application of genetic algorithms and gradient methods has been the
subject of extensive research in the 1990s (Kinnebrock, 1994; Zhang et al. , 1995;
Yang et al. , 1996; Yan et al. , 1997).
Nevertheless, in practical applications of genetic algorithms the encoding of
weight values in chromosomes has proven to be the most crucial problem
(Balakrishnan and Honavar, 1995; Curran and O'Riordan, 2003). However, further
research in this area has borne a great number of possible solutions that can be
classified into two categories:
x direct encoding approaches , in which all parameters that define the neural
network ( i.e. weight values, number of nodes, connectivities, etc. ) or some
of them are encoded in gene code
x indirect encoding approaches , which represent a neural network in terms
of assembly instructions or of recipes.
Direct encoding approaches facilitate the reverse operation of decoding that
consists of back-transformation of genotypes into phenotypes. The best illustration