Evolving Neural and Fuzzy Systems - Computational Intelligence in Time Series Forecasting - page 303

direction in the sense that, given the desired structure of the network to be evolved,
the corresponding connectivity matrix can be generated and translated into the
corresponding binary string.
A direct encoding strategy, although transparent and easy to implement, still
suffers from the scalability problem , which hampers its application in evolving
complex network configurations, because in this case a large connectivity matrix
and much computing time for network evolution are required. In addition, the
potential difficulty of direct encoding strategy is the permutation problem that
disturbs the evolution of proper network architecture.
Indirect encoding strategies are especially popular because they help in
reducing the length of genotypic representation of architectures; this is achieved,
however, at the cost of a reduced feasible search space. In this kind of strategy,
only some characteristics of the architecture ( i.e. those to be evolved) are binary
encoded, which enables a more compact, modular overall network description.
Because being based on a restricted initial information, the indirect decoding
strategies obviously pursue the principle of a growing network, termed
grammatical encoding . Their major advantage is that they favour the modular
design of network structure. Much of pioneer work in this area was done by Kitano
(1990), particularly in defining the matrix rewriting encoding strategy. This
strategy, however, was soon abandoned for the reason that it failed to deliver better
results than the direct encoding strategies.
8.1.1.4 Evolving Complete Network
We now come to the most challenging design task in which the network topology
along with the interconnection weights are simultaneously evolved. The
advantages of such a design approach are, however, accompanied by the
difficulties in finding an adequate representation of genotypes. In the past, apart
from the direct binary encoding that is also applicable here, two additional
encoding strategies have been in use:
x parameterized encoding , in which (instead of a connectivity matrix) the
compact network description is stored in terms of number of layers,
number of neurons within the layers, number of connections between the
layers, etc .
x grammar encoding (Vonk et al. , 1995), particularly matrix grammar
encoding (Kitano, 1990).
In a parameterized encoding network the parameters can be freely encoded. Some
recommendations on this issue have been elaborated by Harp et al . (1990).
Grammar encoding roots in the research achievements of Lindenmayer (1968)
in the area of encoding strategies. Using the biological principle of information
exchange between the cells, Lindenmayer has introduced the so-called L-systems .
To implement this, he defined a special grammar with the parallel representation of
production rules that Boers and Kuiper (1992) later used to evolve neural
networks. The benefits of grammatical encoding are the identification possibility of
network building blocks and the general reusability of development rules. Kitano
(1990) used the productions as the grammar rewriting rules to develop his matrix
rewriting encoding strategy.