Information Technology Reference
In-Depth Information
Gruau (1994) represented neural networks as grammar trees, called cellular
encoding , which is similar to the edge encoding strategy.
In the graph grammar encoding , the network is understood as a lattice made
up of functions and terminals. Each node of the lattice, which is seen as a function
(neurons) or a terminal (input variables), is provided with the information
concerning the connections to other nodes, the weights of the connections, bias,
etc .
An entirely different indirect encoding strategy was proposed for encoding the
developmental rules that are to be optimized instead of direct optimization of the
network architecture. The development rules are similar to the IF-THEN rules used
in production systems, written in recursive form.
Some interesting findings in evolving the learning rules have been reported.
Chalmers (1990) was the first to report on automatic evolving of the delta learning
rule , and Harp and Samad (1991) reported on evolving the rules that can learn and
adapt the network training parameters, such as training speed and network training
accuracy.
However, the inherent problem of encoding neural networks in gene code is
still the well-known permutation problem , created by the fact that different
genotypes can produce equivalent networks, because the fitness and the network
function could produce the permutation of hidden nodes. This is evident from
Figure 8.3, which represents the differently encoded network shown in Figure 8.1.
Both networks are topologically equivalent (Tettamanzi and Tomassini, 2001).
5
6
10
4
3
0100
1010
0101
0110
0011
Figure 8.3. Differently encoded network presented in Figure 8.1
The permutation problem considerably decreases the suitability of the genetic
algorithm as a training tool for feedforward networks.
8.1.1.5 Evolving the Activation Function
So far, we have ignored the evolving issue of the neuron activation function,
assuming silently that it is given in advance by the network expert, preferably as a
sigmoidal function. This is indeed not always the case, but it is assumed for
simplicity that the activation functions of all neurons in a layer or in the entire
network are equally shaped. The first trial to evolve both the activation functions,
placed in nodes as a node transfer function, and the network architecture was done
Search WWH ::




Custom Search