Design of Neural Networks - Gene Expression Programming - page 389

10.2.4 Direct Mutation of Weights and Thresholds
We have already seen that all genetic operators contribute, directly or indi-
rectly, to move the weights and thresholds around. And, in fact, this constant
shuffling of weights and thresholds is more than sufficient to allow an effi-
cient evolution of GEP-NNs as the appropriate number of random weights
and thresholds can be easily created at the beginning of each run. However,
we can also implement special mutation operators that replace the value of a
particular weight or threshold by another in order to maintain a constant
influx of novelty into the pool of weights and thresholds. These operators,
called direct mutation of weights and direct mutation of thresholds, randomly
select particular targets in the arrays where the weights/thresholds are kept,
and randomly generate a new real-valued number.
Consider, for instance, the array:
W i,j = {-0.433, -1.823, 1.255, 0.028, -1.755, -0.036, -0.128, -1.163 , 1.806, 0.083}
encoding the weights of gene j in chromosome i . Now suppose a mutation
occurred at position 7, replacing the weight -1.163 occupying that position
by -0.494, giving:
W i,j = {-0.433, -1.823, 1.255, 0.028, -1.755, -0.036, -0.128, -0.494 , 1.806, 0.083}
The consequences of this kind of mutation are very diverse: they might be
neutral in effect (for instance, when the gene itself is neutral or when the
weight/threshold is not used in the sub-NN) or they might have manifold
effects. The latter occurs whenever the weight/threshold modified happens
to be used in more than one place in the sub-NN (Figure 10.4).
Interestingly, if all the other operators are being used, this kind of muta-
tion seems to contribute very little to adaptation and, indeed, better results
are obtained when this operator is switched off or used at very low rates.
This suggests that, thanks to the constant restructuring of the neural network
architecture achieved by the chromosomal operators (that is, those that oper-
ate directly on the genome sequence), a well dimensioned initial diversity of
weights and thresholds is more than sufficient to allow their evolutionary
tuning. Typically, I use a small mutation rate of 0.01 for the weights/thresh-
olds and an array length of 10 weights/thresholds for domain lengths equal
to or less than 20. For larger domains one could increase the number of
elements but most of the times an array of length 10 works very well.