Information Technology Reference
In-Depth Information
10
Design of Neural Networks
An artificial neural network (NN) is a computational device that consists of
many simple connected units (neurons) that work in parallel. The connec-
tions between the units or nodes are usually weighted by real-valued weights.
Weights are the primary means of learning in neural networks, and a learning
algorithm is usually used to adjust the weights.
Structurally, a neural network has three different classes of units: input
units, hidden units, and output units. An activation pattern is presented on its
input units and spreads in a forward direction from the input units through
one or more layers of hidden units to the output units. The activation coming
into a unit from other units is multiplied by the weights on the links over
which it spreads. All incoming activation is then added together and the unit
becomes activated only if the incoming result is above the unit's threshold.
In summary, the basic elements of a neural network are the units, the con-
nections between units, the weights, and the thresholds. And these are the
elements one must encode in a linear chromosome in order to simulate fully
a neural network so that it can adapt in a particular selection environment.
For that I created a special chromosomal organization that explores some of
the elements of the architecture used for polynomial induction (see chapter 7
Polynomial Induction and Time Series Prediction).
In this chapter we are going to learn how to modify the chromosomes of
gene expression programming so that a complete neural network, including
the architecture, the weights, and thresholds, can be totally encoded in a
linear chromosome. Furthermore, we will see how this chromosomal organi-
zation allows also the training (evolution, in this case) of the neural network
using the selection and modification mechanisms of gene expression
programming, allowing the discovery of solutions in the form of neural net-
works. And as conventional neural networks, these evolvable neural networks
can obviously be used to solve different kinds of problems: from symbolic
