Self-Organizing Maps and Unsupervised Classification - Neural Networks: Methodology and Applications - page 407

Fig. 7.14. 2-D topological map. The network has two layers: an input layer contains
the observations, and a representation layer, for which a topology must be defined
(distance δ between neurons and neighborhood function). Each neuron c stands for
a reference vector w c ; it is fully connected to the input layer. The connection weight
vector of each neuron c is the reference vector w c associated to neuron c

map. A 2-layer neural network provides a joint representation of the map and
of the reference vectors (see Fig. 7.14):

• Observations are present in the input layer of the network. The state of

each unit is a component of one observation. Therefore, the number of
neurons of that layer is equal to the dimension of input space.

• The second layer is the neuronal map. The structure of the map may

be decided a priori. In more flexible versions, the structure can evolve
during training. The neurons simply compute a distance. Each neuron c is
connected to all input units. The reference vector that is associated to the
current neuron c of the second layer is actually the vector of connection
weights afferent to neuron c . Each neuron has n afferent connections since
it is connected to all units of the input layer. When an observation z
is presented to the input layer, the output of neuron c of the map is

2 .

z − w c

During training, the network connection weights change using various up-
dating rules. Thus, the neurons of the map compute their distances to the
current observation in parallel. The main feature of the self-organization
process is to focus the adaptation process on the most active area of the
map. Kohonen's original algorithm, which is the simplest one, considers that
the active zone is the neighborhood of the neuron c that is closest to the
observation under consideration, i.e. whose output

2 is smallest. That
neighborhood generates topological constraints that lead to self-organization.
As indicated in the previous section, it models in a simple way, the lateral
coupling between an active neuron and its neighbors on the connection graph

z − w c