Self-Organizing Maps and Unsupervised Classification - Neural Networks: Methodology and Applications - page 398

computing the value for which the gradient of the cost function is zero.
That defines the new reference vector set

= r∈C K ( δ ( c,r )) Z r w c

r∈C K ( δ ( c,r )) n r

,

where Z r = z i ∈A, χ ( z i )= r z i is the sum of all observations of the training
set A that are allocated to neuron r . Note that each new reference vector
is the center of mass of the mean vector of the subsets P r ∩

A , each center

of mass being weighted by K ( δ ( c,r )) n r .

To summarize, we get the following algorithm:

Batch Algorithm of Topological Maps: T Fixed

1. Initialization : t = 0. Select the p reference vectors (randomly, in general),
the structure of the map and its size, the maximum number of iterations
N iter .
2. Iteration t . The reference vector set W t− 1 is known from the previous

step,
Allocation phase : update the allocation function χ t that is associated to
W t− 1 . Then each observation z i is allocated to a reference vector accord-
ing to

χ T ( z ) = arg max
r∈C

2 =argmax
r∈C

d T ( z , w r );

K T ( δ ( c,r )) z − w c

c∈C

Minimization phase : apply relation

=
r∈C

K ( δ ( c,r )) Z r /
r∈C

w c K ( δ ( c,r )) n r

to compute the new set W t of reference vectors.
3. Iterate until the maximum iteration is reached, or until J som stabilizes in
a local minimum according to a stopping criterion.

As for k -means, a close look at the behavior of self-organizing maps for sim-
ple examples gives insight into the implementation problems that may arise.
The following numerical experiment illustrates the role of the temperature
parameter T in the minimization. The data are the same as on Fig. 7.2 in the
section on k -means. As mentioned before, the data are a sample of a uniform
mixture of four Gaussian modes with partial pairwise overlap. On Fig. 7.9, the
results (topological graph and quantization) are displayed in data space. Ko-
honen's representations are used. The observations and the reference vectors
are shown on the same diagram. The map-induced topology of neighboring
neurons is shown as well. Reference vectors that are relative to neighboring
neurons on the graph are connected by edges on the picture. At initialization,
reference vectors were selected randomly around the center of the observation