Self-Organizing Maps and Unsupervised Classification - Neural Networks: Methodology and Applications - page 410

K T ( δ ( c,r ))
z i ∈P r

2

2
c

2 +
z i ∈P c

1

= z i − w c z i − w r

r = c

+ K T ( δ ( c,c ))
c

2 .

z i − w c

z i ∈P c

This decomposition gives two terms, the sum of which must be minimized:

The second term is I of k -means, weighted by K T ( δ ( c,c )) = K (0). Its
influence is controlled by the temperature parameter T : the smaller the
temperature, the more influential that term during minimization. It tends
to build a partition into compact subsets, and the reference vectors tend
to be the centers of mass of the partition subsets.

•

• The first term enforces the topological consistency constraint: if two neu-

rons r and c are close on the map, K T ( δ ( c,r )) is large, because δ ( c,r )is
small. Minimizing that term decreases the distance between the subsets P c
and P r that are allocated to c and r . Thus, proximity on the map enforces
proximity in the data set.

The above form of J som also gives insight into the presentation of the algo-
rithm as consisting in two different steps that depend on the temperature T
(see above the section on batch optimization algorithm of topological maps).
The first step occurs when T is large: the first term is dominant, and the task
of the algorithm is mainly to guarantee the topological consistency of the
map. The second step occurs at lower temperature. In that case, the second
term becomes dominant and the algorithm essentially minimizes the inertia
of the partition. The temperature allows performing the appropriate tradeoff
between the two terms of J som . Since the topological self-organization occurs
during the first part of training, then the minimization is useful to obtain
subsets that are as compact as possible. It is the k -means phase of the algo-
rithm that consists in approximating locally the data distribution. Thus, the
algorithm may be cursorily described as a version of the k -means algorithm
subject to the constraint of topological consistency of the reference vectors
with the map.
The following experiment gives insight into the difference between SOM
and k -means. We consider again the example that was displayed on Fig. 7.2[d]
to illustrate k -means. In that case, a topological map with a 1D chain structure
is used with four neurons, and the parameters of the map are estimated from
the training set, generated from a mixture of four Gaussians.
The four reference vectors were initialized at the bottom right of the fig-
ure just as for the previous k -means experiment. The two solutions that are
obtained by k -means and SOM are shown on Fig. 7.15. The map topology
constraint allows locating the four neurons at the centers of the four Gaussian
modes. Thus, the SOM algorithm was able to determine the solution of the
k -means problem under the topological consistency constraint (Fig. 7.15 [b]);