Self-Organizing Maps and Unsupervised Classification - Neural Networks: Methodology and Applications - page 408

of the cortical map. Thus, at the end of training, neuronal connection weights
have converged in order to guarantee that a neuron has discriminative abilities,
i.e. that it is active only for a subset of the observations of the training set. A
neuron c , which is represented by the reference vector w c , may be considered
as an average observation that is a compressed representation of the data set
P c of the observations that it has been assigned. Thus, the whole neuronal
map performs a vector quantization of the whole data set D, which is obtained
by the analysis of the training set A . The quality of the quantization (faithful
or not) strongly depends on whether the training set is representative or not.

7.3.6 Architecture and Adaptive Topological Maps

Self-Organizing Maps produce simple representations of data that are embed-
ded in spaces of very large dimension. That representation is performed in
a low-dimension discrete set C with a graph structure. The problem of the
choice of architecture consists in selecting a suitable graph structure for the
map, i.e. a structure that is appropriate for the specific problem of interest.
Therefore, one must define a measure of the adequacy of a map to the prob-
lem of interest. The data set D and the map C are related in two ways: the
embedding of C into D that maps each neuron c of C onto a reference vector
w c of C , and the allocation function χ of D into C , which associates to each
observation vector in D aneuron c of the map. Those two mappings have to
be topologically consistent in the following sense:

• Two neurons that are neighbors in the map C must be represented by two

reference vectors that are close in D .

• Reciprocally, data that are approximately similar must be allocated by χ

to the same neuron or to neighboring neurons.

If the dimension of the map does not fit with the underlying dimension of the
data cloud (dimension of the manifold that is generated by the observations),
two observations that are close in data space D may be allocated to distant
neurons in the map. Yet, the topological consistency is an interesting property
because it allows reducing the dimension of the data while retaining similari-
ties. In previous sections, it was assumed that the graph structure of the map
was given a priori. That choice was not data-driven, which has shortcomings:
it does not guarantee the adequacy between the structure of the map and the
internal structure of the data distribution.
Usually, in applications, the dimension of the data space may be very large
if the number of features describing the data is large, but the observations are
not distributed uniformly in the data set. They are located in specific regions
with various concentrations. Reference vectors must be located in high-density
regions, and one must avoid wasting reference vectors by locating them in void
regions. The choice of the graph structure of the map is very important be-
cause, when it is appropriate, it guarantees the topological consistency of the
map and a good representation of the underlying data probability distribution.