Geoscience Reference
In-Depth Information
corresponding to a unique cluster. ART networks use a simple method of representation wherein
each cluster is represented using the weight vector for an individual prototype unit. Similarities
drive the clustering process. Vectors that are grouped into the same cluster are similar which means
that associated input patterns are close to each other in terms of input space. If an input vector is
close to a prototype, then it is considered a member of the prototype's cluster, with local differences
being attributed to unimportant features or to noise. When the input data and stored prototype are
sufficiently similar, then these two items are said to resonate (from which the name of this tech-
nique is obtained). It should be stressed that there is no set number of clusters - additional output
nodes (clusters) being created as and when needed. An important item in this implementation of the
stability-plasticity dilemma is the control of partial matching between new feature vectors and the
number of stored (learned) patterns that the system can tolerate. Indeed, any clustering algorithm
that does not have a pre-specified number of clusters, or does not in some manner limit the growth
of new clusters, must have some other mechanism or parameter for controlling cluster resolution and
for preventing excessive one-to-one mapping. Each ART network has a vigilance parameter (VP)
that is used for this purpose and which is explained in the next section.
The ART-1 learning algorithm has two major phases. In the first phase, input patterns are pre-
sented and activation values calculated for the output neurons. This defines the winning neuron. The
second phase calculates the mismatch between the input pattern and the current pattern associated
with the winning neuron. If the mismatch is below a threshold (VP), then the old pattern is updated
to accommodate the new one. But if the mismatch is above the threshold, then the procedure con-
tinues to look for a better existing concept-output unit or it will create a new concept-output unit.
ART networks will be stable for a finite set of training examples because even with additional
iterations, the final clusters will not change from those produced using the original set of training
examples. Thus, these tools possess incremental clustering capabilities and can handle an infinite
stream of input data. ART networks also do not require large memories for storing training data
because their cluster prototype units contain implicit representation of all previous input encoun-
ters. However, ART networks are sensitive to the presentation order of the training examples and
might produce different clusterings on the same input data when the presentation order of patterns is
varied. Similar effects are also present in incremental versions of traditional clustering techniques,
for example,
k
-means clustering is also sensitive to the initial selection of cluster centres.
Most ART networks are intended for unsupervised classification. The simplest of the ART
networks are ART-1 that uses discrete data (Carpenter and Grossberg 1987a) and ART-2 which
uses continuous data (Carpenter and Grossberg 1987b). A more recent addition to this collection
is a supervised version of ART-1 called ARTMAP (Carpenter et al. 1991). There is also fuzzy
ARTMAP, a generalisation of ARTMAP for continuous data, that was created with the replace-
ment ART-1 in ARTMAP with fuzzy ART (Carpenter et al. 1992). In this instance, fuzzy ART
synthesises fuzzy logic and ART by exploiting the formal similarities between (1) the computations
of fuzzy subsethood and (2) the dynamics of prototype choice, search and learning. This approach is
appealing because it provides an agreeable integration of clustering with supervised learning on the
one hand and fuzzy logic and ART on the other. A comparison of fuzzy ARTMAP with backpropa-
gation and maximum likelihood classification for a real-world spectral pattern recognition problem
is reported in Gopal and Fischer (1997).
13.7.4 S
elf
-o
rganiSing
f
eature
M
aP
Another important class of powerful recurrent CNNs are self-organising feature maps (otherwise
referred to as self-organising feature map
[SOFM] networks or Kohonen networks). SOFM net-
works are used for vector quantisation and data analysis and these tools have been foremost and in
the main developed by Kohonen (1982, 1989). These quantitative
mapping tools
, which are at least
in part based on the structure of the mammalian brain, will, in addition to the classification process,
also attempt to preserve important topological relationships. Although the standard implementation
