Information Technology Reference
In-Depth Information
5.1 Introduction
Unsupervised learning [16] techniques are applicable where supervised learning is
not: if desired outputs for the learning machine are unavailable, one can still try to
discover the structure underlying a dataset. Since data can be always interpreted in
many different ways, some bias is needed to determine which aspects of the input's
structure should be captured in the output of the learning machine.
In general, unsupervised learning has the goal of finding useful representations
of the given data, for example, by:
- grouping examples to clusters,
- reduction of data dimensionality,
- discovering hidden causes of the data, or
- modeling the data density.
If unsupervised learning is successful, the produced representations can be ap-
plied to tasks, such as data compression, outlier detection, classification or to make
other learning tasks easier.
The last application refers to the preprocessing step of pattern recognition sys-
tems. One of the most important problems in pattern recognition is the extraction of
meaningful features from input signals. To compute symbolic information, such as
the class of an observed object, it is often useful to aggregate characteristic aspects
of the observation into a feature vector that is presented to a classification system.
This generally reduces the dimensionality of the data and facilitates generalization
by discarding aspects of the signal that correspond to variances not relevant for clas-
sification or to noise.
A variety of feature extraction methods exist, e.g., for the problem of hand-
written digit recognition [242]. Some methods use the normalized pixel image as
input for a powerful statistical or neural classifier [22]. Others use features having
a medium degree of abstraction, such as moments [204] or coefficients of the KL-
transformation [86]. The most abstract features are extracted by methods that use the
digit's structure for recognition [21]. All these features usually need specific tuning
towards the task at hand. This makes the transfer to other applications difficult. For
this reason, it would be desirable to construct abstract features from a dataset of
example images by means of unsupervised learning techniques.
The Kalman filter and non-negative matrix factorization are unsupervised learn-
ing methods that have already been discussed in Chapter 3.
Clustering. One of the best known methods of unsupervised learning is the K -
means algorithm [145] for clustering of input vectors. It is also known as LBG
method [144] for vector quantization. The algorithm assumes that the data vectors x i
can be grouped into K clusters and replaced by the mean µ c i of the assigned cluster
c i without much loss of information. The K -means algorithm optimizes iteratively
a squared error criterion:
X
N
k 2 .
k x i µ c i
(5.1)
i =1
Search WWH ::




Custom Search