Modeling Methodology: Dimension Reduction and Resampling Methods - Neural Networks: Methodology and Applications - page 212

Fig. 3.3. Percentage of explained variance

1995] for analyzing and reducing the dimensions of nonlinear distributions. It
may be viewed as a nonlinear extension of principal component analysis. CCA
uses a more local criterion than PCA, which allows it to keep the local topol-
ogy of the distribution of input points. An analysis of this method, together
with examples of applications may be found in [Herault 1993; Vigneron 1997].
Figure 3.4 shows CCA applied to dimension reduction of nonlinear data
structures: on the left, the left-hand part shows a set of points defined in
R

2 . The dimension
reduction may therefore be seen as a “nonlinear” projection that retains the
proximity of points and therefore the local topology of the distribution.
In closed structures, such as a sphere or a cylinder, dimensionality re-
duction will inevitably result in some local distortion, as shown in Fig. 3.5,
which shows an example of the projection of a sphere on the plane. The main
principle of CCA is the gradual control of local distortion, during training.
Since the main goal of CCA is a dimensionality reduction that preserves the
local topology, it is ideally suited for the representation of nonlinear varieties.
A variety in

3 , and the right-hand part shows a representation in

R

R p may be defined roughly as a set of points, the local dimensions
of which are smaller than p . The envelope of a sphere defined in

3 is an
example: the dimension of the variety is 2. More strictly, a variety of dimension
q in R q is a sub-set of R n obtained by applying a function defined by R q in R q .

R