Information Technology Reference
In-Depth Information
Inertia and Matrix Norm
That result is familiar in linear algebra since the inertia of the scatter diagram
corresponds to the Frobenius matrix norm, which is expressed as a function
of singular values,
=
i,j
=
j
X
F
x
2
ij
σ
2
j
.
The projection matrix
P
p×q
associated with the first
q
axesistherefore
represented by the first
q
vectors of matrix
V
p×p
. The contribution to the
inertia of each main axis is given by the ratio of
σ
2
j
to the sum
σ
1
+
σ
2
+
···
+
σ
2
p
.
The contribution of the first
q
axes is:
q≤p
σ
2
j
q
j
=1
σ
2
I
q
=
j
⇒
I
q
=
I
p
.
p
j
=1
σ
2
j
j
=1
The quality of the dimension reduction depends on the value of
q
.Thereis
no general rule for determining the best value. A few rules used to determine
the number
q
of components [Saporta 1990] may be mentioned:
•
The part of the explained inertia to contribute at least a fixed percentage
of the inertia,
•
Kaiser's rule,which retains eigenvalues larger than the average of eigenval-
ues (for reduced centered data, that consists in retaining the eigenvalues
that are larger greater than 1, since the sum of the eigenvalues is equal
to
n
),
•
The “scree test” which, from the curve of
I
q
as a function of
q
=1
,
2
,...,n
,
selects the value of
q
that corresponds to the 1st break in the gradient, as
shown in the example given in Fig. 3.3 with a break in the gradient from
the 4th eigenvalue.
Before applying PCA systematically, it must be remembered that the so-
called principal component is defined from the criterion concerning the inertia
of the scatter diagram. For certain problems, the principal component is by
far not the most informative aspect. For example, in a set of human faces of
several different races, the recognition of race is based more fully on the sec-
ondary components; the first component is more representative of the average
characteristics of the faces.
3.5 Curvilinear Component Analysis
For more complex distributions, dimensionality reduction may require nonlin-
ear processing. Curvilinear component analysis was proposed by [Demartines
Search WWH ::
Custom Search