MCA EXIN NEURON - Neural-based Orthogonal Data Fitting: The EXIN Neural Networks - page 34

2.4 RAYLEIGH QUOTIENT GRADIENT FLOWS

Let : M → be a smooth function defined on a manifold M and let D :
M → T ∗ M denote the differential, that is, the section of the cotangent bundle 5
T ∗ M defined by

D (w) : T w M → ξ → D (w) ξ ≡ D | w (ξ ) (2.39)

where D (w) is the derivative of at w . To be able to define the gradient

vector field of , a Riemannian metric 6 · , · w on M must be specified. Then the

consequent gradient grad : M → TM is determined uniquely by the following
two properties:

1. Tangency condition:

grad (w) ∈ T w M ∀ w ∈ M (2.40)

2. Compatibility condition:

D (w) ξ = grad (w) , ξ ∀ ξ ∈ T w M (2.41)

n

If M = is endowed with its standard Riemannian metric defined by

T n , then the associated gradient vector is just

ξ , η = ξ η ∀ ξ , η ∈

∂ T

∂w 1 (w) , ∂ ∂w 2 (w) , ... , ∂

∇ (w) =

∂w n (w)

nxn is a symmetric matrix denoting a smooth map which represents
the Riemannian metric, then

n

If Q : →

grad (w) = Q − 1 (w) ∇ (w) (2.42)

which shows the dependence of grad (w) on the metric Q (w) .

Let S n − 1

n n . Denote

= { w ∈ | w = 1 } denote the unit sphere in

: S n − 1 r (w , R ) → the restriction of the Rayleigh quotient of the auto-

space is T w S n − 1 correlation matrix on the unit sphere. The tangent =

5 The tangent bundle TM of a manifold M is the set-theoretic disjoint union of all tangent spaces T w M
of M (i.e., TM =∪ w ∈ M T w M ). The cotangent bundle T ∗ M of M is defined as T ∗ M =∪ w ∈ M T w M ,

where T w M = Hom ( T w M , ) is the dual (cotangent) vector space of T w M .

6 The inner product is determined uniquely by a positive definite symmetric matrix Q ∈

n × n

such

that u , v = u T Q v holds ∀ u , v ∈

n ,a Riemannian metric on M is a family of nondegenerate inner
products · , · w defined on each tangent space T w M , such that · , · w depends smoothly on w ∈ M ;
thus, the Riemannian metric on

n n n × n n , Q (w)

is just a smooth map Q : → such that ∀ w ∈

n defines

is a real positive definite symmetric n × n matrix. Every nondegenerate inner product on

n . The converse is not true. A Riemannian manifold is a smooth manifold
endowed with a Riemannian metric.

a Riemannian metric on