Correlation analysis - Statistical Signal Processing of Complex-Valued Data - page 97

unless all singular values of C have even multiplicity. Thus, unlike in the rotational and
reflectional case, the solution to the maximization problem ( 4.26 ) does not lead to a
diagonal matrix, but a matrix K

H

= E with diagonal blocks instead. This means

that generally the internal description ( ) cannot be made cross-proper because this

would require a transformation jointly operating on x and y - yet all we have at our
disposal are widely linear transformations separately operating on x and y .

,

4.2 Invariance properties

In this section, we take a closer look at the properties of CCA, MLR, and PLS. Our
focus will be on the invariance properties that characterize these correlation analysis
techniques.

4.2.1 Canonical correlations

Canonical correlation analysis (CCA), which was introduced by Hotelling (1936 ), is
an extremely popular classical tool for assessing multivariate association. Canonical
correlations have a number of important properties. The following is an immediate
consequence of the fact that the canonical vectors have identity correlation matrix, i.e.,
R ξξ = I and R ωω = I in the rotational and reflectional case, and R ξξ = I and R ωω =

I

in the total case.

Result 4.2. Canonical correlations (rotational, reflectional, and total) satisfy 0 ≤

k i ≤ 1 .

A key property of canonical correlations is their invariance under nonsingular linear
transformation, which can be more precisely stated as follows.

Result 4.3. Rotational and reflectional canonical correlations are invariant under non-
singular linear transformation of x and y , i.e., ( x My ) have the same

rotational and reflectional canonical correlations for all nonsingular N

, y ) and ( Nx ,

C n × n ∈ and

C m × m . Total canonical correlations are invariant under nonsingular widely linear
transformation of x and y , i.e., ( x

M ∈

, y ) and ( N x , My ) have the same total canonical

n × n m × m . Moreover, the canonical

correlations for all nonsingular N ∈ W and M ∈ W

correlations of ( x , y ) and ( y , x ) are identical.

We will show this for rotational correlations. The rotational canonical correlations k i
are the singular values of C , or, equivalently, the nonnegative roots of the eigenvalues of
CC H . Keeping in mind that the nonzero eigenvalues of XY are the nonzero eigenvalues
of YX , we obtain

ev ( CC H ) ev ( R − 1 / 2

xx R xy R − 1 yy R xy R − H / 2

= )

xx

ev ( R − 1

xx R xy R − 1 yy R xy )

= . (4.36)