Environmental Engineering Reference
In-Depth Information
Copulas in ICA
x
; each element
x
i
, separately, is sufficient. This
property, in particular, recommends the copulae
family as a fertile point of departure for dependence
models.
We use parametric copulae families as estima-
tors for the dependency in
x
under broad depen-
dence conditions. Specifically, the copula ap-
proach offers a generalized 'engine' for the
contrast functions - measures of statistical depen-
dence - which characterize ICA analysis. This
yields a copula based version of Independent
Component Analysis (CICA) - where we model
and rotate dependency information in
x
via
copulae families.
This version - CICA - replaces non-parametric,
higher order proxies for independence with para-
metric examples from the copula literature. This
parametric modeling appeals:
The copula measure of dependency is defined via
its density, on a multivariate
x
=
( , ...,
x
x
k
)
,as
1
dF
dF x
x
x
( )
( )
dC
x
=
x
(2)
( )
∏
i
i
is the
multivariate copula density
for
x
. Here
dC
(
x
is the full derivative of a distribution func-
tion which takes the marginal distributions
F
, ...,
as its arguments. The copula distribu-
tion, then, is a distribution function on the space
of the marginals to the unit hypercube,
(
F
x
x
k
1
k
I .
The mutual information (see Kullback [1959]),
for a multivariate
X
with distribution function
F
, ...
F
)
X
X
1
k
1. To the duality between information mini-
mization within the component outputs
and likelihood maximization for the rotated
source model and
2. To the partitioning of the full likelihood of
the outputs into model fit and dependence
minimization.
F
(
X
is
dF
dF
X
i
≡
∫
MI
x
dF
x
log
X
(3)
( )
( )
(
)
∏
Ω
where Ωis the probability space for
X
. Using
equation (2) above, this can be re-expressed as
Here, we can construct ICA via copula based
measures of association on partite reductions of
x
- in direct analogy to the PCA via covariance
matrix we can view the ICA procedures as or-
thogonalizations of higher order tensors to capture
non-elliptical dependence. The flexibility of par-
tite reduction allows us to suggest appropriate
copula families for non-gaussian dependence
pathologies - specifically extreme value, non-
monotone and inhomogenous data - within a
multivariate set. This is a consistent framework
for fully parameterized ICA.
∫
I
MI
X
≡
dC
u
log dC
u
=
MI
u
=
E
log dC
u
( )
( )
(
( ))
( )
(
(
( )))
θ
θ
θ
k
(4)
~ ,
W h e n
T
F dF
=
f
t h e n
( ) ( ( ) ( )
E is called the
entropy
for
t
(see Ash 1965) and here,
−
H T
=
f T log T
=
∫
I
MI
X
= −
H
u
dC log dC
u
u
( )
( )
( )
(
( ))
k
(5)
The mutual information then—as the expected
value of the log of the copula density—can be
computed, or estimated, from a parametric copula.
The mutual information then - as the expected
