Digital Signal Processing Reference
In-Depth Information
Fig. 6.1
A family of divergences built from a convexity gap
Sym
+
→ R
+
We build a
family
of divergences from a convex generator
F
:
as
follows:
J
(α,β)
F
(
P
,
Q
)
=
(
F
(
P
)
F
(
Q
))
β
−
F
((
PQ
)
α
)
≥
0
,
(6.11)
=
< α, β <
with equality holds when
P
Q
,for0
1. The divergence is guaranteed
non-negative only for
. Figure
6.1
depicts the divergence as a line segment
lying inside the convexity gap induced by
F
. Common convex matrix generators are
α
=
β
X
T
X
F
(
X
)
=
tr
(
)(
the quadratic matrix entropy
),
(6.12)
F
(
X
)
=−
log det
X
(
the matrix Burg entropy
),
(6.13)
F
(
X
)
=
tr
(
X
log
X
−
X
)(
the von Neumann entropy
).
(6.14)
1
In particular, the Burbea-Rao divergence [
15
] is obtained by choosing
α
=
β
=
2
:
F
P
F
(
P
)
+
F
(
Q
)
+
Q
BR
F
(
P
,
Q
)
=
−
≥
0
.
(6.15)
2
2
Choosing
F
, we get the
Jensen-von Neumann
divergence,
the matrix counterpart of the celebrated Jensen-Shannon divergence. An interest-
ing property is that asymptotic skew Jensen divergences are equivalent to Bregman
divergences:
(
X
)
=
tr
(
X
log
X
−
X
)
1
α
J
(α,α)
F
B
F
(
,
)
=
(
,
),
P
Q
lim
α
→
0
P
Q
(6.16)
