Databases Reference
In-Depth Information
F I GU R E 10 . 22
Contours of constant probability.
10.6.1 Pyramid Vector Quantization
As the dimension of the input vector increases, something interesting happens. Suppose we
are quantizing a random variable
X
with
pdf f
X
(
. Suppose we
block samples of this random variable into a random vector
X
. A result of Shannon's, called
the
asymptotic equipartition property
(AEP), states that for sufficiently large
L
and arbitrarily
small
X
)
and differential entropy
h
(
X
)
<
log
f
X
(
X
)
+
h
(
X
)
(6)
L
for all but a set of vectors with a vanishingly small probability [
3
]. This means that almost all
the
L
-dimensional vectors will lie on a contour of constant probability given by
=−
log
f
X
(
X
)
h
(
X
)
(7)
L
Given that this is the case, Sakrison [
143
] suggested that an optimum manner to encode the
sourcewould be to distribute 2
RL
points uniformly in this region. Fischer [
144
] used this insight
