Databases Reference
In-Depth Information
But this is simply the probability that
X
⊕
Y
=
1. Therefore, the maximum value that
1
P
(
X
⊕
Y
=
1
)
can have is
D
. Our assumptions were that
D
<
p
and
p
2
, which means
1
is closest to
2
while being less than or equal to
D
that
D
<
2
. Therefore,
P
(
X
⊕
Y
=
1
)
when
P
(
X
⊕
Y
=
1
)
=
D
. Therefore,
I
(
X
;
Y
)
H
b
(
p
)
−
H
b
(
D
)
(65)
We can show that for
P
(
X
=
0
|
Y
=
1
)
=
P
(
X
=
1
|
Y
=
0
)
=
D
, this bound is achieved.
That is, if
P
(
X
=
0
|
Y
=
1
)
=
P
(
X
=
1
|
Y
=
0
)
=
D
, then
I
(
X
;
Y
)
=
H
b
(
p
)
−
H
b
(
D
)
(66)
1
Therefore, for
D
<
p
and
p
2
,
R
(
D
)
=
H
b
(
p
)
−
H
b
(
D
)
(67)
1
Finally, if
p
>
2
, then we simply switch the roles of
p
and 1
−
p
. Putting all this together,
the rate distortion function for a binary source is
H
b
(
p
)
−
H
b
(
D
)
for
D
<
min
{
p
,
1
−
p
}
R
(
D
)
=
(68)
0
otherwise
Examp l e 8 . 5 . 4 : Rate Distortion Function for the Gaussian
Source
Suppose we have a continuous amplitude source that has a zero mean Gaussian
pdf
with
variance
2
. If our distortion measure is given by
σ
2
d
(
x
,
y
)
=
(
x
−
y
)
(69)
our distortion constraint is given by
E
2
(
X
−
Y
)
D
(70)
Our approach to finding the rate distortion function will be the same as in the previous
example; that is, find a lower bound for
I
(
X
;
Y
)
given a distortion constraint, and then show
that this lower bound can be achieved.
First we find the rate distortion function for
D
2
:
<σ
I
(
X
;
Y
)
=
h
(
X
)
−
h
(
X
|
Y
)
(71)
=
h
(
X
)
−
h
(
X
−
Y
|
Y
)
(72)
h
(
X
)
−
h
(
X
−
Y
)
(73)
In order to minimize the right-hand side of Equation (
73
), we have to maximize the second
term subject to the constraint given by Equation (
70
). This term is maximized if
X
−
Y
is
