Cryptography Reference
In-Depth Information
More specifically, if
f
is a convex function, then
Jensen's inequality
applies:
E
[
f
(
X
)]
≥
f
(
E
[
X
])
Most of the basic inequalities in information theory follow directly from
Jensen's inequality.
Last but not least, the
conditional expected value
E
[
X
|A
] of a random variable
X
given event
A
can be computed as follows:
]=
x∈X
E
[
X
|A
P
X|A
(
x
)
·
x
4.2.5
Independence of Random Variables
Let
X
and
Y
be two random variables over the same sample space Ω.
X
and
Y
are
independent
if for all
x
,theevents(
X
=
x
) and (
Y
=
y
) are
independent. This is formally expressed in Definition 4.3.
∈X
and
y
∈Y
Definition 4.3 (Independent random variables)
Two random variables
X
and
Y
are
statistically independent
(or
independent
in short) if and only if
P
XY
(
x, y
)=
P
X
(
x
)
·
P
Y
(
y
)
for all
x
∈X
and
y
∈Y
.
Definition 4.3 basically says that the joint probability distribution of two
independent random variables
X
and
Y
is equal to the product of their marginal
distributions.
If two random variables
X
and
Y
are independent, then the conditional
probability distribution
P
X|Y
of
X
given
Y
is
P
X|Y
(
x, y
)=
P
XY
(
x, y
)
P
Y
(
y
)
P
X
(
x
)
P
Y
(
y
)
P
Y
(
y
)
=
=
P
X
(
x
)
for all
x
=0. This basically means that knowing
the value of one random variable does not tell anything about the distribution of the
other (and vice versa). Furthermore, if
X
and
Y
are independent random variables,
then
∈X
and
y
∈Y
with
P
Y
(
y
)
E
[
XY
]=
E
[
X
]
·
E
[
Y
]
.
