Databases Reference
In-Depth Information
Because the logarithm base 2 probably does not appear on your calculator, let's briefly
review logarithms. Recall that
log
b
x
=
a
means that
b
a
=
x
Therefore, if we want to take the log base 2 of
x
2
a
log
2
x
=
a
⇒
=
x
we want to find the value of
a
. We can take the natural log (log base
e
), which we will write
as
ln
, or log base 10 of both sides (which do appear on your calculator). Then
2
a
ln
(
)
=
ln
x
⇒
a
ln 2
=
ln
x
and
ln
x
ln 2
=
a
Example2.2.1:
Let
H
and
T
be the outcomes of flipping a coin. If the coin is fair, then
1
2
P
(
H
)
=
P
(
T
)
=
and
1bit
If the coin is not fair, then we would expect the information associated with each event to be
different. Suppose
i
(
H
)
=
i
(
T
)
=
1
8
,
7
8
P
(
H
)
=
P
(
T
)
=
Then
0.193 bits
At least mathematically, the occurrence of a head conveys much more information than the
occurrence of a tail. As we shall see later, this has certain consequences for how the information
conveyed by these outcomes should be encoded.
i
(
H
)
=
3 bits
,
i
(
T
)
=
If we have a set of independent events
A
i
, which are sets of outcomes of some experiment
S
, such that
A
i
=
S
where
S
is the sample space, then the average self-information associated with the random
experiment is given by
P
P
H
=
(
A
i
)
i
(
A
i
)
=−
(
A
i
)
log
b
P
(
A
i
)
