Cryptography Reference
In-Depth Information
Month
Message
January
00010100100000101001110010101010100000010101001000101101000000000000
February
000011001000101010000100101001001010101010000010101001011001000000001
March
010110101000001010100100100001101001000000000000000000000000000000000010
April
010000101010000010100100100100101001100000000000000000000000000000000011
May
000110101000001010110010000000000000000000000000000000000000000000000000
June
00010100101010101001110010001010000000000000000000000000000000000000001
July
010101001010101010011000101100100000000000000000000000000000000000000010
August
010000101010101010001110101010101010011010101000000000000000000000000011
September
101001101000101010100000101010001000101010011010100001001000101010100100
October
100111100100111101010110010001010100110101000010010001010101001000000001
November
100111001001111010101100100010101001101010000100100010101010010000000010
December
100010001000101010000110100010101001101010000100100010101010010000000011
TABLE A2.2
R
= log
2
26
4.7004397181410921603968126542567.
R
of the entropy is called the absolute rate of a language. For English,
the value above says that each letter contains about 5 bits of information. In truth, the actual
amount of information in each letter is much lower than this.
To find a better estimate of the entropy of a language, we may want to compute the
entropy for messages of size 1, 2, . . . ,
This upper bound
N
, and use some averaging technique to obtain an
estimate.
r
N
= [
E
(
M
1
)/1 +
E
(
M
2
)/2 + . . . +
E
(
M
N
)/
N
]/
N
Here
M
i
represents messages of length
i
. If we use large values of
N
, and if we assume
the entropy converges to some value as
N
approaches infinity, we can get a good estimate
of the entropy of a language.
r
= lim
r
N
(as
N
→
).
We call this value
r
the rate of a language. Many studies have been done to compute
r
for English, and the best estimates obtained so far are around
r
= 1.3.
Search WWH ::
Custom Search