Information Theory - Introduction to Cryptography with Java Applets - page 357

TABLE A2.1

Month Code

January 0000

February 0001

March 0010

April 0011

May 0100

June 0101

July 0110

August 0111

September 1000

October 1001

November 1010

December 1011

Questions
What is the entropy of a message that signifies
1. A day of the week?
2. A day in the month of May?
3. A time of day in hours, minutes, and seconds?
What entropy means to a cryptanalyst is that the analyst needs only to learn at most only
4 bits of a message representing a month to discern the month. For example, consider the
12 messages in Table A2.2, which represent all of the months in a year:
In this case, examining only the first 2 bits and the last 2 bits of one of these messages
will tell you the month. We say that the number of bits required to determine the meaning
of a message is the uncertainty of a message. In general, entropy and uncertainty are equal.
Obviously, for the cryptanalyst, the lower the entropy, the better.

AII.2

RATE OF A LANGUAGE
What is the entropy of any English message? This is what the analyst really wants to know.
If we are using only upper case letters (there are 26 such letters), then certainly the entropy
is no more than