Unformatted text preview: ight be stored as one of two 6-byte ASCII strings: “MALE” or “FEMALE.” Formally, the amount of information in a message M is measured by the entropy of a message, denoted by H(M). The entropy of a message indicating sex is 1 bit; the entropy of a message indicating the day of the week is slightly less than 3 bits. In general, the entropy of a message measured in bits is log2 n, in which n is the number of possible meanings. This assumes that each meaning is equally likely. The entropy of a message also measures its uncertainty. This is the number of plaintext bits needed to be recovered when the message is scrambled in ciphertext in order to learn the plaintext. For example, if the ciphertext block “QHP*5M” is either “MALE” or “FEMALE, ” then the uncertainty of the message is 1. A cryptanalyst has to learn only one well-chosen bit to recover the message. Rate of a Language
For a given language, the rate of the language is r = H(M)/N in which N is the length of the message. The rate of normal English takes various values between 1.0 bits/letter and 1.5 bits/letter, for large values of N. Shannon, in , said that the entropy depends on the length of the text. Specifically he indicated a rate of 2.3 bits/letter for 8-letter chunks, but the rate drops to between 1.3 and 1.5 for 16-letter chunks. Thomas Cover used a gambling estimating technique and found an entropy of 1.3 bits/character . (I’ll use 1.3 in this book.) The absolute rate of a language is the maximum number of bits that can be coded in each character, assuming each character sequence is equally likely. If there are L characters in a language, the absolute rate is: R = log2 L This is the maximum entropy of the individual characters. For English, with 26 letters, the absolute rate is log2 26, or about 4.7 bits/letter. It should come as no surprise to anyone that the actual rate of English is much less than the absolute rate; natural language is highly redundant. The redundancy of a language, denoted D, is defined by: D=R-r Given that the rate of English is 1.3, the r...
View Full Document
- Fall '10
- Cryptography, Bruce Schneier, Applied Cryptography, EarthWeb, Search Search Tips