Unformatted text preview: be further, see Problems 28, 29, 30, 31, and
S. W. Ho and R. W. Yeung, “On the discontinuity of the Shannon information measures,” IEEE Transactions of Information
Theory, IT56, no. 12, pp. 53625374, 2009. Thursday, 26 December, 13 Finite Alphabet vs Countable
Alphabet
• All Shannon’s information measures are continuous when the alphabets
are ﬁxed and ﬁnite.
• For countable alphabets, Shannon’s information measures are everywhere
discontinuous.
• To probe further, see Problems 28, 29, 30, 31, and
S. W. Ho and R. W. Yeung, “On the discontinuity of the Shannon information measures,” IEEE Transactions of Information
Theory, IT56, no. 12, pp. 53625374, 2009. Thursday, 26 December, 13 Deﬁnition 2.23 Let p and q be two probability distributions on a common
alphabet...
View
Full Document
 Spring '14
 RaymondYeumg
 Information Theory, Mathematical analysis, Continuous function, Px, Binary logarithm, lim h

Click to edit the document details