*This preview shows
pages
1–3. Sign up
to
view the full content.*

This
** preview**
has intentionally

**sections.**

*blurred***to view the full version.**

*Sign up*
**Unformatted text preview: **Harvard SEAS ES250 Information Theory Homework 4 Solutions 1. Find the channel capacity of the following discrete memoryless channel: Z X Y where Pr { Z = 0 } = Pr { Z = a } = 1 2 . The alphabet for x is X = { , 1 } . Assume that Z is independent of X . Observe that the channel capacity depends on the value of a . Solution : Y = X + Z X { , 1 } , Z { ,a } We have to distinguish various cases depending on the values of a . a = 0. In this case, Y = X , and max I ( X ; Y ) = max H ( X ) = 1. Hence the capacity is 1 bit per transmission. a negationslash = 0 , 1. In this case, Y has four possible values 0, 1, a , and 1 + a . Knowing Y , we know the X which was sent, and hence H ( X | Y ) = 0. Hence max I ( X ; Y ) = max H ( X ) = 1, achieved for an uniform distribution on the input X . a = 1. In this case, Y has three possible output values, 0, 1, and 2. The channel is identical to the binary erasure channel with a = 1 / 2. The capacity of this channel is 1 / 2 bit per transmission. a = 1. This is similar to the case when a = 1 and the capacity here is also 1 / 2 bit per transmission. 2. Consider a 26-key typewriter. (a) If pushing a key results in printing the associated letter, what is the capacity C in bits? (b) Now suppose that pushing a key results in printing that letter or the next (with equal proba- bility). Thus A A or B , , Z Z or A . What is the capacity? (c) What is the highest rate code with block length one that you can find that achieves zero probability of error for the channel in part (b). Solution : (a) If the typewriter prints out whatever key is struck, then the output Y , is the same as the input X , and C = max I ( X ; Y ) = max H ( X ) = log 26 , attained by a uniform distribution over the letters. 1 Harvard SEAS ES250 Information Theory (b) In this case, the output is either equal to the input (with probability 1 2 ) or equal to the next letter (with probability 1 2 ). Hence H ( Y | X ) = log 2 independent of the distribution of X , and hence C = max I ( X ; Y ) = max H ( Y ) log 2 = log 26 log 2 = log 13 attained for a uniform distribution over the output, which in turn is attained by a uniform distribution on the input. (c) A simple zero error block length one code is the one that uses every alternate letter, say A,C,E, ,W,Y. In this case, none of the codewords will be confused, since A will produce either A or B, C will produce C or D, etc. The rate of this code, R = log( codewords) Block length = log 13 1 = log 13 In this case, we can achieve capacity with a simple code with zero error. 3. Consider a binary symmetric channel with Y i = X i Z i , where is mod 2 addition, and X i ,Y i { , 1 } ....

View
Full
Document