7.20 - EE 376A Prof. T. Weissman Information Theory Feb 24,...

Info iconThis preview shows pages 1–3. Sign up to view the full content.

View Full Document Right Arrow Icon
EE 376A Information Theory Prof. T. Weissman Feb 24, 2010 Homework Set #5 Solution 1. The Z channel. The Z-channel has binary input and output alphabets and transition probabilities p ( y | x ) given by the following matrix: Q = ± 1 0 1 / 2 1 / 2 ² x, y ∈ { 0 , 1 } Find the capacity of the Z-channel and the maximizing input probability distribution. Solutions: The Z channel. First we express I ( X ; Y ), the mutual information between the input and output of the Z-channel, as a function of x = Pr( X = 1): H ( Y | X ) = Pr( X = 0) · 0 + Pr( X = 1) · 1 = x H ( Y ) = H 2 (Pr( Y = 1)) = H 2 ( x/ 2) I ( X ; Y ) = H ( Y ) - H ( Y | X ) = H 2 ( x/ 2) - x Since I ( X ; Y ) = 0 when x = 0 and x = 1, the maximum mutual information is obtained for some value of x such that 0 < x < 1. Using elementary calculus, we determine that d dx I ( X ; Y ) = 1 2 log 2 1 - x/ 2 x/ 2 - 1 , which is equal to zero for x = 2 / 5. (It is reasonable that Pr( X = 1) < 1 / 2 because X = 1 is the noisy input to the channel.) So the capacity of the Z-channel in bits is H (1 / 5) - 2 / 5 = 0 . 722 - 0 . 4 = 0 . 322. 2. Channel capacity: Calculate the capacity of the following channels with probability transition matrices: (a) X = Y = { 0 , 1 , 2 } p ( y | x ) = 1 / 3 1 / 3 1 / 3 1 / 3 1 / 3 1 / 3 1 / 3 1 / 3 1 / 3 (1) (b) X = Y = { 0 , 1 , 2 } p ( y | x ) = 1 / 2 1 / 2 0 0 1 / 2 1 / 2 1 / 2 0 1 / 2 (2) 1
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
(c) X = Y = { 0 , 1 , 2 , 3 } p ( y | x ) = p 1 - p 0 0 1 - p p 0 0 0 0 q 1 - q 0 0 1 - q q (3) Channel Capacity:
Background image of page 2
Image of page 3
This is the end of the preview. Sign up to access the rest of the document.

This note was uploaded on 10/27/2010 for the course ECE 221 taught by Professor Sd during the Spring '10 term at Huston-Tillotson.

Page1 / 6

7.20 - EE 376A Prof. T. Weissman Information Theory Feb 24,...

This preview shows document pages 1 - 3. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online