{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

# 7.20 - EE 376A Prof T Weissman Information Theory Homework...

This preview shows pages 1–3. Sign up to view the full content.

EE 376A Information Theory Prof. T. Weissman Feb 24, 2010 Homework Set #5 Solution 1. The Z channel. The Z-channel has binary input and output alphabets and transition probabilities p ( y | x ) given by the following matrix: Q = ± 1 0 1 / 2 1 / 2 ² x, y ∈ { 0 , 1 } Find the capacity of the Z-channel and the maximizing input probability distribution. Solutions: The Z channel. First we express I ( X ; Y ), the mutual information between the input and output of the Z-channel, as a function of x = Pr( X = 1): H ( Y | X ) = Pr( X = 0) · 0 + Pr( X = 1) · 1 = x H ( Y ) = H 2 (Pr( Y = 1)) = H 2 ( x/ 2) I ( X ; Y ) = H ( Y ) - H ( Y | X ) = H 2 ( x/ 2) - x Since I ( X ; Y ) = 0 when x = 0 and x = 1, the maximum mutual information is obtained for some value of x such that 0 < x < 1. Using elementary calculus, we determine that d dx I ( X ; Y ) = 1 2 log 2 1 - x/ 2 x/ 2 - 1 , which is equal to zero for x = 2 / 5. (It is reasonable that Pr( X = 1) < 1 / 2 because X = 1 is the noisy input to the channel.) So the capacity of the Z-channel in bits is H (1 / 5) - 2 / 5 = 0 . 722 - 0 . 4 = 0 . 322. 2. Channel capacity: Calculate the capacity of the following channels with probability transition matrices: (a) X = Y = { 0 , 1 , 2 } p ( y | x ) = 1 / 3 1 / 3 1 / 3 1 / 3 1 / 3 1 / 3 1 / 3 1 / 3 1 / 3 (1) (b) X = Y = { 0 , 1 , 2 } p ( y | x ) = 1 / 2 1 / 2 0 0 1 / 2 1 / 2 1 / 2 0 1 / 2 (2) 1

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
(c) X = Y = { 0 , 1 , 2 , 3 } p ( y | x ) = p 1 - p 0 0 1 - p p 0 0 0 0 q 1 - q 0 0 1 - q q (3) Channel Capacity:
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

### Page1 / 6

7.20 - EE 376A Prof T Weissman Information Theory Homework...

This preview shows document pages 1 - 3. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online