This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: ECE 534: Elements of Information Theory, Fall 2010 Homework 6 Solutions October 19, 2010 1. Problem 7.2. Additive noise channel (Matteo Carminati) . Find the channel capacity of the following discrete memoryless channel: Where PrZ = 0 = PrZ = a = 1 2 . The alphabet for x is X = 0 , 1. Assume that Z is indepen dent of X. Observe that the channel capacity depends on the value of a. Solution: We can identify two different values for the capacity of the channel according to the value of a . • First of all, we can note that the outputs of the channel are nonoverlapped if a has a value different from 1 or 1. Thus, in these cases the channel we are analysing is a noisy channel with nonoverlapping outputs. It is known that the capacity of this channel is 1[ bit ] since: C = max I ( X ; Y ) = max( H ( Y ) H ( Y  X )) (mutual information definition) = max( H ( Y )) (from nonoverlapping outputs) = 1 (max of entropy for a binary variable) • In the second case, when a = 1 or a = 1, the outputs of the channel can overlap: if a = 1, Y can be 1 either if X = 0 or X = 1, and if a = 1, Y can be 0 either if X = 0 or X = 1. Let’s compute the channel capacity for a = 1, the same value and a similar expression can be found for a = 1. H ( Y ) = p ( y = 0)log 2 p ( y = 0) p ( y = 1)log 2 p ( y = 1) p ( y = 2)log 2 p ( y = 2) = 2 4 log 2 1 4 1 2 log 2 1 2 = 1 + 1 2 H ( Y  X ) = p ( x = 0) H ( Y  x = 0) p ( x = 1) H ( Y  x = 1) = 1 2 + 1 2 = 1 Thus C = max I ( X ; Y ) = max( H ( Y ) H ( Y  X )) = 1 + 1 2 1 = 1 2 1 2. Problem 7.7. Cascade of binary symmetric channels (Davide Basilio Bartolini) . Show that a cascade of n identical independent binary symmetric channels, X → BSC → X 1 → ... → X n 1 → BSC → X n , each with raw error probability p , is equivalent to a single BSC with error probability 1 2 (1 (1 2 p ) n ) and hence that lim n →∞ I ( X ; X n ) = 0 if p 6 = 0 , 1 . No encoding or decod ing takes place at the intermediate terminals X 1 ,...,X n 1 . Thus, the capacity of the cascade tends to zero. Solution: The conditional probability distribution p ( y  x ) for each of the BSCs may be expressed by the transition probability matrix A , given by: A = 1 p p p 1 p The transition matrix for the cascade is given by A n = A n ; it is possible to exploit the singular...
View
Full Document
 Fall '11
 hoangxinhung

Click to edit the document details