This preview shows pages 1–2. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: ECE 255AN Fall 2011 Homework set 3  solutions 1 Entropy of P is 7 / 4. Hence T 4 . 25 contains all sequences of length four with probabilities between 2 6 and 2 8 . All sequences except five sequences satisfy this property. The five sequences are (1 / 2 , 1 / 2 , 1 / 2 , 1 / 2) and (1 / 2 , 1 / 2 , 1 / 2 , 1 / 4) and their permutations. Hence  T 4 . 25  = 251 and Pr ( T 4 . 25 ) = 13 / 16. 2 a) T n inf is the set of all possible sequences. T n is the set of all sequences with probability 2 nH ( X ) . Two of them are equal, when all sequences have same probability. This is possible if and only if the distribution is uniform. b) T n 1 is the set of all sequences with probability less than 2 n nH ( P ) and greater than 2 n nH ( P ) . Let p min be the lowest probability of an alphabet and p max be that of the highest. Highest probability of a sequence is p n max and the lowest probability is p n min . Therefore for the given condition, it is necessary that p n max ≤ 2 1 nH ( P ) and p n min ≥ 2 n nH ( P ) . Rearranging, p max ≤ 2 1 H ( P ) p min ≥ 2 1 H ( P ) 3 a) We prove by induction that Pr ( X i = 1) = 1 2 for all i . Clearly it is true for i = 1 , 2. Assume its true for n ≥ 2. For n + 1, Pr ( X n +1 ) = 1) = 1 n n X i =1 Pr ( X i ) = 1) = 1 n n 2 = 1 2 Hence by induction, Pr ( X i = 1) = 1 2 for all i , therefore H ( X i ) = 1. b) lim n →∞ H ( X n 1 ) n = lim n →∞ H ( X n  X n 1 1 ) ≤ H ( X n  X 2 1 ), as conditioning increases the en tropy....
View
Full
Document
This document was uploaded on 01/19/2012.
 Fall '09

Click to edit the document details