sol3 - ECE 255AN Fall 2011 Homework set 3 - solutions 1...

Info iconThis preview shows pages 1–2. Sign up to view the full content.

View Full Document Right Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: ECE 255AN Fall 2011 Homework set 3 - solutions 1 Entropy of P is 7 / 4. Hence T 4 . 25 contains all sequences of length four with probabilities between 2- 6 and 2- 8 . All sequences except five sequences satisfy this property. The five sequences are (1 / 2 , 1 / 2 , 1 / 2 , 1 / 2) and (1 / 2 , 1 / 2 , 1 / 2 , 1 / 4) and their permutations. Hence | T 4 . 25 | = 251 and Pr ( T 4 . 25 ) = 13 / 16. 2 a) T n inf is the set of all possible sequences. T n is the set of all sequences with probability 2- nH ( X ) . Two of them are equal, when all sequences have same probability. This is possible if and only if the distribution is uniform. b) T n 1 is the set of all sequences with probability less than 2 n- nH ( P ) and greater than 2- n- nH ( P ) . Let p min be the lowest probability of an alphabet and p max be that of the highest. Highest probability of a sequence is p n max and the lowest probability is p n min . Therefore for the given condition, it is necessary that p n max ≤ 2 1- nH ( P ) and p n min ≥ 2- n- nH ( P ) . Rearranging, p max ≤ 2 1- H ( P ) p min ≥ 2- 1- H ( P ) 3 a) We prove by induction that Pr ( X i = 1) = 1 2 for all i . Clearly it is true for i = 1 , 2. Assume its true for n ≥ 2. For n + 1, Pr ( X n +1 ) = 1) = 1 n n X i =1 Pr ( X i ) = 1) = 1 n n 2 = 1 2 Hence by induction, Pr ( X i = 1) = 1 2 for all i , therefore H ( X i ) = 1. b) lim n →∞ H ( X n 1 ) n = lim n →∞ H ( X n | X n- 1 1 ) ≤ H ( X n | X 2 1 ), as conditioning increases the en- tropy....
View Full Document

This document was uploaded on 01/19/2012.

Page1 / 4

sol3 - ECE 255AN Fall 2011 Homework set 3 - solutions 1...

This preview shows document pages 1 - 2. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online