hw3_soln_fall2011 - ECE 563, Fall 2011 Homework 3 Solution...

Info iconThis preview shows pages 1–3. Sign up to view the full content.

View Full Document Right Arrow Icon
—————————————————————————————————————————————————— 3.13, Cover and Thomas Calculation of Typical Set To clarify the notion of typical set A ( n ) ± , and the smallest set of high probability B ( n ) δ , we will calculate the set for a simple example. consider a sequence of i.i.d. binary random variables, X 1 ,X 2 , ··· n , where the probability that X i = 1 is .6 (and therefore the probability that X i = 0 is .4). a. Calculate H ( X ). Solution: H ( X )= ± x ∈{ 0 , 1 } p ( x ) · log p ( x . 6 · log . 6+ . 4 · log . 4= . 9710 b. With n =25and ± = . 1, which sequences fall in the typical set A ( n ) ± ? Solution: Note that in general: A ( n ) ± = ² ( x 1 ,x 2 , n ) X H ( X ) ± ≤− 1 n log p ( x 1 2 , n ) H ( X )+ ± ³ For this problem: A ( n ) ± = ² x n ∈{ 0 , 1 } n H ( X ) ± 1 n log p ( x n ) H ( X ± ³ = ² x 25 0 , 1 } 25 . 9710 . 1 1 25 log p ( x 25 ) . 9710 + . 1 ³ = ² x 25 0 , 1 } 25 . 8710 1 25 log ( ( . 6) k · ( . 4) 25 k ) 1 . 0710 ³ = { the sequences where k , the number of ones = 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 , 19 } What is the probability of the typical set? Solution: In Matlab: sum(binopdf(11:19,25,.6))=.9362 How many elements are their in the typical set? Solution: Elements = ´ 25 11 µ + ´ 25 12 µ + ´ 25 13 µ + ´ 25 14 µ + ´ 25 15 µ + ´ 25 16 µ + ´ 25 17 µ + ´ 25 18 µ + ´ 25 19 µ =2 6 , 366 , 510 c. How many elements are there in the smallest set that has probability . 9? Solution: This will be done in two ways: First we compute this directly by adding the number of sequences with k =12 , 13 , 25 including just enough of the sequences with 12 ones to equal . 9 probability. We start from the bottom since . 6 >. 4 Hence, in MATLAB, we have [ . 9 sum ( binopdf (13 : 25 , 25 ,. 6))] ·¸ ¹ probability mass lef t in k =12 / ( . 6 (12) . 4 (13) ) ·¸ ¹ probability mass of one element + sum ( NchooseK (25 , 13 : 25)) ·¸ ¹ total elements in k =13 , 14 , ··· 25 . 0458 e +007 or we can use the formula | B ( n ) δ | = | B (25) . 1 | · nH 25 . 9710 . 0283 e + 007 Question 1: ECE 563, Fall 2011 Homework 3 Solution
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
d. How many elements are there in the intersection of the sets in (b) and (c)?
Background image of page 2
Image of page 3
This is the end of the preview. Sign up to access the rest of the document.

Page1 / 8

hw3_soln_fall2011 - ECE 563, Fall 2011 Homework 3 Solution...

This preview shows document pages 1 - 3. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online