# hw3_soln_fall2011 - ECE 563 Fall 2011 Homework 3 Solution...

This preview shows pages 1–3. Sign up to view the full content.

—————————————————————————————————————————————————— 3.13, Cover and Thomas Calculation of Typical Set To clarify the notion of typical set A ( n ) ± , and the smallest set of high probability B ( n ) δ , we will calculate the set for a simple example. consider a sequence of i.i.d. binary random variables, X 1 ,X 2 , ··· n , where the probability that X i = 1 is .6 (and therefore the probability that X i = 0 is .4). a. Calculate H ( X ). Solution: H ( X )= ± x ∈{ 0 , 1 } p ( x ) · log p ( x . 6 · log . 6+ . 4 · log . 4= . 9710 b. With n =25and ± = . 1, which sequences fall in the typical set A ( n ) ± ? Solution: Note that in general: A ( n ) ± = ² ( x 1 ,x 2 , n ) X H ( X ) ± ≤− 1 n log p ( x 1 2 , n ) H ( X )+ ± ³ For this problem: A ( n ) ± = ² x n ∈{ 0 , 1 } n H ( X ) ± 1 n log p ( x n ) H ( X ± ³ = ² x 25 0 , 1 } 25 . 9710 . 1 1 25 log p ( x 25 ) . 9710 + . 1 ³ = ² x 25 0 , 1 } 25 . 8710 1 25 log ( ( . 6) k · ( . 4) 25 k ) 1 . 0710 ³ = { the sequences where k , the number of ones = 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 , 19 } What is the probability of the typical set? Solution: In Matlab: sum(binopdf(11:19,25,.6))=.9362 How many elements are their in the typical set? Solution: Elements = ´ 25 11 µ + ´ 25 12 µ + ´ 25 13 µ + ´ 25 14 µ + ´ 25 15 µ + ´ 25 16 µ + ´ 25 17 µ + ´ 25 18 µ + ´ 25 19 µ =2 6 , 366 , 510 c. How many elements are there in the smallest set that has probability . 9? Solution: This will be done in two ways: First we compute this directly by adding the number of sequences with k =12 , 13 , 25 including just enough of the sequences with 12 ones to equal . 9 probability. We start from the bottom since . 6 >. 4 Hence, in MATLAB, we have [ . 9 sum ( binopdf (13 : 25 , 25 ,. 6))] ·¸ ¹ probability mass lef t in k =12 / ( . 6 (12) . 4 (13) ) ·¸ ¹ probability mass of one element + sum ( NchooseK (25 , 13 : 25)) ·¸ ¹ total elements in k =13 , 14 , ··· 25 . 0458 e +007 or we can use the formula | B ( n ) δ | = | B (25) . 1 | · nH 25 . 9710 . 0283 e + 007 Question 1: ECE 563, Fall 2011 Homework 3 Solution

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
d. How many elements are there in the intersection of the sets in (b) and (c)?
This is the end of the preview. Sign up to access the rest of the document.

## This note was uploaded on 10/24/2011 for the course ELECTRICAL ECE 571 taught by Professor Kelly during the Spring '11 term at University of Illinois, Urbana Champaign.

### Page1 / 8

hw3_soln_fall2011 - ECE 563 Fall 2011 Homework 3 Solution...

This preview shows document pages 1 - 3. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online