{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

midterm_1

# midterm_1 - (a Show X → Y,Z → W forms a Markov chain(b...

This preview shows pages 1–2. Sign up to view the full content.

EE 376A/Stat 376A Information Theory Prof. T. Weissman Monday, February 5, 2007 Sample Midterm 1. ( 25 points ) True or False? If the inequality is true, prove it, otherwise, give a counterexample: (a) H ( X, Y | Z ) H ( X | Z ) (b) H ( X | Z ) H ( Z ) (c) H ( X, Y, Z ) - H ( X, Y ) H ( X, Z ) - H ( X ) (d) H ( X | Z ) H ( X ) - H ( Z ) 2. ( 25 points ) Variable length coding (a) Suppose X p , where p = 10 34 , 8 34 , 7 34 , 5 34 , 3 34 , 1 34 . Find the binary Huffman code for X . 1

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
3. (20 points) AEP Let X i be iid p ( x ), x ∈ { 1 , 2 , · · · , m } . Let μ = EX , and H = - p ( x ) log p ( x ). Let A n = { x n ∈ X n : | - 1 n log p ( x n ) - H | ≤ } . Let B n = { x n X n : | 1 n n i =1 X i - μ | ≤ } . (a) Does Pr { X n A n } → 1 ? (b) Does Pr { X n A n B n } → 1 ? (c) Show | A n B n | ≤ 2 n ( H + ) , for all n . (d) Show | A n B n | ≥ 1 2 2 n ( H - ) , for n sufficiently large. 4. ( 25 points ) Markov chain. Suppose X Y ( Z, W ) forms a Markov chain.
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: (a) Show X → ( Y,Z ) → W forms a Markov chain. (b) Find I ( X ; W,Y ). 5. ( 30 points ) Two looks. Compare the mutual information I ( X ; Y 1 ,Y 2 ) that ( Y 1 ,Y 2 ) provide about X to the sum of the mutual information I ( X ; Y 1 )+ I ( X ; Y 2 ) for each of the following two probability mass functions. (a) ( 15 points ) Two independent looks: p ( x,y 1 ,y 2 ) = p ( x ) p ( y 1 | x ) p ( y 2 | x ) . (b) ( 15 points ) One look at two independents:: p ( x,y 1 ,y 2 ) = p ( y 1 ) p ( y 2 ) p ( x | y 1 ,y 2 ) 2...
View Full Document

{[ snackBarMessage ]}

### Page1 / 2

midterm_1 - (a Show X → Y,Z → W forms a Markov chain(b...

This preview shows document pages 1 - 2. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online