midterm_1 - (a) Show X ( Y,Z ) W forms a Markov chain. (b)...

Info iconThis preview shows pages 1–2. Sign up to view the full content.

View Full Document Right Arrow Icon
EE 376A/Stat 376A Information Theory Prof. T. Weissman Monday, February 5, 2007 Sample Midterm 1. ( 25 points ) True or False? If the inequality is true, prove it, otherwise, give a counterexample: (a) H ( X,Y | Z ) H ( X | Z ) (b) H ( X | Z ) H ( Z ) (c) H ( X,Y,Z ) - H ( X,Y ) H ( X,Z ) - H ( X ) (d) H ( X | Z ) H ( X ) - H ( Z ) 2. ( 25 points ) Variable length coding (a) Suppose X p , where p = ± 10 34 , 8 34 , 7 34 , 5 34 , 3 34 , 1 34 ² . Find the binary Huffman code for X . 1
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
3. (20 points) AEP Let X i be iid p ( x ), x ∈ { 1 , 2 , ··· ,m } . Let μ = EX , and H = - p ( x ) log p ( x ). Let A n = { x n ∈ X n : | - 1 n log p ( x n ) - H | ≤ ± } . Let B n = { x n X n : | 1 n n i =1 X i - μ | ≤ ± } . (a) Does Pr { X n A n } → 1 ? (b) Does Pr { X n A n B n } → 1 ? (c) Show | A n B n | ≤ 2 n ( H + ± ) , for all n . (d) Show | A n B n | ≥ 1 2 2 n ( H - ± ) , for n sufficiently large. 4. ( 25 points ) Markov chain. Suppose X Y ( Z,W ) forms a Markov chain.
Background image of page 2
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: (a) Show X ( Y,Z ) W forms a Markov chain. (b) Find I ( X ; W,Y ). 5. ( 30 points ) Two looks. Compare the mutual information I ( X ; Y 1 ,Y 2 ) that ( Y 1 ,Y 2 ) provide about X to the sum of the mutual information I ( X ; Y 1 )+ I ( X ; Y 2 ) for each of the following two probability mass functions. (a) ( 15 points ) Two independent looks: p ( x,y 1 ,y 2 ) = p ( x ) p ( y 1 | x ) p ( y 2 | x ) . (b) ( 15 points ) One look at two independents:: p ( x,y 1 ,y 2 ) = p ( y 1 ) p ( y 2 ) p ( x | y 1 ,y 2 ) 2...
View Full Document

This note was uploaded on 12/01/2010 for the course ADLAC 1023 at Stanford.

Page1 / 2

midterm_1 - (a) Show X ( Y,Z ) W forms a Markov chain. (b)...

This preview shows document pages 1 - 2. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online