This preview shows pages 1–3. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: EE 376B/Stat 376B Handout #26 Information Theory Thursday, June 1, 2006 Prof. T. Cover Solutions to Practice Final Each problem is worth 20 points. 1. Kolmogorov complexity n ? 6 n What is the Kolmogorov complexity (to first order) of a square with a rectangle eaten out of a corner? The square and rectangle are on an n n grid with the axes lined up with the borders. Solution: Kolmogorov complexity This question is similar to question 1f of homework set 6. To describe the square, one needs 2log( n ) bits to describe the upper left hand corner, and log( n ) bits to specify the length of an edge. We can then describe the rectangle by first describing the corner of the square it intersects, this takes log(4) bits, and then specify the opposing corner of the rectangle, which takes 2log( n ) bits. K (square with a rectangle eaten out of a corner  n ) 5log( n ) + c. 2. Slepian Wolf Let X 1 = U Z 1 , X 2 = U Z 2 , where U Bern( p ), Z 1 Bern( 1 ), and Z 2 Bern( 2 ), and let U, Z 1 , and Z 2 be independent. What rates ( R 1 ,R 2 ) suffice to describe X 1 and X 2 ? 1 Decoder X n 2 X n 1 j ( X n 2 ) 2 nR 2 i ( X n 1 ) 2 nR 1 X n 1 , X n 2 Solution: Slepian Wolf We know R 1 H ( X 1  X 2 ) R 2 H ( X 2  X 1 ) R 1 + R 2 H ( X 1 ,X 2 ) . H ( X 1 ,X 2 ) = H ( p 1 2 + p 1 2 , p 1 2 + p 1 2 , p 1 2 + p 1 2 , p 1 2 + p 1 2 ) . H ( X 1  X 2 ) = H ( X 1 ,X 2 ) H ( X 2 ) = H ( p 1 2 + p 1 2 , p 1 2 + p 1 2 , p 1 2 + p 1 2 , p 1 2 + p 1 2 ) H B ( 2 * p ) where H B is the binary entropy function and 2 * p = p 2 + p 2 . Similarly, H ( X 2  X 1 ) = H ( X 1 ,X 2 ) H ( X 1 ) = H ( p 1 2 + p 1 2 , p 1 2 + p 1 2 , p 1 2 + p 1 2 , p 1 2 + p 1 2 ) H B ( 1 * p ) . Plugging back into the SlepianWolf region gives R 1 H ( p 1 2 + p 1 2 , p 1 2 + p 1 2 , p 1 2 + p 1 2 , p 1 2 + p 1 2 ) H B ( 2 * p ) R 2 H ( p 1 2 + p 1 2 , p 1 2 + p 1 2 , p 1 2 + p 1 2 , p 1 2 + p 1 2 ) H B ( 1 * p ) R 1 + R 2 H ( p 1 2 + p 1 2 , p 1 2 + p 1 2 , p 1 2 + p 1 2 , p 1 2 + p 1 2 ) ....
View
Full
Document
This note was uploaded on 06/10/2008 for the course ECE 376B taught by Professor Tomcover during the Spring '05 term at Stanford.
 Spring '05
 TomCover

Click to edit the document details