hw7sol - EE 376B/Stat 376B Information Theory Prof. T....

Info iconThis preview shows pages 1–3. Sign up to view the full content.

View Full Document Right Arrow Icon
EE 376B/Stat 376B Handout #19 Information Theory Tuesday, May 30, 2006 Prof. T. Cover Solutions to Homework Set #7 1. Images. Consider an n × n array x of 0’s and 1’s . Thus x has n 2 bits. ( a ) ( b ) ( c ) Find the Kolmogorov complexity K ( x | n ) (to first order) if (a) x is a horizontal line. (b) x isasquare . (c) x is the union of two lines, each line being vertical or horizontal. (d) x is a random array. (e) x is a rectangle lined up with the axes. (f) x is the union of two such rectangles meeting in a corner. Solution: Images. (a) The program to print out an image of one horizontal line is of the form For 1 i n { Set pixels on row i to 0; } Set pixels on row r to 1; Print out image. Since the computer already knows n , the length of this program is K ( r | n )+ c , which is log n + c . Hence, the Kolmogorov complexity of a line image is K (line | n ) log n + c. (1)
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
(b) For a square, we have to tell the program the coordinates of the top left corner, and the length of the side of the square. This requires no more than 3 log n bits, and hence K (square | n ) 3log n + c. However, we can save some description length by first describing the length of the side of the square and then the coordinates. Knowing the length of the side of the square reduces the range of possible values of the coordinates. Even better, we can count the total number of such squares. There is one n × n square, four ( n - 1) × ( n - 1) squares, nine ( n - 2) × ( n - 2) squares, etc. The total number of squares is 1 2 +2 2 +3 2 + ··· + n 2 = n ( n + 1)(2 n +1) 6 n 3 3 . Since we can give the index of a square in a lexicographic ordering,
Background image of page 2
Image of page 3
This is the end of the preview. Sign up to access the rest of the document.

Page1 / 5

hw7sol - EE 376B/Stat 376B Information Theory Prof. T....

This preview shows document pages 1 - 3. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online