hw7sol

# hw7sol - EE 376B/Stat 376B Information Theory Prof T Cover...

This preview shows pages 1–3. Sign up to view the full content.

EE 376B/Stat 376B Handout #19 Information Theory Tuesday, May 30, 2006 Prof. T. Cover Solutions to Homework Set #7 1. Images. Consider an n × n array x of 0’s and 1’s . Thus x has n 2 bits. ( a ) ( b ) ( c ) Find the Kolmogorov complexity K ( x | n ) (to ﬁrst order) if (a) x is a horizontal line. (b) x isasquare . (c) x is the union of two lines, each line being vertical or horizontal. (d) x is a random array. (e) x is a rectangle lined up with the axes. (f) x is the union of two such rectangles meeting in a corner. Solution: Images. (a) The program to print out an image of one horizontal line is of the form For 1 i n { Set pixels on row i to 0; } Set pixels on row r to 1; Print out image. Since the computer already knows n , the length of this program is K ( r | n )+ c , which is log n + c . Hence, the Kolmogorov complexity of a line image is K (line | n ) log n + c. (1)

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
(b) For a square, we have to tell the program the coordinates of the top left corner, and the length of the side of the square. This requires no more than 3 log n bits, and hence K (square | n ) 3log n + c. However, we can save some description length by ﬁrst describing the length of the side of the square and then the coordinates. Knowing the length of the side of the square reduces the range of possible values of the coordinates. Even better, we can count the total number of such squares. There is one n × n square, four ( n - 1) × ( n - 1) squares, nine ( n - 2) × ( n - 2) squares, etc. The total number of squares is 1 2 +2 2 +3 2 + ··· + n 2 = n ( n + 1)(2 n +1) 6 n 3 3 . Since we can give the index of a square in a lexicographic ordering,
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

### Page1 / 5

hw7sol - EE 376B/Stat 376B Information Theory Prof T Cover...

This preview shows document pages 1 - 3. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online