# hw7 - (c 0 n 1 followed by any arbitrary...

This preview shows pages 1–2. Sign up to view the full content.

EE 376B/Stat 376B Handout #16 Information Theory Thursday, May 18, 2006 Prof. T. Cover Due Thursday, May 25, 2006 Homework Set #7 1. Images. Consider an n × n array x of 0’s and 1’s . Thus x has n 2 bits. ( a ) ( b ) ( c ) Find the Kolmogorov complexity K ( x | n ) (to first order) if (a) x is a horizontal line. (b) x is a square. (c) x is the union of two lines, each line being vertical or horizontal. (d) x is a random array. (e) x is a rectangle lined up with the axes. (f) x is the union of two such rectangles meeting in a corner. 2. Complexity of the sum. Let n be an integer. (a) Argue that K ( n ) log n + 2 log log n + c. (b) Argue that K ( n 1 + n 2 ) K ( n 1 ) + K ( n 2 ) + c. (c) Give an example in which n 1 and n 2 are complex but the sum is relatively simple. 1

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
3. Monkeys on a computer. Suppose a random program is typed into a computer. Give a rough estimate of the probability that the computer prints the following sequence: (a) 0 n followed by any arbitrary sequence. (b) π 1 π 2 . . . π n followed by any arbitrary sequence, where π i is the ith bit in the expansion of
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: (c) 0 n 1 followed by any arbitrary sequence. (d) (Optional) ω 1 ω 2 ...ω n followed by any arbitrary sequence, where Ω = ∑ U ( p ) halts 2-l ( p ) , Ω = .ω 1 ω 2 ..., where Ω is the probability that the com-puter halts. 4. Image Complexity. Consider two binary subsets A and B (of an n × n grid). For example, ¶ ± ‡ · ¤ § ¥ ƒ Find general upper and lower bounds, in terms of K ( A | n ) and K ( B | n ), for (a) K ( A c | n ) . (b) K ( A ∪ B | n ) . (c) K ( A ∩ B | n ) . 5. Random program Suppose that a random program (symbols i.i.d. uniform over the symbol set) is fed into the nearest available computer. To our surprise the ﬁrst n bits of the binary expansion of 1 / √ 2 are printed out. Roughly what would you say the probability is that the next output bit will agree with the corresponding bit in the expansion of 1 / √ 2 ? 2...
View Full Document

{[ snackBarMessage ]}

### What students are saying

• As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

Kiran Temple University Fox School of Business ‘17, Course Hero Intern

• I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

Dana University of Pennsylvania ‘17, Course Hero Intern

• The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

Jill Tulane University ‘16, Course Hero Intern