This preview shows pages 1–5. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: Module 9, Lecture 1 Algorithmic Information Theory G.L. Heileman Module 9, Lecture 1 Background Randomness Intuitively, we can tell a random number when we see one. Consider the following binary numbers: 01010101010101010101 11001100111010011011 The first string can be constructed using a simple rule: write one 0 followed by one 1, and repeat. Furthermore, if you were asked to predict the next set of digits for this string, youd have confidence in guessing 01. For the second string, however, there appears to be no rule governing its construction, and therefore youd have a hard time guessing the next set of digits for this string. That is, the second string appears more random. I generated it by flipping a coin 20 times, writing a 1 if the outcome was heads, and a 0 if tails. G.L. Heileman Module 9, Lecture 1 Background Randomness Tossing a coin is the classic way of creating a random number, and it seems that the technique itself should guarantee that the resulting number of random. However, from this perspective, the two strings just presented have the exactly same probability, namely 2 20 . Specifically, tossing a coin 20 times yields each of the 2 20 possible 20digit binary strings with the same probability. Clearly, our intuitive notion of randomness (a patternless number), and the probabilistic notion of randomness are contradictory. A more sensible definition, based on information theory, does not depend on the origin of the number, and only considers the characteristics of the sequence itself. G.L. Heileman Module 9, Lecture 1 Algorithmic Information Theory Assume that you had friend on a boat in the middle of the ocean, and that it is very expensive communication with him. Your friend needs a table of trigonometric function, and has asked him to send them to him using the channel. You have a few choices: 1 Using an appropriate encoding, send the entire book of numbers over the channel. 2 Send instructions for calculating the numbers in the table using basic trionometric, e.g., Eulers equation e j = cos + jsin . Both approaches transmit the same information, but the second is far less expensive. In fact, its a compressed representation of the first approach. Now your friend asks you for all of the most recent closing prices of the stocks on the New York Stock Exchange. It doesnt seem that you can do much better than send the entire set of prices using the best encoding you can come up with. I.e., there doesnt seem to be a formula that we could use to compress this information....
View
Full
Document
This note was uploaded on 05/06/2010 for the course ECE 549 taught by Professor G.l.heileman during the Spring '10 term at University of New Brunswick.
 Spring '10
 G.L.Heileman

Click to edit the document details