{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

# 13-mdl - Algorithmic Information Theory Consider the...

This preview shows pages 1–5. Sign up to view the full content.

Algorithmic Information Theory Consider the following Strings: 0101010101010101010101010101010101010101010101010101010101010101 1100100001100001110111101110110011111010010000100101011110010110 How much information is in each of the strings? Intuitively, very little in the first one because we can describe/compress the string as 32 01 . It seems that we can only describe the second string by its own representation, that is, the only way to describe it is by repeating its sequence of zeros and ones. Therefore, the second string seems to contain more information. We define the quantity of information contained in an object to be the size of that object’s smallest representation or description. So, | 32 01 | < | 1100100001100001110111101110110011111010010000100101011110010110 | – p. 1/1

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Minimal Description Lengths Observation: We can use algorithms to describe the structure of strings. We say that M, w is a description of some string x if running the ma- chine M with w as input results in string x , formally x = M, w Note: If we assume that we encode all our descriptions and strings as binary sequences and if w is a binary sequence, then M, w = M w Example: A trivial description. Consider the machine I , I = "On input w , where w is a binary string: 1. accept ." That is, x = I, x = I x . – p. 2/1
Minimal Description Lengths Definition: Let x be a binary string. The minimal length description of x , written d ( x ) , is the shortest string M, w where TM M on input w halts with x on its tape. The Kolmogorov complexity of x , K ( x ) , is K ( x ) = | d ( x ) | . Example: Let, x = 0101010101010101010101010101010101010101010101010101010101010101 , then consider the machine M 1 , M 1 = "On input w , where w is a binary string: 1. Write the string w 31 times on the tape right after the input string. 1. accept ." Now we have, x = M 1 01 . If we assume that | M 1 | < | x | − 2 then d ( x ) = M 1 01 and therefore, K ( x ) = | d ( x ) | = | M 1 01 | – p. 3/1

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Minimal Description Lengths The next theorem states that repeating a string does not significantly increase the information content of the overall string. Theorem: x q [ K ( xx ) K ( x ) + q ] . Proof: Let d ( x ) = N, w , that is N, w is a minimal lengths description of of x . Now, consider the machine M , M = "On input T, s , where T is a TM and s is a string: 1. Run T on s until it halts and produces a string z .
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

### Page1 / 19

13-mdl - Algorithmic Information Theory Consider the...

This preview shows document pages 1 - 5. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online