NIPS2009_0010_slide - MDL selects Q which leads to minimal...

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
Discrete MDL Predicts in Total Variation www.hutter1.net Main result informal: For any countable class of models M = { Q 1 ,Q 2 ,... } containing the unknown true sampling distribution P , MDL predictions converge to the true distribution in total variation distance. Formally . .. Given x = x 1 ...x , the Q -prob. of z = x +1 x +2 ... is Q ( z | x ) = Q ( xz ) Q ( x ) Use Q = Bayes or Q = MDL instead of P for prediction Total variation distance: d ( P,Q ) := sup A ⊆X ± ± Q [ A | x ] - P [ A | x ] ± ± Bayes ( x ) := Q ∈M Q ( x ) w Q , [ w Q > 0 Q ∈M and Q ∈M w Q = 1 ]
Background image of page 1
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: MDL selects Q which leads to minimal code length for x : MDL x := arg min Q ∈M {-log Q ( x ) + K ( Q ) } , [ ∑ Q ∈M 2-K ( Q ) ≤ 1 ] Theorem 1 (Discrete Bayes&MDL Predict in Total Variation) d ∞ ( P, Bayes | x ) → d ∞ ( P, MDL x | x ) → { almost surely for ℓ ( x ) →∞ } [Blackwell&Dubins 1962] [Hutter NIPS 2009] No independence , ergodicity , stationarity , identifiability , or other assumption...
View Full Document

This note was uploaded on 02/12/2010 for the course COMPUTER S 10586 taught by Professor Jilinwang during the Fall '09 term at Zhejiang University.

Ask a homework question - tutors are online