NIPS2009_0010_slide

# NIPS2009_0010_slide - MDL selects Q which leads to minimal...

This preview shows page 1. Sign up to view the full content.

Discrete MDL Predicts in Total Variation www.hutter1.net Main result informal: For any countable class of models M = { Q 1 ,Q 2 ,... } containing the unknown true sampling distribution P , MDL predictions converge to the true distribution in total variation distance. Formally . .. Given x = x 1 ...x , the Q -prob. of z = x +1 x +2 ... is Q ( z | x ) = Q ( xz ) Q ( x ) Use Q = Bayes or Q = MDL instead of P for prediction Total variation distance: d ( P,Q ) := sup A ⊆X ± ± Q [ A | x ] - P [ A | x ] ± ± Bayes ( x ) := Q ∈M Q ( x ) w Q , [ w Q > 0 Q ∈M and Q ∈M w Q = 1 ]
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: MDL selects Q which leads to minimal code length for x : MDL x := arg min Q ∈M {-log Q ( x ) + K ( Q ) } , [ ∑ Q ∈M 2-K ( Q ) ≤ 1 ] Theorem 1 (Discrete Bayes&MDL Predict in Total Variation) d ∞ ( P, Bayes | x ) → d ∞ ( P, MDL x | x ) → { almost surely for ℓ ( x ) →∞ } [Blackwell&Dubins 1962] [Hutter NIPS 2009] No independence , ergodicity , stationarity , identiﬁability , or other assumption...
View Full Document

## This note was uploaded on 02/12/2010 for the course COMPUTER S 10586 taught by Professor Jilinwang during the Fall '09 term at Zhejiang University.

Ask a homework question - tutors are online