mdl.jdu - A(n)(extremely) brief/crudeintroduction...

Info iconThis preview shows pages 1–11. Sign up to view the full content.

View Full Document Right Arrow Icon
1 A(n) (extremely)  brief/crude introduction  to minimum description  length principle jdu 2006-04
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
2 Outline • Conceptual/non-technical introduction • Probabilities and Codelengths • Crude MDL • Refined MDL • Other topics
Background image of page 2
3 Outline • Conceptual/non-technical introduction • Probabilities and Codelengths • Crude MDL • Refined MDL • Other topics
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
4 Introduction • Example: data compression – Description methods ce: Grnwald et al. (2005)  Advances in Minimum Description Length: Theory and Applications.
Background image of page 4
5 Introduction • Example: regression – Model selection and overfitting – Complexity of the model vs. Goodness of fit ce: Grnwald et al. (2005)  Advances in Minimum Description Length: Theory and Applications.
Background image of page 5

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
6 Introduction • Models vs. Hypotheses ce: Grnwald et al. (2005)  Advances in Minimum Description Length: Theory and Applications.
Background image of page 6
7 Introduction • Crude 2-part version of MDL ce: Grnwald et al. (2005)  Advances in Minimum Description Length: Theory and Applications.
Background image of page 7

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
8 Outline • Conceptual/non-technical introduction • Probabilities and Codelengths • Crude MDL • Refined MDL • Other topics
Background image of page 8
9 Probabilities and Codelengths • Let  X  be a finite or countable set – A code  C ( x ) for  X 1-to-1 mapping from  X  to  U n>0 {0,1} n L C ( x ): number of bits needed to encode  x  using  C P : probability distribution defined on  X P ( x ): the probability of  x A sequence of (usually  iid)  observations  x 1 , x 2 , …, x n : x n
Background image of page 9

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
10 Probabilities and Codelengths • Prefix codes: as examples of uniquely decodable  codes – no code word is a prefix of any other a 0 b 111 c 1011 d 1010 r 110 ! 100 Source: http://www.cs.princeton.edu/courses/archive/spring04/cos126/
Background image of page 10
Image of page 11
This is the end of the preview. Sign up to access the rest of the document.

Page1 / 35

mdl.jdu - A(n)(extremely) brief/crudeintroduction...

This preview shows document pages 1 - 11. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online