# mdl.jdu - A(n(extremely brief/crudeintroduction...

This preview shows pages 1–11. Sign up to view the full content.

1 A(n) (extremely)  brief/crude introduction  to minimum description  length principle jdu 2006-04

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
2 Outline • Conceptual/non-technical introduction • Probabilities and Codelengths • Crude MDL • Refined MDL • Other topics
3 Outline • Conceptual/non-technical introduction • Probabilities and Codelengths • Crude MDL • Refined MDL • Other topics

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
4 Introduction • Example: data compression – Description methods ce: Grnwald et al. (2005)  Advances in Minimum Description Length: Theory and Applications.
5 Introduction • Example: regression – Model selection and overfitting – Complexity of the model vs. Goodness of fit ce: Grnwald et al. (2005)  Advances in Minimum Description Length: Theory and Applications.

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
6 Introduction • Models vs. Hypotheses ce: Grnwald et al. (2005)  Advances in Minimum Description Length: Theory and Applications.
7 Introduction • Crude 2-part version of MDL ce: Grnwald et al. (2005)  Advances in Minimum Description Length: Theory and Applications.

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
8 Outline • Conceptual/non-technical introduction • Probabilities and Codelengths • Crude MDL • Refined MDL • Other topics
9 Probabilities and Codelengths • Let  X  be a finite or countable set – A code  C ( x ) for  X 1-to-1 mapping from  X  to  U n>0 {0,1} n L C ( x ): number of bits needed to encode  x  using  C P : probability distribution defined on  X P ( x ): the probability of  x A sequence of (usually  iid)  observations  x 1 , x 2 , …, x n : x n

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
10 Probabilities and Codelengths • Prefix codes: as examples of uniquely decodable  codes – no code word is a prefix of any other a 0 b 111 c 1011 d 1010 r 110 ! 100 Source: http://www.cs.princeton.edu/courses/archive/spring04/cos126/
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

### Page1 / 35

mdl.jdu - A(n(extremely brief/crudeintroduction...

This preview shows document pages 1 - 11. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online