Review: Sending Models and Data Given the Models Models/Explanation/Theories are combination of random variables. Need to send random variables and their parameters Need to send structure between random variables (next lecture) Then send data given the model Best model minimizes length in bits of sending model and then data given model. CSI 661 - Uncertainty in A.I. Lecture 24 1 Review: Discrete Random Variable Encoding - 1 Non-combinatorial approach Create a Huffman code-book. One code word for each individual state/source code symbol Assign code word lengths according to log(P(X=i) Number of bits to send model/dictionary (code-book) M M - 1 N n[m] ln + 1 - ln( M - 1)!- ( n[m]) ln 2 12 N m =1 Number of bits to send data given model is N.H[X] CSI 661 - Uncertainty in A.I. Lecture 24 2 Review: Discrete Random Variable Encoding - 2 Combintorial Approach Create a Huffman code-book. One code-word for each possible combintorial sequence the events log(S) bits to send one of S equally likely events, Number of bits to send model log M N ! n[m]! m =1 Number of bits to send data log N + M -1CM -1 CSI 661 - Uncertainty in A.I. Lecture 24 3 Continuous Random Variables Number of "events" for continuous data is infinte. But data is actually x AOM/2 Num. bits to specify data given model log(AOM.f(x|)) as f(.) is a pdf, d = m / v But the parameters of the model are also continuous values CSI 661 - Uncertainty in A.I. Lecture 24 4 Discretizing the Parameter Space Need to discretize parameter space into cells AOPV If cell mid-point is the parameter to encode/decode data, coding scheme is highly dependent on apriori specified trivial coding details. Instead measure expected (average) message length using all the parameter values within the cell CSI 661 - Uncertainty in A.I. Lecture 24 5 AOPV Original MML Formulation Message length calculation for normal dist. N Range = log + log + f ( xi ).AOM AOPV AOPV i =1 Range Range ...