90 491 1t px1t log x1t pxt xt qt ph xt

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Suff conds for HMMs Generative Accuracy Theorem 4.6.2 Sufficient conditions for HMM accuracy. An HMM ph (X1:T ) will accurately represent a true discrete distribution p(X1:T ) if the following conditions hold for all t: H (Qt |X<t ) = 0 ph (Xt = xt |qx<t ) = p(Xt = xt |X<t = x<t ). where qx<t = f (x<t ) is the unique state sub-sequence associated with x<t . Quite strong and unrealistic requirements, but they guarantee accuracy nonetheless. ∆ Note {< t} = {1, 2, . . . , t − 1} Prof. Jeff Bilmes EE596A/Winter 2013/DGMs – Lecture 4 - Jan 23rd, 2013 page 4-83 (of 239) HMMs HMMs as GMs Other HMM queries What HMMs can do MPE Summ Proof: Suff conds for HMMs Generative Accuracy Proof. We have for all t: D(p(Xt |X<t )||ph (Xt |X<t )) p(xt |x<t ) = p(x1:t ) log ph (xt |x<t ) x (4.90) (4.91) 1:t p(x1:t ) log = x1:t p(xt |x<t ) qt ph (xt |qt )ph (qt |x<t ) (4.92) p(x1:t ) log p(xt |x<t ) ph (xt |qx<t ) (4.93) p(x1:t ) log = p(xt |x<t ) p(xt |x<t ) (4.94) x1:t = x1:t =0 Prof. Jeff Bilmes (4.95) EE596A/Winter 2013/DGMs – Lecture 4 - Jan 23rd, 2013 page 4-84 (of 239) HMMs HMMs as GMs Other HMM queries What HMMs can do MPE Summ Cont. Proof: Suff conds for HMMs Generative Accuracy ... cont. It then follows, using the above equation, that: D(p(Xt |X<t )||ph (Xt |X<t )) 0= (4.96) t p(x1:t ) log = t x1:t p(x1:T ) log = x1:T p(xt |x<t ) = ph (xt |x<t ) p(x1:T ) log t t p(xt |x<t ) = t ph (xt |x<t ) x x1:T p(x1:T ) log 1:T p(xt |x<t ) (4.97) ph (xt |x<t ) p(x1:T ) ph (x1:T ) (4.98) = D(p(X1:T )||ph (X1:T )) (4.99) Strong conditions H (Qt |X<t ) = 0, not likely to happen in practice. Is this really what we need for an HMM, generative accuracy? We’ll address this again soon. Prof. Jeff Bilmes EE596A/Winter 2013/DGMs – Lecture 4 - Jan 23rd, 2013 page 4-85 (of 239) HMMs HMMs as GMs Other HMM queries What HMMs can do MPE Summ Mealy vs. Moore Machines Mealy vs. Moore finite state automata: Input Symbol State Logic Input Symbol Output Symbol State Memory Prof. Jeff Bilmes Output Symbol Logic EE596A/Winter 2013/DGMs – Lecture 4 - Jan 23rd, 2013 Logic Memory page 4-86 (of 239) HMMs HMMs as GMs Other HMM queries What HMMs can do MPE Summ Mealy vs. Moore Machines Mealy vs. Moore finite state automata: Input Symbol State Logic Input Symbol Output Symbol State Memory Output Symbol Logic Logic Memory Moore machine has only one possible output for each state — the output is a function only of the current state. Prof. Jeff Bilmes EE596A/Winter 2013/DGMs – Lecture 4 - Jan 23rd, 2013 page 4-86 (of 239) HMMs HMMs as GMs Other HMM queries What HMMs can do MPE Summ Mealy vs. Moore Machines Mealy vs. Moore finite state automata: Input Symbol State Logic Input Symbol Output Symbol State Memory Output Symbol Logic Logic Memory Moore machine has only one possible output for each state — the output is a function only of the current state. Mealy machine has only one possible output for each state-input pair — given current state, the input determines both the next state and the current output. Prof. Jeff Bilmes EE596A/Winter 2013/...
View Full Document

This document was uploaded on 04/05/2014.

Ask a homework question - tutors are online