# 90 491 1t px1t log x1t pxt xt qt ph xt

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Suﬀ conds for HMMs Generative Accuracy Theorem 4.6.2 Suﬃcient conditions for HMM accuracy. An HMM ph (X1:T ) will accurately represent a true discrete distribution p(X1:T ) if the following conditions hold for all t: H (Qt |X<t ) = 0 ph (Xt = xt |qx<t ) = p(Xt = xt |X<t = x<t ). where qx<t = f (x<t ) is the unique state sub-sequence associated with x<t . Quite strong and unrealistic requirements, but they guarantee accuracy nonetheless. ∆ Note {< t} = {1, 2, . . . , t − 1} Prof. Jeﬀ Bilmes EE596A/Winter 2013/DGMs – Lecture 4 - Jan 23rd, 2013 page 4-83 (of 239) HMMs HMMs as GMs Other HMM queries What HMMs can do MPE Summ Proof: Suﬀ conds for HMMs Generative Accuracy Proof. We have for all t: D(p(Xt |X<t )||ph (Xt |X<t )) p(xt |x<t ) = p(x1:t ) log ph (xt |x<t ) x (4.90) (4.91) 1:t p(x1:t ) log = x1:t p(xt |x<t ) qt ph (xt |qt )ph (qt |x<t ) (4.92) p(x1:t ) log p(xt |x<t ) ph (xt |qx<t ) (4.93) p(x1:t ) log = p(xt |x<t ) p(xt |x<t ) (4.94) x1:t = x1:t =0 Prof. Jeﬀ Bilmes (4.95) EE596A/Winter 2013/DGMs – Lecture 4 - Jan 23rd, 2013 page 4-84 (of 239) HMMs HMMs as GMs Other HMM queries What HMMs can do MPE Summ Cont. Proof: Suﬀ conds for HMMs Generative Accuracy ... cont. It then follows, using the above equation, that: D(p(Xt |X<t )||ph (Xt |X<t )) 0= (4.96) t p(x1:t ) log = t x1:t p(x1:T ) log = x1:T p(xt |x<t ) = ph (xt |x<t ) p(x1:T ) log t t p(xt |x<t ) = t ph (xt |x<t ) x x1:T p(x1:T ) log 1:T p(xt |x<t ) (4.97) ph (xt |x<t ) p(x1:T ) ph (x1:T ) (4.98) = D(p(X1:T )||ph (X1:T )) (4.99) Strong conditions H (Qt |X<t ) = 0, not likely to happen in practice. Is this really what we need for an HMM, generative accuracy? We’ll address this again soon. Prof. Jeﬀ Bilmes EE596A/Winter 2013/DGMs – Lecture 4 - Jan 23rd, 2013 page 4-85 (of 239) HMMs HMMs as GMs Other HMM queries What HMMs can do MPE Summ Mealy vs. Moore Machines Mealy vs. Moore ﬁnite state automata: Input Symbol State Logic Input Symbol Output Symbol State Memory Prof. Jeﬀ Bilmes Output Symbol Logic EE596A/Winter 2013/DGMs – Lecture 4 - Jan 23rd, 2013 Logic Memory page 4-86 (of 239) HMMs HMMs as GMs Other HMM queries What HMMs can do MPE Summ Mealy vs. Moore Machines Mealy vs. Moore ﬁnite state automata: Input Symbol State Logic Input Symbol Output Symbol State Memory Output Symbol Logic Logic Memory Moore machine has only one possible output for each state — the output is a function only of the current state. Prof. Jeﬀ Bilmes EE596A/Winter 2013/DGMs – Lecture 4 - Jan 23rd, 2013 page 4-86 (of 239) HMMs HMMs as GMs Other HMM queries What HMMs can do MPE Summ Mealy vs. Moore Machines Mealy vs. Moore ﬁnite state automata: Input Symbol State Logic Input Symbol Output Symbol State Memory Output Symbol Logic Logic Memory Moore machine has only one possible output for each state — the output is a function only of the current state. Mealy machine has only one possible output for each state-input pair — given current state, the input determines both the next state and the current output. Prof. Jeﬀ Bilmes EE596A/Winter 2013/...
View Full Document

## This document was uploaded on 04/05/2014.

Ask a homework question - tutors are online