Unformatted text preview: Suﬀ conds for HMMs Generative Accuracy
Theorem 4.6.2
Suﬃcient conditions for HMM accuracy. An HMM ph (X1:T ) will
accurately represent a true discrete distribution p(X1:T ) if the following
conditions hold for all t:
H (Qt X<t ) = 0
ph (Xt = xt qx<t ) = p(Xt = xt X<t = x<t ).
where qx<t = f (x<t ) is the unique state subsequence associated with
x<t .
Quite strong and unrealistic requirements, but they guarantee
accuracy nonetheless.
∆ Note {< t} = {1, 2, . . . , t − 1} Prof. Jeﬀ Bilmes EE596A/Winter 2013/DGMs – Lecture 4  Jan 23rd, 2013 page 483 (of 239) HMMs HMMs as GMs Other HMM queries What HMMs can do MPE Summ Proof: Suﬀ conds for HMMs Generative Accuracy
Proof.
We have for all t:
D(p(Xt X<t )ph (Xt X<t ))
p(xt x<t )
=
p(x1:t ) log
ph (xt x<t )
x (4.90)
(4.91) 1:t p(x1:t ) log =
x1:t p(xt x<t )
qt ph (xt qt )ph (qt x<t ) (4.92) p(x1:t ) log p(xt x<t )
ph (xt qx<t ) (4.93) p(x1:t ) log = p(xt x<t )
p(xt x<t ) (4.94) x1:t =
x1:t =0
Prof. Jeﬀ Bilmes (4.95)
EE596A/Winter 2013/DGMs – Lecture 4  Jan 23rd, 2013 page 484 (of 239) HMMs HMMs as GMs Other HMM queries What HMMs can do MPE Summ Cont. Proof: Suﬀ conds for HMMs Generative Accuracy
... cont.
It then follows, using the above equation, that:
D(p(Xt X<t )ph (Xt X<t )) 0= (4.96) t p(x1:t ) log =
t x1:t p(x1:T ) log =
x1:T p(xt x<t )
=
ph (xt x<t ) p(x1:T ) log
t t p(xt x<t )
=
t ph (xt x<t )
x x1:T p(x1:T ) log
1:T p(xt x<t )
(4.97)
ph (xt x<t ) p(x1:T )
ph (x1:T ) (4.98) = D(p(X1:T )ph (X1:T )) (4.99) Strong conditions H (Qt X<t ) = 0, not likely to happen in practice.
Is this really what we need for an HMM, generative accuracy? We’ll
address this again soon.
Prof. Jeﬀ Bilmes EE596A/Winter 2013/DGMs – Lecture 4  Jan 23rd, 2013 page 485 (of 239) HMMs HMMs as GMs Other HMM queries What HMMs can do MPE Summ Mealy vs. Moore Machines
Mealy vs. Moore ﬁnite state automata:
Input
Symbol
State Logic Input
Symbol Output
Symbol State Memory Prof. Jeﬀ Bilmes Output
Symbol Logic EE596A/Winter 2013/DGMs – Lecture 4  Jan 23rd, 2013 Logic
Memory page 486 (of 239) HMMs HMMs as GMs Other HMM queries What HMMs can do MPE Summ Mealy vs. Moore Machines
Mealy vs. Moore ﬁnite state automata:
Input
Symbol
State Logic Input
Symbol Output
Symbol State Memory Output
Symbol Logic Logic
Memory Moore machine has only one possible output for each state — the
output is a function only of the current state. Prof. Jeﬀ Bilmes EE596A/Winter 2013/DGMs – Lecture 4  Jan 23rd, 2013 page 486 (of 239) HMMs HMMs as GMs Other HMM queries What HMMs can do MPE Summ Mealy vs. Moore Machines
Mealy vs. Moore ﬁnite state automata:
Input
Symbol
State Logic Input
Symbol Output
Symbol State Memory Output
Symbol Logic Logic
Memory Moore machine has only one possible output for each state — the
output is a function only of the current state.
Mealy machine has only one possible output for each stateinput
pair — given current state, the input determines both the next state
and the current output. Prof. Jeﬀ Bilmes EE596A/Winter 2013/...
View
Full
Document
This document was uploaded on 04/05/2014.
 Winter '14

Click to edit the document details