Since hmm is a tree there are no additional ll in

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: α-recursion, and blue order is β -recursion. Prof. Jeff Bilmes EE596A/Winter 2013/DGMs – Lecture 4 - Jan 23rd, 2013 page 4-35 (of 239) HMMs HMMs as GMs Other HMM queries What HMMs can do MPE Summ HMMs and elimination So we have the forward recursion as an elimination order and the backwards recursion as an elimination order. 3 1 10 Q1 X1 9 5 2 Q2 X2 8 7 7 4 Q3 X3 6 9 5 6 Q4 X4 4 10 3 8 Q5 X5 2 1 Green order is α-recursion, and blue order is β -recursion. Since HMM is a tree, there are no additional fill-in edges via the elimination orders we have chosen. Prof. Jeff Bilmes EE596A/Winter 2013/DGMs – Lecture 4 - Jan 23rd, 2013 page 4-35 (of 239) HMMs HMMs as GMs Other HMM queries What HMMs can do MPE Summ HMMs and elimination If we just eliminated the hidden variables successively from left-to-right, we’d get: q1 q3 q4 q5 x1 Prof. Jeff Bilmes q2 x2 x3 x4 x5 EE596A/Winter 2013/DGMs – Lecture 4 - Jan 23rd, 2013 page 4-36 (of 239) HMMs HMMs as GMs Other HMM queries What HMMs can do MPE Summ HMMs and elimination If we just eliminated the hidden variables successively from left-to-right, we’d get: q1 q2 q3 q4 q5 x1 x2 x3 x4 x5 This has the same complexity O(T N 2 ) even though the cliques have gotten very big. Prof. Jeff Bilmes EE596A/Winter 2013/DGMs – Lecture 4 - Jan 23rd, 2013 page 4-36 (of 239) HMMs HMMs as GMs Other HMM queries What HMMs can do MPE Summ HMMs and elimination If we just eliminated the hidden variables successively from left-to-right, we’d get: q1 q2 q3 q4 q5 x1 x2 x3 x4 x5 This has the same complexity O(T N 2 ) even though the cliques have gotten very big. Why? Prof. Jeff Bilmes EE596A/Winter 2013/DGMs – Lecture 4 - Jan 23rd, 2013 page 4-36 (of 239) HMMs HMMs as GMs Other HMM queries What HMMs can do MPE Summ HMMs and elimination Another elimination order will produce a different recursion for HMMs. For example, an inside-outside algorithm of sorts can be defined (normally this is used for computing marginals for stochastic context-free grammars) via the following elimination order. 3 1 Q1 X1 7 5 Q2 X2 10 9 Q3 Q4 X3 X4 8 6 4 Q5 X5 2 Exercise:: Derive the equations and a recursion for this elimination order for general length T sequence. How is it different than standard forward/backward? Are there any advantages to this order? Prof. Jeff Bilmes EE596A/Winter 2013/DGMs – Lecture 4 - Jan 23rd, 2013 page 4-37 (of 239) HMMs HMMs as GMs Other HMM queries What HMMs can do MPE Summ HMM, posteriors But as mentioned before, we want more than just p(x1:T ). We need clique posteriors γt (i) = p(Qt = i|x) and ξt (i, j ) = p(Qt−1 = i, Qt = j |x). We can also get these from α and β. γt (qt ) = p(qt |x1:T ) = p(x1:T |qt )p(qt )/p(x1:T ) = p(x1:t , xt+1:T |qt )p(qt )/p(x1:T ) = p(xt+1:T |qt , x1:t )p(x1:t |qt )p(qt )/p(x1:T ) = p(x1:t , qt )p(xt+1:T |qt )/p(x) = α(qt )β (qt )/p(x) α(qt )β (qt ) = α(qt )β (qt )/ qt Prof. Jeff Bilmes EE596A/Winter 2013/DGMs – Lecture 4 - Jan 23rd, 2013 page 4-38 (of 239) HMMs HMMs as GMs Other HMM queries What HMMs can do MPE Summ HMM, posteriors How best to compute γt (...
View Full Document

Ask a homework question - tutors are online