# Since hmm is a tree there are no additional ll in

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: α-recursion, and blue order is β -recursion. Prof. Jeﬀ Bilmes EE596A/Winter 2013/DGMs – Lecture 4 - Jan 23rd, 2013 page 4-35 (of 239) HMMs HMMs as GMs Other HMM queries What HMMs can do MPE Summ HMMs and elimination So we have the forward recursion as an elimination order and the backwards recursion as an elimination order. 3 1 10 Q1 X1 9 5 2 Q2 X2 8 7 7 4 Q3 X3 6 9 5 6 Q4 X4 4 10 3 8 Q5 X5 2 1 Green order is α-recursion, and blue order is β -recursion. Since HMM is a tree, there are no additional ﬁll-in edges via the elimination orders we have chosen. Prof. Jeﬀ Bilmes EE596A/Winter 2013/DGMs – Lecture 4 - Jan 23rd, 2013 page 4-35 (of 239) HMMs HMMs as GMs Other HMM queries What HMMs can do MPE Summ HMMs and elimination If we just eliminated the hidden variables successively from left-to-right, we’d get: q1 q3 q4 q5 x1 Prof. Jeﬀ Bilmes q2 x2 x3 x4 x5 EE596A/Winter 2013/DGMs – Lecture 4 - Jan 23rd, 2013 page 4-36 (of 239) HMMs HMMs as GMs Other HMM queries What HMMs can do MPE Summ HMMs and elimination If we just eliminated the hidden variables successively from left-to-right, we’d get: q1 q2 q3 q4 q5 x1 x2 x3 x4 x5 This has the same complexity O(T N 2 ) even though the cliques have gotten very big. Prof. Jeﬀ Bilmes EE596A/Winter 2013/DGMs – Lecture 4 - Jan 23rd, 2013 page 4-36 (of 239) HMMs HMMs as GMs Other HMM queries What HMMs can do MPE Summ HMMs and elimination If we just eliminated the hidden variables successively from left-to-right, we’d get: q1 q2 q3 q4 q5 x1 x2 x3 x4 x5 This has the same complexity O(T N 2 ) even though the cliques have gotten very big. Why? Prof. Jeﬀ Bilmes EE596A/Winter 2013/DGMs – Lecture 4 - Jan 23rd, 2013 page 4-36 (of 239) HMMs HMMs as GMs Other HMM queries What HMMs can do MPE Summ HMMs and elimination Another elimination order will produce a diﬀerent recursion for HMMs. For example, an inside-outside algorithm of sorts can be deﬁned (normally this is used for computing marginals for stochastic context-free grammars) via the following elimination order. 3 1 Q1 X1 7 5 Q2 X2 10 9 Q3 Q4 X3 X4 8 6 4 Q5 X5 2 Exercise:: Derive the equations and a recursion for this elimination order for general length T sequence. How is it diﬀerent than standard forward/backward? Are there any advantages to this order? Prof. Jeﬀ Bilmes EE596A/Winter 2013/DGMs – Lecture 4 - Jan 23rd, 2013 page 4-37 (of 239) HMMs HMMs as GMs Other HMM queries What HMMs can do MPE Summ HMM, posteriors But as mentioned before, we want more than just p(x1:T ). We need clique posteriors γt (i) = p(Qt = i|x) and ξt (i, j ) = p(Qt−1 = i, Qt = j |x). We can also get these from α and β. γt (qt ) = p(qt |x1:T ) = p(x1:T |qt )p(qt )/p(x1:T ) = p(x1:t , xt+1:T |qt )p(qt )/p(x1:T ) = p(xt+1:T |qt , x1:t )p(x1:t |qt )p(qt )/p(x1:T ) = p(x1:t , qt )p(xt+1:T |qt )/p(x) = α(qt )β (qt )/p(x) α(qt )β (qt ) = α(qt )β (qt )/ qt Prof. Jeﬀ Bilmes EE596A/Winter 2013/DGMs – Lecture 4 - Jan 23rd, 2013 page 4-38 (of 239) HMMs HMMs as GMs Other HMM queries What HMMs can do MPE Summ HMM, posteriors How best to compute γt (...
View Full Document

Ask a homework question - tutors are online