This preview shows page 1. Sign up to view the full content.
Unformatted text preview: αrecursion, and blue order is β recursion. Prof. Jeﬀ Bilmes EE596A/Winter 2013/DGMs – Lecture 4  Jan 23rd, 2013 page 435 (of 239) HMMs HMMs as GMs Other HMM queries What HMMs can do MPE Summ HMMs and elimination So we have the forward recursion as an elimination order and the
backwards recursion as an elimination order.
3 1 10 Q1 X1 9 5 2 Q2 X2 8 7 7 4 Q3 X3 6 9 5 6 Q4 X4 4 10 3 8 Q5 X5 2 1 Green order is αrecursion, and blue order is β recursion.
Since HMM is a tree, there are no additional ﬁllin edges via the
elimination orders we have chosen. Prof. Jeﬀ Bilmes EE596A/Winter 2013/DGMs – Lecture 4  Jan 23rd, 2013 page 435 (of 239) HMMs HMMs as GMs Other HMM queries What HMMs can do MPE Summ HMMs and elimination If we just eliminated the hidden variables successively from
lefttoright, we’d get:
q1 q3 q4 q5 x1 Prof. Jeﬀ Bilmes q2 x2 x3 x4 x5 EE596A/Winter 2013/DGMs – Lecture 4  Jan 23rd, 2013 page 436 (of 239) HMMs HMMs as GMs Other HMM queries What HMMs can do MPE Summ HMMs and elimination If we just eliminated the hidden variables successively from
lefttoright, we’d get:
q1 q2 q3 q4 q5 x1 x2 x3 x4 x5 This has the same complexity O(T N 2 ) even though the cliques have
gotten very big. Prof. Jeﬀ Bilmes EE596A/Winter 2013/DGMs – Lecture 4  Jan 23rd, 2013 page 436 (of 239) HMMs HMMs as GMs Other HMM queries What HMMs can do MPE Summ HMMs and elimination If we just eliminated the hidden variables successively from
lefttoright, we’d get:
q1 q2 q3 q4 q5 x1 x2 x3 x4 x5 This has the same complexity O(T N 2 ) even though the cliques have
gotten very big.
Why? Prof. Jeﬀ Bilmes EE596A/Winter 2013/DGMs – Lecture 4  Jan 23rd, 2013 page 436 (of 239) HMMs HMMs as GMs Other HMM queries What HMMs can do MPE Summ HMMs and elimination
Another elimination order will produce a diﬀerent recursion for HMMs.
For example, an insideoutside algorithm of sorts can be deﬁned
(normally this is used for computing marginals for stochastic
contextfree grammars) via the following elimination order.
3 1 Q1 X1 7 5 Q2 X2 10 9 Q3 Q4 X3 X4 8 6 4 Q5 X5 2 Exercise:: Derive the equations and a recursion for this elimination order
for general length T sequence. How is it diﬀerent than standard
forward/backward? Are there any advantages to this order? Prof. Jeﬀ Bilmes EE596A/Winter 2013/DGMs – Lecture 4  Jan 23rd, 2013 page 437 (of 239) HMMs HMMs as GMs Other HMM queries What HMMs can do MPE Summ HMM, posteriors
But as mentioned before, we want more than just p(x1:T ).
We need clique posteriors γt (i) = p(Qt = ix) and
ξt (i, j ) = p(Qt−1 = i, Qt = j x). We can also get these from α and
β.
γt (qt ) = p(qt x1:T ) = p(x1:T qt )p(qt )/p(x1:T )
= p(x1:t , xt+1:T qt )p(qt )/p(x1:T )
= p(xt+1:T qt , x1:t )p(x1:t qt )p(qt )/p(x1:T )
= p(x1:t , qt )p(xt+1:T qt )/p(x)
= α(qt )β (qt )/p(x)
α(qt )β (qt ) = α(qt )β (qt )/
qt Prof. Jeﬀ Bilmes EE596A/Winter 2013/DGMs – Lecture 4  Jan 23rd, 2013 page 438 (of 239) HMMs HMMs as GMs Other HMM queries What HMMs can do MPE Summ HMM, posteriors
How best to compute γt (...
View
Full
Document
 Winter '14

Click to edit the document details