{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

531f10EM1

# 531f10EM1 - STAT 531 EM algorithm HM Kim Department of...

This preview shows pages 1–6. Sign up to view the full content.

STAT 531: EM algorithm HM Kim Department of Mathematics and Statistics University of Calgary Fall 2010 1/18

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
In order to estimate θ , introduce the log likelihood function deﬁned as L ( θ ) = ln p ( X | θ ) Since ln( x ) is a strictly increasing function, the value of θ which maximizes p ( X | θ ) also maximizes L ( θ ). The EM algorithm is an iterative procedure for maximizing L ( θ ) . Assume that after the n th iteration the current estimate for θ is given by θ n . Since the objective is to maximize L ( θ ), we wish to compute an updated estimate θ such that, L ( θ ) > L ( θ n ) Equivalently we want to maximize the diﬀerence, L ( θ ) - L ( θ n ) = ln p ( X | θ ) - ln p ( X | θ n ) Fall 2010 2/18
Hidden variables may be introduced purely as an artiﬁce for making the maximum likelihood estimation of θ tractable. In this case, it is assumed that knowledge of the hidden variables will make the maximization of the likelihood function easier. Either way, denote the hidden random vector by Z and a given realization by z . The total probability p ( X | θ ) may be written in terms of the hidden variables z as, p ( X | θ ) = z p ( X | z , θ ) p ( Z | θ ) L ( θ ) - L ( θ n ) = ln ± z p ( X | z , θ ) p ( Z | θ ) ² - ln p ( X | θ n ) Fall 2010 3/18

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
, Convex function f is convex on [ a , b ] if f ( λ x 1 +(1 - λ ) x 2 ) λ f ( x 1 )+(1 - λ ) f ( x 2 ) x 1 , x 2 [ a , b ] , λ [0 , 1] Fall 2010 4/18
, Jensen’s inequality Let f be a convex function deﬁned on an interval I . If x 1 , ··· , x n I and λ 1 , λ 2 , , λ n 0 with λ i = 1, f n i =1 λ i x i !

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

### Page1 / 18

531f10EM1 - STAT 531 EM algorithm HM Kim Department of...

This preview shows document pages 1 - 6. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online