531f10EM1 - STAT 531: EM algorithm HM Kim Department of...

Info iconThis preview shows pages 1–6. Sign up to view the full content.

View Full Document Right Arrow Icon
STAT 531: EM algorithm HM Kim Department of Mathematics and Statistics University of Calgary Fall 2010 1/18
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
In order to estimate θ , introduce the log likelihood function defined as L ( θ ) = ln p ( X | θ ) Since ln( x ) is a strictly increasing function, the value of θ which maximizes p ( X | θ ) also maximizes L ( θ ). The EM algorithm is an iterative procedure for maximizing L ( θ ) . Assume that after the n th iteration the current estimate for θ is given by θ n . Since the objective is to maximize L ( θ ), we wish to compute an updated estimate θ such that, L ( θ ) > L ( θ n ) Equivalently we want to maximize the difference, L ( θ ) - L ( θ n ) = ln p ( X | θ ) - ln p ( X | θ n ) Fall 2010 2/18
Background image of page 2
Hidden variables may be introduced purely as an artifice for making the maximum likelihood estimation of θ tractable. In this case, it is assumed that knowledge of the hidden variables will make the maximization of the likelihood function easier. Either way, denote the hidden random vector by Z and a given realization by z . The total probability p ( X | θ ) may be written in terms of the hidden variables z as, p ( X | θ ) = z p ( X | z , θ ) p ( Z | θ ) L ( θ ) - L ( θ n ) = ln ± z p ( X | z , θ ) p ( Z | θ ) ² - ln p ( X | θ n ) Fall 2010 3/18
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
, Convex function f is convex on [ a , b ] if f ( λ x 1 +(1 - λ ) x 2 ) λ f ( x 1 )+(1 - λ ) f ( x 2 ) x 1 , x 2 [ a , b ] , λ [0 , 1] Fall 2010 4/18
Background image of page 4
, Jensen’s inequality Let f be a convex function defined on an interval I . If x 1 , ··· , x n I and λ 1 , λ 2 , , λ n 0 with λ i = 1, f n i =1 λ i x i !
Background image of page 5

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Image of page 6
This is the end of the preview. Sign up to access the rest of the document.

This note was uploaded on 02/04/2011 for the course STAT 531 taught by Professor Gaborlukacs during the Spring '11 term at Manitoba.

Page1 / 18

531f10EM1 - STAT 531: EM algorithm HM Kim Department of...

This preview shows document pages 1 - 6. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online