312: Stat Lecture 05 Maximum Likelihood Estimation
Moo K. Chung [e-mail] September 14, 2004
^ 1. (Invariance Principle) If is the MLE's of ^ parameter then the MLE of h() is h() for some function h. Proof (partial). Consider likelihood function ^ L(). satisfies dL() = 0. Let = h(). d Then the likelihood function for = h() is given by L(h-1 ()). Differentiating the likelihood with respect to , we have L(h-1 ()) dL() d dL() 1 = = = 0. d d d d h () 2. Loglikelihood. Maximizing L() is equivalent to maximizing ln L() since ln is an increasing function. Example. This technique is best illustrated by finding the MLE of parameters in N (, 2 ). = X, ^ 2 = ^ 1 n
n
5. Additional problem (previous midterm). Let X1 , X2 be a random sample from N (0, 1/). Note that the sample size is 2 and the density function for Xi is f (xi ) = exp(-x2 /2). i 2 Find the likelihood function and use it to obtain the maximum likelihood estimator of . Solution. The likelihood is function L() = /(2) exp - (x2 + x2 )/2 . 1 2 Now get log-likelihood function ln L() = const + ln - (x2 + x2 )/2. 1 2 Differentiate with respect to we get d ln L() 1 1 = - (x2 + x2 ) = 0. 2 d 2 1 Solving the equation, we get ^ = 2/(x2 + x2 ). 1 2 Review problems. Example 6.17., Example 6.18., Exercise 6.23., Exercise 6.29. Read Chapter 7. Homework II. Exercise 6.30., 6.38., 7.4. 7.10. 7.12. 7.14. Due Sept 30. 9:30am.
(Xi - X)2
i=1
are the MLE of and 2 respectively. Note that 2 is not un unbiased estimator of 2 . ^ 3. Asymptotic unbiasness. When the sample ...