This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: Stat 411 â€“ Lecture Notes Supplement Computation of maximum likelihood estimators *â€ â€¡ Ryan Martin Spring 2012 1 Introduction Suppose that sample data X 1 ,...,X n are iid with common distribution having PMF/PDF f Î¸ ( x ). The goal is to estimate the unknown parameter Î¸ . In simple examples (including those presented in class), there is a closedform expression for Ë† Î¸ , the maximum likelihood estimator (MLE), the maximizer of the likelihood function L ( Î¸ ) = Q n i =1 f Î¸ ( X i ). But in many practical problems, there may be more than one solution to the likelihood equation ( âˆ‚/âˆ‚Î¸ )log L ( Î¸ ) = 0, and there may be no nice formula for those solutions. In such cases, one will need to use some numerical methods to compute the MLE. The purpose of this document is to elaborate on this computational aspect. These concepts are not central to our goal of developing the basics of statistical theory and, as such, will not be tested on. However, these are important throughout all of applied and theoretical statistics. In this note I shall focus one just one optimization strategy, namely Newtonâ€™s method. A course on computational statistics would delve deeper into other methods, such as the EM algorithm, simulated annealing, and iteratively reweighted least squares. 2 Newtonâ€™s method Suppose the goal is to solve g ( x ) = 0 for some function g . Newtonâ€™s method is one useful technique, and the basics are presented in early calculus courses. The idea is based on the fact that, locally, any differentiable function can be suitably approximated by a linear function. This linear function is then used to define a recursive procedure that will, under suitable conditions, eventually find the desired solution. 2.1 Single parameter problems Consider a problem where the unknown parameter Î¸ is a number. An example is where the underlying distribution is Pois ( Î¸ ). The next subsection deals with the case of a vector * Version: February 15, 2012 â€ Please do not distribute these notes without the authorâ€™s consent ( [email protected] ) â€¡ These notes are meant solely to supplement inclass lectures. The author makes no guarantees that these notes are free of typos or other, more serious errors. 1 of parameters. Write â€˜ ( Î¸ ) for the loglikelihood log L ( Î¸ ). Assume that â€˜ ( Î¸ ) is twice differentiable with respect to Î¸ ; this is not really a practical restriction, in fact, our good theoretical properties of MLEs assumes this and more....
View
Full
Document
This note was uploaded on 03/12/2012 for the course STAT 411 taught by Professor Staff during the Spring '08 term at Ill. Chicago.
 Spring '08
 STAFF

Click to edit the document details