This preview shows pages 1–12. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: Least Squares Maximum Likelihood Estimation Introduction to Least Squares Estimation (LSE) Example: Estimate the mean of a distribution based on a sample of n values x 1 , x 2 , x 3 , , x n. Let = the population mean Then the Least Squares estimate of is the value that minimizes the sum of squares Intuitively, the quantity SS measures total (squared) deviation from the mean of the population, so its minimum value will be the least squares estimate. ( 29 ( 29 ( 29 ( 29 2 2 2 2 1 2 3 2 1 ... ( ) n n i i SS x x x x x = =  +  +  + +  =  Analytical Method To find the value of that minimizes SS we set the derivative with respect to equal to zero. ( 29 ( 29 ( 29 ( 29 ( 29 ( 29 ( 29 ( 29 ( 29 [ ] 1 2 3 1 2 3 1 2 3 ( ) 2 1 2 1 2 1 ... 2( )( 1) 2 ... ( ) = 2 ... n n n d SS x x x x d x x x x x x x x n = + + + + =  + + + +  + + + + Set the derivative = 0 so that ( ) d SS d = 0 = 0 Therefore, the least squares estimate of is the sample mean 1 2 3 ... n x x x x n + + + + 1 2 3 1 ... n i n i x x x x x n n = + + + + = = 1 n i i x x n = = Conclusion For a sample of n observations the least squares estimate of the population mean is given by the sample mean 1 n i x x n = Example A sample of seven values is obtained from a population: x 1 = 230, x 2 = 275, x 3 = 317, x 4 = 283, x 5 = 305, x 6 = 315, x 7 = 284 Find the least squares estimate of the population mean . Solution 230 275 317 283 305 315 291 288 7 7 i x x + + + + + + = = = The Least Square Estimate of is given by: Definition: Likelihood Function For a discrete random variable: Let x 1 , x 2 , x 3 , ., x n be the values of a discrete random variable X with parameter . Then the likelihood function of the sample is given by where P(x) is the population probability mass function. 1 2 3 1 ( ) ( ) ( ) ( )... ( ) ( ) n n i i L P x P x P x P x P x = = = Likelihood Function For a continuous random variable Let x 1 , x 2 , x 3 , ., x n be the values of a continuous random variable X with parameter . Then the likelihood function of the sample is given by where f(x) is the population probability density function. 1 2 3 1 ( ) ( ) ( ) ( )... ( ) ( ) n n i i L f x f x f x f x f x = = = Definition: Maximum Likelihood Estimate The value of that maximizes the likelihood function is called the maximum likelihood estimate . Example of MLE We illustrate the method of maximum likelihood estimation by demonstrating how to estimate the mean of a normal population with known standard deviation of = 5. Assume that two different experimenters have a preconceived idea of the population mean. John thinks the mean is 90 and Mary thinks the mean is 100....
View
Full
Document
This note was uploaded on 02/28/2012 for the course MANAGEMENT MGSC 272 taught by Professor Smith during the Spring '12 term at McGill.
 Spring '12
 smith

Click to edit the document details