747 part 18 maximum likelihood estimation average

Info icon This preview shows pages 8–16. Sign up to view the full content.

View Full Document Right Arrow Icon
™  7/47
Image of page 8

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Part 18: Maximum Likelihood Estimation Average Time Until Failure Estimating the average time until failure, , of light bulbs. yi = observed life until failure. f(yi|) = (1/ )exp(-yi/) L() = Πi f(yi|)= -n exp(- Σyi/ ) logL() = -nlog () - Σyi/ Likelihood equation: ∂logL()/∂ = -n/ + Σyi/2 =0 Solution: MLE = Σyi /n. Note: E[yi]=  Note, ∂logf(yi|)/∂ = -1/ + yi/2 Since E[yi]= , E[∂logf()/∂]=0. (One of the Regularity conditions discussed below) ™  8/47
Image of page 9
Part 18: Maximum Likelihood Estimation The Linear (Normal) Model Definition of the likelihood function - joint density of the observed data, written as a function of the parameters we wish to estimate. Definition of the maximum likelihood estimator as that function of the observed data that maximizes the likelihood function, or its logarithm. For the model: yi = x i + i, where i ~ N[0,2], the maximum likelihood estimators of and 2 are b = ( XX )-1 Xy and s2 = ee /n. That is, least squares is ML for the slopes, but the variance estimator makes no degrees of freedom correction, so the MLE is biased. ™  9/47
Image of page 10

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Part 18: Maximum Likelihood Estimation Normal Linear Model The log-likelihood function = i log f(yi| ) = sum of logs of densities. For the linear regression model with normally distributed disturbances logL = i [ -½log 2 - ½log 2 - ½(yi – xi )2/2 ]. = -n/2[log2 + log2 + s2/2] s2 = ee /n ™  10/47
Image of page 11
Part 18: Maximum Likelihood Estimation Likelihood Equations The estimator is defined by the function of the data that equates log-L/ to 0 . (Likelihood equation) The derivative vector of the log-likelihood function is the score function . For the regression model, g = [logL/ , logL/2]’ = logL/ = i [(1/2) xi (yi - xi ) ] = X/2 . logL/2 = i [ - 1/(22) + (yi - xi )2/(24)] = -N/22 [1 – s2/2] For the linear regression model, the first derivative vector of logL is (1/2) X ( y - X ) and (1/22) i [(yi - xi )2/2 - 1] (K1) (11) Note that we could compute these functions at any and 2. If we compute them at b and ee /n, the functions will be identically zero. ™  11/47
Image of page 12

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Part 18: Maximum Likelihood Estimation Information Matrix The negative of the second derivatives matrix of the log- likelihood, - H = forms the basis for estimating the variance of the MLE. It is usually a random matrix. ™  12/47 2 log ' i i f - ∂ ∂ θ θ
Image of page 13
Part 18: Maximum Likelihood Estimation Hessian for the Linear Model Note that the off diagonal elements have expectation zero. ™  13/47 ∂ ∂ ∂ ∂σ - ∂ ∂ ∂σ ∂ ∂σ ∂σ - σ σ - - σ σ β β β θ θ β β β β 2 2 2 2 2 2 2 2 2 i 2 i i 2 2 i i 2 4 i i logL logL logL = - logL logL 1 (y ) 1 = 1 1 (y ) (y ) 2 i i i i i i i x x x x x x x
Image of page 14

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Part 18: Maximum Likelihood Estimation Information Matrix (which should look familiar). The off diagonal terms go to zero (one of the assumptions of the linear model).
Image of page 15
Image of page 16
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern