# 21MLE - MAXIMUM LIKELIHOOD and REML ESTIMATION IN THE...

This preview shows pages 1–6. Sign up to view the full content.

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: MAXIMUM LIKELIHOOD and REML ESTIMATION IN THE GENERAL LINEAR MODEL c 2011 Dept. Statistics (Iowa State University) Stat 511 section 21 1 / 23 I Suppose f ( w | θ ) is the probability density function ( pdf ) or probability mass function ( pmf ) of a random vector w , where θ is a k × 1 vector of parameters. I Given a value of the parameter vector θ , f ( w | θ ) is a real-valued function of w . I The likelihood function L ( θ | w ) = f ( w | θ ) is a real-valued function of θ for a given value of w . I L ( θ | w ) is not a pdf. R θ L ( θ | w ) d θ 6 = 1 . I For any potential observed value w , define ˆ θ ( w ) to be a parameter value at which L ( θ | w ) attains its maximum value. ˆ θ ( w ) is a maximum likelihood estimator (MLE) of θ . I Invariance property of MLES: The MLE of a function of θ , say g ( θ ) , is the function evaluated at the MLE of θ : d g ( θ ) = g ( ˆ θ ) c 2011 Dept. Statistics (Iowa State University) Stat 511 section 21 2 / 23 I Often much more convenient to work with l ( θ | w ) = log L ( w | θ ) . I If l ( θ | w ) is differentiable, candidates for the MLE of θ can be found by equating the score function ∂ l ( θ | w ) ∂ θ ≡ ∂ l ( θ | w ) ∂ θ . . . ∂ l ( θ | w ) ∂ θ k to and solving for θ I The score equations are ∂ l ( θ | w ) ∂ θ = ⇒ ∂ l ( θ | w ) ∂θ j = ∀ j = 1 ,..., k I One strategy for obtaining an MLE is to find solution(s) of the score equations and verify that at least one such solution maximizes l ( θ | w ) . I If the solution(s) to the score equations lie outside the appropriate parameter space, they are not MLE’s c 2011 Dept. Statistics (Iowa State University) Stat 511 section 21 3 / 23 I Example: Normal Theory Gauss-Markov Linear Model y |{z} n × 1 = X |{z} n × p β |{z} p × 1 + |{z} n × 1 ∼ N ( ,σ 2 I ) θ |{z} ( p + 1 ) × 1 = β σ 2 f ( y | θ ) = exp {- 1 2 ( y- x β ) ( σ 2 I )- 1 ( y- x β ) } ( 2 π ) n / 2 | σ 2 I | 1 / 2 = 1 ( 2 πσ 2 ) n / 2 exp- 1 2 σ 2 ( y- x β ) ( y- x β ) l ( θ | y ) =- n 2 log 2 πσ 2- 1 2 σ 2 ( y- x β ) ( y- x β ) I The score function is ∂ l ( θ | y ) ∂ θ = " ∂ l ( θ | y ) ∂ β ∂ l ( θ | y ) ∂σ 2 # = " 1 σ 2 ( x y- x x β ) ( y- x β ) ( y- x β ) 2 σ 4- n 2 σ 2 # I The score equations are ∂ l ( θ | y ) ∂ θ = ⇔ x x β = x y σ 2 = ( y- x β ) ( y- x β ) n c 2011 Dept. Statistics (Iowa State University) Stat 511 section 21 4 / 23 I Any solution lies in the appropriate parameter space: β ∈ IR p , σ 2 ∈ IR + . I A solution to the score equations is " ˆ β ( y- x ˆ β ) ( y- x ˆ β ) n # , where ˆ β is a solution to the normal equations I Need to show this is a maximum of the likelihood function I We already know that any solution to the normal equations, β = ˆ β , minimizes ( y- X β ) ( y- X β ) for β ∈ IR p ....
View Full Document

## This note was uploaded on 02/11/2012 for the course STAT 511 taught by Professor Staff during the Spring '08 term at Iowa State.

### Page1 / 23

21MLE - MAXIMUM LIKELIHOOD and REML ESTIMATION IN THE...

This preview shows document pages 1 - 6. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online