This preview shows pages 1–6. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: MAXIMUM LIKELIHOOD and REML ESTIMATION IN THE GENERAL LINEAR MODEL c 2011 Dept. Statistics (Iowa State University) Stat 511 section 21 1 / 23 I Suppose f ( w  θ ) is the probability density function ( pdf ) or probability mass function ( pmf ) of a random vector w , where θ is a k × 1 vector of parameters. I Given a value of the parameter vector θ , f ( w  θ ) is a realvalued function of w . I The likelihood function L ( θ  w ) = f ( w  θ ) is a realvalued function of θ for a given value of w . I L ( θ  w ) is not a pdf. R θ L ( θ  w ) d θ 6 = 1 . I For any potential observed value w , define ˆ θ ( w ) to be a parameter value at which L ( θ  w ) attains its maximum value. ˆ θ ( w ) is a maximum likelihood estimator (MLE) of θ . I Invariance property of MLES: The MLE of a function of θ , say g ( θ ) , is the function evaluated at the MLE of θ : d g ( θ ) = g ( ˆ θ ) c 2011 Dept. Statistics (Iowa State University) Stat 511 section 21 2 / 23 I Often much more convenient to work with l ( θ  w ) = log L ( w  θ ) . I If l ( θ  w ) is differentiable, candidates for the MLE of θ can be found by equating the score function ∂ l ( θ  w ) ∂ θ ≡ ∂ l ( θ  w ) ∂ θ . . . ∂ l ( θ  w ) ∂ θ k to and solving for θ I The score equations are ∂ l ( θ  w ) ∂ θ = ⇒ ∂ l ( θ  w ) ∂θ j = ∀ j = 1 ,..., k I One strategy for obtaining an MLE is to find solution(s) of the score equations and verify that at least one such solution maximizes l ( θ  w ) . I If the solution(s) to the score equations lie outside the appropriate parameter space, they are not MLE’s c 2011 Dept. Statistics (Iowa State University) Stat 511 section 21 3 / 23 I Example: Normal Theory GaussMarkov Linear Model y {z} n × 1 = X {z} n × p β {z} p × 1 + {z} n × 1 ∼ N ( ,σ 2 I ) θ {z} ( p + 1 ) × 1 = β σ 2 f ( y  θ ) = exp { 1 2 ( y x β ) ( σ 2 I ) 1 ( y x β ) } ( 2 π ) n / 2  σ 2 I  1 / 2 = 1 ( 2 πσ 2 ) n / 2 exp 1 2 σ 2 ( y x β ) ( y x β ) l ( θ  y ) = n 2 log 2 πσ 2 1 2 σ 2 ( y x β ) ( y x β ) I The score function is ∂ l ( θ  y ) ∂ θ = " ∂ l ( θ  y ) ∂ β ∂ l ( θ  y ) ∂σ 2 # = " 1 σ 2 ( x y x x β ) ( y x β ) ( y x β ) 2 σ 4 n 2 σ 2 # I The score equations are ∂ l ( θ  y ) ∂ θ = ⇔ x x β = x y σ 2 = ( y x β ) ( y x β ) n c 2011 Dept. Statistics (Iowa State University) Stat 511 section 21 4 / 23 I Any solution lies in the appropriate parameter space: β ∈ IR p , σ 2 ∈ IR + . I A solution to the score equations is " ˆ β ( y x ˆ β ) ( y x ˆ β ) n # , where ˆ β is a solution to the normal equations I Need to show this is a maximum of the likelihood function I We already know that any solution to the normal equations, β = ˆ β , minimizes ( y X β ) ( y X β ) for β ∈ IR p ....
View
Full
Document
This note was uploaded on 02/11/2012 for the course STAT 511 taught by Professor Staff during the Spring '08 term at Iowa State.
 Spring '08
 Staff
 Statistics, Probability

Click to edit the document details