{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

maximum likelihood extimator

maximum likelihood extimator - CHAPTER 3 MAXIMUM LIKELIHOOD...

Info icon This preview shows pages 1–2. Sign up to view the full content.

View Full Document Right Arrow Icon
March 14, 2003 CHAPTER 3. MAXIMUM LIKELIHOOD AND M-ESTIMATION 3.1 Maximum likelihood estimates — in exponential families. Let ( X, B ) be a measurable space and { P θ , θ Θ } a measurable family of laws on ( X, B ), dominated by a σ -finite measure v . Let f ( θ, x ) be a jointly measurable version of the density ( dP θ /dv )( x ) by Theorem 1.3.3. For each x X , a maximum likelihood estimate (MLE) of θ is any θ ˆ = θ ˆ ( x ) such that f ( ˆ θ, x ) = sup { f ( φ, x ) : φ Θ } . In other words, θ ˆ ( x ) is a point at which f ( · , x ) attains its maximum. In general, the supremum may not be attained, or it may be attained at more than one point. If it is attained at a unique point θ ˆ , then θ ˆ is called the maximum likelihood estimate of θ . A measurable function θ ˆ ( · ) defined on a measurable subset B of X is called a maximum likelihood estimator if for all x B , θ ˆ ( x ) is a maximum likelihood estimate of θ , and for v -almost all x not in B , the supremum of f ( · , x ) is not attained at any point. Examples . (i) For each θ > 0 let P θ be the uniform distribution on [0 , θ ], with f ( θ, x ) := 1 [0 ] ( x ) for all x . Then if X 1 , . . . , X n are observed, i.i.d. ( P θ ), the MLE of θ is X ( n ) := max( X 1 , . . . , X n ). Note however that if the density had been defined as 1 [0 ) ( x ), its supremum for given X 1 , . . . , X n would not be attained at any θ . The MLE of θ is the smallest possible value of θ given the data, so it is not a very reasonable estimate in some ways. For example, it is not Bayes admissible. (ii). For P θ = N ( θ, 1) n on R n , with usual densities, the sample mean X is the MLE of n θ . For N (0 , σ 2 ) n , σ > 0, the MLE of σ 2 is j =1 X j 2 /n . For N ( m, σ 2 ) n , n 2, the MLE n of ( m, σ 2 ) is ( X, j =1 ( X j X ) 2 /n ). Here recall that the usual, unbiased estimator of σ 2 has n 1 in place of n , so that the MLE is biased, although the bias is small, of order 1 /n 2 as n → ∞ . The MLE of σ 2 fails to exist (or equals 0, if 0 were allowed as a value of σ 2 ) exactly on the event that all X j are equal for j n , which happens for n = 1, but only with probability 0 for n 2. On this event, f (( X, σ 2 ) , x ) + as σ 0. In general, let Θ be an open subset of R k and suppose f ( θ, x ) has first partial deriva- tives with respect to θ j for j = 1 , . . . , k , forming the gradient vector k θ f ( θ, x ) := { ∂f ( θ, x ) /∂θ j } j =1 .
Image of page 1

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Image of page 2
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern