9. Maximum Likelihood Estimators (Jan21,24,26)

# 9. Maximum Likelihood Estimators (Jan21,24,26) - Statistics...

This preview shows pages 1–3. Sign up to view the full content.

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Statistics 3858 : Maximum Likelihood Estimators 1 Method of Maximum Likelihood In this method we construct the so called likelihood function, that is L ( θ ) = L ( θ ; X 1 ,X 2 ,...,X n ) = f n ( X 1 ,X 2 ,...,X n ; θ ) The function f n is either the joint pdf or joint pmf of the random variables X 1 ,X 2 ,...,X m , and the notation denotes the dependence of this distribution on θ , where θ ∈ Θ. The notation for the likelihood function L usually suppresses the random data X 1 ,X 2 ,...,X n and we usually write this as L ( θ ). Sometimes we also wish to denote the dependence on n and write L n ( θ ). When we want to denote the dependence on the random variables X 1 ,X 2 ,...,X n we write L ( θ ; X 1 ,X 2 ,...,X n ) = f n ( X 1 ,X 2 ,...,X n ; θ ). When we have observed data X 1 = x 1 ,X 2 = x 2 ,...,X n = x n , where the lower case letters denote the specific observed data we then have the observed likelihood L ( θ ) = L ( θ ; x 1 ,x 2 ,...,x n ) = f n ( x 1 ,x 2 ,...,x n ; θ ) For given random variables the maximum likelihood estimator, say ˆ θ n is the argument θ ∈ Θ for which L ( ˆ θ n ) = max θ ∈ Θ L ( θ ) Notice this means that ˆ θ n = ˆ θ n ( X 1 ,X 2 ,...,X n ) which says that ˆ θ n is a function of the random data or random variables X 1 ,X 2 ,...,X n . Sometimes we may find this explicitly by finding this function, say h so that ˆ θ n = h ( X 1 ,X 2 ,...,X n ) For observed data x 1 ,x 2 ,...,x n this says ˆ θ n = h ( x 1 ,x 2 ,...,x n ) Mathematically this says we maximize over possible θ the function L ( θ ) = L ( θ ; x 1 ,x 2 ,...,x n ) = f n ( x 1 ,x 2 ,...,x n ; θ ) by treating this is a function with argument θ and treating x 1 ,x 2 ,...,x n as given and known numbers. 1 MLE 2 Since our goal is to maximize the likelihood can use anything proportional to the joint pdf or joint pmf, provided the constant of proportionality does not involve θ . Therefore we use as the likelihood function L anything positively proportional to the joint pdf or joint pmf L ( θ ) = L ( θ ; x 1 ,x 2 ,...,x n ) ∝ f n ( x 1 ,x 2 ,...,x n ; θ ) (1) Let ˆ θ n be the element θ ∈ Θ which maximizes L ( θ ) = L ( θ ; X 1 ,...X n ). We denote this value as ˆ θ n = argmax θ ∈ Θ L ( θ ) . Sometimes we are interested in a particular subset of Θ and the maximization may be done over that set. In any case the idea is the same but the maximization is restricted to θ in this particular subset. For example in a normal model the parameter space is Θ = R × R + , but we may require the mean is 0 and so compute a restricted MLE over the set Θ ′ = { (0 ,σ 2 ) : σ 2 > } ⊂ Θ. Of course we could also work a new statistical model with parameter space R + , where the parameter corresponds to σ 2 ....
View Full Document

## This note was uploaded on 01/17/2012 for the course AM 1234 taught by Professor Qqqq during the Spring '11 term at UWO.

### Page1 / 15

9. Maximum Likelihood Estimators (Jan21,24,26) - Statistics...

This preview shows document pages 1 - 3. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online