This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: Statistics 3858 : Maximum Likelihood Estimators 1 Method of Maximum Likelihood In this method we construct the so called likelihood function, that is L ( θ ) = L ( θ ; X 1 ,X 2 ,...,X n ) = f n ( X 1 ,X 2 ,...,X n ; θ ) The function f n is either the joint pdf or joint pmf of the random variables X 1 ,X 2 ,...,X m , and the notation denotes the dependence of this distribution on θ , where θ ∈ Θ. The notation for the likelihood function L usually suppresses the random data X 1 ,X 2 ,...,X n and we usually write this as L ( θ ). Sometimes we also wish to denote the dependence on n and write L n ( θ ). When we want to denote the dependence on the random variables X 1 ,X 2 ,...,X n we write L ( θ ; X 1 ,X 2 ,...,X n ) = f n ( X 1 ,X 2 ,...,X n ; θ ). When we have observed data X 1 = x 1 ,X 2 = x 2 ,...,X n = x n , where the lower case letters denote the specific observed data we then have the observed likelihood L ( θ ) = L ( θ ; x 1 ,x 2 ,...,x n ) = f n ( x 1 ,x 2 ,...,x n ; θ ) For given random variables the maximum likelihood estimator, say ˆ θ n is the argument θ ∈ Θ for which L ( ˆ θ n ) = max θ ∈ Θ L ( θ ) Notice this means that ˆ θ n = ˆ θ n ( X 1 ,X 2 ,...,X n ) which says that ˆ θ n is a function of the random data or random variables X 1 ,X 2 ,...,X n . Sometimes we may find this explicitly by finding this function, say h so that ˆ θ n = h ( X 1 ,X 2 ,...,X n ) For observed data x 1 ,x 2 ,...,x n this says ˆ θ n = h ( x 1 ,x 2 ,...,x n ) Mathematically this says we maximize over possible θ the function L ( θ ) = L ( θ ; x 1 ,x 2 ,...,x n ) = f n ( x 1 ,x 2 ,...,x n ; θ ) by treating this is a function with argument θ and treating x 1 ,x 2 ,...,x n as given and known numbers. 1 MLE 2 Since our goal is to maximize the likelihood can use anything proportional to the joint pdf or joint pmf, provided the constant of proportionality does not involve θ . Therefore we use as the likelihood function L anything positively proportional to the joint pdf or joint pmf L ( θ ) = L ( θ ; x 1 ,x 2 ,...,x n ) ∝ f n ( x 1 ,x 2 ,...,x n ; θ ) (1) Let ˆ θ n be the element θ ∈ Θ which maximizes L ( θ ) = L ( θ ; X 1 ,...X n ). We denote this value as ˆ θ n = argmax θ ∈ Θ L ( θ ) . Sometimes we are interested in a particular subset of Θ and the maximization may be done over that set. In any case the idea is the same but the maximization is restricted to θ in this particular subset. For example in a normal model the parameter space is Θ = R × R + , but we may require the mean is 0 and so compute a restricted MLE over the set Θ ′ = { (0 ,σ 2 ) : σ 2 > } ⊂ Θ. Of course we could also work a new statistical model with parameter space R + , where the parameter corresponds to σ 2 ....
View
Full
Document
This note was uploaded on 01/17/2012 for the course AM 1234 taught by Professor Qqqq during the Spring '11 term at UWO.
 Spring '11
 qqqq

Click to edit the document details