This preview shows pages 1–8. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: Theory of Point Estimation Dr. Phillip YAM 2010/2011 Spring Semester Reference: Chapter 6 of “Probability and Statistical Inference” by Hogg and Tanis. Section 6.1 Point estimation I Estimating characteristics of the distribution from the corresponding ones from the sample; I Sample mean ¯ x can be thought of as an estimate of the distribution mean μ I Sample mean s 2 can be used as an estimate of the distribution variance σ 2 I What makes an estimate good? Can we say the closeness of the estimate to the true value? Section 6.1 Point estimation I The functional form of the p.d.f is known but depends on an unknown parameter θ . I The parameter space Ω. I Example: f ( x ; θ ) = 1 /θ exp ( x /θ ) for a positive number θ . I The experimenter needs a point estimate for the parameters Section 6.1 Point estimation I Repeat the experiment n independent times, observe the sample X 1 , X 2 ,..., X n . I Estimate the parameter by using the observations x 1 , x 2 ,..., x n . I Estimator for θ is a function (statistic) u ( X 1 , X 2 ,..., X n ). Suppose that X follows Bernoulli distribution with success probability p , f ( x ; p ) = p x (1 p ) 1 x . Pr ( X 1 = x 1 , X 2 = x 2 ,... X n = x n ) = n Y i =1 p x i (1 p ) 1 x i = p ∑ x i (1 p ) n ∑ x i . I Find the value p that maximizes it. I p value most likely to have produced these sample values. The joint p.d.f. is called the likelihood function. Section 6.1 Point estimation The first derivative or gradient of the likelihood function: L ( p ) = ( ∑ x i ) p ∑ x i 1 (1 p ) n ∑ x i ( n ∑ x i ) p ∑ x i (1 p ) n ∑ x i 1 . p ∑ x i (1 p ) n ∑ x i ∑ x i p n ∑ x i 1 p = 0 . p = ∑ n i =1 x i n = ¯ x . It can be shown that L 00 (¯ x ) < 0, so that L (¯ x ) is a maximum. ( ∑ n i =1 X i ) / n = X , is called the maximum likelihood estimator . Section 6.1 Point estimation When finding a maximum likelihood estimator, it is often easier to find the value of the parameter that maximizes the natural logarithm of the likelihood function rather than the value of the parameter that maximizes the likelihood function itself. ln L ( p ) = n X i =1 x i ! ln p + n n X i =1 x i ! ln(1 p ) . d [ln L ( p )] dp = n X i =1 x i ! 1 p + n n X i =1 x i ! 1 1 p = 0 , Section 6.1 Point estimation The joint p.d.f. of X 1 , X 2 , ··· , X n , namely, L ( θ 1 ,θ 2 , ··· ,θ m ) = f ( x 1 ; θ 1 , ··· ,θ m ) f ( x 2 ; θ 1 , ··· ,θ m ) ··· f ( x n ; θ 1 , ··· ,θ m ) , ( θ 1 ,θ 2 , ··· ,θ m ) ∈ Ω , When regarded as a function of θ 1 ,θ 2 , ··· ,θ m , is called the likelihood function . ˆ θ 1 = u 1 ( X 1 , ··· , X n ) , ˆ θ 2 = u 2 ( X 1 , ··· , X n ) , ....
View
Full
Document
This note was uploaded on 06/05/2011 for the course STATISTICS 2006 taught by Professor Ho during the Spring '11 term at CUHK.
 Spring '11
 Ho
 Statistics, Probability

Click to edit the document details