319lec8_2010 - Chapter 6 Parameter Estimation and...

Chapter 6: Parameter Estimation and
Evaluation
References:
Chapter 9, Sections 9.1°9.13 (only for references)
6.1 Point estimation
Suppose f X
i
g n
i =1
is an i.i.d. random sample from some population f
X
( x ) : [Recall the popu-
lation distribution f
X
( x ) is the distribution of each X
i
: ]
Often, one assumes
f
X
( x ) = f ( x; ° ) ;
where f ( x; ° ) is unknown in functional form, but ° is unknown. For example, if f X
i
g is
i.i.d. N ( ±; ² 2 ) ; then
f
X
( x ) = f ( x; ° )
=
1
p
2 ³² 2
exp[ °
1
2 ²
2
( x ° ± )
2
] ;
where ° = ( ±; ²
2
) :
Suppose the random sample X n = f X
1
; X
2
; ::; X
n
g has a realization x n = f x
1
; x
2
; :::; x
n
g :
Then the realization x n is called a data set. A random sample X n can generate many di/erent
data sets.
Estimator versus Estimate
De°nition [Estimator and Estimate]: Suppose ° is a parameter of a population, and
is unknown. An estimator
^
° =
^
° ( X
1
; :::; X
n
) of ° is a statistic that depends on the sample
information and whose realizations provide an approximation to °: A speci±c realization of that
r.v. is called an estimate.
De°nition [Point Estimator]: A point estimator
^
° is an estimator for ° that only gives a
single number for each sample realization x n = ( x
1
; x
2
; :::; x
n
) .
Example: Suppose X
n
= ( X
1
; :::; X
n
) is a random sample with ( ±; ²
2
) : Then we have ° =
±;
^
° =
°
X
n
= n
° 1
P
n
i =1
X
i
:
Example: Suppose X n = ( X
1
; :::; X
n
) is a random sample where X
i
± Bernoulli ( p ) ; 0 < p < 1 :
Then ° = p;
^
° = ^ p
n
=
°
X
n
:
Example: Suppose X n = ( X
1
; :::; X
n
) is a random sample with ( ±; ² 2 ) : Then we have ° =
² 2 ;
^
° = S 2
n
= ( n ° 1) ° 1
P
n
i =1
( X
i
°
°
X
n
) 2 :
1

Example: Suppose X n = ( X
1
; :::; X
n
) is a random sample with ( ±
1
; ² 2
1
) ; and Y m = ( Y
1
; :::; Y
m
)
is a random sample with ( ±
2
; ² 2
2
) : Then we have ° = ±
1
° ±
2
; and
^
° =
°
X
n
°
°
Y
m
:
Example: Suppose X n = ( X
1
; :::; X
n
) is a random sample with ( ±
1
; ² 2
1
) ; and Y m = ( Y
1
; :::; Y
m
)
is a random sample with ( ±
2
; ²
2
2
) : Then ° = ²
2
1
=²
2
2
and
^
° = S
2
n
=S
2
m
:
Notations of Estimators and Estimates:
parameter ( ° ) Estimator (
^
° ) Estimate
±
°
X
n
° x
n
²
2
S
2
n
s
2
n
² S
n
s
n
p ^ p
n
p
n
:
Example: Price-earnings ratios for a random sample of ten stocks traded in NYSE on a partic-
ular date is
10 ; 16 ; 5 ; 10 ; 12 ; 8 ; 4 ; 6 ; 5 ; 4 :
Find the point estimates of the population mean, variance and standard deviation, and the
proportion of stocks in the population for which the price-earnings ratio exceeded 8.5.
ANS: ±; ² 2 ; ² and p = P ( X > 8 : 5) :
° x
n
= 80 = 10 = 8 :
s
2
x
= ( n ° 1)
° 1
(
P
n
i =1
x
2
° n ° x
2
n
) = (782 ° 10 ² 8
2
) = 9 = 15 : 78 :
s
x
=
p
15 : 78 = 3 : 97 :
^ p
x
= 4 = 10 = 0 : 4 :
Questions: For any ° , ther are more than one estimators.
Example: Let ° = ±; then the following are estimators for ° = ± :
(i)
^
°
1
=
°
X
n
:
(ii)
^
°
2
= X
1
:
Which one is a better estimator?
How good is an estimator
^
° for ° ? We need to use some appropriate criterion to measure how
good the estimator is.
De°nition [Mean Squared Error (MSE)]: The MSE of an estimator
^
° is de±ned as
MSE (
^
° ) = E (
^
° ° ° )
2
:
Remarks: MSE(
^
° ) measures the variations or deviations of the estimator
^
° from the true
parameter ° . The smaller MSE(
^
° ) ; the better the estimator
^
°: A smaller MSE means that there
is smaller variation between
^
° and °:
De°nition [Relative E¢ ciency]: