Berkeley UC Department of Statistics STAT 210A: Introduction to Mathematical Statistics Solutions - Problem Set 8 Fall 2006 Issued: Thursday, November 2, 2006 Due: Thursday, November 9, 2006
Graded exercises
Problem 8.1 (a) From the definitions given, we can write: S1 1 0 S2 1 1 S3 1 1 = . . . . . . Sn 1 1 Sn+1 1 1 S = M E
0 0 1 1 1
0 0 E1 E2 0 0 0 0 E3 . . . 1 0 En 1 1 En+1
It follows that the density of S is given by:
n
f (S1 , S2 , . . . , Sn+1 ) = |M |
-1
fE (E(S)) =
i=1 n
exp (-(Si - si-1 )) I(Si - Si-1 0)
= exp (-Sn-1 )
i=1
I(Si - Si-1 0)
The characteristic function of an exponential random variable with mean 1 is given by E (t) = (1 - it)-1 and hence the distribution of Sn+1 has characteristic function Sn+1 = (1 - it)-n+1 , that is, Sn+1 Gamma(1, n + 1) with density fSn+1 (s) = sn exp(-s) sn exp(-s) and so: n! (n+1) = f (S1 , S2 , . . . , Sn |Sn+1 ) = fS (S1 , S2 , . . . , Sn )I(Sn Sn+1 ) fSn+1 (Sn+1 ) n I(Si+1 - Si 0) = n! i=1 n Sn+1
Noticing that conditional on Sn+1 the realized value for Sn+1 can be treated as a constant, we get by rescaling: S1 S2 Sn Sn f( , ,..., |Sn+1 ) = n!I( 1) Sn+1 Sn+1 Sn+1 Sn+1 1
n
I(
i=1
Si Si-1 ) I(S1 0) Sn+1 Sn+1
On the other hand, a sample of sorted uniform variables have density fO given by: fO (U(1) , U(2) , . . . , U(n) ) I U(n) 1 i = nn I(U(i) Ui-1 ) I (U1 0)
To determine the constant of the density, notice that there are n! possible orderings of a sample U1 , U2 , . . . , Un . Letting (i )n! denote an enumeration of the possible i=1 permutations of n objects, we have that:
n!
1 =
[0,1]n n!
fU (u1 , . . . , un )du =
i=1 [0,1]n
fU (u1 , . . . , un )I (i (u) is increasing) du
=
i=1 [0,1]n
fU (u1 , . . . , un )I (i (u) is increasing) du fU (u1 , . . . , un )I (1 (u) is increasing) du
[0,1]n
= n!
By noticing that the last integrand equals fO , it follows that: fO (U(1) , U(2) , . . . , U(n) ) = n!I U(n) 1 which establishes the result. (b) From the CLT, we know that: k Now: n+1 Sk k - n+1 n+1 = = n+1 k n+1 k n+1 k
k i=1 (Ei k i=1 (Ei
i = nn I(U(i) Ui-1 ) I (U1 0)
- 1)
k
N (0, 1)
d
- 1)
k
k i=1 (Ei
- 1)
k
and the result follows by applying Slutsky's theorem after noting that: lim k n+1 = = lim n n+1 lim k n
k n, n p
n
k p n
p
(c) The marginal for each component of the random vector follows from item (c) (a little adaptation needed for the second component, but essentially the same argument). The cross term can be proven to be zero by noticing that Sk = k Ei and Sn+1 - Sk = i=1 n+1 i=k+1 Ei involve sums over disjoint sets of independent random variables. It follows that they are independent for all n and k. If each component of a sequence of random 2
vectors is independent from all others and each of them converges in distribution, the corresponding sequence of random vectors converges in distribution to the product of the marginal limiting distributions. This can be proven formally by using characteristic functions (the characteristic function argument due to Ying Xu). (d) From part a, we know that U(pn) = SSk = Sk +(SSk -Sk ) for k such that p = n+1 n+1 consider the mapping h(x1 , x2 ) = x1x1 2 . It follows that: +x U(pn) = h(Sk , Sn+1 - Sk ) The desired result follows from applying the delta method to h(Sk , Sn+1 -Sk ) combined to the result from item c. (e) Letting F denote the cdf of X, we have that X = F -1 (U ), where U has an uniform distribution on [0, 1]. Since F is monotone, we also have X(k) = F -1 (U(k) ). Letting the k p-quantile be given by letting X(pn) = X(k) = F -1 (U(k) ) for p = n , the result follows from the delta method applied to the result from (e) and noticing that: F -1 (x) x = 1
F (x) x d d d d k n.
Now
=
1 f (x)
(f) We first notice that a similar argument to the one used in item (c) above yields: Sk1 p1 0 0 n+1 - p1 d S 2 -S 0 n + 1 kn+1 k1 - p2 + p1 0 p2 - p1 Sn+1 -Sk2 0 0 1 - p2 - (1 - p )
n+1 2
Defining: ~ h(x1 , x2 , x3 ) =
x1 x1 +x2 +x3 x1 +x2 x1 +x2 +x3
we can use the delta method as in item (c) to conclude that: ~ n + 1 h (Sk1 , Sk2 , Sk3 ) - p1 p2 =
d
n+1 0,
U(np1 ) U(np2 )
-
p1 p2
N
d
p1 (1 - p1 ) p1 (1 - p2 ) p1 (1 - p2 ) p2 (1 - p2 )
Finally, noticing that X(np1 ) , X(np2 ) = F -1 (U(np1 ) ), F -1 (U(np2 ) ) and letting: H(x1 , x2 ) = we have that: H(x1 , x2 ) =
1 f (x1 )
F -1 (x1 ) F -1 (x2 ) 0
1 f (x2 )
0
and the result follows from the delta method analogously as in item (e). 3
Problem 8.2 The prior information can be incorporated into the maximum likelihood estimation by adding constraints to the maximization problem. the Thus, restricted maximum likelihood estimate is given by: ^ (X) = arg max log p(X|) 1 2 s.t. 3 3 ^ (X) = arg max X log s.t.
1 3 1- 2 3
+ log(1 - )
If X = 1, log p(x|) = log(), while for X = 0, log p(x|) = log(1 - ). It follows that: 1 3 , if X = 0 ^ (X) = 2 otherwise 3, Now, let (X) = X. The MLE estimate can be rewritten as: 2 1 ^ (X) = (X) + X 3 3 Hence: ^ - (X) = ^ - (X) ^ E - (X)
2
=
2
=
^ R(, (X)) = ^ R(, (X)) = ^ R(, (X)) - R(, (X)) =
1 2 ( - (X)) + ( - X) 3 3 4 2 1 2 (( - (X))) + ( - (X)) ( - X) + (( - X))2 9 3 9 1 4 2 2 E (( - (X))) + E (( - X)) 9 9 4 1 R(, (X)) + E (( - X))2 9 9 (1 - ) 4 R(, (X)) + 9 9 (1 - ) 5 1 24 24 5 - 2 - + = - 2 + - 9 9 4 36 36 36
The quadratic polynomial above have roots 1 and 2 satisfying 1 < 0.30 < 0.70 < 2 , 1 2 ^ so R(, (X)) < R(, (X)) for all [ 3 , 3 ]. Problem 8.3 (a) When x and y are known, we can rescale X and Y so they have unit variance. Hence, we assume without loss of generality that var(X) = var(Y ) = 1. In this case: 1 1 1 1
-1
= 1 - 2 = (1 - 2 )-1 4 1 - - 1
The log-likelihood function for a single observation is given by: 1 L(X, Y |) = - log(2) - 2 1 = - log(2) - 2 1 1 Xi 1 - Xi log(1 - 2 ) - 2) Yi Yi - 1 2 2(1 - 1 1 log(1 - 2 ) - (X 2 + Yi2 ) + (Xi Yi ) 2 2(1 - 2 ) i (1 - 2 )
Now, for the MLE estimate under regularity conditions, we know: with: I() = -E Using Mathematica 5.2, we get: I() = - = 4 + E(Xi2 + Yi2 ) + 32 E(Xi2 + Yi2 ) - 6EXi Yi - 23 EXi Yi - 1 (1 - 2 )3 1 - 4 1 + 2 = (1 - 2 )3 (1 - 2 )2 2 L(X, Y |) 2 n MLE - N 0, (I())-1 ^
d
(b) In this case, the log-likelihood function is given by: 1 1 1 1 2 2 2 2 L(X, Y |, X , Y ) = - log(2) - log(1 - 2 ) - log(X ) - log(Y ) 2 2 2 2 -2 -1 -1 1 Xi X -X Y Xi - -1 -1 -2 Yi -X Y Y 2(1 - 2 ) Yi 1 1 1 1 2 2 = - log(2) - log(1 - 2 ) - log(X ) - log(Y ) 2 2 2 2 Yi2 Yi Xi Xi2 - - - 2 ) 2 2 ) 2 2(1 - X 2(1 - Y (1 - 2 ) 2 2
Y X
Now: 2 2 I(, X , Y ) = -E
2 2 2 L(Xi ,Yi ;,X ,Y ) 2 2 2 2 L(Xi ,Yi ;,X ,Y ) 2 X 2 2 2 L(Xi ,Yi ;,X ,Y ) 2 (X )2 2 2 2 L(Xi ,Yi ;,X ,Y ) 2 Y 2 2 2 L(Xi ,Yi ;,X ,Y ) 2 2 X Y 2 2 2 L(Xi ,Yi ;,X ,Y ) 2 (Y )2
5
with:
2 2 2 L(Xi , Yi ; , X , Y ) 2 2 2 2 L(Xi , Yi ; , X , Y ) 2 (X )2 2 2 2 L(Xi , Yi ; , X , Y ) 2 (Y )2 2 2 2 L(Xi , Yi ; , X , Y ) 2 X 2 2 2 L(Xi , Yi ; , X , Y ) 2 Y 2 2 2 L(Xi , Yi ; , X , Y ) 2 2 X Y
=
1 + 2 (1 - 2 )2 2 - 2 = - 2 4(1 - 2 )2 X = - = - = - = - 2 - 2 2 4(1 - 2 )2 Y
2 2X (1 2 2Y (1
- 2 )
- 2 )
2 2 2 4X Y (1 - 2 )
(c) The verification can be done by multiplying the given matrix to the I() matrix derived in item b and noticing that the result is the identity matrix. The asymptotic variance of each of the components can be taken from the diagonal of [I()]-1 and so:
d 2 4 2 n ^ - X N 0, 2X X d 2 4 ^ 2 n Y - Y N 0, 2Y d n (^ - ) N 0, (1 - 2 )2
(d) We have that the ratio of the asymptotic variances of the estimates of in (a) and (b) is: var(MLE ) var(^) =
(1-2 )2 1+2 (1 - 2 )2
= (1 + 2 )-1 So, unless = 0, the estimator in item (a) has a lower asymptotic variance than the one in (b). Problem 8.4 For a Bernoulli random variable, we have: E X = var X = (1 - ) So, for an i.i.d. sequence of Bernoulli random variables: n(X - ) N (0, (1 - )) 6
d
Applying the delta method with a g function yields: n g(X) - g() N 0, g ()
d 2
(1 - )
So, we want to determine g(.) such that, for some constant K > 0: g () That is: g () = Hence: g() = K Letting w =
1 t, dw = - 2t dt so: 2
(1 - ) = K 2
K (1 - )
1 t(1 - t)
dt
g() = 2K
dw (1 - w2 ) = 2K arcsin( ) + K1
1
Problem 8.5 Itens (a), (b) and (c) follow directly from applying the results from problem 8.1.e and 8.1.f (we use this instead of 8.1.d to account for the fact that the distribution is uniform over [0, 2] and not the unit interval): (a) Noticing that the
1 2
quantile of the Uni[0, 2] equals and that f (x) = N
d
1 2 I(0
x 2)
n X( 1 n) -
2
0,
1 2
1-
1 2
1 2 2
= N 0, 2
(b) The
2 3
3 quantile of the Uni[0, 2] equals 2 so applying the delta method to g(x) = 2 x: 3
n
2 X 2 - 3 ( 3 n)
N
d
0,
82 9
(c) From problem 8.1.f: n
X( 1 n) - 2 4 X( 3 n) - 3 2
4
N
d
0 0
,
32 4 2 4
2 4 32 4
Now, applying the delta method to g(x1 , x2 ) = 1 (x1 + x2 ) yields: 2 n 1 X( 1 n) + X( 3 n) - 4 4 2 7 N
d
0,
2 2
(d) The ARE are given by the ratio of the variances. In the following table, the columns correspond to the denominator and the rows to the numerator of the asymptotic variance ^ ratios. We see that c is the most asymptotically efficient among the three estimators: ^ ^ ^ b c a 9 ^ a 1.0 2.0 8 8 9 ^b 1.0 9 4 1 4 ^ c 1.0
2 9
8