soln7_229Aspr07

# Soln7_229Aspr07 - EECS 229A Solutions to Homework 7 1 Problem 11.6 on pg 401 of the text Solution Biased estimates may be better Spring 2007 The

This preview shows pages 1–3. Sign up to view the full content.

EECS 229A Spring 2007 * * Solutions to Homework 7 1. Problem 11.6 on pg. 401 of the text. Solution : Biased estimates may be better The notation used in this problem is ﬂawed. The estimator in (11.319) based on n samples is not the same as the estimator in (11.318) based on n - 1 samples. We will use the notation ˜ S 2 n for the estimator of (11.319). Note that S n and ˜ S n are estimators of the standard deviation. (a) E [ ¯ X n ] = E [ 1 n n X k =1 X k ] = 1 n E [ X k ] = μ . Thus ¯ X n is an unbiased estimator of the mean. (b) We have E [ S 2 n ] = E [ 1 n n X k =1 ( X k - ¯ X n ) 2 ] = 1 n n X k =1 E [( X k - ¯ X n ) 2 ] ( a ) = E [( X 1 - ¯ X n ) 2 ] = E [( n - 1 n X 1 - 1 n n X l =2 X l ) 2 ] = ( n - 1 n ) 2 σ 2 + ( n - 1) 1 n 2 σ 2 = n - 1 n σ 2 , where step (a) is by symmetry. Further, since ˜ S 2 n = n n - 1 S 2 n , we have E [ ˜ S 2 n ] = σ 2 . Thus S 2 n is a biased estimator of the variance, while ˜ S 2 n is an unbiased estimator of the variance. (c) We have E [( S 2 n - n - 1 n σ 2 ) 2 ] = E [ S 4 n ] - ( n - 1 n ) 2 σ 4 = ( n - 1 n ) 2 E [ ˜ S 4 n ] - ( n - 1 n ) 2 σ 4 = ( n - 1 n ) 2 E [( ˜ S 2 n - σ 2 ) 2 ] . 1

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This shows that the variance of the biased estimator of variance in (11.318) is strictly smaller than the variance of the unbiased estimator of variance in (11.319). 2. Problem 11.7 on pg. 401 of the text. Solution : Fisher information and relative entropy We write ln f ( x ; θ 0 ) = ln f ( x ; θ ) + ( θ 0 - θ ) ∂θ ln f ( x ; θ ) + 1 2 ( θ 0 - θ ) 2 2 2 θ ln f ( x ; θ ) + o x (( θ 0 - θ ) 2 ) , where, for each ﬁxed x , o x (( θ 0 - θ ) 2 ) is a function that when divided by ( θ 0 - θ ) 2 goes to 0 as θ 0 θ . We next integrate both sides of this equality against f ( x ; θ ) dx . We get Z f ( x ; θ )ln f ( x ; θ 0 ) dx = Z f ( x ; θ )ln f ( x ; θ ) dx + ( θ 0 - θ ) Z f ( x ; θ ) ∂θ ln f ( x ; θ ) dx + 1 2 ( θ 0 - θ ) 2 Z f ( x ; θ ) 2 2 θ ln f ( x ; θ ) dx + o (( θ 0 - θ ) 2 ) , (1) if R f ( x ; θ ) o x (( θ 0 - θ ) 2 ) dx = o (( θ 0 - θ ) 2 ), which we assume. We observe that Z f ( x ; θ ) ∂θ ln f ( x ; θ ) dx = Z f ( x ; θ ) 1 f ( x ; θ ) ∂θ f ( x ; θ ) dx = Z ∂θ f ( x ; θ ) dx ( a ) = ∂θ Z f ( x ; θ ) dx = 0 , where in step (a) we assume that we can interchange the derivative and the integral.
This is the end of the preview. Sign up to access the rest of the document.

## This note was uploaded on 10/25/2010 for the course ECE 544 taught by Professor Liu during the Spring '10 term at Ill. Chicago.

### Page1 / 7

Soln7_229Aspr07 - EECS 229A Solutions to Homework 7 1 Problem 11.6 on pg 401 of the text Solution Biased estimates may be better Spring 2007 The

This preview shows document pages 1 - 3. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online