soln7_229Aspr07

Soln7_229Aspr07 - EECS 229A Solutions to Homework 7 1 Problem 11.6 on pg 401 of the text Solution Biased estimates may be better Spring 2007 The

Info iconThis preview shows pages 1–3. Sign up to view the full content.

View Full Document Right Arrow Icon
EECS 229A Spring 2007 * * Solutions to Homework 7 1. Problem 11.6 on pg. 401 of the text. Solution : Biased estimates may be better The notation used in this problem is flawed. The estimator in (11.319) based on n samples is not the same as the estimator in (11.318) based on n - 1 samples. We will use the notation ˜ S 2 n for the estimator of (11.319). Note that S n and ˜ S n are estimators of the standard deviation. (a) E [ ¯ X n ] = E [ 1 n n X k =1 X k ] = 1 n E [ X k ] = μ . Thus ¯ X n is an unbiased estimator of the mean. (b) We have E [ S 2 n ] = E [ 1 n n X k =1 ( X k - ¯ X n ) 2 ] = 1 n n X k =1 E [( X k - ¯ X n ) 2 ] ( a ) = E [( X 1 - ¯ X n ) 2 ] = E [( n - 1 n X 1 - 1 n n X l =2 X l ) 2 ] = ( n - 1 n ) 2 σ 2 + ( n - 1) 1 n 2 σ 2 = n - 1 n σ 2 , where step (a) is by symmetry. Further, since ˜ S 2 n = n n - 1 S 2 n , we have E [ ˜ S 2 n ] = σ 2 . Thus S 2 n is a biased estimator of the variance, while ˜ S 2 n is an unbiased estimator of the variance. (c) We have E [( S 2 n - n - 1 n σ 2 ) 2 ] = E [ S 4 n ] - ( n - 1 n ) 2 σ 4 = ( n - 1 n ) 2 E [ ˜ S 4 n ] - ( n - 1 n ) 2 σ 4 = ( n - 1 n ) 2 E [( ˜ S 2 n - σ 2 ) 2 ] . 1
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
This shows that the variance of the biased estimator of variance in (11.318) is strictly smaller than the variance of the unbiased estimator of variance in (11.319). 2. Problem 11.7 on pg. 401 of the text. Solution : Fisher information and relative entropy We write ln f ( x ; θ 0 ) = ln f ( x ; θ ) + ( θ 0 - θ ) ∂θ ln f ( x ; θ ) + 1 2 ( θ 0 - θ ) 2 2 2 θ ln f ( x ; θ ) + o x (( θ 0 - θ ) 2 ) , where, for each fixed x , o x (( θ 0 - θ ) 2 ) is a function that when divided by ( θ 0 - θ ) 2 goes to 0 as θ 0 θ . We next integrate both sides of this equality against f ( x ; θ ) dx . We get Z f ( x ; θ )ln f ( x ; θ 0 ) dx = Z f ( x ; θ )ln f ( x ; θ ) dx + ( θ 0 - θ ) Z f ( x ; θ ) ∂θ ln f ( x ; θ ) dx + 1 2 ( θ 0 - θ ) 2 Z f ( x ; θ ) 2 2 θ ln f ( x ; θ ) dx + o (( θ 0 - θ ) 2 ) , (1) if R f ( x ; θ ) o x (( θ 0 - θ ) 2 ) dx = o (( θ 0 - θ ) 2 ), which we assume. We observe that Z f ( x ; θ ) ∂θ ln f ( x ; θ ) dx = Z f ( x ; θ ) 1 f ( x ; θ ) ∂θ f ( x ; θ ) dx = Z ∂θ f ( x ; θ ) dx ( a ) = ∂θ Z f ( x ; θ ) dx = 0 , where in step (a) we assume that we can interchange the derivative and the integral.
Background image of page 2
Image of page 3
This is the end of the preview. Sign up to access the rest of the document.

This note was uploaded on 10/25/2010 for the course ECE 544 taught by Professor Liu during the Spring '10 term at Ill. Chicago.

Page1 / 7

Soln7_229Aspr07 - EECS 229A Solutions to Homework 7 1 Problem 11.6 on pg 401 of the text Solution Biased estimates may be better Spring 2007 The

This preview shows document pages 1 - 3. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online