# lectr16 - 16 Mean Square Estimation Given some information...

This preview shows pages 1–7. Sign up to view the full content.

1 16. Mean Square Estimation Given some information that is related to an unknown quantity of interest, the problem is to obtain a good estimate for the unknown in terms of the observed data. Suppose represent a sequence of random variables about whom one set of observations are available, and Y represents an unknown random variable. The problem is to obtain a good estimate for Y in terms of the observations Let represent such an estimate for Y . Note that can be a linear or a nonlinear function of the observation Clearly represents the error in the above estimate, and the square of n X X X , , , 2 1 . , , , 2 1 n X X X ) ( ) , , , ( ˆ 2 1 X X X X Y n ϕ ϕ = = (16-1) ) ( ϕ . , , , 2 1 n X X X ) ( ˆ ) ( X Y Y Y X ϕ ε - = - = (16-2) 2 | | ε PILLAI

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
2 the error. Since is a random variable, represents the mean square error. One strategy to obtain a good estimator would be to minimize the mean square error by varying over all possible forms of and this procedure gives rise to the M inimization of the M ean S quare E rror (MMSE) criterion for estimation. Thus under MMSE criterion,the estimator is chosen such that the mean square error is at its minimum. Next we show that the conditional mean of Y given X is the best estimator in the above sense. Theorem1: Under MMSE criterion, the best estimator for the unknown Y in terms of is given by the conditional mean of Y gives X . Thus Proof : Let represent an estimate of Y in terms of Then the error and the mean square error is given by ε } | | { 2 ε E ), ( ϕ ) ( ϕ n X X X , , , 2 1 }. | { ) ( ˆ X Y E X Y = = ϕ (16-3) ) ( ˆ X Y ϕ = ). , , , ( 2 1 n X X X X = , ˆ Y Y - = ε } | ) ( | { } | ˆ | { } | | { 2 2 2 2 X Y E Y Y E E ϕ ε σ ε - = - = = (16-4) PILLAI } | | { 2 ε E
3 Since we can rewrite (16-4) as where the inner expectation is with respect to Y , and the outer one is with respect to Thus To obtain the best estimator we need to minimize in (16-6) with respect to In (16-6), since and the variable appears only in the integrand term, minimization of the mean square error in (16-6) with respect to is equivalent to minimization of with respect to }] | { [ ] [ X z E E z E z X = }] | ) ( | { [ } | ) ( | { z 2 z 2 2 X X Y E E X Y E Y X n n n n n n n n n n ϕ ϕ σ ε - = - = . X + - - = - = . ) ( } | ) ( | { }] | ) ( | { [ 2 2 2 dx X f X X Y E X X Y E E X ϕ ϕ σ ε (16-6) (16-5) , ϕ 2 ε σ . ϕ , 0 ) ( X f X , 0 } | ) ( | { 2 - X X Y E ϕ ϕ ϕ 2 ε σ } | ) ( | { 2 X X Y E ϕ - . ϕ PILLAI

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
4 Since X is fixed at some value, is no longer random, and hence minimization of is equivalent to This gives or But since when is a fixed number Using (16-9) ) ( X ϕ } | ) ( | { 2 X X Y E ϕ - . 0 } | ) ( | { 2 = - X X Y E ϕ ϕ (16-7) 0 } | ) ( {| = - X X Y E ϕ (16-8) ), ( } | ) ( { X X X E ϕ ϕ = (16-9) ) ( , X x X ϕ = ). ( x ϕ PILLAI . 0 } | ) ( { } | { = - X X E X Y E ϕ
5 in (16-8) we get the desired estimator to be Thus the conditional mean of Y given represents the best estimator for Y that minimizes the mean square error. The minimum value of the mean square error is given by As an example, suppose is the unknown. Then the best MMSE estimator is given by Clearly if then indeed is the best estimator for Y }. , , , | { } | { ) ( ˆ 2 1 n X X X Y E X Y E X Y = = = ϕ n X X X , , , 2 1 . 0 )} | {var( ] } | ) | ( | { [ } | ) | ( | { ) var( 2 2 2 min = - = - = X Y E X X Y E Y E E X Y E Y E X Y n n n n n n n n n n n σ (16-11) (16-10) 3 X Y = , 3 X Y = 3 ˆ X Y = . } | { } | { ˆ 3 3 X X X E X Y E Y = = = (16-12) PILLAI

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
6 in terms of X . Thus the best estimator can be nonlinear.
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

### What students are saying

• As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

Kiran Temple University Fox School of Business ‘17, Course Hero Intern

• I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

Dana University of Pennsylvania ‘17, Course Hero Intern

• The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

Jill Tulane University ‘16, Course Hero Intern