lectr16 - 16 Mean Square Estimation Given some information...

Info icon This preview shows pages 1–7. Sign up to view the full content.

View Full Document Right Arrow Icon
1 16. Mean Square Estimation Given some information that is related to an unknown quantity of interest, the problem is to obtain a good estimate for the unknown in terms of the observed data. Suppose represent a sequence of random variables about whom one set of observations are available, and Y represents an unknown random variable. The problem is to obtain a good estimate for Y in terms of the observations Let represent such an estimate for Y . Note that can be a linear or a nonlinear function of the observation Clearly represents the error in the above estimate, and the square of n X X X , , , 2 1 . , , , 2 1 n X X X ) ( ) , , , ( ˆ 2 1 X X X X Y n ϕ ϕ = = (16-1) ) ( ϕ . , , , 2 1 n X X X ) ( ˆ ) ( X Y Y Y X ϕ ε - = - = (16-2) 2 | | ε PILLAI
Image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
2 the error. Since is a random variable, represents the mean square error. One strategy to obtain a good estimator would be to minimize the mean square error by varying over all possible forms of and this procedure gives rise to the M inimization of the M ean S quare E rror (MMSE) criterion for estimation. Thus under MMSE criterion,the estimator is chosen such that the mean square error is at its minimum. Next we show that the conditional mean of Y given X is the best estimator in the above sense. Theorem1: Under MMSE criterion, the best estimator for the unknown Y in terms of is given by the conditional mean of Y gives X . Thus Proof : Let represent an estimate of Y in terms of Then the error and the mean square error is given by ε } | | { 2 ε E ), ( ϕ ) ( ϕ n X X X , , , 2 1 }. | { ) ( ˆ X Y E X Y = = ϕ (16-3) ) ( ˆ X Y ϕ = ). , , , ( 2 1 n X X X X = , ˆ Y Y - = ε } | ) ( | { } | ˆ | { } | | { 2 2 2 2 X Y E Y Y E E ϕ ε σ ε - = - = = (16-4) PILLAI } | | { 2 ε E
Image of page 2
3 Since we can rewrite (16-4) as where the inner expectation is with respect to Y , and the outer one is with respect to Thus To obtain the best estimator we need to minimize in (16-6) with respect to In (16-6), since and the variable appears only in the integrand term, minimization of the mean square error in (16-6) with respect to is equivalent to minimization of with respect to }] | { [ ] [ X z E E z E z X = }] | ) ( | { [ } | ) ( | { z 2 z 2 2 X X Y E E X Y E Y X n n n n n n n n n n ϕ ϕ σ ε - = - = . X + - - = - = . ) ( } | ) ( | { }] | ) ( | { [ 2 2 2 dx X f X X Y E X X Y E E X ϕ ϕ σ ε (16-6) (16-5) , ϕ 2 ε σ . ϕ , 0 ) ( X f X , 0 } | ) ( | { 2 - X X Y E ϕ ϕ ϕ 2 ε σ } | ) ( | { 2 X X Y E ϕ - . ϕ PILLAI
Image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
4 Since X is fixed at some value, is no longer random, and hence minimization of is equivalent to This gives or But since when is a fixed number Using (16-9) ) ( X ϕ } | ) ( | { 2 X X Y E ϕ - . 0 } | ) ( | { 2 = - X X Y E ϕ ϕ (16-7) 0 } | ) ( {| = - X X Y E ϕ (16-8) ), ( } | ) ( { X X X E ϕ ϕ = (16-9) ) ( , X x X ϕ = ). ( x ϕ PILLAI . 0 } | ) ( { } | { = - X X E X Y E ϕ
Image of page 4
5 in (16-8) we get the desired estimator to be Thus the conditional mean of Y given represents the best estimator for Y that minimizes the mean square error. The minimum value of the mean square error is given by As an example, suppose is the unknown. Then the best MMSE estimator is given by Clearly if then indeed is the best estimator for Y }. , , , | { } | { ) ( ˆ 2 1 n X X X Y E X Y E X Y = = = ϕ n X X X , , , 2 1 . 0 )} | {var( ] } | ) | ( | { [ } | ) | ( | { ) var( 2 2 2 min = - = - = X Y E X X Y E Y E E X Y E Y E X Y n n n n n n n n n n n σ (16-11) (16-10) 3 X Y = , 3 X Y = 3 ˆ X Y = . } | { } | { ˆ 3 3 X X X E X Y E Y = = = (16-12) PILLAI
Image of page 5

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
6 in terms of X . Thus the best estimator can be nonlinear.
Image of page 6
Image of page 7
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern