X \u02c6 X is orthogonal to the subspace spanned by the observations Y 1 Y 2 Y n ie

# X ˆ x is orthogonal to the subspace spanned by the

This preview shows page 65 - 68 out of 138 pages.

X ˆ X is orthogonal to the subspace spanned by the observations Y 1 , Y 2 , . . . , Y n , i.e., E bracketleftbig ( X ˆ X ) Y i bracketrightbig = 0 , i = 1 , 2 , . . . , n , hence E( Y i X ) = E( Y i ˆ X ) = n summationdisplay j =1 h j E( Y i Y j ) , i = 1 , 2 , . . . , n Define the cross covariance of Y and X as the n -vector Σ Y X = E bracketleftbig ( Y E( Y ))( X E( X )) bracketrightbig = σ Y 1 X σ Y 2 X . . . σ Y n X For n = 1 this is simply the covariance The above equations can be written in vector form as Σ Y h = Σ Y X If Σ Y is nonsingular, we can solve the equations to obtain h = Σ 1 Y Σ Y X EE 278B: Random Vectors 4 – 14
Thus, if Σ Y is nonsingular then the best linear MSE estimate is: ˆ X = h T Y = Σ T Y X Σ 1 Y Y Compare this to the scalar case, where ˆ X = Cov( X, Y ) σ 2 Y Y Now to find the minimum MSE, consider MSE = E bracketleftbig ( X ˆ X ) 2 bracketrightbig = E bracketleftbig ( X ˆ X ) X bracketrightbig E bracketleftbig ( X ˆ X ) ˆ X bracketrightbig = E bracketleftbig ( X ˆ X ) X bracketrightbig , since by orthogonality ( X ˆ X ) ˆ X = E( X 2 ) E( ˆ XX ) = Var( X ) E ( Σ T Y X Σ 1 Y Y X ) = Var( X ) Σ T Y X Σ 1 Y Σ Y X Compare this to the scalar case, where minimum MSE is Var( X ) Cov( X, Y ) 2 σ 2 Y If X or Y have nonzero mean, the MMSE affine estimate ˆ X = h 0 + h T Y is determined by first finding the MMSE linear estimate of X E( X ) given Y E( Y ) (minimum MSE for ˆ X and ˆ X are the same), which is ˆ X = Σ T Y X Σ 1 Y ( Y E( Y )) , and then setting ˆ X = ˆ X + E( X ) (since E( ˆ X ) = E( X ) is necessary) EE 278B: Random Vectors 4 – 15 Example Let X be the r.v. representing a signal with mean μ and variance P . The observations are Y i = X + Z i , for i = 1 , 2 , . . . , n , where the Z i are zero mean uncorrelated noise with variance N , and X and Z i are also uncorrelated Find the MMSE linear estimate of X given Y and its MSE For n = 1 , we already know that ˆ X 1 = P P + N Y 1 + N P + N μ To find the MMSE linear estimate for general n , first let X = X μ and Y i = Y i μ . Thus X and Y are zero mean The MMSE linear estimate of X given Y is given by ˆ X n = h T Y , where Σ Y h = Σ Y X , thus P + N P · · · P P P + N · · · P . . . . . . . . . . . . P P · · · P + N h 1 h 2 . . . h n = P P . . . P EE 278B: Random Vectors 4 – 16
By symmetry, h 1 = h 2 = · · · = h n = P nP + N . Thus ˆ X n = P nP + N n summationdisplay i =1 Y i Therefore ˆ X n = P nP + N parenleftbigg n summationdisplay i =1 ( Y i μ ) parenrightbigg + μ = P nP + N parenleftbigg n summationdisplay i =1 Y i parenrightbigg + N nP + N μ The mean square error of the estimate: MSE n = P E( ˆ X n X ) = PN nP + N Thus as n → ∞ , MSE n 0 , i.e., the linear estimate becomes perfect (even though we don’t know the complete statistics of X and Y ) EE 278B: Random Vectors 4 – 17 Linear Innovation Sequence Let X be the signal and Y be the observation vector (all zero mean) Suppose the Y i s are orthogonal , i.e., E( Y i Y j ) = 0

#### You've reached the end of your free preview.

Want to read all 138 pages?

Stuck? We have tutors online 24/7 who can help you get unstuck.
Ask Expert Tutors You can ask You can ask You can ask (will expire )
Answers in as fast as 15 minutes