X
−
ˆ
X
is orthogonal to the subspace spanned by the observations
Y
1
, Y
2
, . . . , Y
n
, i.e.,
E
bracketleftbig
(
X
−
ˆ
X
)
Y
i
bracketrightbig
= 0
,
i
= 1
,
2
, . . . , n ,
hence
E(
Y
i
X
) = E(
Y
i
ˆ
X
) =
n
summationdisplay
j
=1
h
j
E(
Y
i
Y
j
)
,
i
= 1
,
2
, . . . , n
•
Define the
cross covariance
of
Y
and
X
as the
n
-vector
Σ
Y
X
= E
bracketleftbig
(
Y
−
E(
Y
))(
X
−
E(
X
))
bracketrightbig
=
σ
Y
1
X
σ
Y
2
X
.
.
.
σ
Y
n
X
For
n
= 1
this is simply the covariance
•
The above equations can be written in vector form as
Σ
Y
h
= Σ
Y
X
•
If
Σ
Y
is nonsingular, we can solve the equations to obtain
h
= Σ
−
1
Y
Σ
Y
X
EE 278B: Random Vectors
4 – 14

•
Thus, if
Σ
Y
is nonsingular then the best linear MSE estimate is:
ˆ
X
=
h
T
Y
= Σ
T
Y
X
Σ
−
1
Y
Y
•
Compare this to the scalar case, where
ˆ
X
=
Cov(
X, Y
)
σ
2
Y
Y
•
Now to find the minimum MSE, consider
MSE = E
bracketleftbig
(
X
−
ˆ
X
)
2
bracketrightbig
= E
bracketleftbig
(
X
−
ˆ
X
)
X
bracketrightbig
−
E
bracketleftbig
(
X
−
ˆ
X
)
ˆ
X
bracketrightbig
= E
bracketleftbig
(
X
−
ˆ
X
)
X
bracketrightbig
,
since by orthogonality (
X
−
ˆ
X
)
⊥
ˆ
X
= E(
X
2
)
−
E(
ˆ
XX
)
= Var(
X
)
−
E
(
Σ
T
Y
X
Σ
−
1
Y
Y
X
)
= Var(
X
)
−
Σ
T
Y
X
Σ
−
1
Y
Σ
Y
X
•
Compare this to the scalar case, where minimum
MSE
is
Var(
X
)
−
Cov(
X, Y
)
2
σ
2
Y
•
If
X
or
Y
have nonzero mean, the MMSE affine estimate
ˆ
X
=
h
0
+
h
T
Y
is
determined by first finding the MMSE linear estimate of
X
−
E(
X
)
given
Y
−
E(
Y
)
(minimum MSE for
ˆ
X
′
and
ˆ
X
are the same), which is
ˆ
X
′
= Σ
T
Y
X
Σ
−
1
Y
(
Y
−
E(
Y
))
, and then setting
ˆ
X
=
ˆ
X
′
+ E(
X
)
(since
E(
ˆ
X
) = E(
X
)
is necessary)
EE 278B: Random Vectors
4 – 15
Example
•
Let
X
be the r.v. representing a signal with mean
μ
and variance
P
. The
observations are
Y
i
=
X
+
Z
i
, for
i
= 1
,
2
, . . . , n
, where the
Z
i
are zero mean
uncorrelated noise with variance
N
, and
X
and
Z
i
are also uncorrelated
Find the MMSE linear estimate of
X
given
Y
and its MSE
•
For
n
= 1
, we already know that
ˆ
X
1
=
P
P
+
N
Y
1
+
N
P
+
N
μ
•
To find the MMSE linear estimate for general
n
, first let
X
′
=
X
−
μ
and
Y
′
i
=
Y
i
−
μ
. Thus
X
′
and
Y
′
are zero mean
•
The MMSE linear estimate of
X
′
given
Y
′
is given by
ˆ
X
′
n
=
h
T
Y
′
, where
Σ
Y
h
= Σ
Y
X
,
thus
P
+
N
P
· · ·
P
P
P
+
N
· · ·
P
.
.
.
.
.
.
.
.
.
.
.
.
P
P
· · ·
P
+
N
h
1
h
2
.
.
.
h
n
=
P
P
.
.
.
P
EE 278B: Random Vectors
4 – 16

•
By symmetry,
h
1
=
h
2
=
· · ·
=
h
n
=
P
nP
+
N
. Thus
ˆ
X
′
n
=
P
nP
+
N
n
summationdisplay
i
=1
Y
′
i
Therefore
ˆ
X
n
=
P
nP
+
N
parenleftbigg
n
summationdisplay
i
=1
(
Y
i
−
μ
)
parenrightbigg
+
μ
=
P
nP
+
N
parenleftbigg
n
summationdisplay
i
=1
Y
i
parenrightbigg
+
N
nP
+
N
μ
•
The mean square error of the estimate:
MSE
n
=
P
−
E(
ˆ
X
′
n
X
′
) =
PN
nP
+
N
Thus as
n
→ ∞
,
MSE
n
→
0
, i.e., the linear estimate becomes perfect (even
though we don’t know the complete statistics of
X
and
Y
)
EE 278B: Random Vectors
4 – 17
Linear Innovation Sequence
•
Let
X
be the signal and
Y
be the observation vector (all zero mean)
•
Suppose the
Y
i
s are
orthogonal
, i.e.,
E(
Y
i
Y
j
) = 0

#### You've reached the end of your free preview.

Want to read all 138 pages?

- Spring '14
- 1Staff
- Signal Processing, Probability theory, EE 278B