Stochastic Processes
nctuee07f
Homework 2
Solutions
1. The solution to this problem is put on the ﬁnal page.
2. This problem is very important. It states that, when the jointly Gaussian random vectors
y
and
z
are dependent, conditioning on
z
can always be replaced by conditioning on another
Gaussian random vector ˆ
z
, where ˆ
z
and
y
are statistically independent. We will use this
property later this semester when discussing the recursive minimum mean squared estimator,
or the socalled
Kalman ﬁlter
.
(a) Let
s
,
[
y
T
z
T
]
T
be the (
m
+
r
)
×
1 vector collecting
y
and
z
. In topic 3, we learn that
the conditional mean
E
[
x

y
,
z
] =
E
[
x

s
] =
m
x
+
K
xs
K
s

1
(
s

m
s
)
,
(1)
where
m
x
and
m
s
are the mean vector of
x
and
s
, respectively. In the following, the
notation
m
*
refers to the mean vector of
*
.
We ﬁrst carry out
K
xs
and
K
s

1
. The crosscovariance matrix between
x
and
s
is an
n
×
(
m
+
r
) matrix and can be written in a block matrix form as
K
xs
=
E
h
(
x

m
x
)(
s

m
s
)
T
i
= [
K
xy

K
xz
]
,
where
K
xy
and
K
xz
are the crosscovariance matrix of [
x
and
y
, and [
x
and
z
, respectively.
Since
y
and
z
are independent, their crosscovariance matrix
K
yz
is a zero matrix. There
fore, the covariance matrix of
s
is
K
s
=
E
••
y

m
y
z

m
z
‚
·
[(
y

m
y
)
T
,
(
z

m
z
)
T
]
‚
=
K
y
0
0
K
z
.
It is clear that
K
s

1
=
K
y

1
0
0
K
z

1
.
This can be seen by directly expanding the matrix multiplication
K
s
K
s

1
, and show the
result is an identity matrix. With straightforward manipulations, we ﬁnd
K
xs
K
s

1
(
s

m
s
) = [
K
xy
K
y

1
,
K
xz
K
z

1
]
·
•
y

m
y
z

m
z
‚
=
K
xy
K
y

1
(
y

m
y
) +
K
xz
K
z

1
(
z

m
z
)
=
E
[
x

y
]

m
x
+
E
[
x

z
]

m
x
.
(2)
Substituting (2) into (1) yields
E
[
x

y
,
z
] =
E
[
x

y
] +
E
[
x

z
]

m
x
,
when
y
and
z
are statistically independent.
(b) This part is somewhat involved, and its result is very important, as I mentioned. Intu