Basic Multivariate Normal Theory
[
Prerequisite probability background:
Univariate theory of random variables, expectation, vari
ance, covariance, moment generating function, independence and normal distribution.
Other
requirements:
Basic vectormatrix theory, multivariate calculus, multivariate change of vari
able.]
1
Random Vector
A random vector
U
∈
R
k
is a vector (
U
1
, U
2
,
· · ·
, U
k
)
T
of scalar random variables
U
i
defined
over a common probability space. The expectation and variance of
U
are defined as:
E
U
:=
E
U
1
E
U
2
.
.
.
E
U
k
,
Var
U
:= E(
{
U

E
U
}{
U

E
U
}
T
) =
Var
U
1
Cov(
U
1
, U
2
)
. . .
Cov(
U
1
, U
k
)
Cov(
U
2
, U
1
)
Var
U
2
. . .
Cov(
U
2
, U
k
)
.
.
.
.
.
.
.
.
.
.
.
.
Cov(
U
k
, U
1
)
Cov(
U
k
, U
2
)
. . .
Var
U
k
.
It is easy to check that if
X
=
a
+
BU
where
a
∈
R
p
and
B
is a
p
×
k
matrix, then
X
is
a random vector in
R
p
with E
X
=
a
+
B
E
U
, Var
X
=
B
Var(
U
)
B
T
. You should also note
that if Σ = Var
U
for some random vector
U
, then Σ is a
k
×
k
nonnegative definite matrix,
because for any
a
∈
R
k
,
a
T
Σ
a
= Var(
a
T
U
)
≥
0.
2
Multivariate Normal
Definition
1
.
A random vector
U
∈
R
k
is called a normal random vector if for every
a
∈
R
k
,
a
T
U
is a (one dimensional) normal random variable.
Theorem 1.
A random vector
U
∈
R
k
is a normal random vector if and only if one can
write
U
=
m
+
AZ
for some
m
∈
R
k
and
k
×
k
matrix
A
where
Z
= (
Z
1
,
· · ·
, Z
k
)
T
with
Z
i
IID
∼
Normal
(0
,
1)
.
Proof.
“If part”
. Suppose
U
=
m
+
AZ
with
m
,
A
and
Z
as in the statement of the theorem.
Then for any
a
∈
R
k
,
a
T
U
=
a
T
m
+
a
T
AU
=
b
0
+
n
∑
i
=1
b
i
Z
i
for some scalars
b
0
, b
1
,
· · ·
, b
k
.
But a linear combination of independent (one dimensional)
normal variables is another normal, so
a
T
U
is a normal variable.
“Only if part”
Suppose
U
is a normal random vector. It suffices to show that a
V
=
m
+
AZ
with
Z
as in the statement of the theorem, and suitably chosen
m
and
A
, has the same
distribution as
U
.
For any
a
∈
R
k
, the moment generating function
M
U
(
a
) of
U
at
a
is
1