When the pair (
X, Y
) is continuous, the same equation defines the
conditional
probability density function for
Y
given
X
. Note that if
X
and
Y
are independent,
then the factorization
f
XY
(
x, y
) =
f
X
(
x
)
f
Y
(
y
) implies that
f
Y

X
(
y

x
) =
f
Y
(
y
), so
that the conditional probability mass or density function for
Y
given
X
is just the
marginal probability mass or density function for
Y
.
Conditional expectations may easily be defined using conditional probability
density or mass functions. For any function
g
, we define
E
(
g
(
Y
)

X
=
x
) =
X
y
g
(
y
)
f
Y

X
(
y

x
)
when (
X, Y
) is discrete, and
E
(
g
(
Y
)

X
=
x
) =
Z
∞
∞
g
(
y
)
f
Y

X
(
y

x
)d
y
when (
X, Y
) is continuous. Letting
g
(
y
) =
y
, we obtain expressions for the simple
case
E
(
Y

X
=
x
).
When
X
and
Y
are independent,
f
Y

X
(
y

x
) =
f
Y
(
y
), and
so there is no difference between conditional and unconditional expectation – we
simply have
E
(
g
(
Y
)

X
=
x
) =
E
(
g
(
Y
)) for any function
g
.
An important property of conditional expectations is the
law of iterated expecta
tions
. To state this law, we need to define a slightly different version of conditional
expectation. For any function
g
, define
E
(
g
(
Y
)

X
) =
X
y
g
(
y
)
f
Y

X
(
y

X
)
when (
X, Y
) is discrete, and
E
(
g
(
Y
)

X
) =
Z
∞
∞
g
(
y
)
f
Y

X
(
y

X
)d
y
when (
X, Y
) is continuous. This is almost the same as our definition of
E
(
g
(
Y
)

X
=
x
) above, but we have left
X
as a random quantity, rather than specifying that
we know it is equal to some fixed value
x
.
This makes
E
(
g
(
Y
)

X
) a random
quantity. Think of
E
(
g
(
Y
)

X
) as your prediction of
g
(
Y
) in terms of
X
. Different
realizations of
X
will give you a different prediction of
g
(
Y
), so your prediction
E
(
g
(
Y
)

X
) must itself be random, but in a way that depends only on
X
. Since
E
(
g
(
Y
)

X
) is random, we can consider taking its expected value:
E
(
E
(
g
(
Y
)

X
)).
The law of iterated expectations states that
E
(
E
(
g
(
Y
)

X
)) =
E
(
g
(
Y
))
.
To see why the law of iterated expectations is true, suppose that the pair
(
X, Y
) is continuous with joint pdf
f
XY
. The conditional expectation
E
(
g
(
Y
)

X
)
8