302, Stat Introduction to Probability
Jiahua Chen
January-April 2011
Jiahua Chen ()
Lecture 16
January-April 2011
1 / 23
Conditional distribution: continuous random variables
Consider the case where X and Y have joint density function f (x , y ). Similar to the discrete case, we may attempt to compute P ( Y = y |X = x ) = P (X = x , Y = y ) . P (X = x )
Yet this is not feasible because P (X = x ) = 0. However, the notation of conditional distribution is just as desirable.
Jiahua Chen ()
Lecture 16
January-April 2011
2 / 23
Continuous random variables
Suppose fX (x ) > 0 in a neighborhood of x = a. Let a = [a, a + ], b = [b , b + ] for some > 0. It is seen that P (X a ) fX (a) > 0. Similarly, P (X a , Y b ) f (a, b )2 .
Jiahua Chen ()
Lecture 16
January-April 2011
3 / 23
Continuous random variables
Hence,
P ( Y b |X a ) f (a, b ) = fX ( a )
which is is well dened. We hence dene fY | X ( y | x ) = f (x , y ) fX ( x )
as the conditional probability density function of Y given X = x . You may notice that this expression is very close to the expression of the conditional pmf in discrete case.
Jiahua Chen ()
Lecture 16
January-April 2011
4 / 23
Example
Suppose the joint pdf of X and Y is given by f (xy ) = for 0 < x < 1 and 0 < y < 1. What is the conditional pdf of X given Y = y ? What is the conditional pdf of Y given X = x ? 12 x (2 x y ) 5
Jiahua Chen ()
Lecture 16
January-April 2011
5 / 23
Example
Recall the general form of conditional pdfs f (x , y )/fX (x ) and f (x , y )/FY (y ) we need only nd the marginal pdfs to answer these two questions. What is the conditional pdf of X given Y = y ?
The marginal pdf of Y is fY (y ) =
1 x =0
2 12 x (2 x y )dx = (4 3y ). 5 5
Hence, when x (0, 1), the conditional pdf of y fY |X (y |x ) = for 0 < y < 1. 6x ( 2 x y ) 4 3y
Jiahua Chen ()
Lecture 16
January-April 2011
6 / 23
Example
Recall the general form of conditional pdfs f (x , y )/fX (x ) and f (x , y )/FY (y ) we need only nd the marginal pdfs to answer these two questions. What is the conditional pdf of X given Y = y ?
The marginal pdf of Y is fY (y ) =
1 x =0
2 12 x (2 x y )dx = (4 3y ). 5 5
Hence, when x (0, 1), the conditional pdf of y fY |X (y |x ) = 6x ( 2 x y ) 4 3y
for 0 < y < 1. The conditional pdf of Y given, for instance, X = 1 is not dened. The conditional pdf of Y at y (0, 1) equals 0.
Jiahua Chen () Lecture 16 January-April 2011 6 / 23
Example
What is the conditional pdf of Y given X = x ?
The marginal pdf of X is fX (x ) =
1 y =0
12 6 x (2 x y )dy = (3x x 2 ). 5 5
Hence, for 0 < y < 1 given any X = x (0, 1), fY |X (y |x ) = 2(2 x y ) 2x ( 2 x y ) = . 3 2x 3x 2x 2
Jiahua Chen ()
Lecture 16
January-April 2011
7 / 23
Example: Bivariate normal
Two random variables X and Y have bivariate normal distribution if their joint pdf is given by f (x , y ) = where g (x , y ) = (x x )(y y ) (y y )2 (x x )2 2 + . 2 2 x x y y 1 2x y 1 2 exp{ 1 g (x , y ) } 2( 1 2 )
Note x , y are variables in the joint pdf, while x , y are means, x , y are standard deviations of X and Y , plus is the correlation coecient. Because of above, x , y are positive, and [1, 1]. Simply, the density function is given by exp(g (x , y )) where g (x , y ) is a positive denite quadratic form in x , y .
Jiahua Chen () Lecture 16 January-April 2011 8 / 23
Bivariate normal: marginal distribution
Apparently, the marginal distributions of X and Y are both normal. By complete the square, we nd g (x , y ) = (x x )(y y ) (y y )2 (x x )2 2 + 2 2 x x y y
=
(x x ) (y y ) x y
2
+ ( 1 2 )
(y y )2 2 y
Jiahua Chen ()
Lecture 16
January-April 2011
9 / 23
Bivariate normal: marginal distribution
The marginal pdf of Y is hence given by fY ( y ) = C exp{ exp{ 1 g (x , y )}dx 2( 1 2 )
=C
(x x ) (y y ) 1 2) 2( 1 x y 2 (y y ) exp{ } 2 2y (y y )2 } 2 2y
2
}dx
= C exp{
Note C depends parameter values only and its value changes from one line to another. However, its exact value is not important in our computation.
Jiahua Chen () Lecture 16 January-April 2011 10 / 23
Bivariate normal: marginal distribution
It is seen that the pdf of Y is proportional to exp{ which has to be
(y y )2 } 2 2y
2 or N (y , y ).
(y y )2 1 exp{ } 2 2y 2y
Jiahua Chen ()
Lecture 16
January-April 2011
11 / 23
Bivariate normal: marginal and conditional distribution
2 Similarly, the marginal distribution of X is N (x , x ).
Before we give the conditional pdf of X given Y = y , have a look again: f (x , y ) = where g (x , y ) = (x x )(y y ) (y y )2 (x x )2 2 + . 2 2 x x y y 1 2x y 1 2 exp{ 1 g (x , y ) 2( } 1 2 )
Jiahua Chen ()
Lecture 16
January-April 2011
12 / 23
Bivariate normal: marginal and conditional distribution
and that
(x x ) (y y ) g (x , y ) = x y
2
+ ( 1 2 )
(y y )2 . 2 y
We nd the conditional pdf of X given Y = y is fX |Y (x |y ) = C exp
(x x ) (y y ) 1 2) 2( 1 x y
1
2 2x (1 2 )
2
= C exp
with
( x x |y ) 2
x |y = x +
Jiahua Chen ()
x (y y ). y
January-April 2011 13 / 23
Lecture 16
Bivariate normal: marginal and conditional distribution
The form fX |Y (x |y ) = C exp 1
2 2x (1 2 )
( x x |y ) 2
implies that X |Y = y is normally distributed with conditional mean x |y = x + and conditional variance
2 2 x |y = x (1 2 ). 2 This is a reduction from x .
x (y y ). y
Jiahua Chen ()
Lecture 16
January-April 2011
14 / 23
Bivariate normal: marginal and conditional distribution
Having conditional variance
2 2 x |y = x (1 2 ).
implies that knowing the value of Y is helpful to predict the observed value of X .
Jiahua Chen ()
Lecture 16
January-April 2011
15 / 23
Conditional mean and conditional variance
When X and Y are discrete, the conditional pmf of Y given X = x is given by p (x , y ) P ( Y = y |X = x ) = = pY | X ( y | x ) . pX (x ) When X and Y are continuous, the conditional pdf of Y given X = x is given by f (x , y ) = fY | X ( y | x ) . fX ( x ) Note only they are called pmf and pdf, they are indeed pmf and pdf (as function of y ).
Jiahua Chen ()
Lecture 16
January-April 2011
16 / 23
Conditional mean and conditional variance
To avoid confusion, I use X = a instead of X = x in the following. For discrete r.v.s, we have (1) 1 P (Y = y |X = a) 0 for all y . (2) y P (Y = y |X = a) = 1. For continuous r.v.s, we have f ( a ,y ) (1) fY |X (y |a) = f (a) 0 for all y . X (2) y fY |X (y |a)dy = 1. Both indicate that the conditional distribution is also distribution.
Jiahua Chen ()
Lecture 16
January-April 2011
17 / 23
Conditional mean and conditional variance
We may compute various moments of the conditional distribution: For discrete one, we have E [g ( Y ) |X = a ] = For continuous one, we have
g (y )P (Y
y
= y |X = a ) .
E [g ( Y ) |X = a ] =
g (y )fY |X (y |a)dy .
The conditional expectation of g (Y ) depends on the specic value a we choose for X .
We usually use x instead of a for a potential value of X .
Jiahua Chen ()
Lecture 16
January-April 2011
18 / 23
Example
Let X1 and X2 be the arrival times of rst two students for my oce hour. Assume that X1 has exponential distribution with pdf ( x > 0) f1 (x ) = exp(x ) Assume that given X1 = a, the pdf of X2 is given by f2|1 (x |X1 = a) = exp((x a)). for x > a. What is their joint pdf?
Jiahua Chen ()
Lecture 16
January-April 2011
19 / 23
Example
The joint pdf is given by f ( x1 , x2 ) = f 2 | 1 ( x2 | x1 ) f 1 ( x1 ) and we must keep close track of these 1s and 2s. The answer is f (x1 , x2 ) = exp((x2 x1 )) exp(x1 )
= 2 exp(x2 )
for > x2 > x1 > 0.
The range is crucial in this computation.
What is the conditional pdf of X1 given X2 = b ?
Jiahua Chen ()
Lecture 16
January-April 2011
20 / 23
Example
What is the conditional pdf of X1 given X2 = b ? Let us nd the marginal pdf of X2 : for any b > 0,
b
f2 ( b ) =
f (x1 , b )dx1 =
0
2 exp(b )dx1 = 2 b exp(b )
Do you know the name of this distribution?
The conditional pdf of X1 given X2 = b is hence f 1 | 2 ( x1 | b ) = for 0 < x1 < b . 2 exp(b ) 1 = 2 b exp( b ) b
Jiahua Chen ()
Lecture 16
January-April 2011
21 / 23
Example
Keeping tracking the range is hard. You may go as follows: f 1 | 2 ( x1 | b ) = 2 exp(b )I (0 < x1 < b ) 1 = I ( 0 < x1 < b ) . 2 b exp( b )I (0 < x < b ) b 1
Note that this is a function of x1 , and b is regarded as a number.
Jiahua Chen ()
Lecture 16
January-April 2011
22 / 23
Example
Keeping tracking the range is hard. You may go as follows: f 1 | 2 ( x1 | b ) = 2 exp(b )I (0 < x1 < b ) 1 = I ( 0 < x1 < b ) . 2 b exp( b )I (0 < x < b ) b 1
Note that this is a function of x1 , and b is regarded as a number.
Does it help to use b instead of x2 here?
For instance, if b = 20mins , then f1|2 (x |20) = which is uniform on [0, 20]. What is the conditional expectation and variance of X1 given X2 = 20?
Jiahua Chen () Lecture 16 January-April 2011 22 / 23
1 I (0 < x < 20) 20
Example
Given X2 = 20 mins, then f1|2 (x |20) = Hence E (X1 |X2 = 20) =
2 E (X1 |X2 = 20) =
1 I (0 < x < 20). 20 1 20, 2 1 ?? = 202 . 3 ?? =
Therefore, var(X1 |X2 = 20) =
1 12
Replace 20 by x2 yourself to repeat the computation/derivation.
202 .
Jiahua Chen ()
Lecture 16
January-April 2011
23 / 23