This preview has intentionally blurred sections. Sign up to view the full version.View Full Document
Unformatted text preview: An important property of conditional expectations is the law of iterated expecta- tions . To state this law, we need to define a slightly different version of conditional expectation. For any function g , define E ( g ( Y ) | X ) = X y g ( y ) f Y | X ( y | X ) when ( X,Y ) is discrete, and E ( g ( Y ) | X ) = Z ∞-∞ g ( y ) f Y | X ( y | X )d y when ( X,Y ) is continuous. This is almost the same as our definition of E ( g ( Y ) | X = x ) above, but we have left X as a random quantity, rather than specifying that we know it is equal to some fixed value x . This makes E ( g ( Y ) | X ) a random quantity. Think of E ( g ( Y ) | X ) as your prediction of g ( Y ) in terms of X . Different realizations of X will give you a different prediction of g ( Y ), so your prediction E ( g ( Y ) | X ) must itself be random, but in a way that depends only on X . Since E ( g ( Y ) | X ) is random, we can consider taking its expected value: E ( E ( g ( Y ) | X )). The law of iterated expectations states that E ( E ( g ( Y ) | X )) = E ( g ( Y )) . To see why the law of iterated expectations is true, suppose that the pair ( X,Y ) is continuous with joint pdf f XY . The conditional expectation E ( g ( Y ) | X ) 8 is a random quantity depending on X- in other words, it is a function of X . Therefore, its expected value is given by E ( E ( g ( Y ) | X )) = Z ∞-∞ E ( g ( Y ) | x ) f X ( x )d x. Plugging in our definition for E ( g ( Y ) | X ), we obtain E ( E ( g ( Y ) | X )) = Z ∞-∞ Z ∞-∞ g ( y ) f Y | X ( y | x )d y f X ( x )d x = Z ∞-∞ Z ∞-∞ g ( y ) f Y | X ( y | x ) f X ( x )d x d y. The product of the conditional density f Y | X ( y | x ) and the marginal density f X ( x ) is the joint density f XY ( x,y ), so we now have E ( E ( g ( Y ) | X )) = Z ∞-∞ Z ∞-∞ g ( y ) f XY ( x,y )d x d y = Z ∞-∞ g ( y ) Z ∞-∞ f XY ( x,y )d x d y = Z ∞-∞ g ( y ) f Y ( y )d y = E ( g ( Y )) . A similar argument can be used to justify the law of iterated expectations when the pair ( X,Y ) is discrete. 8 Convergence in probability Suppose we have an infinite sequence of random variables Z 1 ,Z 2 ,Z 3 ,... . Imagine that, as we move along this sequence, the random variables Z n start to settle down to some constant value. For instance, it might be the case that the n th variable in our sequence, Z n , has the uniform distribution on (1- 1 /n, 1 + 1 /n ). In this case, as n increases, the distribution of Z n becomes more and more tightly concentrated around one. In some sense, Z n converges to one as n → ∞ . But since each Z n is random, we need to find a suitable way to define this convergence that involves a statement about the random behavior of the different Z n ’s....
View Full Document
- Spring '08
- Normal Distribution, Probability theory, probability density function