session5

session5 - EE278: Review Session # 5 Han-I Su May 4, 2009...

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: EE278: Review Session # 5 Han-I Su May 4, 2009 Han-I Su () EE278: Review Session # 5 May 4, 2009 1 / 12 Outline 1 Basic Probability Detection and Estimation MMSE Linear Estimate Functions of random variables Inequalities Homework Hints 2 3 4 5 6 Han-I Su () EE278: Review Session # 5 May 4, 2009 2 / 12 Basic Probability Some properties of events, discrete random variables (PMF), continuous random varialbes (PDF), and mixed random variables are similar. Law of total probability Events: P(B ) = i P(Ai ∩ B ) if Ai ’s partition Ω PMF: pX (x ) = y pXY (x , y ) PDF: fX (x ) = fXY (x , y )dy Mixed: fY (y ) = θ pΘ (θ )fY |Θ (y |θ ) fY (y )pΘ|Y (θ |y )dy pΘ (θ ) = Han-I Su () EE278: Review Session # 5 May 4, 2009 3 / 12 Bayes rule Event: P(Aj |B ) = Mixed: P(B |Aj ) P(Aj ) if Ai ’s partition Ω i P(B |Ai ) P(Ai ) fY |Θ (y |θ ) pΘ (θ ) fY |Θ (y |θ )pΘ (θ ) pΘ|Y (θ |y ) = θ fY |Θ (y |θ ) = pΘ|Y (θ |y ) fY (y ) pΘ|Y (θ |y )fY (y )dy The denominator is obtained from the law of total probability An explanation: pΘ|Y (θ |y ) is a PMF over Θ, which is proportional to fY |Θ (y |θ )pΘ (θ ) with normalization constant fY (y ) Han-I Su () EE278: Review Session # 5 May 4, 2009 4 / 12 Detection and Estimation Channel pY |X or fY |X Y Decoder/ Estimator X ˆ X (Y ) Detection X is discrete Given Y = y , find decision rule D (y ) to minimize the probability of error MSE Estimation X can be discrete or continuous ˆ Given Y = y , find estimate X (y ) to minimize MSE Han-I Su () EE278: Review Session # 5 May 4, 2009 5 / 12 Example (discrete X ) X and Z are independent, and Y = X + Z . pX (x ) = 1/3 for x = x0 , x1, x2 and fZ (z ) = (1/2)e −|z | fY |X (y |x )pX (x ) x1 x0 y x2 Plotting fY |X (y |x )pX (x ) helps finding decision rules or computing estimates In general the signal distribution pX (x ) and the channel fY |X (y |x ) are given or easier to be found The given Y = y gives us three intersections fY |X (y |x0)pX (x0 ), fY |X (y |x1)pX (x1 ), and fY |X (y |x2)pX (x2 ) Han-I Su () EE278: Review Session # 5 May 4, 2009 6 / 12 Detection Given Y = y , the probability of sending xi is proportional to fY |X (y |xi )pX (xi ) Since fY |X (y |x2 )pX (x2 ) is greater than the other two, D (y ) = x2 MSE Estimation The MMSE estimate is E(X |Y ) and the MMSE is E(Var(X |Y )) Han-I Su () EE278: Review Session # 5 May 4, 2009 7 / 12 Detection Given Y = y , the probability of sending xi is proportional to fY |X (y |xi )pX (xi ) Since fY |X (y |x2 )pX (x2 ) is greater than the other two, D (y ) = x2 MSE Estimation The MMSE estimate is E(X |Y ) and the MMSE is E(Var(X |Y )) How do we compute E(X |Y )? Han-I Su () EE278: Review Session # 5 May 4, 2009 7 / 12 Detection Given Y = y , the probability of sending xi is proportional to fY |X (y |xi )pX (xi ) Since fY |X (y |x2 )pX (x2 ) is greater than the other two, D (y ) = x2 MSE Estimation The MMSE estimate is E(X |Y ) and the MMSE is E(Var(X |Y )) How do we compute E(X |Y )? 1 By definition of conditional expectation, we need to find pX |Y (x |y ), which is a PMF for X Han-I Su () EE278: Review Session # 5 May 4, 2009 7 / 12 Detection Given Y = y , the probability of sending xi is proportional to fY |X (y |xi )pX (xi ) Since fY |X (y |x2 )pX (x2 ) is greater than the other two, D (y ) = x2 MSE Estimation The MMSE estimate is E(X |Y ) and the MMSE is E(Var(X |Y )) How do we compute E(X |Y )? 1 2 By definition of conditional expectation, we need to find pX |Y (x |y ), which is a PMF for X Since fY |X (y |x ) is easier to find, we need the Bayes rule. Recall that pX |Y (xi |y ) is proportional to fY |X (y |xi )pX (xi ) with normalization Han-I Su () EE278: Review Session # 5 May 4, 2009 7 / 12 Detection Given Y = y , the probability of sending xi is proportional to fY |X (y |xi )pX (xi ) Since fY |X (y |x2 )pX (x2 ) is greater than the other two, D (y ) = x2 MSE Estimation The MMSE estimate is E(X |Y ) and the MMSE is E(Var(X |Y )) How do we compute E(X |Y )? 1 2 3 By definition of conditional expectation, we need to find pX |Y (x |y ), which is a PMF for X Since fY |X (y |x ) is easier to find, we need the Bayes rule. Recall that pX |Y (xi |y ) is proportional to fY |X (y |xi )pX (xi ) with normalization Fix some Y = y at which we know explicitly fY |X (y |xi )pX (xi ). In the example, z = y − x > 0 and z < 0 correspond to (1/2)e −(y −x ) and (1/2)e y −x Han-I Su () EE278: Review Session # 5 May 4, 2009 7 / 12 Detection Given Y = y , the probability of sending xi is proportional to fY |X (y |xi )pX (xi ) Since fY |X (y |x2 )pX (x2 ) is greater than the other two, D (y ) = x2 MSE Estimation The MMSE estimate is E(X |Y ) and the MMSE is E(Var(X |Y )) How do we compute E(X |Y )? 1 2 3 4 By definition of conditional expectation, we need to find pX |Y (x |y ), which is a PMF for X Since fY |X (y |x ) is easier to find, we need the Bayes rule. Recall that pX |Y (xi |y ) is proportional to fY |X (y |xi )pX (xi ) with normalization Fix some Y = y at which we know explicitly fY |X (y |xi )pX (xi ). In the example, z = y − x > 0 and z < 0 correspond to (1/2)e −(y −x ) and (1/2)e y −x If fY |X (y |x0 )pX (x0 ) = 0.2, fY |X (y |x1 )pX (x1 ) = 0.1, fY |X (y |x2 )pX (x2 ) = 0.5, then E(X |Y ) = 0.2x0 + 0.1x1 + 0.5x2 0.2 + 0.1 + 0.5 May 4, 2009 7 / 12 Han-I Su () EE278: Review Session # 5 MMSE Linear Estimate The MMSE linear estimate of X given Y is ˆ = Cov(X , Y ) (Y − E(Y )) + E (X ) X 2 σY and its MSE is 2 MSE = (1 − ρ2 )σX XY We only need to know the first and second moments of X and Y . In general, computing moments is easier than finding distributions Han-I Su () EE278: Review Session # 5 May 4, 2009 8 / 12 Functions of Random Variables The distribution of X is given and Y = g (X ) Discrete: pX (x ) pY (y ) = x :g (x )=y Continuous: f Y (y ) = x :g (x )=y f X (x ) | g (x )| CDF: FY (y ) = P{Y ≤ y } = P{X : g (X ) ≤ y } Note that pY (y ) and fY (y ) should be in terms of y instead of x Application: Y = FX (x ) is U[0, 1] Han-I Su () EE278: Review Session # 5 May 4, 2009 9 / 12 Inequalities Union of events: P( i =1 n n Ai ) ≤ i =1 P(Ai ) Caichy-Schwarz: (E(XY ))2 ≤ E(X 2 ) E(Y 2 ); equality iff X = aY Jensen: if g (x ) is convex, then E(g (X )) ≥ g (E(X )) Markov: if X ≥ 0 and a > 1, then 1 P{X ≥ a E(X )} ≤ a Chebyshev: 1 P{|X − E (X )| ≥ aσX } ≤ 2 a EE278: Review Session # 5 May 4, 2009 10 / 12 Han-I Su () Homework Hints Problem 1: Uncorrelated: E (XZ ) = E (X )E (Z ) Problem 2: Since we know the distribution of Z |X = x , compute the expectations associated with Z via iterative expectations. For example, E(XZ ) = EX [E(XZ |X )] = EX [X E(Z |X )] Problem 3: The linear MMSE is an upper bound on MMSE. The bound is tight if the MMSE estimate is the same as MMSE linear estimate. Han-I Su () EE278: Review Session # 5 May 4, 2009 11 / 12 Problem 4: Let Ai = {i < X ≤ i + 1} for i = −k , . . . , k − 1. Then k −1 k −1 ˜ E(X X ) = i =−k ˜ E(X X |Ai ) P(Ai ) = i =−k E X 1 i+ 2 Ai P(Ai ) You will also need k=1 i 2 = k (k + 1)(2k + 1)/6 i Problem 5: E (X |Y ) is Gaussian, so the distribution is determined by the mean and variance. E (X |Y ) is also the MMSE linear estimate aY + b, and thus we can find the mean and variance For part b, E(Y 2 |X = x ) = (E(Y |X = x ))2 + Var(Y |X = x ) Han-I Su () EE278: Review Session # 5 May 4, 2009 12 / 12 ...
View Full Document

This note was uploaded on 04/07/2010 for the course EE 278 taught by Professor Balajiprabhakar during the Spring '09 term at Stanford.

Ask a homework question - tutors are online