This preview shows pages 1–3. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: EE 278 Monday, July 27, 2009 Statistical Signal Processing Handout #7 Homework #3 Solutions 1. (20 points) Schwarz inequality. a. Consider the quadratic equation in the parameter a : 0 = E (( X + aY ) 2 ) = E ( X 2 ) + 2 aE ( XY ) + a 2 E ( Y 2 ) . Since it is the expected value of a nonnegative random variable, E (( X + aY ) 2 ) ≥ 0 . If E (( X + aY ) 2 ) > 0 there are two imaginary solutions, while if E (( X + aY ) 2 ) = 0 there is one real solution. Thus the discriminant must satisfy 4( E ( XY )) 2 4 E ( X 2 ) E ( Y 2 ) ≤ , which we can rewrite as E (( XY )) 2 ≤ E ( X 2 ) E ( Y 2 ) . Here is another proof. Let a = ± s E ( X 2 ) E ( Y 2 ) . Plugging these value into E (( X + aY ) 2 ) ≥ 0, we obtain E ( XY ) ≤ p E ( X 2 ) E ( Y 2 ) and E ( XY ) ≤ p E ( X 2 ) E ( Y 2 ) Combining the two inequalities for E ( XY ) yields  E ( XY )  ≤ p E ( X 2 ) E ( Y 2 ) ⇒ E 2 ( XY ) ≤ E ( X 2 ) E ( Y 2 ) which is what we set out to prove. b. If X = cY then E (( cY · Y )) 2 = c 2 ( E ( Y 2 )) 2 = E (( cY ) 2 ) E ( Y 2 ) = E ( X 2 ) E ( Y 2 ) . Conversely, if E 2 ( XY ) = E ( X 2 ) E ( Y 2 ) then the discriminant of the quadratic equation in part (a) is 0. Therefore E (( X + aY ) 2 ) = 0, which means that X + aY = 0 with probability 1. Hence X = aY with probability 1. Clearly, a = ± s E ( X 2 ) E ( Y 2 ) . c. The square of correlation coefficient is by definition ρ 2 X,Y = Cov 2 ( X, Y ) Var( X )Var( Y ) . If we define random variables U = X E ( X ) and V = Y E ( Y ), then ρ 2 X,Y = ( E ( UV )) 2 E ( U 2 ) E ( V 2 ) ≤ 1 , where the inequality follows from the Schwarz Inequality. Therefore  ρ X,Y  ≤ 1 . d. By the Schwarz Inequality, E 2 ( XY ) ≤ E ( X 2 ) E ( Y 2 ) ⇒ E ( XY ) ≤ p E ( X 2 ) E ( Y 2 ) . Therefore E (( X + Y ) 2 ) = E ( X 2 ) + 2 E ( XY ) + E ( Y 2 ) ≤ E ( X 2 ) + 2 p E ( X 2 ) p E ( Y 2 ) + E ( Y 2 ) = p E ( X 2 ) + p E ( Y 2 ) 2 . 2. (20 points) Jensen Inequality. a. For two x points with nonzero p X ( x ), the proof follows by the definition of convexity. Now, suppose that the proof holds for n x points with nonzero p X ( x ). To prove that it holds for n + 1 points, we need to show that g n +1 X i =1 p X ( x i ) x i ! ≤ n +1 X i =1 p X ( x i ) g ( x i ) . Define n X i =1 p X ( x i ) = λ, so p X ( x n +1 ) = 1 λ. Then, n +1 X i =1 p X ( x i ) g ( x i ) = (1 λ ) g ( x n +1 ) + λ n X i =1 p X ( x i ) λ g ( x i ) ≥ (1 λ ) g ( x n +1 ) + λg n X i =1 p X ( x i ) λ x i ! ≥ g (1 λ ) x n +1 + λ n X i =1 p X ( x i ) λ x i ! = g n +1 X i =1 p X ( x i ) x i ! . where the first inequality follows from the induction hypothesis and the second follows from the definition of convexity. b. Once a function g ( X ) has been shown to be convex, applying Jensen’s inequality E ( g ( X )) ≥ g ( E ( X )) gives the inequality relationship....
View
Full
Document
This note was uploaded on 10/26/2011 for the course ECE 180 taught by Professor Eggers during the Spring '11 term at Aarhus Universitet.
 Spring '11
 Eggers
 Signal Processing

Click to edit the document details