This preview shows pages 1–3. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: EE263 S. Lall 2009.10.21.02 Homework 5 Solutions Due Thursday 10/29 1. Projection matrices. 44020 A matrix P R n n is called a projection matrix if P = P T and P 2 = P . (a) Show that if P is a projection matrix then so is I P . (b) Suppose that the columns of U R n k are orthonormal. Show that UU T is a projection matrix. (Later we will show that the converse is true: every projection matrix can be expressed as UU T for some U with orthonormal columns.) (c) Suppose A R n k is full rank, with k n . Show that A ( A T A ) 1 A T is a projection matrix. (d) If S R n and x R n , the point y in S closest to x is called the projection of x on S . Show that if P is a projection matrix, then y = Px is the projection of x on range( P ). (Which is why such matrices are called projection matrices . . . ) Solution. (a) To show that I P is a projection matrix we need to check two properties: i. I P = ( I P ) T ii. ( I P ) 2 = I P . The first one is easy: ( I P ) T = I P T = I P because P = P T ( P is a projection matrix.) The show the second property we have ( I P ) 2 = I 2 P + P 2 = I 2 P + P (since P = P 2 ) = I P and we are done. (b) Since the columns of U are orthonormal we have U T U = I . Using this fact it is easy to prove that UU T is a projection matrix, i.e. , ( UU T ) T = UU T and ( UU T ) 2 = UU T . Clearly, ( UU T ) T = ( U T ) T U T = UU T and ( UU T ) 2 = ( UU T )( UU T ) = U ( U T U ) U T = UU T (since U T U = I ) . (c) First note that ( A ( A T A ) 1 A T ) T = A ( A T A ) 1 A T because ( A ( A T A ) 1 A T ) T = ( A T ) T ( ( A T A ) 1 ) T A T = A ( ( A T A ) T ) 1 A T = A ( A T A ) 1 A T . Also ( A ( A T A ) 1 A T ) 2 = A ( A T A ) 1 A T because ( A ( A T A ) 1 A T ) 2 = ( A ( A T A ) 1 A T )( A ( A T A ) 1 A T ) = A ( ( A T A ) 1 A T A ) ( A T A ) 1 A T = A ( A T A ) 1 A T (since ( A T A ) 1 A T A = I ) . 1 EE263 S. Lall 2009.10.21.02 (d) To show that Px is the projection of x on range( P ) we verify that the error x Px is orthogonal to any vector in range( P ). Since range( P ) is nothing but the span of the columns of P we only need to show that x Px is orthogonal to the columns of P , or in other words, P T ( x Px ) = 0. But P T ( x Px ) = P ( x Px ) (since P = P T ) = Px P 2 x = (since P 2 = P ) and we are done. 2. Gradient of some common functions. 42090 Recall that the gradient of a differentiable function f : R n R , at a point x R n , is defined as the vector f ( x ) = f x 1 . . . f x n , where the partial derivatives are evaluated at the point x . The first order Taylor approx imation of f , near x , is given by f tay ( z ) = f ( x ) + f ( x ) T ( z x ) ....
View Full
Document
 Fall '08
 BOYD,S

Click to edit the document details