This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: EE263 S. Lall 2009.10.21.02 Homework 5 Solutions Due Thursday 10/29 1. Projection matrices. 44020 A matrix P ∈ R n × n is called a projection matrix if P = P T and P 2 = P . (a) Show that if P is a projection matrix then so is I − P . (b) Suppose that the columns of U ∈ R n × k are orthonormal. Show that UU T is a projection matrix. (Later we will show that the converse is true: every projection matrix can be expressed as UU T for some U with orthonormal columns.) (c) Suppose A ∈ R n × k is full rank, with k ≤ n . Show that A ( A T A ) − 1 A T is a projection matrix. (d) If S ⊆ R n and x ∈ R n , the point y in S closest to x is called the projection of x on S . Show that if P is a projection matrix, then y = Px is the projection of x on range( P ). (Which is why such matrices are called projection matrices . . . ) Solution. (a) To show that I − P is a projection matrix we need to check two properties: i. I − P = ( I − P ) T ii. ( I − P ) 2 = I − P . The first one is easy: ( I − P ) T = I − P T = I − P because P = P T ( P is a projection matrix.) The show the second property we have ( I − P ) 2 = I − 2 P + P 2 = I − 2 P + P (since P = P 2 ) = I − P and we are done. (b) Since the columns of U are orthonormal we have U T U = I . Using this fact it is easy to prove that UU T is a projection matrix, i.e. , ( UU T ) T = UU T and ( UU T ) 2 = UU T . Clearly, ( UU T ) T = ( U T ) T U T = UU T and ( UU T ) 2 = ( UU T )( UU T ) = U ( U T U ) U T = UU T (since U T U = I ) . (c) First note that ( A ( A T A ) − 1 A T ) T = A ( A T A ) − 1 A T because ( A ( A T A ) − 1 A T ) T = ( A T ) T ( ( A T A ) − 1 ) T A T = A ( ( A T A ) T ) − 1 A T = A ( A T A ) − 1 A T . Also ( A ( A T A ) − 1 A T ) 2 = A ( A T A ) − 1 A T because ( A ( A T A ) − 1 A T ) 2 = ( A ( A T A ) − 1 A T )( A ( A T A ) − 1 A T ) = A ( ( A T A ) − 1 A T A ) ( A T A ) − 1 A T = A ( A T A ) − 1 A T (since ( A T A ) − 1 A T A = I ) . 1 EE263 S. Lall 2009.10.21.02 (d) To show that Px is the projection of x on range( P ) we verify that the “error” x − Px is orthogonal to any vector in range( P ). Since range( P ) is nothing but the span of the columns of P we only need to show that x − Px is orthogonal to the columns of P , or in other words, P T ( x − Px ) = 0. But P T ( x − Px ) = P ( x − Px ) (since P = P T ) = Px − P 2 x = (since P 2 = P ) and we are done. 2. Gradient of some common functions. 42090 Recall that the gradient of a differentiable function f : R n → R , at a point x ∈ R n , is defined as the vector ∇ f ( x ) = ∂f ∂x 1 . . . ∂f ∂x n , where the partial derivatives are evaluated at the point x . The first order Taylor approx imation of f , near x , is given by ˆ f tay ( z ) = f ( x ) + ∇ f ( x ) T ( z − x ) ....
View
Full Document
 Fall '08
 BOYD,S
 Linear Algebra, Least Squares, Pixel, Impulse response, Linear least squares

Click to edit the document details