hw5_2009_10_21_02_solutions

hw5_2009_10_21_02_solutions - EE263 S Lall 2009.10.21.02...

This preview shows pages 1–3. Sign up to view the full content.

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: EE263 S. Lall 2009.10.21.02 Homework 5 Solutions Due Thursday 10/29 1. Projection matrices. 44020 A matrix P ∈ R n × n is called a projection matrix if P = P T and P 2 = P . (a) Show that if P is a projection matrix then so is I − P . (b) Suppose that the columns of U ∈ R n × k are orthonormal. Show that UU T is a projection matrix. (Later we will show that the converse is true: every projection matrix can be expressed as UU T for some U with orthonormal columns.) (c) Suppose A ∈ R n × k is full rank, with k ≤ n . Show that A ( A T A ) − 1 A T is a projection matrix. (d) If S ⊆ R n and x ∈ R n , the point y in S closest to x is called the projection of x on S . Show that if P is a projection matrix, then y = Px is the projection of x on range( P ). (Which is why such matrices are called projection matrices . . . ) Solution. (a) To show that I − P is a projection matrix we need to check two properties: i. I − P = ( I − P ) T ii. ( I − P ) 2 = I − P . The first one is easy: ( I − P ) T = I − P T = I − P because P = P T ( P is a projection matrix.) The show the second property we have ( I − P ) 2 = I − 2 P + P 2 = I − 2 P + P (since P = P 2 ) = I − P and we are done. (b) Since the columns of U are orthonormal we have U T U = I . Using this fact it is easy to prove that UU T is a projection matrix, i.e. , ( UU T ) T = UU T and ( UU T ) 2 = UU T . Clearly, ( UU T ) T = ( U T ) T U T = UU T and ( UU T ) 2 = ( UU T )( UU T ) = U ( U T U ) U T = UU T (since U T U = I ) . (c) First note that ( A ( A T A ) − 1 A T ) T = A ( A T A ) − 1 A T because ( A ( A T A ) − 1 A T ) T = ( A T ) T ( ( A T A ) − 1 ) T A T = A ( ( A T A ) T ) − 1 A T = A ( A T A ) − 1 A T . Also ( A ( A T A ) − 1 A T ) 2 = A ( A T A ) − 1 A T because ( A ( A T A ) − 1 A T ) 2 = ( A ( A T A ) − 1 A T )( A ( A T A ) − 1 A T ) = A ( ( A T A ) − 1 A T A ) ( A T A ) − 1 A T = A ( A T A ) − 1 A T (since ( A T A ) − 1 A T A = I ) . 1 EE263 S. Lall 2009.10.21.02 (d) To show that Px is the projection of x on range( P ) we verify that the “error” x − Px is orthogonal to any vector in range( P ). Since range( P ) is nothing but the span of the columns of P we only need to show that x − Px is orthogonal to the columns of P , or in other words, P T ( x − Px ) = 0. But P T ( x − Px ) = P ( x − Px ) (since P = P T ) = Px − P 2 x = (since P 2 = P ) and we are done. 2. Gradient of some common functions. 42090 Recall that the gradient of a differentiable function f : R n → R , at a point x ∈ R n , is defined as the vector ∇ f ( x ) = ∂f ∂x 1 . . . ∂f ∂x n , where the partial derivatives are evaluated at the point x . The first order Taylor approx- imation of f , near x , is given by ˆ f tay ( z ) = f ( x ) + ∇ f ( x ) T ( z − x ) ....
View Full Document

{[ snackBarMessage ]}

Page1 / 12

hw5_2009_10_21_02_solutions - EE263 S Lall 2009.10.21.02...

This preview shows document pages 1 - 3. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online