*This preview shows
page 1. Sign up
to
view the full content.*

**Unformatted text preview: **, xn , y]
If A = Q1 R1 with rank(A) = p , the full QR factorization is
A= Q1 Q2 R1
0 where Q1 Q2 is orthogonal, i.e., columns of Q2 ∈ IRm×(m−p) are
orthonormal, orthogonal to Q1
To ﬁnd Q2 , one can use any matrix A s.t. [A A] is full rank (e.g.,
A = I ), and then apply general Gram-Schmidt process
Q1 are orthonormal vectors obtained form columns of A
Q2 are orthonormal vectors obtained from extra columns A
12 / 27 Complementary subspaces ran(Q1 ) and ran(Q2 ) are complementary subspaces since
they are orthogonal (i.e., every vector in the ﬁrst subspace is
orthogonal to every vector in the second subspace)
the sum is IRm (i.e., every vector in IRm can be expressed as a sum of
two vectors, one from each subspace)
⊥ ran(Q1 ) +ran(Q2 ) = IRm ran(Q2 ) = ran(Q1 )⊥ (ran(Q1 ) = ran(Q2 )⊥ )
We know ran(Q1 ) = ran(A); what is its orthogonal complement
ran(Q2 )? 13 / 27 Complementary subspaces From A = R1 0 Q1
Q2 , we have A z = 0 ⇐⇒ Q1 z = 0 ⇐⇒ z ∈ ran(Q2 )
so ran(Q2 ) = ran(A ) (i.e., columns of Q2 are an orthonormal basis
for ran(A )
We conclude: ran(A) and null(A ) are complementary subspaces
⊥ ran(A) + null(A ) = IRm (recall A ∈ IRm×p
ran(A)⊥ = null(A ) (and null(A )⊥ = ran(A)
called orthogonal decomposition induced by A ∈ IRm×p 14 / 27 Least squares via QR decomposition A ∈ IRm×n , A = QR with Q Q = I , R ∈ IRn×n
The pseudo inverse is
(A A)−1 A = (R Q QR )−1 R Q = R −1 Q The projected point, xls = R −1 Q y
The projection matrix
A(A A)−1 A = AR −1 Q = QQ 15 / 27 Least squares via full QR factorization
Full QR factorization
A= Q1 Q2 R1
0 where Q1 Q2 ∈ IRm×m is orthogonal, R1 ∈ IRn×n is upper
triangular and invertible
Multiplication by orthogonal matrix
Ax − y 2 = Q1 Q2 = Q1 Q2 =
= 2 R1
0 x−y
R1
0 Q1 Q2 R1 x − Q1 y
−Q2 y
2
R1 x − Q1 y
+ 2 x− Q1 Q2 y 2 Q2 y 2 −
Can be minimized by choosing xls = R1 1 Q1 y
16 / 27 Least squares via full QR factorization (cont’d) Least squares minimization
Ax − y 2 = R1 x − Q1 y 2 + Q2 y 2 −
Optimal solution xls = R1 1 Q1 y Residual of the optimal x is
Axls − y = −Q2 Q2 y
Q1 Q1 gives projection onto ran(A)
Q2 Q2 gives projection onto ran(A)⊥ 17 / 27 Gram-Schmidt as tri...

View
Full
Document