571_5.3 - 5.3 LEAST SQUARES PROBLEMS KIAM HEONG KWA pp....

Info iconThis preview shows pages 1–3. Sign up to view the full content.

View Full Document Right Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: 5.3 LEAST SQUARES PROBLEMS KIAM HEONG KWA pp. 223-224 In practice, the coefficient matrix A and the right-hand side b of an m n system A x = b are obtained emprically. In consequence, the system A x = b is usually inconsistent. In this case, we look for a vector b x R n for which A b x is closest to b in the following sense. (Clearly, if x is a solution of A x = b , then it satisfies the desired property.) Define the residual r : R n R m of the system A x = b by setting r ( x ) = b- A x and consider the norm of the residual k r k : R n R given by k r ( x ) k = k b- A x k for each x R n , where k k is the Euclidean norm. Then for a vector b x R n , A b x is closest to b if and only if k r ( b x ) k is a (global) minimum, i.e., k r ( b x ) k k r ( x ) k for all x R n . Such a vector b x that minimizes k r k is called a least squares solution of the system A x = b . Note that minimizing k r k is equivalent to minimizing k r k 2 . Let b x be a least squares solution of A x = b . Then p = A b x R ( A ) , the column space of A . By the definition of a least squares solution, p is an element of R ( A ) that is closest to b . Since R ( A ) R m is a subspace, such a closest vector p always exists and is unique. Moreover, it is characterized by the inclusion b- p R ( A ) = N ( A T ) 1 . In other words, b x is a least squares solution of A x = b if and only if r ( b x ) = b- A b x R ( A ) = N ( A T ) . This is a consequence of the following general statement. Let S R m be a subspace and let b R m but otherwise arbi- trary. There is a unique element p S , called the projection of b Therem 5.3.1 Date : July 15, 2011. 1 The fundamental subspaces theorem from the last section says that N ( A ) = R ( A T ) and N ( A T ) = R ( A ) for any matrix A . 1 2 KIAM HEONG KWA onto S , such that k b- p k < k b- y k for any y S , y 6 = p . In addition, such an element p of S is char- acterized by the condition b- p S . Proof. To begin with, recall that R m = S S , so that b has the unique representation b = p + q , where p S and q S . Hence if y S , so that p- y S , we have k b- y k 2 = k ( b- p ) + ( p- y ) k 2 = k b- p k 2 + k p- y k 2 because b- p = q p- y . Thus we see that k b- y k k b- p k and the equality holds if and only if k p- y k = 0 or p = y . This shows the existence and the uniqueness of such an element p of S in the statement of the theorem. In fact, the above calculation also shows that if p S is such that b- p S , then k b- y k k b- p k for any y S . Conversely, let z S be such that b- z 6 S . Then, in particular, z 6 = p and, as in the above calculation, we have k b- z k 2 = k ( b- p ) + ( p- z ) k 2 = k b- p k 2 + k p- z k 2 , so that k b- z k > k b- p k ....
View Full Document

This note was uploaded on 11/11/2011 for the course MATH 571 taught by Professor Kim during the Summer '08 term at Ohio State.

Page1 / 9

571_5.3 - 5.3 LEAST SQUARES PROBLEMS KIAM HEONG KWA pp....

This preview shows document pages 1 - 3. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online