{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

571_5.3 - 5.3 LEAST SQUARES PROBLEMS KIAM HEONG KWA pp...

Info iconThis preview shows pages 1–3. Sign up to view the full content.

View Full Document Right Arrow Icon
5.3 LEAST SQUARES PROBLEMS KIAM HEONG KWA pp. 223-224 In practice, the coefficient matrix A and the right-hand side b of an m × n system A x = b are obtained emprically. In consequence, the system A x = b is usually inconsistent. In this case, we look for a vector b x R n for which A b x is closest to b in the following sense. (Clearly, if x is a solution of A x = b , then it satisfies the desired property.) Define the residual r : R n R m of the system A x = b by setting r ( x ) = b - A x and consider the norm of the residual k r k : R n R given by k r ( x ) k = k b - A x k for each x R n , where k · k is the Euclidean norm. Then for a vector b x R n , A b x is closest to b if and only if k r ( b x ) k is a (global) minimum, i.e., k r ( b x ) k ≤ k r ( x ) k for all x R n . Such a vector b x that minimizes k r k is called a least squares solution of the system A x = b . Note that minimizing k r k is equivalent to minimizing k r k 2 . Let b x be a least squares solution of A x = b . Then p = A b x R ( A ) , the column space of A . By the definition of a least squares solution, p is an element of R ( A ) that is closest to b . Since R ( A ) R m is a subspace, such a closest vector p always exists and is unique. Moreover, it is characterized by the inclusion b - p R ( A ) = N ( A T ) 1 . In other words, b x is a least squares solution of A x = b if and only if r ( b x ) = b - A b x R ( A ) = N ( A T ) . This is a consequence of the following general statement. Let S R m be a subspace and let b R m but otherwise arbi- trary. There is a unique element p S , called the projection of b Therem 5.3.1 Date : July 15, 2011. 1 The fundamental subspaces theorem from the last section says that N ( A ) = R ( A T ) and N ( A T ) = R ( A ) for any matrix A . 1
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
2 KIAM HEONG KWA onto S , such that k b - p k < k b - y k for any y S , y 6 = p . In addition, such an element p of S is char- acterized by the condition b - p S . Proof. To begin with, recall that R m = S S , so that b has the unique representation b = p + q , where p S and q S . Hence if y S , so that p - y S , we have k b - y k 2 = k ( b - p ) + ( p - y ) k 2 = k b - p k 2 + k p - y k 2 because b - p = q p - y . Thus we see that k b - y k ≥ k b - p k and the equality holds if and only if k p - y k = 0 or p = y . This shows the existence and the uniqueness of such an element p of S in the statement of the theorem. In fact, the above calculation also shows that if p S is such that b - p S , then k b - y k ≥ k b - p k for any y S . Conversely, let z S be such that b - z 6∈ S . Then, in particular, z 6 = p and, as in the above calculation, we have k b - z k 2 = k ( b - p ) + ( p - z ) k 2 = k b - p k 2 + k p - z k 2 , so that k b - z k > k b - p k . This shows that the element p of S such that k b - p k is minimized is characterized by the inclusion b - p S . pp. 224-225 Let b x be a least squares solution of the system A x = b . Recall that b x is characterized by the inclusion r ( b x ) = b - A b x R ( A ) = N ( A ) T . That is, A T ( b - A b x ) = Ar ( b x ) = 0 or equivalently A T A b x = A T b .
Background image of page 2
Image of page 3
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}