{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

571_5.3

# 571_5.3 - 5.3 LEAST SQUARES PROBLEMS KIAM HEONG KWA pp...

This preview shows pages 1–3. Sign up to view the full content.

5.3 LEAST SQUARES PROBLEMS KIAM HEONG KWA pp. 223-224 In practice, the coefficient matrix A and the right-hand side b of an m × n system A x = b are obtained emprically. In consequence, the system A x = b is usually inconsistent. In this case, we look for a vector b x R n for which A b x is closest to b in the following sense. (Clearly, if x is a solution of A x = b , then it satisfies the desired property.) Define the residual r : R n R m of the system A x = b by setting r ( x ) = b - A x and consider the norm of the residual k r k : R n R given by k r ( x ) k = k b - A x k for each x R n , where k · k is the Euclidean norm. Then for a vector b x R n , A b x is closest to b if and only if k r ( b x ) k is a (global) minimum, i.e., k r ( b x ) k ≤ k r ( x ) k for all x R n . Such a vector b x that minimizes k r k is called a least squares solution of the system A x = b . Note that minimizing k r k is equivalent to minimizing k r k 2 . Let b x be a least squares solution of A x = b . Then p = A b x R ( A ) , the column space of A . By the definition of a least squares solution, p is an element of R ( A ) that is closest to b . Since R ( A ) R m is a subspace, such a closest vector p always exists and is unique. Moreover, it is characterized by the inclusion b - p R ( A ) = N ( A T ) 1 . In other words, b x is a least squares solution of A x = b if and only if r ( b x ) = b - A b x R ( A ) = N ( A T ) . This is a consequence of the following general statement. Let S R m be a subspace and let b R m but otherwise arbi- trary. There is a unique element p S , called the projection of b Therem 5.3.1 Date : July 15, 2011. 1 The fundamental subspaces theorem from the last section says that N ( A ) = R ( A T ) and N ( A T ) = R ( A ) for any matrix A . 1

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
2 KIAM HEONG KWA onto S , such that k b - p k < k b - y k for any y S , y 6 = p . In addition, such an element p of S is char- acterized by the condition b - p S . Proof. To begin with, recall that R m = S S , so that b has the unique representation b = p + q , where p S and q S . Hence if y S , so that p - y S , we have k b - y k 2 = k ( b - p ) + ( p - y ) k 2 = k b - p k 2 + k p - y k 2 because b - p = q p - y . Thus we see that k b - y k ≥ k b - p k and the equality holds if and only if k p - y k = 0 or p = y . This shows the existence and the uniqueness of such an element p of S in the statement of the theorem. In fact, the above calculation also shows that if p S is such that b - p S , then k b - y k ≥ k b - p k for any y S . Conversely, let z S be such that b - z 6∈ S . Then, in particular, z 6 = p and, as in the above calculation, we have k b - z k 2 = k ( b - p ) + ( p - z ) k 2 = k b - p k 2 + k p - z k 2 , so that k b - z k > k b - p k . This shows that the element p of S such that k b - p k is minimized is characterized by the inclusion b - p S . pp. 224-225 Let b x be a least squares solution of the system A x = b . Recall that b x is characterized by the inclusion r ( b x ) = b - A b x R ( A ) = N ( A ) T . That is, A T ( b - A b x ) = Ar ( b x ) = 0 or equivalently A T A b x = A T b .
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}