{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

mat 2310 0607_2_note9_1

mat 2310 0607_2_note9_1 - Lecture Note 9-1 Dr Jeff Chak-Fu...

Info iconThis preview shows pages 1–9. Sign up to view the full content.

View Full Document Right Arrow Icon
Lecture Note 9-1: March 19, 2007 Dr. Jeff Chak-Fu WONG Department of Mathematics Chinese University of Hong Kong [email protected] MAT 2310C Linear Algebra and Its Applications Spring, 2007 Produced by Jeff Chak-Fu WONG 1
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
A PPLICATIONS OF R EAL V ECTOR S PACES 1. Least Squares 2. QR-Factorization A PPLICATIONS OF R EAL V ECTOR S PACES 2
Background image of page 2
L EAST S QUARES Prerequisites: 1. Solutions of Linear Systems of Equations 2. The Inverse of a Matrix 3. n -Vectors 4. Orthogonal Complements L EAST S QUARES 3
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
We recall that An m × n linear system A x = b is inconsistent if it has no solution. A x = b is consistent if and only if b belongs to the column space of A (cf. Lecture Note 6, Theorem 0.6, Page 65). Equivalently, A x = b in inconsistent if and only if b is not in the column space of A . Inconsistent systems do indeed arise in many situations and we must determine how to deal with them. L EAST S QUARES 4
Background image of page 4
Our approach is to change the problem so that we do not require that the matrix equation A x = b be satisfied. Instead, we seek a vector ˆ x in R n such that A ˆ x is as close to b as possible. If W is the column space of A , it follows that the vector in W that is closest to b is proj W b . That is, k b - w k , for w in W , is minimized when w = proj W b . Thus, if we find ˆ x such that A ˆ x = proj W b , then we are assured that k b - A ˆ x k will be as small as possible (see Figure 1). W W = Column space of A proj W b b A x = w = b - w A x Figure 1: A solution x produces the vector A x in W closest to b L EAST S QUARES 5
Background image of page 5

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
As shown in the proof of Theorem 0.5 (cf. Lecture Note 8-2, Page 49), b - proj W b = b - A ˆ x is orthogonal to every vector in W (see Figure 2). In other words, b - proj W b = b - A x is orthogonal to W . But W is the column space of A , so follows from Theorem 0.5 (Lecture note 8-2) that b - A x lies in the null space of A T . W W = Column space of A proj W b b Figure 2: It then follows that b - A ˆ x is orthogonal to each column of A , In terms of a matrix equation, we have A T ( A ˆ x - b ) = 0 L EAST S QUARES 6
Background image of page 6
or equivalently, A T A ˆ x = A T b . Thus, ˆ x is a solution to A T A x = A T b . (1) Any solution to (1) is called a least squares solution to the linear system A x = b . ( Warnings: In general, A ˆ x 6 = b .) Equation (1) is called the normal system of equations associated with A x = b , or just the normal system. Note that x is a least squares solution of A x = b if and only if b - A x belongs to W . Now b = A x + ( b - A x ) , A x belongs to W . Observe that if A x = b is consistent, then a solution to this system is a least squares solution. In particular, if A is nonsingular, a least squares solution to A x = b is just the usual solution x = A - 1 b . L EAST S QUARES 7
Background image of page 7

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
To compute a least squares solution ˆ x to the linear system A x = b , we can proceed as follows. Let { w 1 , w 2 , · · · , w m } be an orthonormal basis for the column space W of A . Then Equation (1) of Lecture Note 8-2 yields proj W b = ( b · w 1 ) w 1 + ( b · w 2 ) w 2 + · · · + ( b · w m ) w m .
Background image of page 8
Image of page 9
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}