Normal Equations If b is not in the column space of A , then Ax = b has no solution; the system is inconsistent . This is typical if A is m × n with m > n , which we will assume here. Let us also assume that A has full rank. Since Ax = b has no solution, one may reasonably be interested in ﬁnding a vector x which minimizes the diﬀerence between b and Ax : min x k Ax-b k . (1) Equivalently: ﬁnd a vector y in the column space of A which is closest to b (then x is the unique solution of the consistent system Ax = y ). There are many norms that we might use in (1), but if we use the norm induced by the dot product, then (1) is called the discrete linear least squares problem : min x k Ax-b k 2 . (2) Now suppose that we want to ﬁnd an element of S ≤ R n that is closest to some vector b (which is typically not in S ). Our intuition says that we “drop a perpendicular” from b to S , and that is exactly right: Let y ∈ S be such that b-y ⊥ S , and consider any vector w = y + αz
This is the end of the preview.
access the rest of the document.