This
** preview**
has intentionally

**sections.**

*blurred***to view the full version.**

*Sign up*
**Unformatted text preview: **Least Squares and matrices Jonathan L.F. King University of Florida, Gainesville FL 32611-2082, USA [email protected] Webpage http://www.math.ufl.edu/ ∼ squash/ The Problem Suppose we have a collection K of N points Q 1 ,...,Q j ,...Q N in the plane. Consider now the line L with equation y = βx + α . It has slope β and y-intercept α . At a given point Q = ( x,y ), the vertical ( signed ) distance to L is v := [ α + βx ]- y . 1 : Letting v j denote the vertical distance at Q j , de- fine the least-square distance from K to L by g ( α,β ) := X N j =1 [ v j ] 2 . 1 : Our goal is to find all pairs ( α,β ) which mini- mize g . It will turn out there is a unique min- imum, except in the silly case that all the given points lie on one vertical line. That is, writing Q j as ( x j ,y j ), except when x 1 = ··· = x N . The quantities that we will need are X := ∑ N j =1 x j , Y := ∑ N j =1 y j , S := ∑ N j =1 x j 2 , P := ∑ N j =1 y j x j . ( “ S ” is for Squares and “ P ” is for Product. ) Using Calculus Evidently in computing the first-partials of g we will want to compute them for each v j . From (1) we compute that d v d α = 1, so d d α ( v 2 ) = 2 v · 1 and d v d β = x , so d d β ( v 2 ) = 2 v · x, by the Chain Rule. Consequently d g d α = X N j =1 2 v j and d g d β = X N j =1 2 v j x j . Thus, the pair ( α,β ) is a critical point of g IFF at ( α,β ) we have that 0 = X N j =1 v j and 0 = X N j =1 v j x j . 2 : Recall that v j is α + x j β- y j . So multiplying out and distributing the summations in (2) yields that 0 = Nα + Xβ- Y , 0 = Xα + Sβ- P . 2 : We can rewrite this to say that ( α,β ) is a critical point of g IFF Y = Nα + Xβ , P = Xα + Sβ . 3 : Matrices. Let M denote the matrix [ N X X S ] and let D := Det( M ) note === NS- X 2 . It follows from a standard ♥ 1 inequality that: All the points x 1 ,...,x N are equal IFF D = 0. We henceforth assume that our scatterplot has at least two distinct x-values. Bare-hands computation [ or matrix algebra ] shows that (3) has a unique solution, which is α = 1 D h S Y- XP i , β = 1 D h XY + NP i ....

View
Full
Document