rather good estimate of the original vectorx. Forλ=100 we get a smoother RLS signal,but evidently it is less accurate thanxRLS(10), especially near the boundaries. The RLSsolution forλ=1000 is very smooth, but it is a rather poor estimate of the original signal.In any case, it is evident that the parameterλis chosen via a trade off between data fidelity(closeness ofxtob) and smoothness (size ofLx). The four plots where produced by theMATLAB commandsL=zeros(299,300);for i=1:299L(i,i)=1;L(i,i+1)=-1;endx_rls=(eye(300)+1*L’*L)\b;x_rls=[x_rls,(eye(300)+10*L’*L)\b];x_rls=[x_rls,(eye(300)+100*L’*L)\b];x_rls=[x_rls,(eye(300)+1000*L’*L)\b];figure(2)Downloaded 01/06/17 to 69.91.157.97. Redistribution subject to SIAM license or copyright; see

3.5. Nonlinear Least Squares45for j=1:4subplot(2,2,j);plot(1:300,x_rls(:,j),’LineWidth’,2);hold onplot(1:300,x,’:r’,’LineWidth’,2);hold offtitle([’\lambda=’,num2str(10^(j-1))]);end3.5Nonlinear Least SquaresThe least squares problem considered so far is also referred to as “linear least squares”since it is a method for finding a solution to a set of approximate linear equalities. Thereare of course situations in which we are given a system ofnonlinearequationsfi(x)≈ci,i=1,2,...,m.In this case, the appropriate problem is thenonlinear least squares (NLS) problem, whichis formulated asminmi=1(fi(x)−ci)2.(3.5)As opposed to linear least squares, there is no easy way to solve NLS problems. In Section4.5 we will describe the Gauss–Newton method which is specifically devised to solve NLSproblems of the form (3.5), but the method is not guaranteed to converge to the globaloptimal solution of (3.5) but rather to a stationary point.3.6Circle FittingSuppose that we are givenmpointsa1,a2,...,am∈n. Thecircle fitting problemseeks tofind a circleC(x,r) ={y∈n:∥y−x∥=r}that best fits thempoints. Note that we use the term “circle,” although this terminol-ogy is usually used in the plane(n=2), and here we consider the generaln-dimensionalspacen. Additionally, note thatC(x,r)is the boundary set of the corresponding ballB(x,r). An illustration of such a fit is given in Figure 3.4. The circle fitting problem hasapplications in many areas, such as archaeology, computer graphics, coordinate metrol-ogy, petroleum engineering, statistics, and more. The nonlinear (approximate) equationsassociated with the problem are∥x−ai∥ ≈r,i=1,2,...,m.Since we wish to deal with differentiable functions, and the norm function is not differ-entiable, we will consider the squared version of the latter:∥x−ai∥2≈r2,i=1,2,...,m.Downloaded 01/06/17 to 69.91.157.97. Redistribution subject to SIAM license or copyright; see

46Chapter 3. Least Squares3540455055606570755051015Figure 3.4.The best circle fit(the optimal solution of problem(3.6))of10points denoted by asterisks.The NLS problem associated with these equations isminx∈n,r∈+mi=1(∥x−ai∥2−r2)2.(3.6)From a first glance, problem (3.6) seems to be a standard NLS problem, but in this casewe can show that it is in factequivalentto a linear least squares problem, and therefore