EE103 Section 7

EE103 Section 7 - EE103 Winter 2009 Lecture Notes (SEJ)...

Info iconThis preview shows pages 1–5. Sign up to view the full content.

View Full Document Right Arrow Icon
EE103 Winter 2009 Lecture Notes (SEJ) Section 7 SECTION 7: INTRODUCTION TO OPTIMIZATION: NONLINEAR LEAST SQUARES, THE NEWTON AND GAUSS-NEWTON METHODS ....................... 129 Newton’s Method for Systems of Nonlinear Equations ......................................... 129 Newton’s Algorithm .............................................................................................. 130 Introduction to Optimization and Nonlinear Least Squares ................................ 133 Philosophy of Unconstrained Nonlinear Optimization ......................................... 134 The Newton Method for Optimization .................................................................... 136 Numerical Example of Steepest Decent vs. Newton’s Method ......................... 137 Newton’s Method Applied to Nonlinear Least-squares .................................... 138 A Modification: The Gauss-Newton Method ..................................................... 139 Tutorial Codes (i.e., not elegantly written) . ...................................................... 141
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
EE103 Winter 2009 Lecture Notes (SEJ) Section 7 129 SECTION 7: INTRODUCTION TO OPTIMIZATION: NONLINEAR LEAST SQUARES, THE NEWTON AND GAUSS-NEWTON METHODS This Section introduces the student to Newton’s method for solving a system of nonlinear equations, where the number of variables is equal to the number of equations. Subsequently, we consider the case where there are more nonlinear equations than there are variables. As in the linear case ( , Ax b m n = > ), this leads us to consider the nonlinear least squares problem. The latter problem often arises when considering the estimation of parameters for known functional forms. Newton’s Method for Systems of Nonlinear Equations We are interested in a numerical method for computing a solution of the system of nonlinear equations ( ) () 112 212 12 ,,, 0 , , , 0 0 n n nn fxx x x x = = = # Let T n x xx x = and let T n f ff f = . We have () () () () ,, , T n f xf x f x f x = and we abbreviate the above system, using vector notation, to ( ) 0 fx = an n x n system of nonlinear equations. Given a point , T kk k k n x x = , we use the 1 st order approximation for each of the functions; that is () ( ) ( )( ) () () () ( ) 11 1 22 2 k k k n f x f x x x f x f x x x f x f x x x ≈+ # or in vector-matrix notation:
Background image of page 2
EE103 Winter 2009 Lecture Notes (SEJ) Section 7 130 () () ( ) () () () () ( ) 1 2 k k kk k n k f fx f xf x x x fx fx J x x x Δ ⎡⎤ ⎢⎥ ≈+ ⎣⎦ # As in the single variable case, we solve the (system of) linear equations ( ) ( )( ) 0 k f + −= or, assuming ( ) k f Jx is nonsingular, we write ( notation, not computation! ) ( ) ( ) 1 1 1 k f k k f xx J x x xJ x f x + =− ( ) ( ) ( ) 1 Nf gx x J x f x ∴= However, we do not solve for 1 k x + by inverting ( ) k f . Rather, letting ( ) k yx x , we solve the system of equations (e.g., by LU methods) ( ) ( ) f Jxy fx Let k y be the solution. Therefore, 1 k y x xy + = + Newton's algorithm for a system of nonlinear equations may be succinctly stated. Newton’s Algorithm Select an initial vector x DoWhile (some appropriate stopping condition has not occurred) Compute the Jacobian, f Let y solve the system of linear equations f (assuming f is nonsingular) x ←+ End DoWhile
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
EE103 Winter 2009 Lecture Notes (SEJ) Section 7 131 Example (“Using Matlab as a calculator”): Consider the circle and ellipse 22 12 10 xx + −= 0.5 0 += We wish to find an intersection point. Starting with (1, 1.5) t x = , we’ll use Matlab as a calculator to find a solution and we’ll stop when the addition to the current value of x is less than approximately 3 10 .
Background image of page 4
Image of page 5
This is the end of the preview. Sign up to access the rest of the document.

This note was uploaded on 03/30/2009 for the course EE 103 taught by Professor Vandenberghe,lieven during the Winter '08 term at UCLA.

Page1 / 15

EE103 Section 7 - EE103 Winter 2009 Lecture Notes (SEJ)...

This preview shows document pages 1 - 5. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online