{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

EE103 Section 7

EE103 Section 7 - EE103 Winter 2009 Lecture Notes(SEJ...

This preview shows pages 1–5. Sign up to view the full content.

EE103 Winter 2009 Lecture Notes (SEJ) Section 7 SECTION 7: INTRODUCTION TO OPTIMIZATION: NONLINEAR LEAST SQUARES, THE NEWTON AND GAUSS-NEWTON METHODS ....................... 129 Newton’s Method for Systems of Nonlinear Equations ......................................... 129 Newton’s Algorithm .............................................................................................. 130 Introduction to Optimization and Nonlinear Least Squares ................................ 133 Philosophy of Unconstrained Nonlinear Optimization ......................................... 134 The Newton Method for Optimization .................................................................... 136 Numerical Example of Steepest Decent vs. Newton’s Method ......................... 137 Newton’s Method Applied to Nonlinear Least-squares .................................... 138 A Modification: The Gauss-Newton Method ..................................................... 139 Tutorial Codes (i.e., not elegantly written) . ...................................................... 141

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
EE103 Winter 2009 Lecture Notes (SEJ) Section 7 129 SECTION 7: INTRODUCTION TO OPTIMIZATION: NONLINEAR LEAST SQUARES, THE NEWTON AND GAUSS-NEWTON METHODS This Section introduces the student to Newton’s method for solving a system of nonlinear equations, where the number of variables is equal to the number of equations. Subsequently, we consider the case where there are more nonlinear equations than there are variables. As in the linear case ( , Ax b m n = > ), this leads us to consider the nonlinear least squares problem. The latter problem often arises when considering the estimation of parameters for known functional forms. Newton’s Method for Systems of Nonlinear Equations We are interested in a numerical method for computing a solution of the system of nonlinear equations ( ) () 112 212 12 ,,, 0 , , , 0 0 n n nn fxx x x x = = = # Let T n x xx x = and let T n f ff f = . We have () () () () ,, , T n f xf x f x f x = and we abbreviate the above system, using vector notation, to ( ) 0 fx = an n x n system of nonlinear equations. Given a point , T kk k k n x x = , we use the 1 st order approximation for each of the functions; that is () ( ) ( )( ) () () () ( ) 11 1 22 2 k k k n f x f x x x f x f x x x f x f x x x ≈+ # or in vector-matrix notation:
EE103 Winter 2009 Lecture Notes (SEJ) Section 7 130 () () ( ) () () () () ( ) 1 2 k k kk k n k f fx f xf x x x fx fx J x x x Δ ⎡⎤ ⎢⎥ ≈+ ⎣⎦ # As in the single variable case, we solve the (system of) linear equations ( ) ( )( ) 0 k f + −= or, assuming ( ) k f Jx is nonsingular, we write ( notation, not computation! ) ( ) ( ) 1 1 1 k f k k f xx J x x xJ x f x + =− ( ) ( ) ( ) 1 Nf gx x J x f x ∴= However, we do not solve for 1 k x + by inverting ( ) k f . Rather, letting ( ) k yx x , we solve the system of equations (e.g., by LU methods) ( ) ( ) f Jxy fx Let k y be the solution. Therefore, 1 k y x xy + = + Newton's algorithm for a system of nonlinear equations may be succinctly stated. Newton’s Algorithm Select an initial vector x DoWhile (some appropriate stopping condition has not occurred) Compute the Jacobian, f Let y solve the system of linear equations f (assuming f is nonsingular) x ←+ End DoWhile

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
EE103 Winter 2009 Lecture Notes (SEJ) Section 7 131 Example (“Using Matlab as a calculator”): Consider the circle and ellipse 22 12 10 xx + −= 0.5 0 += We wish to find an intersection point. Starting with (1, 1.5) t x = , we’ll use Matlab as a calculator to find a solution and we’ll stop when the addition to the current value of x is less than approximately 3 10 .
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

Page1 / 15

EE103 Section 7 - EE103 Winter 2009 Lecture Notes(SEJ...

This preview shows document pages 1 - 5. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online