# Register now to access 7 million high quality study materials (What's Course Hero?) Course Hero is the premier provider of high quality online educational resources. With millions of study documents, online tutors, digital flashcards and free courseware, Course Hero is helping students learn more efficiently and effectively. Whether you're interested in exploring new subjects or mastering key topics for your next exam, Course Hero has the tools you need to achieve your goals.

9 Pages

### barrierb

Course: CAAM 236, Spring 2007
School: Monmouth IL
Rating:

Word Count: 1138

#### Document Preview

(Fall EE236A 2007-08) Lecture 13 Convergence analysis of the barrier method complexity analysis of the barrier method convergence analysis of Newtons method choice of update parameter bound on the total number of Newton iterations initialization 131 Complexity analysis well analyze the method of page 1221 with update t+ = t starting point x(t(0)) on the central path O( m log((0)/)) caveats: methods...

Register Now

#### Unformatted Document Excerpt

Coursehero >> Illinois >> Monmouth IL >> CAAM 236

Course Hero has millions of student submitted documents similar to the one
below including study guides, practice problems, reference materials, practice exams, textbook help and tutor support.

Course Hero has millions of student submitted documents similar to the one below including study guides, practice problems, reference materials, practice exams, textbook help and tutor support.
(Fall EE236A 2007-08) Lecture 13 Convergence analysis of the barrier method complexity analysis of the barrier method convergence analysis of Newtons method choice of update parameter bound on the total number of Newton iterations initialization 131 Complexity analysis well analyze the method of page 1221 with update t+ = t starting point x(t(0)) on the central path O( m log((0)/)) caveats: methods with good worst-case complexity dont necessarily work better in practice were not interested in the numerical values for the boundonly in the exponent of m and n doesnt include initialization insights obtained from analysis are more valuable than the bound itself Convergence analysis of the barrier method 132 main result: #Newton iters is bounded by (where (0) = m/t(0)) Outline 1. convergence analysis of Newtons method for m (x) = tc x T i=1 log(bi aT x) i (will give us a bound on the number of Newton steps per outer iteration) 2. eect of on total number of Newton iterations to compute x(t) from x(t) 3. combine 1 and 2 to obtain the total number of Newton steps, starting at x(t(0)) Convergence analysis of the barrier method 133 The Newton decrement Newton step at x: v = 2(x)1(x) = (AT diag(d)2A)1(tc + AT d) where d = (1/(b1 aT x), . . . , 1/(bm aT x)) m 1 Newton decrement at x: (x) = = = i=1 (x)T 2(x)1(x) v T 2(x)v m 1/2 aT v i bi aT x i 2 = diag(d)Av Convergence analysis of the barrier method 134 theorem. if = (x) < 1, then is bounded below and (x) (x(t)) log(1 ) 4 3.5 3 2.5 2 1.5 1 0.5 0 0 0.2 0.4 0.6 0.8 1 if 0.81, then (x) (x(t)) + useful as stopping criterion for Newtons method Convergence analysis of the barrier method 135 proof: w.l.o.g. assume b Ax = 1; let x = x(t), z = 1 + Av = Av < 1 2(x)v = AT Av = (x) = tc AT 1 = z = 1 + Av 0 = AT z = tc m m tc x T i=1 log(bi aT x) i = z Ax z T Ax + T i=1 m i=1 T log(bi aT x) i log zi z T (b Ax) + m m = (1 + Ax) z + = tcT x + i log zi + m i=1 (aT v + log(1 + aT v )) i i tcT x + + log(1 ) Convergence analysis of the barrier method 136 inequalities follow from: 1. log y log z + zy 1 for y, z > 0 2. m i=1 (yi log(1 + yi)) y log(1 y ) if y < 1 Convergence analysis of the barrier method 137 Local convergence analysis x+ = x 2(x)1(x) theorem: if < 1, then Ax+ < b and + 2 ( is Newton decrement at x; + is Newton decrement at x+) gives bound on number of iterations: suppose we start at x(0) with (0) 0.5, then (x) (x(t)) < after fewer than log2 log2(1/ ) iterations called region of quadratic convergence practical rule of thumb: 56 iterations Convergence analysis of the barrier method 138 proof. 1. 2 = m T2 i=1 (ai v ) /(bi aT x)2 < 1 implies aT (x + v ) < bi i i 2. assume b Ax+ = 1; let w = 1 d diag(d)2Av (+)2 = Av + 2 = = Av + m 2 w + Av + Av + (1 di)4 (diaT v )4 i 2(Av +)T (w + Av +) 2 (1) (2) i=1 m = i=1 diag(d)Av 4 = 4 (1) uses AT w = tc + AT 1, AT Av + = tc AT 1 (2) uses Av = Ax+ b Ax + b = 1 + diag(d)11, therefore diaT v = 1 di and wi = (1 di)2 i Convergence analysis of the barrier method 139 Global analysis of Newtons method damped Newton algorithm: x+ = x + sv , v = 2(x)1(x) step size to the boundary: s = 1 where = max aT v i bi aT x i aT v > 0 i ( = 0 if Av 0) theorem. for = s 1/(1 + ), (x + sv ) (x) ( log(1 + )) very simple expression for step size if 0.5, same bound if s is determined by an exact line search (x + (1 + )1v ) (x) 0.09 (hence, convergence) Convergence analysis of the barrier method 1310 proof. dene f (s) = (x + sv ) for 0 s < 1/ f (s) = v T (x + sv ), by integrating the upper bound m f (s) = v T 2(x + sv )T v for Newton direction v : f (0) = f (0) = 2 aT v i bi aT x saT v i i 2 f (s) = i=1 f (0) (1 s)2 twice, we obtain f (s) f (0) + sf (0) f (0) (s + log(1 s)) 2 upper bound is minimized by s = f (0)/(f (0) f (0)) = 1/(1 + ) f (s) f (0) f (0) ( log(1 + )) 2 f (0) ( log(1 + )) (since )) 1311 Convergence analysis of the barrier method Summary given x with Ax < b, tolerance (0, 0.5) repeat 1. Compute Newton step at x: v = 2(x)1(x) 2. Compute Newton decrement: = (v T 2(x)v )1/2 3. If , return(x) 4. Update x: If 0.5, x := x + (1 + )1v where = max{0, maxi aT v/(bi aT x)} i i else, x := x + v upper bound on #iterations, starting at x: log2 log2(1/ ) + 11 ((x) (x(t))) usually very pessimistic; good measure in practice: 0 + 1 ((x) (x(t))) with empirically determined i (0 5, 1 11) Convergence analysis of the barrier method 1312 #Newton steps per outer iteration #Newton steps to minimize (x) = t+cT x theorem. if z > 0, AT z + c = 0, then m m i=1 log(bi aT x) i (x (t )) t b z + + +T log zi + m(1 + log t+) i=1 in particular, for t+ = t, zi = 1/t(bi aT x(t)): i (x(t+)) (x(t)) m( 1 log ) yields estimates for #Newton steps to minimize starting at x(t): 0 + 1m( 1 log ) is an upper bound for 0 = log2 log2(1/ ), 1 = 11 is a good measure in practice for empirically determined 0, 1 Convergence analysis of the barrier method 1313 proof. if z > 0, AT z + c = 0, then m (x) = t c x t+cT x + +T i=1 m log(bi aT x) i log zi t+z T (b Ax) + m(1 + log t+) log zi + m(1 + log t+) i=1 m = t+bT z + i=1 for zi = 1/(t(bi aT x(t)), t+ = t, this yields i (x(t+)) (x(t)) m( 1 log ) Convergence analysis of the barrier method 1314 Bound on total #Newton iters suppose we start on central path with t = t(0) number of outer iterations: log((0)/) #outer iters = log (0) = m/t(0): initial duality gap (0)/: reduction in duality gap upper bound on total #Newton steps: log((0)/) (0 + 1m( 1 log )) log 0 = log2 log2(1/ ), 1 = 11 can use empirical values for i to estimate average-case behavior Convergence analysis of the barrier method 1315 Strategies for choosing independent of m: #Newton steps per outer iter O(m) total #Newton steps O(m log((0)/))) = 1 + / m with independent of m #Newton steps per outer iter O(1) total #Newton steps O( m log((0)/))) follows from: m( 1 log ) 2/2, because x x2/2 log(1 + x) for x > 0 log(1 + / m) log(1 + )/ m for m 1 Convergence analysis of the barrier method 1316 Choice of initial t rule of thumb: given estimate p of p, choose m/t cT x p (since m/t is duality gap) via complexity theory (c.f. page 1312) given dual feasible z , #Newton steps in rst iteration is bounded by an ane function of m t(c x + b z ) + (x) T T i=1 log zi m(1 + logt) = t(cT x + bT z ) m log t + const. choose t to minimize bound; yields m/t = cT x + bT z there are many other ways to choose t Convergence analysis of the barrier method 1317
Find millions of documents on Course Hero - Study Guides, Lecture Notes, Reference Materials, Practice Exams and more. Course Hero has millions of course specific materials providing students with the best way to expand their education.

Below is a small sample set of documents:

Monmouth IL - CAAM - 236
EE236A (Fall 2007-08)Lecture 14 Primal-dual interior-point methods primal-dual path-following Mehrotras corrector step computing the search directions141Central path and complementary slacknesss + Ax b = 0 AT z + c = 0 zisi = 1/t, z 0, i = 1, . . . ,
Monmouth IL - CAAM - 236
EE236A (Fall 2007-08)Lecture 15 Self-dual formulations initialization and infeasibility detection skew-symmetric LPs homogeneous self-dual formulation self-dual formulation151Solution of an LPgiven a pair of primal and dual LPs minimize cT x subject
Monmouth IL - CAAM - 236
EE236A (Fall 2007-08)Lecture 16 Large-scale linear programming cutting-plane method Benders decomposition delayed column generation Dantzig-Wolfe decomposition161Cutting-plane methodminimize cT x subject to Ax b A Rmn, m n general idea: solve sequenc
Monmouth IL - CAAM - 236
EE236A (Fall 2007-08)Lecture 17 Integer linear programming integer linear programming, 0-1 linear programming a few basic facts branch-and-bound171Denitioninteger linear program (ILP) minimize cT x subject to Ax b, x ZncGx = dmixed integer linear
Monmouth IL - CAAM - 236
EE236A (Fall 2007-08)Lecture 1 Introduction and overview linear programming example from optimal control example from combinatorial optimization history course topics software11Linear program (LP)nminimizej =1 ncj xj aij xj bi,j =1 nsubject toi
Monmouth IL - CAAM - 236
EE236A (Fall 2007-08)Lecture 2 Linear inequalities vectors inner products and norms linear equalities and hyperplanes linear inequalities and halfspaces polyhedra21Vectors(column) vector x Rn: x1 x x = .2 . xn xi R: ith component or element of x al
Monmouth IL - CAAM - 236
EE236A (Fall 2007-08)Lecture 3 Geometry of linear programming subspaces and ane sets, independent vectors matrices, range and nullspace, rank, inverse polyhedron in inequality form extreme points the optimal set of a linear program31SubspacesS Rn (S
Monmouth IL - CAAM - 236
EE236A (Fall 2007-08)Lecture 4 The linear programming problem variants of the linear programming problem LP feasibility problem examples and some general applications linear-fractional programming41Variants of the linear programming problemgeneral fo
Monmouth IL - CAAM - 236
EE236A (Fall 2007-08)Lecture 5 Structural optimization minimum weight truss design truss topology design limit analysis design with minimum number of bars51Truss m bars with lengths li and cross-sectional areas xi N nodes; nodes 1, . . . , n are free
Monmouth IL - CAAM - 236
EE236A (Fall 2007-08)Lecture 6 FIR lter design FIR lters linear phase lter design magnitude lter design equalizer design61FIR ltersnite impulse response (FIR) lter:n1y (t) = =0h u(t ),tZ u : Z R is input signal ; y : Z R is output signal hi R a
Monmouth IL - CAAM - 236
EE236A (Fall 2007-08)Lecture 7 Applications in control optimal input design robust optimal input design pole placement (with low-authority control)71Linear dynamical systemy (t) = h0u(t) + h1u(t 1) + h2u(t 2) + single input/single output: input u(t)
Monmouth IL - CAAM - 236
EE236A (Fall 2007-08)Lecture 8 Network optimization network ows minimum cost network ow problem extreme ows shortest path problem bipartite matching81Networksnetwork (directed graph): m nodes connected by n directed arcs arcs are ordered pairs (i, j
Monmouth IL - CAAM - 236
EE236A (Fall 2007-08)Lecture 9 Duality (part 1) the dual of an LP in inequality form weak duality examples optimality conditions and complementary slackness Farkas lemma and theorems of alternatives proof of strong duality91The dual of an LP in inequa
Monmouth IL - CAAM - 236
EE236A (Fall 2007-08)Lecture 10 Duality (part 2) duality in algorithms sensitivity analysis via duality duality for general LPs examples mechanics interpretation circuits interpretation two-person zero-sum games101Duality in algorithmsmany algorithms
Monmouth IL - CAAM - 236
EE236A (Fall 2007-08)Lecture 11 The simplex method extreme points adjacent extreme points one iteration of the simplex method degeneracy initialization numerical implementation111Idea of the simplex methodmove from one extreme point to an adjacent ex
Monmouth IL - CAAM - 236
EE236A (Fall 2007-08)Lecture 12 The barrier method brief history of interior-point methods Newtons method for smooth unconstrained minimization logarithmic barrier function central points, the central path the barrier method121The ellipsoid method 19
Monmouth IL - MATH - MATH427
MT 3802 NUMERICAL ANALYSIS2008/2009Dr Clare E Parnell and Dr Stphane Rgnier e eOctober 1, 2008Chapter 0Handout0.1 NotationThroughout this course we will be using scalars, vectors and matrices. It is essential that you know what they are and can tel
Monmouth IL - MATH - MATH427
MT 3802 NUMERICAL ANALYSIS2008/2009Dr Clare E Parnell and Dr Stphane Rgnier e eOctober 15, 2008Contents0 Handout 0.1 0.2 0.3 Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Introduction . . . . . . . . .
Monmouth IL - MATH - MATH427
Chapter 2Iterative Methods2.1 IntroductionIn this section, we will consider three different iterative methods for solving a sets of equations. First, we consider a series of examples to illustrate iterative methods. To construct an iterative method, we
Monmouth IL - MATH - MATH427
cep 2008/09MT3802 - Numerical AnalysisSOLUTIONS - Tutorial Sheet 1 1. Show that the following satisfy the requirements of a norm specified on a Vector Space: (i)bf =a|f (x)|dxwhere f (x) C[a, b] ; Proof that is satisfies Rule 1 (1.1) f = 0 &amp; conti
Monmouth IL - MATH - MATH427
cep 2008/09MT3802 - Numerical AnalysisSOLUTIONS - Tutorial Sheet 2 1. For a sub-ordinate matrix norm and an invertible matrix A, where Ax = b, show that (i) As AssNUsing Rule 5 (1.9) of sub-ordinate matrix norms we have As A As-1 A2As-2 As-1A As
Monmouth IL - MATH - MATH427
Monmouth IL - MATH - MATH427
Monmouth IL - MATH - MATH427
cep 2008/09MT3802 - Numerical AnalysisTutorial Sheet 1 1. Show that the following satisfy the requirements of a norm specified on a Vector Space: (i)bf =a|f (x)|dxwhere f (x) C[a, b] ;(ii) x = max |xi |1inwhere x Rn .2. An inner product is defi
Monmouth IL - MATH - MATH427
cep 2008/09MT3802 - Numerical AnalysisTutorial Sheet 2 1. For a sub-ordinate matrix norm and an invertible matrix A, where Ax = b, show that (i) (ii) As AssN1 x A-1 A b2. Assume that (A + A)-1 is computed as an approximation to A-1 . Show that (A +
Monmouth IL - MATH - MATH427
Monmouth IL - MATH - MATH427
Monmouth IL - MATH - MATH427
cep 2008/09MT3802 - Numerical AnalysisTutorial Sheet 5 1. Let p1 (x) be the linear interpolant of f (x) at the points (x0 , f (x0 ) and (x1 , f (x1 ), where x1 - x0 = h. Show that for x [x0 , x1 ] f (x) - p1 (x) where f (x) M h2 8 M.Suppose ex on [-
Monmouth IL - CS - CS 399
ScriptEase: A Pattern Based Approach to ScriptingCurtis Onuczko Thursday, January 19, 2006Overview What is ScriptEase? Why use ScriptEase? When to use ScriptEase? How to use ScriptEase Tips and TricksWhat is ScriptEase?Why use ScriptEase?A tool to
Monmouth IL - CS - CS 399
Overview Game Creation ProcessCMPUT 299 Winter 2006 Thursday January 12 ! Making a computer game is a lengthy process, involving many creative people! High-profile productions often take 2-3 years ! Challenging games can involve &gt; 100 people!! This lec
Monmouth IL - CS - CS 399
convexity (series) new modes: adding multiplayerGame DesignCMPUT 299Finnegan Southey XBox Live GamerTag: Aleafeatures qbilities, training, introducing abilities critical gaming save dev qa coverage/replay approachable doors &amp; keys consuming items s
Monmouth IL - CS - CS 399
Lecture Review Sheet: Game Design Finnegan Southey CMPUT 299 Winter 2006 Terms and Definitions MDA n A model for game design that views the user experience as a three part process: mechanics, dynamics, and aesthetics Funativity n A model for types of &quot;fun
Monmouth IL - CS - CS 399
Overview! AnnouncementsWhat have we learned thus far?CMPUT 299 Fall 2005 Tuesday, January 31! ! ! !Design Document next Thursday Meet with producer beforehand BioWare internships (CS only) Lab this week/lab exam! NWN Module Discussion ! Computation
Monmouth IL - CS - CS 399
Management is Fundamentals of Project ManagementCMPUT 299 H. James Hoover Winter 2005 2005-09-22 Version 1.21 CMPUT 299 - Winter 2005 Project Management People Process Problem Solving2People Identify skills - have and missing Identify personalities
Monmouth IL - CS - CS 399
OpeningNarrative:Concepts of Narrative in Interactive New Media Art FormsLecture Goals for the next two classes1. Introduce concepts of traditional narrative, most of which apply to later technologies. 2. Discuss the application of these concepts to n
Monmouth IL - CS - CS 399
Meetings, Bloody Meetings How To Run MeetingsCMPUT 299 H. James Hoover Fall 2005 2005-09-23,27 Version 1.0CMPUT 299 - Fall 2005 How to Run Meetings Written by John Cleese and Antony Jay Starring John Cleese and Robert Hardy Video Arts, 1993 34 minMee
Monmouth IL - CS - CS 399
The Sims 2 More Post-Mortems, SharewareCMPUT 299 Winter 2005 February 16CMPUT 299 - Fall 2005 Game PostmortemsThe Sims 2! What went right! ! ! ! ! Prototyping Understanding the audience Kleenex testing Custom content and community SWAT TeamsCMPUT 29
Monmouth IL - CS - CS 399
Lecture Overview Artificial Intelligence Part ICMPUT 299 Winter 2006 February 28, 2006 ! Lab Exam! Course Evals! Design Issue Presentations ! Artificial Intelligence! Definition! Related concepts! Algorithm ! Time/Memory Cost! Finite State Machines
Monmouth IL - CS - CS 399
Lecture Overview Artificial Intelligence Part IICMPUT 299 Winter 2006 March 2, 2006 ! ! ! ! ! Lab Exam Pizza! Summary (Tuesday) Pathfinding Finite State MachinesCMPUT 299 - Fall 2005 Artificial IntelligenceLab Exam ScoresA+ A B CPizza!! Get together
Monmouth IL - CS - CS 399
The Early Days Graphics and GenresCMPUT 299 ! The very earliest video games (pre-1975) were custom built machines. ! Designed/built by engineers (like a TV).Tennis for Two(Brookhaven Labs,1958)Finnegan Southey XBox Live GamerTag: AleaCMPUT 299 - Fall
Monmouth IL - CS - CS 399
Lecture Review Sheet: Artificial Intelligence Nathan Sturtevant CMPUT 299 Winter 2006 Terms and Definitions Intelligence is usually said to involve mental capabilities such as the ability to reason, plan, solve problems, think abstractly, comprehend ideas
Monmouth IL - CS - CS 399
Lecture Review Sheet: The culture of Gaming Sean Gouglas CMPUT 299 Winter 2006Main Ideas and Concepts A. Introduction Culture describes social actors, the webs of significance that allow for social actors to communicate and understand each other, and the
Canterbury Christ Church University - ENGR - 102
Rochester College - CS - CSC400
THE DIGITAL MILLENNIUM COPYRIGHT ACT OF 1998U.S. Copyright Office SummaryDecember 1998INTRODUCTIONThe Digital Millennium Copyright Act (DMCA)1 was signed into law by President Clinton on October 28, 1998. The legislation implements two 1996 World Inte
National University of Juridical Sciences - ACC - 640M
National University Course Outline ACC640M Governmental and Non-profit Accounting Please Note: We will be using e-college. All handouts, and other class materials are available in e-college under DOC Sharing Tab at top of page. Make copies of your handout
University of Florida - CHM - 0808
University of Florida - CHM - 0808
University of Florida - CHM - 0808
University of Florida - CHM - 0808
University of Florida - CHM - 0808
University of Florida - CHM - 0808
University of Florida - CHM - 0808
University of Florida - CHM - 0808
University of Florida - CHM - 0808
University of Florida - CHM - 0808
University of Florida - CHM - 0808
University of Florida - CHM - 0808
University of Florida - CHM - 0808
University of Florida - CHM - 0808
University of Florida - CHM - 0808
University of Florida - CHM - 0808