# Register now to access 7 million high quality study materials (What's Course Hero?) Course Hero is the premier provider of high quality online educational resources. With millions of study documents, online tutors, digital flashcards and free courseware, Course Hero is helping students learn more efficiently and effectively. Whether you're interested in exploring new subjects or mastering key topics for your next exam, Course Hero has the tools you need to achieve your goals.

12 Pages

### barrier

Course: CAAM 236, Spring 2007
School: Monmouth IL
Rating:

Word Count: 1452

#### Document Preview

(Fall EE236A 2007-08) Lecture 12 The barrier method brief history of interior-point methods Newtons method for smooth unconstrained minimization logarithmic barrier function central points, the central path the barrier method 121 The ellipsoid method 1972: ellipsoid method for (nonlinear) convex nondierentiable optimization (Nemirovsky, Yudin, Shor) 1979: Khachiyan proves that the ellipsoid method...

Register Now

#### Unformatted Document Excerpt

Coursehero >> Illinois >> Monmouth IL >> CAAM 236

Course Hero has millions of student submitted documents similar to the one
below including study guides, practice problems, reference materials, practice exams, textbook help and tutor support.

Course Hero has millions of student submitted documents similar to the one below including study guides, practice problems, reference materials, practice exams, textbook help and tutor support.
(Fall EE236A 2007-08) Lecture 12 The barrier method brief history of interior-point methods Newtons method for smooth unconstrained minimization logarithmic barrier function central points, the central path the barrier method 121 The ellipsoid method 1972: ellipsoid method for (nonlinear) convex nondierentiable optimization (Nemirovsky, Yudin, Shor) 1979: Khachiyan proves that the ellipsoid method applied to LP has polynomial worst-case complexity much slower in practice than simplex very dierent approach from simplex method; extends gracefully to nonlinear convex problems solved important open theoretical problem (polynomial-time algorithm for LP) The barrier method 122 Interior-point methods early methods (1950s1960s) methods for solving convex optimization problems via sequence of smooth unconstrained problems logarithmic barrier method (Frisch), sequential unconstrained minimization (Fiacco & McCormick), ane scaling method (Dikin), method of centers (Huard & Lieu) no worst-case complexity theory; (often) worked well in practice fell out of favor in 1970s new methods (1984) 1984 Karmarkar: new polynomial-time method for LP (projective algorithm) later recognized as closely related to earlier interior-point methods many variations since 1984; widely believed to be faster than simplex for very large problems (over 10,000 variables/constraints) The barrier method 123 Gradient and Hessian dierentiable function f : Rn R gradient and Hessian (evaluated at x): f (x) x1 f (x) x2 f (x) = . . f (x) xn , 2 f (x) = 2 f (x) x2 1 2 f (x) x2 x1 . . 2 f (x) x1 x2 2 f (x) x2 2 ... . . 2 f (x) x1 xn 2 f (x) x2 xn . . 2 f (x) xn x1 2 f (x) xn x2 2 f (x) x2 n 2nd order Taylor series expansion around x: 1 f (y ) f (x) + f (x)T (y x) + (y x)T 2f (x)(y x) 2 The barrier method 124 Positive semidenite matrices a quadratic form is a function f : Rn R with n f (x) = x Ax = i,j =1 T Aij xixj may as well assume A = AT since xT Ax = xT ((A + AT )/2)x A = AT is positive semidenite if xT Ax 0 for all x A = AT is positive denite if xT Ax > 0 for all x = 0 The barrier method 125 Convex dierentiable functions f : Rn R is convex if for all x and y 01 = f (x + (1 )y ) f (x) + (1 )f (y ) f : Rn R is strictly convex if for all x and y 0<<1 = f (x + (1 )y ) < f (x) + (1 )f (y ) for dierentiable f : 2f (x) positive semidenite 2f (x) positive denite for convex dierentiable f : f (x) = 0 x = argmin f = f convex f strictly convex f strictly convex argmin f is unique (if it exists) The barrier method 126 Pure Newton method algorithm for minimizing convex dierentiable f : x+ = x 2f (x)1f (x) x+ minimizes 2nd order expansion of f (y ) at x: 1 f (x) + f (x)T (y x) + (y x)T 2f (x)(y x) 2 2nd-order approx. of f d d d fd x The barrier method d d x+ 127 x+ solves linearized optimality condition: 0 = f (x) + 2f (x)(y x) 1st-order approx. of f d d d 0 f x x+ intepretations suggest method works very well near optimum The barrier method 128 Global behavior pure Newton method can diverge example: f (x) = log(ex + ex), start at x(0) = 1.1 f (x) 3.5 1 f (x) 0.8 0.6 3 2.5 0.4 0.2 2 0 0.2 1.5 0.4 0.6 1 0.8 1 0.5 3 2 1 x 0 1 2 3 3 2 1 x 0 1 2 3 k 1 2 3 4 5 The barrier method x(k) 1.129 100 1.234 100 1.695 100 5.715 100 2.302 104 f (x(k)) f 5.120 101 5.349 101 6.223 101 1.035 100 2.302 104 129 Newton method with exact line search given starting point x repeat 1. Compute Newton direction v = 2f (x)1f (x) 2. Line search. Choose a step size t t = argmint>0 f (x + tv ) 3. Update. x := x + tv until stopping criterion is satised globally convergent very fast local convergence (more later) The barrier method 1210 Logarithmic barrier function minimize cT x subject to aT x bi, i = 1, . . . , m i assume strictly feasible: {x | Ax < b} = m log(bi aT x) Ax < b i dene logarithmic barrier (x) = i=1 + otherwise as x approaches boundary of {x | Ax < b} The barrier method 1211 Derivatives of barrier function m (x) = i=0 m 1 a = AT d Tx i bi ai 2(x) = 1 aiaT = AT diag(d)2A (bi aT x)2 i i i=1 where d = (1/(b1 aT x), . . . , 1/(bm aT x)) m 1 is smooth on C = {x | Ax < b} is convex on C : for all y Rn, y T 2(x)y = y T A diag(d)2Ay = diag(d)Ay strictly convex if rank A = n The barrier method 1212 2 0 The analytic center argmin (if it exists) is called analytic center of inequalities optimality conditions: m (x) = i=1 1 = ai 0 bi aT x i exists if and only if C = {x | Ax < b} is bounded unique if A has rank n dierent descriptions of the same polyhedron may have dierent analytic centers (e.g., adding redundant inequalities moves analytic center) eciently computed via Newtons method (given strictly feasible starting point) The barrier method 1213 Force eld interpretation associate with constraint aT x bi, at point x, the force i Fi = ai bi aT x i Fi points away from constraint plane Fi = 1/dist(x, constraint plane) is potential of 1/r force eld associated with each constraint plane Fi forces balance at analytic center The barrier method 1214 Central path x(t) = argmin(tcT x + (x)) for t > 0 x (we assume minimizer exists and is unique) curve x(t) for t 0 called central path can compute x(t) by solving smooth unconstrained minimization problem (given a strictly feasible starting point) t gives relative weight of objective and barrier barrier traps x(t) in strictly feasible set intuition suggests x(t) converges to optimal as t x(t) characterized by tc + 1 ai = 0 bi aT x(t) i i=1 m The barrier method 1215 example c t = 100 t = 10 t=1 rxac The barrier method 1216 Force eld interpretation imagine a particle in C , subject to forces ith constraint generates constraint force eld Fi(x) = 1 ai bi aT x i is potential associated with constraint forces constraint forces push particle away from boundary of feasible set superimpose objective force eld F0(x) = tc pulls particle toward small cT x t scales objective force at x(t), constraint forces balance objective force; as t increases, particle is pulled towards optimal point, trapped in C by barrier potential The barrier method 1217 Central points and duality recall x = x(t) satises m c+ i=1 ziai = 0, zi = 1 >0 t(bi aT x) i i.e., z is dual feasible and p bT z = cT x + i zi(aT x bi) = cT x m/t i summary: a point on central path yields dual feasible point and lower bound: cT x(t) p cT x(t) m/t (which proves x(t) becomes optimal as t ) The barrier method 1218 Central path and complementary slackness optimality conditions: x optimal Ax b and z s.t. z 0, AT z + c = 0, zi(bi aT x) = 0 i centrality conditions: x is on central path Ax < b and z , t > 0 s.t. z 0, AT z + c = 0, zi(bi aT x) = 1/t i for t large, x(t) almost satises complementary slackness central path is continuous deformation of complementary slackness condition The barrier method 1219 Unconstrained minimization method given strictly feasible x, desired accuracy > 0 1. t := m/ 2. compute x(t) starting from x 3. x := x(t) computes -suboptimal point on central path (and dual feasible z ) solve constrained problem via Newtons method works, but can be slow The barrier method 1220 Barrier method given strictly feasible x, t > 0, tolerance > 0 repeat 1. compute x(t) starting from x, using Newtons method 2. x := x(t) 3. if m/t , return(x) 4. increase t also known as SUMT (Sequential Unconstrained Minimization Technique) generates sequence of points on central path simple updating rule for t: t+ = t (typical values 10 100) steps 14 above called outer iteration; step 1 involves inner iterations (e.g., Newton steps) tradeo: small = few inner iters to compute x(k+1) from x(k), but more outer iters The barrier method 1221 Example minimize cT x subject to Ax b A R10050, Newton with exact line search 1000 100 10 duality gap width of steps shows #Nt. iters per outer iter; height of steps shows reduction in dual. gap (1/) gap reduced by 105 in few tens of Newton iters gap decreases geometrically 10 20 1 0.1 0.01 0.001 0.0001 0 = 50 = 180 = 3 30 40 50 60 can see trade-o in choice of 70 total # Newton iters The barrier method 1222 example continued . . . trade-o in choice of : #Newton iters required to reduce duality gap by 106 90 total # Newton iters 80 70 60 50 40 30 0 20 40 60 80 100 120 140 160 180 200 works very well for wide range of The barrier method 1223 Phase I to compute strictly feasible point (or determine none exists) set up auxiliary problem: minimize w subject to Ax b + w1 easy to nd strictly feasible point (hence barrier method can be used) can use stopping criterion with target value 0 if we include constraint on cT x, minimize w subject to Ax b + w1 cT x M phase I method yields point on central path of original problem many other methods for nding initial primal (& dual) strictly feasible points The barrier method 1224
Find millions of documents on Course Hero - Study Guides, Lecture Notes, Reference Materials, Practice Exams and more. Course Hero has millions of course specific materials providing students with the best way to expand their education.

Below is a small sample set of documents:

Monmouth IL - MATH - MATH427
MT 3802 NUMERICAL ANALYSIS2008/2009Dr Clare E Parnell and Dr Stphane Rgnier e eOctober 1, 2008Chapter 0Handout0.1 NotationThroughout this course we will be using scalars, vectors and matrices. It is essential that you know what they are and can tel
Monmouth IL - MATH - MATH427
MT 3802 NUMERICAL ANALYSIS2008/2009Dr Clare E Parnell and Dr Stphane Rgnier e eOctober 15, 2008Contents0 Handout 0.1 0.2 0.3 Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Introduction . . . . . . . . .
Monmouth IL - MATH - MATH427
Chapter 2Iterative Methods2.1 IntroductionIn this section, we will consider three different iterative methods for solving a sets of equations. First, we consider a series of examples to illustrate iterative methods. To construct an iterative method, we
Monmouth IL - MATH - MATH427
cep 2008/09MT3802 - Numerical AnalysisSOLUTIONS - Tutorial Sheet 1 1. Show that the following satisfy the requirements of a norm specified on a Vector Space: (i)bf =a|f (x)|dxwhere f (x) C[a, b] ; Proof that is satisfies Rule 1 (1.1) f = 0 &amp; conti
Monmouth IL - MATH - MATH427
cep 2008/09MT3802 - Numerical AnalysisSOLUTIONS - Tutorial Sheet 2 1. For a sub-ordinate matrix norm and an invertible matrix A, where Ax = b, show that (i) As AssNUsing Rule 5 (1.9) of sub-ordinate matrix norms we have As A As-1 A2As-2 As-1A As
Monmouth IL - MATH - MATH427
Monmouth IL - MATH - MATH427
Monmouth IL - MATH - MATH427
cep 2008/09MT3802 - Numerical AnalysisTutorial Sheet 1 1. Show that the following satisfy the requirements of a norm specified on a Vector Space: (i)bf =a|f (x)|dxwhere f (x) C[a, b] ;(ii) x = max |xi |1inwhere x Rn .2. An inner product is defi
Monmouth IL - MATH - MATH427
cep 2008/09MT3802 - Numerical AnalysisTutorial Sheet 2 1. For a sub-ordinate matrix norm and an invertible matrix A, where Ax = b, show that (i) (ii) As AssN1 x A-1 A b2. Assume that (A + A)-1 is computed as an approximation to A-1 . Show that (A +
Monmouth IL - MATH - MATH427
Monmouth IL - MATH - MATH427
Monmouth IL - MATH - MATH427
cep 2008/09MT3802 - Numerical AnalysisTutorial Sheet 5 1. Let p1 (x) be the linear interpolant of f (x) at the points (x0 , f (x0 ) and (x1 , f (x1 ), where x1 - x0 = h. Show that for x [x0 , x1 ] f (x) - p1 (x) where f (x) M h2 8 M.Suppose ex on [-
Monmouth IL - CS - CS 399
ScriptEase: A Pattern Based Approach to ScriptingCurtis Onuczko Thursday, January 19, 2006Overview What is ScriptEase? Why use ScriptEase? When to use ScriptEase? How to use ScriptEase Tips and TricksWhat is ScriptEase?Why use ScriptEase?A tool to
Monmouth IL - CS - CS 399
Overview Game Creation ProcessCMPUT 299 Winter 2006 Thursday January 12 ! Making a computer game is a lengthy process, involving many creative people! High-profile productions often take 2-3 years ! Challenging games can involve &gt; 100 people!! This lec
Monmouth IL - CS - CS 399
convexity (series) new modes: adding multiplayerGame DesignCMPUT 299Finnegan Southey XBox Live GamerTag: Aleafeatures qbilities, training, introducing abilities critical gaming save dev qa coverage/replay approachable doors &amp; keys consuming items s
Monmouth IL - CS - CS 399
Lecture Review Sheet: Game Design Finnegan Southey CMPUT 299 Winter 2006 Terms and Definitions MDA n A model for game design that views the user experience as a three part process: mechanics, dynamics, and aesthetics Funativity n A model for types of &quot;fun
Monmouth IL - CS - CS 399
Overview! AnnouncementsWhat have we learned thus far?CMPUT 299 Fall 2005 Tuesday, January 31! ! ! !Design Document next Thursday Meet with producer beforehand BioWare internships (CS only) Lab this week/lab exam! NWN Module Discussion ! Computation
Monmouth IL - CS - CS 399
Management is Fundamentals of Project ManagementCMPUT 299 H. James Hoover Winter 2005 2005-09-22 Version 1.21 CMPUT 299 - Winter 2005 Project Management People Process Problem Solving2People Identify skills - have and missing Identify personalities
Monmouth IL - CS - CS 399
OpeningNarrative:Concepts of Narrative in Interactive New Media Art FormsLecture Goals for the next two classes1. Introduce concepts of traditional narrative, most of which apply to later technologies. 2. Discuss the application of these concepts to n
Monmouth IL - CS - CS 399
Meetings, Bloody Meetings How To Run MeetingsCMPUT 299 H. James Hoover Fall 2005 2005-09-23,27 Version 1.0CMPUT 299 - Fall 2005 How to Run Meetings Written by John Cleese and Antony Jay Starring John Cleese and Robert Hardy Video Arts, 1993 34 minMee
Monmouth IL - CS - CS 399
The Sims 2 More Post-Mortems, SharewareCMPUT 299 Winter 2005 February 16CMPUT 299 - Fall 2005 Game PostmortemsThe Sims 2! What went right! ! ! ! ! Prototyping Understanding the audience Kleenex testing Custom content and community SWAT TeamsCMPUT 29
Monmouth IL - CS - CS 399
Lecture Overview Artificial Intelligence Part ICMPUT 299 Winter 2006 February 28, 2006 ! Lab Exam! Course Evals! Design Issue Presentations ! Artificial Intelligence! Definition! Related concepts! Algorithm ! Time/Memory Cost! Finite State Machines
Monmouth IL - CS - CS 399
Lecture Overview Artificial Intelligence Part IICMPUT 299 Winter 2006 March 2, 2006 ! ! ! ! ! Lab Exam Pizza! Summary (Tuesday) Pathfinding Finite State MachinesCMPUT 299 - Fall 2005 Artificial IntelligenceLab Exam ScoresA+ A B CPizza!! Get together
Monmouth IL - CS - CS 399
The Early Days Graphics and GenresCMPUT 299 ! The very earliest video games (pre-1975) were custom built machines. ! Designed/built by engineers (like a TV).Tennis for Two(Brookhaven Labs,1958)Finnegan Southey XBox Live GamerTag: AleaCMPUT 299 - Fall
Monmouth IL - CS - CS 399
Lecture Review Sheet: Artificial Intelligence Nathan Sturtevant CMPUT 299 Winter 2006 Terms and Definitions Intelligence is usually said to involve mental capabilities such as the ability to reason, plan, solve problems, think abstractly, comprehend ideas
Monmouth IL - CS - CS 399
Lecture Review Sheet: The culture of Gaming Sean Gouglas CMPUT 299 Winter 2006Main Ideas and Concepts A. Introduction Culture describes social actors, the webs of significance that allow for social actors to communicate and understand each other, and the
Canterbury Christ Church University - ENGR - 102
Rochester College - CS - CSC400
THE DIGITAL MILLENNIUM COPYRIGHT ACT OF 1998U.S. Copyright Office SummaryDecember 1998INTRODUCTIONThe Digital Millennium Copyright Act (DMCA)1 was signed into law by President Clinton on October 28, 1998. The legislation implements two 1996 World Inte
National University of Juridical Sciences - ACC - 640M
National University Course Outline ACC640M Governmental and Non-profit Accounting Please Note: We will be using e-college. All handouts, and other class materials are available in e-college under DOC Sharing Tab at top of page. Make copies of your handout
University of Florida - CHM - 0808
University of Florida - CHM - 0808
University of Florida - CHM - 0808
University of Florida - CHM - 0808
University of Florida - CHM - 0808
University of Florida - CHM - 0808
University of Florida - CHM - 0808
University of Florida - CHM - 0808
University of Florida - CHM - 0808
University of Florida - CHM - 0808
University of Florida - CHM - 0808
University of Florida - CHM - 0808
University of Florida - CHM - 0808
University of Florida - CHM - 0808
University of Florida - CHM - 0808
University of Florida - CHM - 0808
University of Florida - CHM - 0808
University of Florida - CHM - 0808
University of Florida - CHM - 0808
University of Florida - CHM - 0808
University of Florida - CHM - 0808
University of Florida - CHM - 0808
University of Florida - CHM - 0808
University of Florida - CHM - 0808
University of Florida - CHM - 0808
University of Florida - CHM - 0808
University of Florida - CHM - 0808
University of Florida - CHM - 0808
University of Florida - CHM - 0808
University of Florida - CHM - 0808
University of Florida - CHM - 0808
University of Florida - CHM - 0808