{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

Global optimization of nonconvex NLPs and MINLPs(Ryoo and Sahinidis)-95

Global optimization of nonconvex NLPs and MINLPs(Ryoo and Sahinidis)-95

Info icon This preview shows pages 1–16. Sign up to view the full content.

View Full Document Right Arrow Icon
Image of page 1

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Image of page 2
Image of page 3

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Image of page 4
Image of page 5

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Image of page 6
Image of page 7

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Image of page 8
Image of page 9

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Image of page 10
Image of page 11

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Image of page 12
Image of page 13

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Image of page 14
Image of page 15

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Image of page 16
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: @ Pergamon 0098-1354(94)00097-2 Computers chem. Engng Vol. 19. No. 5, pp. 551—566, 1995 Copyright © 1995 Elscvicr Science Ltd Printed in Great Britain. All rights reserved 0098-1354/95 $9.5()+().00 GLOBAL OPTIMIZATION OF NONCONVEX NLPs AND MINLPs WITH APPLICATIONS IN PROCESS DESIGN H. S. RYOO and N. V. SAHINIDIS'j' Department of Mechanical & Industrial Engineering, University of Illinois, Urbana-Champaign, 1206 W. Green Street, Urbana. IL 61801, USA. (Received 10 December 1993; final revision received 27 July 1994; received for publication 9 August 1994) Abstract—This paper presents an algorithm for finding global solutions of nonconvex nonlinear programs (NLPs) and mixed—integer nonlinear programs (MINLPs). The approach is based on the solution of a sequence of convex underestimating subproblems generated by evolutionary subdivision of the search region. The key components of the algorithm are new optimality-based and feasibility-based range reduction tests. The former use known feasible solutions and perturbation results to exclude inferior parts of the search region from consideration, while the latter analyze constraints to obtain valid inequalities. Furthermore. the algorithm integrates these devices with an efficient local search heuristic. Computational results demonstrate that the algorithm compares very favorably to several other current approaches when applied to a large collection of global optimization and process design problems. It is typically faster, requires less storage and it produces more accurate results. 1. INTRODUCTION Realistic treatments of physical and engineering systems frequently involve nonlinear models. Model nonlinearities give rise to nonconvexities which, in turn, lead to multiple local optima. Unfortunately, once a local minimum is found, the question of global optimality is often ignored by optimization practitioners. The reason is not so much a lack of understanding of the phenomena or absence of modeling techniques, but rather the absence of eflicient global optimization algorithms. The optimi- zation community seems to be just now starting to explore the subject of continuous global optimiza- tion in a systematic way. Although papers had addressed this topic sporadically since the 1960s (Tuy, 1964; Falk and Soland, 1969), it was not until very recently that the first systematic exposition of deterministic global optimization methods appeared (Horst and Tuy, 1990) and the first journal specializ- ing on global optimization was established (Journal of Global Optimization, Kluwer Academic Publishers, 1991). In the process design literature, global optimization questions were addressed in some of the early works (e.g. Stephanopoulos and Westerberg, 1975; Westerberg and Shah, 1978; Wang and Luus, 1978). More recent papers have proposed a variety of heuristics (Kocis and Grossmann, 1988; Floudas et al., 1989; Salcedo, 1992) and exact algorithms (Swaney, 1990; Floudas and Visweswaran, 1990; Manousiouthakis and Sourlas, 1992; Quesada and Grossmann, 1993). T Author for correspondence. 551 Global optimization techniques can be classified as stochastic or deterministic. Stochastic methods, such as simulated annealing and Monte-Carlo mini- mization and their variations, do not make any assumptions for the problem functions. The meth- ods involve random elements in their search pro- cedure and converge to the global optimum with a probability approaching one as their running time goes to infinity (Tom and Zilinskas, 1989; Schoen, 1991). Deterministic approaches, on the other hand, take advantage of the mathematical structure of the problem and often guarantee finite convergence within a prespecified level of accuracy (Horst and Tuy, 1993). Deterministic approaches include branch-and—bound, cutting plane algorithms and decomposition schemes. These methods were recently reviewed by Horst (1990) and Horst and Tuy (1993). Following the development of branch-and-bound methods for integer programs, application of the same principles was suggested for continuous global optimization problems (Falk and Soland, 1969; McCormick, 1972a, b, 1983). Branch-and-bound methods employ lower and upper bounds of the optimal objective function value over subregions of the search space. Certain subregions are dynami— cally refined while others are excluded from con- sideration based on optimality and feasibility cri- teria. Of critical importance to the success of such methods is the accuracy (tightness) of the bounding procedures used. The bounds are developed over a range of values of the variables involved and become tighter as the search is confined to smaller 552 Objective Objective Variable _> Variable p (a) Lower bounding (b) Upper bounding Objective P 0 @ fathom U;- ..... / L: G Variable . . sudeVide (C) Domain subdivision ((1) Search tree Fig. l. Graphical interpretation of branch-and-bound for continuous global optimization. subregions. The fundamental ideas of the approach are illustrated in Fig. 1. First, a relaxation of the nonconvex problem is constructed. A relaxation R of a nonconvex minimization problem P is obtained by enlarging the feasible region and/or underesti- mating the objective function of P. The relaxation is constructed in such a way that the difference between the optimal objective function values of problems P and P is a nonincreasing function of the size of the feasible region over which the relaxation is developed. Moreover, the relaxations typically used can be solved to their respective global minima by conventional minimization techniques (they are usually convex programs). Once the relaxed prob- lem is solved, a valid lower bound L is thus obtained for the global minimum (Fig. la). Using the relaxed solution as a starting point (if some other more advantageous starting point is unavailable), local minimization techniques can be used to yield a valid upper bound, U, for P (Fig. 1b). At this stage the global minimum is known to be between L and U. If L is sufficiently close to U, the algorithm terminates. If not, the feasible region is subdivided into parts—— e.g. into two parts: one to the right and one to the left of the relaxed solution. A new relaxed problem is solved for each subdivision. This time, the relaxa— tion becomes tighter (by construction) and the lower bound comes closer to the upper bound (Fig. 1c). The process is then repeated for each subdivision until the subdivisions possess lower bounds that either exceed or are sufficiently close to the best found feasible solution of problem P. This leads to a H. S. Rvoo and N. V. SAHINIDIS method for searching over a tree whose nodes corre— spond to relaxed problems (Fig. 1d). At any time during the search, parts of the search region (tree nodes) can be excluded from further consideration by comparing their respective lower bounds to the current upper bound (e.g. node R2 in Fig. 1d). Employing branch-and-bound techniques similar to the one described above in finding global minima of integer programs has consituted an important paradigm for the optimization community during the last two decades. It is now well understood that these problems can be solved to global optimality provided special provisions are taken at the level of modeling and algorithm design (Nemhauser and Wolsey, 1988). Tight formulations are needed and the algorithms must be designed to exploit the special mathematical structure of the problems under consideration (e.g. Sahinidis and Grossmann, 19913, 1992). Although problems involving several hundreds and even thousands of integer variables can be solved to global optimality using today’s algorithms and conventional computer technology, a great difficulty seems to arise in the context of continuous problems. Continuous problems solved to proven global optimality have so far typically involved only a few constraints and variables (see above cited references). Due to the large number of applications of continuous nonconvex program- ming, it is then clear that further algorithmic advances are needed. The purpose of this paper is to present a new banch-and-bound—based method for discrete/ continuous global optimization. The key compo- nents of the method are tests for reducing the ranges of the problem variables. Applied at each node of the search tree, these range reduction tests yield a branch-and-reduce algorithm for global optimiza- tion. The paper is organized as follows: Section 2 presents the main algorithm. Section 3 develops optimality-based and feasibility—based range reduc- tion tests. Computational results with the branch- and-reduce algorithm are presented in Section 4. They involve a collection of several global optimiza- tion and process design problems. Conclusions are drawn in Section 5 and some future research direc- tions are identified. Finally. to facilitate testing of global optimization algorithms, the test problems used in our study and complete information on their solution are provided in the Appendices. 2. THE BRANCH-AND-REDUCE GLOBAL OPTIMIZATION ALGORITHM Although good, sometimes optimal, solutions are usually found in early stages during the search, Global optimization of nonconvex NLPs and MlNLPs 553 branch-and-bound algorithms may take a long time to verify optimality. The key element of the pro- posed algorithm is the introduction of devices for accelerating the convergence of a branch-and-bound algorithm. It should be noted that these devices can be incorporated in the context of other algorithms as well. Branch-and-bound is used here as the basis of the suggested approach as it is a well established and usually the preferred solution method for combina- torial global optimization problems. The global minimization problem considered here is the follow- ing: Problem P: minf(x) s.t. g(x)<0, x EX where f: X—>9]t, g: X—>gt"" and ngt'“. Let R be a relaxation of P. The formal statement of the proposed branch-and-reduce algorithm (orig- inally sketched in Sahinidis, 1992a, b) is as follows: 1. INITIALIZATION. Put R on a list 9’ of sub- problems. Select the convergence parameter a. Select MAXSOLVE: the maximum number of times a node is allowed to be solved in any iteration. Set the lower bound L= — 00 , and the upper bound U: + 00 . Goto 2. 2. SUBPROBLEM SELECTION. If 9’ = @, stop: the current best solution is optimal. Else, choose R,- from 9’ according to a subproblem selection rule (node selection rule) and set 9):: 9P\{R,-} (i.e., delete R,- from 9?). Set SOLVE=I and Goto 3. 3. PRE-PROCESSING (optional). Use optimality— based and feasibility-based tests to tighten variable bounds as much as possible for sub- problem Ri. Goto 4. 4. LOWER BOUNDING. Solve R,-, or bound its solution from below. Let L,- be this lower bound. If the solution, x‘, found for R, is feasible for P and f(x')< U update U<—f(x"), make x' the current best solution, and set 9’[email protected]\{Rj} for all those subproblems R]- for which L12 U—s. If L,2U—s, discard R,- and Goto 2. Else, let L be equal to the smallest of all L, and Goto 5. 5. UPPER BOUNDING (optional). Do local search or other heuristic(s) to find a feasible solution for problem P. If found, update U and the current best solution, and set 9:: 9P\{R,—} for all those subproblems R, for which L12 U-s. Goto 6. 6. POST-PROCESSING (optional). Use optimal- ity-basedandfeasibility—basedteststostrengthen the bounds of as many variables as possible. If Steps 4 or 5 were successful in improving U, apply strengthening tests to all subproblems currently in 9’, otherwise apply strengthening only to R. If SOLVE<MAXSOLVE and this step was successful in reducing the variable bounds for R,, reconstruct R,- using the new bounds, set SOLVE = SOLVE+ 1 and Goto 4. Else, Goto 7. 7. BRANCHING (separation). Apply a branching rule to R,», i.e., generate a set of new subprob- lems R,-1, R- . . , R,q, place them on the list 129 - 91’, and Goto 2. The algorithm involves a search tree. Simple sub- problems (relaxations) are solved at each node of the tree in Step 4. Their solution provides lower bounds, L,—, which can be used to exclude nodes in the tree from further consideration. This is achieved by comparing these lower bounds to the best current upper bound U. At all times the global minimum is bounded between the lowest lower bound L and the value U of the best found feasible solution. Without the optional pre- and post-processing steps, the above algorithm follows closely the origi- nal developments of Falk and Soland (1969) and the branch-and-bound algorithms described elsewhere (e.g. in Horst and Tuy, 1993; McCormick, 1972a, 1976, 1983). Relaxations can be developed in more than one way. If, for example, the functions are separable. valid underestimators can be obtained by summing up underestimators of single-variable func— tions. If separability is not present, it can be usually (although not always) induced to problem P at the expense of introducing additional variables and con- straints (see McCormick, 1972b). Moreover, it is often straightforward to develop convex underesti- mators of single-variable functions in an interval. For example, if a function is concave, its convex underestimation in an interval is a straight line. In any case, the form of the underestimators will depend on the interval to which the variables are restricted. As the search proceeds, these intervals should be made smaller (by the branching rule) so that the underestimators become more accurate. Node selection (Step 2) and branching (Step 7) can also be done in more than one way. The approach followed here in Step 2 of the algorithm is to always select the subproblem with the lowest lower bound. In Step 7, the solution of the relaxed subproblem R, is followed by branching on the variable x, appearing in the underestimating term that contributes most strongly to the deviation of the underestimators from the original functions at the current solution point. Two new subproblems are then created: one 554 where x,- is restricted to the interval [xi-L, xT] and one with x, in [xT.xTJ]. Here x}. xj-U, and xT denote the lower bound, the upper bound and the solution value, respectively, for x,- in subproblem R,. In case a local minimum is known with a solution JEje (xT,x,-U), )2,- is made the branching point instead of xT. This makes the underestimators exact at the candidate solution (Swaney, 1990). Note that, in order for variable x,- to be selected for branching by the algorithm, there must be a nonconvex term that is violated by the underestimating functions at xT. As the underestimators used here are exact at end- points, it follows that, if x, is selected for branching, we have le<xT<xIU. Therefore, the two intervals {x}, xT] and [x T, x/U] of the descendant nodes will be strictly smaller than that of the parent node {x}, x,”]. Also note that the local minimum )2,- can be made the branching point only if it is in the open interval (xT. xi”), i.e. only if it is stictly between the parents node’s bounds. in which case the resulting descen- dant nodes” intervals again become smaller. Remark 1. The above algorithm is described with— out any explicit reference to discrete variables. However, a binary variable x can be modeled within a continuous framework using the standard con— tinuous reformulation Osxs 1, x(1—x)S0. Alternatively, integer variables can be dealt with explicitly as done in standard branch—and—bound algorithms (e.g. Nemhauser and Wolsey, 1988). In either case, the above algorithm applies equally well to NLP and MINLP problems. Remark 2. When the relaxations and branching rules satisfy certain properties, it is well known (e. g. Horst and Tuy, 1993) that, without the optional pre- and post-processing steps, the above algorithm either terminates finitely or else produces an infinite sequence of points approaching the e-global mini- mum of problem P for any e>0. Since the above algorithm prevents infinite repetition of the optional steps, the same convergence characteristics will be preserved when the optional pre- and post- processing steps are included, provided that the range reduction tests are valid, i.e. that these tests do not exclude the parts of the search region that contain the global optimum. Valid tests are deve- loped in the following section. 3. RANGE REDUCTION TESTS 3.1. Optimality-based range reduction tests Consider the relaxed problem solved at the root node of a branch-and-reduce tree: Problem R: minf(x) s.t. g(x)<0, xe X H. S. RYoo and N. V. SAHlNlDIS where f:X—>9t, g:X—>91t"', XngGJt" and where for any x feasible in P we have f(x)$ f(x), g(x)sg(x). Also consider the following per- turbed problem: Problem R(y): qo(y) = minf(x) s.t. g(x)Sy, xe X. The perturbation function (p(y) has the following well-known properties (e.g. Chap. 5 of Minoux, 1986): Theorem 1. Assuming that R has an optimum x* of finite value, 1* is a saddle-point multiplier if and only if the hyperplane with equation 2 = rp(0) —l* - y is a supporting hyperplane at y=0 of the graph of the perturbation function (,0 (y). In this case, we have (p (y) 2 (p (0) — 1* - y, Vy e 9%“. Moreover, the saddle— point exists and the perturbation function is convex if R is a convex problem satisfying standard con— straint qualifications. On the basis of Theorem 1, the following result can be derived: Theorem 2. Assume that R is a convex optimiza- tion problem with an optimal objective function value of L and assume that the constraint x, —x,-U $ 0 is active at the solution of problem R with a multip- lier value of AT>0. Let U be a known upper bound for problem P. The following test is valid and can then be incorporated in the pre- and post-processing steps of the branch-and-reduce algorthm: U—L 17‘ Test l—Kj=x,u— If ij<K,-, then set L Xi <—K". Proof. Consider the following perturbation prob- lem: Problem P(y): (I>(y) = minf(x) s.t. g(x)$y, x e X. Since, for any y, R (y) is a convex relaxation of P(y), we have (p(y) S (P(y). Therefore, an underestimator of cp(y) is also an underestimator of P(y). Consider now the perturbed problem R(y) where only the right hand side of constraint x, vx,“ Sy is perturbed. It follows from Theorem 1 that a valid underestima- tor for (p(y) is L—lTy and hence: L—ATysrp(y)S (My). By requiring that the value of <1>(y) be better than the already known upper bound U, we obtain: L — ATy< U. Finally, consider only nonpositive values for y. Since xj—xj-“Sy is active for y=0, it will also be active in the solution of R(y) for any Global optimization of nonconvex NLPs and MINLPs 555 y$0, i.e. y=x,-—x,”. Substituting this relationship in L—Afy< U above and rearranging yields: U—L 1)" x,- >xl-U - E] A geometric interpretation of Test 1 is provided in Fig. 2. The figure depicts the perturbation function (p and its linear underestimator 2. Test 1 excludes all those values of xj- for which the linear understimator of the relaxed problem exceeds the known upper bound U. Similarly, a valid test can be derived if the lower bounding constraint xl—ijO is active at the solu- tion of problem R with a multiplier value of 17>O: U—L 1* i Test 2—Let n,=x,l-+ . If xlu>3rj, then set U x,- 675,-. Tests 1 and 2 can be applied only if the corres— ponding bound constraint is active and the corres- ponding dual variable is strictly positive. For vari- ables that are not at their bounds at the relaxed solution, similar tests can be developed by probing at the bounds: that is, by temporarily fixing these variables at their bounds and solving the partially restricted relaxed problem. This leads to the follow— ing tests: Test 3—Solve R after setting xi = xf. If 17> 0 continue. L — U A: then set x,U <— 7:1. Test 4—Solve R after setting xi = xJ-L. If )lj-‘> 0 continue. L — U 17 then set x,'“<—— K]. Let nj=x,U— .Ifx,“>.7rj, Let xj=x,»L- .IijL>:rj, Theorem 3. Tests 2~4 are valid, i.e. they do not exclude feasible solutions of P with obj...
View Full Document

{[ snackBarMessage ]}

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern