This preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: @ Pergamon 00981354(94)000972 Computers chem. Engng Vol. 19. No. 5, pp. 551—566, 1995
Copyright © 1995 Elscvicr Science Ltd Printed in Great Britain. All rights reserved
00981354/95 $9.5()+().00 GLOBAL OPTIMIZATION OF NONCONVEX NLPs AND
MINLPs WITH APPLICATIONS IN PROCESS DESIGN H. S. RYOO and N. V. SAHINIDIS'j' Department of Mechanical & Industrial Engineering, University of Illinois, UrbanaChampaign,
1206 W. Green Street, Urbana. IL 61801, USA. (Received 10 December 1993; ﬁnal revision received 27 July 1994;
received for publication 9 August 1994) Abstract—This paper presents an algorithm for ﬁnding global solutions of nonconvex nonlinear
programs (NLPs) and mixed—integer nonlinear programs (MINLPs). The approach is based on the
solution of a sequence of convex underestimating subproblems generated by evolutionary subdivision of
the search region. The key components of the algorithm are new optimalitybased and feasibilitybased
range reduction tests. The former use known feasible solutions and perturbation results to exclude
inferior parts of the search region from consideration, while the latter analyze constraints to obtain valid
inequalities. Furthermore. the algorithm integrates these devices with an efﬁcient local search heuristic.
Computational results demonstrate that the algorithm compares very favorably to several other current
approaches when applied to a large collection of global optimization and process design problems. It is
typically faster, requires less storage and it produces more accurate results. 1. INTRODUCTION Realistic treatments of physical and engineering
systems frequently involve nonlinear models. Model
nonlinearities give rise to nonconvexities which, in
turn, lead to multiple local optima. Unfortunately,
once a local minimum is found, the question of
global optimality is often ignored by optimization
practitioners. The reason is not so much a lack of
understanding of the phenomena or absence of
modeling techniques, but rather the absence of
eﬂicient global optimization algorithms. The optimi
zation community seems to be just now starting to
explore the subject of continuous global optimiza
tion in a systematic way. Although papers had
addressed this topic sporadically since the 1960s
(Tuy, 1964; Falk and Soland, 1969), it was not until
very recently that the ﬁrst systematic exposition of
deterministic global optimization methods appeared
(Horst and Tuy, 1990) and the ﬁrst journal specializ
ing on global optimization was established (Journal
of Global Optimization, Kluwer Academic
Publishers, 1991). In the process design literature,
global optimization questions were addressed in
some of the early works (e.g. Stephanopoulos and
Westerberg, 1975; Westerberg and Shah, 1978;
Wang and Luus, 1978). More recent papers have
proposed a variety of heuristics (Kocis and
Grossmann, 1988; Floudas et al., 1989; Salcedo,
1992) and exact algorithms (Swaney, 1990; Floudas
and Visweswaran, 1990; Manousiouthakis and
Sourlas, 1992; Quesada and Grossmann, 1993). T Author for correspondence. 551 Global optimization techniques can be classiﬁed
as stochastic or deterministic. Stochastic methods,
such as simulated annealing and MonteCarlo mini
mization and their variations, do not make any
assumptions for the problem functions. The meth
ods involve random elements in their search pro
cedure and converge to the global optimum with a
probability approaching one as their running time
goes to inﬁnity (Tom and Zilinskas, 1989; Schoen,
1991). Deterministic approaches, on the other hand,
take advantage of the mathematical structure of the
problem and often guarantee ﬁnite convergence
within a prespeciﬁed level of accuracy (Horst and
Tuy, 1993). Deterministic approaches include
branchand—bound, cutting plane algorithms and
decomposition schemes. These methods were
recently reviewed by Horst (1990) and Horst and
Tuy (1993). Following the development of branchandbound
methods for integer programs, application of the
same principles was suggested for continuous global
optimization problems (Falk and Soland, 1969;
McCormick, 1972a, b, 1983). Branchandbound
methods employ lower and upper bounds of the
optimal objective function value over subregions of
the search space. Certain subregions are dynami—
cally reﬁned while others are excluded from con
sideration based on optimality and feasibility cri
teria. Of critical importance to the success of such
methods is the accuracy (tightness) of the bounding
procedures used. The bounds are developed over a
range of values of the variables involved and
become tighter as the search is conﬁned to smaller 552 Objective Objective Variable _> Variable p (a) Lower bounding (b) Upper bounding Objective P 0
@ fathom U; ..... /
L: G Variable . .
sudeVide (C) Domain subdivision ((1) Search tree Fig. l. Graphical interpretation of branchandbound for
continuous global optimization. subregions. The fundamental ideas of the approach
are illustrated in Fig. 1. First, a relaxation of the
nonconvex problem is constructed. A relaxation R
of a nonconvex minimization problem P is obtained
by enlarging the feasible region and/or underesti
mating the objective function of P. The relaxation is
constructed in such a way that the difference
between the optimal objective function values of
problems P and P is a nonincreasing function of the
size of the feasible region over which the relaxation
is developed. Moreover, the relaxations typically
used can be solved to their respective global minima
by conventional minimization techniques (they are
usually convex programs). Once the relaxed prob
lem is solved, a valid lower bound L is thus obtained
for the global minimum (Fig. la). Using the relaxed
solution as a starting point (if some other more
advantageous starting point is unavailable), local
minimization techniques can be used to yield a valid
upper bound, U, for P (Fig. 1b). At this stage the
global minimum is known to be between L and U. If
L is sufﬁciently close to U, the algorithm terminates.
If not, the feasible region is subdivided into parts——
e.g. into two parts: one to the right and one to the
left of the relaxed solution. A new relaxed problem
is solved for each subdivision. This time, the relaxa—
tion becomes tighter (by construction) and the lower
bound comes closer to the upper bound (Fig. 1c).
The process is then repeated for each subdivision
until the subdivisions possess lower bounds that
either exceed or are sufﬁciently close to the best
found feasible solution of problem P. This leads to a H. S. Rvoo and N. V. SAHINIDIS method for searching over a tree whose nodes corre—
spond to relaxed problems (Fig. 1d). At any time
during the search, parts of the search region (tree
nodes) can be excluded from further consideration
by comparing their respective lower bounds to the
current upper bound (e.g. node R2 in Fig. 1d). Employing branchandbound techniques similar
to the one described above in ﬁnding global minima
of integer programs has consituted an important
paradigm for the optimization community during
the last two decades. It is now well understood that
these problems can be solved to global optimality
provided special provisions are taken at the level of
modeling and algorithm design (Nemhauser and
Wolsey, 1988). Tight formulations are needed and
the algorithms must be designed to exploit the
special mathematical structure of the problems
under consideration (e.g. Sahinidis and Grossmann,
19913, 1992). Although problems involving several
hundreds and even thousands of integer variables
can be solved to global optimality using today’s
algorithms and conventional computer technology,
a great difﬁculty seems to arise in the context of
continuous problems. Continuous problems solved
to proven global optimality have so far typically
involved only a few constraints and variables (see
above cited references). Due to the large number of
applications of continuous nonconvex program
ming, it is then clear that further algorithmic
advances are needed. The purpose of this paper is to present a new
banchandbound—based method for discrete/
continuous global optimization. The key compo
nents of the method are tests for reducing the ranges
of the problem variables. Applied at each node of
the search tree, these range reduction tests yield a
branchandreduce algorithm for global optimiza
tion. The paper is organized as follows: Section 2
presents the main algorithm. Section 3 develops
optimalitybased and feasibility—based range reduc
tion tests. Computational results with the branch
andreduce algorithm are presented in Section 4.
They involve a collection of several global optimiza
tion and process design problems. Conclusions are
drawn in Section 5 and some future research direc
tions are identiﬁed. Finally. to facilitate testing of
global optimization algorithms, the test problems
used in our study and complete information on their
solution are provided in the Appendices. 2. THE BRANCHANDREDUCE GLOBAL OPTIMIZATION
ALGORITHM Although good, sometimes optimal, solutions are
usually found in early stages during the search, Global optimization of nonconvex NLPs and MlNLPs 553 branchandbound algorithms may take a long time
to verify optimality. The key element of the pro
posed algorithm is the introduction of devices for
accelerating the convergence of a branchandbound
algorithm. It should be noted that these devices can
be incorporated in the context of other algorithms as
well. Branchandbound is used here as the basis of
the suggested approach as it is a well established and
usually the preferred solution method for combina
torial global optimization problems. The global
minimization problem considered here is the follow ing: Problem P:
minf(x) s.t. g(x)<0, x EX
where
f: X—>9]t, g: X—>gt"" and ngt'“. Let R be a relaxation of P. The formal statement
of the proposed branchandreduce algorithm (orig
inally sketched in Sahinidis, 1992a, b) is as follows: 1. INITIALIZATION. Put R on a list 9’ of sub
problems. Select the convergence parameter a.
Select MAXSOLVE: the maximum number of
times a node is allowed to be solved in any
iteration. Set the lower bound L= — 00 , and
the upper bound U: + 00 . Goto 2. 2. SUBPROBLEM SELECTION. If 9’ = @, stop:
the current best solution is optimal. Else,
choose R, from 9’ according to a subproblem
selection rule (node selection rule) and set
9):: 9P\{R,} (i.e., delete R, from 9?). Set
SOLVE=I and Goto 3. 3. PREPROCESSING (optional). Use optimality—
based and feasibilitybased tests to tighten
variable bounds as much as possible for sub
problem Ri. Goto 4. 4. LOWER BOUNDING. Solve R,, or bound its
solution from below. Let L, be this lower
bound. If the solution, x‘, found for R, is
feasible for P and f(x')< U update U<—f(x"),
make x' the current best solution, and set
9’[email protected]\{Rj} for all those subproblems R] for
which L12 U—s. If L,2U—s, discard R, and
Goto 2. Else, let L be equal to the smallest of
all L, and Goto 5. 5. UPPER BOUNDING (optional). Do local
search or other heuristic(s) to ﬁnd a feasible
solution for problem P. If found, update U and
the current best solution, and set 9:: 9P\{R,—}
for all those subproblems R, for which
L12 Us. Goto 6. 6. POSTPROCESSING (optional). Use optimal
itybasedandfeasibility—basedteststostrengthen the bounds of as many variables as possible. If
Steps 4 or 5 were successful in improving U,
apply strengthening tests to all subproblems
currently in 9’, otherwise apply strengthening
only to R. If SOLVE<MAXSOLVE and this
step was successful in reducing the variable
bounds for R,, reconstruct R, using the new
bounds, set SOLVE = SOLVE+ 1 and Goto 4.
Else, Goto 7. 7. BRANCHING (separation). Apply a branching
rule to R,», i.e., generate a set of new subprob
lems R,1, R . . , R,q, place them on the list 129  91’, and Goto 2. The algorithm involves a search tree. Simple sub
problems (relaxations) are solved at each node of
the tree in Step 4. Their solution provides lower
bounds, L,—, which can be used to exclude nodes in
the tree from further consideration. This is achieved
by comparing these lower bounds to the best current
upper bound U. At all times the global minimum is
bounded between the lowest lower bound L and the
value U of the best found feasible solution. Without the optional pre and postprocessing
steps, the above algorithm follows closely the origi
nal developments of Falk and Soland (1969) and the
branchandbound algorithms described elsewhere
(e.g. in Horst and Tuy, 1993; McCormick, 1972a,
1976, 1983). Relaxations can be developed in more
than one way. If, for example, the functions are
separable. valid underestimators can be obtained by
summing up underestimators of singlevariable func—
tions. If separability is not present, it can be usually
(although not always) induced to problem P at the
expense of introducing additional variables and con
straints (see McCormick, 1972b). Moreover, it is
often straightforward to develop convex underesti
mators of singlevariable functions in an interval.
For example, if a function is concave, its convex
underestimation in an interval is a straight line. In
any case, the form of the underestimators will
depend on the interval to which the variables are
restricted. As the search proceeds, these intervals
should be made smaller (by the branching rule) so
that the underestimators become more accurate.
Node selection (Step 2) and branching (Step 7) can
also be done in more than one way. The approach
followed here in Step 2 of the algorithm is to always
select the subproblem with the lowest lower bound.
In Step 7, the solution of the relaxed subproblem R,
is followed by branching on the variable x, appearing
in the underestimating term that contributes most
strongly to the deviation of the underestimators
from the original functions at the current solution
point. Two new subproblems are then created: one 554 where x, is restricted to the interval [xiL, xT] and one
with x, in [xT.xTJ]. Here x}. xjU, and xT denote the
lower bound, the upper bound and the solution
value, respectively, for x, in subproblem R,. In case
a local minimum is known with a solution JEje
(xT,x,U), )2, is made the branching point instead of
xT. This makes the underestimators exact at the
candidate solution (Swaney, 1990). Note that, in
order for variable x, to be selected for branching by
the algorithm, there must be a nonconvex term that
is violated by the underestimating functions at xT.
As the underestimators used here are exact at end
points, it follows that, if x, is selected for branching,
we have le<xT<xIU. Therefore, the two intervals
{x}, xT] and [x T, x/U] of the descendant nodes will be
strictly smaller than that of the parent node {x}, x,”].
Also note that the local minimum )2, can be made the
branching point only if it is in the open interval
(xT. xi”), i.e. only if it is stictly between the parents
node’s bounds. in which case the resulting descen
dant nodes” intervals again become smaller. Remark 1. The above algorithm is described with—
out any explicit reference to discrete variables.
However, a binary variable x can be modeled within
a continuous framework using the standard con—
tinuous reformulation Osxs 1, x(1—x)S0.
Alternatively, integer variables can be dealt with
explicitly as done in standard branch—and—bound
algorithms (e.g. Nemhauser and Wolsey, 1988). In
either case, the above algorithm applies equally well
to NLP and MINLP problems. Remark 2. When the relaxations and branching
rules satisfy certain properties, it is well known (e. g.
Horst and Tuy, 1993) that, without the optional pre
and postprocessing steps, the above algorithm
either terminates ﬁnitely or else produces an inﬁnite
sequence of points approaching the eglobal mini
mum of problem P for any e>0. Since the above
algorithm prevents inﬁnite repetition of the optional
steps, the same convergence characteristics will be
preserved when the optional pre and post
processing steps are included, provided that the
range reduction tests are valid, i.e. that these tests
do not exclude the parts of the search region that
contain the global optimum. Valid tests are deve
loped in the following section. 3. RANGE REDUCTION TESTS 3.1. Optimalitybased range reduction tests Consider the relaxed problem solved at the root
node of a branchandreduce tree: Problem R:
minf(x) s.t. g(x)<0, xe X H. S. RYoo and N. V. SAHlNlDIS where
f:X—>9t, g:X—>91t"', XngGJt" and where for any x feasible in P we have f(x)$
f(x), g(x)sg(x). Also consider the following per
turbed problem: Problem R(y): qo(y) = minf(x) s.t. g(x)Sy, xe X. The perturbation function (p(y) has the following
wellknown properties (e.g. Chap. 5 of Minoux,
1986): Theorem 1. Assuming that R has an optimum x*
of ﬁnite value, 1* is a saddlepoint multiplier if and
only if the hyperplane with equation 2 = rp(0) —l*  y
is a supporting hyperplane at y=0 of the graph of
the perturbation function (,0 (y). In this case, we have
(p (y) 2 (p (0) — 1*  y, Vy e 9%“. Moreover, the saddle—
point exists and the perturbation function is convex
if R is a convex problem satisfying standard con—
straint qualiﬁcations. On the basis of Theorem 1, the following result
can be derived: Theorem 2. Assume that R is a convex optimiza
tion problem with an optimal objective function
value of L and assume that the constraint x, —x,U $ 0
is active at the solution of problem R with a multip
lier value of AT>0. Let U be a known upper bound
for problem P. The following test is valid and can
then be incorporated in the pre and postprocessing
steps of the branchandreduce algorthm: U—L
17‘ Test l—Kj=x,u— If ij<K,, then set L
Xi <—K". Proof. Consider the following perturbation prob
lem: Problem P(y): (I>(y) = minf(x) s.t. g(x)$y, x e X. Since, for any y, R (y) is a convex relaxation of P(y),
we have (p(y) S (P(y). Therefore, an underestimator
of cp(y) is also an underestimator of P(y). Consider
now the perturbed problem R(y) where only the
right hand side of constraint x, vx,“ Sy is perturbed.
It follows from Theorem 1 that a valid underestima
tor for (p(y) is L—lTy and hence: L—ATysrp(y)S
(My). By requiring that the value of <1>(y) be better
than the already known upper bound U, we obtain:
L — ATy< U. Finally, consider only nonpositive
values for y. Since xj—xj“Sy is active for y=0, it
will also be active in the solution of R(y) for any Global optimization of nonconvex NLPs and MINLPs 555 y$0, i.e. y=x,—x,”. Substituting this relationship
in L—Afy< U above and rearranging yields: U—L
1)" x, >xlU  E] A geometric interpretation of Test 1 is provided in
Fig. 2. The ﬁgure depicts the perturbation function
(p and its linear underestimator 2. Test 1 excludes all
those values of xj for which the linear understimator
of the relaxed problem exceeds the known upper
bound U. Similarly, a valid test can be derived if the lower
bounding constraint xl—ijO is active at the solu
tion of problem R with a multiplier value of 17>O: U—L
1* i Test 2—Let n,=x,l+ . If xlu>3rj, then set U
x, 675,. Tests 1 and 2 can be applied only if the corres—
ponding bound constraint is active and the corres
ponding dual variable is strictly positive. For vari
ables that are not at their bounds at the relaxed
solution, similar tests can be developed by probing
at the bounds: that is, by temporarily ﬁxing these
variables at their bounds and solving the partially
restricted relaxed problem. This leads to the follow—
ing tests: Test 3—Solve R after setting xi = xf. If 17> 0
continue.
L — U
A:
then set x,U <— 7:1.
Test 4—Solve R after setting xi = xJL. If )lj‘> 0
continue.
L — U
17
then set x,'“<—— K]. Let nj=x,U— .Ifx,“>.7rj, Let xj=x,»L .IijL>:rj, Theorem 3. Tests 2~4 are valid, i.e. they do not
exclude feasible solutions of P with obj...
View
Full Document
 Spring '11
 Staff

Click to edit the document details