Branch and Bound Methods
Stephen Boyd and Jacob Mattingley
Notes for EE364b, Stanford University, Winter 200607
March 11, 2007
Branch and bound
algorithms are methods for global optimization in nonconvex prob
lems [LW66, Moo91].
They are nonheuristic, in the sense that they maintain a provable
upper and lower bound on the (globally) optimal objective value; they terminate with a
certificate proving that the suboptimal point found is
ǫ
suboptimal. Branch and bound al
gorithms can be (and often are) slow, however.
In the worst case they require effort that
grows exponentially with problem size, but in some cases we are lucky, and the methods
converge with much less effort. In these notes we describe two typical and simple examples
of branch and bound methods, and show some typical results, for a minimum cardinality
problem.
1
Unconstrained nonconvex minimization
The material in this section is taken from [BBB91]. The branch and bound algorithm we
describe here finds the global minimum of a function
f
:
R
m
→
R
over an
m
dimensional
rectangle
Q
init
, to within some prescribed accuracy
ǫ
. We let
f
⋆
denote the optimal value,
i.e.
,
f
⋆
= inf
x
∈Q
init
f
(
x
). For a rectangle
Q ⊆ Q
init
we define
Φ
min
(
Q
) = inf
x
∈Q
f
(
x
)
,
so
f
⋆
= Φ
min
(
Q
init
).
The algorithm will use two functions Φ
lb
(
Q
) and Φ
ub
(
Q
) defined for any rectangle
Q ⊆
Q
init
. These functions must satisfy the following conditions. First, they are lower and upper
bounds on Φ
min
(
Q
), respectively: for any
Q ⊆ Q
init
,
Φ
lb
(
Q
)
≤
Φ
min
(
Q
)
≤
Φ
ub
(
Q
)
.
The second condition is (roughly) that the bounds become tight as the rectangle shrinks
to a point. More precisely, as the maximum halflength of the sides of
Q
, denoted by size(
Q
),
goes to zero, the difference between upper and lower bounds uniformly converges to zero,
i.e.
,
∀
ǫ >
0
∃
δ >
0
∀Q ⊆ Q
init
,
size(
Q
)
≤
δ
=
⇒
Φ
ub
(
Q
)
−
Φ
lb
(
Q
)
≤
ǫ.
1
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Figure 1:
Branch and bound example in
R
2
, after 3 iterations. The partition of
the original rectangle is shown at left; the associated binary tree is shown at right.
We mention a third condition, which is not needed to prove convergence of the branch
and bound algorithm, but is needed in practice: The functions Φ
ub
(
Q
) and Φ
lb
(
Q
) should
be cheap to compute.
We now describe the algorithm. We start by computing
L
1
= Φ
lb
(
Q
init
)
,
U
1
= Φ
ub
(
Q
init
)
,
which are lower and upper bounds on
f
⋆
, respectively.
If
U
1
−
L
1
≤
ǫ
, the algorithm
terminates. Otherwise we partition
Q
init
into two rectangles
Q
init
=
Q
1
∪ Q
2
, and compute
Φ
lb
(
Q
i
) and Φ
ub
(
Q
i
),
i
= 1
,
2. Then we have new lower and upper bounds on
f
⋆
,
L
2
= min
{
Φ
lb
(
Q
1
)
,
Φ
lb
(
Q
2
)
} ≤
Φ
min
(
Q
init
)
≤
U
2
= min
{
Φ
ub
(
Q
1
)
,
Φ
ub
(
Q
2
)
}
.
If
U
2
−
L
2
≤
ǫ
, the algorithm terminates. Otherwise, we partition one of
Q
1
and
Q
2
into
two rectangle, to obtain a new partition of
Q
init
into three rectangles, and compute Φ
lb
and
Φ
ub
for these new rectangles. We update the lower bound
L
3
as the minimum over the lower
bounds over the partition of
Q
init
, and similarly for the upper bound
U
3
.
This is the end of the preview.
Sign up
to
access the rest of the document.
 '09
 BOYD,S
 Optimization, lower bound, Upper and lower bounds, Qinit

Click to edit the document details