Nonlinear Programming
1
The problem and the solution concept
We are interested in solving a general constrained optimization problem
minimize
f
(
x
) subject to
x
∈
C,
(1)
where
f
is the
objective function
and
C
⊂
R
N
is the
constraint set
. Such an
optimization problem is called a
linear programming problem
when both the ob
jective function and the constraint are linear. Otherwise, it is called a
nonlinear
programming problem
. If both the objective function and the constraint happen
to be convex, it is called a
convex programming problem
. Thus
Linear Programming
⊂
Convex Programming
⊂
Nonlinear Programming
.
We focus on minimization because maximizing
f
is the same as minimizing
−
f
. If
f
is continuous and
C
is compact, by the BolzanoWeierstrass theorem we
know there is a solution, but the theorem does not tell you how to compute it. In
this section you will learn how to derive necessary conditions for optimality for
general nonlinear programming problems. Oftentimes, the necessary conditions
alone will pin down the solution to a few candidates, so you only need to compare
these candidates.
¯
x
∈
C
is said to be a
global
solution if
f
(
x
)
≥
f
(¯
x
) for all
x
∈
C
.
¯
x
is a
local solution
if there exists
ϵ
>
0 such that
f
(
x
)
≥
f
(¯
x
) whenever
x
∈
C
and
∥
x
−
¯
x
∥
<
ϵ
. If ¯
x
is a global solution, clearly it is also a local solution.
2
Tangent cone and normal cone
Let ¯
x
∈
C
be any point. The
tangent cone
of
C
at ¯
x
is defined by
T
C
(¯
x
) =
y
∈
R
N
(
∃
)
{
α
k
}
≥
0
,
{
x
k
}
⊂
C, y
= lim
k
→∞
α
k
(
x
k
−
¯
x
)
.
That is,
y
∈
T
C
(¯
x
) if
y
points to the same direction as the the limiting direction
of
{
x
k
−
¯
x
}
.
Intuitively, the tangent cone of
C
at ¯
x
consists of all directions
that can be approximated by that from ¯
x
to another point in
C
.
Lemma 1.
T
C
(¯
x
)
is a nonempty closed cone.
Proof.
Setting
α
k
= 0 for all
k
we get 0
∈
T
C
(¯
x
).
If
y
∈
T
C
(¯
x
), then
y
=
lim
α
k
(
x
k
−
¯
x
) for some
{
α
k
}
≥
0 and
{
x
k
}
⊂
C
.
Then for
β
≥
0 we have
β
y
= lim
βα
k
(
x
k
−
¯
x
)
∈
T
C
(¯
x
), so
T
C
(¯
x
) is a cone.
To show that
T
C
(¯
x
) is
closed, let
{
y
l
}
⊂
T
C
(¯
x
) and
y
l
→
¯
y
. For each
l
we can take a sequence such
1
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
2014W Econ 172B
Operations Research (B)
Alexis Akira Toda
that
α
k,l
≥
0, lim
k
→∞
x
k,l
= ¯
x
, and
y
l
= lim
k
→∞
α
k,l
(
x
k,l
−
¯
x
).
Hence we
can take
k
l
such that
∥
x
k
l
,l
−
¯
x
∥
<
1
/l
and
∥
y
l
−
α
k
l
,l
(
x
k
l
,l
−
¯
x
)
∥
<
1
/l
. Then
x
k
l
,l
→
¯
x
and
∥
¯
y
−
α
k
l
,l
(
x
k
l
,l
−
¯
x
)
∥ ≤ ∥
¯
y
−
y
l
∥
+
∥
y
l
−
α
k
l
,l
(
x
k
l
,l
−
¯
x
)
∥ →
0
,
so ¯
y
∈
T
C
(¯
x
).
The dual cone of
T
C
(¯
x
) is called the
normal cone at
¯
x
and is denoted by
N
C
(¯
x
). By the definition of the dual cone, we have
N
C
(¯
x
) =
z
∈
R
N
(
∀
y
∈
T
C
(¯
x
))
⟨
y, z
⟩ ≤
0
.
This is the end of the preview.
Sign up
to
access the rest of the document.
 Winter '08
 Foster,C
 Optimization, tc, lagrange multipliers, Alexis Akira Toda, tangent cone, Akira Toda

Click to edit the document details