Optimization problems in mathematics are problems in which we wish to minimize or
maximize some real-valued function relative to some set of arguments. This latter set is
often called the
set of feasible alternaties
or, simply the
feasible set
. The function to be
minimized or maximized is variously called the
cost function
, the
objective function
or the
criterion
.
In these lectures, we will deal almost exclusively with cost functions,
f
o
, which are deﬁned
on subsets of
R
n
. In this case, the feasible set is often deﬁned, at least partially, by a
set of inequalities or equalities. Such explicit constraints are given in terms of a number
of real-valued functions
f
i
:
R
n
→
R
, i
= 1
,
2
, . . . , m
. But it may well be the case that
constraints are given more abstractly in terms of some prescribed subset
K ⊂
R
n
. This
type of description is particularly useful in treating constraints of a geometric type, or, for
example,
range constraints
in which we prescribe feasible points as ones whose components
lie within certain upper and lower bounds, or, quite commonly, to keep the components
of the feasible set non-negative,
x
i
≥
0.
More explicitly, we can write such a problem as
minimize
f
o
(
x
)
subject to
x
∈ K
f
(
x
)
≤
b
g
(
x
) =
c
.
Here, the vector
x
= (
x
1
, x
2
,
···
, x
n
)
>
is the optimization variable of the problem
1
. The
function
f
o
:
R
n
→
R
is the cost function, the components of the function
f
:
R
n
→
R
m
,
namely
f
i
:
R
n
→
R
, are constraint functions describing inequality constraints, and the
component functions,
g
i
:
R
n
→
R
, of
g
:
R
n
→
R
k
describe equality constraints. The set
K ⊂
R
n
is a (geometric) constraint set.
Deﬁnition 1.1
A vector
x
?
will be called
optimal
if it gives the smallest value to
f
o
amongst all vectors
x
satisfying the constraints, i.e., amongst all feasible vectors.
We need to make some simple observations regarding this standard form. First, we have
stated the problem as a
minimization
problem. Since maximizing a function over some
1
Vectors will always be considered as column vectors. The symbol
>
here designates the operation of
transpose which interchangs columns and rows.
1