Optimization Problems
OneDimensional Optimization
MultiDimensional Optimization
Scientiﬁc Computing: An Introductory Survey
Chapter 6 – Optimization
Prof. Michael T. Heath
Department of Computer Science
University of Illinois at UrbanaChampaign
Copyright c
±
2002. Reproduction permitted
for noncommercial, educational use only.
Michael T. Heath
Scientiﬁc Computing
1 / 74
Optimization Problems
OneDimensional Optimization
MultiDimensional Optimization
Outline
1
Optimization Problems
2
OneDimensional Optimization
3
MultiDimensional Optimization
Michael T. Heath
Scientiﬁc Computing
2 / 74
Optimization Problems
OneDimensional Optimization
MultiDimensional Optimization
Deﬁnitions
Existence and Uniqueness
Optimality Conditions
Optimization
Given function
f
:
R
n
→
R
, and set
S
⊆
R
n
, ﬁnd
x
*
∈
S
such that
f
(
x
*
)
≤
f
(
x
)
for all
x
∈
S
x
*
is called
minimizer
or
minimum
of
f
It sufﬁces to consider only minimization, since maximum of
f
is minimum of

f
Objective
function
f
is usually differentiable, and may be
linear or nonlinear
Constraint
set
S
is deﬁned by system of equations and
inequalities, which may be linear or nonlinear
Points
x
∈
S
are called
feasible
points
If
S
=
R
n
, problem is
unconstrained
Michael T. Heath
Scientiﬁc Computing
3 / 74
Optimization Problems
OneDimensional Optimization
MultiDimensional Optimization
Deﬁnitions
Existence and Uniqueness
Optimality Conditions
Optimization Problems
General continuous optimization problem:
min
f
(
x
)
subject to
g
(
x
) =
0
and
h
(
x
)
≤
0
where
f
:
R
n
→
R
,
g
:
R
n
→
R
m
,
h
:
R
n
→
R
p
Linear programming
:
f
,
g
, and
h
are all linear
Nonlinear programming
: at least one of
f
,
g
, and
h
is
nonlinear
Michael T. Heath
Scientiﬁc Computing
4 / 74
Optimization Problems
OneDimensional Optimization
MultiDimensional Optimization
Deﬁnitions
Existence and Uniqueness
Optimality Conditions
Examples: Optimization Problems
Minimize weight of structure subject to constraint on its
strength, or maximize its strength subject to constraint on
its weight
Minimize cost of diet subject to nutritional constraints
Minimize surface area of cylinder subject to constraint on
its volume:
min
x
1
,x
2
f
(
x
1
, x
2
) = 2
πx
1
(
x
1
+
x
2
)
subject to
g
(
x
1
, x
2
) =
πx
2
1
x
2

V
= 0
where
x
1
and
x
2
are radius and height of cylinder, and
V
is
required volume
Michael T. Heath
Scientiﬁc Computing
5 / 74
Optimization Problems
OneDimensional Optimization
MultiDimensional Optimization
Deﬁnitions
Existence and Uniqueness
Optimality Conditions
Local vs Global Optimization
x
*
∈
S
is
global minimum
if
f
(
x
*
)
≤
f
(
x
)
for all
x
∈
S
x
*
∈
S
is
local minimum
if
f
(
x
*
)
≤
f
(
x
)
for all feasible
x
in
some neighborhood of
x
*
Michael T. Heath
Scientiﬁc Computing
6 / 74
Optimization Problems
OneDimensional Optimization
MultiDimensional Optimization
Deﬁnitions
Existence and Uniqueness
Optimality Conditions
Global Optimization
Finding, or even verifying, global minimum is difﬁcult, in
general
Most optimization methods are designed to ﬁnd local
minimum, which may or may not be global minimum
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
This is the end of the preview.
Sign up
to
access the rest of the document.
 Fall '11
 Wasfy
 Mechanical Engineering, Optimization, Michael T. Heath

Click to edit the document details