lehighlogo
IE417: Nonlinear Programming: Lecture 10
Jeff Linderoth
Department of Industrial and Systems Engineering
Lehigh University
16th February 2006
Jeff Linderoth
IE417:Lecture 10
lehighlogo
Last Time: Conjugate Gradient
Solving
Ax
=
b
Or
min
φ
(
x
)
def
= 1
/
2
x
T
Ax

b
T
x,
with
A
∈ S
n
×
n
++
Conjugate Gradient Algorithm
1
Choose
x
0
.r
0
=
Ax
0

b, d
0
=

d
0
, k
= 0
2
α
k
=

r
T
k
d
k
d
T
k
Ad
k
3
x
k
+1
=
x
k
+
α
k
d
k
4
β
k
+1
=
r
T
k
+1
Ad
k
d
T
k
Ad
k
5
d
k
+1
=

r
k
+1
+
β
T
k
+1
d
k
6
If
r
k
= 0
,
stop.
Else
k
=
k
+ 1
, Go to 2.
Jeff Linderoth
IE417:Lecture 10
lehighlogo
CG for Unconstrained Optimization
min
x
∈
R
n
f
(
x
)
FletcherReeves Conjugate Gradient
1
Given
x
0
.
d
0
=
∇
f
(
x
0
)
,
k
= 0
2
Compute
α
k
.
x
k
+1
=
x
k
+
α
k
d
k
3
Compute
∇
f
(
x
k
+1
)
, If
∇
f
(
x
k
+1
) = 0
,
stop
. Else
β
k
+1
=
∇
f
(
x
k
+1
)
T
∇
f
(
x
k
+1
)
∇
f
(
x
k
)
T
∇
f
(
x
k
)
4
Compute new direction:
d
k
+1
=
∇
f
(
x
k
+1
) +
β
k
+1
d
k
Fletcher Reeves method is globally convergence as long as
your
α
k
satisfies the strong Wolfe conditions (with
c
2
<
1
/
2
)
Jeff Linderoth
IE417:Lecture 10
lehighlogo
Today: QuasiNewton
Recall:
m
k
(
d
) =
f
(
x
k
) +
∇
f
(
x
k
)
T
d
k
+ 1
/
2
d
T
B
k
d
Minimizer of this quadratic function is
d
k
=

B

1
k
∇
f
(
x
k
)
Step:
x
k
+1
=
x
k
+
α
k
d
k
The Question
What can
B
do for you?
Given the gradient information that you have recently seen,
how would you like your model to behave?
Make
∇
m
k
+1
(
·
)
match
∇
f
(
·
)
at the last two points
x
k
+1
=
x
k
+
αd
k
, so
∇
m
k
+1
(0)
=
∇
f
(
x
k
+1
)
∇
m
k
+1
(

α
k
d
k
)
=
∇
f
(
x
k
)
Jeff Linderoth
IE417:Lecture 10
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
lehighlogo
The Secant Condition
If
B
k
+1
is to be symmetric, we have
∇
m
k
+1
(
d
) =
∇
f
(
x
k
+1
) +
B
k
+1
d
So
∇
m
k
+1
(0) =
∇
f
(
x
k
+1
)
This is the end of the preview.
Sign up
to
access the rest of the document.
 Spring '08
 Linderoth
 Optimization, Systems Engineering, Trigraph, QuasiNewton method, Broyden's method, Jeff Linderoth, Fletcher Reeves

Click to edit the document details