This preview shows pages 1–4. Sign up to view the full content.
1
Lecture9
Conjugate Direction Algorithm
(Solution of Linear System or
Minimization of A Quadratic
Function)
Conjugate Gradient
• Linear conjugate gradient: for solving linear
systems
Ax=
b with PD matrix, A.
– Exact solution in
n steps
(Hestenes & Stiefel, 1950s)
– Approximate solution in fewer than
n steps
• Nonlinear conjugate gradient: for solving large
scale nonlinear optimization problems.
– Fletcher and Reeves, 1964
– PolkRibiere, 1969
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document 2
Conjugate Gradient
Or minimize the following function:
b
Ax
=
A
is symmetric PD.
(1)
x
b
Ax
x
x
T
T

=
2
1
)
(
f
(2)
)
(
)
(
x
r
b
x
=

=
∇
f
r
(
x
) is the residual
j
i
0
≠
2200
=
j
T
i
Ap
p
{ }
1
1
0
,
,
,

=
n
p
p
p
S
K
The set
S
is conjugate wrt
A
if
Linear Independence
0
then
0
if
1
2
1
0
1
1
1
1
0
0
=
=
=
=
=
+
+



n
n
n
p
p
p
s
s
s
s
s
s
s
K
K
{ }
1
1
0
,
,
,

=
n
p
p
p
S
K
S
is linearly independent
Conjugate set is also linearly independent.
j
i
0
≠
2200
=
j
T
i
p
3
Conjugate Direction Method
k
k
k
k
p
x
x
a
+
=
+
1
j
i
0
≠
2200
=
j
T
i
Ap
p
Line search
1D minimizer of a quadratic function
x
b
Ax
x
x
T
T

=
2
1
)
(
f
k
T
k
k
T
k
k
Ap
p
p
f
a
∇

=
Convergence Rate of Steepest
Descent
( 29
( 29 ( 29 ( 29
( 29
)
(
0
0
0
)
2
1
(
k
T
k
k
T
k
k
k
k
T
T
k
k
T
k
k
T
k
T
k
k
T
k
T
k
k
T
k
k
T
k
T
k
k
T
k
k
T
k
T
k
k
k
k
T
k
k
T
k
k
k
k
f
Q
f
f
f
Qg
g
g
b
Q
x
Qg
g
g
b
Qg
x
g
b
Qg
x
Qg
g
g
b
Qg
g
Qg
x
g
b
Qg
g
x
g
x
b
g
x
Q
g
x
d
d
g
x
f
d
d
∇
∇
∇
∇
=

=

=

=
=
+
+

=
+


=
=




=

a
a
a
a
a
a
a
a
a
a
a
a
k
k
T
k
k
T
k
k
k
f
f
Q
f
f
f
x
x
∇
∇
∇
∇
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
This is the end of the preview. Sign up
to
access the rest of the document.
This note was uploaded on 06/12/2011 for the course COT 6505 taught by Professor Shah during the Spring '07 term at University of Central Florida.
 Spring '07
 Shah

Click to edit the document details