12cg - EE236C (Spring 2008-09) 12. Conjugate gradient...

Info iconThis preview shows pages 1–5. Sign up to view the full content.

View Full Document Right Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: EE236C (Spring 2008-09) 12. Conjugate gradient method Krylov sequence conjugate gradient method for linear equations convergence analysis preconditioned conjugate gradient method nonlinear conjugate gradient method 121 Unconstrained quadratic minimization minimize f ( x ) = 1 2 x T Ax b T x with A S n ++ equivalent to solving Ax = b residual r = b Ax is negative gradient at x : r = f ( x ) conjugate gradient method invented by Hestenes and Stiefel around 1951 the most widely used iterative method for solving Ax = b , with A can be extended to non-quadratic unconstrained minimization Conjugate gradient method 122 Krylov subspace K = { } , K k = span { b, Ab, . . . , A k- 1 b } for k 1 a sequence of nested subspaces: K K 1 K 2 if K k +1 = K k , then K i = K k for all i k A- 1 b K n (even when K n negationslash = R n ) this follows from Cayley-Hamilton theorem: A n + a 1 A n- 1 + + a n- 1 A + a n I = 0 where det( I A ) = n + a 1 n- 1 + + a n- 1 + a n ; therefore A- 1 b = 1 a n ( A n- 1 b + a 1 A n- 2 b + + a n- 1 b ) Conjugate gradient method 123 Krylov sequence CG algorithm is a recursive method for computing the Krylov sequence x ( k ) = argmin x K k f ( x ) , k x ( n ) = A- 1 b optimality condition for x ( k ) is x ( k ) K k , f ( x ( k ) ) = Ax ( k ) b K k either x ( k ) = A- 1 b , or the residual r k = b Ax ( k ) is orthogonal to K k we will see there is a simple two-term recurrence x ( k +1) = x ( k ) + a k r k + b k ( x ( k ) x ( k- 1) ) Conjugate gradient method 124 Residuals of Krylov sequence the residuals r i = b Ax ( i ) span the Krylov subspaces: K k = span { r , r 1 , . . . , r k- 1 } this follows from r i K i and r i = b Ax ( i ) K i +1 the residuals are mutually orthogonal: r T i r j = 0 for i negationslash = j if j < i , this follows from r j K i and r i K i Conjugate gradient method 125 Conjugate directions the vectors v i = x ( i ) x ( i- 1) satisfy v T i Av j = 0 for i negationslash = j, v T i Av i = v T i r i- 1 proof (assume j < i ) v j = x ( j ) x ( j- 1) K j K i- 1 and Av i = A ( x ( i ) x ( i- 1) ) = r i + r i- 1 K i- 1 follows from the fact that t = 1 is the minimum of f ( x ( i- 1) + tv i ) = f ( x ( i- 1) ) + 1 2 t 2 v T i Av i tv T i r i- 1 Conjugate gradient method 126 Conjugate vectors to simplify notation we scale the vectors v i = x ( i ) x ( i- 1) as p i = bardbl r i- 1 bardbl 2 2 v T i r i- 1 v i ( p i = 0 if v i = 0) p T i r i- 1 = bardbl r i- 1 bardbl 2 2 p T i Ap j = 0 for i negationslash = j , so if p i negationslash = 0 , it is independent of { p 1 , . . . , p i- 1 } the conjugate vectors span the Krylov subspace: K k = span { p 1 , p 2 , . . . , p k } by optimizing f ( x ( k- 1) + p k ) over we can express...
View Full Document

Page1 / 15

12cg - EE236C (Spring 2008-09) 12. Conjugate gradient...

This preview shows document pages 1 - 5. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online