painless-conjugate-gradient

painless-conjugate-gradient - An Introduction to the...

Info iconThis preview shows pages 1–5. Sign up to view the full content.

View Full Document Right Arrow Icon
An Introduction to the Conjugate Gradient Method Without the Agonizing Pain Edition 1 1 4 Jonathan Richard Shewchuk August 4, 1994 School of Computer Science Carnegie Mellon University Pittsburgh, PA 15213 Abstract The Conjugate Gradient Method is the most prominent iterative method for solving sparse systems of linear equations. Unfortunately, many textbook treatments of the topic are written with neither illustrations nor intuition, and their victims can be found to this day babbling senselessly in the corners of dusty libraries. For this reason, a deep, geometric understanding of the method has been reserved for the elite brilliant few who have painstakingly decoded the mumblings of their forebears. Nevertheless, the Conjugate Gradient Method is a composite of simple, elegant ideas that almost anyone can understand. Of course, a reader as intelligent as yourself will learn them almost effortlessly. The idea of quadratic forms is introduced and used to derive the methods of Steepest Descent, Conjugate Directions, and Conjugate Gradients. Eigenvectors are explained and used to examine the convergence of the Jacobi Method, Steepest Descent, and ConjugateGradients. Other topics include preconditioningand the nonlinearConjugateGradient Method. I have taken pains to make this article easy to read. Sixty-six illustrations are provided. Dense prose is avoided. Concepts are explained in several different ways. Most equations are coupled with an intuitive interpretation. Supported in part by the Natural Sciences and Engineering Research Council of Canada under a 1967 Science and Engineering Scholarship and by the National Science Foundation under Grant ASC-9318163. The views and conclusions contained in this document are those of the author and should not be interpreted as representing the official policies, either express or implied, of NSERC, NSF, or the U.S. Government.
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Keywords: conjugate gradient method, preconditioning, convergence analysis, agonizing pain
Background image of page 2
Contents 1. Introduction 1 2. Notation 1 3. The Quadratic Form 2 4. The Method of Steepest Descent 6 5. Thinking with Eigenvectors and Eigenvalues 9 5.1. Eigen do it if I try 9 5.2. Jacobi iterations 11 5.3. A Concrete Example 12 6. Convergence Analysis of Steepest Descent 13 6.1. Instant Results 13 6.2. General Convergence 17 7. The Method of Conjugate Directions 21 7.1. Conjugacy 21 7.2. Gram-Schmidt Conjugation 25 7.3. Optimality of the Error Term 26 8. The Method of Conjugate Gradients 30 9. Convergence Analysis of Conjugate Gradients 32 9.1. Picking Perfect Polynomials 33 9.2. Chebyshev Polynomials 35 10. Complexity 37 11. Starting and Stopping 38 11.1. Starting 38 11.2. Stopping 38 12. Preconditioning 39 13. Conjugate Gradients on the Normal Equations 41 14. The Nonlinear Conjugate Gradient Method 42 14.1. Outline of the Nonlinear Conjugate Gradient Method 42 14.2. General Line Search 43 14.3. Preconditioning 47 A Notes 48 B Canned Algorithms 49 B1. Steepest Descent 49 B2. Conjugate Gradients 50 B3. Preconditioned Conjugate Gradients 51 i
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
B4. Nonlinear Conjugate Gradients with Newton-Raphson and Fletcher-Reeves 52 B5. Preconditioned Nonlinear Conjugate Gradients with Secant and Polak-Ribi`ere
Background image of page 4
Image of page 5
This is the end of the preview. Sign up to access the rest of the document.

This document was uploaded on 10/24/2011.

Page1 / 64

painless-conjugate-gradient - An Introduction to the...

This preview shows document pages 1 - 5. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online