Numerical Mathematics, by Quarteroni, Sacco, Saleri..pdf -...

This preview shows page 1 out of 664 pages.

Unformatted text preview: Texts in Applied Mathematics 37 Editors J.E. Marsden L. Sirovich S.S. Antman Advisors G. Iooss P. Holmes D. Barkley M. Dellnitz P. Newton Texts in Applied Mathematics 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23. 24. 25. 26. 27. 28. Sirovich: Introduction to Applied Mathematics. Wiggins: Introduction to Applied Nonlinear Dynamical Systems and Chaos. Hale/Koc¸ak: Dynamics and Bifurcations. Chorin/Marsden: A Mathematical Introduction to Fluid Mechanics, Third Edition. Hubbard/West: Differential Equations: A Dynamical Systems Approach: Ordinary Differential Equations. Sontag: Mathematical Control Theory: Deterministic Finite Dimensional Systems Second Edition. Perko: Differential Equations and Dynamical Systems, Third Edition. Seaborn: Hypergeometric Functions and Their Applications. Pipkin: A Course on Integral Equations. Hoppensteadt/Peskin: Modeling and Simulation in Medicine and the Life Sciences, Second Edition. Braun: Differential Equations and Their Applications, Fourth Edition. Stoer/Bulirsch: Introduction to Numerical Analysis, Third Edition. Renardy/Rogers: An Introduction to Partial Differential Equations. Banks: Growth and Diffusion Phenomena: Mathematical Frameworks and Applications. Brenner/Scott: The Mathematical Theory of Finite Element Methods, Second Edition. Van de Velde: Concurrent Scientific Computing. Marsden/Ratiu: Introduction to Mechanics and Symmetry, Second Edition. Hubbard/West: Differential Equations: A Dynamical Systems Approach: Higher-Dimensional Systems. Kaplan/Glass: Understanding Nonlinear Dynamics. Holmes: Introduction to Perturbation Methods. Curtain/Zwart: An Introduction to Infinite-Dimensional Linear Systems Theory. Thomas: Numerical Partial Differential Equations: Finite Difference Methods. Taylor: Partial Differential Equations: Basic Theory. Merkin: Introduction to the Theory of Stability of Motion. Naber: Topology, Geometry, and Gauge Fields: Foundations. Polderman/Willems: Introduction to Mathematical Systems Theory: A Behavioral Approach. Reddy: Introductory Functional Analysis: with Applications to Boundary Value Problems and Finite Elements. Gustafson/Wilcox: Analytical and Computational Methods of Advanced Engineering Mathematics. (continued after index) Alfio Quarteroni Riccardo Sacco Fausto Saleri Numerical Mathematics Second Edition With 135 Figures and 45 Tables ABC Alfio Quarteroni Riccardo Sacco SB-IACS-CMS, EPFL 1015 Lausanne, Switzerland and Dipartimento di Matematica-MOX Politecnico di Milano Piazza Leonardo da Vinci, 32 20133 Milano, Italy E-mail: [email protected] Dipartimento di Matematica Politecnico di Milano Piazza Leonardo da Vinci, 32 20133 Milano, Italy E-mail: [email protected] Fausto Saleri Series Editors Dipartimento di Matematica–MOX Politecnico di Milano Piazza Leonardo da Vinci, 32 20133 Milano, Italy E-mail: [email protected] J.E. Marsden S.S. Antman Control and Dynamical Systems 107-81 California Institute of Technology Pasadena, CA 91125 USA [email protected] L. Sirovich Laboratory of Applied Mathematics Department of Biomathematics Mt. Sinai School of Medicine Box 1012 New York, NY 10029-6574 USA Department of Mathematics and Institute for Physical Science and Technology University of Maryland College Oark, MD 20742-4015 USA [email protected] Mathematics Subject Classification (2000): 15-01, 34-01, 35-01, 65-01 ISBN 0939-2475 ISBN-10 3-540-34658-9 Springer Berlin Heidelberg New York ISBN-13 978-3-540-34658-6 Springer Berlin Heidelberg New York Library of Congress Control Number: 2006930676 This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilm or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer-Verlag. Violations are liable to prosecution under the German Copyright Law. Springer is a part of Springer Science+Business Media. springer.com c Springer Berlin Heidelberg 2007 ⃝ The use of general descriptive names, registered names, trademarks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. Typesetting by the Authors and Spi using Springer LATEX macro package Cover design: design & production GmbH, Heidelberg Printed on acid-free paper SPIN: 11304951 37/2244/SPi 543210 Preface Numerical mathematics is the branch of mathematics that proposes, develops, analyzes and applies methods from scientific computing to several fields including analysis, linear algebra, geometry, approximation theory, functional equations, optimization and differential equations. Other disciplines such as physics, the natural and biological sciences, engineering, and economics and the financial sciences frequently give rise to problems that need scientific computing for their solutions. As such, numerical mathematics is the crossroad of several disciplines of great relevance in modern applied sciences, and can become a crucial tool for their qualitative and quantitative analysis. This role is also emphasized by the continual development of computers and algorithms, which make it possible nowadays, using scientific computing, to tackle problems of such a large size that real-life phenomena can be simulated providing accurate responses at affordable computational cost. The corresponding spread of numerical software represents an enrichment for the scientific community. However, the user has to make the correct choice of the method (or the algorithm) which best suits the problem at hand. As a matter of fact, no black-box methods or algorithms exist that can effectively and accurately solve all kinds of problems. One of the purposes of this book is to provide the mathematical foundations of numerical methods, to analyze their basic theoretical properties (stability, accuracy, computational complexity), and demonstrate their performances on examples and counterexamples which outline their pros and cons. This is done using the MATLAB! 1 software environment. This choice satisfies the two fundamental needs of user-friendliness and wide-spread diffusion, making it available on virtually every computer. Every chapter is supplied with examples, exercises and applications of the discussed theory to the solution of real-life problems. The reader is thus in the ideal condition for acquiring the theoretical knowledge that is required to 1 MATLAB is a trademark of The MathWorks, Inc. VI Preface make the right choice among the numerical methodologies and make use of the related computer programs. This book is primarily addressed to undergraduate students, with particular focus on the degree courses in Engineering, Mathematics, Physics and Computer Science. The attention which is paid to the applications and the related development of software makes it valuable also for graduate students, researchers and users of scientific computing in the most widespread professional fields. The content of the volume is organized into four Parts and 13 chapters. Part I comprises two chapters in which we review basic linear algebra and introduce the general concepts of consistency, stability and convergence of a numerical method as well as the basic elements of computer arithmetic. Part II is on numerical linear algebra, and is devoted to the solution of linear systems (Chapters 3 and 4) and eigenvalues and eigenvectors computation (Chapter 5). We continue with Part III where we face several issues about functions and their approximation. Specifically, we are interested in the solution of nonlinear equations (Chapter 6), solution of nonlinear systems and optimization problems (Chapter 7), polynomial approximation (Chapter 8) and numerical integration (Chapter 9). Part IV, which demands a mathematical background, is concerned with approximation, integration and transforms based on orthogonal polynomials (Chapter 10), solution of initial value problems (Chapter 11), boundary value problems (Chapter 12) and initial-boundary value problems for parabolic and hyperbolic equations (Chapter 13). Part I provides the indispensable background. Each of the remaining Parts has a size and a content that make it well suited for a semester course. A guideline index to the use of the numerous MATLAB programs developed in the book is reported at the end of the volume. These programs are also available at the web site address: ˜calnum/programs.html. For the reader’s ease, any code is accompanied by a brief description of its input/output parameters. We express our thanks to the staff at Springer-Verlag New York for their expert guidance and assistance with editorial aspects, as well as to Dr. Martin Peters from Springer-Verlag Heidelberg and Dr. Francesca Bonadei from Springer-Italia for their advice and friendly collaboration all along this project. We gratefully thank Professors L. Gastaldi and A. Valli for their useful comments on Chapters 12 and 13. We also wish to express our gratitude to our families for their forbearance and understanding, and dedicate this book to them. Lausanne, Milan January 2000 Alfio Quarteroni Riccardo Sacco Fausto Saleri Preface to the Second Edition This second edition is characterized by a thourough overall revision. Regarding the styling of the book, we have improved the readibility of pictures, tables and program headings. Regarding the scientific contents, we have introduced several changes in the chapter on iterative methods for the solution of linear systems as well as in the chapter on polynomial approximation of functions and data. Lausanne, Milan September 2006 Alfio Quarteroni Riccardo Sacco Fausto Saleri Contents Part I Getting Started 1 2 Foundations of Matrix Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.1 Vector Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3 Operations with Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3.1 Inverse of a Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3.2 Matrices and Linear Mappings . . . . . . . . . . . . . . . . . . . 1.3.3 Operations with Block-Partitioned Matrices . . . . . . . 1.4 Trace and Determinant of a Matrix . . . . . . . . . . . . . . . . . . . . . . 1.5 Rank and Kernel of a Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.6 Special Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.6.1 Block Diagonal Matrices . . . . . . . . . . . . . . . . . . . . . . . . 1.6.2 Trapezoidal and Triangular Matrices . . . . . . . . . . . . . . 1.6.3 Banded Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.7 Eigenvalues and Eigenvectors . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.8 Similarity Transformations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.9 The Singular Value Decomposition (SVD) . . . . . . . . . . . . . . . . 1.10 Scalar Product and Norms in Vector Spaces . . . . . . . . . . . . . . 1.11 Matrix Norms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.11.1 Relation between Norms and the Spectral Radius of a Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.11.2 Sequences and Series of Matrices . . . . . . . . . . . . . . . . . 1.12 Positive Definite, Diagonally Dominant and M-matrices . . . . 1.13 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 3 5 6 7 8 9 10 11 12 12 12 13 13 15 17 18 22 Principles of Numerical Mathematics . . . . . . . . . . . . . . . . . . . . . 2.1 Well-posedness and Condition Number of a Problem . . . . . . . 2.2 Stability of Numerical Methods . . . . . . . . . . . . . . . . . . . . . . . . . 2.2.1 Relations between Stability and Convergence . . . . . . 2.3 A priori and a posteriori Analysis . . . . . . . . . . . . . . . . . . . . . . . 33 33 37 40 42 25 26 27 30 X Contents 2.4 2.5 2.6 Sources of Error in Computational Models . . . . . . . . . . . . . . . . Machine Representation of Numbers . . . . . . . . . . . . . . . . . . . . . 2.5.1 The Positional System . . . . . . . . . . . . . . . . . . . . . . . . . . 2.5.2 The Floating-point Number System . . . . . . . . . . . . . . . 2.5.3 Distribution of Floating-point Numbers . . . . . . . . . . . 2.5.4 IEC/IEEE Arithmetic . . . . . . . . . . . . . . . . . . . . . . . . . . 2.5.5 Rounding of a Real Number in its Machine Representation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.5.6 Machine Floating-point Operations . . . . . . . . . . . . . . . Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 45 45 46 49 49 50 52 54 Part II Numerical Linear Algebra 3 Direct Methods for the Solution of Linear Systems . . . . . . . 3.1 Stability Analysis of Linear Systems . . . . . . . . . . . . . . . . . . . . . 3.1.1 The Condition Number of a Matrix . . . . . . . . . . . . . . . 3.1.2 Forward a priori Analysis . . . . . . . . . . . . . . . . . . . . . . . . 3.1.3 Backward a priori Analysis . . . . . . . . . . . . . . . . . . . . . . 3.1.4 A posteriori Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 Solution of Triangular Systems . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.1 Implementation of Substitution Methods . . . . . . . . . . 3.2.2 Rounding Error Analysis . . . . . . . . . . . . . . . . . . . . . . . . 3.2.3 Inverse of a Triangular Matrix . . . . . . . . . . . . . . . . . . . 3.3 The Gaussian Elimination Method (GEM) and LU Factorization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.1 GEM as a Factorization Method . . . . . . . . . . . . . . . . . 3.3.2 The Effect of Rounding Errors . . . . . . . . . . . . . . . . . . . 3.3.3 Implementation of LU Factorization . . . . . . . . . . . . . . 3.3.4 Compact Forms of Factorization . . . . . . . . . . . . . . . . . . 3.4 Other Types of Factorization . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4.1 LDMT Factorization . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4.2 Symmetric and Positive Definite Matrices: The Cholesky Factorization . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4.3 Rectangular Matrices: The QR Factorization . . . . . . . 3.5 Pivoting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.6 Computing the Inverse of a Matrix . . . . . . . . . . . . . . . . . . . . . . 3.7 Banded Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.7.1 Tridiagonal Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.7.2 Implementation Issues . . . . . . . . . . . . . . . . . . . . . . . . . . 3.8 Block Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.8.1 Block LU Factorization . . . . . . . . . . . . . . . . . . . . . . . . . 3.8.2 Inverse of a Block-partitioned Matrix . . . . . . . . . . . . . 3.8.3 Block Tridiagonal Systems . . . . . . . . . . . . . . . . . . . . . . . 3.9 Sparse Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 60 60 62 65 65 66 67 69 70 70 73 78 78 80 81 81 82 84 87 91 92 93 94 96 97 97 98 99 Contents 3.10 3.11 3.12 3.13 3.14 3.15 4 XI 3.9.1 The Cuthill-McKee Algorithm . . . . . . . . . . . . . . . . . . . 102 3.9.2 Decomposition into Substructures . . . . . . . . . . . . . . . . 103 3.9.3 Nested Dissection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 Accuracy of the Solution Achieved Using GEM . . . . . . . . . . . . 106 An Approximate Computation of K(A) . . . . . . . . . . . . . . . . . . 108 Improving the Accuracy of GEM . . . . . . . . . . . . . . . . . . . . . . . . 112 3.12.1 Scaling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112 3.12.2 Iterative Refinement . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 Undetermined Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117 3.14.1 Nodal Analysis of a Structured Frame . . . . . . . . . . . . . 117 3.14.2 Regularization of a Triangular Grid . . . . . . . . . . . . . . . 120 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123 Iterative Methods for Solving Linear Systems . . . . . . . . . . . . 125 4.1 On the Convergence of Iterative Methods . . . . . . . . . . . . . . . . . 125 4.2 Linear Iterative Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128 4.2.1 Jacobi, Gauss-Seidel and Relaxation Methods . . . . . . 128 4.2.2 Convergence Results for Jacobi and Gauss-Seidel Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130 4.2.3 Convergence Results for the Relaxation Method . . . . 132 4.2.4 A priori Forward Analysis . . . . . . . . . . . . . . . . . . . . . . . 133 4.2.5 Block Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134 4.2.6 Symmetric Form of the Gauss-Seidel and SOR Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135 4.2.7 Implementation Issues . . . . . . . . . . . . . . . . . . . . . . . . . . 137 4.3 Stationary and Nonstationary Iterative Methods . . . . . . . . . . . 138 4.3.1 Convergence Analysis of the Richardson Method . . . 139 4.3.2 Preconditioning Matrices . . . . . . . . . . . . . . . . . . . . . . . . 141 4.3.3 The Gradient Method . . . . . . . . . . . . . . . . . . . . . . . . . . . 148 4.3.4 The Conjugate Gradient Method . . . . . . . . . . . . . . . . . 152 4.3.5 The Preconditioned Conjugate Gradient Method . . . 158 4.3.6 The Alternating-Direction Method . . . . . . . . . . . . . . . . 160 4.4 Methods Based on Krylov Subspace Iterations . . . . . . . . . . . . 160 4.4.1 The Arnoldi Method for Linear Systems . . . . . . . . . . . 164 4.4.2 The GMRES Method . . . . . . . . . . . . . . . . . . . . . . . . . . . 167 4.4.3 The Lanczos Method for Symmetric Systems . . . . . . . 168 4.5 The Lanczos Method for Unsymmetric Systems . . . . . . . . . . . 170 4.6 Stopping Criteria . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173 4.6.1 A Stopping Test Based on the Increment . . . . . . . . . . 174 4.6.2 A Stopping Test Based on the Residual . . . . . . . . . . . 175 4.7 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175 4.7.1 Analysis of an Electric Network . . . . . . . . . . . . . . . . . . 176 4.7.2 Finite Difference Analysis of Beam Bending . . . . . . . . 178 4.8 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180 XII 5 Contents Approximation of Eigenvalues and Eigenvectors . . . . . . . . . . 183 5.1 Geometrical Location of the Eigenvalues . . . . . . . . . . . . . . . . . . 183 5.2 Stability and Conditioning Analysis . . . . . . . . . . . . . . . . . . . . . . 186 5.2.1 A priori Estimates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187 5.2.2 A posteriori Estimates . . . . . . . . . . . . . . . . . . . . . . . . . . 190 5.3 The Power Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192 5.3.1 Approximation of the Eigenvalue of Largest Module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192 5.3.2 Inverse Iteration ....
View Full Document

  • Left Quote Icon

    Student Picture

  • Left Quote Icon

    Student Picture

  • Left Quote Icon

    Student Picture