Numerical Recipes in PASCAL - The Art of Scientific Computing [1 ed.] 0521375169, 0521375320, 0521387663, 0521376750, 0521375339, 0521387671

Table of contents :
Title Page
Contents
Preface to the Pascal Edition
Preface
Acknowledgements
List of Computer Programs
1 PRELIMINARIES
1.0 Introduction
1.1 Program Organization and Control Structures
1.2 Conventions for Scientific Computing in Pascal
1.3 Error, Accuracy, and Stability
2 SOLUTION OF LINEAR ALGEBRAIC EQUATIONS
2.0 Introduction
2.1 Gauss-Jordan Elimination
2.2 Gaussian Elimination with Backsubstitution
2.3 LU Decomposition
2.4 Inverse of a Matrix
2.5 Determinant of a Matrix
2.6 Tridiagonal Systems of Equations
2.7 Iterative Improvement of a Solution to Linear Equations
2.8 Vandermonde Matrices and Toeplitz Matrices
2.9 Singular Value Decomposition
2.10 Sparse Linear Systems
2.11 Is Matrix Inversion an N^3 Process?
3 INTERPOLATION AND EXTRAPOLATION
3.0 Introduction
3.1 Polynomial Interpolation and Extrapolation
3.2 Rational Function Interpolation and Extrapol
3.3 Cubic Spline Interpolation
3.4 How to Search an Ordered Table
3.5 Coefficients of the Interpolating Polynomial
3.6 Interpolation in Two or More Dimensions
4 INTEGRATION OF FUNCTIONS
4.0 Introduction
4.1 Classical Formulas for Equally-Spaced Abscissas
4.2 Elementary Algorithms
4.3 Romberg Integration
4.4 Improper Integrals
4.5 Gaussian Quadratures
4.6 Multidimensional Integrals
5 EVALUATION OF FUNCTIONS
5.0 Introduction
5.1 Series and Their Convergence
5.2 Evaluation of Continued Fractions
5.3 Polynomials and Rational Functions
5.4 Recurrence Relations and Clenshaw's Recurrence Formula
5.5 Quadratic and Cubic Equations
5.6 Chebyshev Approximation
5.7 Derivatives or Integrals of a Chebyshev-approximated Function
5.8 Polynomial Approximation from Chebyshev Coefficients
6 SPECIAL FUNCTIONS
6.0 Introduction
6.1 Gamma Function, Beta Function, Factorials, Binomial Coefficients
6.2 Incomplete Gamma Function, Error Function, Chi-Square Probability Function, Cumulative Poisson Distribution
6.3 Incomplete Beta Function, Student's Distribution, F-Distribution, Cumulative Binomial Distribution
6.4 Bessel Functions of Integer Order
6.5 Modified Bessel Functions of Integer Order
6.6 Spherical Harmonics
6.7 Elliptic Integrals and Jacobian Elliptic Functions
7 RANDOM NUMBERS
7.0 Introduction
7.1 Uniform Deviates
7.2 Transformation Method: Exponential and Normal Deviates
7.3 Rejection Method: Gamma, Poisson, Binomial Deviates
7.4 Generation of Random Bits
7.5 The Data Encryption Standard
7.6 Monte Carlo Integration
8 SORTING
8.0 Introduction
8.1 Straight Insertion and Shell's Method
8.2 Heapsort
8.3 Indexing and Ranking
8.4 Quicksort
8.5 Determination of Equivalence Classes
9 ROOT FINDING AND NONLINEAR SETS OFEQUATIONS
9.0 Introduction
9.1 Bracketing and Bisection
9.2 Secant Method and False Position Method
9.3 Van Wijngaarden-Dekker-Brent Method
9.4 Newton-Raphson Method Using Derivative
9.5 Roots of Polynomials
9.6 Newton-Raphson Method for Nonlinear Systems of Equations
10 MINIMIZATION OR MAXIMIZATION OF FUNCTIONS
10.0 Introduction
10.1 Golden Section Search in One Dimension
10.2 Parabolic Interpolation and Brent's Method in One Dimension
10.3 One-Dimensional Search with First Derivatives
10.4 Downhill Simplex Method in Multidimensions
10.5 Direction Set (Powell's) Methods in Multidimensions
10.6 Conjugate Gradient Methods in Multidimensions
10.7 Variable Metric Methods in Multidimensions
10.8 Linear Programming and the Simplex Method
10.9 Combinatorial Minimization: Method of Simulated Annealing
11 EIGENSYSTEMS
11.0 Introduction
11.1 Jacobi Transformations of a Symmetric Matrix
11.2 Reduction of a Symmetric Matrix to Tridiagonal Form: Givens and Householder Reductions
11.3 Eigenvalues and Eigenvectors of a Tridiagonal Matrix
11.4 Hermitian Matrices
11.5 Reduction of a General Matrix to Hessenberg Form
11.6 The QR Algorithm for Real Hessenberg Matrices
11.7 Improving Eigenvalues and/or Finding Eigenvectors by Inverse Iteration
12 FOURIER TRANSFORM SPECTRAL METHODS
12.0 Introduction
12.1 Fourier Transform of Discretely Sampled Data
12.2 Fast Fourier Transform (FFT)
12.3 FFT of Real Functions, Sine and Cosine Transforms
12.4 Convolution and Deconvolution Using the FFT
12.5 Correlation and Autocorrelation Using the FFT
12.6 Optimal (Wiener) Filtering with the FFT
12.7 Power Spectrum Estimation Using the FFT
12.8 Power Spectrum Estimation by the Maximum Entropy (All Poles)Method
12.9 Digital Filtering in the Time Domain
12.10 Linear Prediction and Linear Predictive Coding
12.11 FFT in Two or More Dimensions
13 STATISTICAL DESCRIPTION OF DATA
13.0 Introduction
13.1 Moments of a Distribution: Mean, Variance, Skewness, and so forth
13.2 Efficient Search for the Median
13.3 Estimation of the Mode for Continuous Data
13.4 Do Two Distributions Have the Same Means or Variances?
13.5 Are Two Distributions Different?
13.6 Contingency Table Analysis of Two Distributions
13.7 Linear Correlation
13.8 Nonparametric or Rank Correlation
13.9 Smoothing of Data
14 MODELING OF DATA
14.0 Introduction
14.1 Least Squares as a Maximum Likelihood Estimator
14.2 Fitting Data to a Straight Line
14.3 General Linear Least Squares
14.4 Nonlinear Models
14.5 Confidence Limits on Estimated Model Parameters
14.6 Robust Estimation
15 INTEGRATION OF ORDINARY DIFFERENTIAL EQUATIONS
15.0 Introduction
15.1 Runge-Kutta Method
15.2 Adaptive Stepsize Control for Runge-Kutta
15.3 Modified Midpoint Method
15.4 Richardson Extrapolation and the Bulirsch-Stoer Method
15.5 Predictor-Corrector Methods
15.6 Stiff Sets of Equations
16 TWO POINT BOUNDARY VALUE PROBLEMS
16.0 Introduction
16.1 The Shooting Method
16.2 Shooting to a Fitting Point
16.3 Relaxation Methods
16.4 A Worked Example: Spheroidal Harmonics
16.5 Automated Allocation of Mesh Points
16.6 Handling Internal Boundary Conditions or Singular Points
17 PARTIAL DIFFERENTIAL EQUATIONS
17.0 Introduction
17.1 Flux-Conservative Initial Value Problems
17.2 Diffusive Initial Value Problems
17.3 Initial Value Problems in Multidimensions
17.4 Fourier and Cyclic Reduction Methods for Boundary Value Problems
17.5 Relaxation Methods for Boundary Value Problems
17.6 Operator Splitting Methods and ADI
APPENDIX A: References
APPENDIX B: Table of Program Dependencies
Index

Citation preview

Published by the Press Syndicate of the University of Cambridge The Pitt Building, Trumpington Street, Cambridge CB2 1RP 40 West 20th Street, New York, NY 10011-4211, USA 10 Stamford Road, Oakleigh, Melbourne 3166, Australia Copyright @Cambridge University Press 1986, 1989 except for computer programs and procedures, which are Copyright O Numerical Recipes Software 1986, 1989. All Rights Reserved. First published 1989 Reprinted 1990, 1992, 1994 Printed in the United States of America Typeset in T EX

The computer programs in this book are available in several machinereadable formats. To purchase diskettes in IBM-compatible personal computer format, use the order form at the back of the book or write to Cambridge University Press, 110 Midland Avenue, Port Chester, NY 10573. Also available from Cambridge University Press is the Numerical Recipes Example Book (Pascal), Revised Edition which provides demonstration programs that illustrate the use of each program in this book. This book, the Example Book, and the diskettes are also available in editions in the FORTRAN and C languages. Technical questions, corrections, and requests for information on other available formats and additional software products should be directed to Numerical Recipes Software, P.O. Box 243, Cambridge, MA 02238. Inquiries regarding OEM and site licenses should also be directed to this address.

Library of Congress Cataloging-in-Publication Data Numerical recipes in Pascal. Bibliography: p. Includes index. 1. Pascal (Computer program language) I. Press, William H. QA76.73.P2N87 1989 005.13'3 89-15841

British Library Cataloguing in Publication Data Numerical recipes in Pascal. 1. Numerical analysis. Applications of computer systems I. Press, William H. 519.4 ISBN 0-521.;.37516-9 hardback ISBN ISBN ISBN ISBN ISBN ISBN

0-521-37516-9 0-521-37532-0 0-521-38766-3 0-521-37675-0 0-521-37533-9 0-521-38767-1

book Pascal diskette (IBM) Pascal diskette (Macintosh) Example book Example diskette (IBM) Example diskette (Macintosh)

Contents Preface to the Pascal Edition

1

Preface

xiii

List of Computer Programs

xvii

PRELIMINARIES 1.0 1.1 1.2 1.3

2

xi

1

Introduction 1 Program Organization and Control Structures Conventions for Scientific Computing in Pascal Error, Accuracy, and Stability 23

4 14

SOLUTION OF LINEAR ALGEBRAIC EQUATIONS 2.0 Introduction 27 2.1 Gauss-Jordan Elimination 31 2.2 Gaussian Elimination with Backsubstitution 37 2.3 LU Decomposition 39 2.4 Inverse of a Matrix 46 2.5 Determinant of a Matrix 47 2.6 Tridiagonal Systems of Equations 48 2.7 Iterative Improvement of a Solution to Linear Equations 2.8 Vandermonde Matrices and Toeplitz Matrices 52 2.9 Singular Value Decomposition 61 2.10 Sparse Linear Systems 74 84 2.11 Is Matrix Inversion an N3 Process?

3

INTERPOLATION AND EXTRAPOLATION 3.0 3.1 3.2 3.3 3.4 3.5 3.6

4

Introduction 87 Polynomial Interpolation and Extrapolation 90 Rational Function Interpolation and Extrapolation Cubic Spline Interpolation 97 How to Search an Ordered Table 101 Coefficients of the Interpolating Polynomial 104 Interpolation in Two or More Dimensions 107

Introduction 116 Classical Formulas for Equally-Spaced Abscissas Elementary Algorithms 124 Romberg Integration 129 Improper Integrals 130 138 Gaussian Quadratures Multidimensional Integrals 144 v

49

87

93

INTEGRATION OF FUNCTIONS 4.0 4.1 4.2 4.3 4.4 4.5 4.6

27

116 117

GJL.....5

Contents

.....J

EVALUATION OF FUNCTIONS

149

5.0 Introduction 149 5.1 Series and Their Convergence 150 5.2 Evaluation of Continued Fractions 153 155 5.3 Polynomials and Rational Functions 5.4 Recurrence Relations and Clenshaw's Recurrence Formula 159 5.5 Quadratic and Cubic Equations 163 5.6 Chebyshev Approximation 165 5.7 Derivatives or Integrals of a Chebyshev-approximated Function 170 5.8 Polynomial Approximation from Chebyshev Coefficients 172

6

SPECIAL FUNCTIONS

175

6.0 Introduction 175 6.1 Gamma Function, Beta Function, Factorials, Binomial Coefficients 176 6.2 Incomplete Gamma Function, Error Function, Chi-Square Probability Function, Cumulative Poisson Distribution 180 6.3 Incomplete Beta Function, Student's Distribution, F - Distribution, Cumulative Binomial Distribution 186 6.4 Bessel Functions of Integer Order 191 6.5 Modified Bessel Functions of Integer Order 197 6.6 Spherical Harmonics 202 6.7 Elliptic Integrals and Jacobian Elliptic Functions 205

7

RANDOM NUMBERS 7.0 7.1 7.2 7.3 7.4 7.5 7.6

8

Introduction 212 Uniform Deviates 213 Transformation Method: Exponential and Normal Deviates 222 Rejection Method: Gamma, Poisson, Binomial Deviates 226 Generation of Random Bits 233 The Data Encryption Standard 239 Monte Carlo Integration 249

SORTING 8.0 8.1 8.2 8.3 8.4 8.5

9

212

Introduction 254 Straight Insertion and Shell's Method Heapsort 258 Indexing and Ranking 261 Quicksort 264 Determination of Equivalence Classes

254 255

267

ROOT FINDING AND NONLINEAR SETS OF EQUATIONS 9.0 Introduction 270 9.1 Bracketing and Bisection 274 9.2 Secant Method and False Position Method

279

270

Contents 9.3 Van Wijngaarden-Dekker-Brent Method 283 9.4 Newton-Raphson Method Using Derivative 286 9.5 Roots of Polynomials 292 9.6 Newton-Raphson Method for Nonlinear Systems of Equations

10 MINIMIZATION OR MAXIMIZATION OF FUNCTIONS 10.0 Introduction 309 10.1 Golden Section Search in One Dimension 312 10.2 Parabolic Interpolation and Brent's Method in One Dimension 10.3 One-Dimensional Search with First Derivatives 322 lOA Downhill Simplex Method in Multidimensions 326 10.5 Direction Set (Powell's) Methods in Multidimensions 331 10.6 Conjugate Gradient Methods in Multidimensions 339 10.7 Variable Metric Methods in Multidimensions 346 10.8 Linear Programming and the Simplex Method 351 10.9 Combinatorial Minimization: Method of Simulated Annealing

11 EIGENSYSTEMS

vii

305

309

318

366

375

11.0 Introduction 375 11.1 Jacobi Transformations of a Symmetric Matrix 382 11.2 Reduction of a Symmetric Matrix to Tridiagonal Form: Givens and Householder Reductions 389 11.3 Eigenvalues and Eigenvectors of a Tridiagonal Matrix 397 11.4 Hermitian Matrices 404 11.5 Reduction of a General Matrix to Hessenberg Form 405 11.6 The QR Algorithm for Real Hessenberg Matrices 410 11.7 Improving Eigenvalues and/or Finding Eigenvectors by Inverse Iteration 418

12 FOURIER TRANSFORM SPECTRAL METHODS 12.0 12.1 12.2 12.3 12.4 12.5 12.6 12.7 12.8

422

Introduction 422 Fourier Transform of Discretely Sampled Data 427 Fast Fourier Transform (FFT) 431 FFT of Real Functions, Sine and Cosine Transforms 438 Convolution and Deconvolution Using the FFT 449 457 Correlation and Autocorrelation Using the FFT 459 Optimal (Wiener) Filtering with the FFT 463 Power Spectrum Estimation Using the FFT Power Spectrum Estimation by the Maximum Entropy (All Poles) Method 473 12.9 Digital Filtering in the Time Domain 478 12.10 Linear Prediction and Linear Predictive Coding 487 12.11 FFT in Two or More Dimensions 493

Contents ~'---------------------------

13 STATISTICAL DESCRIPTION OF DATA

498

13.0 Introduction 498 13.1 Moments of a Distribution: Mean, Variance, Skewness, and so forth 499 13.2 Efficient Search for the Median 503 13.3 Estimation of the Mode for Continuous Data 507 13.4 Do Two Distributions Have the Same Means or Variances? 13.5 Are Two Distributions Different? 515 13.6 Contingency Table Analysis of Two Distributions 523 13.7 Linear Correlation 532 13.8 Nonparametric or Rank Correlation 536 13.9 Smoothing of Data 543

14 MODELING OF DATA 14.0 14.1 14.2 14.3 14.4 14.5 14.6

547

Introduction 547 Least Squares as a Maximum Likelihood Estimator Fitting Data to a Straight Line 553 General Linear Least Squares 558 Nonlinear Models 572 Confidence Limits on Estimated Model Parameters Robust Estimation 590

548

580

15 INTEGRATION OF ORDINARY DIFFERENTIAL EQUATIONS 15.0 Introduction 599 15.1 Runge-Kutta Method 602 15.2 Adaptive Stepsize Control for Runge-Kutta 607 15.3 Modified Midpoint Method 614 15.4 Richardson Extrapolation and the Bulirsch-Stoer Method 15.5 Predictor-Corrector Methods 624 15.6 Stiff Sets of Equations 628

16 TWO POINT BOUNDARY VALUE PROBLEMS 16.0 16.1 16.2 16.3 16.4 16.5 16.6

Introduction 633 The Shooting Method 637 Shooting to a Fitting Point 641 Relaxation Methods 645 A Worked Example: Spheroidal Harmonics 658 Automated Allocation of Mesh Points 666 Handling Internal Boundary Conditions or Singular Points

17 PARTIAL DIFFERENTIAL EQUATIONS 17.0 17.1 17.2 17.3

Introduction 673 Flux-Conservative Initial Value Problems Diffusive Initial Value Problems 693 Initial Value Problems in Multidimensions

509

599

617

633

669 673

681 700

Contents

GJ

-----------------------------' 17.4 Fourier and Cyclic Reduction Methods for Boundary Value Problems 704 17.5 Relaxation Methods for Boundary Value Problems 710 17.6 Operator Splitting Methods and ADI 718

APPENDIX A: References

727

APPENDIX B: Table of Program Dependencies

732