Numerical Analysis of Systems of Ordinary and Stochastic Differential Equations [Reprint 2010 ed.] 9783110944662

Table of contents :
Preface
1. Numerical Solution of the Cauchy Problem for Systems of Ordinary Differential Equations
1.1. The Cauchy Problem for an ODE System. Linear Systems. Stiff Systems
1.2. Single-step Methods. Main Definitions
1.3. Rosenbrock Type Methods. The Convergence Theorem
1.4. Taylor Expansion of Exact and Numerical Solutions of the Cauchy Problem
1.5. Consistency of RTMs
1.6. A-stability of RTMs
1.7. Practical Applications of RTMs to the Solution of Stiff ODEs
1.8. Some Generalizations of RTMs
1.9. Numerical Experiments
2. Statistical Simulation of the Cauchy Problem Solution for Systems of Stochastic Differential Equations
2.1. Elements of Probability Theory, Stochastic Processes and Statistical Simulation
2.2. Cauchy Problem for a SDE System. Main Definitions
2.3. Construction of SDEs with Given Probability Characteristics of Solution
2.4. Linear SDE Systems with Additive and Multiplicative Noise
2.5. Mean Square Stability of SDE Solutions. Stiff in Mean Square SDE Systems. Oscillatory Stochastic Systems
2.6. Simple Numerical Methods Generalizing the Explicit Runge-Kutta Methods
2.7. Families of Numerical Methods for Solving SDE Systems. Theorem of Convergence
2.8. Mean Square Consistency of Methods
2.9. Mean Square Stability of Methods
2.10. Numerical Methods for Solving Linear SDE Systems
2.11. Variable Step Algorithms for Solving SDEs
2.12. Numerical Solution of SDE System with Poisson Component
2.13. Applying SDE for Numerical Solution of Linear Elliptic and Parabolic Equations
2.14. Statistical Simulation of the SDE Solutions in Problems of Analysis and Synthesis of Automated Control
2.15. Numerical Experiments
References

Citation preview

Numerical Analysis of Systems of Ordinary and Stochastic Differential Equations

NUMERICAL ANALYSIS OF SYSTEMS OF ORDINARY AND STOCHASTIC DIFFERENTIAL EQUATIONS S.S. Artemiev and T.A. Averina

III MS?Ill Utrecht, The Netherlands, 1997

VSPBV P.O. Box 346 3700 AH Zeist The Netherlands

©VSPBV 1997 First published in 1997 ISBN 90-6764-250-9

All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, without the prior permission of the copyright owner.

Printed in The Netherlands by Ridderprint bv, Ridderkerk.

CONTENTS Preface 1.

2.

Numerical Solution of the Cauchy Problem for Systems of Ordinary Differential Equations 1.1. The Cauchy Problem for an ODE System. Linear Systems. Stiff Systems 1.2. Single-step Methods. Main Definitions 1.3. Rosenbrock Type Methods, The Convergence Theorem 1.4. Taylor Expansion of Exact and Numerical Solutions of the Cauchy Problem 1.5. Consistency of RTMs 1.6. Λ-stability of RTMs 1.7. Practical Applications of RTMs to the Solution of Stiff ODEs 1.8. Some Generalizations of RTMs 1.9. Numerical Experiments Statistical Simulation of the Cauchy Problem Solution for Systems of Stochastic Differential Equations 2.1. Elements of Probability Theory, Stochastic Processes and Statistical Simulation 2.2. Cauchy Problem for a SDE System. Main Definitions 2.3. Construction of SDEs with Given Probability Characteristics of Solution 2.4. Linear SDE Systems with Additive and Multiplicative Noise 2.5. Mean Square Stability of SDE Solutions, Stiff in Mean Square SDE Systems. Oscillatory Stochastic Systems 2.6. Simple Numerical Methods Generalizing the Explicit Runge-Kutta Methods 2.7. Families of Numerical Methods for Solving SDE Systems. Theorem of Convergence 2.8. Mean Square Consistency of Methods 2.9. Mean Square Stability of Methods 2.10. Numerical Methods for Solving Linear SDE Systems 2.11. Variable Step Algorithms for Solving SDEs

vii

1 3 10 15 19 22 25 33 38 41

49 50 57 64 73 79 85 90 94 106 116 120

vj

contents

2.12. Numerical Solution of SDE System with Poisson Component 2.13, Applying SDE for Numerical Solution of Linear Elliptic and Parabolic Equations 2.14, Statistical Simulation of the SDE Solutions in Problems of Analysis and Synthesis of Automated Control 2.15. Numerical Experiments References

128 133 139 154 169

PREFACE The motion of a physical object or the course of a chemical reaction can frequently be described by a system of first order ordinary differential equations (ODE), resolved with respect to the derivative. The stiff ODE systems arise in the mathematical simulation of automated regulation and control systems, of chemical kinetics reactions, of electronic circuits and electric chains. As far as the simulating ODEs are nonlinear in the general case, and if the dimensionality of the system is rather large, then numerical integration is a real means of obtaining the Cauchy problem solution. The first Chapter of the book deals with the problems of numerical solution of the Cauchy problem for stiff ODE systems by Rosenbrodc-type methods (RTMs). The general solutions of consistency equations are obtained, which enable us to construct RTMs from the first to the fourth orders. The conditions are considered in which RTMs of various orders are A-stabie or L-stable. The RTMs are chosen specially for application in the solution of the stiff ODE systems with oscillatory solutions. On the basis of embedded RTMs of the first, second and third orders, a variable order variable step algorithm is constructed for the solution of stiff autonomous ODE systems. On the basis of the second order RTMs, a variable step algorithm is constructed for the solution of stiff non-autonomous ODE systems with oscillatory solutions. The requirement of an adequate mathematical description of various dynamic processes brought about the construction of models, taking into account the random fluctuations of the trajectories of these processes. The dynamic models described by stochastic differential equations (SDEs) are the most popular ones. A powerful incentive for the development of the SDEs theory was its relationship to the linear parabolic and elliptic equations in partial derivatives κ The mathematical models of dynamic systems described by SDEs are used in quite different fields: statistical radio engineering, statistical mechanics and physics, automated control, chemistry, reliability theory. The SDE systems are applied for the solution of many problems of mathematical physics, for instance, for the Bolz m aim equation. The second Chapter of the book deals with problems of the statistical simulation of the solution of the Cauchy problem for SDE systems. The mean-square convergence theorem is proved and Taylor expansions of the numerical solutions are constructed for the introduced families of numerical methods of SDE solution in the sense of I to and Stratonovich. Special numerical methods for solving linear SDE systems with an arbitrary high order of mean-square convergence are constructed. The concept of mean-square stability of numerical methods and criteria for their verification have been developed. The concept of stiff in the mean-square sense SDE systems has been introduced. A variable stepsize algorithm with automatical accuracy control for solving auto-oscillating and stiff SDE has been constructed. Applications of the numerical methods of SDE solution to the partial differential equations and to the analysis and synthesis problems of automated control of stochastic systems are considered. The book will be useful for specialists in computational mathematics and physics, in probability theory and automated control theory.

Chapter 1

Numerical Solution of the Cauchy Problem for Systems of Ordinary Differential Equations The motion of a physical object or the course of a chemical reaction can frequently be described by a system of first order ordinary differential equations (ODEs), resolved with respect to the derivative. The stiff ODE systems arise in the mathematical simulation of automated regulation and control systems, of chemical kinetics reactions, of electronic circuits and electric chains, while reducing partial differentia] equations to an ODE system by difference approximation of an operator containing the derivatives with respect to the spatial variables. As far as the simulating ODEs are nonlinear in the general case, and if the dimensionality of the system is rather large, then numerical integration is a real means of obtaining the Cauchy problem solution. The efforts to apply the classical explicit numerical Runge-Kutta methods and the linear multistep methods to the solution of stiff ODE systems encountered considerable complications because, for the absolute stability of a numerical method of solution of the asymptotically stable ODE systems the stepsize used for calculations should not exceed a certain value, which is inversely proportional to the maximal eigenvalue of the Jacobi matrix from the right-hand side of the ODE system. In the case of stiff systems, this requirement makes the stepsize very small over the entire integration interval, while it could vary to a high extent in the course of calculations for the sake of accuracy. In other words, while one applies classical explicit numerical methods for the solution of stiff ODE systems, the stepsize must have the same order of magnitude on the sections of rapid and slow variations of the solution. This is not efficient and brings about large computer costs. In connection with the stiffness problem, the demand for new efficient numerical methods of processing ODE systems arose. The work by Dahlquist in 1963 [32], where ,4-stability of the numerical method was introduced as a new notion, served as a push forward in construction of such methods. The /1-stable methods are related to the rational approximation of the exponential function, in contrast to the polynomial approximation for the explicit Runge-Kutta methods, ,4-stable methods proved to be efficient for the solution of stiff ODE systems because no integration stepsize restrictions were demanded by the absolute stability requirement. Α-stable methods require much more calculation on each integration step than classical explicit numerical methods, but combined with the procedure of stepsize variation in the solution of stiff ODE systems, they can reduce the computing time by several orders of magnitude.

2

/. Numerical Solution of the Cauchy Problem

In 1963, Rosenbrock [93] proposed numerical solution of ODE systems by methods that differ from the explicit Runge-Kutta methods in the use of the regularization [/ - Λ,α|£(ίη, y«)]"1 in each calculation of the right-hand side of the ODE system. Here |^(i,y) is the Jacobi matrix of the right-hand side, / is the unit matrix, h is the integration stepsize. The Rosenbrock-type methods (RTMs) are single-step methods; hence they do not require a specific procedure of initiating calculations and, unlike the implicit methods, there is no necessity to construct an iterative procedure for finding a numerical solution. It suffices to solve only linear systems of algebraic equations. Further investigations of the RTMs have shown that, by the corresponding choice of the parameter "a", they become A-stable. Let us list, also, some other positive features of the single-step RTM in comparison with the implicit linear multistep methods: • the convergence order of Λ-stable RTM may be higher than two; • RTMs can be used for solution of the ODE systems with discontinuous right-hand sides, e.g., in analysis of discrete control systems; • variation of the integration stepsize can be performed at each node of the mesh, not altering the stability of the method; • in application of the RTMs to variable step algorithms it is not required to recalculate the method parameters after changing the integration stepsize; • the transition from a method of one order to another is easily carried out in the calculation process. It is also possible to employ other singlestep methods of different types along with the RTMs for solution of the same Cauchy problem. The first Chapter of the book deals with the problems of numerical solution of the Cauchy problem for stiff ODE systems by the Rosenbrock-type methods. The general solutions of consistency equations are obtained, which enable us to construct RTMs from the first to the fourth orders. The conditions are considered, in which RTMs of various orders are /4-stable or L-stable. The RTMs are chosen specially for application in the solution of stiff ODE systems with oscillatory solutions. On the basis of the embedded RTMs of the first, second and third orders, a variable order variable step algorithm is constructed for the solution of stiff autonomous ODE systems. On the basis of the second order RTM, a variable step algorithm is constructed for the solution of stiff non-autonomous ODE systems with oscillatory solutions. Methods for the numerical solution error estimation used in the algorithms, procedures of solution accuracy control and integration stepsize variation are described. The results of numerical experiments

LI. The Cauchy Problem for an ODE System

3

are presented, which demonstrate certain properties and features of the constructed variable step algorithms, 1,1. The Cauchy Problem for an ODE System. Linear Systems. Stiff Systems Let us introduce some definitions and concepts, connected with the numerical solution of ODEs [29, 32, 34, 41, 42, 46, 53, 54, 55, 76, 78, 91, 100]. The Cauchy problem for an ODE system is stated in the following way: it is required to find a solution y(t) = (yi(i), ...,y^(t))T of the ODE system ).

*0 m in formulae (1.4.5). For the linear homogeneous ODE system (1.1.3), the Taylor formula (1.4.4) takes the form

For the non-autonomous ODE system (1.1.1), the Taylor formula for the numerical solution of the Cauchy problem of the two-stage RTM has the form

22

1. Numerical Solution of the Cauchy Problem

.

(1.4.6)

In formula (1.4.6) the function / and its partial derivatives are calculated at the point (i n , yn}.

1.5.

Consistency of RTMs

According to the definition, the RTM is consistent with order p, if the value of the local error equals Q(hp+l] as h -» 0 at each node of the mesh. If we assume yn = y(i n )j then we conclude from {1.2.2), that 1/2.

(1.6.9)

For α - 1/2 we obtain |β(ΐω}| = 1 for all u>, which corresponds to property 2) from (1.1.13). Note, however, that for α = 1/2 the RTM possesses one drawback. On the real half-line (—00, 0) the growth function of the RTM with a = 1/2 demonstrates the following asymptotic behaviour:

which may cause "parasitic" oscillations of the numerical solution in calculations of the stiff ODE systems with large Ke(-Xjh) for some eigenvalues of the Jacob! matrix, Taking into account representation (1.2.8) for the growth function, from (1.6.8) we have )αω2

sM

= ΪΪΑ?·

(L6 10)

-

It is easy to derive for α = 1/2 from (1.6.10), that c(7T/2) = -0.237,

β(π/2) = 0.972,

while cos(π/2) = 0, sin (π/2) — 1. A low accuracy of approximation of the functions COS(LJ) and sin (ω) by c(cj) and $(ω) and the above-described drawback do not enable us to recommend the RTM with α = 1/2 for numerical integration of the stiff ODE systems with oscillatory solutions, particularly if it is required to estimate the frequency of the solution oscillations in the course of a numerical experiment. . We see from (1.6.8) that the RTM is L-stable for a = 1. Note that application of an L-stable RTM is most efficient while solving the superstiff ODE systems, when the value Re(-Xjh) may be arbitrarily large for some eigenvalues of the Jacobi matrix. m = 2. With regard to (1.6.2) for RTM of order 2 we obtain

28

L Numerical Solution of the Cauchy Problem

-

{1_az}2

.

From (1.6.7) we conclude that inequality (1.6.4) is fulfilled for all ω if (1/2-2α)(1/2-α)2 1/4.

(1.6.12)

L-stability requirement (1.6.3) for the RTM of order 2 assumes the form α2 -2α +1/2 = 0.

Both solutions of this equation a = ^- satisfy (1.6,12). These were the particular parameters α which Rosenbrock applied in his method [93]. The growth function (1.6.11) for α = 1/4 admits the form

whence |Α(ΐω)| = 1 for all ω and C(W)

1 - 3/8ω2 + l/256u>4 = 1+«16' '

(1.6.14)

-

The functions c(w) and s(w) from (1.6.14) approximate cos (ω) and sin(w) with sufficient accuracy, for instance, c(jr/2) = 0.074, β(ττ/2) = 0.997, which enables us to recommend the RTM with the growth function (1.6.13) for numerical integration of the stiff ODE systems with oscillatory solutions, though, just as in the previous case with α — 1/2, one has to take into account the possibility of "parasitic" oscillations of the numerical solution while integrating superstiff ODE systems. m = 3. With regard to (1.6.2), for the 3-rd order RTM we obtain -I- (1/2 - 3α + 3α 2 )ζ 2 + (1.6.15) With regard to (1.6.5), we obtain

1.6. Α-stability of RTMs

29

μ^3) = 1/2 - 3α, μ£] = 1/6 - 3o/2 + 3α2.

According to (1.6.7), the 3-stage RTM of order 3 is Λ-stable if the inequalities

2α3 > 3α2 - 3α/2 + 1/6 > Ο

[ l.o.lo)

hold. It is proved in [35], that inequalities (1.6.16) are consistent if 1/3 < α < 1.0685790.

(1.6.17)

L-stabi ty requirement (1.6.3) for the RTM of order 3 takes the form a3 - 3α2 + 3o/2 - 1/6 - 0.

(1.6.18)

The only real root of (1.6.18), lying in the interval (1.6.17), is α = 0.4358665215. There is no 3-stage RTM of order 3 yet, which could be recommended for numerical integration of the stiff ODE systems with oscillatory solutions. τη = 4. With regard to (1.6.2) for the 4-th order RTM we obtain

R(z) =

{l + (1 - 4a)z + (1/2 - 4α + 6α V +

* t l

tLjo 1

(1/6 - 2α + 6α2 - 4α 3 )^ 3 + (1/24 - 2α/3 + 3α2 - 4α3 + α 4 )ζ 4 }

(1.6.19)

and by formula (1.6.5)

μ(2} = 1/2- 4α, j44) = 1/2 - 2α + 6α2,

μ(*} = 1/24 - 2α/3 + 3α2 - 4α3.

According to (1.6.7), the four-stage RTM of order 4 is /1-stable if the following inequalities hold 8α5 - 17α4 + 32α3/3 - 17α2/6 + α/3 - 1/72 < Ο, , , 2α4 > 4α3 - 3α2 + 2α/3 - 1/24 > 0.

(1.6.20)

It is proved in [35] that conditions (1.6.20) hold for 0.3943376 < a < 1.2805798.

(1.6,21)

L-stability requirement (1.6.3) for the RTM of order 4 assumes the form ο 4 - 4α3 + 3α2 - 2α/3 + 1/24 = 0.

(1.6.22)

The only real root of (1.6.22), lying in the interval (1.6.21) is a = 0.57281606. For the growth function of the RTM of order 4 for α - 0.3943376 we obtain

30

/. Numerical Solution of the Cauchy Problem

|H(«r/2)| = 0,992,

c(jr/2) = 0.01,

β(π/2) = 0.992,

which enables us to recommend RTM with such a parameter α for numerical integration of the stiff ODE systems with oscillatory solutions. For m > 5 the Α-stability testing for RTM becomes a complex problem, which can be solved only with the help of a computer. For instance, for m = 5, a method is Λ-stable under the conditions

< 0,

where

/45> = 1/2 -5a,

/

5

= l/6-5a/2+10a 22,

/45) = 1/24 - 5a/6 + 5a2 - 10a3, /45) = 1/120 -5a/24 + 5a 2 /3-oa 3 + 5a4, It is demonstrated in [66] that, for m = 5, the RTM is Α-stable as 0.246506 0 for all A € 3; 6) Ρ(Ω) = 1; oo

oo

B) P( U An) = Σ P(An) for An e S, Ai Π A, = 0, i φ j. '

n=l

Definition 2.1.3. Λ random variable on the probability space {Ω,9, Ρ} is any £r- measurable function ζ : Ω —» . Characteristics of a random variable:

is the distribution function of ξ. If the representation X

F(x} = I α '~J — 00

holds, the function p^(x) is said to be ί/ie density function of a continuous random variable ξ. OQ

-]

is the mean of a continuous random variable ξ.

κ

is the mean of a discrete random variable ξ. 00

Dt= J(x-mt — 00

is the variance of a continuous random variable ξ.

κ k=l

is the variance of a discrete random variable .

52

2. Statistical Simulation of the Conchy Problem Solution

Let A be some event from σ-algebra o, where P(A) > 0. Then P(A}

is a conditional function of the distribution ζ relative to A.

is conditional density of distribution ξ relative to A. 00

τη(ξ/Α) = Ι is conditional mean ξ relative to A. The concepts of conditional function of distribution, conditional density of distribution and conditional mean are generalized with respect to any σ-subalgebras ^SQ C £f. 2.1.2. Random vectors Definition 2.1.4. Let ξι, ξ% be random variables. Then

= ((£1 - "*i)(6 - m 2 )> is the covariance of the random variables ξι and £2- Here m, is the mean value of &, i — 1,2, {·} is the operation of taking the mean value. Definition 2.1.5. Let £, i = Ι , . , . , Λ Γ , be random variables. Then the vector ξ = (ξι, . . . ,£ΛΓ) iR s^id to be the Ν -dimensional random vector, Definition 2.1.6. The function

is said to be the joint distribution function of the random variables &, ΐ = 1, . . . , ΛΓ, or function of distribution of the Ν -dimensional random vector ξ. If the following representation holds

—oo

the function Ρ ξ ( χ \ , . . . , Χ Λ Γ } is said to be ifte density of distribution of the Ν-dimensional random vector ξ.

2.1. Elements of Probability Theory

53

Definition 2.1,7. The symmetric, positive definite matrix DS = ((ξ -

is said to be the covariance matrix of the vector ξ. The matrix Γ ς = is said to be the second moment matrix of the vector ξ. Definition 2.1.8. The random variables £,, i = 1, . . ., JV, are independent in totality if Ν

2,1.3. Stochastic processes Definition 2.1.9. A stochastic process ξ ( · ) on the probability space { Ω , ο , Ρ } is a family of random variables £(ί,ω), where t > 0 is a real parameter, ω € Ω, Definition 2.1.10. A function £(i,u?) for a fixed ω is said to be α trajectory of the stochastic process. Characteristics of a stochastic process: The main probable characteristics of a stochastic process are its consistent finite- dimensional distributions

ii < t 2 < ... < ίκ, Κ = 1,2, ----

!%(*.*) = is ine one- dimensional distribution function of the stochastic process da:

is f/ie one-dtmensiona/ density function of the stochastic process oo

mΐξ(ί) f(t) = / —ooo —

is the mean of the stochastic process

xpf(t,x]dx

54

2, Statistical Simulation of the Cauchy Problem Solution

is the covariance matrix of the stochastic process

is the second moments matrix of the stochastic process f (·).

is the correlation function of the stochastic process

is the mutual correlation function of the processes f i ( - ) and £ϊ{·). Definition 2.1.11. A stochastic process ξ(·) is said to be Gaussian if for any finite set of times ij , . . . , tjf the random variables £(£j), . - . , £{i/c) have a joint normal distribution. Definition 2.1.12. A stochastic process £(·) is stationary in a broad sense if its mean is constant for all t > 0, and the correlation function is

Definition 2.1.13. A stochastic process ξ(·) is said to be α Markov process if for a fixed f(i) the random variables ξ(«), κ > t do not depend on £(s), s < ί, Definition 2.1.14. A stochastic process ^(*) is said to be a process with independent increments if for all ίο < ίι < . . . < IK the random variables (>), ί(ίι) - ξ(ίο)ι - · Μ ξ (ίίί) - ξ(*κ-\) are independent in totality. 2.1.4. Mathematical statistics Characteristics for a sample from a random variable; K

is ifte sample mean. Here ξ^, A; = 1, . . . , K", are independent sample values of«.

is the sample variance.

2.1. Elements of Probability Theory

*!, = -

55

A

is f/i€ sample covariance of the random variables ξι and £ 2 .

is ί/ie sample (empirical) distribution function of ξ. Here jx is the number of sample elements £n such that £,· < or. In practice the samples from continuous distributions frequently subject to α grouping. As this takes place the individual sample values are not given, but only the number of sample values is indicated, which are in the intervals of a certain mesh. Then each j'-th interval of the mesh is accepted as the basis of the rectangular of height ^-, where Δ is the length of the interval. The figure obtained is called the histogram of a sample.

2.1.5. Statistical simulation of discrete and continuous random variables The process of obtaining, on a computer, a sequence of sample values of a random variable ξ with the given distribution function ίξ(ζ) is called the numerical simulation of the random variable ξ. Random variables are usually simulated by transforming one or several independent values of the random variable o, distributed uniformly in the interval (0, 1): Fa(x) = ar, Q