Computational Physics : With Worked Out Examples in FORTRAN and MATLAB 9783110515145, 3110515148

Citation preview

Michael Bestehorn Computational Physics

Also of Interest Computational Physics. Mit Beispielen in Fortran und Matlab Michael Bestehorn, 2016 ISBN 978-3-11-037288-5, e-ISBN (PDF) 978-3-11-037303-5, e-ISBN (EPUB) 978-3-11-037304-2

Biomedical Imaging. Principles of Radiography, Tomography and Medical Physics Tim Salditt, Timo Aspelmeier, Sebastian Aeffner, 2017 ISBN 978-3-11-042668-7, e-ISBN (PDF) 978-3-11-042669-4, e-ISBN (EPUB) 978-3-11-042351-8 Dynamics of Solid Structures. Methods using Integrodifferential Relations Georgy Viktorovich Kostin, Vasily V. Saurin, 2017 ISBN 978-3-11-051623-4, e-ISBN (PDF) 978-3-11-051644-9, e-ISBN (EPUB) 978-3-11-051625-8 Computational Physics of Electric Discharges in Gas Flows Sergey T. Surzhikov, 2012 ISBN 978-3-11-027033-4, e-ISBN (PDF) 978-3-11-027041-9

Computational Sciences Ponnadurai Ramasami (Ed.), 2017 ISBN 978-3-11-046536-5, e-ISBN (PDF) 978-3-11-046721-5, e-ISBN (EPUB) 978-3-11-046548-8

Michael Bestehorn

Computational Physics | With Worked Out Examples in FORTRAN® and MATLAB®

Physics and Astronomy Classification Scheme 2010 05.10.Ln, 05.45.-a, 05.65.+b, 07.05.Mh, 44.05.+e, 45.10.-b, 47.11.-j, 47.85.-g Author Prof. Dr. Michael Bestehorn BTU Cottbus-Senftenberg Dept. of Statistical Physics and Nonlinear Dynamics Erich-Weinert-Str. 1 03046 Cottbus Germany [email protected]

ISBN 978-3-11-051513-8 e-ISBN (PDF) 978-3-11-051514-5 e-ISBN (EPUB) 978-3-11-051521-3 Library of Congress Control Number: 2018934247 Bibliographic information published by the Deutsche Nationalbibliothek The Deutsche Nationalbibliothek lists this publication in the Deutsche Nationalbibliografie; detailed bibliographic data are available on the Internet at http://dnb.dnb.de. © 2018 Walter de Gruyter GmbH, Berlin/Boston Cover image: Michael Bestehorn Typesetting: le-tex publishing services GmbH, Leipzig Printing and binding: CPI books GmbH, Leck ♾ Printed on acid-free paper Printed in Germany www.degruyter.com

Contents 1 1.1 1.2 1.2.1 1.2.2 1.2.3 1.2.4 1.3 1.3.1 1.3.2 1.3.3

Introduction | 1 Goal, contents, and outline | 1 The environment required for program development | 4 Operating system | 4 Software packages | 5 Graphics | 5 Program development and a simple script | 6 A first example – the logistic map | 7 Map | 7 FORTRAN | 8 Problems | 11

2 2.1 2.1.1 2.1.2 2.1.3 2.1.4 2.2 2.2.1 2.2.2 2.2.3 2.3 2.3.1 2.3.2 2.3.3 2.4 2.4.1 2.4.2 2.4.3 2.5 2.5.1 2.5.2 2.5.3

Nonlinear maps | 12 Frenkel–Kotorova model | 12 Classical formulation | 12 Equilibrium solutions | 13 The standard map | 13 Problems | 14 Chaos and Lyapunov exponents | 15 Stability, butterfly effect, and chaos | 15 Lyapunov exponent of the logistic map | 16 Lyapunov exponents for multidimensional maps | 17 Affine maps and fractals | 20 Sierpinski triangle | 22 About ferns and other plants | 23 Problems | 24 Fractal dimension | 25 Box-counting | 25 Application: Sierpinski triangle | 26 Problem | 27 Neural networks | 27 Perceptron | 28 Self-organized maps: Kohonen’s model | 34 Problems | 39

3 3.1 3.1.1 3.1.2

Dynamical systems | 40 Quasilinear differential equations | 40 Example: logistic map and logistic ODE | 41 Problems | 42

VI | Contents

3.2 3.2.1 3.2.2 3.2.3 3.2.4 3.2.5 3.2.6 3.2.7 3.3 3.3.1 3.3.2 3.3.3 4 4.1 4.1.1 4.1.2 4.2 4.2.1 4.2.2 4.2.3 4.3 4.3.1 4.3.2 4.3.3 4.4 4.4.1 4.4.2 4.4.3 4.4.4 4.4.5 4.5 4.5.1 4.5.2 4.5.3 4.5.4 4.5.5 4.5.6 4.6 4.6.1 4.6.2

Fixed points and instabilities | 43 Fixed points | 43 Stability | 43 Trajectories | 44 Gradient dynamics | 45 Special case N = 1 | 45 Special case N = 2 | 45 Special case N = 3 | 47 Hamiltonian systems | 50 Hamilton function and canonical equations | 50 Symplectic integrators | 51 Poincaré section | 56 Ordinary differential equations I | 58 Newton’s mechanics | 58 Equations of motion | 58 The mathematical pendulum | 59 Numerical methods of the lowest order | 60 Euler method | 60 Numerical stability of the Euler method | 60 Implicit and explicit methods | 63 Higher order methods | 64 Heun’s method | 64 Problems | 67 Runge–Kutta method | 67 RK4 applications: celestial mechanics | 73 Kepler problem: closed orbits | 73 Quasiperiodic orbits and apsidal precession | 76 Multiple planets: is our solar system stable? | 76 The reduced three-body problem | 79 Problems | 86 Molecular dynamics (MD) | 86 Classical formulation | 86 Boundary conditions | 87 Microcanonical and canonical ensemble | 88 A symplectic algorithm | 89 Evaluation | 90 Problems | 94 Chaos | 95 Harmonically driven pendulum | 95 Poincaré section and bifurcation diagrams | 97

Contents |

4.6.3 4.6.4 4.6.5 4.7 4.7.1 4.7.2 4.7.3 4.7.4 4.7.5

VII

Lyapunov exponents | 98 Fractal dimension | 106 Reconstruction of attractors | 109 ODEs with periodic coefficients | 111 Floquet theorem | 111 Stability of limit cycles | 112 Parametric instability: pendulum with an oscillating support | 112 Mathieu equation | 114 Problems | 116

5 5.1 5.1.1 5.1.2 5.2 5.2.1 5.2.2 5.3 5.3.1 5.3.2 5.4 5.4.1 5.4.2 5.4.3 5.4.4 5.5 5.5.1 5.5.2 5.5.3 5.6

Ordinary differential equations II | 117 Preliminary remarks | 117 Boundary conditions | 117 Example: ballistic flight | 118 Finite differences | 118 Discretization | 119 Example: Schrödinger equation | 122 Weighted residual methods | 128 Weight and base functions | 128 Example: Stark effect | 130 Nonlinear boundary value problems | 132 Nonlinear systems | 132 Newton–Raphson | 133 Example: the nonlinear Schrödinger equation | 134 Example: a moonshot | 136 Shooting | 140 The method | 140 Example: free fall with quadratic friction | 141 Systems of equations | 142 Problems | 143

6 6.1 6.1.1 6.1.2 6.1.3 6.2 6.2.1 6.2.2 6.2.3 6.2.4

Partial differential equations I, basics | 144 Classification | 144 PDEs of the first order | 144 PDEs of the second order | 147 Boundary and initial conditions | 149 Finite differences | 153 Discretization | 153 Elliptic PDEs, example: Poisson equation | 156 Parabolic PDEs, example: heat equation | 162 Hyperbolic PDEs, example: convection equation, wave equation | 167

VIII | Contents

6.3 6.3.1 6.3.2 6.3.3 6.4 6.4.1 6.4.2 6.4.3 6.4.4

Alternative discretization methods | 173 Chebyshev spectral method | 173 Spectral method by Fourier transformation | 178 Finite-element method | 182 Nonlinear PDEs | 186 Real Ginzburg–Landau equation | 186 Numerical solution, explicit method | 188 Numerical solution, semi-implicit method | 189 Problems | 190

7 7.1 7.1.1 7.1.2 7.2 7.2.1 7.2.2 7.2.3 7.3 7.3.1 7.3.2 7.3.3 7.3.4 7.4 7.4.1 7.4.2 7.4.3

Partial differential equations II, applications | 193 Quantum mechanics in one dimension | 193 Stationary two-particle equation | 193 Time-dependent Schrödinger equation | 196 Quantum mechanics in two dimensions | 202 Schrödinger equation | 202 Algorithm | 202 Evaluation | 203 Fluid mechanics: flow of an incompressible liquid | 204 Hydrodynamic basic equations | 204 Example: driven cavity | 206 Thermal convection: (A) square geometry | 211 Thermal convection: (B) Rayleigh–Bénard convection | 218 Pattern formation out of equilibrium | 225 Reaction-diffusion systems | 225 Swift–Hohenberg equation | 234 Problems | 238

8 8.1 8.1.1 8.1.2 8.1.3 8.2 8.2.1 8.2.2 8.3 8.3.1 8.3.2 8.4 8.4.1 8.4.2

Monte Carlo methods (MC) | 240 Random numbers and distributions | 240 Random number generator | 240 Distribution function, probability density, mean values | 241 Other distribution functions | 242 Monte Carlo integration | 245 Integrals in one dimension | 245 Integrals in higher dimensions | 247 Applications from statistical physics | 250 Two-dimensional classical gas | 250 The Ising model | 255 Differential equations derived from variational problems | 265 Diffusion equation | 265 Swift–Hohenberg equation | 267

Contents | IX

A A.1 A.1.1 A.1.2 A.1.3 A.1.4 A.2 A.2.1 A.2.2 A.3 A.3.1 A.3.2 A.3.3 A.3.4 A.3.5 A.4 A.4.1 A.4.2 A.4.3

Matrices and systems of linear equations | 270 Real matrices | 270 Eigenvalues and eigenvectors | 270 Characteristic polynomial | 270 Notations | 271 Normal matrices | 271 Complex matrices | 272 Notations | 272 Jordan canonical form | 273 Inhomogeneous systems of linear equations | 274 Regular and singular system matrices | 274 Fredholm alternative | 275 Regular matrices | 275 LU decomposition | 276 Thomas algorithm | 278 Homogeneous systems of linear equations | 279 Eigenvalue problems | 279 Diagonalization | 279 Application: zeros of a polynomial | 282

B B.1 B.2 B.2.1 B.2.2 B.2.3 B.2.4 B.2.5 B.2.6 B.3 B.3.1 B.3.2 B.3.3 B.4 B.4.1 B.4.2 B.4.3 B.4.4 B.4.5 B.4.6 B.4.7

Program library | 284 Routines | 284 Graphics | 285 init | 285 contur | 285 contur1 | 285 ccontu | 286 image | 286 ccircl | 286 Runge–Kutta | 286 rkg | 286 drkg | 287 drkadt | 287 Miscellaneous | 287 tridag – Thomas algorithm | 287 ctrida | 287 dlyap_exp – Lyapunov exponents | 288 schmid – orthogonalization | 288 FUNCTION volum – volume in n dimensions | 288 FUNCTION deter – determinant | 289 random_init – random numbers | 289

X | Contents

C C.1 C.2 C.3 C.4 C.5 C.6 C.7

Solutions of the problems | 290 Chapter 1 | 290 Chapter 2 | 291 Chapter 3 | 292 Chapter 4 | 292 Chapter 5 | 301 Chapter 6 | 303 Chapter 7 | 306

D D.1 D.2 D.2.1 D.2.2 D.2.3

README and a short guide to FE-tools | 310 README | 310 Short guide to finite-element tools from Chapter 6 | 313 mesh_generator | 313 laplace_solver | 315 grid_contur | 315

Index | 317

1 Introduction 1.1 Goal, contents, and outline The present text tries to show, by means of examples coming from different corners of physics, how physical and mathematical questions can be answered using a computer. Nowadays, theoretical physicists are mainly concerned with two subjects: one is the devising of models, normally in the form of differential equations, ordinary or partial, while the other one is their approximate solution, normally with the help of larger or smaller computers. In a certain way, computational physics can be seen as the link between theory and experiment. The often involved equations need a numerical treatment, usually in connection with certain simplifications and approximations. Once the code is written, the experimental part begins. A computer on which, for instance, a turbulent pipe flow is simulated, can to some extent be regarded as a test station with easy access to output values such as flow rates, velocities, temperatures, and also to input parameters like material constants or geometric boundary conditions. Problems from classical mechanics normally rely on ordinary differential equations, while those of electrodynamics, quantum mechanics, continuum mechanics, or hydrodynamics are formulated by partial differential equations, which are often nonlinear. This is neither a numeric book, nor a tutorial for a special computer language for which we refer to the literature [1]. On the other hand, most of the discussed algorithms are realized in FORTRAN. FORTRAN is an acronym composed of FORmula TRANslator. This already indicates that it is more appropriate and convenient for solving problems originating from theoretical physics or applied mathematics, like managing complex numbers or solving differential equations. During the past 10 years, the software package MATLAB has enjoyed more and more popularity in research as well as in teaching [2]. The notion is formed as another acronym from MATrix LABoratory. Thus MATLAB shows its strengths in the interactive treatment of problems resulting from vector or matrix calculus, but other problems related to differential equations can also be tackled. Programming is similar to FORTRAN and for beginners, many things are easier to implement and to handle. In addition, MATLAB captivates with its comfortable and convenient graphic output features. MATLAB is devised as a standalone system. It runs under Windows and Linux. No additional text editor, compiler, or software libraries are needed. However, contrary to FORTRAN, a MATLAB code once written is not compiled to an effective machine code but rather interpreted line by line. Processes may thus become incredibly slow. MATLAB may have some advantages in program development and testing, but if in the end a fast and effective code is desired (which should usually be the case), a compiler https://doi.org/10.1515/9783110515145-001

2 | 1 Introduction

language is inevitably preferred. Nevertheless we shall give some examples also in MATLAB, mainly those based on matrix calculation and linear algebra. In the meantime, highly developed computer algebra systems exist and can be used to check the formulas derived in the book, but we do not go into this matter. The motto of the book could read ‘Learning by Doing.’ Abstract formulations are avoided as much as possible. Sections where theory dominates are followed by an example or an application. The chapters of course can be read in sequence, albeit not necessarily. If someone is interested for instance in partial differential equations, it is possible to start with Chapter 6 and consult Chapters 3-5 for potential issues. The same is true for the statistic part, Chapter 8. It should be understandable also without knowledge of the preceding chapters. Depending on the subject, certain basic knowledge in (theoretical) physics is required. For Chapters 4, 5 and 7, classical mechanics and a little bit of quantum mechanics should be at hand, while for Chapter 7, additional basics in hydrodynamics won’t hurt. In Chapter 8 we presume a certain familiarity with the foundations of statistical mechanics. In any case, it is recommended to have a computer or notebook running nearby and, according to the motto, experiment with and check the presented routines immediately while reading. The topics up to Chapter 4, and to some extent also in Chapter 5, can be categorized under the notion low-dimensional systems, problems that can be described with only a few variables. Besides iterative maps, applications from Newton’s classical mechanics come to mind, for instance, the computations of planetary orbits or the motion of a pendulum (Figure 1.1). Naturally for this case, ordinary differential equations play a

2

0

–2

–2

0

2

Fig. 1.1: Time evolution of low-dimensional systems from Chapter 4. Left: two-planet problem with gravitational interaction. The orbit of the inner planet (red) is unstable and the planet is thrown out of the system. Right: deterministic chaos for the driven pendulum, Poincaré section.

1.1 Goal, contents, and outline | 3

1

0.8

0.6

0.4

0.2

0 0

0.2

0.4

0.6

0.8

1

Fig. 1.2: Numerical solutions of field equations, see Chapter 7. Left: time-dependent Schrödinger equation, probability density, and center of mass of an electron in a constant magnetic field. Right: Hydrodynamic vortex for the driven cavity.

central role. For three dimensions and more (or better: degrees of freedom), chaotic behavior becomes possible, if not the rule. Large parts of Chapters 2-4 are dedicated to the so-called deterministic chaos. Starting with Chapter 5 high-dimensional (or infinite-dimensional) systems come into the focus, normally described by field theories in the form of partial differential equations. These can be linear as is very often seen in quantum mechanics or electrodynamics, but also nonlinear as in hydrodynamics (Figure 1.2) or in diffusion-driven macroscopic pattern formation (Figure 1.3). Caused by nonlinearities, patterns may emerge on most diverse spatio-temporal scales, the dynamics becoming chaotic or turbulent (Figure 1.4).

Fig. 1.3: Macroscopic pattern formation for the example of a reaction-diffusion system, time series. Out of a large-scale oscillating solution, nuclei emerge spontaneously followed by the generation of small-scaled Turing structures. The numerical integration of the Brusselator equations is shown in Chapter 7.

4 | 1 Introduction

Fig. 1.4: Two-dimensional flow field showing turbulence, time series. Due to a vertical temperature gradient, an initially stable layer becomes unstable (top). Warmer air (red) rises in the form of ‘plumes’ in the colder air (blue) (from Chapter 7).

Fig. 1.5: Monte Carlo simulation of a gas comprised of 1000 particles with hardcore rejection and long range attraction from Chapter 8. Temperature decreases from left to right, and a phase transition in a condensed state can be detected.

Another chapter is devoted to Monte Carlo methods. Here, randomness plays a key role in the determination of equilibrium states. Using Monte Carlo simulations, phase transitions can be studied without resorting to differential equations (Figure 1.5). The application on the Ising model of spatially arranged spins is a prominent example.

1.2 The environment required for program development 1.2.1 Operating system We shall discuss programs mainly written in FORTRAN, but also in MATLAB. MATLAB is conceived as its own environment, where codes can be written, developed, and run without leaving the MATLAB application. The operating system of the performing computer plays a minor role, so Windows can be used as well. On the other hand, writing and developing FORTRAN codes is quite different. One needs a compiler, a

1.2 The environment required for program development |

5

text editor, and a convenient file system. For more ambitious applications, software libraries like LAPACK should be at hand. Also a graphic library is important to have at least the possibility to plot results in a pleasant and appealing way. If one wants to have all these possibilities preferably free of charge, the only possibility is to do it under Linux. Thus, we assume the reader has a Linux operating system running where GNU FORTRAN 95 is installed, together with the important libraries. A certain code written with a text editor, e.g., emacs or xemacs, can be compiled via $ gfortran program.f to a binary file named a.out. That one can be executed with $ ./a.out This of course is the most simple case where no further libraries are involved.

1.2.2 Software packages Besides a compiler, e.g., gfortran from GNU, we shall need external routines to solve standard problems, mainly from the LAPACK-Library (Linear Algebra PACKage). LAPACK can be also installed for free via the internet [4]. If the source code resorts to LAPACK routines, one should use the compiler options -l and -L: gfortran program.f -L/usr/lib -llapack The option -L/. . . points to the path where the following library is located on the computer.

1.2.3 Graphics For graphic output we employ the package PGPLOT [5]. All figures in this first chapter and most of the book are generated with PGPLOT. Of course the reader can use other packages if available, but PGPLOT provides all necessary basic tools and is also free of charge. The compiler options for a source that relies both on LAPACK and GNUPLOT should look like this: gfortran program.f -L/usr/lib -llapack -lpgplot if the PGPLOT-Library is installed in /usr/lib.

6 | 1 Introduction

1.2.4 Program development and a simple script Program development can be considered as an iterative process. The desired result is approached slowly and reached, if at all, after N iterations. One iteration step is built up of the following points: (1) editing, (2) compiling, (3) executing, (4) go back to (1) In (1), an ASCII file is generated using a text editor. This file is the source and usually has the extension .f, say, name.f. The editor is invoked, e.g., emacs by $ emacs name.f After finishing the first round of editing the source file, compiling and possible linking with libraries (2) can be achieved by a script file gfortran -O1 $1.f -o $1 -L/usr/lib -llapack -lpgplot Let us call the script containing the above line ’make_f95’. Before executing the file, it must receive the attribute executable, which is done by $ chmod u+x make_f95 Running the script with $ ./make_f95 name initializes the variable $1 with name. Further options (optimization, etc.) of the FORTRAN compiler can be found in the built-in manual by $ man gfortran The option -o transfers the name in $1 to the created binary executable, instead of calling it a.out (default). (3) If the compiler passes without error messages, one finds in the working directory a binary file, here name, that can simply be executed with $ ./name

1.3 A first example – the logistic map |

7

1.3 A first example – the logistic map 1.3.1 Map To inspire the reader a bit, we conclude this chapter with a numerically very simple example. We study the map x n+1 = ax n (1 − x n ) ,

n = 0, 1, 2. . .

0≤a≤4,

0 ≤ x0 ≤ 1

(1.1)

which is known as the logistic map. In this way we can also demonstrate the usage of the graphic package PGPLOT. The recursion rule (1.1) can be considered a simple model of the temporal evolution of a certain population if x n denotes the population density at time (in the year) n. The term ax n alone leads to an exponential increase if a > 1, otherwise leading to the extinction of the species. For a > 1, the nonlinearity −ax2n restricts the growth due to lack of resources (limited foods) and leads to a saturation. The constraint 0 ≤ a ≤ 4 originates from the condition 0 ≤ x n ≤ 1 for all x n . Depending on a, we have one or two stationary solutions, namely (1)

xs = 0

for all

(2)

xs = 1 − 1/a

a,

which are computed from

(i)

for

1 3. To this end we examine the behavior of an infinitesimal deviation (2) |ϵ0 | ≪ 1 of xs (2) x n = xs + ϵ n (1.5) under iteration. Inserting (1.5) into (1.1) yields (Taylor expansion) (2)

(2)

(2)

(2)

xs + ϵ n+1 = f(xs + ϵ n ) = f(xs ) + f 󸀠 (xs ) ϵ n + O(ϵ2n )

(1.6)

or, considering only linear terms in ϵ n (2)

ϵ n+1 = f 󸀠 (xs ) ϵ n .

(1.7) (2)

The small deviation ϵ0 will grow during iteration if |f 󸀠 (xs )| > 1, which leads with (1.4) to the two regions of instability a3.

8 | 1 Introduction The first one is excluded by (1.2). What happens for a > 3? The computer solutions introduced below show a periodic behavior, (Figure 1.6) x n = x n−2 = xp1 ,

x n+1 = x n−1 = xp2

or xp1 = f(f(xp1 )) ,

xp2 = f(f(xp2 )) .

Applying again a linear stability analysis shows that the periodic solution becomes unstable for a > 1 + √6 and a more complex periodic cycle where x n alters now between four values arises (Problems).

1.3.2 FORTRAN Numerically we can study the logistic map with the following code

C c

c C

c

PROGRAM logist REAL*8 :: a,x,amin,amax ! double precision (8 bytes) region for a amin=2.8; amax=4. initialize graphics (pgplot) CALL INIT(SNGL(amin),SNGL(amax),0.,1.,7.,1.,1) itmax=200 ! number of iterations for each a ivor=1000 ! number of hidden iterations before plotting tiny=1.E-6 ! initial value for x da=(amax-amin)/1000. ! step width for a loop for 1000 a values DO a=amin,amax,da logistic map x=tiny; ic=0 ! x0 DO it=1,itmax+ivor x=a*x*(1.-x) IF(it > ivor) THEN after pre-iter. plot a pixel at (a,x) with color ic ic=MOD(ic,15)+1; CALL PGSCI(ic) CALL PGPT(1,SNGL(a),SNGL(x),-1) ENDIF ENDDO ENDDO

1.3 A first example – the logistic map | 9

1 x 0.8

0.6

0.4

0.2 a 0

3

3.5

4

Fig. 1.6: Logistic map, period doublings, and (2) chaos. The stationary solution xs (bold) becomes unstable for a > 3.

c labels for axes CALL PGSCI(1); CALL PGSCH(2.5) CALL PGTEXT(3.5,0.02,'a'); CALL PGTEXT(1.2,0.9,'x') c steady solution x_{\text{s}}(a) CALL PGSLW(5) ! in bold CALL PGMOVE(SNGL(amin),1.-1./amin) da=(amax-amin)/99. DO as=amin,amax,da xs=1.-1./as CALL PGDRAW(as,xs) ENDDO c terminate graphic CALL PGEND END Compiling and executing creates a figure like Figure 1.6 on the screen. The subroutine init comprises PGPLOT calls for initializing the graphic screen. It is described in more detail in Appendix B. A quick glance at the program shows that the implicit type declarations of FORTRAN have been utilized. That means it is not necessary to declare variables in the header of each routine if they are real or integer scalar quantities. Each variable beginning with a letter from a to h and then from o to z is defined real (single precision) as default, while all variables beginning with i to n are integers (4 bytes). On the other hand you can often read in FORTRAN guides (normally written by mathematicians or computer scientists) sentences like we advise strongly against using type conventions in FORTRAN. Of course the usage of default declarations seems to be a bit

10 | 1 Introduction

sloppy but it saves a lot of typing and going back and forth while writing a program. Those who do not like this way can put the statement IMPLICIT NONE in the first line of each routine and switch off the implicit declarations (there are also compiler options that effect the same). However, each variable has to then be defined explicitly. 1.3.2.1 MATLAB The same code can be written in MATLAB. The syntax is very similar to FORTRAN but easier to write. The graphics calls are different. A code that produces the same Figure 1.6 is shown here: clear; amin=2.8; amax=4.0; itmax=200; ivor=1000; tiny=1.e-6;y=zeros(1,imax);xx=zeros(1,imax); da=(amax-amin)/1000; for a=2.8:da:4.0 x=tiny; for i=1:itmax+ivor; x=a*x*(1-x); if i>ivor y(i-ivor)=x; xx(i-ivor)=a; % to save CPU time it is essential to end % store the x-values for each a and end % then plot them all together plot(xx,y,'.'); end However, execution time in MATLAB is normally much longer than in a compiler language. To demonstrate this, we remove all graphic output and compare the two codes, first in MATLAB: clear; t=cputime; itmax=100000000; ivor=itmax; y=zeros(1,imax); a=3.95; x=0.01; for i=1:ivor+itmax x=a*x*(1-x); if i>ivor y(i-ivor)=x; end end cputime-t % prints the CPU time used

Bibliography |

11

then in FORTRAN REAL*8 :: a,x,y(100000000) common y ! outwit the FORTRAN optimizer! itmax=100000000; ivor=itmax a=3.95; x=0.01 DO it=1,itmax+ivor x=a*x*(1.-x) IF(it.gt.ivor) THEN y(it-ivor)=x ENDIF ENDDO CALL CPU_TIME(s) WRITE(6,*) s END Both codes are running on an HP ProDesk 400 G3 MT, the MATLAB code under Windows 7 Professional, the FORTRAN code under Linux using GNU-FORTRAN 95. MATLAB needs about 3 seconds for the 20×106 iterations. If the optimizer in FORTRAN is turned off (-O0), FORTRAN takes 0.95 seconds. On the first optimization level (-O1), we get 0.65 seconds, using the second up to the fourth we find 0.6 seconds. Starting with -O2, it is necessary to outflank the FORTRAN optimizer and to write the y-array into a dummy COMMON-block. Otherwise gfortran would recognize the computed quantities inside the iteration loop are not used further and would just skip the loop completely.

1.3.3 Problems With pencil and paper: 1. Compute the period two cycle xp1 , xp2 of the logistic map. 2. Show that this cycle becomes unstable if a > 1 + √6. And to code: Change the code for the logistic map to scale up certain regions in the x-a plane. The regions could be defined using the mouse cursor.

Bibliography [1] [2] [3] [4] [5]

S. J. Chapman, FORTRAN 95/2003 for Scientists and Engineers, McGraw Hill Book Co. (2007). Matlab – Mathworks, http://mathworks.com/products/matlab/ Maplesoft – Mathworks, http://www.maplesoft.com/solutions/education/ LAPACK – Linear Algebra PACKage – Netlib, www.netlib.org/lapack/ T. J. Pearson, Guide to the graphics package PGPLOT, Caltech, USA: http://www.astro.caltech.edu/~tjp/pgplot/old_manual.ps.gz

2 Nonlinear maps In this chapter we continue with recursion formulas where, starting from an initial (set of) number(s), all subsequent values of one (ore more) variable(s) follow in a unique and deterministic way.

2.1 Frenkel–Kotorova model 2.1.1 Classical formulation As a simple model of a solid body in an externally given, periodic potential V(x) with V(x) = V(x + 1)

(2.1)

we examine a one-dimensional chain of N mass points, each with mass m. The mass points are connected with chains having the spring rate D = 1 and the equilibrium length zero (Figure 2.1). They are located at x n and have the momentum p n . The Hamilton function reads: N

H(x n , p n ) = ∑ [ n=1

p2n 1 + V(x n ) + (x n − x n−1 )2 ] . 2m 2

(2.2)

To state the dynamic problem one may write up the canonical equations ṗ n = −

∂H , ∂x n

ẋ n =

∂H ∂p n

(2.3)

and have to solve 2N coupled ordinary differential equations.

V(x)

y2 x1

y3 x2

D x3

D xN

x

Fig. 2.1: Sketch of a one-dimensional chain of springs in an external periodic potential. https://doi.org/10.1515/9783110515145-002

2.1 Frenkel–Kotorova model | 13

2.1.2 Equilibrium solutions We shall come back in detail to problems like (2.3) in the next chapters. Here we concentrate only on steady configurations of the chain where ṗ n = 0, ẋ n = 0, p n = 0. From (2.3) follows the equilibrium condition ∂H =0 ∂x n

(2.4)

V 󸀠 (x n ) + (x n − x n−1 ) − (x n+1 − x n ) = 0 ,

(2.5)

or where the prime denotes the derivative. We define the reciprocal density at location x n y n ≡ x n − x n−1

(2.6)

and obtain from (2.5) and (2.6) the two-dimensional map y n+1 = y n + V 󸀠 (x n ) x n+1 = x n + y n+1 = x n + y n + V 󸀠 (x n ) .

(2.7)

From a certain initial pair (x0 , y0 ) the whole series (x n , y n ) is uniquely determined. Of course nothing is said about the stability of these solutions. At this stage we only know that they are at equilibrium where the net force on each mass point vanishes. Due to (2.4), the potential energy has an extremum, but this might be a maximum as well.

2.1.3 The standard map To go on we must specify V(x). With (2.1) we assume V(x) =

K cos(2πx) (2π)2

(2.8)

with the control parameter K. Then (2.7) takes the form x n+1 = x n + y n − y n+1 = y n −

K sin(2πx n ) 2π

K sin(2πx n ) . 2π

(2.9)

The recursion relation (2.9) is called Standard map, Chirikov map or Circle map and is discussed in detail in the literature (e.g., [1]).

14 | 2 Nonlinear maps

The two-dimensional map (2.9) is easy to code: PROGRAM circle_map REAL*8 :: x,y,pi2,vor CHARACTER*1 :: c INTEGER :: pgcurs pi2=4.*ASIN(1.D0) ! 2*Pi amp=1. ! value of K imax=10000 ! number of iterations for each initial value CALL INIT(0.,1.,0.,0.5,10.,0.5,1) ncol=15 ! use 15 colors DO k=pgcurs(x0,y0,c) ! mouse query x=x0; y=y0 ! initial values ic=mod(ic,ncol)+1; CALL PGSCI(ic) DO i=1,imax y=y-amp/pi2*SIN(pi2*x); x=x+y ! circle map xp=MOD(x,1.); yp=MOD(y,1.) ! plot modulo 1 CALL PGPNTS(1,xp,yp,-1,1) ENDDO if(ICHAR(c).eq.32) EXIT ENDDO CALL PGEND END For each initial value, 10 000 points are plotted. The initial pair is determined using the mouse. The map is periodic in y → y ± m, x → x ± m󸀠 with integers m, m󸀠 and is symmetric with respect to the origin. Therefore it is sufficient to take the initial values from the region 0 ≤ x ≤ 1 , 0 ≤ y ≤ 1/2 . Two fixed points x n+1 = x n , y n+1 = y n exist. At x, y = 0 we have a center, and x = 1/2, y = 0 is called saddle point (see Chapter 3). Figure 2.2 shows several series for K = 1.

2.1.4 Problems 1. 2.

Examine (2.9) for different K. For K = 0 you will not need your computer. Plot the chain length for fixed x0 as a function of y0 . Try different values for x0 .

2.2 Chaos and Lyapunov exponents | 15

Fig. 2.2: Standard map for K = 1.

2.2 Chaos and Lyapunov exponents As already known from the discussion of the logistic map in Section 1.3, there exist regular (periodic or quasiperiodic) and chaotic regimes if the control parameters or, at least for the Standard map, the initial conditions are changed. Instead of only looking qualitatively at the patterns of the iteration maps, we shall now derive a more quantitative measure for chaos.

2.2.1 Stability, butterfly effect, and chaos Let us turn again to the logistic map (1.1). Prevenient to the chaotic regime exists a cascade of period doublings where the relative distance of doubling or bifurcation points decreases with increasing a. The first few bifurcation points can be computed analytically, but the effort required increases quickly. Therefore it is desirable to find a quantity that reflects the stability of a series for any arbitrary a. Butterfly effect: What do we mean by stability? A descriptive definition of chaos is given by the development of some tiny disturbances of variables and/or parameters: if an (infinitesimally) small cause, namely a perturbation of i.e., the initial value y0 = x0 + ϵ after a certain time (here: number of iterations) is amplified to a finite and measurable outcome, a completely different series y n may result. Then the series starting

16 | 2 Nonlinear maps

at x0 is called unstable. A butterfly flaps its wings and perturbs the atmosphere very slightly. Provided sufficiently unstable weather conditions, this ‘butterfly effect’ may completely change the further development of the weather after a certain amount of time.

2.2.2 Lyapunov exponent of the logistic map Thus we are looking for a criterion that describes the divergence of two initially infinitesimally close series x n and y n if n ≫ 1. Let us start the first iteration with x0 . Then we have for x n x n = f(f(f(. . .f(x0 ) . . . ))) ≡ f (n) (x0 ) , (2.10) where f (n) denotes the nth iterate of f . In parallel, a second series y1 , y2 . . .y n is computed for a closely nearby initial value x0 + ϵ, |ϵ| ≪ 1 and reads y n = f (n) (x0 + ϵ) .

(2.11)

To see if the two series are receding from each other, we compute their distance at each n and find with ϵ → 0 󵄨 󵄨 󵄨󵄨󵄨 f (n) (x0 + ϵ) − f (n) (x0 ) 󵄨󵄨󵄨 󵄨󵄨 = lim 󵄨󵄨󵄨󵄨ϵ d f (n) (x)󵄨󵄨󵄨󵄨 . d n = |x n − y n | = 󵄨󵄨󵄨󵄨ϵ (2.12) 󵄨 ϵ→0 󵄨󵄨 󵄨󵄨 󵄨 ϵ 󵄨󵄨 󵄨󵄨 󵄨 x0 󵄨 dx If we assume that this distance increases (unstable) or decreases (stable) exponentially with n, d n = d0 eλn = |ϵ| eλn , (2.13) the series is unstable if λ > 0, stable if λ < 0. With (2.12) we obtain from (2.13) 1 󵄨󵄨󵄨󵄨 d (n) 󵄨󵄨󵄨󵄨 ln 󵄨󵄨 f (x)󵄨󵄨 , n→∞ n 󵄨󵄨 x0 󵄨󵄨 dx

λ = lim

(2.14)

where λ is called the Lyapunov exponent¹. For the evaluation of (2.14) the chain rule must be applied d (n) 󵄨󵄨󵄨󵄨 f (x)󵄨󵄨 = d x f(x0 )d x f(x1 ). . .d x f(x n ) dx 󵄨󵄨 x0 leading finally to

󵄨󵄨 d 󵄨󵄨 1 n 󵄨 󵄨 ∑ ln 󵄨󵄨󵄨 f(x)󵄨󵄨󵄨 . n→∞ n 󵄨󵄨 dx 󵄨󵄨 x=x k k=0

λ = lim

(2.15)

This relation is easy to code. Of course it is not possible to take infinite n, one should choose a sufficiently large n, say 1000, to find a convergent result. Based on the code logist from Chapter 1, the program could look like this:

1 Alexander M. Lyapunov, Russian mathematician and physicist, 1857–1918.

2.2 Chaos and Lyapunov exponents | 17

chaos

periodic windows

Fig. 2.3: Lyapunov exponent of the logistic map. The chaotic regime commences at about a ≈ 3.55 and λ > 0. Inside the chaotic regime, periodic windows of different size persist.

PROGRAM lyapunov REAL*8 :: a,x,flyap .... DO it=1,itmax+ivor ! iteration loop x=a*x*(1.-x) ! Logistic map IF(it.GT.ivor.AND.ABS(x-.5).GT.1.E-30) * fly=fly+LOG(ABS(a*(1.-2.*x))) ! sum for Lyap. exp. ENDDO flyap=flyap/FLOAT(itmax) ! Lyapunov exponent .... plot flyap as function of a ... Figure 2.3 shows the Lyapunov exponent over the control parameter a. One finds negative λ for the fixed points and for the periodic series, they are stable. But in the chaotic regions, the Lyapunov exponent turns out to be positive. At the bifurcation points a = a k , λ vanishes, since the 2k−1 -periodic cycle gets unstable at a k and gives way to the 2k -periodic solution, which then becomes stable from a > a k through a = a k+1 .

2.2.3 Lyapunov exponents for multidimensional maps The method can be generalized to N-dimensional maps. Let x⃗ n+1 = f ⃗(x⃗ n ) with x,⃗ f ⃗ in R N . The distance d n now turns into the length of a vector d⃗ n and instead of (2.12) one has d⃗ n = L(n) (x⃗ 0 ) d⃗ 0 , |d⃗ 0 | = ϵ ≪ 0

18 | 2 Nonlinear maps

with the Jacobi matrix (n)

L ij (x⃗ 0 ) =

(n) 󵄨󵄨󵄨

∂f i ∂x j

󵄨󵄨 󵄨󵄨 󵄨󵄨 󵄨󵄨x⃗ 0

of the nth iterate. With d⃗ n = d⃗ 0 eλn ≡ d⃗ 0 σ the eigenvalue problem (L(n) − σ i ) d⃗ 0 = 0 results. Knowing the N eigenvalues σ i we may finally compute the N Lyapunov exponents 1 (2.16) λ i = lim ln σ i . n→∞ n The Jacobi matrix of the nth iterate can be written as a product of the simple Jacobi matrices: L (n) (x⃗ 0 ) = L(x⃗ n ) L(x⃗ n−1 ) . . . L(x⃗ 0 ) .

(2.17)

As an example we take the Standard map (2.9). Because of 1 − K cos(2πx) L(x)⃗ = ( −K cos(2πx)

1 ) , 1

x⃗ = (x, y)

we have here the special situation det L = 1 independent on x.⃗ From (2.17) it follows det L(n) = 1 and as a consequence σ 1 σ 2 = 1. Since L(n) is a real valued matrix, we may distinguish between the two cases (i) σ i real valued, one σ k ≥ 1 and λ k ≥ 0, (ii) σ 1 = σ ∗2 complex valued, |σ i | = 1, and Re λ i = 0, Case (i) belongs to an unstable series (saddle point, see Chapter 3), case (ii) denotes a stable one (center). How can we compute the Lyapunov exponents in practice? To achieve a convergent outcome, one should do at least a few thousand iterations and the product (2.17) will accordingly include as many matrix multiplications. In the chaotic regime where the series diverge, an overflow error is unavoidable due to the exponential growth with n. On the other hand, the relation (2.17) is linear in x⃗ 0 and L can be normalized after each iteration. If we choose the maximum norm 1 L(̄ x⃗ i ) = L(x⃗ i ) si

with

s i = max(|L jk (x⃗ i )|) , jk

2.2 Chaos and Lyapunov exponents | 19

the product (n) L̄ (x⃗ 0 ) = L(̄ x⃗ n ) L(̄ x⃗ n−1 ) . . . L(̄ x⃗ 0 )

(2.18)

will not diverge anymore but differs from (2.17) in a possibly rather large factor n

s = ∏ si . i=0

The eigenvalues will then differ in the same factor and instead of (2.16) we obtain n 1 1 (ln σ̄ i + ln s) = lim (ln σ̄ i + ∑ ln s k ) . n→∞ n n→∞ n k

λ i = lim

(2.19)

All we have to do is to sum up ln s k during the iteration which will not lead to an overflow due to the logarithm. The code section computing λ in connection with the program circle_map could look like this: REAL*8 :: x,y,bs,s,sp,det,amp,fm1(2,2) complex xl1,xl2 .... x=x0; y=y0 ! init. values for iteration fm1=fm(pi2*x,amp) ! Jacobi matrix, defined as function below bs=MAXVAL(ABS(fm1)) ! maximum norm fm1=fm1/bs ! normalization s=LOG(bs) ! norm to s DO i=1,imax ! iteration loop y=y-amp/pi2*SIN(pi2*x) x=x+y c recursive matrix multiplication fm1=MATMUL(fm(pi2*x,amp),fm1) bs=MAXVAL(ABS(fm1)) fm1=fm1/bs s=s+LOG(bs) ENDDO c compute eigenvalues of fm1 det=(fm1(1,1)*fm1(2,2)-fm1(1,2)*fm1(2,1)) sp=fm1(1,1)+fm1(2,2) xl1=(log(.5*(sp+csqrt(cmplx(sp**2-4.*det,0.))))+s) xl2=(log(.5*(sp-csqrt(cmplx(sp**2-4.*det,0.))))+s) .. plot result

20 | 2 Nonlinear maps

c Jacobi matrix CONTAINS ! makes function fm visible in MAIN FUNCTION fm(x,amp) REAL*8 :: fm(2,2),x,amp fm(1,1)=1.-amp*COS(x) fm(1,2)=1. fm(2,1)=-amp*COS(x) fm(2,2)=1. END FUNCTION fm END If this section is integrated into a double loop over the initial values x0 , y0 , we can scan the whole plane with a certain resolution. The largest Lyapunov exponent, i.e., the maximum of the real parts of xl1, xl2 are then stored in a two-dimensional array and finally contour lines or a colored bitmap can be plotted. This can be done using the routine image, which is based on the PGPLOT routine PGPIXL and explained in detail in Appendix B. The colors can be initialized using the routine ccircl (Appendix B): CALL CCIRCL(2,100) ! initializes the color circle for the ! color indices 2..100 Figure 2.4 shows the color-coded largest Lyapunov exponent for three different values of K with a resolution of 500 × 500. Blue regions correspond to a regular, nonchaotic series, and all other colors reveal a positive Lyapunov exponent and denote chaotic behavior. The complete code can be found on the book’s website [2].

2.3 Affine maps and fractals Next we turn to linear, bijective maps composed of the three operations translation: rotation: scaling and shear:

q⃗ 󸀠 = q⃗ + a⃗ q⃗ 󸀠 = L R q⃗ q⃗ 󸀠 = L S q⃗ .

We restrict ourselves to two dimensions, q⃗ = (x, y) , where L R is the rotation matrix LR = (

cos θ sin θ

− sin θ ) cos θ

2.3 Affine maps and fractals

| 21

0.5 0.4 0.3 0.2 0.1 0 0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0.5 0.4 0.3 0.2 0.1 0 0.5 0.4 0.3 0.2 0.1 0

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Fig. 2.4: Largest Lyapunov exponent of the Standard map for K = 0.5 (top), K = 1 (middle), K = 1.5 (bottom). In the blue regions, the series are not chaotic.

and L s is the scaling-shear matrix LS = (

sx 0

b ) . sy

A compound transformation may read q⃗ 󸀠 = L R L S q⃗ + a⃗ ,

(2.20)

in which the different mappings do not commute, i.e., the sequence is important (in fact rotation and scaling commute if s x = s y , b = 0). If a triangle with area A is

22 | 2 Nonlinear maps transformed, one finds with Det(L R ) = 1 A󸀠 = Det(L S ) A = s x s y A . If the mapping (2.20) is performed recursively, q⃗ n+1 = L R L S q⃗ n + a⃗ ,

(2.21)

and s x s y < 1, a self-similar structure is obtained.

2.3.1 Sierpinski triangle As an application we construct the Sierpinski triangle. With the FORTRAN call CALL RANDOM_NUMBER(x) we activate the random number generator. If x is a scalar it contains after the call an equally distributed random number in the interval 0≤xplot ‘fd.dat’ with lines Then the slope can be determined at a convenient position where the curve is more or less straight. If we take, e.g., points 3 and 8 to construct the slope triangle, we find d f ≈ 1.585. The value of the fractal dimension can also be computed analytically using another construction for the Sierpinski gasket, see [3]. One then obtains d f = log(3)/ log(2) = 1.5849625. . .

2.4.3 Problem Determine the fractal dimension of the fern and the tree of Figure 2.6 numerically.

2.5 Neural networks In about 1990 the discipline neuro-informatics emerged. It comprises certain subjects from physics, mathematics, chemistry, and medicine, and aims to study and to understand the way the (human) brain works. One approach is the modeling of biological intelligence such as memory, learning, and logical relations using neural networks. Here we can only take a short look at this huge and still developing area. We have to concentrate on two examples, the perceptron and the so-called Kohonen maps. For further details we refer to the literature [6].

28 | 2 Nonlinear maps

Fig. 2.9: Perceptron with N = 5 input neurons. Since there is only one processing layer, it is also called single-layer perceptron (SLP).

2.5.1 Perceptron The human brain is built of about 80 billion neurons that are linked among each other with more than 1014 contacts, or synapses. These connections are dynamic and can be intensified or attenuated, depending on their usage. This time dependent adaption process is considered the key to learning. The perceptron is a very simple system realizing this concept. It possesses an input layer of N neurons S1 . . . S N which are not connected to each other. One processing layer, namely the adjustable weighted connections (synapses) w j , links the input neurons to a single output neuron (Figure 2.9). Each neuron has two states, active (S j = 1) or passive (S j = −1). The coupling of the input layer to the output neuron is given by the simple relation N

S o = sign (∑ w j S j )

(2.31)

j

where {1 for x ≥ 0 sign(x) = { −1 for x < 0 { is the slightly modified sign function. The strength of the connections is described by the synaptic weights w j . If w j < 0, the connection is inhibitive; otherwise it is activating. During the learning process, the values of the w j are changed accordingly. 2.5.1.1 Learning rule What do we mean by learning process? A neural network is not coded and does not contain any inherent rules or connections. One offers input/output pairs that are assigned to each other (learned). Assume we have M such pairs denoted by (n)

(S j = x j , S o = y(n) ) ,

n = 1...M .

(2.32)

2.5 Neural networks | 29

(n)

The network learns the assignment x j → y(n) by dynamically changing the synaptic weights w i . For the optimal case, as many pairs as possible out of (2.32) should fulfill the relation (2.31): N

!

(n)

y(n) = sign (∑ w j x j ) .

(2.33)

j

In 1949, D. Hebb² formulated what is nowadays called Hebb’s learning rule or Hebbian learning, which says that the synaptic weights are adjusted to the activity of the input and output neurons they do connect. Let ∆w i denote the modification of w i at each learning step, then Hebb’s rule can be written up as ∆w j =

1 (n) (n) y xj . N

(2.34)

If there was only one pair (M = 1) to learn and initially all w j = 0, we find after one step w j = N1 y x j . Inserting this into (2.33) yields y = sign (y

1 N 2 ∑ x ) = sign(y) , N j j

which is fulfilled for all y = ±1. In practice, the network should learn of course more than just one pair. Then one has to modify the rule (2.34): each pair changes the weights only if it is not yet learned correctly (Rosenblatt rule³): (n)

{ 1 y(n) x j ∆w j = { N 0 {

if

(n)

y(n) (∑Nj w j x j ) ≤ 0

(2.35)

otherwise .

In the ideal case when all pairs have been learned correctly, the algorithm stops by itself. However, as we shall see below, the capability to learn depends on the input/output pairs (2.32), and the complete learning of all pairs is rather an exception. But let us first continue with an example. 2.5.1.2 Prime numbers We wish to teach our perceptron to distinguish between prime and not prime numbers. The input of an integer K > 0 is realized by activating input neuron no. K to +1. All other neurons are in the rest state −1. Obviously, the range of input integers is restricted to K ≤ N. The output neuron should be active (+1) if K is prime.

2 Donald Hebb, Canadian psychologist, 1904–1985. 3 Frank Rosenblatt, American psychologist, 1928–1971.

30 | 2 Nonlinear maps

1. 2. 3.

To gain a clear and flexible code we adjust the three subtasks Generation of the input/output pairs Evaluation Learning step, adjustment of the synaptic weights

to the three subroutines 1. SUBROUTINE inout 2. SUBROUTINE eval 3. SUBROUTINE learn. The main program is then a simple loop over the repeated learning of integers as long as a stop-bit (here istop) is set: PARAMETER (n=100, m=1)

! number of neurons, input(n) and ! output(m) layer

C INTEGER :: ie(n),ia(m),ial(m) REAL :: w(n,m) w=-1. ! initiate all weights with -1 imax=n ! max. number of different input/output pairs k=1; istop=0 DO WHILE(istop.EQ.0) CALL inout(ie,ial,n,m,k) CALL eval(ie,ia,w,n,m) CALL learn(ie,ia,ial,w,n,m,istop,imax) wskal=MAXVAL(ABS(w)) w=w/wskal ! scaling of weights k=MOD(k,imax+1)+1 k1=k1+1 CALL synplo(w,n,m,k1) ! plot of the synaptic weights ENDDO write(6,*)'learning steps needed:',k1 END The array ie contains the value of the input neurons, ia those of the outputs (here only one). For K, the natural numbers 1, 2, 3, . . . , N are successively offered. The routine learn sets the stop-bit if imax numbers are recognized correctly one after the other without changing w. The subroutines read in detail:

2.5 Neural networks | 31

SUBROUTINE inout(ie,ia,n,m,k) INTEGER :: ie(n),ia(m) c provides input/output pairs ie=-1 ie(k)=1 ! input layer ia(1)=iprim(k) ! output neuron END The integer-function iprim(k) returns +1 if k is prime, otherwise −1. We leave the coding of iprim to the reader. SUBROUTINE eval(ie,ia,w,n,m) INTEGER :: ie(n),ia(m) REAL :: w(n,m) DO j=1,m s=SUM(w(1:n,j)*FLOAT(ie)) ia(j)=SIGN(1,FLOOR(s)) ENDDO END This is the coding of formula (2.31). The subroutine learn finally realizes Rosenblatt’s rule (2.35):

c

SUBROUTINE learn(ie,ia,ial,w,n,m,istop,imax) INTEGER :: ie(n),ia(m),ial(m) REAL :: w(n,m) modify weights according to Rosenblatt istop=0 ll=ll+1 DO j=1,m IF(ia(j).NE.ial(j)) THEN ll=0 w(1:n,j)=w(1:n,j)+float(ial(j)*ie)/FLOAT(n) ENDIF ENDDO IF(ll.EQ.imax) istop=1 ! set stop-bit END

If the computed result ia differs from the one that should be learned, ial, the weights are adjusted accordingly. The stop-bit istop is set if imax numbers are learned correctly in a row. In Figure 2.10 we show the learning steps needed as a function of N for different initializations of the weights w i .

32 | 2 Nonlinear maps 1e+07 8e+06

w = random w=1

K1

6e+06 4e+06 2e+06

Fig. 2.10: The number K1 of the learning steps needed depends strongly on the number of neurons N of the input layer, but also on the initialization of the synaptic weights w i .

w = −1 0

0

200

400

600

800 1000

N

2.5.1.3 Logic gates and linear separability We extend the switching condition (2.31) according to N

S o = sign (∑ w j S j − ϑ) .

(2.36)

j

Now the output neuron becomes active if the sum of its inputs exceeds the threshold value ϑ, which can be different for each neuron. To indicate the threshold, we write its value in the respective output neuron. To train the perceptron as a ‘normal’ binary computer it should at least be able to learn the basic logic functions NOT, AND, OR, and XOR. Taking an appropriate value for ϑ, this is possible for the first three functions, but not for XOR (Figure 2.11), as already proven by M. Minsky⁴ in the 60s. The truth table of an XOR gate looks like S1

S2

So

−1 1 −1 1

−1 1 1 −1

−1 −1 1 1

−1

1/2

1/2

1/2

1/2

0

1/2

−1/2

NOT

AND

OR

Fig. 2.11: The logic functions NOT, AND, and OR can be realized by an SLP.

4 Marvin Minsky, American artificial intelligence researcher, 1927–2016.

2.5 Neural networks

| 33

Let a1 = −1 (passive) and a2 = 1 (active). Then we must have w1 a1 + w2 a1 < ϑ w1 a2 + w2 a2 < ϑ

(2.37)

w1 a1 + w2 a2 > ϑ w1 a2 + w2 a1 > ϑ . Subtracting the third from the first inequality yields w2 (a1 − a2 ) < 0 , subtracting the fourth from the second leads to w2 (a2 − a1 ) < 0 ,

which is obviously a contradiction. In N-dimensional vector space, all possible input patterns of N neurons lay on the corners of an N-dimensional hypercube. Only these patterns that can be separated by an N − 1-dimensional hyperplane can be distinguished by the (single-layer) perceptron, Figure 2.12. If we try to teach our perceptron the XOR function, the learning process would never come to an end. Problems of this kind are called not linearly separable.

S2

S2 S1

AND

S2 S1

OR

S1

XOR

Fig. 2.12: For AND and OR the input states assigned to the same output state can be separated by a straight line; for XOR this is impossible.

Unfortunately most of the problems are not linearly separable, at least for larger N. A solution can be the extension to more than one processing layer (multilayer perceptron), for which we refer to the literature. 2.5.1.4 Decimal to binary converter Taking M perceptrons having one output neuron each in parallel, one arrives at a perceptron with M neurons in the output layer, Figure 2.13. The synaptic weights can now be cast in a rectangular matrix w ij ,

i = 1...N ,

j = 1...M

34 | 2 Nonlinear maps

Fig. 2.13: An SLP having more than one output neuron. Here, N = 5 input neurons are connected with M = 3 output neurons via the rectangular weight matrix w ij .

and Hebb’s rule (2.34) turns into ∆w ij =

1 (n) (n) y xi . N j

(2.38)

Such an extended perceptron can not only distinguish between two properties like prime or not prime, but between 2M states. As an example, we wish to teach the perceptron how to convert decimals into binary numbers. During the learning process we offer again integer values K running from 0 to N. The routine inout creates the input ie according to ie=-1 IF(k.ne.0) ie(k)=1 The related output is the binary image of K, which can be obtained in FORTRAN using the BTEST statement: ie=-1 DO j=0,m-1 IF(BTEST(k,j)) ie(m-j)=1 ENDDO (the most significant bit is ie(1)). The complete listing is provided in [2]. Figure 2.14 shows the synaptic connections for N = 15, M = 4 when finally all numbers 1. . . 15 have been learned after 80 learning cycles (w ij = 1 initialized). Negative (inhibitive) connections are drawn dashed.

2.5.2 Self-organized maps: Kohonen’s model Since there exist no horizontal connections within each layer of the perceptron, the spatial arrangement of the neurons plays no role. But the brain works differently: sim-

2.5 Neural networks

1

2

3

4

5

7

6

8

9

10

11

12

13

| 35

14 15

Fig. 2.14: The SLP as a decimal to binary converter. Inhibiting connections w < 0 are shown dashed. In the output layer the most significant bit is on the left-hand side.

ilar stimuli are processed in nearby regions of the cortex. This constitutes the basic idea of the self-organized maps developed by the Finnish engineer Teuvo Kohonen in 1982. 2.5.2.1 The model We resort to the case where the neurons are located in a plane layer representing a two-dimensional network. To each neuron we may assign a vector I r⃗ = ( ) J with (I, J) = (0, 0). . .(M − 1, N − 1) (Figure 2.15). Each neuron is connected with an input layer consisting of L neurons by synapses with the weights wℓr ⃗ ,

ℓ = 1...L

(the upper index ℓ denotes the neurons of the input layer). The Kohonen algorithm modifies Hebb’s learning rule in the following way: 1. Initializing. Take an appropriate initial value for the weights wℓr ⃗ , e.g., randomly distributed or all zero or ±1. 2. Input signals. The signals or patterns to learn are offered as vℓ , either one by one or in a random sequence. 3. BMU. Identify the Best Matching Unit (BMU). The BMU is the neuron that has the smallest (euclidean) distance to the learning signal. If L

L

ℓ

ℓ

∑(vℓ − wℓr ⃗ 󸀠 )2 ≤ ∑(vℓ − wℓr ⃗ )2

for all

r⃗ ,

the BMU is located at r ⃗ 󸀠 . Then r ⃗ 󸀠 is called the center of excitation.

(2.39)

36 | 2 Nonlinear maps

J Fig. 2.15: 25 neurons arranged in a quadratic processing layer.

I

4. Dynamics. During the adjustment step all synaptic weights are adapted according to (2.40) ∆wℓr ⃗ = ϵ h(d) (vℓ − wℓr ⃗ ) , with d ≡ |r ⃗ − r ⃗ 󸀠 | for all r.⃗ Except for the new function h(d) this is exactly Hebb’s rule (2.34). The function h(d) is unimodal and has its maximum at d = 0. It could be for instance a Gaussian of width σ h(d) = exp(−d2 /2σ 2 ) . (2.41)

5. 6. 7.

Note that in this step not only the BMU is changed but also the neurons lying in its neighborhood. Refinement. To refine the spatial structure, σ is decreased after each learning step. If σ ≥ σ min go to 2. The algorithm stops when only the BMU is changed.

2.5.2.2 Color maps To be more clear we wish to present an application: a Kohonen map sorting colors. Colors can be classified by a three-component vector measuring their red, green, and blue composition, called the RGB value. An RGB value of (1/0/0) denotes red, (0/1/0) green, (1/1/0) yellow, (0.5,0.5,0.5) gray and (1/1/1) white. Each neuron bears three synaptic values, namely (1)

wr⃗

for red ,

(2)

wr⃗

for green ,

(3)

wr⃗

for blue ,

where (ℓ)

0 ≤ wr⃗ ≤ 1 . The code that implements the Kohonen algorithm can be found in [2] (the array R is assigned to w(1) , G to w(2) and B to w(3) using an EQUIVALENCE statement). Figure 2.16 shows color maps formed by a layer of 15 × 15 neurons after learning 100, 500, and 3000 randomly generated RGB patterns. For a monotonically decreasing

2.5 Neural networks | 37

Fig. 2.16: Evolution of a color map consisting of 15 × 15 neurons applying Kohonen’s algorithm. Upper left: initial state; upper right: after t = 100 learned random patterns; bottom left: after t = 500; bottom right: after t = t max = 3000.

σ we use the relation σ = σ 0 ⋅ (0.05)t/t max ,

σ0 = 5 ,

tmax = 3000 ,

in addition we put ϵ = 0.03. It turns out that the completely random color pattern from the beginning is eventually organized in such a way that similar colors are assigned to neurons located close to each other. 2.5.2.3 Traveling Salesman Problem Next we wish to study a one-dimensional configuration of N neurons in a row. Each neuron in the chain bears a two-dimensional vector xi w⃗ i = ( ) yi pointing at a certain location in 2D real space, e.g., of a road map. If one passes the neurons one after the other, the adjoint locations (x i , y i ) constitute a certain path on the road map. If we offer certain given points (places) to the Kohonen algorithm as learning inputs, the path will be matched closer and closer to these points. This leads us to the famous Traveling Salesman Problem (TSP). The task of the TSP is to find the shortest connection between K given places on a road map with the constraint that each place is only touched once. It turns out that there exist 12 (K − 1)! different closed paths to fulfill this condition. But which is the shortest? Of course one could try all paths, but the computational effort increases exponentially with K. Even for a rather small number of places the largest computers would be overcharged. To apply Kohonen’s method for an approximate solution, we take 30 prescribed locations, for instance, randomly chosen positions on a map; see the red points in Figure 2.17. In each learning step we select randomly one of these locations as the input

38 | 2 Nonlinear maps

Fig. 2.17: Two approximate solutions of the TSP computed with Kohonen’s algorithm based on a chain of 40 neurons (black dots). A certain number of places, here 30 (red dots), should be connected by the shortest path. Shown is the situation after 20 (top right), 80 (bottom left), and 2000 (bottom right) learning steps. In the beginning, the chain is initialized on a circle with radius R = 0.15 (solid lines) and R = 0.5 (dashed). The path length after 2000 steps is 4.176 (solid) and 4.304 (dashed).

and adjust the synaptic weights according to (2.40). Also in Figure 2.17 we show the states of the network after 20, 80, and 2000 learning steps for two different initial values of the weight vectors w⃗ i . The solid lines correspond to initial values on a circle with radius R = 0.15, the dashed ones with R = 0.5. After 2000 steps all places are passed by the paths. For the parameters we use ϵ = 0.8 and σ = σ 0 ⋅ (0.02)t/t max ,

σ0 = 2 ,

tmax = 4000 .

Bibliography |

39

Considering the figure, it is evident that different initial configurations lead to at least slightly different solutions. But only one path can be the shortest and Kohonen’s algorithm can solve the TSP only approximately.

2.5.3 Problems 1. 2.

Code the routines for the perceptron as a decimal to binary converter. Solve the TSP for K locations and N neurons applying the Kohonen algorithm.

Bibliography [1] J. Argyris, G. Faust, M. Haase and R. Friedrich, An Exploration of Dynamical Systems and Chaos, Springer (2015). [2] FORTRAN codes at http://www.degruyter.com/books/978-3-11-051513-8 [3] W. Kinzel and G. Reents, Physics by Computer, Springer (1997). [4] B. Mandelbrot, The Fractal Geometry of Nature, Freeman and Co. New York (1983). [5] M. F. Barnsley, Fractals Everywhere, Dover Pub. (2012). [6] H. Ritter, T. Martinetz and K. Schulten, Neural Computation and Self-Organizing Maps: An Introduction, Addison–Wesley Pub. (1992).

3 Dynamical systems The foundation of physical theories is normally based on differential equations. In field theories like quantum mechanics, electrodynamics, or hydrodynamics, one is confronted with (a system of) partial differential equations F k [r,⃗ t, Ψ i (r,⃗ t), ∂ t Ψ i (r,⃗ t), ∂ tt Ψ i (r,⃗ t), . . . , ∂ x n Ψ i (r,⃗ t), ∂ x n x m Ψ i (r,⃗ t) . . . , ] = 0 , (3.1) while in Newton’s classical mechanics ordinary differential equations G k [x i (t), d t x i (t), d2tt x i (t), . . . ] = 0

(3.2)

are the focus of attention. Systems of the form (3.1) can be expanded into a certain basis for further numerical analysis, according to N

Ψ i (r,⃗ t) = ∑ a ik (t) ϕ ik (r)⃗

(3.3)

k

and are thereby converted to the form (3.2). We postpone the treatment of partial differential equations to Chapters 6 and 7 and commence here with problems of the kind of (3.2).

3.1 Quasilinear differential equations Let t be the independent variable, for instance, time. To begin with, we restrict the treatment to one dependent variable, say x and search for the function x(t). In classical mechanics this corresponds to a one-dimensional motion. Quasilinear differential equations are linear in the highest occurring derivative. Then, (3.2) can be written as x(N) = f (t, x, x(1) , x(2) , . . . .x(N−1) )

(3.4)

with

dk x . dt k Equation (3.4) is equivalent to a system of N first order ordinary differential equations (ODEs), obtained by the substitution x = x(t) ,

x k+1 = x(k) ,

x1 = x , as

x(k) ≡

k = 1...N −1

d t x1 = x2 d t x2 = x3 .. . d t x N−1 = x N d t x N = f(t, x1 , x2 , . . .x N ) ,

https://doi.org/10.1515/9783110515145-003

(3.5)

3.1 Quasilinear differential equations | 41

or, using vector notation

d x⃗ = f ⃗(t, x)⃗ dt

(3.6)

with f ⃗ = (x2 , x3 , . . .x N , f). It is sufficient to study first order systems as (3.6). Equations of this kind but for arbitrary f ⃗ constitute a dynamical system with N as the number of ⃗ lives in an N-dimensional state space (or phase degrees of freedom. The vector x(t) space). If we consider the definition of the derivative by the differential quotient ⃗ + ∆t) − x(t) ⃗ d x⃗ x(t = lim , dt ∆t→0 ∆t we can write (3.6) for small but finite ∆t approximately (Taylor expansion) in the form of an N-dimensional map ⃗ + ∆t) = x(t) ⃗ + f ⃗(t, x(t))∆t ⃗ x(t + O(∆t2 ) ,

(3.7)

⃗ we can compute x(t ⃗ + ∆t) and then where ∆t is denoted as step size. From each x(t) iteratively all the following values ⃗ + m∆t) x(t for m integer. It is clear that in addition to the function f ⃗ we need initial values ⃗ 0 ) = x⃗ 0 . x(t

(3.8)

⃗ 0 ) uniquely onto the vector Formally, the dynamical system (3.6) maps the vector x(t ⃗ 1 ). This can be also expressed by the help of the time-evolution operator U:⃗ x(t ⃗ 1 , t0 )[x(t ⃗ 0 )] . ⃗ 1 ) = U(t x(t

(3.9)

Equations (3.6,3.8) define an initial value problem, and (3.7) provides a first numerical approximation scheme for its solution, the so-called Euler-forward method or explicit Euler method.

3.1.1 Example: logistic map and logistic ODE In context with the logistic map from Chapter 1 we examine the first order ODE (logistic ODE) dx (3.10) = (a − 1)x − ax2 , dt defining the behavior of x(t) for any given x(0) = x0 (one-dimensional initial value problem). An exact solution can be found by the separation of variables x(t) =

a−1 , a + c e(1−a)t

(3.11)

42 | 3 Dynamical systems where the integration constant c = (a − ax0 − 1)/x0 is determined by the initial condition. As t → ∞ one obtains the asymptotic solutions {0 xs = { 1 − 1/a {

if

a1

(3.12)

which coincide with the fixed points of the logistic map. How is the ODE (3.10) linked to the logistic map (1.1)? Following (3.7) we express dx/dt by the differential quotient dx x(t + ∆t) − x(t) ≈ dt ∆t

(3.13)

which becomes exact for ∆t → 0. Inserting (3.13) on the left-hand side of (3.10) one finds after a short calculation x n+1 = ax n ∆t (1 −

1 1 (1 − ) − x n ) . a ∆t

(3.14)

Here, x n denotes the x values at the time n∆t, or x n = x(n∆t). If we put now ∆t = 1, Equation (3.14) becomes equivalent to (1.1). In other words, the discretized form of the ODE (3.10) corresponds mathematically to the logistic map. This explains the similar ‘temporal’ development up to a < 3, i.e., the asymptotic approach to xs and the correct asymptotic solutions. What cannot be explained are the further bifurcations of the solutions of the logistic map for a > 3 and their chaotic behavior for even larger a. All this is of course not included in the solution (3.11). How can we explain the much richer dynamic behavior of the discretized form (3.14)? To get an answer we have to look at the stability of (3.14). A stationary solution (fixed point) of (3.14) reads xs = 1 − 1/a. If we put x(t) = xs + u(t) and linearize with respect to u, we can show that for ∆t = 1 the fixed point xs becomes numerically unstable as soon as a > 3. Coming from the logistic ODE, all the bifurcations through the chaotic sequences originate from a too-large time step during discretization (see Problems) and vanish with a smaller ∆t. However, then (3.14) is no longer equivalent to the logistic map.

3.1.2 Problems With a pencil, show that (3.14) becomes numerically unstable if ∆t > 2/(a − 1). And to code: 1. Plot the function (3.11) using PGPLOT for different initial conditions and different values of a. 2. Examine the map (3.14) as a numerical solution of the ODE (3.10) for different time steps ∆t and different a.

3.2 Fixed points and instabilities | 43

3.2 Fixed points and instabilities We continue with the N-dimensional dynamical system (3.6) with a given initial condition. For the time being we concentrate on autonomous systems, i.e., the right-hand sides of (3.15) do not depend explicitly on t: dx i = f i (x1 . . .x N ) , dt

i = 1...N .

(3.15)

3.2.1 Fixed points For an autonomous system, fixed points as a stationary solution of (3.15) may exist. They can be found by solving the algebraic system (0)

(0)

0 = f i (x1 . . .x N ) ,

i = 1...N .

(3.16)

We note that for somewhat involved nonlinear f i , solutions of (3.16) are normally only accessible numerically, even for small N.

3.2.2 Stability (0)

If a fixed point x i is known, the behavior of the system in its neighborhood is of interest. How do small deviations evolve? Are they amplified or do they fade away in time? If the deviations stay small or decay, the fixed point is called stable; if they depart exponentially, we have an unstable fixed point. If N > 1, the disturbed trajectories may in addition oscillate around the fixed point. This is called oscillatory or Hopf bifurcation¹. If one restricts to infinitesimal deviations u i (t) from the fixed point (0)

x i (t) = x i

+ u i (t) ,

(3.17)

one may linearize (3.15) with respect to u. This is called linear stability analysis. Inserting (3.17) into (3.15) yields after linearization the system N du i = ∑ L ij u j dt j

with the Jacobi matrix L ij =

󵄨 ∂f i 󵄨󵄨󵄨 󵄨 . ∂x j 󵄨󵄨󵄨󵄨x=⃗ x⃗ (0)

(3.18)

(3.19)

For an autonomous system, the L ij are constant and u j = q j eλt

1 Eberhard Hopf, German-American mathematician, 1902–1983.

(3.20)

44 | 3 Dynamical systems

transforms (3.18) into a linear eigenvalue problem N

∑ [L ij − λ δ ij ] q j = 0 .

(3.21)

j

Due to (3.20), the real parts of the N eigenvalues λ k characterize the stability of the (0) fixed point x i . Let the eigenvalues be ordered according to Re(λ1 ) ≥ Re(λ2 ) ≥ . . . ≥ Re(λ N ). We distinguish between the four cases: (i) All λ k have a negative real part: the fixed point is stable. (ii) Re(λ1 ) > 0, and Im(λ1 ) = 0; the fixed point is monotonically unstable. (iii) Re(λ1 ) > 0, and Im(λ1 ) ≠ 0; the fixed point is oscillatory (Hopf) unstable. (iv) Re(λ1 ) = 0, critical point, center if Im(λ1 ) ≠ 0; the fixed point is marginally stable. (i) is also named node or stable focus if Im(λ1 ) ≠ 0, (ii) is called saddle point, and (iii) is an unstable focus.

3.2.3 Trajectories ⃗ 0 ) be the location of a particle in the N-dimensional state space at time t0 . The Let x(t ⃗ corresponds to a line (trajectory) and (3.15) creates a flow. Given parametrization x(t) f , the time-evolution is deterministic: how the particle evolves for t > t0 is determined ⃗ 0 ). But then it can be easily seen that crossings of trajectories are excluded. solely by x(t ⃗ 0 ) there would be two possible different paths for Taking a potential cross point as x(t t > t0 . But this would contradict the deterministic behavior described by the flow (3.15). This is formulated as the No-Intersection Theorem: Two distinct trajectories in N-dimensional state space cannot intersect, nor can a single trajectory cross itself. The theorem is valid for arbitrary N. If f ⃗ is taken as a particle density flow, the source strength N

Q(x)⃗ = ∇ ⋅ f ⃗ = ∑ i

∂f i ∂x i

(3.22)

can be defined. Knowing Q, we can compute the temporal change of a volume element ⃗ ∆V that swims in the flow at x(t): ⃗ ⃗ ⃗ d t ∆V(x(t)) = ∆V(x(t)) Q(x(t)) .

(3.23)

⃗ Normally Q depends on x⃗ and thus Q(x(t)) depends on time. For a dissipative system, one finds ⟨Q⟩ ≤ 0 with a convenient time average ⟨. . . ⟩. Then every volume element whose corners travel along trajectories shrinks in the course of time and vanishes at infinite times.

3.2 Fixed points and instabilities |

45

3.2.4 Gradient dynamics If the circulation (or curl) of f ⃗ vanishes everywhere, a potential function U exists with fi = −

∂U(x1 , . . .x N ) . ∂x i

Then one has Q(x)⃗ = − ∑ i

and L ij = −

∂2 U ∂x2i

∂2 U . ∂x j ∂x i

As a consequence, the Jacobi matrix is symmetric and has only real eigenvalues, excluding the existence of oscillatory instabilities. On the other hand one easily shows with N dU ∂U dx i =∑ = −|f ⃗|2 ≤ 0 dt ∂x dt i i that U decays with t monotonically until a fixed point is reached where f ⃗ = 0.

3.2.5 Special case N = 1 The most simple case is N = 1 and reads d t x = f(x) . If the solution is bounded, due to the No-Intersection Theorem, every temporal development must eventually end in a stable fixed point: lim x(t) = xs

t→∞

with

f(xs ) = 0 and

d x f|xs < 0 .

(3.24)

A potential of U(x) = − ∫ dx f(x) always exists, which must have at least one minimum at xs because of (3.24).

3.2.6 Special case N = 2 A direct consequence of the No-Intersection Theorem is the Poincaré–Bendixson Theorem²: 2 Henri Poincaré, French physicist and mathematician, 1854–1912; Otto Bendixson, Swedish mathematician, 1861–1935.

46 | 3 Dynamical systems

q2

q2

q2

q1

q1

λ2 < λ 1 < 0

λ1 > 0, λ2 < 0

λ 1 > λ2 > 0

q2

q2

q1

q2

q1

q1

Re λi < 0

Re λi = 0

q1

Re λi > 0

Fig. 3.1: Fixed points and their neighborhood in two-dimensional state space, from left to right, top row: λ i real valued, stable node, saddle point, unstable node. Bottom row: λ i complex valued, stable focus, center, unstable focus.

If in two-dimensional state space the trajectories are confined to a finite size region (i.e., do not go to infinity), they either end on a stable fixed point or approach a stable limit cycle as t → ∞. A limit cycle is a periodic solution in state space, that is, a closed trajectory x⃗ lc with x⃗ lc (t) = x⃗ lc (t + T) where T denotes the period. Near any fixed point, the local behavior of the trajectories is restricted in two dimensions to only a few qualitatively different possibilities, shown in Figure 3.1. 3.2.6.1 Example: Van der Pol Oscillator Van der Pol’s equation reads d2 x dx − μ(1 − x2 ) +x=0 dt dt2 and shows for μ > 0 self-excited nonlinear oscillations as the solution. We shall examine the system in the form (3.15): d t x1 = x2 d t x2 = −x1 + μ(1 − x21 ) x2 . (0)

(0)

(3.25)

There exists only one fixed point, x1 = x2 = 0. Its Jacobi matrix turns out to be L=(

0 −1

1 ) μ

3.2 Fixed points and instabilities | 47

and has the eigenvalues λ12 = μ/2 ± √μ 2 /4 − 1 . Depending on μ, the fixed point exhibits different characteristics: μ ≤ −2: −2 < μ < 0: μ = 0: 0 < μ < 2: 2 ≤ μ:

stable node stable focus center, crit. point unstable focus unstable node

λ i ϵ R, λ i ϵ C, λ i ϵ C, λ i ϵ C, λ i ϵ R.

For the source strength introduced in (3.22) one has Q(x i ) = μ(1 − x21 ) . Taking polar coordinates x1 = r cos φ, x2 = r sin φ one finds for r ≫ 1, μ > 0 d t r ≈ −μr3 cos2 φ sin2 φ ≤ 0 and the trajectories remain in a bounded region. For r < 1, it follows from d t r = μr(1 − r2 cos2 φ) sin2 φ , that the region is limited for μ < 0. Without solving the ODEs (3.25) explicitly we obtain the following scenario: If μ < 0, all trajectories starting somewhere inside the circle x21 (0) + x22 (0) < 1 end at the stable fixed point (focus or node) x1 = x2 = 0. If μ > 0, the fixed point becomes unstable. Since all trajectories remain in a bounded region in state phase and no other (stable) fixed point exists, the Poincaré– Bendixson Theorem applies: for t → ∞ all trajectories approach asymptotically a limit cycle circuiting the unstable fixed point.

3.2.7 Special case N = 3 In the preceding paragraph we saw that in the bounded two-dimensional state space only regular behavior is possible. All trajectories asymptotically either reach fixed points or limit cycles. This changes for N = 3. A new object comes into play, the socalled strange attractor. It can be thought as a certain trajectory having both attractive and repelling character. The first strange attractor was discovered by E. Lorenz³ as a numerical solution of a system of nonlinear ODEs [1].

3 Edward Lorenz, American mathematician and meteorologist, 1917–2008.

48 | 3 Dynamical systems

3.2.7.1 Example: the Lorenz equations In the 60s, Lorenz derived a model of three coupled nonlinear ODEs intended for weather forecasting: dx1 = −α (x1 − x2 ) dt dx2 (3.26) = (δ + 1) x1 − x2 − x1 x3 dt dx3 = −β x3 + x1 x2 . dt Thereby, α, β > 0 denote system parameters and δ is the control parameter or bifurcation parameter. The system (3.26) is nowadays called Lorenz equations. It possesses the three fixed points (i) x i = 0 ,

(ii) x1 = x2 = ±√βδ , x3 = δ ,

the latter two existing only if δ ≥ 0. We wish to examine their stability and begin with the fixed point (i). The Jacobi matrix reads −α L = (δ + 1 0

α −1 0

0 0) −β

(3.27)

and has the eigenvalues λ12 = −

1+α 1 ± √(1 + α)2 + 4αδ , 2 2

λ3 = −β .

For δ < 0 all λ have a negative real part and the fixed point is a stable node or a stable focus. At δ ≥ 0 the fixed point turns into a saddle node with one unstable and two stable directions (or manifolds, that is the eigenvectors of (3.27)). For (ii) (δ > 0), the Jacobi matrix reads −α α 0 L=( 1 (3.28) −1 ∓√βδ) ±√βδ ±√βδ −β with the eigenvalues as roots of the polynomial P(λ) = λ3 + λ2 (1 + β + α) + λβ(1 + δ + α) + 2βαδ .

(3.29)

Since all the coefficients of (3.29) are positive, real eigenvalues may only occur with λ < 0. Evaluation of (3.29) for certain fixed parameters (α = 10, β = 8/3) shows that for δ larger than ≈ 0.35 the spectrum has the form λ1 = λ∗2 ϵ C, λ3 < 0. The real parts

3.2 Fixed points and instabilities |

49

Fig. 3.2: The Lorenz attractor for α = 10, β = 8/3, δ = 24. Shown is the projection of a trajectory on the x1 -x2 -plane. The two fixed points (ii) are marked by asterisks.

of the conjugate complex pair become positive if δ > δc =

α(α + β + 3) 451 −1= ≈ 23.7 . α−β−1 19

Then the two stable focus nodes (ii) turn into unstable saddle foci having a twodimensional unstable and a one-dimensional stable manifold. The two fixed points become oscillatory unstable with the Hopf frequency (δ = δ c ) ω c = Im λ1 = √

2αβ(α + 1) ≈ 9.62 α−β−1

and finally the Lorenz attractor emerges, an almost two-dimensional object in state space; see Figure 3.2. A peculiarity of (3.26) is its constant source strength Q L = −α − 1 − β < 0 , the Lorenz equations are dissipative in the complete state space. From (3.23) it follows that the size of each arbitrary volume element ∆V0 exponentially decays in time ∆V(t) = ∆V0 eQ L t . Every volume element is eventually mapped onto the 2 + ϵ-dimensional attractor.

50 | 3 Dynamical systems

3.3 Hamiltonian systems We continue with autonomous systems without explicit time-dependence as (3.15).

3.3.1 Hamilton function and canonical equations In many problems formulated in classical mechanics the forces are derived from a potential V(q1 , . . .q N ) where q1 . . . .q N are the (generalized) positions. To achieve a complete description, the momenta p1 . . .p N need to be known. The Hamilton⁴ function, or for short, the Hamiltonian N p2 H(q i , p i ) = ∑ i + V(q i ) (3.30) 2m i coincides with the total mechanical energy of the system and is a constant of motion. The canonical equations are equivalent to Newton’s equations of motion and can be formulated as [2] ∂H dt qi = ∂p i (3.31) ∂H dt pi = − . i = 1...N . ∂q i Since we have two sets of variables, each of them with N variables, the motion described by (3.31) has 2N degrees of freedom. However, this is the mathematical definition of a degree of freedom. In mechanics, each pair of conjugated variables q i , p i corresponds to one degree of freedom, halving their total number. To exclude further confusion we adopt the notion mechanical degree of freedom for the number of pairs of variables (here N), degree of freedom for the number of first order ODEs in (3.31) (here 2N). The system (3.31) has the form of (3.15) but, due to the special form of f ⃗, N

Q = ∇ ⋅ f⃗ = ∑ [ i

∂2 H ∂2 H − ]=0 ∂q i ∂p i ∂p i ∂q i

applies. The size of volume elements is constant in time, and the dynamics in (3.31) is called conservative. If one wishes to know the time dependence of some variable A(q i , p i , t) one must use the chain rule according to N N ∂A dq i ∂A dp i ∂A ∂H ∂A ∂H dA ∂A − ) = {A, H} − = ∑( + ) = ∑( dt ∂t ∂q dt ∂p dt ∂q ∂p ∂p i i i i i ∂q i i i

4 William R. Hamilton, Irish mathematician and physicist, 1805–1865.

3.3 Hamiltonian systems |

51

where the abbreviation N

{A, B} ≡ ∑ ( i

∂A ∂B ∂A ∂B − ) ∂q i ∂p i ∂p i ∂q i

is called the Poisson bracket. Nonexplicitly time-dependent variables (∂A/∂t = 0) for which the Poisson bracket vanishes are constants of motion (dA/dt = 0). This of course is especially true for H.

3.3.2 Symplectic integrators The structure created by the flow (3.31) has symplectic geometry [2]. Normally this geometry is destroyed when (3.31) is integrated numerically. Then a nonvanishing source strength is generated and constants of motion, especially the total energy, are no longer conserved. So-called symplectic methods have been developed to avoid such problems. Here we can only give a short overview, see also [3]. For the sake of simplicity we restrict ourselves to a two-dimensional phase space corresponding to one mechanical degree of freedom, that is H = H(p, q) and dt q =

∂H , ∂p

dt p = −

∂H . ∂q

(3.32)

We abbreviate x⃗ = (q, p) and write (3.32) as ⃗ x]⃗ d t x⃗ = H[

(3.33)

with the (in general nonlinear) operator ⃗ x]⃗ = ( ∂H/∂p| x⃗ ) . H[ − ∂H/∂q| x⃗ ⃗ x]⃗ indicates that H⃗ acts on x,⃗ and the result of A remark on the notation: writing H[ the operation is a vector of the same dimension. Successive operations of H⃗ are then ⃗ H[. ⃗ . . x]⃗ . . . ]. It is convenient to omit the brackets and write H⃗ x⃗ instead denoted by H[ ⃗ x]. ⃗ In the same manner we write products of different operators, e.g., of H[ ⃗ . A⃗ B⃗ x⃗ ≡ A[⃗ B[⃗ x]] ⃗ As already done in (3.9) we may introduce the time-evolution operator U(t) ⃗ x(0) ⃗ = U(t) ⃗ x(t) with (an autonomous system assumed) ⃗ , ⃗ ⃗ = exp(t(T⃗ + V)) U(t) = exp(t H)

(3.34)

which can be seen by insertion into (3.33). In the last step, we have defined the operators 0 ∂T/∂p ) (3.35) ) , V⃗ = ( T⃗ = ( −∂V/∂q 0

52 | 3 Dynamical systems

with H(p, q) = T(p) + V(q) . Thus the Hamiltonian should be separable into a momentum-dependent kinetic part and into a space-dependent potential part. Each part operator U⃗ T (t) = exp(t T)⃗ ,

U⃗ V (t) = exp(t V)⃗

(3.36)

generates a time-evolution being trivial in a certain sense. The evolution U⃗ T is that of a free particle where p =const., q = (p/m)t, whereas U⃗ V describes the ‘motion’ of an infinitely heavy mass point, where q =const., p = −(dV/dq)t. The trick to constructing symplectic methods is the conversion of (3.34) into products of U⃗ T and U⃗ V . But since T⃗ and V⃗ do not commute, we cannot simply write U⃗ = U⃗ T U⃗ V . The transformation must be done applying the Campbell–BakerHausdorff relation exp(C) = exp(A) exp(B) 1 1 C = A + B + [A, B] + ([A, [A, B]] + [B, [B, A]]) + . . . 2 12

(3.37)

([A, B] = AB − BA denotes the commutator). If A, B is of O(t), then [A, B] = O(t2 ), [A, [A, B]] = O(t3 ) and so on. We divide t into many small intervals (time steps) of ⃗ ⃗ ∆t and instead of U(t), we look at U(∆t). The formula (3.37) can then be considered an expansion with respect to ∆t and can be truncated at a certain desired order in ∆t. After a longer manipulation it can be shown that ⃗ U(t) = U⃗ T (a n ∆t)U⃗ V (b n ∆t)U⃗ T (a n−1 ∆t). . . U⃗ T (a1 ∆t)U⃗ V (b 1 ∆t) + O(∆t n+1 )

(3.38)

where the coefficients a k , b k can be chosen arbitrarily but must fulfill the constraints n

∑ ak = 1 , k

n

∑ bk = 1 .

(3.39)

k

Since both operators (3.36) are symplectic, this holds for the product (3.38) as well. Taking certain a k , b k , we may then generate methods having a given order in the time step, where all of them conserve the symplectic geometry of the Hamiltonian flow. The relation (3.38) is strongly simplified if we take T⃗ T⃗ x⃗ = 0 ,

V⃗ V⃗ x⃗ = 0

into account. The Taylor expansion stops after the second term 1 U⃗ T = exp(∆t T)⃗ = 1 + ∆t T⃗ + ∆t2 T⃗ T⃗ + . . . = 1 + ∆t T⃗ , 2 which also holds for U⃗ V . Then we can substitute U⃗ T = 1 + ∆t T⃗ , in (3.38).

U⃗ V = 1 + ∆t V⃗

3.3 Hamiltonian systems

| 53

Let us state the scheme in the lowest order n = 1. Due to (3.39) a1 = b 1 = 1 and with (3.38) we obtain ⃗ ⃗ + ∆t) = U(∆t) ⃗ = (1 + ∆t T)⃗ (1 + ∆t V)⃗ x(t) ⃗ + O(∆t2 ) . x(t x(t) Taking (3.35) and ∂T(p)/∂p = p/m we finally have (

0 p/m qn q n+1 ) = (1 + ∆t ( )) [( ) + ∆t ( ∂V )] − ∂q (q n ) p n+1 pn 0 =(

0 (p n − ∆t ∂V qn ∂q (q n )) /m) ) + ∆t ( ∂V ) + ∆t ( − (q ) pn 0 n ∂q

=(

p n+1 /m qn ) + ∆t ( ∂V ) . − ∂q (q n ) pn

(3.40)

Thus, the iteration formula for one time step for p and q read ∂V (q n ) ∆t ∂q p n+1 = qn + ∆t . m

p n+1 = p n − q n+1

(3.41)

The one-step scheme (3.41) is analogous to the two-step scheme q n+1 = 2q n − q n−1 −

∆t2 ∂V (q n ) , m ∂q

(3.42)

which can easily be shown by eliminating p n . The scheme (3.42) is also known as the Verlet algorithm. It can be derived alternatively from Newton’s second law discretizing the second time derivative by d2 q dt2

→

q n+1 − 2q n + q n−1 . ∆t2

A particular scheme of the 3rd order is generated from (3.38), taking a1 = a2 = 1/2 , and

b1 = 1 ,

b2 = 0 ,

⃗ U(t) = U⃗ T (∆t/2) U⃗ V (∆t) U⃗ T (∆t/2) + O(∆t3 ) .

Without going into detail we quote the iteration rule ∂V (q n ) ∆t/2 ∂q p n+1/2 = qn + ∆t m ∂V = p n+1/2 − (q n+1 ) ∆t/2 ∂q

p n+1/2 = p n − q n+1 p n+1

(3.43)

which is similar to the ‘Leapfrog’ version of Verlet’s algorithm where space and momentum variables are evaluated at ∆t/2 differing times.

54 | 3 Dynamical systems

3.3.2.1 Example: Hénon–Heiles model To demonstrate the advantage of the symplectic method compared to a simple Euler method, we shall explore in some detail the Hénon–Heiles model [4]. The model describes the two-dimensional motion of a star near the center of a disk-shaped galaxy. Taking a suitable scaling (m = 1), the Hamiltonian reads H(p1 , p2 , q1 , q2 ) = E =

1 2 1 2 p + p + V(q1 , q2 ) 2 1 2 2

(3.44)

with

1 2 1 2 1 (3.45) q + q + q21 q2 − q32 . 2 1 2 2 3 A transformation to polar coordinates immediately shows the 3-fold rotational symmetry of V: 1 1 V(r, φ) = r2 ( + r sin 3φ) . 2 3 The motion of the star is bounded close to the origin as long as the total energy (3.44) V(q1 , q2 ) =

E < V0 =

1 . 6

For a larger E, the star may leave the triangle (see Figure 3.3), first in the directions of φ = 90°, 210° and 330°. 2

1

0

–1

–2 –2

–1

0

1

2

Fig. 3.3: Potential V(q 1 , q 2 ) of the Hénon–Heiles model. Inside the triangle around the origin the motion of the star is bounded if its total energy is less than 1/6. Solid lines: V = 1/6, 1/12, 1/24, 1/48, 1/96, dashed: V = 1/3, 2/3, 1.

3.3 Hamiltonian systems |

55

The canonical equations (3.31) read d t q1 = p1 d t q2 = p2

(3.46)

d t p1 = −q1 − 2q1 q2 d t p2 = −q2 − q21 + q22 . The iteration (3.41) turns into p1,n+1 = p1,n − (q1,n + 2q1,n q2,n ) ∆t p2,n+1 = p2,n − (q2,n + q21,n − q22,n ) ∆t q1,n+1 = q1,n + p1,n+1 ∆t

(3.47)

q2,n+1 = q2,n + p2,n+1 ∆t . The Euler algorithm with the same order looks almost identical. One only has to substitute p i,n+1 with p i,n in the third and fourth equation. However, the results differ vastly. For the initial conditions q1 = 0.1 ,

q2 = 0 ,

p1 = 0 ,

p2 = 0.1 ,

(E = 0.01)

both methods provide nonclosed quasiperiodic orbits around the origin. For the Euler code, their distance from the center increases at each revolution; after some 100 cycles the star leaves the bounded region and disappears into infinity. Accordingly the energy increases constantly, whereas using the symplectic method the star remains in the bounded region and its energy is constant for many 100 000 cycles, as it should be (Figure 3.4).

Fig. 3.4: Comparison between the Euler and the symplectic method, both of the same order ∆t 2 . Left frame: r = √ q 21 + q 22 over t; right frame: difference of the total energies.

56 | 3 Dynamical systems

3.3.3 Poincaré section For larger values of E and/or for different initial conditions, the trajectories can become more involved, and chaotic regions may appear. The trajectories can be visualized in a comfortable way with the Poincaré section method. To this end, one places an (N−1)-dimensional hyperplane, the Poincaré plane, somewhere in the N-dimensional phase space and marks the points of intersection of a trajectory for many revolutions. Periodic motion is then correlated to single isolated points on the plane, quasiperiodic behavior to curves, and chaotic regions to fractal objects having a dimension between one and two on the Poincaré plane. (n) Let ξ i , i = 1 . . . N − 1 be the coordinates of the nth intersection with the plane. (n) Then from each set ξ i the values for the next intersection point follows uniquely. Instead of a system of N ODEs, one finds the relation ξ ⃗ (n+1) = f ⃗(ξ ⃗ (n) )

(3.48)

which is nothing other than an (N − 1)-dimensional map, called a Poincaré map. However, computing f ⃗ in (3.48) requires solving the ODEs of the full system. 3.3.3.1 Example: Hénon–Heiles model For the Poincaré plane we choose q1 = 0 and obtain for (3.48) ξ1 = q2 (t s ) ,

ξ2 = p1 (t s ) ,

ξ3 = p2 (t s )

with t s from q1 (t s ) = 0. Since a conserved quantity exists, namely the total energy E (3.44), we can reduce the three-dimensional hyperplane further to a two-dimensional plane, for instance q2 (t s ), p2 (t s ). Since p21 (t s ) ≥ 0, all intersection points on this plane must be located inside the region −g(q2 ) ≤ p2 ≤ g(q2 )

with

g(q2 ) = √2E − q22 +

2 3 q . 3 2

A code for the symplectic integration, plotting of trajectories, and of the Poincaré section is provided in [5]. The initial values q2 (0), p2 (0) can be set using the mouse cursor, p1 (0) for a given E is computed from (3.44). Figure 3.5 shows some solutions for E = 0.12 and different initial values on the Poincaré plane (p2 , q2 ) as well as in position space (q1 , q2 ).

Bibliography [1] [2] [3] [4]

E. N. Lorenz, Deterministic nonperiodic flow, J. Atmos. Sci. 20, 130 (1963). J. V. José and E. J. Saletan, Classical Dynamics, Cambridge Univ. Press (1998). J. M. Thijssen, Computational Physics, Cambridge Univ. Press (2007). M. Hénon and C. Heiles, The applicability of the third integral of motion: Some numerical experiments, Astrophys. J. 69 (1964). [5] FORTRAN codes at http://www.degruyter.com/books/978-3-11-051513-8

Bibliography

| 57

0.5 0.5

0

0

–0.5 –0.5 –1

–0.5

0

0.5

1

–0.5

0

0.5

–0.5

0

0.5

–0.5

0

0.5

0.5 0.5

0

0

–0.5 –0.5 –1

–0.5

0

0.5

1

0.5 0.5

0

0

–0.5 –0.5 –1

–0.5

0

0.5

1

Fig. 3.5: Poincaré plane (left), trajectories in position space (right) for E = 0.12, and several initial conditions. Dashed: Bounded area defined by the total energy.

4 Ordinary differential equations I, initial value problems 4.1 Newton’s mechanics The present chapter mainly deals with problems arising from classical mechanics. We shall further investigate ordinary differential equations (ODEs) having the form G k [x i (t), ẋ i (t), ẍ i (t)] = 0 . The highest derivative occurring is now of the second order, normally in time. From here on we denote the first time derivative of x by x,̇ the second by x.̈

4.1.1 Equations of motion The second law of Newton provides equations of motion for N mass points at the positions r ⃗i (t), i = 1. . .N m i r ̈⃗i = F⃗ i (r1⃗ , . . . r ⃗N , r1̇⃗ , . . . r ̇⃗N , t)

(4.1)

with the force F⃗ i acting on the mass point i. The necessary initial conditions are normally given by positions and velocities at t = t0 r ⃗i (t0 ) = r ⃗i (0) ,

r ̇⃗i (t0 ) = v⃗ i (0) .

(4.2)

Often, the forces do not explicitly depend on time, F⃗ i = F⃗ i (r ⃗j , r ̇⃗j ) and (4.1) is called an autonomous system. If, on the other hand F⃗ i = F⃗ i (r ⃗j ) with

∂V(r1⃗ . . . r ⃗N ) F⃗ i (r ⃗j ) = − , ∂ r i⃗

the system (4.1) is denoted as conservative. In this case, the total energy E=

1 ∑ m i (r ̇⃗i )2 + V(r1⃗ . . . r ⃗N ) 2 i

(4.3)

constitutes a constant of motion. For a translational invariant system, a further condition F⃗ i = F⃗ i (r ⃗j − r ⃗k , r ̇⃗j , t) . holds for the forces. https://doi.org/10.1515/9783110515145-004

4.1 Newton’s mechanics |

59

4.1.2 The mathematical pendulum We commence our elaboration with the equation of motion of the damped onedimensional mathematical pendulum: φ̈ + α φ̇ + Ω20 sin φ = 0 ,

Ω20 = g/ℓ ,

(4.4)

realized for instance by a bar pendulum with a massless bar. Here, ℓ is the bar length, α > 0 the damping rate and φ denotes the displacement angle from the vertical. The equivalent first-order system for the two variables φ(t) and ω(t) reads φ̇ = ω ω̇ = −α ω − Ω20 sin φ

(4.5)

with the initial conditions φ(0) = φ0 ,

ω(0) = ω0 .

The system (4.5) has two fixed points (rest positions) φ̇ = ω̇ = 0: (0)

φ0 = 0 , (0)

(0)

φ1 = π , (0)

where φ0 has the properties of a stable focus, φ1 of an unstable saddle node. Multiplication of (4.4) by φ̇ and integration with respect to t yields 1 2 φ̇ − Ω20 cos φ = E0 − R(t) 2

(4.6)

with the integration constant E0 and a monotonically growing function R(t) = α ∫ dt φ̇ 2 . The expression on the left-hand side of (4.6) can be identified with the total mechanical energy E/mℓ2 of the system, R corresponds to the energy dissipated by friction and transformed to heat: R(t) = E0 − E(t) . Since E(t) ≥ −Ω20 and Ṙ ≥ 0, the rest position φ = 0, ω = 0 is asymptotically reached with t → ∞. During this process the mechanical energy R(t → ∞) = E0 + Ω20 has been converted to heat and the pendulum has come to rest, independently of its initial conditions. As long as E > E c with E c = Ω20 , the mass point passes through the upper unstable rest position (rotation), and if E < E c one observes oscillations around the lower rest position, also called libration.

60 | 4 Ordinary differential equations I For the conservative case without damping (α = 0), one finds E = E0 and the pendulum, once moving, never comes to rest. Depending on E0 the motion persists either as oscillation or rotation. For E0 = E c the motion is an infinitely long approach to the upper rest position. If the initial condition is also chosen with the upper rest position, the trajectory in phase space (φ-ω-plane) consists of a closed loop with initial and end points at the very same fixed point, also called a homoclinic orbit or separatrix.

4.2 Numerical methods of the lowest order 4.2.1 Euler method Next we try to solve the system (4.5) numerically using the Euler method. Introducing the discrete variables φ n = φ(n∆t), ω n = ω(n∆t), and the time step ∆t, the iteration rule reads φ n+1 = φ n + ω n ∆t (4.7) ω n+1 = ω n − (α ω n + Ω20 sin φ n ) ∆t . Figure 4.1 shows phase space and energy over t for different α and different time steps. For α = 0 (Figure 4.1 top) we expect a constant energy, which is obviously not the case. Moreover, all trajectories should be closed (periodic motion) instead of spiraling outwards. The simple Euler forward method produces an artificial negative(!) damping, the energy increases as well as the amplitude. After a certain time the separatrix is crossed and the oscillations turn into rotations. This behavior is qualitatively independent of ∆t. Figure 4.1 shows the damped case in the middle; the result looks more realistic. The trajectories spiral inwards and the energy reaches the value −Ω20 asymptotically. In contrary the behavior shown in Figure 4.1 bottom is again ruled by numerical artifacts. For larger time steps (∆t = 0.15) again a kind of ‘negative damping’ is obtained and the energy increases again. Let us draw some preliminary conclusions: – The results for α = 0 are all wrong (negative numerical ‘damping’). – For α > 0 we find qualitatively correct results for small ∆t. – Also for the damped case the energy grows continuously and the trajectories get off of the stable (!) fixed point if ∆t is chosen too large.

4.2.2 Numerical stability of the Euler method As we have already remarked in Section 3.1 the time step of the Euler method is bounded from above by the demand for numerical stability. We start with the general

4.2 Numerical methods of the lowest order

61

|

6 2 4 0 2

–2 0 –2

0

2

0

100

200

300

0

100

200

300

0

100

200

300

0.9 2 0.92 0.94 0 0.96 –2

–0.98 –1 –2

0

2

2

0

0 –0.5 –2 –1 –2

0

2

Fig. 4.1: Trajectories in phase space (left) and total energy over t for the damped pendulum (Ω0 = 1), computed using Euler’s method. Top α = 0, ∆t = 0.05, middle: α = 0.1, ∆t = 0.05, bottom: α = 0.1, ∆t = 0.15. The behaviors shown in the top and the bottom frames do not even qualitatively correspond to reality but are rather a product of numerical errors and instability.

62 | 4 Ordinary differential equations I

autonomous system

̇⃗ = f ⃗(x(t)) ⃗ x(t) .

(4.8)

The Euler method provides the discretization ⃗ + ∆t) = x(t) ⃗ + f ⃗(x(t))∆t ⃗ x(t + O(∆t2 ) .

(4.9)

⃗ = Let x⃗ (0) be a stable fixed point f ⃗(x⃗ (0) ) = 0. Then we may linearize (4.9) with x(t) (0) ⃗ for small deviations u⃗ around this fixed point: x⃗ + u(t) ⃗ ∆t ⃗ + ∆t) = u(t) ⃗ + L ⋅ u(t) u(t with the Jacobi matrix L ij =

󵄨 ∂f i 󵄨󵄨󵄨 󵄨 . ∂x j 󵄨󵄨󵄨󵄨x⃗ (0)

(4.10)

(4.11)

Due to the stability of x⃗ (0) , the eigenvalues of L must all have a nonpositive real part. We write (4.10) as ⃗ ⃗ + ∆t) = Q Ex ⋅ u(t) (4.12) u(t with QEx = 1 + ∆t L . To reflect the behavior close to a stable fixed point, the deviations u⃗ should not grow while iterating (4.12). Thus the spectral radius of Q Ex must be less or equal to one: ρ(Q Ex ) ≤ 1 .

(4.13)

This condition yields an upper boundary for ∆t. To demonstrate the method we take again the mathematical pendulum. Linearizing around the stable fixed point φ = ω = 0 yields the Jacobi matrix L=(

0 −Ω20

1 ) . −α

For the sake of simplicity we exclude here the overdamped case and assume α < 2Ω0 . Then the spectral radius of Q Ex turns out to be ρ = √1 − α ∆t + ∆t2 Ω20 and the stability criterion reads ∆t ≤

α . Ω20

This of course explains the strange behavior of the solutions shown in Figure 4.1. The stability condition at Ω0 = 1 simply reads ∆t ≤ α , which is only fulfilled for the solutions of the middle row in Figure 4.1. For the undamped case α = 0 the Euler method is unstable for arbitrarily small steps ∆t.

4.2 Numerical methods of the lowest order

| 63

4.2.3 Implicit and explicit methods Since the dependent variables at time t+ ∆t occur explicitly on the left-hand side of the iteration rule, the Euler forward method belongs to the group of explicit integration schemes. For an implicit method instead of (4.9) one finds the rule ⃗ + ∆t) = x(t) ⃗ + f ⃗(x(t ⃗ + ∆t))∆t + O(∆t2 ) . x(t

(4.14)

To compute the variables at the next iteration step, one must first solve (4.14) for ⃗ + ∆t). This is only possible in a unique way for a linear function f ⃗, x(t f ⃗(x)⃗ = A ⋅ x⃗ , and one obtains ⃗ + ∆t) = x(t) ⃗ (1 − ∆t A) ⋅ x(t or ⃗ ⃗ + ∆t) = Q Im ⋅ x(t) x(t

(4.15)

with = 1 − ∆t A . Q −1 Im Now the stability is determined from the spectral radius of (1−∆t A)−1 , which normally yields a much weaker restriction. One finds for the linearized pendulum ρ(Q Im ) =

1 √1 + α∆t + Ω20 ∆t2

,

E

0.4

Δt = 0.01 0.2 Δt = 0.03 Δt = 0.1 0

0

50

100

150 t

200

250

300

Fig. 4.2: Harmonic oscillator, implicit first-order scheme, α = 0. Although closed trajectories and a constant energy are expected, the method computes a damped solution. The artificial numerical damping goes with ∆t. Despite being stable for arbitrary time steps, the implicit method is still not very accurate.

64 | 4 Ordinary differential equations I which is less than one for all ∆t, even if α = 0. Thus the implicit method for the linearized pendulum (that is the harmonic oscillator) is unconditionally stable and therefore much more suitable than the explicit scheme. On the other hand, the numerical truncation error is still of order ∆t2 and this time generates a positive damping, leading to decaying oscillations in time even if α = 0, Figure 4.2.

4.3 Higher order methods The Euler method converges slowly and provides inaccurate and partially nonphysical results. Mainly for conservative systems this may lead to a qualitatively wrong behavior of the solutions. To increase the accuracy, higher order methods have been developed where the truncation error is of order ∆t3 or even better. In the following we introduce two higher order methods. Both are explicit and belong to the Runge–Kutta class.

4.3.1 Heun’s method We begin with the one-dimensional system dx = f(t, x) , dt

(4.16)

which is generalized later to n dimensions. We integrate (4.16) over t t+∆t

t+∆t

∫ dt󸀠 t

dx = x(t + ∆t) − x(t) = ∫ dt󸀠 f(t󸀠 , x(t󸀠 )) dt󸀠

(4.17)

t

and obtain the still exact iteration rule t+∆t

x(t + ∆t) = x(t) + ∫ dt󸀠 f(t󸀠 , x(t󸀠 )) .

(4.18)

t

If we approximate the integral by the square rule according to t+∆t

∫ dt󸀠 f(t󸀠 , x) ≈ f(t, x(t)) ∆t , t

the Euler method results. Taking the more accurate trapezoidal rule t+∆t

∫ dt󸀠 f(t󸀠 , x) ≈ (f(t, x(t)) + f (t + ∆t, x(t + ∆t))) t

∆t , 2

4.3 Higher order methods | 65

one obtains for (4.18), x(t + ∆t) = x(t) + (f(t, x(t)) + f (t + ∆t, x(t + ∆t)))

∆t . 2

(4.19)

Since x(t + ∆t) now occurs on the right-hand side as well, this is an implicit (or better a semi-implicit) algorithm. To convert (4.19) to an explicit scheme one computes x(t+∆t) on the right-hand side applying the Euler method x(t + ∆t) = x(t) + f(t, x(t)) ∆t and finally finds x(t + ∆t) = x(t) + (f(t, x(t)) + f (t + ∆t, x(t) + f(t, x(t)) ∆t))

∆t . 2

(4.20)

This is called Heun’s method, or sometimes the second-order Runge–Kutta method. 4.3.1.1 Accuracy To which order of ∆t is the truncation error in Heun’s method? We write (4.20) as (for the sake of simplicity we assume f(x) instead of f(t, x)) x(t + ∆t) − x(t) = (f(x(t)) + f (x(t) + f(x(t)) ∆t))

∆t 2

and expand the left-hand side with respect to ∆t L.H.S. =

dx 1 d2 x 2 1 d3 x 3 ∆t + ∆t + . . . ∆t+ dt 2 dt2 6 dt3 1 df d2 f 1 df 2 = f∆t + f∆t2 + (f 2 2 + f ( ) ) ∆t3 + . . . , 2 dx 6 dx dx

the right-hand side with respect to f∆t: R.H.S. =

df 1 d2 f 2 2 ∆t f ∆t ) + . . . , (2f + f∆t + 2 dx 2 dx2

where . . . denote terms of the order ∆t4 . Both sides coincide up to the order ∆t2 , that is, the truncation error is of the order ∆t3 and thus one order smaller than that of Euler’s method. 4.3.1.2 Numerical stability Similar to the Euler method we may formulate one iteration step for Heun’s method after linearization around a fixed point according to ⃗ ⃗ + ∆t) = Q H ⋅ x(t) x(t with Q H = 1 + ∆t L +

1 2 2 ∆t L 2

66 | 4 Ordinary differential equations I

and the Jacobi matrix L as in (4.11). Again, numerical stability is given by the restriction ρ(Q H ) ≤ 1, which eventually leads to 󵄨󵄨 󵄨󵄨 1 󵄨 󵄨 ρ = max 󵄨󵄨󵄨1 + ∆tλ i + ∆t2 λ2i 󵄨󵄨󵄨 ≤ 1 , 󵄨󵄨 2 i 󵄨󵄨 where λ i are the eigenvalues of L. For the example of the harmonic oscillator with λ12 = −

α 1 ± i√4Ω20 − α 2 2 2

wie obtain the stability limit: −α +

1 2 1 1 α ∆t − αΩ20 ∆t2 + Ω40 ∆t3 = 0 . 2 2 4

This provides an upper time step which is for α > 0, larger than that for Euler’s method (Figure 4.3, ∆t c ≈ 1.4). However, for the undamped case α = 0, Heun’s method is still unstable for all ∆t. Taking the symplectic method from Chapter 3, one iteration step for the harmonic oscillator can be formulated according to (3.41): φ φ = Qs ( ) ( ) ω n ω n+1 with (see Problems) 1 − Ω20 ∆t2 Qs = ( −Ω20 ∆t

n

Runge–Ku

tta

Heu

ρ

Runge–Ku

ρ

(4.21)

sympl.

tta

Heu n

Eul

Eul

er

er

∆t − α∆t2 ) . 1 − α∆t

pl.

sym Δt

Δt

Fig. 4.3: Spectral radii over ∆t for the harmonic oscillator, Ω0 = 1, left: damped, α = 1/2, right: undamped, α = 0. Especially for the undamped case the symplectic method discussed in Chapter 3 is superior to any other, even higher-order method.

4.3 Higher order methods | 67

The stability is given by the spectral radius of Q s . As long as both eigenvalues are complex one finds ρ(Q s ) = √| Det Q s | = √|1 − α∆t| . For the special (Hamiltonian) case of the undamped oscillator α = 0, one obtains the pleasant result ρ = 1, and the symplectic method should be stable for ∆t < 2/α. However, for ∆t ≥ (−α + √α 2 + 4Ω20 )/2Ω20 the complex conjugate pair of eigenvalues of Q s becomes real valued and the spectral radius quickly exceeds one; compare Figure 4.3. Despite this restriction and especially for a weak damping (α ≪ Ω), the symplectic method is much better suited than other methods.

4.3.2 Problems 1.

For the Crank–Nicolson method, the right-hand side of the ODEs is written as a linear combination of explicit and implicit terms. Let y(x) be the solution of dy(x) = f(y(x)) dx ̃ ̃ 0 ) = y(x0 ), and y(x) the numerically obtained approximation. Starting with y(x the Crank–Nicolson scheme provides ̃ 0 + ∆x) − y(x ̃ 0) 1 y(x ̃ 0 + ∆x))] . ̃ 0 )) + f(y(x = [f(y(x ∆x 2

(4.22)

Show that the numerical error after one time step reads ̃ 0 + ∆x) − y(x0 + ∆x)| = O(∆x3 ) . |y(x 2.

Derive the amplification matrix (4.21) for the symplectic method.

4.3.3 Runge–Kutta method Taking a higher order algorithm increases both step size and accuracy. Even higher order methods are possible, however, the numerical effort would also increase. For Heun’s method one has to evaluate f at two different function values for each iteration step. To reduce the algorithm to practice one should always find a compromise between accuracy and effort. An algorithm that has stood the test of time is the already in 1895 developed Runge–Kutta method (RK). RK exists in different orders; normally the fourth order scheme (which we shall abbreviate by RK4) is taken. For the sake of simplicity we explain the principle at a method of the second order. Let x = x(t) be the desired solution of the ODE ẋ = f(t, x(t)) .

68 | 4 Ordinary differential equations I We expand x at t + ∆t/2 x(t) = x(t + ∆t/2) −

∆t 1 ∆t 2 ̇ + ∆t/2) + ( ) x(t ̈ + ∆t/2) + O(∆t3 ) x(t 2 2 2

x(t + ∆t) = x(t + ∆t/2) +

∆t 1 ∆t 2 ̈ + ∆t/2) + O(∆t3 ) ̇ + ∆t/2) + ( ) x(t x(t 2 2 2

and subtract the two equations ̇ + ∆t/2) + O(∆t3 ) . x(t + ∆t) = x(t) + ∆t x(t It results in an iteration scheme of the order of ∆t2 . Similar to Heun’s method one must know ẋ = f for a time later than t to compute the right-hand side. Again this is accomplished by the Euler method ̇ + ∆t/2) = f (t + ∆t/2, x(t + ∆t/2)) = f (t + ∆t/2, x(t) + ∆t/2 f(t, x(t))) + O(∆t2 ) x(t and one iteration step takes the following form: k 1 = ∆t f(t, x(t)) k 2 = ∆t f(t + ∆t/2, x(t) + k 1 /2)

(4.23)

x(t + ∆t) = x(t) + k 2 . Thus an explicit scheme of order ∆t2 is constructed where f must be evaluated twice at each step. Although it looks different, the scheme (4.23) is similar to Heun’s method (4.20) up to the order ∆t3 , which can be shown by expanding (4.23) with respect to ∆t/2 at t and t + ∆t. In the same manner the fourth order RK scheme is derived: k 1 = ∆t f(t, x(t)) k 2 = ∆t f(t + ∆t/2, x(t) + k 1 /2) k 3 = ∆t f(t + ∆t/2, x(t) + k 2 /2) k 4 = ∆t f(t + ∆t, x(t) + k 3 ) 1 x(t + ∆t) = x(t) + (k 1 + 2 k 2 + 2 k 3 + k 4 ) , 6 and is easily coded in FORTRAN95:

C C C C C

SUBROUTINE rk4(x,t,n,dt,eq) integrates the ODE system defined in eq from t to t+dt x dependent variables x(n) t independent variable n number of dependent var. dt time step

(4.24)

4.3 Higher order methods | 69

C

eq the ride-hand sides of the ODEs (subroutine) REAL, DIMENSION(n) :: x,f1,f2,f3,f4 CALL eq(f1,x,t); f1=dt*f1 CALL eq(f2,x+.5*f1,t+dt/2.); f2=dt*f2 CALL eq(f3,x+.5*f2,t+dt/2.); f3=dt*f3 CALL eq(f4,x+f3,t+dt) x=x+(f1+2.*f2+2.*f3+f4*dt)/6. END

The error for RK4 is of the order ∆t5 and the restriction for the step size originates now from the stability condition (the detailed computation is left for the reader) 󵄨󵄨 1 1 1 4 4 󵄨󵄨󵄨󵄨 󵄨 ρ = max 󵄨󵄨󵄨1 + ∆tλ i + ∆t2 λ2i + ∆t3 λ3i + ∆t λ i 󵄨󵄨 < 1 . 󵄨󵄨 2 6 24 i 󵄨󵄨

(4.25)

An evaluation of ρ is also shown in Figure 4.3. As expected, the stability region is much larger than those of the other schemes having a lower order. 4.3.3.1 Example: mathematical pendulum with RK4 As an application we use RK4 to solve the equations for the nonlinear pendulum with f ⃗ from (4.5). Note that even for the problematic conservative case α = 0 convergent results and constant energy are obtained if the time step is not chosen to be too large (ρ ≈ 1 in (4.25)); see Figure 4.4. A code section computing a trajectory with initial values φ = 2, ω = 0 could read as PROGRAM pendulum_rk4 REAL, DIMENSION(2) :: y

! variables phi, omega

... y=(/2.,0./) ! initial values for phi and omega t=0.; dt=0.01 DO WHILE(t.LT.tend) ! time iteration loop begins t=t+dt CALL rk4(y,t,2,dt,pendulum_deq) ! compute one time step with rk4 ... ENDDO ... If the subroutine rk4 has been placed in the program library (see Appendix B), only the subroutine pendulum_deq has to be specified in the file below the main section (MAIN), here pendulum_rk4. This can be done even inside MAIN using the CONTAINS statement. Then the parameters, here omega and alpha, are global and also defined in the subroutine. Moreover, MAIN recognizes pendulum_deq as the name of a subroutine and transfers it accordingly to rk4.

70 | 4 Ordinary differential equations I

0 2 0.2 –0.4 0 –0.6 –0.8

–2

–1 –2

0

2

0

100

200

300

0

100

200

300

0.5 2

0 0 –0.5 –2 –1 –2

0

2

Fig. 4.4: The undamped mathematical pendulum (α = 0) with RK4. For a time step too large, numerical friction distorts the result (top, ∆t = 1), for ∆t = 0.1 (bottom) the energy is conserved well for many periods (Ω0 = 1, T = 2π).

... CONTAINS SUBROUTINE pendulum_deq(rhside,y,t) REAL, DIMENSION(2) :: rhside,y rhside(1)=y(2) rhside(2)=-alpha*y(2)-omega**2*SIN(y(1)) END SUBROUTINE pendulum_deq END PROGRAM pendulum_rk4 The other, more traditional ‘FORTRAN77-like’ way is to avoid the CONTAINS statement and the sometimes confusing use of global variables by defining pendulum_deq by an

4.3 Higher order methods | 71

EXTERNAL statement in MAIN. The parameter transfer to the subroutine defining the right-hand sides of the ODEs can then be realized by a COMMON block. 4.3.3.2 RK4 with adaptive step size Up to here the step size ∆t was constant and, besides stability restrictions, chosen rather arbitrarily. This is the most easiest way but may not always be the optimal one. A step width too large renders the results inaccurate, while a step size too small unnecessarily increases the number of iterations and therefore the computational effort. An algorithm adapting the step size during the computations can improve the situation considerably. On the one hand one can thus prescribe a desired accuracy (or upper error limit); on the other hand the method can automatically decrease the step size at times where x⃗ is slowly varying and vice versa. ⃗ and Using RK4 the truncation error is of O(∆t5 ). Let us start the integration at x(t) ⃗ ⃗ compare two numerically computed x1 (t + ∆t), x2 (t + ∆t), one found with step size ∆t (one iteration), the other with ∆t/2 and two consecutive iterations. Let d(∆x) be the euclidean distance d(∆t) = |x⃗ 1 − x⃗ 2 | , then d(∆t) = |c|∆t5 where c depends somehow on the fifth derivative of x⃗ at time t. If we compute d for two different step sizes we may eliminate c and find d(∆t1 ) ∆t1 5 ) . =( d(∆t2 ) ∆t2 This can be solved for ∆t2 : ∆t2 = ∆t1 (

d(∆t2 ) 1/5 . ) d(∆t1 )

Now we compute one iteration with a certain fixed ∆t1 and require d(∆t2 ) = ϵ as the desired accuracy. Then the step width to achieve this accuracy reads ∆t2 = ∆t1 (

1/5 ϵ . ) d(∆t1 )

In practice the method is more robust if the exponents are taken variably, depending whether ∆t increases or decreases: ∆t2 = ∆t1 (

p ϵ ) d(∆t1 )

and {1/5 if p={ 1/4 if {

d 0 ,

(4.26)

were proposed by Lotka (1920) and Volterra (1931) to model the interaction between a prey n1 (t) and a predator n2 (t) population. They are nowadays known as the Lotka– Volterra model and constitute the most simple predator-prey system. 1. Explain the terms on the right-hand sides of (4.26). Show that performing a proper scaling of time, n1 and n2 , (4.26) can be cast into the form ṅ̃ 1 = a ñ 1 − ñ 1 ñ 2 ñ̇ 2 = −ñ 2 + ñ 1 ñ 2 , 2. 3.

a>0

Compute the fixed points of (4.27) and examine their stability. Show that W(ñ 1 , ñ 2 ) = ñ 1 + ñ 2 − ln ñ 1 − a ln ñ 2

(4.27)

(4.28)

is a conserved quantity. 4. Compute numerical solutions of (4.27) using RK4 and check the conservation of (4.28). Plot the trajectories in state space for several a and different initial conditions.

4.4 RK4 applications: celestial mechanics | 73

4.4 RK4 applications: celestial mechanics The basic subject of classical celestial mechanics is the computation of the trajectories of point masses (planets or stars) interacting with Newton’s law of gravity. Only the most simple situation of two bodies in an otherwise empty universe can be solved exactly. This situation is known as the two-body or Kepler problem. Its solutions are the well-known ellipses, parabolas, or hyperbolas.

4.4.1 Kepler problem: closed orbits Two bodies (point masses) with mass m i are located at r i⃗ (t). Between both bodies a force according to Newton’s law of gravity is assumed. The equations of motion read r1⃗ − r2⃗ |r1⃗ − r2⃗ |3 m1 ̈ r2⃗ − r1⃗ = − r2⃗̈ = −G m1 r1⃗ m2 |r2⃗ − r1⃗ |3

r1̈⃗ = −G m2

(4.29)

with Newton’s gravity constant G ≈ 6.67 ⋅ 10−11 m3 /(kg s2 ). Introducing relative coordinates η⃗ = r1⃗ − r2⃗ , the six mechanical degrees of freedom can be reduced to three (this is a consequence of the conservation of the total momentum) η⃗ η̈⃗ = −G M 3 η

(4.30)

with the total mass M = m1 + m2 . Taking the center of mass as the origin of the coordinate system, one has the relations r1⃗ =

m2 η⃗ , M

r2⃗ = −

m1 η⃗ . M

(4.31)

Due to the central forces the total angular momentum is conserved and the motion can be further restricted to a plane, leaving only two mechanical degrees of freedom. Without loss of generality, we can study the 2D problem with η⃗ = (x, y). After appro1/3 one obtains the two ODEs of second order without any ̃ priate scaling |η| = |η|(GM) further parameters x ẍ = − 3 r (4.32) y ÿ = − 3 r 2 2 √ where r = x + y . The total energy measured in multiples of GM 2/3 reads E=

1 1 2 (ẋ + ẏ 2 ) − 2 r

(4.33)

and represents another constant of motion. Together with the constant angular momentum L = x ẏ − y ẋ (4.34)

74 | 4 Ordinary differential equations I

the number of dynamical degrees of freedom in (4.32) is further reduced from four to two. But this excludes chaotic behavior and one expects at least for bounded solutions either fixed points or periodic solutions¹. Fixed points with a finite E do not exist for the Kepler problem and the periodic solutions correspond to the closed orbits of the planets. An obvious solution of (4.32) is a circular orbit with radius R, x = R cos(ωt) ,

y = R sin(ωt) .

Inserting this into (4.32) yields 1 , (4.35) R3 in agreement with Kepler’s third law. Numerical solutions of (4.32) can be achieved with the help of RK4. The subroutine defining the system (4.32) takes the form ω2 =

SUBROUTINE kepler_deq(rhside,y,t) REAL, DIMENSION(4) :: rhside,y r32=(y(1)**2+y(3)**2)**(3/2.) rhside(1)=y(2) rhside(2)=-y(1)/r32 rhside(3)=y(4) rhside(4)=-y(3)/r32 END Taking the initial conditions x(0) = R ,

y(0) = 0 ,

̇ x(0) =0,

̇ y(0) = L/R ,

one finds a circular orbit for L = L0 = √R with radius R, otherwise ellipses, or for even larger L, hyperbolas. For L < L0 , the initial point coincides with the aphelion (this is the largest distance from the planet to the sun), and for L > L0 , with the perihelion of the ellipse. For E, (4.33) is evaluated as E=

L2 1 − , 2R2 R

i.e., if L > √2R one has E > 0 and no bounded solutions exist anymore. The two bodies are free and move on hyperbolas around their common center of mass. In general there is now friction in the celestial mechanics of mass points. Then it may become crucial how numerical friction will modify the results. How far is the energy (4.33) conserved? To shed light on this issue we show in Figure 4.6 the value of E after 10 000 revolutions as a function of ∆t for a circular orbit with R = 1, L = 1,

1 However, this applies only for the 1/r-potential, see the remarks in Section 4.4.2.

4.4 RK4 applications: celestial mechanics | 75

Fig. 4.6: Deviation of the total energy of a planet revolving on a circular orbit (R = 1, L = 1) from the exact value E = −1/2 after 10 000 periods, plotted over the time step ∆t. Although only of the order ∆t 2 , the symplectic algorithm is superior to the fourth order Runge–Kutta method for larger ∆t.

ω = 1. One recognizes a very precise energy conservation up to time steps of about 0.05. Note that the orbital period is 2π, and thus for ∆t = 0.05, one circle is resolved by about 120 time steps only. Nevertheless the total energy after t = 10 000 ⋅ 2π is about E = −0.50024, a deviation of less than 0.05%. Taking the symplectic method of the lowest order one time step is easily integrated by

C

SUBROUTINE symplect(y,dt) symplectic integrator for one time step REAL, DIMENSION(4) :: y r32=(y(1)**2+y(3)**2)**(3/2.) ! denominator y(2)=y(2)-y(1)/r32*dt y(4)=y(4)-y(3)/r32*dt y(1)=y(1)+y(2)*dt y(3)=y(3)+y(4)*dt END

As it can be seen also from Figure 4.6, the symplectic method is not always preferable. For a larger time step it conserves the energy better, but for smaller time steps the accuracy of RK4 is much higher.

76 | 4 Ordinary differential equations I

1

0.5 0

–0.5

–1 –1

–0.5

0

0.5

1

Fig. 4.7: Perihelion precession for a tiny deviation from the 1/r-potential. Orbits computed with RK4 and R = 1, L = 0.7, and ϵ = 0.05.

4.4.2 Quasiperiodic orbits and apsidal precession For E < 0 all orbits are closed. This is a special property of the 1/r-potential and any tiny deviation 1 V(r) = − (1+ϵ) , |ϵ| ≪ 1 r leads to nonclosed, quasiperiodic trajectories whose perihelion turns slowly around the central mass (Figure 4.7). For ϵ ≠ 0 the only closed orbit is the circle (for a more detailed discussion see Problems, Section 4.4.5). Previously we argued that the motion is reduced due to the conservation laws on a plane with two dynamical degrees of freedom. But how then can crossing trajectories emerge as depicted in Figure 4.7? Apparently there exist two possible paths after each ̇ y), ẏ = y(x, ̇ y). crossing point. This is a consequence of the not unique relation ẋ = x(x, If we eliminate for instance ẏ with the help of (4.34) from the energy law (4.33) we find ̇ y) having two different solutions. a quadratic equation for x(x,

4.4.3 Multiple planets: is our solar system stable? Things are getting much more involved if the system is extended to more than one planet circling the central star. Analytic solutions of the equations of motions can be found only by perturbation theory and for very special cases. In 1887 the King of Sweden Oscar II asked the question ‘is our solar system stable?’ and offered a prize of 2500 crowns for the answer. Even nowadays this question is not definitely decided. On the one hand, in 1890 Henri Poincaré could find a kind of counter-evidence. He mathematically showed that already for the three-body problem, i.e., a planet close to a double star, no regular orbits can exist. On the other hand, the French astronomer Jacques Laskar computed numerically in 1989 the orbits of the

4.4 RK4 applications: celestial mechanics | 77

four inner planets, and in 1994, of all planets in our solar system for the next 25 billion years and found that the trajectories are ‘weakly chaotic,’ but collisions between the planets are rather unlikely in the next 200 million years [1, 2]. A further indication to the instability of certain orbit radii is the distribution of asteroids between Mars and Jupiter. One observes the so-called Kirkwood gaps occurring at orbit radii having a period of revolution TA as a rational of the period of Jupiter: TA =

TJ , n

n = 2, 7/3, 5/2, 3, . . . .

(4.36)

Resonances occur along those orbits, which pull out the asteroids located close to these radii in a short time (at least on the astronomic scale); see also left frame of Figure 1.1. The interaction of three celestial bodies can be examined numerically. Based on (4.29) we consider the extended ODE system r1⃗ − r2⃗ r1⃗ − r3⃗ − G m3 3 ⃗ ⃗ | r1 − r2 | |r1⃗ − r3⃗ |3 r2⃗ − r1⃗ r2⃗ − r3⃗ − G m3 r2̈⃗ = −G m1 |r2⃗ − r1⃗ |3 |r2⃗ − r3⃗ |3 r3⃗ − r1⃗ r3⃗ − r2⃗ − G m2 . r3̈⃗ = −G m1 3 |r3⃗ − r1⃗ | |r3⃗ − r2⃗ |3 r1̈⃗ = −G m2

(4.37)

Again the number of degrees of freedom can be reduced due to the conservation of the total momentum. Taking the center of mass as the origin of the coordinate system, we may eliminate for instance r3⃗ : r3⃗ = −α 1 r1⃗ − α 2 r2⃗ .

(4.38)

Here, α i = m i /m3 denote the dimensionless mass ratios. Scaling the time according to Gm3 1/2 τ=t( 3 ) ℓ where ℓ is a length to be determined later, one finds the dimensionless system r1⃗ − r3⃗ r1⃗ − r2⃗ − |r1⃗ − r2⃗ |3 |r1⃗ − r3⃗ |3 r2⃗ − r3⃗ r2⃗ − r1⃗ − r2̈⃗ = −α 1 3 |r2⃗ − r1⃗ | |r2⃗ − r3⃗ |3

r1̈⃗ = −α 2

(4.39)

with r3⃗ from (4.38). The energies of the three bodies in units of ℓ/Gm23 read Ei =

1 α i (r ̇⃗i )2 + ∑ V ik , 2 k=i̸

V ik = −

αi αk , | r ⃗ i − r ⃗k |

(4.40)

78 | 4 Ordinary differential equations I

which of course are not conserved. The total energy, however, E=

1 1 ∑ α i (r ̇⃗i )2 + ∑ V ik 2 i 2 i,k=i̸

(4.41)

constitutes again a constant of motion and can be used as a control value for any numerical integration. In addition, the three components of the total angular momentum L⃗ = ∑ m i (r ⃗i × r ̇⃗i ) i

must also be conserved. Counting the number of dynamical degrees of freedom in three dimensions, one finds 4 × 3 (number of equations) −1 (energy) −3 (momentum) = 8, more than enough to expect chaotic behavior. We shall however continue the treatment with the restriction to two spatial dimensions. This of course is a special case that can be realized if for all bodies the initial condition reads z i (0) = ż i (0) = 0. The number of degrees of freedom is then reduced to only 4 × 2 − 1 − 1 = 6 but still enough to leave room for chaos. The system (4.39) can be integrated by either applying RK4 or using a symplectic algorithm. Depending on the α i and on the initial conditions, various qualitatively different motions can be obtained. A solution for relatively small α i is depicted in Figure 4.8. Because of m3 ≫ m1 , m2 , the mass point denoted by i = 3 (blue) plays the role of the central star. It is revolved by the ‘planets’ with i = 2, 3. Affected by the (weak) interaction, the orbits are no longer closed and show weakly chaotic behavior. If the α i are larger, the solutions become very irregular and chaotic. Often one body can absorb enough energy to escape the system even if the total energy is negative, E < 0. This mass point detaches from the other two over the course of time and for t → ∞, a two-body problem remains, showing periodic and closed Kepler orbits (Figure 4.9). Both the solutions shown in Figure 4.8 and Figure 4.9 have been computed with a symplectic method of order ∆t2 , and the changes of total energy and angular momentum have been recorded. With the time step ∆t = 2 ⋅ 10−6 the maximal relative deviations 󵄨󵄨 X 󵄨 󵄨 max − Xmin 󵄨󵄨󵄨 ∆X = 2 󵄨󵄨󵄨 󵄨 󵄨󵄨 Xmax + Xmin 󵄨󵄨󵄨 in the time interval 0 < t < 2600 are for Figure 4.8 ∆E = 4.8 ⋅ 10−7 ,

∆L = 3.0 ⋅ 10−12 ,

∆E = 2.2 ⋅ 10−5 ,

∆L = 2.9 ⋅ 10−12 .

and for Figure 4.9 The restriction of the motion on a plane strongly increases the probability of collisions between two bodies. Here of course, a 3D-computation would be more realistic.

4.4 RK4 applications: celestial mechanics

10

0.1

Ei

|ri (t)|

1

0.01 0.001

500

1000

1500 2000

2500

–0.0022 –0.0024 –0.0026 –0.0028 –0.0030 –0.0032 –0.0034 –0.0036 –0.0038 –0.0040 –0.0042

500

t

1000

| 79

1500 2000

2500

t

Fig. 4.8: The three-body problem and numerical solutions for α 1 = α 2 = 0.001. Left: the distances |r i | from the origin; right: the energies E 1 , E 2 from (4.40). The two lighter bodies (red, green) circuit the heavy one (blue) on weakly distorted circular orbits.

1000

0

100

–0.005 –0.01

1

Ei

|ri (t)|

10

–0.015

0.1

–0.02

0.01 0.001 500

1000 1500 2000 2500 t

–0.025

500

1000 1500 2000 2500 t

Fig. 4.9: Same as Figure 4.8, but for α 1 = 0.015, α 2 = 0.025. Now the evolution seems much more chaotic. At t ≈ 100 the at first inner planet (green) moves outwards. It continues absorbing energy from the other two bodies and reaches even larger orbits. Eventually it may escape from the influence of the two other masses (E 2 ≈ 0, t > 1400). Then the two remaining bodies form a stable periodic Kepler system.

4.4.4 The reduced three-body problem To arrive from (4.37) at the so-called reduced three-body problem, the following approximations are in order: – All masses move in the same plane (2D). – One of the three masses is light enough (a small planet, an asteroid, or a space ship) not to influence the motion of the two other ones. Let us assume m1 ≪ m2 , m3 . Then m1 is called the test body (TB). – The two heavy bodies (named main bodies, e.g., sun and Jupiter or a double star) move in circles around their common center of mass with the angular frequency ω. Lengths and time are scaled to their distance and period so that |r2⃗ − r3⃗ | = 1 and ω = 1.

80 | 4 Ordinary differential equations I

4.4.4.1 The explicitly time-dependent equations of motion An important system parameter is provided by the mass relation μ=

m3 , m2 + m3

0i

4.5.2 Boundary conditions For the majority of MD simulations, periodic boundary conditions are assumed. The integration domain with periodic length L is thought to be repeated in each space direction infinitely many times. Each particle is not only interacting with every other one in L but also with the ones in the periodic copies. In this way a much larger system is simulated. Wall effects, however, cannot be studied since the system is in fact infinite.

88 | 4 Ordinary differential equations I

A much more simple way is to take rigid, impermeable, and perfectly reflecting walls around the integration domain. If while integrating a particle position x i , y i outside of the domain turns out, say on the right-hand side x i > L, the particle is to be reflected. Then its new position and x-velocity are simply set to x i → 2L − x i ,

ẋ i → −ẋ i .

The same can be done for y and correspondingly for the walls at x = 0, y = 0. The particles interact with the walls in the form of elastic collisions where energy and absolute value of momentum are conserved.

4.5.3 Microcanonical and canonical ensemble The macroscopic state variables of a closed many-particle system are the internal energy E, the volume V (in 2D V = L2 ), and the number of particles N. All three quantities are constants of motion and can be adjusted by the system and the initial conditions, respectively. Using the notions of statistical mechanics, such a system is called a member of the microcanonical ensemble. Each member is represented by a point in the 4N-dimensional state space, the Γ-space. When time goes on, these points travel on trajectories in a 4N − 1-dimensional hyperplane E(x1 , y1 . . .x N , y N , u 1 , v1 . . .u N , v N ) = const. where u i = ẋ i , v i = ẏ i denote the particle velocities. We may assign a temperature to each ensemble member using the equipartition theorem according to f kB T = Ek 2

(4.63)

where f denotes the number of mechanical degrees of freedom. During the temporal evolution the temperature will change and reaches its equilibrium value T0 for t → ∞. Scaling the temperature with ϵ/k B and as in (4.60), the energy with ϵ one obtains with f = 2N: (4.64) T = Ek /N . If one is interested in the thermal behavior of the simulated material, one normally prescribes the temperature by coupling the system to a heat bath. Then, the internal energy will adjust itself accordingly and the ensemble is called canonical. There are different ways to fix the temperature to a desired value. These algorithms are called thermostats. Due to (4.63) Ek is given through T. Thus, after a certain number of time steps one can repeatedly multiply the speed of each particle with the factor √Ts /T, fulfilling (4.63). Here, Ts denotes the prescribed nominal temperature of the heat bath.

4.5 Molecular dynamics (MD) | 89

Another way to realize a thermostat is assuming a particle-speed-dependent friction in the equations of motion. If we add to the right-hand sides of (4.57) the terms −γ(T) r ̇⃗i and take

Ts ) , (4.65) T it can be shown that the temperature Ts is reached exponentially in time with exp(−2γ0 t) (Problems). Each particle is thereby permanently in contact with the heat bath, releasing energy to the bath if T > Ts , receiving energy for T < Ts . γ(T) = γ0 (1 −

4.5.4 A symplectic algorithm Here we shall only briefly describe the algorithm; a detailed and commented listing can be found in [3]. To integrate (4.57) with (4.65) a symplectic scheme is recoṁ a system of 4N first-order mended. Introducing the particle velocities (u, v) = (x,̇ y), ODEs is obtained having the form: N

u̇ i = 24 ∑ f(r ij ) (x i − x j ) − γ(T) u i j=i̸

ẋ i = u i

(4.66)

N

v̇ i = 24 ∑ f(r ij ) (y i − y j ) − γ(T) v i j=i̸

ẏ i = v i , which after time discretization can be brought into the symplectic scheme (see Section 3.3.2) (n+1)

= ui

(n+1)

= xi

ui xi

+ [24 ∑ f (r ij ) (x i [ j=i̸

(n)

+ ui

(n+1)

= vi

(n+1)

= yi

vi yi

N

(n)

(n+1)

(n)

(n)

(n) (n) − x j ) − γ(T) u i ] ∆t ]

∆t (4.67)

N

(n)

+ [24 ∑ f (r ij ) (y i [ j=i̸

(n)

+ vi

(n+1)

(n)

(n)

(n) (n) − y j ) − γ(T) v i ] ∆t ]

∆t .

4.5.4.1 Optimization To find the forces acting on a certain particle (called reference particle in the following) one must sum over all N − 1 pair forces caused by the other particles. This corresponds to N(N − 1) evaluations of the pair potential at each time step. The program

90 | 4 Ordinary differential equations I

can be vastly optimized if only those particles located within a certain distance r m to the reference particle are taken into account. The Lennard-Jones potential decreases with 1/r6 , thus it should be sufficient to take say r m = 4r0 . Then U(r m ) ≈ −0.0005 can safely be neglected against U(r0 ) = −1. To restrict the pair sum on the neighbors laying in a sphere (or disc in 2D) with radius r m , one must know the ambient particles for the reference particle, demanding a certain bookkeeping during the dynamical process. This can be achieved most easily with a two-dimensional array NB. Let the particles be numbered with k = 1 . . . N (ordinal number). Then the first index of NB is assigned to the ordinal number of the reference particle, say k, while the second one lists the neighbors. There, NB(k, 1) is equal to the number of neighbors of the reference particle k. If we have, for instance, NB(k, 1) = 5, particle number k has five neighbors inside r m , whose ordinal numbers are stored in NB(k, 2) through NB(k, 6). Since the particle positions change continuously, the neighborhood array must be updated, taking an additional computational effort. However, depending on the maximal particle speed this update is not necessary after each single time step. In practice it proves to be sufficient to recompute NB after, say, 20–30 steps. 4.5.4.2 Initial conditions As usual in classical mechanics one needs two complete sets of initial conditions, namely particle positions and velocities at t = 0. The particles, for instance, can be ordered on a square grid with width r0 . This of course cannot correspond to an equilibrium state since next-nearest neighbors also interact (Problems). The velocities can be chosen close to thermal equilibrium according to a Maxwell–Boltzmann distribution, applying the Box–Muller method described in more detail in Section 8.1.3.3. On this behalf, one may take (0)

= √−2T ln(1 − ξ1 ) cos(2πξ2 )

(0)

= √−2T ln(1 − ξ1 ) sin(2πξ2 )

ui vi

(4.68)

with two independent, equally distributed random numbers ξ1 , ξ2 in [0,1).

4.5.5 Evaluation 4.5.5.1 Phases in configuration space Figure 4.14 shows three snapshots of 1600 particles, each one for a different temperature. For large T, the gaseous state is clearly distinguished. The main difference between ‘liquid’ and ‘solid’ is, apart from small fluctuations, the constant particle position. In Figure 4.15 we plot the distance of two initially neighbored particles over time. Averaging over all distances one should find for the liquid state a behavior as ⟨r2 ⟩ = D t .

4.5 Molecular dynamics (MD)

| 91

Fig. 4.14: Configurations of 1600 particles and different temperatures, each after t = 200.

Fig. 4.15: Distance of two initially neighbored particles as a function of time.

For the solid state, the diffusion constant D goes to zero. Taking only 1600 particles, a rather large percentage is close to phase separating walls, and surface effects may play an important role. Surface effects can be avoided by applying periodic boundary conditions in space, for which we refer to the literature, e.g., [5]. 4.5.5.2 Pair correlation function In 3D configuration space, the pair correlation function g(r) describes the probability of finding a particle at distance r from another particle in a spherical shell with volume 4πr2 dr. In 2D space, the shell turns into a ring with the area 2πrdr. Let n(r) be

92 | 4 Ordinary differential equations I

Fig. 4.16: Scaled pair correlation function for several temperatures, solid: T = 0.12, dash-dotted: T = 0.44, dashed: T = 1.25.

the number of particles located on the ring [r, r + ∆r], N

n(r) = ∑ Θ(r ij − r) Θ(r + ∆r − r ij ) ,

(4.69)

i,j=i̸

where Θ denotes the step function, and g(r) can be computed as: g(r) 2πr∆r =

1 n(r) . N(N − 1)

(4.70)

To find n(r), one needs a histogram. To this end, the particles n(r) laying on the ring are counted and stored for several discrete values of r i = i∆r at a certain time. To avoid fluctuations, one averages (4.69) over a certain time interval (time average = ensemble average). Figure 4.16 shows the (scaled) pair correlation functions for the three different temperatures from Figure 4.14. Clearly visible is the transition from a far order (solid state) to a near order (liquid) and finally to a pure repulsion (gaseous) if temperature is increased. 4.5.5.3 Specific heat The specific heat of a mechanically closed system (constant volume) is given by cv =

dE dT

(4.71)

4.5 Molecular dynamics (MD) |

93

Fig. 4.17: Internal energy E p over T , averaged over a time interval ∆t = 100. When close to a phase transition, E p varies strongly. Simulation with N = 1600.

with the total energy E as (4.60). There are different ways to compute cv from MD simulations. For a canonical ensemble one finds 1 Var(E) cv = kb T2 with the variance Var(E) = ⟨E2 ⟩ − ⟨E⟩2 and ⟨. . . ⟩ as the ensemble or time average. However, for the computations described above, E is a conserved quantity and thus Var(E) = 0. If, on the other hand, one substitutes in (4.60) Ek = NT, (4.71) yields dEp . (4.72) dT Ep (T) can be easily gained from an MD simulation. The dependence of Ep on T is shown in Figure 4.17. The strong increase around T ≈ 0.45 indicates a phase transition. Indeed, cv should diverge in the thermodynamic limit at a phase transition. Another method to compute cv was developed by Lebowitz et al. [6]. There, one needs only the variance of the kinetic energy (4.61) and gets: cv = N +

cv =

N 1−

Var(E k ) NT 2

.

(4.73)

Figure 4.18 shows cv after (4.73) from the same simulation as used for Figure 4.17. The data is rather noisy. A maximum can nevertheless be clearly recognized at T ≈ 0.45, as well as the correct asymptotic behavior of cv . For small T the equipartition theorem provides E = 2NT (two degrees of freedom from translation and another two from oscillation around rest position) and therefore cv = 2N. For a large T one expects ‘almost free’ particles (perfect gas) with cv = N.

94 | 4 Ordinary differential equations I 10000

8000

cv

6000

4000 2N 2000 N 0 0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

T Fig. 4.18: Specific heat from (4.73). For T → 0 one expects c v = 2N (elastic solid), for T → ∞ c v = N (perfect gas). Simulation with N = 1600.

4.5.6 Problems 1. Thermalization A method to keep the temperature of a many-particle system constant at Ts is to introduce friction terms into the equation of motion as done in (4.66): d2tt r ⃗i = F⃗ i − γ(T, Ts ) v⃗ i where γ(T, Ts ) = γ0 (1 −

(4.74)

Ts ) . T

For the interaction-free case F⃗ i = 0, derive from (4.74) a first-order ODE for T(t). Solve this equation and show that T approaches Ts exponentially in time with exp(−2γ0 t). 2. 2D-equilibrium configurations An (infinitely extended) 2D-many-particle system at equilibrium forms a periodic lattice. Consider the two configurations: square and hexagonal (Figure 4.19). Let the two-particle interaction be given by a Lennard-Jones potential as in (4.59). Compute the equilibrium distances r S and r H and consider (i) Only nearest neighbors (blue), (ii) Also next nearest neighbors (red), (iii) Also next-next nearest neighbors (green). Which of the two configurations is the more stable one?

4.6 Chaos |

95

Fig. 4.19: Two possible periodic configurations in 2D.

4.6 Chaos As it already discussed in Chapter 3 but in even more pronounced examples in the two previous sections of the present chapter, chaotic behavior is more the rule than the exception if the phase space possesses enough dimensions. Chaotic trajectories may appear both in conservative (Section 3.3, 4.4.3, 4.4.4) and in dissipative systems (Section 3.2.7, 4.5). For Hamiltonian systems, initial conditions normally play a decisive role. For closely neighbored initial values, qualitatively different behaviors very often emerge, such as periodic, quasiperiodic, or chaotic; see for instance Figure 4.13. For a dissipative system, typically the initial conditions play a minor role and are ‘forgotten’ after a certain, by friction effects defined, relaxation time (there are exceptions: a trajectory starting, e.g., close to a stable fixed point or limit cycle, will stay close and stationary or periodic behavior will persist. However, this does not exclude more involved or even chaotic behavior for trajectories starting in another region of state space).

4.6.1 Harmonically driven pendulum We continue working with low-dimensional systems (N > 2) having the form dy i = f i (y1 , y2 , . . . y N ) , dt

y i (0) = a i ,

i = 1...N ,

(4.75)

where friction is included (dissipative systems, no energy conservation). Let us begin with the harmonically driven nonlinear mathematical pendulum in 2D phase space φ̈ + α φ̇ + Ω20 sin φ = A cos ωt .

(4.76)

96 | 4 Ordinary differential equations I

This is a nonautonomous problem with explicit time dependence. It can be transformed to an autonomous one by introducing an additional dependent variable y1 = φ, y2 = φ,̇ y3 = ωt having now the form (4.75) and N = 3: ẏ 1 = y2 ẏ 2 = −α y2 − Ω20 sin y1 + A cos y3

(4.77)

ẏ 3 = ω . The corresponding RK4-subroutine pendulum_deq reads: SUBROUTINE pendulum_deq(rhside,y,t) REAL, DIMENSION(3) :: rhside,y COMMON /PARAM/ alpha,omega0,a,omega rhside(1)=y(2) rhside(2)=-alpha*y(2)-omega0**2*SIN(y(1))+a*COS(y(3)) rhside(3)=omega END Clearly the last equation in (4.77) can be immediately integrated and one may just as well (and with less effort) work with the nonautonomous original system: rhside(1)=y(2) rhside(2)=-alpha*y(2)-omega0**2*SIN(y(1))+a*COS(omega*t) However, the following considerations are valid only for autonomous systems and we shall resort to (4.77).

4

4

2

2

0

0

–2

–2

–4

–4 –2

0

2

–2

0

2

Fig. 4.20: Chaos for the driven pendulum. Left: phase space, y 2 over y 1 ; right: Poincaré section, intersection points with the planes y 3 = 3π/2 + 2nπ. A = 1, ω = 0.8, Ω0 = 1, α = 0.1.

4.6 Chaos

| 97

Fig. 4.21: Near a separatrix, tiny effects due to the driving force may turn into qualitatively different behavior. Solid: without external driving; dashed: possible trajectories with external force.

In the left frame of Figure 4.20 a chaotic trajectory is shown in phase space (y1 , y2 ). The existence of a separatrix of the nondriven system is an important though not necessary ingredient for the emergence of chaos. If the trajectory comes close to the separatrix, a suitable phase and amplitude of the driving force can push the phase point to the other side of the separatrix, and the motion then changes qualitatively, here from oscillation to rotation (Figure 4.21).

4.6.2 Poincaré section and bifurcation diagrams To visualize three-dimensional trajectories, certain projections are needed. In this way Figure 4.20 shows a projection onto the y1 -y2 -plane. Another possibility is to prepare a stroboscopic snapshot, for instance synchronized with the driving frequency ω in the case of the pendulum. As already discussed in Section 3.3.3, a Poincaré section results. In the right frame of Figure 4.20 the y1 -y2 -values are plotted when the driving force undergoes a zero-crossing with a positive slope, that is for ωt = 3π/2 + 2nπ. If the Poincaré section for t → ∞ consists of a finite number of separated points, the trajectory is periodic; a dense line is assigned to quasiperiodicity (cut through a torus) and filled areas with fractal gaps can be interpreted as chaotic behavior. A bifurcation diagram is obtained if some control parameter is changed and the value for a dependent variable is plotted at certain consecutive times over this parameter. Figure 4.22 (bottom) shows such a diagram where the value of y1 has been plotted over the strength of the driving force, again for fixed times ωt = 3π/2 + 2nπ.

98 | 4 Ordinary differential equations I

0.1 0 –0.1 –0.2 –0.3 0.9

1

1.1

1.2

1.3

1.4

0.9

1

1.1

1.2

1.3

1.4

2.5 2 1.5 1 0.5 0

Fig. 4.22: Top: the two nontrivial Lyapunov exponents for the driven pendulum with friction. Parameters same as in Figure 4.20. Bottom: bifurcation diagram for the driven pendulum, ordinate: |y 1 | at ωt = 3π/2 + 2nπ, abscissa: A.

4.6.3 Lyapunov exponents Let y⃗ (0) (t) be a numerically determined solution of (4.75). We ask for its stability by performing a linear stability analysis: ⃗ = y⃗ (0) (t) + u(t) ⃗ . y(t) Contrary to the linearization around a fixed point, the Jacobi matrix depends now on time ⃗ d u(t) ⃗ (4.78) = L(t) u(t) dt with 󵄨 ∂f i 󵄨󵄨󵄨 󵄨 . L ij (t) = ∂y j 󵄨󵄨󵄨󵄨y⃗ (0) (t) Due to the linear character of (4.78) one may assume that for large times, exponential behavior emerges ⃗ |u(t)| ∼ h(t)e σt , t → ∞ with a bounded h(t). Then the sign of σ σ = lim

t→∞

1 ⃗ ln |u(t)| t

(4.79)

4.6 Chaos | 99

classifies the stability of y⃗ (0) . For σ > 0 arbitrarily small deviations will grow exponentially over the course of time, the trajectory is unstable, and chaotic behavior is encountered. The σ defined as (4.79) is called the largest Lyapunov exponent. How can we compute σ? One nearby possibility is to integrate (4.78) numerically and evaluate (4.79) for large t. However, due to the exponential increase of |u|⃗ for a positive σ this is not practical since a numerical overflow would soon occur. 4.6.3.1 Determination of the largest Lyapunov exponent To construct a method that works we begin with the time-evolution operator Q. We ⃗ as may then formally write u(t) ⃗ ⃗ = Q(t, 0) u(0) . u(t)

(4.80)

⃗ ⃗ As a matter of fact, u(t), and therefore σ, depends on u(0). Thus one finds as many different Lyapunov exponents as the number of possible linearly independent initial conditions, namely N, the dimension of the state space. Often it is sufficient to know the largest σ because it distinguishes between chaotic and regular dynamics. Since Q(t, t0 ) = Q(t, t1 ) Q(t1 , t0 ) ,

t > t1 > t0

we may write (4.80) as a product ⃗ ⃗ = Q(t, t − ∆T) Q(t − ∆T, t − 2∆T) . . .Q(∆T, 0) u(0) . u(t) Introducing the abbreviations Q k ≡ Q(k∆T, (k − 1)∆T) ,

⃗ u⃗ k ≡ u(k∆T) ,

k = 0, 1, 2. . .

we obtain u⃗ k = Q k Q k−1 . . .Q1 u⃗ 0

(4.81)

where k = t/∆T. Taking a small enough ∆T avoids an overflow at a single step u⃗ k = Q k u⃗ k−1 . After each step one may normalize according to û k =

u⃗ k , dk

d k = |u⃗ k |

and obtains Q1 û 0 = d0 d1 Q k Q k−1 . . . ⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟ Q2 û 1 = d0 d1 . . .d k û k . u⃗ k = d0 Q k Q k−1 . . . Q 2 ⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟ = û 1 d1

= û 2 d2

From here one receives k

|u⃗ k | = ∏ dℓ ℓ=0

100 | 4 Ordinary differential equations I

d1 d4

d2 û1 d0

û0

û2

û3 d3

t = Tv

t = Tv + ΔT

t = Tv + 2ΔT

û4 y(0) (t)

t = Tv + 3ΔT

Fig. 4.23: Numerical computation of the largest Lyapunov exponent. After a constant time interval the absolute value of the deviation d k is determined. Then the deviation is normalized to one without changing its direction.

and finds by substitution in (4.79) σ = lim

k→∞

k 1 1 k ∑ ln dℓ , ln (∏ dℓ ) = lim k∆T k→∞ k∆T ℓ=0 ℓ=0

(4.82)

the Lyapunov exponent to the initial condition û 0 . The following algorithm illustrated in Figure 4.23 has to be implemented: 1. After a certain relaxation time T v compute the reference trajectory from (4.75). 2. Take an arbitrary û 0 , d0 = 1 at time t = T v , and set ℓ = 1. 3. Integrate (4.75), and in the meantime (4.78), over the interval ∆T. ⃗ + ℓ∆T)|. Normalize u⃗ to û ℓ and store dℓ . 4. Determine dℓ = |u(t 5. ℓ := ℓ + 1 , t = t + ∆T. 6. If ℓ < k go to 3. 7. Compute σ according to (4.82). If we denote ln dℓ as a local Lyapunov exponent, (4.82) is the average value of these local exponents. The limit k → ∞ has to be substituted by a large k. But what does large mean? We define the Lyapunov exponent computed after k iterations: σk =

1 k ∑ ln dℓ k∆T ℓ=0

and find for the next iteration σ k+1 =

k 1 k ( ∑ ln dℓ + ln d k+1 ) = σ k + ∆σ (k + 1)∆T ℓ=0 k+1

and with k ≫ 1 ∆σ = σ k+1 − σ k ≈

1 ln d k+1 . k∆T

(4.83)

4.6 Chaos

| 101

Obviously, ∆σ ∼ 1/k converges with k → ∞. The sum in (4.82) can be truncated if the error |∆σ| reaches a fixed lower bound. Alternatively the relative error 󵄨󵄨 ∆σ 󵄨󵄨 󵄨󵄨 󵄨󵄨 󵄨󵄨 󵄨󵄨 < ϵrel 󵄨󵄨 σ k 󵄨󵄨 can be used as a truncation criterion. An N-dimensional system possesses N Lyapunov exponents, also called Lyapunov spectra. The above described method provides the largest one. This is easy to ⃗ for t → ∞ in the base v̂ k , built by the understand if we decompose the deviation u(t) ̂ N different u 0 : ⃗ = c1 v̂ 1 eσ1 t + . . .c N v̂ N eσ N t , u(t) where σ k denotes now the ordered Lyapunov spectrum according to σ1 ≥ σ2 ⋅ ⋅ ⋅ ≥ σ N . ⃗ The constants c k are determined by the initial value u(0). As time goes on u⃗ will orient parallel to v̂ 1 , independently from the initial conditions. However, if c1 would exactly vanish one would expect u⃗ parallel to v̂2 , etc. In this way one could at least in principle obtain the complete spectrum, what, however, cannot work in practice. Numerically there is always a tiny component in the direction v⃗ 1 that increases exponentially fast and dominates the deviations after a certain time ∼ 1/σ 1 . We shall see below how the complete spectrum can be found anyway. But first let us prove an important theorem. 4.6.3.2 Theorem: one Lyapunov exponent vanishes for all trajectories that do not terminate at a fixed point We further assume bounded systems with ⃗ |y(t)| ≤ D1 ,

̇⃗ ⃗ |y(t)| = |f ⃗(y(t))| ≤ D2 ,

Di > 0

and show the simple proof of the theorem. Differentiating (4.75) yields N

ÿ i = ∑ j

N ∂f i ẏ j = ∑ L ij ẏ j , ∂y j j

from where it follows that with u⃗ = ẏ⃗ the time derivative of each solution of (4.75) also provides a solution of the linear system (4.78). In particular this is valid for the reference trajectory y⃗ (0) (t) itself, i.e., the deviation points always tangential to the trajectory (Figure 4.24). This special (marginal) Lyapunov exponent holds with (4.79) as σ m = lim

t→∞

1 1 ln |ẏ⃗ (0) (t)| ≤ lim ln |D2 | = 0 , t→∞ t t

and therefore σm ≤ 0 .

(4.84)

102 | 4 Ordinary differential equations I

û1 û0

û4

û2 û3

y(0) (t)

Fig. 4.24: A deviation pointing in the direction of the reference trajectory has a vanishing Lyapunov exponent if the trajectory does not terminate at a fixed point.

Assuming on the other hand σ m < 0, due to |ẏ⃗ (0) (t)| ∼ eσ m t , one arrives at

t→∞

|ẏ⃗ (0) (t)| = 0 ,

t→∞

and the trajectory would end at a fixed point. Therefore, if the trajectory does not terminate at a fixed point as assumed, the only possibility remaining is σm = 0 .

4.6.3.3 Higher order Lyapunov exponents How can we compute the complete Lyapunov spectrum? To this end we introduce the Lyapunov exponents of the order p; the quantity defined with (4.79) is then the Lyapunov exponent of the first order corresponding to the average rate of contraction of a vector (one-dimensional). Accordingly, the exponent of the order p stands for the mean rate of contraction of the volume of a p-dimensional parallelepiped: 1 ln |V p (t)| . t→∞ t

σ (p) = lim

(4.85)

Considering a parallelepiped moving along trajectories and spanned by the base v⃗ k , one can show that σ (p) = σ 1 + σ 2 + . . . σ p holds. If all N orders of the Lyapunov exponent are known, we can compute the exponents of the first order according to σ 1 = σ (1) σ 2 = σ (2) − σ (1) σN = σ

(N)

−σ

(N−1)

(4.86) .

4.6.3.4 Classification If all first-order exponents are known, the reference trajectory y⃗ (0) (t) can be classified (Table 4.1). In particular, a positive Lyapunov exponent denotes the divergence of the

4.6 Chaos |

103

Tab. 4.1: Classification of trajectories of an autonomous, dissipative system in three-dimensional state space with the help of the signs of their three Lyapunov exponents. σ1

σ2

σ3

− 0 0 +

− − 0 0

− − − −

trajectory terminates at a fixed point stable limit cycle (periodic dynamics) stable torus (quasiperiodic dynamics) strange attractor (chaotic dynamics)

trajectories. If, however, the dynamics is bounded on a finite region in state space, contractions must also be present and at least one of the Lyapunov exponents must be negative. An N-dimensional volume element V N (t) whose corners move along trajectories in state space is contracted in the temporal average if the system is dissipative; see Section 3.2.3. For the sake of simplicity let us first assume div f ⃗ = c with c < 0 and c as constant. From (3.23) follows V N (t) = V N (0) e ct and with (4.85), N

σ (N) = ∑ σ k = c = div f ⃗ = Tr L .

(4.87)

k=1

The sum over all Lyapunov exponents corresponds to the divergence of f (or the trace of the Jacobi matrix L) and is negative for a dissipative system. Hence it follows also that at least one Lyapunov exponent must be less than zero. Thus, for a three-dimensional state space all possible combinations are listed in Table 4.1. But in general, div f ⃗ is not a constant. Then one can still compute the average and σ

(N)

t

N

1 = ∑ σ k = lim ∫ dt div f ⃗ t→∞ t k=1

(4.88)

0

holds, an expression being also negative for dissipative systems. For the example of the damped pendulum one obtains 3

σ (3) = ∑ σ k = −α . k=1

For Hamiltonian systems the equations of motions have canonical form (see Section 3.3) and div f ⃗ = 0 results. The sum over all Lyapunov exponents vanishes. 4.6.3.5 Numerical determination of all Lyapunov exponents Next we present a method for the computation of the complete spectrum σ n and follow mostly [7]. Starting with a p-dimensional parallelepiped, due to the exponentially different growth rates of the different directions, all the spanning vectors will point soon

104 | 4 Ordinary differential equations I

in more or less one direction, namely in that assigned to the largest Lyapunov exponent. This can be avoided if the spanning vectors are orthogonalized repeatedly after certain time intervals, applying a Gram–Schmidt procedure for instance. We demonstrate the algorithm for the three-dimensional state space, N = 3: 1. Select three orthonormal unit vectors û 1 , û 2 , û 3 , (û i ⋅ û j ) = δ ij , t = 0, ℓ = 0. 2. Integrate (4.78) from t through t + ∆T: w⃗ i = Q(t + ∆T, t) û i 3.

(p)

Compute the volumes Vℓ , which are spanned by w⃗ 1 . . . w⃗ p : (p)

Vℓ

= √| det V|

with the help of the p × p matrix N

(i)

(j)

V ij = ∑ w n w n n (i)

and w k as the kth component of the vector w⃗ i . 4. Using a Schmidt–Gram method, determine the next orthogonal set û i in such a way that the first p vectors û i span the subspace defined by w⃗ 1 till w⃗ p : û 1 =

w⃗ 1 , |w⃗ 1 |

u⃗ 2 = w⃗ 2 − c12 û 1 ,

û 2 =

u⃗ 3 = w⃗ 3 − c13 û 1 − c23 û 2 , 5. 6. 7.

u⃗ 2 , |u⃗ 2 | û 3 =

c12 = û 1 ⋅ w⃗ 2 , u⃗ 3 , |u⃗ 3 |

c13 = û 1 ⋅ w⃗ 3 ,

c23 = û 2 ⋅ w⃗ 3

t := t + ∆T, ℓ := ℓ + 1 If ℓ < k go to 2. After sufficiently many steps (large k) the Lyapunov exponent of the order p can be computed: 1 k (p) σ (p) = ∑ ln Vℓ k∆T ℓ=1 and from there according to (4.86) finally the whole spectrum σ 1 . . .σ N .

The subroutine dlyap_exp implementing the steps explained above is described in the Appendix B. 4.6.3.6 Example: the driven pendulum We demonstrate the method for the driven pendulum (4.77). The Jacobi matrix in (4.78) reads 0 1 0 (4.89) L(t) = (−Ω20 cos y1(0) (t) −α −A sin y3(0) (t)) . 0 0 0

4.6 Chaos

| 105

Due to the last equation (4.77) there are no fixed points and at least one Lyapunov exponent must vanish. Because of the simple structure of (4.89) we can make some analytic conclusions. From the third line it follows immediately u̇ 3 = 0 ,

u 3 = const. (0)

(0)

For small A we may assume self-consistently |y1 |, |y2 | ∼ A and linearize (0)

cos y1 ≈ 1 .

(4.90)

In particular for u 3 = 0 the two equations for u 1 , u 2 are equivalent to the damped pendulum having the solutions α |(u 1 , u 2 )| ∼ h i (t) exp (− ) , 2

i = 1, 2

where h i (t) is an oscillating bounded function. With (4.79) one finds the two Lyapunov exponents σ 2 = σ 3 = −α/2 . Putting u 3 = 1, the two equations for u 1 , u 2 correspond to the driven harmonic oscillator. The long-time behavior turns out to be y1 ∼ sin(ωt + β) , providing σ 1 = 0. Therefore, for small A the solution y⃗ (0) (t) is a stable limit cycle (Table 4.1).

0 –0.05 –0.1 0.45

0.5

0.55

0.6

0.65

0.7

0.75

0.8

0.45

0.5

0.55

0.6

0.65

0.7

0.75

0.8

2.5 2 1.5 1 0.5 0

Fig. 4.25: Lyapunov exponents and bifurcation diagram for the driven pendulum with friction; parameters as in Figure 4.20.

106 | 4 Ordinary differential equations I

For larger A the linearization (4.90) is invalid, the limit cycle becomes unstable, and eventually chaotic trajectories emerge. Figure 4.25 shows numerical results for smaller A, while Figure 4.22 shows those for larger values. Code. In the code published on the website, instead of the original three-dimensional autonomous system (4.77), the equivalent two-dimensional nonautonomous system ẏ 1 = y2 ẏ 2 = −α y2 − Ω20 sin y1 + A cos ωt

(4.91)

is used. The Jacobi matrix then reduces to L(t) = (

0 (0) −Ω20 cos y1 (t)

1 ) −α

(4.92)

and only two Lyapunov exponents exist. The one vanishing for the 3D-system does not occur (the theorem Section 4.6.3.2 is only valid for autonomous systems). 4.6.3.7 Lyapunov time For dynamical systems originating from physical problems, the initial conditions are only known at a finite accuracy ∆ϵ(0). However, if the dynamics is chaotic, small errors may exponentially grow in time and the largest Lyapunov exponent σ 1 constitutes a measure for their growth rate (dimension 1/time). If σ 1 is known, a time t∗ can be estimated after the initial error has exceeded a certain size L, typically the circumference of the attractor in state space. Then the initial conditions will no longer play any role and the albeit fully deterministic theory (the differential equations (4.75)) does not allow for any forecasting for times beyond t∗ . With L = ∆ϵ(t∗ ) = ∆ϵ(0) e σ1 t one finds t∗ =

∗

1 L ln ( ) σ1 ∆ϵ(0)

for the so-called Lyapunov time. Depending on atmospheric conditions, for the weather forecast, t∗ lies in the region between only a few hours up to a maximum of two weeks. Studying the Centennial Calendar or similar products is therefore proven useless.

4.6.4 Fractal dimension Initiated by the Euclidean notion of dimension and analogue to the concept already discussed in Section 2.4.1, we may assign a dimension to an object in state space. A fixed point has the dimension d = 0, a limit cycle d = 1, a torus d = 2, etc. As it

4.6 Chaos |

107

follows from the No-Intersection theorem, a chaotic attractor cannot be completely fit into a plane and therefore must have a dimension larger than 2. On the other hand it normally does not fill the whole state space leading for N = 3 to a fractal dimension 2 0. (6.133)

By the help of an appropriate scaling of t, x, u, transform the Fisher–Kolmogorov equation to the normal form ∂ t u(x, t) = ∂2xx u(x, t) + u(x, t) − u 2 (x, t) .

(6.134)

6.4 Nonlinear PDEs |

191

Fig. 6.27: Numerical solution of the real Ginzburg–Landau equation (6.126) obtained with the scheme (6.131) for ε = 0.1, time series.

2. 3.

Show with a linear stability analysis that the fixed point (stationary solution) u (0) = 0 is unstable, and that the fixed point u (0) = 1 is stable. Monotonic solutions that connect the two fixed points u = u(x − vt) = u(ξ) ,

u(ξ → −∞) = 1 ,

u(ξ → ∞) = 0 ,

v>0

(6.135)

are denoted as fronts or kinks. Therefore the front moves with a certain velocity v towards the fixed point u = 0, i.e., to the right-hand side. Find a lower limit for v.

192 | 6 Partial differential equations I, basics

4. Show that for a certain v and κ u(x − vt) =

5.

6.

1 (1 +

2 Ce−κ(x−vt))

,

C>0

(6.136)

is a solution of (6.134). Determine v and κ. What is the meaning of the (arbitrary but positive) constant C? Release the restriction u ≥ 0. With the help of the analogy to a one-dimensional motion of a point mass in a potential, show that a stationary, localized solution of (6.134) exists. Determine this solution by integrating the law of energy conservation of the mass point. Perform a linear stability analysis u(x, t) = u 0 (x) + w(x) exp(λt) about the solution u 0 (x) found in 5. and derive a stationary Schrödinger equation for the disturbances w(x) of the form (−d2xx + V(x)) w(x) = −λw(x) .

(6.137)

Find the ground state by expanding V(x) up to the order x2 . What can you say about the stability of u 0 (x)? And to code: 1. Verify the results of 1–4 solving (6.134) numerically with an FTCS method and the appropriate boundary conditions u(x = 0) = 1, u(x = L) = 0. Assume a large enough L. 2. Solve the Schrödinger equation (6.137) writing for instance a MATLAB code. Analyze its spectrum and its eigenfunctions.

Bibliography [1] A. P. S. Selvadurai, Partial differential equations in mechanics, Vol. 1, 2, Springer (2010). [2] J. D. Jackson, Classical Electrodynamics, Wiley & Sons (1998). [3] L. D. Landau and E. M. Lifshitz, Fluid Mechanics, Vol. 6, Butterworth–Heinemann, 2nd edn. (1987). [4] FORTRAN codes at http://www.degruyter.com/books/978-3-11-051513-8 [5] W.Guo, G. Labrosse and R.Narayanan, The Application of the Chebyshev-Spectral Method in Transport Phenomena, Springer (2012). [6] T. I. Zohdi, A Finite Element Primer for Beginners: The Basics, Springer (2015). [7] L. D. Landau, On the Theory of Phase Transitions, Zh. Eksp. Teor. Fiz. 7, (1937). [8] D. J. Higham and N. J. Higham, MATLAB Guide, SIAM (2005).

7 Partial differential equations II, applications Now we shall apply the methods and schemes learned in the last chapter to problems from several fields of physics. Let us start with some simple quantum mechanical examples.

7.1 Quantum mechanics in one dimension 7.1.1 Stationary two-particle equation We first consider two interacting particles in one spatial dimension. The particles, each with mass m, are located at x1 , x2 and connected with a spring having the spring constant −α. The stationary Schrödinger equation thus reads Ĥ Ψ(x1 , x2 ) = E Ψ(x1 , x2 ) , where Ψ is the two-particle wave function and the Hamiltonian is given as ℏ2 ∂2 1 ∂2 Ĥ = − ( 2 + 2 ) − α (x1 − x2 )2 . 2m ∂x1 ∂x2 2

(7.1)

We shall also look at the case of a repelling interaction α > 0. In addition the particles are trapped in a potential well with length L and infinitely high walls, Ψ(0, x2 ) = Ψ(L, x2 ) = Ψ(x1 , 0) = Ψ(x1 , L) = 0 . With the scaling L π4 ℏ2 π2 ℏ2 ̃ ̃ E α , E = x̃ i , α = π mL4 2mL2 the stationary Schrödinger equation takes the dimensionless form xi =

(

∂2 ∂2 ̃ Ψ =0. + 2 + α̃ (x̃ 1 − x̃ 2 )2 + E) 2 ∂ x̃ 1 ∂ x̃ 2

(7.2)

(7.3)

In the following we leave the tildes. For the decoupled case α = 0 the exact solutions simply read (N → ∞) Ψ(x1 , x2 ) =

2 N ∑ cℓm sin ℓx1 sin mx2 π ℓ,m=1

(7.4)

where the coefficients cℓm are determined from the initial condition. For the energy, one finds the levels (7.5) E = Eℓm = ℓ2 + m2 , which are two-fold degenerated if ℓ ≠ m, reflecting the symmetry x1 ↔ x2 . The decoupled problem is similar to the single particle Schrödinger equation in a twodimensional potential well. https://doi.org/10.1515/9783110515145-007

194 | 7 Partial differential equations II, applications Inserting (7.4) in (7.3) with α ≠ 0 and multiplying with 2 sin kx1 sin nx2 π we obtain after integrating over x1 , x2 the linear system ∑ M knℓm cℓm = E c kn

(7.6)

ℓ,m

with the matrix elements ππ

M knℓm = (ℓ2 + m2 ) δℓk δ mn −

4α ∫∫ dx1 dx2 (x1 − x2 )2 sin kx1 sin nx2 sin ℓx1 sin mx2 π2 00

32αkℓmn −1 + (−1)k+ℓ + (−1)m+n − (−1)k+ℓ+m+n { { − , { { π2 (ℓ2 − k 2 )2 (m2 − n2 )2 { { { { { { 4αmn (1 + (−1)m+n ) { { , − { { (m2 − n2 )2 ={ { { 4αkℓ (1 + (−1)k+ℓ ) { { { , − { { (k 2 − ℓ2 )2 { { { { { 3(n2 + k 2 ) − π2 k 2 n2 { α + k 2 + n2 , { 6k 2 n2

k ≠ ℓ , m ≠ n k = ℓ , m ≠ n k ≠ ℓ , m = n k=ℓ, m=n. (7.7)

As happens quite often in quantum mechanics, one is confronted with the task to diagonalize a large matrix. To convert M into a two-dimensional matrix, one can confine each pairs of indices to one: (k, n) → k 󸀠 = k + (n − 1)N

(7.8)

and the same for (ℓ, m). N is the number of modes included in (7.4). The larger N, the better spatial resolution of the wave functions is achieved. However, M has N 2 × N 2 elements and the numerical effort grows fast with N 4 . Figure 7.1 shows the lower energy levels depending on α for N = 10. Note that the degeneracy vanishes due to the coupling. On the other hand, for large α nearly degenerated states occur again where a symmetric and an antisymmetric state belong to (almost) the same energy value. Figure 7.2 depicts contour lines of the 12 lowest states for α = 10. The probability densities are largest near the corners of the line x1 + x2 = L, corresponding to the furthest possible separation of the two particles. In Figure 7.3 we show the states for attractive interaction, α = −10. Here, the probability density is maximal along the line x1 = x2 , hence the particles prefer to be close to each other.

7.1 Quantum mechanics in one dimension | 195

20

10

Ψ22

Ψ00 E

0 Ψ12 + Ψ21

Ψ12 – Ψ21

Ψ21

Ψ12

–10

–20 –20

–10

0

10

20

α Fig. 7.1: Energy levels as a function of α. The states Ψ ℓm , Ψ mℓ , degenerated for α = 0, split up. Inset: by linear combination of two degenerated states (bottom), new states can be constructed (top).

Fig. 7.2: The 12 lowest states in the x1 , x2 -plane, from top left to bottom right, α = 10. The states (1,2), (3,4) (5,6) are almost degenerated, compare Figure 7.1.

196 | 7 Partial differential equations II, applications

Fig. 7.3: The lowest states for α = −10. Here, degeneracy is completely removed.

7.1.2 Time-dependent Schrödinger equation The time-dependent one-particle Schrödinger equation already appeared in Chapter 5. We wish to continue our study again in one spatial dimension. Scaling of x as in (7.2) leads to its dimensionless form −

̃ ∂2 Ψ(x,̃ t)̃ ̃ x,̃ t)̃ Ψ(x,̃ t)̃ = i ∂Ψ(x,̃ t) , + U( 2 ∂ x̃ ∂t̃

(7.9)

where in addition time has been scaled with t̃ =

π2 ℏ t 2mL2

and

2mL2 Ũ = 2 2 U . π ℏ In what follows we leave all tildes again. 7.1.2.1 The infinitely high potential well Again we consider a potential well with infinitely high walls {0 , U(x) = { ∞, {

0 0. Since the two diffusion constants D i are surely positive, this can only be valid for all k if b 1 b 2 < 0 , und a2 < 0 holds. Now consider a1 as a control parameter that can be increased from outside by varying e.g., temperature or pressure. The mode becoming unstable at first then has a wave number k = ±k c that minimizes (7.81), (a1 (k c ) = a1c ). The zero of the derivative of (7.81) with respect to k yields k 2c =

a2 b1 b2 1 a1 a2 + √− = ( + ) . D2 D1 D2 2 D1 D2

(7.82)

For a1 > a1c the fixed point w0i becomes unstable and the mode (7.78) with k = k c grows exponentially in time. This spatially periodic structure is nowadays called Turing instability or Turing structure. 7.4.1.2 Hopf instability To compute the critical point we simply assumed λ = 0 in (7.80). But there is another way for an instability to occur: λ may be complex valued and the mode oscillates in time. Then one has either a damped (old, stable state) or an exponentially increasing oscillation, depending on the sign of the real part of the eigenvalue λ. Thus the critical point is now determined by Re(λ) = 0 or λ = iω

(7.83)

228 | 7 Partial differential equations II, applications

with a real valued frequency ω. Inserting (7.83) into (7.80) and separating real and imaginary parts one obtains the two equations ω2 = (D1 k 2 − a1 )(D2 k 2 − a2 ) − b 1 b 2 0 = (D1 + D2 )k 2 − a1 − a2 .

(7.84a) (7.84b)

Equation (7.84b) determines the critical point depending on the wave number: a1 = (D1 + D2 )k 2 − a2 .

(7.85)

Obviously the mode with k = 0 becomes unstable at first as soon as a1 ≥ −a2 . Hence the oscillatory mode comes with a long-wave, even homogeneous, spatial structure, corresponding to kc = 0 . (7.86) As already noted in Section 3.2, instabilities belonging to a pair of conjugate complex eigenvalues are also denoted as Hopf instabilities, and the frequency at the critical point is named Hopf frequency. It follows from (7.84a) with k = 0: ω c = √a1 a2 − b 1 b 2 = √−a21 − b 1 b 2 .

(7.87)

To obtain a real valued frequency, b 1 b 2 < −a21 must hold. Let b 1 < 0 and b 2 > 0. Then the substance w1 creates via the coupling b 2 the substance w2 , while w2 diminishes w1 via b 1 . This is why (7.74) is also denoted as an activator-inhibitor system with w1 as the concentration of the activator and w2 as that of the inhibitor. Temporal oscillations of a spatially uniform state were first experimentally observed by Boris Belousov in 1950 and are now named Belousov–Zhabotinskii reactions. 7.4.1.3 Codimension-two But what happens now if one of the control parameters, say a1 , is increased over the threshold? The uniform stationary state becomes unstable, but how so exactly is determined by the other parameters. Depending on the values of a1 , b 1 , and b 2 as well as on the diffusion constants, the homogeneous state will give way either to a time periodic (Hopf) or to a space periodic (Turing) structure. From mathematics, the notion codimension is familiar. The codimension of an object is defined as the difference of the dimension of the room in which the object lives and the dimension of the object itself. A plane in the three-dimensional space possesses the codimension 3 − 2 = 1. Instability conditions like (7.81) or (7.85) determine the value of one parameter related to the others. They define an n −1-dimensional subspace of the n-dimensional parameter space and therefore have the codimension one.

7.4 Pattern formation out of equilibrium | 229

Fig. 7.20: Sketch of Turing and Hopf instability for the two-equation system (7.76). If increasing a1 and a2 < a2CD2 , the Turing instability comes first (a), if a2 > a2CD2 the Hopf instability occurs directly above threshold (b). For (c), a2 = a2CD2 .

But now we can adjust the parameters in such a way that the Turing and Hopf modes are unstable at the same a1 . This additionally restricts the parameter setting into an n − 2-dimensional subspace. A ‘double’ instability of this kind is named codimensiontwo instability (CD2); see also Figure 7.20. Inserting the critical wave number (7.82) into (7.81), one finds the smallest necessary value for a1 to get a Turing instability: a1T =

D1 D1 a2 + 2√−b 1 b 2 . D2 D2

(7.88)

In the same way one obtains for the oscillatory instability from (7.85) with (7.86) a1H = −a2 .

(7.89)

Equating the two expressions finally leads to a condition for a2 : a2CD2 = −2

√−b 1 b 2 D1 D2 , D1 + D2

a1CD2 = −a2CD2 .

(7.90)

This value is denoted as codimension-two-point. Obviously, for a2 > a2CD2 the Hopf mode becomes unstable first; for a2 < a2CD2 the Turing mode is observed at threshold. For a2 ≈ a2CD2 one expects the simultaneous emergence of both modes and a particularly rich spatio-temporal behavior of the solutions of the fully nonlinear equations. 7.4.1.4 The Brusselator During the 60s, a model based on a hypothetical chemical reaction scheme was devised by Prigogine¹⁵ and coworkers at the University of Brussels. The model was moti-

15 Ilya Prigogine, Russian–Belgian physical chemist, 1917–2003.

230 | 7 Partial differential equations II, applications

vated by the chemical oscillations discovered about ten years before by Belousov¹⁶ and consisted in its original form of two coupled ODEs (K = 2). Later, diffusion terms were added and spatio-temporal pattern formation was examined extensively. The system has the form of (7.74) and may show, depending on parameters, either Hopf or Turing instabilities. (A) Equations and linear analysis. For the Brusselator one finds f1 = A − (B + 1) w1 + w21 w2 ,

f2 = B w1 − w21 w2 ,

(7.91)

and the only fixed point is given by w01 = A ,

w02 = B/A .

Since there are only two free parameters (A, B), the four coefficients a i , b i of the Jacobian (7.77) are not independent from each other and read b1 = A2 ,

a1 = B − 1 ,

a2 = −A2 ,

b 2 = −B .

(7.92)

If A is considered to be fixed and B is defined as a variable control parameter, the relations (7.81, 7.85) can be solved for B: (T)

Bc = (H)

Bc

A2 + (D1 A2 + D2 )k 2 + D1 D2 k 4 D2 k2

=(D1 + D2 )k 2 + A2 + 1 .

(7.93)

From (7.82) we obtain for the Turing mode the critical wave number k 2c =

A √D1 D2

.

(7.94)

The conditions (7.88, 7.89) turn into 2

B = T

(T) B c (k c )

D1 = (A√ + 1) , D2

(H)

B H = B c (0) = A2 + 1 ,

(7.95)

from which one finds the codimension-two point by equating B T = B H : A CD2 =

2√D1 D2 , D2 − D1

B CD2 =

(D1 + D2 )2 . D2 − D1

(7.96)

Figure 7.21 shows the parameter plane for D1 = 1/2, D2 = 1. Depending on A, both instability types are possible.

16 Boris P. Belousov, Russian chemist, 1893–1970.

7.4 Pattern formation out of equilibrium | 231

Fig. 7.21: Parameter plane for the Brusselator with D2 = 2D1 . On the left-hand side of the dashed line the homogeneous state becomes oscillatory unstable; on the right-hand side one observes Turing patterns.

(B) Semi-implicit pseudospectral method. Next we wish to integrate numerically the fully nonlinear system applying a semi-implicit pseudospectral scheme as already done in Section 7.3.4.3. Chemical reactions are often performed on thin gels and a twodimensional representation is natural. It is of further advantage to transform the basic system to the deviations u i from the fixed point u 1 = w1 − A ,

u 2 = w2 − B/A

as in (7.76). For u i = u i (x, y, t) one derives the system ∂ t u 1 = D1 ∆u 1 + (B − 1) u 1 + A2 u 2 + f(u 1 , u 2 )

(7.97a)

2

∂ t u 2 = D2 ∆u 2 − B u 1 − A u 2 − f(u 1 , u 2 )

(7.97b)

with the nonlinearity f(u 1 , u 2 ) = (

B + u 2 ) u 21 + 2A u 1 u 2 . A

(7.98)

Similar to (7.69), the semi-implicit formulation leads to an inhomogeneous system of equations 1 1 (7.99) M k u⃗ k (t + ∆t) = u⃗ k (t) + f k (u 1 (t), u 2 (t))( ) −1 ∆t with u⃗ k = (u k1 , u k2 ) and u k , f k as the Fourier transform of u and f . Since the third dimension is absent, M k is here a simple 2 × 2-matrix Mk = (

1 ∆t

+ D1 k2 + 1 − B B

−A2 1 ∆t

+ D2 k2 + A2

) ,

(7.100)

which can be inverted explicitly: M −1 k =

1 2 2 1 ∆t + D 2 k + A ( Det M k −B

A2 1 ∆t

+ D1 k2 + 1 − B

) .

232 | 7 Partial differential equations II, applications t = 200

t = 200

Fig. 7.22: Depending on the value of A, for the plane (2D) Brusselator either (asymptotically stationary) Turing structures (left, A = 0.8), or spatially large-scaled temporally oscillating Hopf patterns (right, A = 0.35) emerge. For D1 /D2 = 0.1 it follows that A CD2 ≈ 0.7.

t = 15

t = 26

t = 34

t = 50

Fig. 7.23: Spreading fronts can be seen for other parameters in the oscillatory region. They are similar to the chemical nonequilibrium patterns observed experimentally in the Belousov–Zhabotinsky reaction. Time series for D1 /D2 = 0.5, A = 0.2, B = 1.2.

7.4 Pattern formation out of equilibrium | 233

(C) Results. We begin with D1 = 0.1, D2 = 1. For A above the codimension-twopoint A CD2 ≈ 0.7 we obtain Turing patterns mostly in the form of more or less regular hexagons, including some point and line defects. Below the CD2-point, large-scaled regions in space oscillating with about the Hopf frequency can be seen, in agreement with the linear theory, as in Figure 7.22. Closer to the CD2-point a complex interchange of Turing and Hopf modes is observed; compare also Figure 1.3 from the introduction. Qualitatively different structures, namely the experimentally known target or spiral patterns, are found for less differing diffusion constants, for instance D1 = 0.5, D2 = 1, and much farther in the Hopf region, A = 0.2, B = 1.2; see Figure 7.23. We conclude this section presenting a MATLAB code for the semi-implicit pseudospectral scheme based on the built-in 2D fast-Fourier-transform FFT2. clear n=512; % % parameter input d=input('d1/d2? '); acd2=2.*sqrt(d)/(1.-d) a=input('value for a? bh=1.+a^2 % bt=(1.+a*sqrt(d))^2 % b=input('value for b?

mesh points

% cd-2 point '); threshold Hopf mode threshold Turing mode '); ba=b/a;

xkc=sqrt(a/sqrt(d)); dx=2.*pi/xkc/15.; dx2=dx^2; dt=0.1; % step sizes dkx2=(2.*pi/n/dx)^2; for k=1:n; % coefficients of M for j=1:n; xk2=((k-n/2)^2+(j-n/2)^2)*dkx2; det=(1./dt+d*xk2+1.-b)*(1./dt+xk2+a^2)+b*a^2; m22(k,j)=(1./dt+d*xk2+1.-b)/det; m11(k,j)=(1./dt+xk2+a^2)/det; m12(k,j)=a^2/det; m21(k,j)=-b/det; end end m11=ifftshift(m11); m12=ifftshift(m12); m22=ifftshift(m22); m21=ifftshift(m21); u1=0.2*(rand(n)-.5); u2=0.2*(rand(n)-.5);

% initial conditions

234 | 7 Partial differential equations II, applications

t=0; tend=1000; n=0; for t=0:dt:tend; n=n+1; fn=(u1.^2).*(ba+u2)+2.*a*u1.*u2; rhs1=u1/dt+fn; rhs2=u2/dt-fn;

% time loop % nonlinearity % rhs in real space

fs1=fft2(rhs1); fs2=fft2(rhs2); % Fourier transform rhs1=m11.*fs1+m12.*fs2; % comp. in Fourier space rhs2=m21.*fs1+m22.*fs2; u1=real(ifft2(rhs1)); u2=real(ifft2(rhs2)); % back to real space if(n==100) n=0; contour(u1); drawnow; t end

% plotting

end

7.4.2 Swift–Hohenberg equation 7.4.2.1 Motivation Turing structures originate from instabilities where the fastest growing, most unstable modes have a finite wave number k c . If we evaluate for instance (7.80) for the Brusselator, a growth rate λ(k 2 ) is obtained in the super-critical domain B > B c as shown in Figure 7.24. The function λ(k 2 ) seems to be rather involved, including a square root. If

rg

λ

en

be

0.01

oh −H

2.5

3

3.5

r sse lato

–0.02

Bru

–0.01

k2 2

Sw ift

0

k2c

–0.03

–0.04

Fig. 7.24: The basic idea behind the Swift–Hohenberg equation is to expand the eigenvalue λ(k2 ) of a Turing instability, here for the Brusselator, in the neighborhood of the critical point, leading to the polynomial (7.101). Parameters: D1 = 0.1, D2 = 1, A = 0.8, B c = 1.57, B = 1.01B c .

7.4 Pattern formation out of equilibrium | 235

we restrict ourselves only to the region of the unstable (linearly growing) modes, we may expand λ at B c , k 2c , according to: λ ≈ a(B − B c ) − b(k 2 − k 2c )2 with a=

∂λ 󵄨󵄨󵄨󵄨 , 󵄨 ∂B 󵄨󵄨󵄨B c ,k2c

b=−

(7.101)

󵄨 1 ∂2 λ 󵄨󵄨󵄨 󵄨󵄨 . 2 2 2 ∂(k ) 󵄨󵄨󵄨B c ,k2c

In position space one may simply replace k 2 with −∆ and obtain a linear PDE whose solution reflects the instability behavior (7.101): ∂ t Ψ(x, y, t) = a(B − B c )Ψ(x, y, t) − b(∆ + k 2c )2 Ψ(x, y, t) .

(7.102)

After appropriate scaling of time and space, (7.102) takes the form ∂ t Ψ(x, y, t) = εΨ(x, y, t) − (∆ + 1)2 Ψ(x, y, t)

(7.103)

with the sole control parameter ε ∼ B − B c . To account for a nonlinear saturation of the linearly unstable modes one adds to (7.103) the most simple terms that guarantee global stability (bounded solutions): ∂ t Ψ(x, y, t) = εΨ(x, y, t) − (∆ + 1)2 Ψ(x, y, t) + A Ψ 2 (x, y, t) − Ψ 3 (x, y, t) .

(7.104)

Therewith we have derived the Swift–Hohenberg equation [9], nowadays a standard model of pattern formation. It describes the spatio-temporal pattern evolution close to the critical point of a (monotonic) Turing instability¹⁷. A similar ‘normal form’ can be derived for an oscillatory (Hopf) bifurcation and yields a complex Ginzburg–Landau equation; see Problems. The Swift–Hohenberg equation can be formulated as a gradient system: ∂t Ψ = −

δL[Ψ] . δΨ

(7.105)

Here, δ/δΨ denotes the functional derivative and L is the functional 2A 3 1 4 1 Ψ − Ψ } , L[Ψ] = − ∫ dx dy {εΨ 2 − [(∆ + 1)Ψ]2 + 2 3 2

(7.106)

also called the Lyapunov functional. As one easily shows it follows from (7.105) that dt L ≤ 0

(7.107)

and L decreases monotonically as long as a stationary state (minimum or secondary minimum of L) is reached asymptotically. The property (7.107) can be used as a check as well as a stop criterion for any numerical method.

17 The original work was for A = 0. The extension with a square term was first proposed and studied by H. Haken [6].

236 | 7 Partial differential equations II, applications

7.4.2.2 Algorithm For periodic boundary conditions it is easy to construct a pseudospectral scheme as already done in Section 7.4.1.4. Since we now have only one equation to iterate, the matrix inversion turns into a simple division in Fourier space: Ψ k (t + ∆t) =

f k (Ψ(t)) 1/∆t − ε + (1 − k 2 )2

(7.108)

with the Fourier transform f k of f(Ψ) =

Ψ + A Ψ2 − Ψ3 . ∆t

Another possibility is the discretization in position space with finite differences. Let N be the number of mesh points in each direction (for simplicity we assume a square geometry). We organize the node values in a one-dimensional vector Φ k , k = 1 . . . N 2 as in (7.23) and formulate a large (N 2 ) system of equations N2

∑ M ij Φ j (t + ∆t) = j

Φ i (t) + A Φ2i (t) − Φ3i (t) ∆t

where M has to be inverted. Of course M is a rather large matrix with (N 2 )2 elements but most of them are zero and special routines for sparse matrices can apply. MATLAB is suited to treat sparse problems. A simple code could look like that: clear; n=200; dx=0.5; dx2=dx^2; dt=3; tend=1000; eps=0.1; a=0.; lap=-gallery('poisson',n); lap2=lap*lap; % Laplace and biharmonic FD in 2D pot=speye(n^2)+lap/dx2; % matrix needed for Lyapunov potential m=(1/dt-eps+1)*speye(n^2)+2*lap/dx2+lap2/dx2^2; % matrix M psi=zeros(n); phi=rand(1,n^2)-0.5; % random initial values k=0; for t=0:dt:tend % time loop till tend k=k+1; phi=(phi/dt-phi.^3+a*phi.^2)/m; % matrix inversion lyap=eps*phi.^2-.5*phi.^4+2./3.*a*phi.^3-(phi*pot).^2; % potential lyap_plot(k)=-0.5*dx2*sum(lyap); t_plot(k)=t; % plot values

7.4 Pattern formation out of equilibrium | 237

5

200

0

150

–5

100

–10

50

–15

0

50

100

150

200

50

100

150

200

Fig. 7.25: Numerical solution of the Swift–Hohenberg equation at t = 200 for ε = 0.1, A = 0 (right) and Lyapunov functional (7.106) over t (left), computed on a 200 × 200 grid and for a random-dot initial condition. 10

200

0

150

–10

100

–20

50

–30

0

50

100

150

200

50

100

150

200

Fig. 7.26: The same as in Figure 7.25 but for A = 0.5.

if(mod(k,10)==0) % plott each 10 time steps psi(:)=phi; % compute 2D Psi from Phi subplot(1,2,1); plot(t_plot,lyap_plot) subplot(1,2,2); contourf(psi) drawnow; t end end It produces output as shown in Figures 7.25, 7.26. After t = 200 the depicted patterns are not completely stationary but still evolving towards their fixed points. Performing a linear analysis around a perfectly regular stripe solution, it turns out that at a critical value √3ε Ac = 2

238 | 7 Partial differential equations II, applications

stripes become unstable and give way to hexagons already found for the Brusselator. If the symmetry Ψ → −Ψ is broken, i.e., A ≠ 0, one always finds hexagons directly above threshold ε = 0 since A c = 0. Thus, hexagons are the typical (generic) structure for a Turing instability with no further symmetry restrictions. As a secondary instability for larger ε stripes, or for more involved nonlinearities, squares may also occur. Boundary conditions. The Swift–Hohenberg equation is of the fourth order in the spatial derivatives and thus needs two independent boundary conditions along each side. Taking the gallery MATLAB call, the matrix ‘lap’ contains the 2D Laplace FDrepresentation of Figure 6.8a, cut at the boundaries. This corresponds to Ψ = 0 along all sides. The biharmonic operator ‘lap2’ in the form of Figure 6.9 is obtained by squaring the matrix ‘lap.’ Then a second boundary condition is automatically built in of the form of ∆Ψ = 0, corresponding together with the first condition to ∂ ξξ Ψ = 0 where ξ stands for the variable perpendicular to the boundary. Very often, the Swift–Hohenberg equation is solved with the boundary conditions Ψ =0,

n̂ ⋅ ∇Ψ = 0

(7.109)

with n̂ normal to the boundaries. Conditions of the form (7.109) can be realized by modifying the diagonal elements of the matrix ‘lap2’ that belong to the points next to the four sides, according to for j=1:n:n^2 j1=j+n-1; lap2(j,j)=lap2(j,j)+2; lap2(j1,j1)=lap2(j1,j1)+2; end for j=1:n j1=j-n^2-n-1 lap2(j,j)=lap2(j,j)+2; lap2(j1,j1)=lap2(j1,j1)+2; end This code section must be included in the code on page 236 before computing the matrix ‘m’ in line 5. The modification of the diagonal elements can be understood by assuming a ‘virtual line’ outside of the integration domain, as explained in more detail in Section 6.2.2.1.

7.4.3 Problems Similar to the derivation in Section 7.4.2.1, a model equation valid for the weakly nonlinear case close to an oscillatory (Hopf) instability can be found. This equation is also named the complex Ginzburg–Landau equation and reads: ∂ t Ψ(x, t) = [ε + iω0 + (1 + ic1 ) ∂2xx ] Ψ(x, t) − (1 + ic3 )|Ψ(x, t)|2 Ψ(x, t)

(7.110)

239

Bibliography |

where ε, ω0 , c i ϵ R and Ψ ϵ C. Assume periodic boundary conditions in the domain 0 ≤ x ≤ L: Ψ(x = 0, t) = Ψ(x = L, t) . 1.

Without loss of generality, the frequency ω0 can be put to zero. Why? Show that for ε > 0 an exact solution of (7.110) reads Ψ = A exp(iΩt)

2.

with A, Ω ϵ R. Determine A and Ω. Examine the stability of the solution (7.111). Show that it becomes unstable if c1 c3 < −1 .

3.

(7.111)

(7.112)

Solve (7.110) for ε = 0.1 numerically using an FTCS scheme in the stable regime, i.e., for c1 = 1/2 , c3 = −1/2 , and in the unstable regime, i.e., with c1 = 1/2 ,

4.

c3 = −4 .

Take L = 800, ε = 0.1 and the step sizes ∆x = 0.8 und ∆t = 0.05. As the initial condition, assume a random dot distribution of Ψ(x, 0) in the interval 0.9A . . . 1.1A. Represent the solution graphically in a space-time diagram. If (7.112) is fulfilled, (7.110) possesses spatio-temporal solutions that are chaotic. Examine them by determining their largest Lyapunov exponent Λ for different values of c3 and fixed c1 = 1/2. Plot Λ as a function of c3 .

Bibliography [1] L. D. Landau and E. M. Lifshitz, Fluid Mechanics, Vol. 6, Butterworth–Heinemann, 2nd edn. (1987). [2] C. A. J. Fletcher, Computational Techniques for Fluid Dynamics 2, Springer (2013). [3] FORTRAN codes at http://www.degruyter.com/books/978-3-11-051513-8 [4] C. H. Bruneau, The 2D lid-driven cavity problem revisited, M. Saad, Computers & Fluids 35, 326 (2006). [5] P. N. Swarztrauber and R. A. Valent, FFTPACK5 – a FORTRAN library of fast Fourier transforms, http://www.netlib.org/fftpack, February 11, 2018. [6] H. Haken, Synergetics: Introduction and Advanced Topics, Springer (2012). [7] A. M. Turing: The chemical basis of morphogenesis, Phil. Trans. R. Soc. London B 237, 37 (1952). [8] J. D.Murray, Mathematical Biology, Springer (2008). [9] J. Swift, P. C. Hohenberg, Hydrodynamic fluctuations at the convective instability, Phys. Rev. A15, 319 (1977).

8 Monte Carlo methods (MC) The terms ‘Monte Carlo methods’ or ‘Monte Carlo simulations’ refer to a variety of numerical methods mainly based on random numbers. On the one hand these can be problems arising from physics where fluctuations or other random processes play a central role, for example thermal motion in statistical physics, trajectories of particles in disordered media, or equilibrium phase transitions. On the other hand, mathematical problems like the approximate computation of high-dimensional integrals or the determination of extreme values in a high-dimensional variable space can be often only tackled by MC methods.

8.1 Random numbers and distributions 8.1.1 Random number generator Since random numbers play a crucial role, we wish to start with their properties and their realization on a computer. Because all processes on computers are in principle deterministic, one should better talk about ‘pseudorandom numbers’. A pseudorandom number generator is implemented in FORTRAN95. The statement CALL RANDOM_NUMBER(x) assigns to the real variable x an equally distributed random number in the interval 0≤x 2.

248 | 8 Monte Carlo methods (MC)

8.2.2.1 Example As an example we wish to compute the integral 1

1

J n = ∫⋅ ⋅ ⋅ ∫ dx1 dx2 . . . dx n (x1 + x2 + . . . x n )2 . 0

(8.23)

0

It can be solved exactly as n n2 + . (8.24) 12 4 Evaluation using the rectangle rule with N mesh points in each dimension yields J nA =

N

N

2

J nR ≈ (∆x)n ∑ ⋅ ⋅ ⋅ ∑ (x i1 + x i2 + . . . x i n ) , i 1 =1

x i = ∆x (i − 1) .

i n =1

Monte Carlo integration provides J nM ≈

1 NT 2 ∑ (ξ1 + ξ2 + . . . ξ n ) N T i =1 1

with the equally distributed, uncorrelated random numbers 0 ≤ ξi ≤ 1 . To compare the results we take the same number of summands for each method, leading to N = (N T )1/n . The relative error in percent is given as 󵄨󵄨 R,M 󵄨 󵄨󵄨J n − J nA 󵄨󵄨󵄨 󵄨; Er(%) = 100 ⋅ 󵄨 J nA see Figure 8.3. 8.2.2.2 Simple sampling and importance sampling To determine integrals of the form (8.16) with (8.17), we assumed equally distributed sampling points x i ; this is also called simple sampling. If f(x) varies strongly, the deviation from the mean according to (8.21), and hence the error, can become large. On the other hand we can rewrite (8.16) for an arbitrary, nonzero function ρ(x) as: b

b

J = ∫ dx f(x) = ∫ dx [ a

a

f(x) ] ρ(x) . ρ(x)

(8.25)

If ρ(x) is the probability density of x, we can interpret (8.25) as the mean ⟨f/ρ⟩ and compute the variance b

Var(f/ρ) = ∫ dx [ a

f(x) 2 ] ρ(x) − J 2 . ρ(x)

(8.26)

8.2 Monte Carlo integration

|

249

Fig. 8.3: Numerical computation of the integral (8.23) for n = 1, 4, 7 room dimensions. Monte Carlo integration compared to rectangle rule. Relative error in percentage over the total number of mesh points N T = N n . For the rectangle rule N = 3 . . . 20 have been used in each dimension. The MC error −1/2 goes with N T (straight line). For n = 1 the rectangle rule is superior, whereas for n = 4, MC is already more precise.

Taking for ρ an appropriate PDF, (8.26) can be minimized. Vanishing functional derivative δ/δρ of b

F[ρ] = Var(f/ρ) + λ ∫ dx ρ a

with λ as Lagrange parameter originating from the constraint (normalization) b

∫ dx ρ(x) = 1

(8.27)

a

yields −f 2 /ρ 2 + λ = 0 or ρ=

|f| , √λ

where λ is to be determined with (8.27). The error becomes smaller the more the shape of ρ resembles that of |f|. Adjusting the PDF according to the form of the integrand is named importance sampling, and can be readily extended to higher dimensions.

250 | 8 Monte Carlo methods (MC)

8.3 Applications from statistical physics Now let us turn to the other important field of Monte Carlo applications, namely to problems where randomness and fluctuations play an intrinsic role. To this end we first revisit many-particle systems. In Chapter 4 we examined such systems solving Newton’s equations of motion for certain interaction potentials. Applying now the methods of statistical physics, one considers an ensemble of macroscopically identical systems that differ microscopically, i.e., by different initial conditions of the particles. Averaging over the ensemble one can compute certain macroscopic values like density or temperature. In thermal equilibrium one may also consider snapshots of the same system at different times, making use of the identity of ensemble average and time average.

8.3.1 Two-dimensional classical gas 8.3.1.1 Hard spheres model We start with a two-dimensional gas of N hard spheres, or better hard discs, each having the diameter d0 . Let the particles be randomly distributed in a certain given volume V, say the unit square. We draw the positions r i⃗ from an equal distribution in the unit square r ⃗i = (x i , y i ) , x i ϵ [0..1] , y i ϵ [0..1] , i = 1 . . . N . However, the particles must not overlap. Thus we have to choose the positions fulfilling the constraints (8.28) d ij = √(x i − x j )2 + (y i − y j )2 ≥ d0 for each pair i ≠ j. The code section for the initialization could look as: PARAMETER (n=1000) REAL, DIMENSION(2,n) :: REAL, DIMENSION(2) ::

! number of particles x ! positions xn ! new positions

.... d0 = ... ! sphere diameter, minimal distance c draw n particle positions randomly DO i=1,n 11 CALL RANDOM_NUMBER(xn) ! random position DO j=1,i-1 IF(d(xn,x(:,j)).LT.d0) GOTO 11 ! distances ok? ENDDO x(:,i)=xn ! yes, accept position ENDDO .... The function d computes the Euclidean distance according to (8.28).

8.3 Applications from statistical physics

|

251

To create a new configuration one assigns fresh random coordinates x k , y k to an arbitrarily chosen particle k = 1 . . . N. If all constraints (8.28) are fulfilled the new position is accepted. Otherwise the coordinates are rejected and drawn again. Hence one produces successively different ensemble members in accordance with the macroscopic conditions V = const., N = const. Since there is no motion of particles, a mechanical kinetic energy is not defined. Nevertheless, the interaction between the particles can be easily considered. A hard core repulsion is already manifested in the restriction (8.28). How can we model an additional attractive force? Let the interaction for instance be expressed by an attractive Coulomb potential u(r i⃗ − r ⃗j ) = −

α , d ij

α>0.

Then one computes the total energy for each configuration according to N

N

U(r1⃗ , . . . r ⃗N ) = ∑ ∑ u(r ⃗i − r ⃗j ) .

(8.29)

i=1 j=i+1

One could be tempted to find the ‘ground state’ with the lowest energy just by testing as much as possible configurations. Assuming N = 1000, we had to vary 2000 variables (3000 in 3D) in the region [0, 1]. To detect a minimum in such a high-dimensional space seems to be rather hopeless. 8.3.1.2 Metropolis algorithm A quite elegant method to an approximate solution of the problem is provided by the Metropolis algorithm¹. In particular we shall here consider the canonical ensemble where volume, particle number, and temperature are prescribed. A central part of the statistical description is the partition sum [2] Z(N, V, T) = ∑ exp(−E i /k B T)

(8.30)

i

with Boltzmann’s constant k B and energy E i of the microstate i. The sum runs over all macroscopically possible microstates. If one deals with continuous variables as for the hard spheres gas model, the sum is converted into a 2N-dimensional integral according to Z(N, V, T) ∼ ∫ d2 r1⃗ ⋅ ⋅ ⋅ ∫ d2 r N⃗ exp(−U(r1⃗ . . . r ⃗N )/k B T) .

(8.31)

If the partition sum is known, the probability in finding a state with energy E i in the canonical ensemble is expressed by ρi =

1 exp(−E i /k B T) , Z

1 Devised by Metropolis, Rosenbluth, Rosenbluth, Teller and Teller in 1953 [1].

(8.32)

252 | 8 Monte Carlo methods (MC)

also denoted as Boltzmann distribution. With (8.32) we can compute mean values as ⟨Y⟩ = ∑ Y i ρ i ,

(8.33)

i

where again the sum runs over all states and has to be replaced by the appropriate integrals for the case of continuous variables. How can the sums or high-dimensional integrals for the computations of Z be evaluated numerically? It is possible to apply MC-integration. Normally (8.32) is concentrated in a very narrow region around its mean ⟨E⟩ = ∑ E i ρ i i

and simple sampling will not work effectively. A much better strategy is to draw only states with E i ≈ ⟨E⟩. Hence one varies the state variables not arbitrarily but rather in such a way that the system moves towards thermal equilibrium. Let X i be a set of variables describing the system in the state i, i.e., the position coordinates of the gas from Section 8.3.1.1. Now one seeks a sequence of states X0 → X1 → . . . X n that converges preferably quickly to equilibrium. Each state should be uniquely determined by its predecessor; such a sequence is denoted as a Markov chain. Let P(X i → X j ) be the transition probability from state i into state j. A stationary distribution ρ(X) is given if the condition ρ(X i )P(X i → X j ) = ρ(X j )P(X j → X i ) , known as detailed balance, holds. For ρ as (8.32) one obtains P(X i → X j ) = P(X j → X i )

ρ(X j ) = P(X j → X i ) exp(−∆E ji /k B T) ρ(X i )

(8.34)

with ∆E ji = E j − E i . If E i > E j , the transition i → j should always take place (approximation to the equilibrium): P(X i → X j ) = 1 if

∆E ji < 0 .

From (8.34) one finds for the reverse transition j → i P(X j → X i ) = exp(−∆E ij /k B T)

and

∆E ij > 0 .

8.3 Applications from statistical physics

| 253

Importance sampling with the Metropolis algorithm is built by the following steps: 1. Choose an initial configuration X0 . Compute E0 (X0 ). i = 1. 2. Construct the next configuration X i by modifying the position of an arbitrary particle. Compute E i (X i ) and ∆E i,i−1 3. If ∆E i,i−1 ≤ 0, accept the new configuration X i . i = i + 1. Go to 2. 4. If ∆E i,i−1 > 0, draw an equally distributed random number 0 ≤ r < 1. If r < exp(−∆E i,i−1 /k B T), accept the new configuration X i . i = i + 1. Go to 2. 5. If r > exp(−∆E i,i−1/k B T), restore the old configuration X i = X i−1 . Go to 2. We exemplify steps 1–5 with a code for MC simulation of the 2D gas with the interaction potential given in (8.29). For the complete listing see [3]. Step 1 Initial position is computed with program section on page 250. Computation of E0 : E0=0. DO i=1,n-1 DO j=i+1,n E0=E0-1./d(x(1:2,i),x(1:2,j)) ENDDO ENDDO Step 2 100

CALL RANDOM_NUMBER(rk) k=rk*FLOAT(n)+1 ! random particle k Ek=0. DO i=1,n IF(i.NE.k) Ek=Ek-1./d(x(1:2,i),x(1:2,k)) ! contribution of k to E0 ENDDO xn=x(1:2,k) ! store old config. CALL RANDOM_NUMBER(x(1:2,k)) ! new random position of k Ekn=0. DO i=1,n ! constraints fulfilled? (hard spheres)? IF(i.NE.k) THEN IF(d(x(1:2,K),x(1:2,i)).LT.dmin) THEN x(1:2,K)=xn GOTO 100 ! No, then draw again ENDIF Ekn=Ekn-1./d(x(1:2,k),x(1:2,i)) ! contribution of k to E0 ENDIF ENDDO dE=Ekn-Ek ! delta E

254 | 8 Monte Carlo methods (MC)

Fig. 8.4: (N = 500), configuration at T = 2000 (left frame) and drop formation at T = 20 (right).

Step 3 IF(dE.LT.0.) THEN E0=E0+dE ! new E0 GOTO 100 ! next configuration ENDIF Step 4 CALL RANDOM_NUMBER(r) IF(r.LT.EXP((-dE/t)) THEN E0=E0+dE ! new E0 GOTO 100 ! next configuration ENDIF Step 5 x(1:2,k)=xn ! restore old config, E0 unchanged GOTO 100 ! next draw Figure 8.4 presents two particle configurations for high and low T. Scaling of temperature and distances are such that α = k B = 1 and d0 = 0.01. Obviously, a circular assignment of the particles minimizes the energy for lower T and a single drop is formed. For T = 2000 one obtains a gaseous state. The transition becomes visible if a histogram of the particle distances is recorded as a function of temperature. From the histogram one finds the average particle distance as ⟨r⟩ = see Figure 8.5.

∑i r i H(r i ) ; ∑i H(r i )

8.3 Applications from statistical physics

|

255

0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 0

200

400

600

800

1000 1200 1400 1600 1800 2000

Fig. 8.5: Mean particle distance ⟨r⟩ over temperature for N = 500, d 0 = 0.01.

Fig. 8.6: Drop formation under gravity for g = 103 (left) and g = 104 (right) at T = 20.

The influence of gravitation in the negative y-direction can be readily included by adding the potential energy of the form N

Up = g ∑ yi

(8.35)

i

to (8.29). For g > 0 the particles ‘move’ downwards and eventually a drop having a certain contact angle is formed on the lower wall. For larger g the drop is squeezed and the contact angle becomes smaller; see Figure 8.6.

8.3.2 The Ising model Certain metals such as iron or nickel show spatial regions where spins orient themselves in a parallel fashion and create a permanent magnetic field, a property called

256 | 8 Monte Carlo methods (MC)

ferromagnetism. If the metal is heated above a critical temperature T c named Curie temperature, polarization and magnetic field vanish. Hence these metals undergo a phase transition from a magnetic to a nonmagnetic state or vice versa with temperature as the control parameter and magnetization as the order parameter. A simplified model describing ferromagnetism was devised by E. Ising² almost 100 years ago [4]. Thereby, N spins are located on a regular periodic lattice (in Ising’s original work on a one-dimensional chain) and each spin has two possible orientations, say up and down, denoted by ±1. If only the nearest neighbor interaction is taken into account, the energy is given by N

γ

N

E(s1 , . . . s N ) = −ε ∑ ∑ s i s j − H ∑ s i , i jϵNN(i)

(8.36)

i

where ε measures the coupling strength, NN(i) denote the indices of the nearest neighbors and H stands for an external, homogeneous magnetic field. The amount γ of nearest neighbors depends on the room dimension n as well as on the geometry of the periodic grid. One finds if n = 1, chain

2 { { { { { {4 { { { γ = {6 { { { { 6 { { { { {8

if n = 2, square lattice if n = 2, hexagonal lattice if n = 3, simple cubic lattice if n = 3, body-centered cubic lattice.

Taking (8.36) we compute the partition sum (we shall use from here the abbreviation β = 1/k B T) Z(H, T) = ∑ ∑ ⋅ ⋅ ⋅ ∑ exp(−βE(s1 . . . s N )) , (8.37) s1 s2

sN

where each sum runs only over the two values s i = −1, 1. Hence we have to evaluate 2N expressions in total. With the partition sum at hand, we can compute the free energy F(H, T) = −k B T ln Z(H, T)

(8.38)

and from there further important macroscopic quantities like the internal energy U(H, T) = −k B T 2

∂ F ( ) , ∂T k B T

(8.39)

∂U , ∂T

(8.40)

the specific heat C(H, T) = or the overall magnetization N

M(H, T) = ⟨∑ s i ⟩ = − i

2 Ernst Ising, German physicist, 1900–1998.

∂ F ( ) . ∂H k B T

(8.41)

8.3 Applications from statistical physics

| 257

8.3.2.1 Mean-field approximation A rather crude ‘solution’ to the problem is achieved applying the mean-field approximation. Although this method in its simplest form yields inaccurate or even wrong results, it shows at least qualitatively why a phase transition can take place and why the critical temperature depends both on coupling strength and room dimension. With (8.36) the energy of a single spin reads γ

E s = −ε s ∑ s j − Hs = −εγs s̄NN − Hs ,

(8.42)

jϵNN(s)

with s̄NN =

1 γ

γ

∑ sj

(8.43)

jϵNN(s)

as the mean of the neighbor spins. With the partition sum (two states, s = ±1) ̄ + βH) + exp(−βεγ s̄NN − βH) = 2 cosh(βεγ sNN ̄ + βH) Z s = exp(βεγ sNN the mean of s with given s̄NN s̄ =

1 +1 ̄ + βHs) = tanh(βεγ s̄NN + βH) ∑ s exp(βεγs sNN Z s s=−1

(8.44)

is computed. One now makes the strongly simplified assumption that the mean of the neighbors (8.43) is equal to the mean (8.44) and obtains the transcendental equation s̄ = tanh(βεγ s̄ + βH)

(8.45)

from which s̄ and therewith the magnetization can be determined. We start with the case H = 0 where the trivial solution s̄ = 0 exists. However, for βεγ > 1

(8.46)

this solution becomes unstable and two new real branches ±s̄1 appear (three intersections of tanh(βεγ s)̄ and s̄ exist). The two new branches are identified with the ferromagnetic phases. The Curie temperature is given from (8.46) k B T c = εγ . It is proportional to the coupling strength ε and depends on the number of neighbors and the room dimension. The magnetization is visualized easiest by solving (8.45) with respect to T εγ s̄ T= k B atanh(s)̄ and plotting T over s̄ ϵ [−1, 1], Figure 8.7 left. In the moment where an external field is applied, a preferred direction exists. Then, one of the two nontrivial states becomes

258 | 8 Monte Carlo methods (MC)

1

s̅

1

0.5

s̅

0.5

0

1

2

3

Tc

–3 –2 –1 0

4

–0.5

–0.5

–1

–1

1

2

3

H

Fig. 8.7: Left: magnetization for H = 0 over T . Right: magnetization over external field H for T = 2. The dashed line is computed from (8.45). It is unstable where its slope is negative and the true process follows the arrows. Scaling: kB = ε = 1 and γ = 4, T c = 4.

more stable, and the other can still be metastable for a certain range of H, while a hysteresis loop shows up: Figure 8.7, right frame. Based on the mean-field approach, magnetic phases should occur also in one dimension (spin chain) at finite T = 2ε/k B . However, in real life this is not the case, which we briefly demonstrate in the next section. 8.3.2.2 The one-dimensional Ising model Consider a chain of N spins having equal distances. The relation (8.36) turns into N−1

N

E = −2 ε ∑ s i s i+1 − H ∑ s i i

(8.47)

i

and the partition sum can be computed exactly. For the sake of simplicity we shall concentrate on the case H = 0 and substitute 2ε = ϵ. Using (8.47) one finds from (8.37) Z(T) = ∑ ∑ ⋅ ⋅ ⋅ ∑ exp(βϵs1 s2 ) exp(βϵs2 s3 ) . . . exp(βϵs N−1 s N ) . s1 s2

(8.48)

sN

The sum over s1 can be evaluated: ∑ exp(βϵs1 s2 ) = exp(βϵs2 ) + exp(−βϵs2 ) = 2 cosh(βϵs2 ) = 2 cosh(βϵ) s1

and shows to be independent on s2 . Next, the sum over s2 is executed in the same way and so on, and finally one arrives at Z(T) = 2N (cosh(βϵ))N−1 ≈ (2 cosh(βϵ))N .

(8.49)

From (8.39) the internal energy U(T) = −ϵN tanh (

ϵ ) kB T

(8.50)

8.3 Applications from statistical physics

259

|

1

0.5

0.4

M 0.5

0.3 C –2

0.2

0

–1

1 H

2

–0.5

0.1 0 0

1

2

3

4

–1

T

Fig. 8.8: Left: specific heat per spin for the one-dimensional Ising model (H = 0); right: magnetization per spin depending on an external magnetic field for T = 1 (solid) and T = 2 (dashed), kB = ϵ = 1.

and from (8.40) the specific heat C(T) =

ϵ2 N k B T 2 cosh(ϵ/k B T)

(8.51)

can be determined. The shape of C(T) could allude to a phase transition for a specific T c ≈ 1, Figure 8.8, left. However, this is not the case as we shall see if we compute the magnetization according to (8.41). To this aim one must know the partition sum for H ≠ 0. This can be achieved also exactly but the computation is more involved; for details we refer to the literature, e.g., [2]. Here, we only present the result (N ≫ 1): Z(H, T) = exp(Nβϵ) [(cosh(βϵ) + √sinh2 (βH) + exp(−4βϵ)]

N

.

(8.52)

We finally get M(H, T) =

N sinh(βH) √sinh2 (βH) + exp(−4βϵ)

and from there M(0, T) = 0 ,

for T > 0 .

No spontaneous magnetization occurs for finite temperature; see also Figure 8.8, right. Applying thermodynamics, this can be understood by an estimate of the free energy F = U − TS .

260 | 8 Monte Carlo methods (MC)

(a) Fig. 8.9: One-dimensional chain of spins, (a) magnetic ground state, (b) dislocation.

(b)

In a homogeneous magnetic phase all spins are aligned (Figure 8.9 (a)), the entropy is S = 0, and the internal energy reads U = −Nϵ. If, on the other hand, a dislocation as shown in Figure 8.9 (b) is included, the internal energy increases by 2ϵ, but in the meantime the entropy increases also by k B ln N since there exist N possibilities to place the dislocation. Thus, the free energy is in total changed by ∆F = 2ϵ − k B T ln N , which becomes negative for T > 2ϵ/(k B ln N). Since F decreases on the way to thermodynamic equilibrium and finally takes a minimum there, more and more dislocations are created by thermal fluctuations and the order vanishes. In the thermodynamic limit N → ∞ fluctuations at even infinitesimal temperature are large enough to destroy a one-dimensional magnetic structure independently from the coupling strength ϵ. Why then does the mean-field approximation in one dimension yield a value of T c = 2ε/k B ? If one assumes as in (8.45) for all spins the same mean value, the influence of fluctuations is considerably diminished, leading to larger values of T c in general. For only one spatial dimension, even a qualitatively wrong result can turn out. 8.3.2.3 The two-dimensional Ising model Coupling is stronger in two room dimensions and may finally lead to ordered states at finite T. A dislocation in 2D (a grain boundary) on an M × M lattice costs the internal energy 2Mϵ and thus changes the free energy by ∆F = 2Mϵ − k B T ln M , an expression that is nonnegative for finite T and large enough M. Onsager³ [5] was able to find an exact solution of the two-dimensional Ising model using transfer matrices. He derived for T c the relation Tc =

2ε ε . ≈ 2.269 k B asinh(1) kB

The derivation is however rather involved and lengthy and an exact solution in three dimensions is still absent.

3 Lars Onsager, Norwegian-American physicist, 1903–1976.

8.3 Applications from statistical physics

|

261

The Metropolis algorithm introduced in Section 8.3.1.2 can be readily applied on the Ising model. We shall use a square grid built up by N = M × M spins. The method can be extended in a straightforward manner to three dimensions. We refer to the procedure given on page 253. The initial state (step 1) may consist either of equally distributed (hot start-up) or of parallel orientated spins (cold startup). To compute E, (8.36) is evaluated with γ = 4. A new configuration is created by changing the direction of a randomly drawn spin. How can ∆E be computed if spin s k changes to −s k ? Let the initial energy be γ

E({s}) = −ε s k

∑ s j − H s k + rest ,

(8.53)

jϵNN(k)

and after changing s k γ

E({s󸀠 }) = ε s k

∑ s j + H s k + rest ,

(8.54)

jϵNN(k)

where ‘rest’ denotes all terms that do not depend on s k . Subtracting the two equations above yields (8.55) ∆E = E({s󸀠 }) − E({s}) = 2s k h k with the abbreviation

γ

hk = ε

∑ sj + H .

(8.56)

jϵNN(k)

In two dimensions the sum in (8.56) can only take one out of the five values −4,−2,0,2,4 and one finds ∆E = ±2(4ε + H, 2ε + H, H, −2ε + H, −4ε + H) . The Boltzmann factors are needed only for the cases ∆E > 0. Assuming |H| < 2ε and taking the scaling T=

ε ̃ T, kB

H = ε H̃

the five Boltzmann factors ̃ T)̃ exp(−4T)̃ , exp(±2H/

̃ T)̃ exp(−8T)̃ , exp(±2H/

̃ T)̃ exp(−2H/

occur. It suffices to compute the four different exponential functions for one time at the beginning of the program what increases its speed considerably. A code section containing steps 2–5 of page 253 could look like this (for the complete code see [3]):

262 | 8 Monte Carlo methods (MC)

ex(0)=0.5 ex(1)=EXP(-4./T) ! the four exp-functions ex(2)=EXP(-8./T) exh(1)=EXP(2.*H/T) exh(2)=EXP(-2.*H/T) .... c should the spin is(i,j) be changed? in=-is(i,j)*(is(i,j+1)+is(i,j-1)+is(i+1,j)+is(i-1,j)) ! sum over ! 4 neighbors IF(in.GT.0) THEN ! Delta E > 0, change! is(i,j)=-is(i,j) ELSE ! Delta E 0, draw a random number r out of [0,1]. If r < exp(−∆F/T), accept the new configuration; otherwise restore the old one. What is here the role of the ‘temperature’? Analogue to thermodynamics, the possibility to accept a Metropolis step with increasing F increases with T. Thus the system may leave a secondary minimum to reach eventually the main minimum. Too large temperatures however may yield rather noisy and not converging results and one has to play around a bit to find the appropriate T. Also a ‘time-dependent,’ slowly decreasing temperature could help in some cases. 8.4.1.2 Implementation with finite differences Approximating the derivatives and the integral in (8.60) with finite differences, the functional F turns into a function of N = M × M variables M 1 1 F(Ψ ij ) = ∑ [ (Ψ i+1,j − Ψ i−1,j )2 + (Ψ i,j+1 − Ψ i,j−1)2 − ρ ij Ψ ij ∆x2 ] . 8 8 ij

(8.63)

If only one certain Ψ mn is changed, ∆F can be effectively evaluated just as (8.53) by dividing (8.63) in terms which contain Ψ mn and the rest. After some manipulations one arrives at 1 2 1 F = Ψ mn − Ψ mn (Ψ m+2,n + Ψ m−2,n + Ψ m,n+2 + Ψ m,n−2 ) − ρ mn Ψ mn ∆x2 + rest . 2 4 ̃ Let Ψ mn be the new value at node m, n. Then the modification of F reads 1 ̃2 2 ) (Ψ mn − Ψ mn 2 1 + (Ψ̃ mn − Ψ mn ) (−ρ mn ∆x2 − Ψ m+1,n − Ψ m−1,n − Ψ m,n+1 − Ψ m,n−1 ) . 4 In the last formula we have halved the step size and replaced the nodes m, n + 2 with m, n + 1 and so on. ∆F =

8.4.1.3 Results Figure 8.13 shows a ‘development’ of the initial condition Ψ ij = 0 on a 100 × 100 grid with ∆x = 1. Two point sources ρ 20,20 = −1 ,

ρ 80,80 = 1

8.4 Differential equations derived from variational problems | 267

Fig. 8.13: Solution of the Poisson equation for two point charges applying the Metropolis algorithm. The snapshots are taken from left to right after 106 , 108 , 2 ⋅ 109 variations of the node values and for a temperature of T = 4 ⋅ 10−5 .

are given. At each step, the new node value is achieved by Ψ̃ mn = Ψ mn + αξ

(8.64)

with an equally distributed random variable ξ ϵ [−0.5, 0.5] and α = 0.1, while the temperature is fixed with T = 4 ⋅ 10−5 . The boundary values Ψ0,j , Ψ M+1,j , etc. are not changed and remain as zero (Dirichlet condition).

8.4.2 Swift–Hohenberg equation As demonstrated in Section 7.4.2, the Swift–Hohenberg equation may also be derived from a functional (7.106). Thus we can again use the Metropolis algorithm. Changing a node value Ψ mn now causes the change of the potential ∆L ε 1 10 4 2 2 = (Ψ̃ mn − Ψ mn )( − + − 2 + 4 ) 2 2 2 ∆x ∆x ∆x 8 2 + (Ψ̃ mn − Ψ mn ) ( ( 2 − ) (Ψ m+1,n + Ψ m−1,n + Ψ m,n+1 + Ψ m,n−1 ) ∆x ∆x4 2 + 4 (Ψ m+1,n+1 + Ψ m−1,n+1 + Ψ m+1,n−1 + Ψ m−1,n−1 ) ∆x 1 + 4 (Ψ m,n+2 + Ψ m,n−2 + Ψ m+2,n + Ψ m−2,n ) ) ∆x 1 1 4 3 3 4 − A(Ψ̃ mn − Ψ mn ) + (Ψ̃ mn − Ψ mn ). 3 4 The expression is more involved because (7.106) contains the Laplacian. Figure 8.14 shows two states on a 100 × 100 grid for different parameters, each obtained after 2 ⋅ 109 variations. The node modifications were done as in (8.64) but with α = 0.01, ∆x = 1/2, T = 2.5 ⋅ 10−4 . Figure 8.15 depicts the corresponding potentials. The temperature chosen in Figure 8.16 is obviously too high.

268 | 8 Monte Carlo methods (MC)

Fig. 8.14: Two solutions of the Swift–Hohenberg equation by variation of the Lyapunov potential (7.106) with the help of the Metropolis algorithm. Left frame: ε = 0.1, A = 0, right frame: ε = 0.1, A = 0.5. As in Section 7.4.2, one finds stripes for A = 0 and hexagons for A = 0.5. 0.8 0.6 0.4 0.2 0 –0.2 –0.4 –0.6 –0.8 200

600

1000

1400

1800

1 0.5 0 –0.5 –1 –1.5 –2 –2.5 –3 –3.5 200

600

1000

1400

1800

Fig. 8.15: Lyapunov functional over the number of variations (in millions) for the two parameter sets of Figure 8.14, T = 2.5 ⋅ 10−4 . 12.6 12.2 11.8 11.4 11 10.6

0

400

800

1200

1600

2000

Fig. 8.16: Lyapunov functional for ϵ = 0.1, A = 0, but now for T = 2.5 ⋅ 10−3 . At higher temperatures the fluctuations are clearly larger and it takes much longer until an equilibrium (stable) state is reached.

Bibliography

| 269

Bibliography [1] N. Metropolis, A. Rosenbluth, M. Rosenbluth, A. Teller and E. Teller, Equation of State Calculations by Fast Computing Machines, J. Chem. Phys. 21, 1087 (1953). [2] K. Huang, Statistical Mechanics, Wiley & Sons (1987). [3] FORTRAN codes at http://www.degruyter.com/books/978-3-11-051513-8 [4] E. Ising, Beitrag zur Theorie des Ferromagnetismus, Z. Phys, 31, 253 (1925). [5] L. Onsager, Crystal statistics I. A two dimensional model with order-disorder transition, Phys. Rev. 65, 117 (1944).

A Matrices and systems of linear equations In this appendix, we concentrate on methods for the solution to large linear systems of equations. These can be homogeneous

or inhomogeneous

A ⋅ x⃗ = 0

(A.1)

A ⋅ x⃗ = b⃗ ,

(A.2)

where x,⃗ b⃗ denote vectors in RN and A is an N × N square matrix. We begin with some properties and definitions of square matrices.

A.1 Real matrices Let A be a real-valued N × N matrix, A ij ∈ R.

A.1.1 Eigenvalues and eigenvectors Each vector v⃗ i in RN for which A ⋅ v⃗ i = λ i v⃗ i

with

v⃗ i , λ i ∈ C

(A.3)

holds is called the eigenvector of A and belongs to the eigenvalue λ i . The eigenvalues are either real valued or come as conjugate-complex pairs; the same is found for the eigenvectors.

A.1.2 Characteristic polynomial The polynomial of degree N P(λ) = det(A − λ 1) = (λ1 − λ)(λ2 − λ) . . . (λ N − λ)

(A.4)

is called the characteristic polynomial of A. Its N zeros λ i are the eigenvalues of A and are also denoted as the spectrum of A. Multiple zeros: Let λ i be a k i -fold zero of P, P(λ) = (λ1 − λ)k1 (λ2 − λ)k2 . . . (λ s − λ)k s . The eigenvalue λ i then has the algebraic multiplicity k i .

https://doi.org/10.1515/9783110515145-009

A.1 Real matrices | 271

A.1.3 Notations –

The transposed matrix A T is obtained by interchanging rows and columns of A: A Tij = A ji .

–

(A.5)

The inverse matrix A−1 is defined as A−1 ⋅ A = A ⋅ A−1 = 1 .

(A.6)

It is computed by solving the linear, inhomogeneous system of equations (A.6), in components: N

∑ A ij A−1 jk = δ ik .

(A.7)

j

– – –

For a symmetric matrix A = A T holds. For an orthogonal matrix A −1 = A T holds. Because of (A.7) all rows and all columns are pairwise orthogonal. A normal matrix commutes with its transposed, A ⋅ A T = A T ⋅ A. Symmetric and orthogonal matrices are normal.

A.1.4 Normal matrices Let A be a normal matrix. An orthogonal matrix T exists with T −1 ⋅ A ⋅ T = B

(A.8)

where B is diagonal, B ij = b i δ ij . A and B are called similar and have the same characteristic polynomial and thus the same spectrum λ i = b i . The eigenvectors of A (A T ) form the columns (rows) of the transformation matrix T (T −1 ). A.1.4.1 Symmetric matrices A symmetric N × N matrix has the following properties: 1. All N eigenvalues and eigenvectors are real valued. 2. Eigenvectors belonging to different eigenvalues are orthogonal. 3. If an eigenvalue has the algebraic multiplicity k (k-fold degenerated), it has k linearly independent eigenvectors that can be orthogonalized applying Gram– Schmidt orthogonalization. 4. From 2. and 3. follows: the N eigenvectors form a complete orthogonal system in RN , i.e., they span the complete RN . 5. The transformation matrix T in (A.8) is orthogonal.

272 | A Matrices and systems of linear equations

A.1.4.2 Orthogonal matrices The spectrum of an orthogonal matrix M is located on the unit circle in the complex plane: |λ j | = 1 , λ j ∈ C or λ j = exp(iφ j ) , Because of det M =

∏Ni

φj ∈ R .

λ i it immediately follows with A.1.1 det M = ±1 .

Example: the rotational matrix in two dimensions cos θ D=( sin θ

− sin θ ) cos θ

has the characteristic polynomial (cos θ − λ)2 + sin2 θ = 0 and the eigenvalues λ12 = exp(±iθ) .

A.2 Complex matrices Let A be a complex N × N matrix, A ij ∈ C.

A.2.1 Notations The adjoint matrix A+ is obtained by interchanging rows and columns and forming the conjugate-complex of each element A+ij = A∗ji .

(A.9)

For the inverse matrix A −1 (A.6) and (A.7) hold. Special matrices: – The self-adjoint matrix with A = A+ . Self-adjoint matrices are also named Hermitian. – The unitary matrix with A −1 = A + . – The normal matrix with A ⋅ A+ = A+ ⋅ A. – Self-adjoint and unitary matrices are also normal.

A.2 Complex matrices

| 273

A.2.2 Jordan canonical form Each arbitrary, complex or real-valued, N × N matrix A can be transformed by an invertible matrix C (linear transformation) into a matrix having Jordan canonical form, or for short, into a Jordan matrix: J 1 (λ1 ) 0 C −1 ⋅ A ⋅ C = J = ( 0 ... ( 0

0 J 2 (λ2 ) 0

0 0 J 3 (λ3 )

0 0 0

... ... ... )

(A.10)

J m (λ m ))

...

with the matrices (Jordan blocks) J i (λ i ) = λ i ,

(

λi 0

1 ) , λi

λi (0 0

1 λi 0

0 1) , λi

etc.

(A.11)

of length 1, – –

–

2,

3,

etc.

J i (λ i ) contributes to the spectrum of A with the sole (algebraic 1,2,3,. . . -fold) eigenvalue λ i . The geometric multiplicity of the eigenvalue λ i is equal to the number of the linearly independent eigenvectors belonging to λ i . It is thereby equal to the number of Jordan blocks with λ i in J. A normal matrix has the following properties: – Algebraic and geometric multiplicity are equal. – All Jordan blocks have the length one. – All normal matrices are diagonalizable.

Example: The matrix λ1 0 J=( 0 0

1 λ1 0 0

0 0 λ1 0

0 0 ) 0 λ2

is already in Jordan canonical form and has the characteristic polynomial P(λ) = (λ1 − λ)3 (λ2 − λ) . It follows for the eigenvalues

algebraic multiplicity geometric multiplicity

λ1

λ2

3 2

1 1

274 | A Matrices and systems of linear equations

Hence, J possesses only three eigenvectors, two belonging to λ1 and one to λ2 . Thus, the eigenvectors do not form a complete orthogonal system.

A.3 Inhomogeneous systems of linear equations We begin with N-dimensional systems having the form A ⋅ x⃗ = b⃗ .

(A.12)

A.3.1 Regular and singular system matrices The system (A.12) possesses a unique solution if the system matrix A is invertible. It reads (A.13) x⃗ = A−1 ⋅ b⃗ . Hence, det A ≠ 0 and A is called a regular matrix. Then the rank of A is rank(A) = N , and A is built up by N linearly independent rows (and columns). Conversely, if det A = 0 and rank(A) = N − K, there are K linearly dependent rows and A is named singular. The kernel of A ker(A) denotes the K-dimensional subspace of RN spanned by the vectors v⃗1 , . . . , v⃗ K for which A ⋅ v⃗ i = 0 ,

i = 1...K .

The kernel is also called the null space. For a singular A, depending on b⃗ there exists either no or infinitely many solutions to (A.12). There is no solution if b⃗ is partially or completely in ker(A): (a) no solution if (b) infinitely many solutions if

v⃗+i ⋅ b⃗ ≠ 0 v⃗+i ⋅ b⃗ = 0

for at least one i ≤ K for all i ≤ K .

Here, v⃗ +i denotes the eigenvectors of A+ (or the left-sided eigenvectors of A): (A+ − λ i 1) ⋅ v⃗+i = 0 , and λ1 , . . . λ K = 0, λ K+1 , . . . λ N ≠ 0.

A.3 Inhomogeneous systems of linear equations | 275

For case (b), the solutions are found by expanding x⃗ and b⃗ into the base v⃗ i : N

x⃗ = ∑ y i v⃗ i ,

N

N

i=1

i=K+1

b⃗ = ∑ β i v⃗ i = ∑ β i v⃗ i

i=1

with β i = v⃗ +i ⋅ b.⃗ From (A.12) we obtain: N

N

N

N

i=1

i=1

i=K+1

i=K+1

∑ y i A v⃗ i = ∑ y i λ i v⃗ i = ∑ y i λ i v⃗ i = ∑ β i v⃗ i and because the v⃗ i are linearly independent yi =

v⃗+i ⋅ b⃗ , λi

i = K +1...N

must hold. The other y i , i = 1 . . . K remain undetermined.

A.3.2 Fredholm alternative Hence, the solutions of (A.12) read either x⃗ = A −1 ⋅ b⃗ ,

if rank(A) = N

or, given that b⃗ ∈ ̸ ker(A), K v+i ⋅ b⃗ v⃗ i + ∑ y i v⃗ i , λi i=K+1 i=1 N

x⃗ = ∑

if rank(A) = N − K .

These two possibilities are named Fredholm alternative.

A.3.3 Regular matrices The matrix inversion to numerically solve (A.12) can be achieved by applying some standard routines provided e.g., by the LAPACK library. Depending on the form of A different routines are to be considered, a documentation is found for instance in¹ http://www.netlib.org/lapack/explore-html/files.html The different names of the routines are logically derived and composed from their functions, data types, etc., for details see http://www.netlib.org/lapack/individualroutines.html

1 Date: November 2017.

276 | A Matrices and systems of linear equations

To solve (A.12), the matrix A does not need to be inverted completely, it suffices to transform it into a triangular shape, applying Gauss elimination (for details see e.g., [1]). Once (A.12) is in triangular form A󸀠 ⋅ x⃗ = b⃗ 󸀠 with

A󸀠11 A󸀠 = (

. A󸀠22

. . A󸀠33

(A.14) . . . .

A󸀠1N . . ) ,

(A.15)

A󸀠NN

0

the system (A.14) is easily solved by backward substitution and we obtain x i :=

b 󸀠i − ∑Nk=i+1 A󸀠ik x k A󸀠ii

,

(A.16)

where the index i = N, N − 1, . . . 1 runs backwards. If the matrix in (A.15) is regular, det A 󸀠 = ∏Ni A󸀠ii ≠ 0 and as a consequence all A󸀠ii ≠ 0.

A.3.4 LU decomposition LU decomposition provides a direct, noniterative solution method of an inhomogeneous system of linear equations. The matrix A is written as the product of two matrices L⋅U =A (A.17) where L is a lower (only nonzero elements on the diagonal and below) and U an upper (nonzero elements on the diagonal and above) triangular matrix. Taking a 4×4 matrix as an example, we have 0 0 U11 U12 U13 L11 0 L21 L22 0 0 0 U22 U23 )⋅( ( L31 L32 L33 0 0 0 U33 0 0 0 L41 L42 L43 L44

U14 A11 U24 A21 )=( U34 A31 U44 A41

A12 A22 A32 A42

A13 A23 A33 A43

A14 A24 ) . A34 A44 (A.18)

Then (A.12) can be written as A ⋅ x⃗ = (L ⋅ U) ⋅ x⃗ = L ⋅ (U ⋅ x)⃗ = b⃗ .

(A.19)

y⃗ = U ⋅ x⃗

(A.20)

L ⋅ y⃗ = b⃗ ,

(A.21)

Introducing the vector we first solve

A.3 Inhomogeneous systems of linear equations | 277

which, because of the triangular form of L, is simple: y i :=

i−1 1 [ b i − ∑ L ij y j ] . L ii j=1 [ ]

(A.22)

Here we imply that the sum is zero for i = 1 and for y j occurring on the right-hand side the values computed in the same loop are used (forward substitution). As a second step, we solve U ⋅ x⃗ = y⃗ (A.23) by backward substitution just as in (A.14). The main task is to find the decomposition (A.17), or, in other words, to determine the matrices L and U. This is done by solving the inhomogeneous system K

∑ L ik U kj = A ij ,

i = 1...N ,

j = 1...N ,

(A.24)

k=1

where because of the special structure of L and U the sum runs only up to K = min(i, j) . These are N 2 equations for 2× N 2+N = N 2 + N unknowns and the system is underdetermined, meaning the LU decomposition is not unique. Thus, we can arbitrarily make the N elements in the diagonal of L 1: 2

L ii = 1 ,

i = 1...N .

(A.25)

A direct solution of (A.24) is given by the Crout algorithm [3]: 1. j = 1. 2. Compute the j coefficients i−1

U ij := A ij − ∑ L ik U kj ,

i = 1...j .

(A.26)

k=1

3.

Compute the N − j coefficients j−1

L ij := 4.

1 [A ij − ∑ L ik U kj ] , U jj k=1

i = j +1...N .

(A.27)

j := j + 1. If j ≤ N go to 2.

Following the algorithm step-by-step, it turns out that on the right-hand sides only elements are needed that have already been computed in prior steps.

278 | A Matrices and systems of linear equations

A.3.4.1 Pivoting A division occurs in (A.27), which may cause numerical problems if the corresponding U jj is small. One may show that Crout’s algorithm (as other elimination methods too) is unstable if U jj becomes small compared to other matrix elements. A solution is provided by reordering the system. One may interchange arbitrarily rows and columns but then has to swap also the elements of the vectors x⃗ and b⃗ accordingly. Finding an appropriate permutation that leads to a large-valued divisor (the pivot element in (A.27)) is called ‘pivoting.’ One distinguishes between full and partial pivoting. In the first case, rows and columns of A are interchanged pairwise, in the latter only the rows. How can we find the optimal pivot element? Normally it suffices to take that element with the largest absolute value. To compare elements in different rows, the matrix A should be scaled first. One can use the largest element in each row and scale this to one, i.e., one computes for each row i a scaling factor s−1 i = max |A ij | j

and then scales A󸀠ij = s i A ij ,

b 󸀠i = s i b i .

A.3.5 Thomas algorithm If the matrix A has many zeros, the methods can often be simplified considerably. The special case of a tridiagonal matrix (only nonzeros in the diagonal, and in one upper and lower diagonal) occurs often from finite differences discretization and can be solved quite effectively. Let A ⋅ x⃗ = y⃗ , with a tridiagonal A given as

( A=( (

b1 a2

c1 b2

0 c2 .

. ai

. bi

ci .

) ) . ) .

(0

. aN

(A.28)

bN )

In a first step, (A.28) is reduced to triangular form 1 ( A󸀠 = ( (

c󸀠1 1

0 c󸀠2 .

. 1

) ) , )

c󸀠i .

(0

. 1)

(A.29)

A.4 Homogeneous systems of linear equations | 279

which is achieved by substituting c󸀠1 =

c1 , b1

and c󸀠i :=

ci , b i − a i c󸀠i−1

y󸀠i :=

y󸀠1 =

y1 b1

y i − a i y󸀠i−1

b i − a i c󸀠i−1

(A.30)

i = 2...N .

(A.31)

i = N − 1, . . . 1 .

(A.32)

,

The system A󸀠 x⃗ = y⃗ 󸀠 can then be readily solved by (A.16): x N = y󸀠N

and

x i := y󸀠i − x i+1 c󸀠i ,

The receipt (A.30-A.32) is known as the ‘Thomas algorithm’ and can be extended straightforwardly to matrices having more than one upper and lower diagonal.

A.4 Homogeneous systems of linear equations A.4.1 Eigenvalue problems Now we turn to the homogeneous equations (A.3) B ⋅ x⃗ − λ x⃗ = 0 ,

(A.33)

that can be brought into the form (A.1) by substituting B − λ 1 = A. The system (A.33) has nontrivial solutions only if the determinant det(B − λ 1) = 0 vanishes (solvability condition). As already stated in Section A.1.2, this yields the characteristic polynomial P(λ) whose N zeros correspond to the eigenvalues of B. To each eigenvalue λ i , an eigenvector v⃗ i is assigned so that B ⋅ v⃗ i = λ i v⃗ i ,

i = 1...N ,

(A.34)

where not all eigenvalues must be different, see Section A.2.2.

A.4.2 Diagonalization The central task is to find eigenvalues and eigenvectors for a given matrix. As outlined in Section A.2.2, each matrix can be transformed into Jordan canonical form: C−1 ⋅ B ⋅ C = J .

280 | A Matrices and systems of linear equations

Since the transformation does not change the spectrum, the eigenvalues of A and J are equal and can simply be read off from the diagonal of J: λ i = J ii . Thus, the spectrum is known if an appropriate C has been determined. If J is diagonal, J ij = λ i δ ij , B is said to be diagonalizable and its desired eigenvectors are identical with the columns of the matrix C (for matrices which can not be brought into diagonal form there exist special methods which we shall not discuss here). How can we obtain the transformation C? In practice, two different methods are applied, sometimes also a combination of both methods. We shall present a short overview of both, for more details see the literature, e.g., [1] or [2]. A.4.2.1 Transformation method The idea is to find different invertible transformation matrices P i that change and simplify the original matrix in a specific way. This can be the zeroing of certain elements (Jacobi transformation), or of complete rows or columns (Householder transformation). Thus, one has −1 −1 B 󸀠 = P−1 k . . . P2 P1 B P1 P2 . . . P k . The resulting matrix B󸀠 is similar to B and must not already be in diagonal form. Anyway, it should be easier to handle than the original matrix. Finally, the better conditioned B󸀠 can be further diagonalized by factorization. A.4.2.2 Factorization method Assume that the matrix B can be factored into a right matrix F R and a left one F L according to B = FL ⋅ FR . (A.35) A new matrix can be built by commutation B󸀠 = FR ⋅ FL .

(A.36)

Conversely, multiplying (A.35) with F −1 L from the left side yields F R = F −1 L ⋅B, and, inserting this into (A.36), B󸀠 = F −1 L ⋅ B ⋅ FL .

(A.37)

Hence, B and B󸀠 are similar and have the same spectrum. Next, one constructs the sequence (n) ⋅ F nL = F nR ⋅ F nL (A.38) B (n+1) = F −1 nL ⋅ B

A.4 Homogeneous systems of linear equations |

281

with B (0) = B and F nL , F nR as factors of B(n) . One can show that B(n) converges for n → ∞ to a triangular matrix if the factorization (A.35) fulfills certain properties [2]. This will be described in greater detail in the next section. A.4.2.3 LU factorization Now we use in (A.35) the LU decomposition introduced in detail in Section A.3.4 B=L⋅U. The iteration rule (A.38) thus reads (n) ⋅ Ln = U n ⋅ Ln B(n+1) = L−1 n ⋅B

(A.39)

and one complete iteration step looks as follows: 1. Factorize B in L and U. 2. Compute B󸀠 = U ⋅ L. 3. Put B := B 󸀠 . 4. If B is not in triangular form, go to 1. At each step B comes closer to an upper triangular matrix. If B has reached this form it will not change further, hence each upper triangular matrix is a fixed point of the map (A.39). This is evident because an upper triangular matrix B factorizes in a trivial way into an upper triangular matrix U = B and the unit matrix as a ‘lower triangular matrix’ L. Then one has L U = U L and B = B 󸀠 . A.4.2.4 QR factorization Not every matrix can be LU decomposed. Conversely, the so-called QR decomposition B=Q⋅R with R again as the upper triangular matrix but now an orthogonal matrix Q is always possible. Instead of (A.39) we obtain B(n+1) = Q −1 ⋅ B(n) ⋅ Q n = R n ⋅ Q n . n

(A.40)

We list the following properties without proof, for details see [2]: – If all eigenvalues of B have different absolute values, B (n) converges for n → ∞ to an upper triangular matrix. The eigenvalues are ordered with increasing absolute value in the diagonal. – If k eigenvalues have the modulus |λ i |, i = 1 . . . k, B(n) converges to an upper triangular matrix with the exception of a diagonal block matrix of order k whose eigenvalues are λ i .

282 | A Matrices and systems of linear equations

A.4.3 Application: zeros of a polynomial When given a polynomial of degree N in the form N

P(x) = ∑ a k x k ,

(A.41)

k=0

whose N complex zeros P(x i ) = 0 ,

xi ∈ C ,

i = 1...N

are to be determined, several methods are available. Here we introduce an algorithm leading to a linear eigenvalue problem. First, we normalize (A.41) b k = a k /a N ,

k = 0...N −1

and write the polynomial as N−1

̃ P(x) = xN + ∑ bk xk , k=0

having of course still the same zeros as P. We construct the N × N-Frobenius matrix 0 1 ( ( F=( (. (. . ( One can show that

0 1

0

1

−b 0 −b 1 −b 2 ) ) . ) ) . . ) . −b N−1 )

(A.42)

̃ det(F − x 1) = (−1)N P(x)

holds, i.e., the N eigenvalues of F correspond to the zeros of P. The matrix (A.42) has Hessenberg form, this is a triangular matrix with an additional secondary diagonal. Such matrices can be effectively diagonalized, e.g., using the LAPACK routine DHSEQR:

C C C C

SUBROUTINE zeros(a,n,z) computes all zeros of the polynomial p = a(n)*x^n + a(n-1)*x^(n-1) ... a(0) = 0 a(0:n) [in]: coefficients (not changed) real*8 z(n) [out]: the complex zeros, complex*16

Bibliography | 283

C

C

C

Double precision version IMPLICIT REAL*8 (a-h,o-z) COMPLEX*16 z(n) DIMENSION a(0:n),zr(n),zi(n),frob(n,n),work(1000) an=a(n); a=a/an ! normalization of polynomial Frobenius matrix (Hessenberg) frob=0. frob(1:n,n)=-a(0:n-1) DO i=1,n-1 frob(i+1,i)=1. ENDDO Lapack call, computes the eigenvalues of frob CALL DHSEQR('E','N',n,1,n,frob,n,zr,zi,dumy,1,work,1000,info) z=CMPLX(zr,zi) a=a*an ! re-do normalization END

Bibliography [1] L. Hogben (editor), Handbook of Linear Algebra, Chapman & Hall (2007). [2] J. Stoer and R. Bulirsch, Introduction to Numerical Analysis, Springer (2010). [3] W. H. Press, B. P. Flannery, S. A. Teukolsky and W. T. Vetterling, Numerical Recipes, Cambridge Univ. Press (2007).

B Program library B.1 Routines We wish to supply and to describe some useful and often needed routines. They can be compiled separately and linked by specifying the corresponding object file (.o) during the compilation process of the main program. If the routines are gathered e.g., in a file named fortlib.f, one compiles once with gfortran -c fortlib.f The option -c tells the compiler not to create an executable but rather generate an object file that is named for this case fortlib.o. If a routine from the library is used, one must modify the script file used for compilation, (e.g., make_f95 from Section 1.2.4), by adding the fortlib object file. Finally, the script file should look like this (if LAPACK is also needed): gfortran -O2 $1.f -o $1 fortlib.o -L/usr/lib/ -lpgplot -llapack The complete library contains the following sources:

Name

purpose

graphics init contur contur1 ccontu image ccircl

graphics initialization contour lines contour lines contour lines (colored) bitmap (colored) initialize color circle

Runge–Kutta rkg drkg drkadt

Runge–Kutta Runge–Kutta (double precision) Runge–Kutta, adaptive time step (double prec.)

miscellaneous tridag ctrida dlyap_exp schmid volum deter random_init

Thomas algorithm for tridiagonal matrix Thomas algorithm for complex tridiag. matrix computation of (all) Lyapunov exponents Schmidt–Gram orthogonalization volume spanned by vectors (function) determinant of a matrix (function) initialization of random generator

https://doi.org/10.1515/9783110515145-010

see book page 8 56 199 20 20 67 71 278 104

241

B.2 Graphics

| 285

In the following, we only present the comment headers of the respective routines. Where not specified otherwise, implicit type declaration is assumed and real variables are in single precision (real*4). The complete listing is provided in ‘http://www.degruyter.com/books/978-3-11-051513-8’.

B.2 Graphics All routines in this section resort to programs of the PGPLOT library (see chapter 1).

B.2.1 init SUBROUTINE init(x0,x1,y0,y1,size,aspect,frame) c initializes a graphics window c x0,x1 [In] x-region c y0,y1 [In] y-region c size [In] size of window in inches c aspect [in] aspect ratio (y/x) c frame [In] if frame='Y' or 'y' a box is plotted c

B.2.2 contur

c c c c c c

SUBROUTINE contur(a,idim,jdim,n) Plots n contour lines of an array a. The contour values are chosen equidistant between min(a) and max(a). If n=0: only the a=zero line is plotted. a(idim,jdim) [In] 2D array idim, jdim [In] dimensions of a n [In] number of lines

B.2.3 contur1

c c c c c

SUBROUTINE contur1(a,idim,jdim,n) plots contour lines as 'contur' n=0 [In] only zero line plotted n>0 n lines between 0 and max(a) (equidistant) n 3. A linear stability analysis yields for the perturbations ϵ of (C.3) ϵ n+1 = A ϵ n with the amplification factor A=

d (2) 󵄨󵄨󵄨󵄨 . f (x)󵄨󵄨 󵄨󵄨 x p ,p dx 1 2

Forming the derivative of the second iterate (C.1) we obtain A = −4a3 x3 + 6a3 x2 − 2(1 + a)a2 x + a2 . Using (C.2) we find after some calculations A = −a2 + 2a + 4 . The periodic solution (C.3) becomes unstable if |A| > 1 . This is the case if either a < 3, which is excluded from the existence of (C.3), or if a > 1 + √6 . https://doi.org/10.1515/9783110515145-011

C.2 Chapter 2

| 291

C.2 Chapter 2 Problems 2.1.4 1. For K = 0: y n+1 = y n ,

x n+1 = x n + y n ,

horizontal rows of dots, which are dense if y n is irrational, otherwise points with gaps. 2. The chain length after N iterations is given as ℓ = x N − x0 , thus for K = 0 ℓ = N y0 , a linear function of the initial value y0 . For small values of K the length deviates from the straight line for certain rational y0 = 1/M, M integer, and locks in the form of steps of a staircase. For larger K this effect increases and chaotic regions occur in addition; see Figure C.1. 104

104

8000

8000

6000

6000

K = 0.2

4000

4000

2000

2000

0

0

0.2

0.4

0.6

0.8

1

0

104

104

8000

8000

6000

K = 0.5

0

0.2

0.4

0.6

1

6000 K = 0.7

4000

4000

2000

2000

0

0.8

0

0.2

0.4

0.6

0.8

1

0

K = 1.0

0

0.2

0.4

0.6

0.8

Fig. C.1: Chain length as function of y 0 for several K after 10 000 iterations. For larger K an irregular staircase (devil’s staircase) occurs.

1

292 | C Solutions of the problems

C.3 Chapter 3 Problems 3.1.2 We study the map x n+1 = f(x n ) with f(x) = a ∆t x (1 −

1 1 + − x) a a∆t

and

df 1 1 = a ∆t (1 − + − 2x) . dx a a∆t In particular we wish to compute the stability of the fixed point xs = 1 − 1/a

and find for the amplification factor A=

df 󵄨󵄨󵄨󵄨 󵄨 = 1 + (1 − a)∆t . dx 󵄨󵄨󵄨xs

For xs > 0, one must have a > 1 and therefore A ≤ 1. The numerical stability limit is thus given by A = −1, or 2 . ∆t = a−1

C.4 Chapter 4 Problems 4.3.2 ̃ 0 + ∆x)) on the left-hand side of (4.22) can be written as a Taylor 1. The expression f(y(x expansion with respect to g: ̃ 0 + ∆x)) = f(y(x0 ) + g) f(y(x = f(y(x0 )) +

df 1 d2 f 2 g + O(∆x3 ) . g+ dy 2 dy2

Now taking g from the expansion 1 ̃ 0 + ∆x) = y(x0 ) + y󸀠 ∆x + y󸀠󸀠 ∆x2 + O(∆x3 ) , y(x ⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟ 2 ≡g

we obtain up to the order

∆x2

̃ 0 + ∆x)) = f(y(x0 )) + f(y(x

df 1 1 d2 f 󸀠2 2 y ∆x + O(∆x3 ) . (y󸀠 ∆x + y󸀠󸀠 ∆x2 ) + dy 2 2 dy2

Inserting into (4.22) yields ̃ 0 + ∆x) = y(x0 ) + ∆x ⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟ f(y(x0 )) + y(x y󸀠

∆x2 df 󸀠 ∆x3 1 df 󸀠󸀠 1 d2 f 󸀠2 y ] +O(∆x4 ) . y + [ y + 2 ⏟⏟⏟⏟⏟⏟⏟⏟⏟ dy 2 ⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟ 2 dy 2 dy2 y󸀠󸀠

1 󸀠󸀠󸀠 2y

C.4 Chapter 4 |

293

On the other hand, when expanding the exact solution we obtain 1 󸀠󸀠 2 1 󸀠󸀠󸀠 3 y ∆x + y ∆x + O(∆x4 ) . 2 6 Subtracting the two last equations finally yields the error y(x0 + ∆x) = y(x0 ) + y󸀠 ∆x +

̃ 0 + ∆x) − y(x0 + ∆x)| = |y(x

1 󸀠󸀠󸀠 3 |y |∆x 12

which is of order ∆x3 . 2. The symplectic scheme of (4.5) reads φ n+1 = φ n + ω n+1 ∆t ω n+1 = ω n − (αω n + Ω20 φ n ) ∆t or φ φ A( ) = B( ) ω n+1 ω n with A=(

1 0

−∆t ) , 1

Thus, Q s in (4.21) reads

B=(

1 −Ω20 ∆t

0 ) . 1 − α∆t

Q s = A−1 B .

Problems 4.3.3.3 1. Meaning of the expressions: α 1 : increase of prey by reproduction, exponential growth α 2 : limitation of prey by predator, eating β 1 : decease of predator, exponential decrease β 2 : increase of predator due to prey food, eating After scaling: t = τ t̃ ,

n i = γ i ñ i

with τ = 1/β 1 ,

γ1 = β 1 /β 2 ,

γ2 = β 1 /α 2

one arrives at the form (4.27) with the sole remaining (control) parameter a = α 1 /β 1 . 2. The fixed points are computed from 0 = a ñ 1 − ñ 1 ñ 2 0 = −ñ 2 + ñ 1 ñ 2 as

(C.4)

294 | C Solutions of the problems (i) ñ 1 = ñ 2 = 0 Stable for a < 0, node, unstable for a > 0, saddle node. (ii) ñ 1 = 1 ,

ñ 2 = a.

This fixed point exists only for a > 0 since ñ i as a concentration is by definition positive. Linearizing: 1 u1 ñ 1 ( ) = ( ) + ( ) e λt a u2 ñ 2 leads after insertion in (4.27) to the solvability condition det (

λ −a

1 ) = λ2 + a = 0 λ

and λ12 = ±i√a . Hence the second fixed point is always marginally stable (Re λ = 0), but has a nonvanishing imaginary part. This is denoted as a center. 3. Time derivative of W yields dW/dt = d ñ 1 /dt + d ñ 2 /dt −

d ñ 1 /dt d ñ 2 /dt −a ̃n1 ñ 2

If we insert (4.27), the right-hand side vanishes, showing that W is conserved. 4. Numerical solutions are obtained straightforwardly using Runge–Kutta; see Figure C.2. 5

4

3

2

1

0 0

1

2

3

4

5

Fig. C.2: Numerical solution for a = 1 and different initial conditions, n2 over n1 .

C.4 Chapter 4 |

295

Problems 4.4.5 1. Transformation to a constantly rotating frame of references turns (4.55) into (tildes removed): ẋ = v ẏ = w v̇ = −2Ω0 w + Ω20 x − y rα

ẇ = 2Ω0 v + Ω20 y −

x rα

(C.5)

The system (C.5) possesses infinitely many fixed points lying on the circle √x20 + y20 = −2/α

r0 = Ω0 , each corresponding to the same circular orbit in the nonrotating frame. Without loss of generality we can take the fixed point: x 0 = r0 ,

y0 = 0 ,

v0 = w0 = 0 .

Linearization around this point yields for the deviations q⃗ the system λ q⃗ = J q⃗ with the Jacobi matrix 0 0 J=( 2 αΩ0 0

0 0 0 0

1 0 0 2Ω0

0 1 ) . −2Ω0 0

(C.6)

The eigenvalues of J read λ12 = ±Ω0 √α − 4 ,

λ34 = 0 .

The circular orbit is unstable if one of the eigenvalues has a positive real part, which is the case for α > 4. If we compute the total energy of the planet in the nonrotating system it reads α−4 1 , E0 = 2(α − 2) r0α−2 a value becoming positive if α > 4. Thus, the planet being initially on a circle could leave the system and vanish at infinity, independently of r0 (this reasoning applies of course only if the potential energy goes to zero for r → ∞, i.e., for α > 2). For α < 4 one finds λ12 = ±iω with ω = Ω 0 √4 − α and the planet will oscillate around the circular orbit when slightly disturbed. The condition for a closed orbit is resonance k ω = Ω0 , ℓ

296 | C Solutions of the problems where k and ℓ are two coprime integers. The orbit is closed after one revolution if ℓ = 1, leaving the condition for α α = 4 − k2 ,

k = 1, 2, 3, . . .

In fact there are infinitely many sharp values for α where circular orbits stay closed when slightly perturbed. We mention the first three ones, together with the behavior of the force on distance and the potential: 1. α = 3: Kepler potential, F ∼ −1/r2 , U ∼ −1/r. 2. α = 0: 2D-harmonic oscillator, F ∼ −r, U ∼ r2 . 3. α = −5: anharmonic oscillator, F ∼ −r6 , U ∼ r7 . To show numerically the existence of closed orbits we integrate the system (C.5) with the initial conditions x(0) = 1 ,

y(0) = 0 ,

v(0) = 0 ,

w(0) = 1 + ϵ

for 100 revolutions and average the x(t i ) − x(0) value after each revolution i: ⟨δx⟩ =

1 100 ∑ |x(t i ) − x(0)| 100 i

where t i is found from y(t i ) = 0 and w(t i ) > 0. The small deviation ϵ = 0.001 starts the instability. The result for α = −6 through α = 4 is shown in Figure C.3. Clearly, the deviations after one revolution get very small for the selected values α = −5, 0, 3, indicating a closed orbit.

0.1000000 ⟨δx⟩ 0.0100000

0.0010000

0.0001000

Kepler anharmonic oscillator

0.0000100

0.0000010 –6

harmonic oscillator α

–5

–4

–3

–2

–1

0

1

2

Fig. C.3: Averaged deviation from initial value x as a function of α.

3

4

C.4 Chapter 4

| 297

2. is left for the reader. 3. The Lagrangian points L1 through L3 are located on the x-axis. Thus it suffices to evaluate the Jacobi potential (4.46) at y = 0: V(x, 0) = −

1−μ μ x2 − − =0. |x + μ| |x + μ − 1| 2

Zeroing its derivative leads to the conditions for the fixed points ∂ x V(x, 0) =

1−μ μ (x + μ) + (x + μ − 1) − x = 0 . |x + μ|3 |x + μ − 1|3

After some calculations one finds a 5th degree polynomial: P μ (x) = − x5 + x4 (2 − 4μ) + x3 (6μ − 6μ 2 − 1) + x2 (6μ 2 − 4μ 3 − 2μ) + x(2μ3 − μ 4 − μ 2 ) + sign(x + μ) [x2 (1 − μ) + x(4μ − 2μ2 − 2) + (1 − μ)3 ] + μ sign(x + μ − 1)(x + μ)2 .

(C.7)

Due to the moduli one must distinguish between the three cases 1. x < −μ, left of m2 , sign(x + μ) = −1, sign(x + μ − 1) = −1 2. −μ < x < 1 − μ, between m2 and m3 , sign(x + μ) = 1, sign(x + μ − 1) = −1 3. 1 − μ < x, right of m3 , sign(x + μ) = 1, sign(x + μ − 1) = 1. The roots of (C.7) can be computed for instance using the program zeros of Section A.4.3. Figure C.4 shows the Lagrangian points depending on the mass relation μ.

1 0.5 0 –0.5 –1

0

0.2

0.4

0.6

0.8

1

Fig. C.4: Solid: x-coordinates of the three Lagrangian points L 1 , L 2 , L 3 over the mass parameter μ; dashed: position of the two main bodies.

298 | C Solutions of the problems

Problems 4.5.6 1. Thermalization: Multiplying (4.74) with v⃗ i and summation over all particles for F⃗ i = 0 yields 1 d (C.8) ∑ v2 = −γ ∑ v2i . 2 dt i i i From the equipartition theorem in 2D it follows 1 ∑ v2 = NT 2 i i and then from (C.8) d t T = −2γT = −2γ0 (T − Ts ) or T − Ts ∼ exp(−2γ0 t) . 2. 2D-equilibrium configurations: (i) If only the four nearest neighbors (blue) are considered, the potential energy reads U GQ = 4 U(r) for a square lattice, and U GH = 6 U(r) for a hexagonal grid, where we take U from (4.59). The equilibrium positions are found from dU GQ,H =0 (C.9) dr as r Q = r H = 21/6 ≈ 1.12246. (ii) Including also the red particles, we find U GQ = 4 U(r) + 4 U(√2r) and U GH = 6 U(r) + 6 U(√3r) and from the equilibrium condition (C.9) r Q = (65/36)1/6 ≈ 1.10349 ,

r H = ((27 + 1/27)/14)1/6 ≈ 1.11593 .

(iii) Including all plotted particles, we finally obtain U GQ = 4 U(r) + 4 U(√2r) + 4 U(2r) and U GH = 6 U(r) + 6 U(√3r)) + 6 U(2r) ,

C.4 Chapter 4 |

1/6

r Q = (2

1 + 2−6 + 2−12 ) 1 + 2−3 + 2−6

299

1/6

≈ 1.10100,

r H = (2

1 + 3−6 + 2−12 ) 1 + 3−3 + 2−6

≈ 1.1132.

The more stable configuration is that with the lower energy. Computing U G for the given equilibrium positions, one finds for all cases U GH < U GQ , independently on the number of neighbors taken into account. Hence the lattice should manifest a hexagonal geometry for low temperatures, in agreement with Figure 4.14, left frame.

Problems 4.7.5 1. Linearizing (4.102) with respect to the upper rest position yields u̇ 1 = u 2 sin(π + u 1 ) u̇ 2 = −Ω20 (1 + a sin ωt) ⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟ =−u1

or ü − Ω20 (1 + a sin ωt) u = 0

(C.10)

with u = u 1 . The same scaling as in Section 4.7.4 leads to ü − (p + 2b sin 2t)u = 0 .

(C.11)

Substituting the ansatz u(t) = u 0 cos(t + α) e λt , and neglecting the term sin(3t + α), one obtains (λ2 − 1 − p) (cos t cos α − sin t sin α) − 2λ (sin t cos α + cos t sin α) − b (sin t cos α − cos t sin α) = 0 or sin t [−(λ2 − 1 − p) sin α − (2λ + b) cos α] + cos t [(λ2 − 1 − p) cos α − (2λ − b) sin α] = 0 . The two square brackets must vanish, leading to the system (

−λ2 + 1 + p −2λ + b

sin α −2λ − b )=0. )( cos α λ2 − 1 − p

Its solvability condition constitutes a polynomial of 2nd degree in λ2 having the roots λ2 = p − 1 ± √b 2 − 4p .

300 | C Solutions of the problems

b

Re λ = 0

p 1

Fig. C.5: The upper rest position is stable in the shaded area.

For a nonpositive Re(λ), λ2 must be real valued and λ2 ≤ 0. The first condition leads to b ≥ 2√p , the latter to b ≤ 1+p. In addition, 0 ≤ p ≤ 1 must hold. This results in a stability diagram sketched in Figure C.5. 2. The damped Mathieu equation is transformed to the dimensionless normal form ̃ =0 ü + β u̇ + (p + 2b sin 2t)u

(C.12)

by the scaling of Section 4.7.4 and β = 2α/ω. To find the Floquet exponents one numerically integrates the system (4.104), but now with 0 L(t)̃ = ( −(p + 2b sin 2t)̃

1 ) , −β

(C.13)

from t ̃ = 0 through t ̃ = T = π. Determining the monodromy matrix according to (4.106) one finds the Floquet multipliers (4.107). Figure C.6 shows the contour lines max(|σ 1 |, |σ 2 |) = 1 that mark the onset of instability. The tips of the tongues no longer reach the p-axis as in Figure 4.30 but are shifted to finite critical amplitudes that are necessary to overcome friction.

C.5 Chapter 5 |

301

Fig. C.6: Stability chart for the damped Mathieu equation (C.12) with β = 0.1.

C.5 Chapter 5 Problems 5.6 1. Exact solution of (5.4). Let α > 0. The set (5.4) is written dimensionless with the scaling ̃ , t = t/α

(x, y) = (x,̃ y)̃ ⋅ g/α 2

and reads ẍ̃ = −ẋ̃ + β̃ ỹ ÿ̃ = −ẏ̃ − 1 .

(C.14) (C.15)

Only one parameter β̃ = β/α 2 is left. We leave all tildes. Different ways to solve the system are possible. One can compute y from (C.15) and obtains y(t) =

T f(t) − t f(T)

with f(t) = 1 − e−t .

(C.16)

302 | C Solutions of the problems The boundary conditions y(0) = y(T) = 0 are already implemented in (C.16). From (C.14) and (C.15) y can be eliminated: d4t x + 2d3t x + d2t x = −β . Integrating twice yields ẍ + 2ẋ + x = −

β 2 t +At+B. 2

Putting β 2 t 2 with A = b − 2β, B = a + 2b − β, leads to the homogeneous equation x(t) = X(t) + a + b t − Ẍ + 2Ẋ + X = 0 , which is solved by X(t) = (c t − a) e−t . Here, x(0) = 0 is already included. Thus one finds for x x(t) = (c t − a) e−t + a + b t −

β 2 t . 2

(C.17)

The coefficients b, c are determined by inserting (C.17) and (C.16) in (C.14) and comparing the coefficients: βT , b=c+β. c= f(T) The remaining coefficient a follows from the boundary condition x(T) = L: a=−

T 2 β(1 + 3e−T ) L − βT + . f(T) 2f 2 (T)

2. Solution of the nonlinear Schrödinger equation. The solution given is verified straightforwardly by insertion. Discussion for arbitrary C: The integral found by separation of variables from (5.45) has no analytic form. We discuss a ‘graphic solution’ coming from the analogy to a classical one-dimensional motion of a mass point in a potential. Writing (5.42) in the form Φ󸀠󸀠 = −EΦ + γ Φ3 = −

dV(Φ) dΦ

with

(C.18)

E 2 γ 4 (C.19) Φ − Φ , 2 4 (C.18) is identical with the equation of motion of a mass point with m = 1 in the potential (C.19) (x plays the role of time, Φ that of the position). Equation (5.45) corresponds to the conservation of energy and C/2 denotes the total energy V(Φ) =

C 1 󸀠 2 = (Φ ) + V(Φ) . 2 2

C.6 Chapter 6

V

V

E2 /4γ

V

Φ

Φ

| 303

V

Φ

Φ E 2 /4γ

(a)

(d)

(c)

(b)

Fig. C.7: The four different potentials. Nonlinear waves can exist for (a), (b) and (d).

We examine the four cases (a) E > 0, γ > 0, Figure C.7 (a). For C < E2 /2γ, nonlinear oscillations exist around Φ = 0. They have the square amplitude Φ2m = (E − √E2 − 2γC) /γ

(C.20)

and correspond to plane, nonlinear waves in position space. For C > E2 /2γ or C < 0, Φ diverges for x → ±∞. (b) E > 0, γ < 0, Figure C.7 (b). Nonlinear waves for C > 0. (c) E < 0, γ > 0, Figure C.7 (c). Diverging solutions for all C. (d) E < 0, γ < 0, Figure C.7 (d). If E2 /2γ < C < 0, nonlinear waves about Φ = ±√E/γ. For C > 0 nonlinear waves about Φ = 0 with square amplitude (C.20).

C.6 Chapter 6 Problems 6.4.4 1. We obtain (6.134) after scaling ̃ , t = t/α

x = x̃ √D/α ,

u = ũ ⋅ (α/β) .

2. (i) Linearization with u 0 = 0, u = u 0 + w: ∂ t w = ∂2xx w + w yields with w ∼ e λt e ikx λ(k) = 1 − k 2 . The fixed point is unstable for any plane wave with |k| < 1. (ii) Linearization about u 0 = 1: ∂ t w = ∂2xx w − w yields λ(k) = −1 − k 2 . The fixed point is stable for all k.

304 | C Solutions of the problems 3. Inserting u = u(ξ) in (6.134) leads to −v∂ ξ u = ∂2ξξ u + u − u 2 . Since u → 0 for ξ → ∞, we can linearize for large ξ according to: −v∂ ξ u = ∂2ξξ u + u . The ansatz u ∼ e−σξ , σ > 0 yields v=σ+

1 σ

and hence a lower limit for the speed: vmin = 2 for

σ=1.

4. Inserting (6.136) in (6.134) leads to 2C e−κξ (κ 2 − 1 − κv) − C2 e−2κξ (2κv + 4κ 2 + 1) = 0 . Since this must be valid for all ξ , the expressions in the brackets must vanish separately and one computes κ = 1/√6 .

v = −5/√6 ≈ −2.04 , By the substitution C = e κx0

it becomes evident that C shifts the front to the right side by x0 . 5. The stationary form of (6.134) can be written as u 󸀠󸀠 = u 2 − u = −

dV(u) du

with

u2 u3 − 2 3 and, as in problem 5.6(2), interpreted as one-dimensional motion in the potential V. The ‘energy law’ thus reads 1 󸀠2 (C.21) (u ) + V(u) = E . 2 For the special value E = 1/6, the ‘motion’ is along the homoclinic orbit of the fixed point u = 1. Thereby, u goes from u = 1 to u = −1/2 and back to u = 1; see Figure C.8. For E = 1/6, (C.21) may be integrated by the separation of variables: V(u) =

x − x0 = √3 ∫

2u + 1 du = 2 atanh (√ ) . 3 √1 − 3u 2 + 2u 3

After some short manipulations one obtains the stationary solution u 0 (x) = 1 −

3 2

2 cosh ((x − x0 )/2)

,

(C.22)

corresponding to a localized solution at x = x0 with u 0 (x → ±∞) = 1 and u 0 (x0 ) = −1/2.

C.6 Chapter 6

|

305

V

V = 1/6

u = –1/2

u=1

u Fig. C.8: Potential and homoclinic orbit for E = 1/6.

6. We examine the stability of (C.22) with x0 = 0. Linearization of (6.134) about u 0 yields (6.137) with the potential V(x) = 2u 0 − 1 = 1 −

3 cosh2 (x/2)

.

Expanding 3 2 x , 4 turns (6.137) into the equation of the quantum mechanical harmonic oscillator: V(x) ≈ −2 +

−d2xx w +

3 2 x w = (2 − λ)w . 4

The ground state is a Gaussian w ∼ exp(−(√3/4) x2 ), for its ‘energy’ one finds (2 − λ) = √3/2 or λ = 2 − √3/2 ≈ 1.13. Hence, λ > 0 and the solution (C.22) is proved to be unstable. The following MATLAB code computes the lowest three states of (6.137) with d x w(−L/2) = d x w(L/2) = 0 with L = 50 and n = 1000 mesh points using a finite difference scheme. The output is shown in Figure C.9. clear; n=1000; dx=50/(n+1); dx2=dx^2; x=-25+dx:dx:25-dx; a=-1/dx2*ones(n-1,1); % tridiagonal matrix for k=1:n b(k)=2/dx2+1-3/cosh(x(k)/2)^2; end b(1)=b(1)-1/dx2; b(n)=b(n)-1/dx2; m=diag(a,1)+diag(a,-1)+diag(b,0); % boundary conditions [v,e]=eigs(m,3,'sa'); % compute eigenvalues and vectors plot(x,v(1:n,1),x,v(1:n,2),x,v(1:n,3)); % output -diag(e)

306 | C Solutions of the problems 0.2 0.15

1

0.1 3

0.05 0 –0.05

2

–0.1 –0.15 –25

–20

–15

–10

–5

0

5

10

15

20

25

Fig. C.9: The three lowest states.

The values for λ are printed and read λ = 1.25, 0, −0.75. Note that the second (marginal, λ = 0) mode corresponds to a lateral shift of u 0 with w ∼ d x u 0 . It can be easily shown that w = d x u 0 is an exact solution of (6.137) with λ = 0.

C.7 Chapter 7 Problems 7.4.3 1. With the transformation

Ψ = Ψ̃ exp(iω0 t)

ω0 vanishes from (7.110). Inserting (7.111) in (7.110) yields (from here ω0 = 0): iΩ = ε − (1 + ic3 ) A2 or, after separating real and imaginary parts, ε − A2 = 0 ,

Ω + c3 A2 = 0

and from there A = √ε ,

Ω = −c3 ε .

2. Linear stability analysis about (7.111). Inserting Ψ = (A + w(x, t)) exp(iΩt) in (7.110) and linearization with respect to w gives ẇ = (1 + ic1 ) ∂2xx w − (1 + ic3 ) ε (w + w∗ ) ẇ ∗ = (1 − ic1 ) ∂2xx w∗ − (1 − ic3 ) ε (w + w∗ ) .

C.7 Chapter 7

| 307

With w ∼ exp(λt) exp(ikx) one arrives at the linear system (λ + (1 + ic1 ) k 2 + (1 + ic3 ) ε) w + (1 + ic3 ) ε w∗ = 0 (1 − ic3 ) ε w + (λ + (1 − ic1 ) k 2 + (1 − ic3 ) ε) w∗ = 0 . Its solvability condition is a 2nd degree polynomial in λ having the roots λ1,2 = −k 2 − ε ± √ε2 − c21 k 4 − 2c1 c3 εk 2 .

(C.23)

Expansion with respect to k 2 yields λ = −(1 + c1 c3 ) k 2 + O(k 4 ) for small k. Thus, λ > 0 for arbitrarily small k if 1 + c1 c3 < 0 . All plane wave disturbances with 0 ≤ k 2 ≤ k 2c and k 2c = −2ε

1 + c1 c3 1 + c21

are growing exponentially in time, Figure C.10. This is called Benjamin–Feir instability.

Fig. C.10: Largest λ from (C.23) for 1 + c 1 c 3 < 0 (solid) and 1 + c 1 c 3 > 0 (dashed).

3. Figure C.11 shows the real part of Ψ in an xt-diagram for c1 = 1/2, c3 = −3. It can be easily seen that the initial condition (7.111) becomes unstable and after t ≈ 250 gives way to a spatio-temporal chaotic evolution. 4. The largest Lyapunov exponent Λ is computed determining first a numerical solution Ψ N (x, t) as in 3. This is the reference trajectory; compare also Section 4.6.3.1. In

308 | C Solutions of the problems 2000

1500

1000

500

0 0

200

400

600

800

Fig. C.11: xt-diagram of Re(Ψ) from t = 0 through t = 2000.

parallel, one integrates the linearized DEQ for the complex valued deviations from the reference trajectory u(x, t): ∂ t u(x, t) = [ε+(1 + ic1 ) ∂2xx ] u(x, t) − (1 + ic3 )[2|Ψ N (x, t)|2 u(x, t) + Ψ N (x, t)2 u ∗ (x, t)] which is obtained by substituting Ψ(x, t) = Ψ N (x, t) + u(x, t) into (7.110) and linearizing around Ψ N . For u(x, 0) we choose a randomly distributed but normalized initial condition ∑ |u(x j , 0)|2 = 1 ,

with

x j = j∆x .

j

After a certain time δt (e.g., δt = 200∆t) the growth of |u| in the interval δt is computed: 1/2 2]

. s(δt) = [∑ |u(x j , δt)| j ] [ Thereafter u(x, δt) is again normalized. The continued numerical integration then yields s(2δt) and so on. At the end, the largest Lyapunov exponent is found as the mean value of all local exponents according to Λ = lim

K→∞

1 K 1 ∑ ln s(kδt) . K k=1 δt

(C.24)

For each single value of c3 the integration is carried on up to t = 34 000. After changing (increasing or decreasing) c3 , the evaluation of the corresponding Lyapunov exponent according to (C.24) is started after an initial run until t = 15 000.

C.7 Chapter 7

| 309

Fig. C.12: Largest Lyapunov exponent Λ of the complex Ginzburg–Landau equation over c 3 for c 1 = 1/2. A transition to chaos showing a wide hysteretic region results.

Figure C.12 depicts Λ over c3 for a fixed c1 = 1/2. From (7.112) chaotic dynamics is expected for c3 ≤ −2. Obviously, the behavior of Λ depends on the previous states and a large hysteretic region shows up in which both chaotic and regular dynamics may emerge at the same values of c3 .

D README and a short guide to FE-tools D.1 README File with description of the program examples on http://www.degruyter.com/books/978-3-11-051513-8

The codes are ordered along their appearance in the book chapters. In each subdirectory a script file make_f95 exists for compilation. In addition the packages pgplot, installed in /usr/lib/ lapack, installed in /usr/lib/lapack/ fftpack, installed in /usr/lib/ (only for chapter 7) are required. In this directory: README fortlib.f fortlib.o

this file (see also appendix D.1) the library described in appendix B object files, linked by make_f95

TI stands for Terminal Input -----------chap2/ lyap_2d.f computes the largest Lyapunov exponent of the standard map, sect. 2.2.3. TI: K (control parameter)

https://doi.org/10.1515/9783110515145-012

D.1 README | 311

dbc_slp.f decimal to binary converter from sect. 2.5.1.4 TI: following the learning phase, input of the decimal number to be converted kohonen_colormap.f color map after Kohonen as shown in Figure 2.16. TI: none tsp.f Traveling Salesman Problem from sect. 2.5.2.3 TI: R (radius of the initial circle) -----------chap3/ henon_heiles.f Plots the Poincaré section and trajectories from the Henon-Heiles model from sect. 3.3.3.1, compare Figure 3.5. The value 'e' for the energy can be modified in the code. TI: initial values set by mouse

-----------chap4/ reduc_3_b_p.f Plots and computes trajectories of the reduced three body problem from sect. 4.4.4.4. mu=1/4, can be modified in the code. TI: value of energy 'e'. Then set initial value by mouse in allowed region (red). molecular_dynamics.f Molecular dynamics simulation from sect. 4.5.4. On the left-hand side particles are depicted in config. space; the right-hand side shows the pair correlation function. Simulations are also possible in a constant gravity field. TI: temperature (0.1 - 2.0), gravitation (ca 1), tend (e.g. 200)

312 | D README and a short guide to FE-tools

pendulum_lyapunov_exp.f Bifurcation diagram (bottom) and two Lyapunov exponents of the driven pendulum from sect. 4.6.3.6. TI: region for a, resolution

-----------chap6/ shallow_water.f Solution of 2D shallow water eqs. from sect. 6.2.4.2 with FTCS. Here for two oscillating point sources. TI: none grid_contur.f mesh_generator.f laplace_solver.f Program tools for FE, see separate short guide in fe_guide.pdf house.grd, house.knd, house.ppm kidney.grd, kidney.knd, kidney.ppm example files for FE fe_guide.pdf guide for the FE codes, see also appendix D.2

-----------chap7/ driven_cavity.f solution of the hydrodynamic basic equations from sect. 7.3.2.3. Plot of stream lines as in Figures 7.9., 7.10 TI: Reynolds number benard_convection.f Solution of the hydrodynamic eqs. from sect. 7.3.4.2 in 3D following sect. 7.3.4.3. Plot of temperature field. Rayleigh number, Biot number, tend can be changed in the code. Note: the library FFTPACK is needed (see make_95) TI: none

D.2 Short guide to finite-element tools from Chapter 6

|

313

-----------chap8/ mc_gas.f compute the equilibrium of a many-particle system based on the Metropolis algorithm from sect. 8.3.1.2. In addition: constant gravitation. TI: temperature, gravitation. Note: units as in Figures 8.4 - 8.6 ising.f solution of the Ising model with Metropolis algorithm from sect. 8.3.2.3. temperature values are scanned, mean magnetization is computed. TI: external magnetic field

D.2 Short guide to finite-element tools from Chapter 6 The three codes for grid generation, Laplace solving, and evaluation: program

purpose

[I]nput/[O]utput

mesh_generator

generates a 2D-triangular mesh

[I]: ppm-file domain boundary [O] grid file

laplace_solver

solves Laplace equation with Dirichlet b.c.

[I]: grid file [O] node file

grid_contur

plots contour lines

[I] node file, grid file

Example files: name

purpose

created by

kidney.ppm, house.ppm kidney.grd, house.grd kidney.knd, house.knd

domain bound. grid files node files

Xfig (or similar); see Figure D.1 mesh_generator laplace_solver

D.2.1 mesh_generator TI stands for ‘Terminal Input’. TI: name of ppm-file (P6-format, binary, one byte/pixel), contains the domain boundary created by a graphics program, e.g., Xfig. The dimension of the bitmap can be as large as 5000 x 5000.

314 | D README and a short guide to FE-tools

Fig. D.1: The two domain boundaries ‘house.ppm’ and ‘kidney.ppm.’ The latter also contains inner boundaries (multiply connected domain).

TI: number of mesh points in the x-direction, defines the resolution of the triangular grid. Reasonable values for the examples are 30–60. The triangular grid is plotted (red). The outer boundary points are marked in blue. If some are missing they must be added manually using the mouse. If finished, type ‘s’ on keyboard. All mesh points left and right from the external boundaries are then deleted automatically. Boundary points are now marked by circles. Mouse modes: There are three modes, which one can toggle between using the keys ‘d, r, z.’ Key ‘d’: delete mode: Clicked points near the mouse cursor are deleted. Key ‘r’: boundary mode: Clicked points close to the mouse cursor are marked as boundary points. Inner boundaries (example ‘kidney’) have to be marked by hand and afterwards drawn to the desired position. Here, single points can also be set (point charges). Key ‘z’: draw mode: Clicked points near the mouse cursor can be drawn, e.g., onto the boundary. The new location is obtained by positioning the mouse cursor and repeated clicking. Attention: avoid crossing lines! Key ‘s’: TI for grid file name (output, .grd) and termination of the program.

D.2 Short guide to finite-element tools from Chapter 6

| 315

D.2.2 laplace_solver TI: grid file name. In the code the boundary node values have to be defined one by one. The number of corresponding mesh points is obtained from mesh_generator. TI: node file name. Here, the node values are stored after computation.

D.2.3 grid_contur TI: node file name. Then request for the value of the contour line(s) to be plotted. Termination by Ctrl-d.

What could be improved? The package can be extended and improved in various ways. Here, we only note a few ideas: – Handling of the boundary conditions. Instead of setting the (Dirichlet) boundary values in the code laplace_solver it could be better to define them in an extra file, or by mouse input. – Other boundary conditions (Neumann, mixed). – Automatic detection of the inner boundaries. – Extension to the Poisson equation. – Extension to time-dependent problems (diffusion equation, Schrödinger equation). – Extension to 3D.

Index A acceptance-rejection method 242 activator-inhibitor system 228 ADI method 159 algebraic multiplicity 270 anharmonic oscillator 127 apsidal precession 76 autonomous system 43 B ballistic flight 118 Barnsley ferns 23 Belousov–Zhabotinskii reaction 228 bifurcation diagram 97, 105 biharmonic operator 154 boundary conditions 117, 149 – Dirichlet 150, 152 – Neumann 150, 152, 157 – Robin 150, 157 Boussinesq approximation 212 box-counting method 25, 108 Box–Muller method 90, 245 Brusselator 229 Burgers equation 146 butterfly effect 15 C Campbell–Baker-Hausdorff relation 52 canonical ensemble 88, 251 canonical equations 50 capacity dimension 107 celestial mechanics 73 characteristics 144 Chebyshev spectral method 173 Chebyshev–Gauss–Lobatto 174 Chirikov map 13 Circle map 13 codimension-two 228 collocation method 129 color maps 36 conservative system 58 contact angle 255 convection 211 – equation 145, 167 – fully nonlinear problem 219 – linear stability analysis 215 – Rayleigh–Bénard 218 https://doi.org/10.1515/9783110515145-013

correlation dimension 107 Courant number 167 Crank–Nicolson method 67, 208 Crout algorithm 277 cyclic reduction 161 cyclotron frequency 202 D decimal to binary converter 33 degree of freedom 41 detailed balance 252 differential quotient 153 discretization – biharmonic operator 154 – Laplace operator 154 distribution function 241 driven cavity 206 drop formation 255 dynamical system 41 E eigenvalue problems 279 elliptic PDE 149 embedding dimension 110 ensemble 250 Euler method 60 expectation value 241 F finite differences 118, 153 finite element – evaluation 185 – Laplace solver 185 – mesh generation 185 – method 182 fixed point 43 Floquet – exponent 111 – theorem 111 flow field 204 flow-apart time 201 FORTRAN 1 fractal dimension 25, 106 fractals 20 Fredholm alternative 275 Frenkel–Kotorova model 12

318 | Index

frozen time analysis 216 FTCS scheme 163 G Galerkin method 129 Gauss–Seidel method 158 geometric multiplicity 273 Ginzburg–Landau equation 186 – complex 238 gnuplot 26 gradient dynamics 187 H Hamilton function 50 Hamiltonian system 50 hard spheres 250 harmonic oscillator 125 heat equation 151, 162, 176 Hebb’s learning rule 29 Hénon–Heiles model 54 Heun’s method 64 homoclinic orbit 60 Hopf instability 43, 227 hyperbolic PDE 149 hysteresis 265 I implicit declarations 10 implicit method 63 importance sampling 248, 253 incompressible fluids 204 initial conditions 149 Ising model 255 – one-dimensional 258 – two-dimensional 260 J Jacobi – matrix 18, 43 – method 158 – potential 80 Jordan canonical form 273 K Kepler problem 73 kernel 274 Kirkwood gap 77 Kohonen algorithm 35

L Lagrange polynomials 174 Lagrangian points 81 LAPACK 5 Laplace operator 154 Lax scheme 168 leapfrog scheme 169 learning process 28 least-squares method 129 Lennard-Jones potential 86 linear separability 32 linear stability analysis 7, 43 Linux 4 logistic map 7 logistic ODE 41 Lorenz attractor 49 Lorenz equations 48 Lotka–Volterra system 72 LU decomposition 276 LU factorization 281 Lyapunov exponent 98 – higher order 102 – logistic map 16 – multidimensional maps 17 – Standard map 18 Lyapunov functional 235 Lyapunov time 106

M Markov chain 252 mathematical pendulum 59, 95 Mathieu equation 114 MATLAB 1 Maxwell–Boltzmann distribution 90 mean-field approximation 257 Metropolis algorithm 251 – 2D-gas 253 – diffusion equation 265 – Ising model 260 – Swift–Hohenberg equation 267 microcanonical ensemble 88 molecular dynamics 86 monodromy matrix 111 Monte Carlo integration 245 Monte Carlo methods (MC) 240 moonshot 136

Index | 319

N Navier–Stokes equations 204 neural networks 27 Newton–Raphson 133 No-Intersection Theorem 44 normal distribution 243 null space 274 O orbit-type diagram 85 Oscar II 76 over-relaxation 158 P pair correlation function 91 parabolic PDE 149 parametric instability 112 partition sum 251 PDF 241 perceptron 28 PGPLOT 5 pivoting 278 Poincaré – map 56 – section 56, 97 Poincaré–Bendixson Theorem 45 Poisson bracket 51 Poisson equation 156 Prandtl number 213 pressure 204 probability density 241 pseudospectral method 221, 231 Q QR factorization 281 quasilinear differential equations 40 R random numbers 240 Rayleigh number 213 reaction-diffusion systems 225 reconstruction 109 recurrence time 197 regularization 82 Reynolds number 207 Rosenblatt rule 29 Runge–Kutta method 67 – with adaptive step size 71

S Schmidt–Gram method 104 Schrödinger equation 122 – nonlinear 134 – stationary 125 – stationary two-particle 193 – time-dependent 196 self-organized maps 34 self-similar 23 separatrix 60 shallow water equations 171 shooting method 140 Sierpinski triangle 22 simple sampling 248 specific heat 92 spectral method 178 spiral patterns 233 standard deviation 241 Standard map 13 Stark effect 123, 130 strange attractor 47 stream function 205 subdomain method 129 Swift–Hohenberg equation 235 – Metropolis algorithm 267 symplectic method 51, 66, 89 T Takens, embedding theorem 110 target patterns 233 Thomas algorithm 278 three-body problem 76 – reduced 79 time-evolution operator 41, 197 transformation method 242 transport equation 212 traveling salesman problem 37 Trojans 81 Turing instability 226 U upwind scheme 168 V Van der Pol oscillator 46 variance 241 variational problem 184 velocity field 205 Verlet algorithm 53

320 | Index

von Neumann stability analysis 166 vorticity 205 W wave equation 169 wave packet 197

Z zeros of a polynomial 282