A Primer on Radial Basis Functions with Applications to the Geosciences 978-1-61197-402-7

Citation preview

Downloaded 11/04/15 to 132.239.1.231. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

A Primer on Radial Basis Functions with Applications to the Geosciences

CB87_Fornberg-Flyer-FM.indd 1

8/21/2015 10:03:20 AM

Downloaded 11/04/15 to 132.239.1.231. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

CBMS-NSF REGIONAL CONFERENCE SERIES IN APPLIED MATHEMATICS A series of lectures on topics of current research interest in applied mathematics under the direction of the Conference Board of the Mathematical Sciences, supported by the National Science Foundation and published by SIAM. Garrett Birkhoff, The Numerical Solution of Elliptic Equations D. V. Lindley, Bayesian Statistics, A Review R. S. Varga, Functional Analysis and Approximation Theory in Numerical Analysis R. R. Bahadur, Some Limit Theorems in Statistics Patrick Billingsley, Weak Convergence of Measures: Applications in Probability J. L. Lions, Some Aspects of the Optimal Control of Distributed Parameter Systems Roger Penrose, Techniques of Differential Topology in Relativity Herman Chernoff, Sequential Analysis and Optimal Design J. Durbin, Distribution Theory for Tests Based on the Sample Distribution Function Sol I. Rubinow, Mathematical Problems in the Biological Sciences P. D. Lax, Hyperbolic Systems of Conservation Laws and the Mathematical Theory of Shock Waves I. J. Schoenberg, Cardinal Spline Interpolation Ivan Singer, The Theory of Best Approximation and Functional Analysis Werner C. Rheinboldt, Methods of Solving Systems of Nonlinear Equations Hans F. Weinberger, Variational Methods for Eigenvalue Approximation R. Tyrrell Rockafellar, Conjugate Duality and Optimization Sir James Lighthill, Mathematical Biofluiddynamics Gerard Salton, Theory of Indexing Cathleen S. Morawetz, Notes on Time Decay and Scattering for Some Hyperbolic Problems F. Hoppensteadt, Mathematical Theories of Populations: Demographics, Genetics and Epidemics Richard Askey, Orthogonal Polynomials and Special Functions L. E. Payne, Improperly Posed Problems in Partial Differential Equations S. Rosen, Lectures on the Measurement and Evaluation of the Performance of Computing Systems Herbert B. Keller, Numerical Solution of Two Point Boundary Value Problems J. P. LaSalle, The Stability of Dynamical Systems D. Gottlieb and S. A. Orszag, Numerical Analysis of Spectral Methods: Theory and Applications Peter J. Huber, Robust Statistical Procedures Herbert Solomon, Geometric Probability Fred S. Roberts, Graph Theory and Its Applications to Problems of Society Juris Hartmanis, Feasible Computations and Provable Complexity Properties Zohar Manna, Lectures on the Logic of Computer Programming Ellis L. Johnson, Integer Programming: Facets, Subadditivity, and Duality for Group and Semi-Group Problems Shmuel Winograd, Arithmetic Complexity of Computations J. F. C. Kingman, Mathematics of Genetic Diversity Morton E. Gurtin, Topics in Finite Elasticity Thomas G. Kurtz, Approximation of Population Processes Jerrold E. Marsden, Lectures on Geometric Methods in Mathematical Physics Bradley Efron, The Jackknife, the Bootstrap, and Other Resampling Plans M. Woodroofe, Nonlinear Renewal Theory in Sequential Analysis D. H. Sattinger, Branching in the Presence of Symmetry R. Temam, Navier–Stokes Equations and Nonlinear Functional Analysis Miklós Csörgo, Quantile Processes with Statistical Applications J. D. Buckmaster and G. S. S. Ludford, Lectures on Mathematical Combustion R. E. Tarjan, Data Structures and Network Algorithms

CB87_Fornberg-Flyer-FM.indd 2

8/21/2015 10:03:20 AM

Downloaded 11/04/15 to 132.239.1.231. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

Paul Waltman, Competition Models in Population Biology S. R. S. Varadhan, Large Deviations and Applications Kiyosi Itô, Foundations of Stochastic Differential Equations in Infinite Dimensional Spaces Alan C. Newell, Solitons in Mathematics and Physics Pranab Kumar Sen, Theory and Applications of Sequential Nonparametrics László Lovász, An Algorithmic Theory of Numbers, Graphs and Convexity E. W. Cheney, Multivariate Approximation Theory: Selected Topics Joel Spencer, Ten Lectures on the Probabilistic Method Paul C. Fife, Dynamics of Internal Layers and Diffusive Interfaces Charles K. Chui, Multivariate Splines Herbert S. Wilf, Combinatorial Algorithms: An Update Henry C. Tuckwell, Stochastic Processes in the Neurosciences Frank H. Clarke, Methods of Dynamic and Nonsmooth Optimization Robert B. Gardner, The Method of Equivalence and Its Applications Grace Wahba, Spline Models for Observational Data Richard S. Varga, Scientific Computation on Mathematical Problems and Conjectures Ingrid Daubechies, Ten Lectures on Wavelets Stephen F. McCormick, Multilevel Projection Methods for Partial Differential Equations Harald Niederreiter, Random Number Generation and Quasi-Monte Carlo Methods Joel Spencer, Ten Lectures on the Probabilistic Method, Second Edition Charles A. Micchelli, Mathematical Aspects of Geometric Modeling Roger Temam, Navier–Stokes Equations and Nonlinear Functional Analysis, Second Edition Glenn Shafer, Probabilistic Expert Systems Peter J. Huber, Robust Statistical Procedures, Second Edition J. Michael Steele, Probability Theory and Combinatorial Optimization Werner C. Rheinboldt, Methods for Solving Systems of Nonlinear Equations, Second Edition J. M. Cushing, An Introduction to Structured Population Dynamics Tai-Ping Liu, Hyperbolic and Viscous Conservation Laws Michael Renardy, Mathematical Analysis of Viscoelastic Flows Gérard Cornuéjols, Combinatorial Optimization: Packing and Covering Irena Lasiecka, Mathematical Control Theory of Coupled PDEs J. K. Shaw, Mathematical Principles of Optical Fiber Communications Zhangxin Chen, Reservoir Simulation: Mathematical Techniques in Oil Recovery Athanassios S. Fokas, A Unified Approach to Boundary Value Problems Margaret Cheney and Brett Borden, Fundamentals of Radar Imaging Fioralba Cakoni, David Colton, and Peter Monk, The Linear Sampling Method in Inverse Electromagnetic Scattering Adrian Constantin, Nonlinear Water Waves with Applications to Wave-Current Interactions and Tsunamis Wei-Ming Ni, The Mathematics of Diffusion Arnulf Jentzen and Peter E. Kloeden, Taylor Approximations for Stochastic Partial Differential Equations Fred Brauer and Carlos Castillo-Chavez, Mathematical Models for Communicable Diseases Peter Kuchment, The Radon Transform and Medical Imaging Roland Glowinski, Variational Methods for the Numerical Solution of Nonlinear Elliptic Problems Bengt Fornberg and Natasha Flyer, A Primer on Radial Basis Functions with Applications to the Geosciences

CB87_Fornberg-Flyer-FM.indd 3

8/21/2015 10:03:20 AM

Downloaded 11/04/15 to 132.239.1.231. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

Bengt Fornberg University of Colorado Boulder, Colorado

Natasha Flyer

National Center for Atmospheric Research Boulder, Colorado

A Primer on Radial Basis Functions with Applications to the Geosciences

SOCIETY FOR INDUSTRIAL AND APPLIED MATHEMATICS PHILADELPHIA

CB87_Fornberg-Flyer-FM.indd 5

8/21/2015 10:03:21 AM

Downloaded 11/04/15 to 132.239.1.231. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

Copyright © 2015 by the Society for Industrial and Applied Mathematics. 10 9 8 7 6 5 4 3 2 1 All rights reserved. Printed in the United States of America. No part of this book may be reproduced, stored, or transmitted in any manner without the written permission of the publisher. For information, write to the Society for Industrial and Applied Mathematics, 3600 Market Street, 6th Floor, Philadelphia, PA 19104-2688 USA. Trademarked names may be used in this book without the inclusion of a trademark symbol. These names are used in an editorial context only; no infringement of trademark is intended. IBM is a registered trademark of IBM, Inc. www.ibm.com Intel is a registered trademark of Intel Corporation or its subsidiaries in the United States and other countries. Mathematica is a registered trademark of Wolfram Research, Inc. MATLAB is a registered trademark of The MathWorks, Inc. For MATLAB product information, please contact The MathWorks, Inc., 3 Apple Hill Drive, Natick, MA 01760-2098 USA, 508-647-7000, Fax: 508-647-7001, [email protected], www.mathworks.com. NVidia is a registered trademark of NVIDIA Corporation in the U.S. and other countries. Publisher Acquisitions Editor Developmental Editor Managing Editor Production Editor Copy Editor Production Manager Production Coordinator Compositor Graphic Designer

David Marshall Elizabeth Greenspan Gina Rinelli Kelly Thomas Ann Manning Allen Bruce Owens Donna Witzleben Cally Shrader Techsetters, Inc. Lois Sellers

Library of Congress Cataloging-in-Publication Data Fornberg, Bengt. A primer on radial basis functions with applications to the geosciences / Bengt Fornberg, University of Colorado, Boulder, Colorado, Natasha Flyer, National Center for Atmospheric Research, Boulder, Colorado. pages cm. -- (CBMS-NSF regional conference series in applied mathematics ; 87) Includes bibliographical references and index. ISBN 978-1-611974-02-7 1. Radial basis functions. 2. Geology--Mathematical models. I. Flyer, Natasha. II. Title. QA223.F67 2015 512.7'3--dc23 2015022855 is a registered trademark.

CB87_Fornberg-Flyer-FM.indd 6

8/21/2015 10:03:21 AM

Downloaded 11/04/15 to 132.239.1.231. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

Contents Preface

ix

1

Brief Summary of Finite Difference Methods 1.1 Finite difference formulas . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Application of FD formulas to PDEs . . . . . . . . . . . . . . . . . . . .

2

Brief Summary of Pseudospectral Methods 2.1 Periodic (Fourier) PS methods . . . . . 2.2 Nonperiodic PS methods . . . . . . . . 2.3 PS methods in polar domains . . . . . 2.4 Concluding remarks of PS methods . .

1 1 9

. . . .

19 19 33 37 38

3

Introduction to Radial Basis Functions 3.1 General background on RBFs . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 Near-flat RBFs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3 Some additional issues with regard to RBF approximations . . . . . .

39 39 48 68

4

Global RBFs for Solving PDEs 4.1 A few larger-scale applications . . . . . . . . . . . . . . . . 4.2 Time-independent PDEs . . . . . . . . . . . . . . . . . . . . 4.3 Time-dependent PDEs . . . . . . . . . . . . . . . . . . . . . 4.4 Static local node refinement for time-dependent PDEs

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

91 . 92 . 92 . 95 . 101

5

RBF-Generated FD (RBF-FD) Methods 109 5.1 Basic concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109 5.2 RBF-FD for elliptic equations . . . . . . . . . . . . . . . . . . . . . . . . . 120 5.3 RBF-FD for time-dependent convection-type PDEs . . . . . . . . . . . 124

6

Global RBF Applications to Geo-Modeling: Spherical Domains 6.1 Vector versus scalar variables and operators on a sphere . . . . . . . . . 6.2 Shallow-water equations on a sphere . . . . . . . . . . . . . . . . . . . . . 6.3 Thermal convection in a 3-D spherical shell . . . . . . . . . . . . . . . .

133 133 134 141

7

RBF-FD Applications to Geo-Modeling: Spherical Domains 7.1 Shallow-water equations on a sphere . . . . . . . . . . . . . . . . . . . . . 7.2 Practical guidelines to solving hyperbolic PDEs on a sphere with C ∞ RBF-FD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.3 A 3-D ellipitic PDE with irregular boundaries: Global electric circuit

149 149

vii

157 161

Downloaded 11/04/15 to 132.239.1.231. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

viii

Contents

8

RBF-FD Applications to Geo-Modeling: Limited-Area Domains 167 8.1 Solving the Navier-Stokes equations on a rectangular domain with RBF-FD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167 8.2 Forward seismic modeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173

A

Introduction to RBFs via Cubic Splines 179 A.1 Cubic splines in 1-D . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179 A.2 Generalization to multiple dimensions . . . . . . . . . . . . . . . . . . . . 182 A.3 Improvement in accuracy from algebraic to spectral . . . . . . . . . . . 183

B

Spherical Harmonics B.1 Introduction of Fourier series via an eigenvalue problem . . . B.2 Introduction of Fourier expansion via 2-D Taylor expansions B.3 Uniform point sets on the unit sphere . . . . . . . . . . . . . . . B.4 Introduction of SPH via an eigenvalue problem . . . . . . . . . B.5 Introduction of SPH via 3-D Taylor expansions . . . . . . . . . B.6 Some concluding comments on SPH expansions . . . . . . . .

C

D

. . . . . .

. . . . . .

. . . . . .

. . . . . .

185 185 185 186 186 188 190

Some Node Distribution Strategies C.1 Halton nodes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C.2 Halton and random nodes on the surface of a sphere . . . . . . . . C.3 Node placing (NP) method . . . . . . . . . . . . . . . . . . . . . . . . C.4 Maximal determinant (MD) and minimal energy (ME) node sets

. . . .

. . . .

. . . .

191 191 192 192 195

Cartesian Vector Operators on a Sphere

. . . . . .

199

Bibliography

201

Index

219

Downloaded 11/04/15 to 132.239.1.231. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

Preface This book is focused on a powerful numerical methodology for solving PDEs to high accuracy in any number of dimensions: Radial Basis Functions (RBFs). During the past decade, this method has been shown to apply to a wide range of PDEs, arising, for example, in fluid mechanics, wave motions, astro- and geosciences, mathematical biology, computational electromagnetics, etc. In the past few years, the approach has advanced from being mainly “just another method that can be made to work on small toy problems” to one that can compete highly successfully against the very best previous approaches on some large benchmark problems. So far, its greatest successes in this direction have arguably been in the geosciences. Our focus in this monograph will be on how, when, and why RBF-based approaches work, more by means of examples and heuristic explanations than by rigorous theoretical arguments. Instead of trying to tread a careful path between the opposite sins of excessively intuitive arguments and formal rigor, we will systematically choose the former. The RBF approach is generally attributed to Rolland Hardy [143], who in 1971 proposed it for the purpose of interpolating scattered 2-D data. However, some of the key theorems underlying its numerical stability go back to the 1930s (Buchner [20], Schoenberg [241]). It was recognized by Kansa [162, 163] in 1990 that the ability of RBFs to provide accurate approximations for derivatives of functions known only at scattered data locations offered a novel opportunity for the numerical solution of PDEs. In this monograph, we will extend the “RBFs-for-PDEs story” both backward and forward in time. Conceptually, there is a very logical progression that starts with finite difference (FD) and pseudospectral (PS) methods and then, via global RBFs, leads to RBFgenerated FD (RBF-FD) methods. While RBFs by now are quite well established, the RBF-FD approach is still an emerging methodology. Although the progression FD ⇒ PS ⇒ RBF ⇒ RBF-FD is not quite how it always was perceived while the developments occurred, each of the last three methodologies is in fact closely linked to the preceding one. This book starts with brief introductions to FD and PS methods—limited to the extent that is needed for providing the perspective that we wish to convey about RBF and RBF-FD methods when these are applied to the task of solving PDE problems, in particular, as these arise in the geosciences. Finite difference methods: These were first proposed for solving PDEs in 1911 [221], and they have remained a dominant methodology ever since. Generally, they are easy to implement but are more restrictive than, for example, finite elements in terms of geometric flexibility. Pseudospectral methods: For applications in very simple geometries (intervals in 1D, rectangular or circular domains in 2-D, periodic boxes in 3-D, spherical shells, etc.) it was noted in the early 1970s that the order of accuracy of FD methods often can be increased indefinitely and that this sometimes can offer spectacular computational efficiencies. Another way to arrive at the same PS methods is via expansions in orthogonal ix

Downloaded 11/04/15 to 132.239.1.231. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

x

Preface

functions, such as Fourier, Chebyshev, and spherical harmonics (SPH). These PS methods soon became prominent for solving PDEs in numerous areas, including fluid dynamics (such as direct numerical simulations of turbulent flows), weather forecasting, long time evolution of linear and nonlinear waves, and computational electromagnetics. Radial basis functions: It transpires that all PS methods can be seen as highly specialized (and typically not optimal) cases of RBFs applied to PDEs. RBFs generalize PS methods away from their severe geometric limitations and their dependence on very regular node layouts (which for PS methods makes it complicated to carry out local refinements in critical solution areas). This can be done while preserving their spectral accuracy (beyond any power of the typical node spacing). Both the coding effort and the computational cost of RBFs are independent of how simple or complicated the geometry might be. In a recent large-scale 3-D geophysical flow application, an RBF-based code on a standard PC [291] competed very favorably against all previous methodologies, even when these were implemented on large supercomputer systems. In spite of these successes, computational cost and “scalability” to large computer systems remained lingering concerns. Radial basis function-generated FD methods: This takes us almost back to FD methods—where the numerical journey of the present monograph started. It has recently been discovered that using RBFs to create generalized FD methods might offer the best opportunity yet for combining the strengths of all the previous approaches. In particular, RBF-FD methods can offer (i) numerical stability even when using explicit time stepping of purely convective problems on irregular node layouts, (ii) very high computational speed (since they only rely on local approximations, and they also give rise to sparse rather than to full matrix problems), (iii) accuracy levels approaching those of PS and global RBF methods, (iv) easy opportunities for local (adaptive) refinements, and (v) excellent opportunities for large-scale parallel computing (from GPU boards to supercomputers with vast numbers of processors). Applications of RBF and RBF-FD methods in the geosciences: While FD and PS methods by now have long histories, the RBF approach has only in the past few years taken the crucial steps up to full-blown applications and then further from just showing feasibility to demonstrating actual cost advantages, in some cases over all previously available methods. This book follows quite closely the lectures that were given by the present authors at the NSF-CBMS Regional Research Conference “Radial Basis Functions—Mathematical Developments and Applications,” held June 20–24, 2011, at the University of Massachusetts, Dartmouth. The lecture notes have here been edited and expanded so that this book also can serve as a textbook for a semester-length graduate course on RBFs (and, in particular, on their application to PDEs). Although some background materials are included here on ODE solvers, basic finite differences, etc., it is nevertheless recommended that students first complete some more introductory course on numerical methods before proceeding to the present material. Acknowledgments: This book project would not have been possible without the generous help of many organizations and individuals. The Regional Research Conference was supported by NSF under the grant DMS 1040883. NSF has also provided individual support to the authors when developing much of the present materials. Furthermore, NCAR is supported by the NSF. The conference was superbly organized by Saeja Kim, Sigal Gottlieb, Alfa Heryudono, and Cheng Wang. Several colleagues have assisted not only with helpful discussions but also by giving detailed comments to early versions of the present manuscript. For this, we want especially to thank Nick Trefethen and Grady Wright. We also owe great thanks to SIAM and, in particular, to Sara Murphy, Elizabeth Greenspan, and Gina Rinelli for making the publication process run very smoothly and pleasantly.

Downloaded 11/04/15 to 132.239.1.231. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

Chapter 1

Brief Summary of Finite Difference Methods

This chapter provides a brief summary of FD methods, with a special emphasis on the aspects that will become important in the subsequent chapters.

1.1 Finite difference formulas Finite differences (FD) approximate derivatives by combining nearby function values using a set of weights. Several different algorithms for determining such weights are mentioned in Sections 1.1.1–1.1.5. In the very simplest case, illustrated in Figure 1.1, we use the mathematical definition of a derivative f  (x) = lim

h→0

f (x + h) − f (x) h

(1.1)

to arrive at a two-node FD formula. Taylor expansion of (1.1) shows that f (x + h) − f (x) h h 2  = f  (x) + f  (x) + f (x) + . . . = f  (x) + O(h 1 ); h 2! 3! f (x+h)− f (x)

i.e., the approximation f  (x) ≈ is accurate to first order. The FD weights at the h nodes x and x + h are in this case [−1 1] / h. The FD stencil can graphically be illustrated as ← entry for f  , value {1}  1 1   ← entries for f , values {− h , h } (1.2) ↑ ↑ x x + h ← spatial locations. The open circle indicates a (typically) unknown derivative value and the filled squares (typically) known function values. While the compactness of this approximation is convenient (it uses only two adjacent function values), its low order of accuracy (first order; exact only for polynomials up to degree one) makes it almost entirely useless for practical computing. Before considering the application of FD formulas to tasks such as approximating ODEs and PDEs (ordinary and partial differential equations), we consider next some different procedures for creating higher-order FD approximations. 1

Downloaded 11/04/15 to 132.239.1.231. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

2

Chapter 1. Brief Summary of Finite Difference Methods

Figure 1.1. Illustration of the approximation f  (x) ≈ accurate as h → 0.

rise run

=

f (x+h)− f (x) ; h

this is increasingly

1.1.1 Some direct approaches for generating FD stencils The three approaches described next are flexible and conceptually quite straightforward but also rather inefficient in terms of their operation count. Even so, it should be noted that the linear systems approach (Section 1.1.1.3) will become prominent later in the RBF and RBF-FD contexts. For simplicity of notation, we do not describe the approaches in their most general form but choose the specific example of finding the weight vector 1 1 [− 2 0 2 ]/h in the second-order approximation to the first derivative 1

f  (x) ≈

− 2 f (x − h) +

1 2

f (x + h)

h

.

(1.3)

1.1.1.1 Derivative of Lagrange’s interpolation polynomial

The value for x does not influence the weights in a formula such as (1.3), so we can assume that the stencil is centered at x = 0. The Lagrange interpolation polynomial p(x), taking the desired values at the nodes x = −h, 0, h, becomes p(x) =

(x−0)(x−h) (−h−0)(−h−h)

f (−h) +

(x+h)(x−h) (0+h)(0−h)

(x+h)(x−0)

f (0) + (h+h)(h−0) f (+h).

Differentiating this polynomial with respect to x and then setting x = 0 gives p  (0) = 1 1 (− 2 f (−h) + 0 f (0) + 2 f (+h))/h, in agreement with (1.3). 1.1.1.2 Taylor expansions

Expressing f (−h), f (0), f (h) by Taylor expansion around x = 0 gives ⎧ ⎨ f (−h) f (0) ⎩ f (h)

h

= f (0) − 1! f  (0) + = f (0), h = f (0) + 1! f  (0) +

h2 2! h2 2!

f  (0) − + . . . , (1.4) 

f (0) + . . . .

We want to find weights w−1 , w0 , w1 such that w−1 f (−h) + w0 f (0) + w1 f (h) = 0 f (0) + 1 f  (0) + 0 f  (0) + . . . .

Downloaded 11/04/15 to 132.239.1.231. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

1.1. Finite difference formulas

3

Using the expansions from (1.4) and equating coefficients for f (0), f  (0), f  (0) gives rise to a linear system to solve for the unknown coefficients ⎡ ⎤⎡ ⎤ ⎡ ⎤ 1 1 1 w−1 0 ⎢ − h 0 h ⎥⎣ ⎦ ⎣ = 1 ⎦, (1.5) w ⎣ 0 1! 1! ⎦ h2 h2 w 0 0 2! 1 2! with the solution w−1 = −

1 , 2h

w0 = 0, w1 =

1 . 2h

1.1.1.3 Use of monomial test functions

Continuing with the same example, we want the formula f  (0) ≈ w−1 f (−h) + w0 f (0) + w1 f (h) to be exact for as high degree polynomials as possible. Enforcing it in turn for the monomials f = 1, f = x and f = x 2 gives f =1 f =x f = x2

⇒ ⇒ ⇒

w−1 w−1 (−h) w−1 (−h)2

+w0

+w1 +w1 (h) +w1 (h)2

= 0, = 1, = 0,

equivalent to (1.5). In the more general case of finding the weights w1 , w2 , . . . , wn to use at locations x1 , x2 , . . . , xn for approximating a linear operator L at some location x = xc , we similarly solve the system ⎡ ⎢ ⎢ ⎢ ⎣

1 x1 .. .

1 x2 .. .

··· ··· .. .

1 xn .. .

x1n−1

x2n−1

···

xnn−1

⎤⎡ ⎥⎢ ⎥⎢ ⎥⎢ ⎦⎣

w1 w2 .. .

⎤

⎡

⎥ ⎢ ⎥ ⎢ ⎥=⎢ ⎦ ⎣

wn

L 1| x=xc L x| x=xc .. . L x n−1 | x=xc

⎤ ⎥ ⎥ ⎥. ⎦

(1.6)

The successive lines of this system enforce that the set of weights lead to the correct result for the functions 1, x, x 2 , . . . , x n−1 and thus, by linearity, for all polynomials up through degree n − 1. This direct linear systems approach is very flexible and easy to implement. However, it is not computationally fast (O(n 3 ) operations), and the coefficient matrix can become ill-conditioned. It will, however, become the primary approach in the context of RBF methods.

1.1.2 Padé-based algorithm for equispaced grids When the nodes have a uniform spacing h (as has been the case in the examples above), a particularly short symbolic algebra algorithm was presented in 1998 [91]. We generalize the stencil (1.2) to

Downloaded 11/04/15 to 132.239.1.231. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

4

Chapter 1. Brief Summary of Finite Difference Methods

Here, the numbers s, d , and n describe the stencil shape. In the illustration above, these take the values 3/2, 3, and 7, respectively. The weights, one at each node point, relate nodal values of the mth derivative of f with the nodal values of the function f . In Mathematica (version 7 and higher), the complete code is t = PadeApproximant[xs (Log[x]/h)m ,{x,1,{n,d}}]; CoefficientList[{Denominator[t],Numerator[t]},x] with similar codes in other symbolic languages. The following are three typical applications of this algorithm: The choices s = 1, d = 0, n = 2, m = 2 describe a stencil of the shape

Example 1. 

for approximating the second derivative (since m = 2). The algorithm pro   duces the output  2 h , {1, −2, 1} , corresponding to the explicit second-order accurate formula for the second derivative f  (x) ≈ { f (x − h) − 2 f (x) + f (x + h)}



(1.7)

The choices s = 0, d = 2, n = 2, m = 2 describe a stencil of the shape

Example 2. 

1 . h2



, again for approximating the second derivative (since m = 2). The algorithm    produces the output  

2 h 5h 2 h 2 , , , {1, −2, 1} 12 6 12 corresponding to the compact (implicit) fourth-order accurate formula for the second derivative 1  5 1 1 . f (x − h) + f  (x) + f  (x + h) ≈ { f (x − h) − 2 f (x) + f (x + h)} 12 6 12 h2

The choices s = −2, d = 2, n = 1, m = 1 describe a stencil of the shape

Example 3. 







(1.8)



for approximating the first derivative. The output 

  4h 23h 5h ,− , , {−1, 1} 12 3 12

is readily rearranged into f (x + h) = f (x) +

h (23 f  (x) − 16 f  (x − h) + 5 f  (x − 2h)) , 12

(1.9)

which we later (in Section 1.2.1.2) will encounter as the third-order Adams-Bashforth method for solving ODEs.

Downloaded 11/04/15 to 132.239.1.231. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

1.1. Finite difference formulas

5

Table 1.1. Weights for centered FD approximations of the first derivative on an equispaced grid (omitting the factor 1/h).

Order

Weights

2 1 12 3 20 1 5

4 1

6

− 60

8 .. .

1 280

↓

↓

↓

Limit

1 4

1 −3

1 2

···

4

− 105

−2

1

0

−3

2

0

−4

3

0

−5

4

0 .. .

↓ −1

0

1 2 2 3 3 4 4 5

1

− 12 3

1 60 4 105

− 20 1

−5

1

− 280

↓

↓

↓

↓

1

1 −2

1 3

−4

1

···

Table 1.2. Weights for centered FD approximations of the second derivative on an equispaced grid (omitting the factor 1/h 2 ).

Order 2 4

1

6 1

8 .. .

− 560

Limit

2 − 42

↓ ···

1 90 8 315

↓ 2 32

− 12 3

− 20 1

−5

Weights 1 −2 5 4 −2 3

2 − 22

− 12

− 72

−5

1 90 8 315

↓

↓

↓

↓

↓

π2 −3

2 12

2 − 22

2 32

− 42

− 18

205

↓ 2 12

1

4 3 3 2 8 5

49

3 2 8 5

↓

1 3

− 20 1

1

− 560 2

···

In every case, the weights will be the optimal ones with regard to formal order of accuracy. The algorithm is particularly convenient to use when only a small number of stencils are considered and when one wants to obtain the weights in exact rational form rather than as floating point numbers. Table 1.1 shows the lowest-order centered FD formulas for the first derivative and Table 1.2 for the second derivative. The existence of infinite order limits (indicated by the bottom line in each of the two tables) will play a key role in Section 2.1.1 when we introduce PS methods. The special case illustrated in Example 3 can be generalized to include all the main classes of linear multistep methods. With m = 1 and accuracy order p ≥ 1, the appropriate settings for s, d , and n become Adams-Bashforth (AB) Adams-Moulton (AM) Backward Differentiation (BD)

s = 1 − p, s = 2 − p, s = p,

d = p − 1, d = p − 1, d = 0,

n = 1. n = 1. n = p.

1.1.3 Algorithms for arbitrarily spaced grids FD approximations based on equispaced grids are very accurate when they are centered (extending equally far to both sides) but tend to lose accuracy when boundaries are approached and they have to become increasingly one-sided. The common remedy is to gradually cluster nodes more densely as the boundary is approached, as will be discussed further in Section 2.2.1. A number of effective algorithms for calculating FD weights are available for such (nonequispaced) cases.

6

Chapter 1. Brief Summary of Finite Difference Methods

Downloaded 11/04/15 to 132.239.1.231. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

1.1.3.1 FD approximations at select points

We consider here first the case when one merely wants a few stencils. In the case of nodes located at xi , i = 1, 2, . . . , n, one can obtain the weights for d p /d x p | x=z , p = 0, 1, . . . , m (where the location z may or may not coincide with any of the node locations) by means of function c = weights(z,x,m) % Calculates FD weights. The parameters are: % z location where approximations are to be accurate, % x vector with x-coordinates for grid points, % m highest derivative that we want to find weights for % c array size (m+1,length(x)), containing (as output) in % successive rows the weights for derivatives 0,1,...,m. n = length(x); c = zeros(m+2,n); c(2,1) = 1; x1 = x(ones(1,n),:); A = x1’-x1; b = cumprod([ones(n,1),A],2); rm = cumsum(ones(m+2,n-1))-1; d = diag(b); d(1:n-1) = d(1:n-1)./d(2:n); for i = 2:n mn = min(i,m+1); c(2:mn+1,i) = d(i-1)*(rm(1:mn,1).*c(1:mn,i-1)-(x(i-1)-z)*... c(2:mn+1,i-1)); c(2:mn+1,1:i-1) = ((x(i)-z)*c(2:mn+1,1:i-1)-rm(1:mn,1:i-1).*... c(1:mn,1:i-1))./(x(i)-x1(1:mn,1:i-1)); end c(1,:) = [];

For example, the statement weights(0, −2 : 2, 6) returns the output 0 0.0833 -0.0833 -0.5000 1.0000 0 0

0 -0.6667 1.3333 1.0000 -4.0000 0 0

1.0000 0 -2.5000 0 6.0000 0 0

0 0.6667 1.3333 -1.0000 -4.0000 0 0

0 -0.0833 -0.0833 0.5000 1.0000 0 0

This output shows the optimal weights to be applied to function values at x = −2, −1, 0, 1, 2, for approximating the zeroth up through the sixth derivative at x = 0. Since the approximation point z coincides with one of the data points, the top line tells the obvious fact that the most accurate “interpolation” is to just use that data value. We recognize lines 2 and 3 from the fourth-order approximations in Tables 1.1 and 1.2, respectively. The last two lines are all zero, reflecting the fact that there exist no formulas for the fifth and sixth derivatives that extend over only five node points. The derivation of the algorithm, given in [86, 91], is based on recursions that follow from Lagrange’s interpolation formula. If one also wants weights for shorter stencil widths (based on xi , i = 1, 2, . . . , ν; ν = 1, 2, . . . , n), these can be picked up “for free,” as otherwise discarded intermediate results after each step in the loop for i = 2:n. In that case, the algorithm costs only four arithmetic operations per calculated weight. If these shorter stencils are not wanted, the weights algorithm presented in [230] has a somewhat lower operation count and can be coded particularly efficiently in C++. However, that code is much longer, and it runs in MATLAB several times slower than the present algorithm.

1.1. Finite difference formulas

7

Downloaded 11/04/15 to 132.239.1.231. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

1.1.3.2 Algorithms for differentiation matrices (DMs)

In case one wants to employ global FD stencils (extending over all the nodes; the case with nonperiodic PS methods), one typically needs a sequence of weight sets, providing approximations that are accurate at each of the nodes xi in turn. Repeated use of the algorithm in Section 1.1.3 would be comparatively slow for producing such differentiation matrices (DMs), as it would not utilize the fact that the many separate cases are all based on the same node set. Several specialized algorithms for calculating such DMs have been presented [149, 278]. The algorithm and MATLAB code by Weideman and Reddy [276] is often preferred and is downloadable from the Web [275]. Once a DM has been calculated, the derivative approximations at all the nodes are obtained by a single matrix × vector multiplication ⎤ ⎡ ⎡ (m) ⎤ ⎡ u(x ) ⎤ u (x1 ) 1 ⎢ .. ⎥ ⎥ ⎣ ⎢ .. ⎦ DM ⎣ . ⎦. ⎦≈ ⎣ . u (m) (xn )

u(xn )

1.1.4 Errors when applying FD formulas to given functions The application of FD formulas gives rise to different types of errors: Truncation errors: These are expressed by the leading error terms that we have quoted above, such as O(h 2 ), O(h 4 ), etc. In most applications, higher orders (especially above second order) are preferable. We will introduce pseudospectral (PS) methods as the limit of increasing order FD methods. Rounding errors: The numerator in an approximation such as (1.7) can in standard arithmetic be evaluated to an accuracy of 10−16 ; i.e., if the stencil involves a division by h 2 , the rounding error becomes O(10−16 /h 2 ). Total error: The best accuracy is usually obtained when h is chosen so that the two error types above match. In the present example, O(h 2 ) matches O(10−16 /h 2 ) when h ≈ 10−4 , producing a total error around 10−8 . Higher-order FD methods fare better in this type of analysis, but higher derivatives make the situation much worse since, for the pth derivative, the rounding error becomes of the form O(10−16 /h p ). FD formulas are for this reason only rarely used for derivatives beyond the third or fourth. In the case of FD formulas for analytic functions, there are some options available for greatly reducing the rounding errors: If a function is known to be analytic and can be computed also for complex arguments, Cauchy’s integral formula leads to FD approximations that do not use points along the real axis but instead (for example) around a circle in the complex plane, centered at the location at which we want the derivative approximation. In this case, good accuracy does not require the circle radius to be small, and it turns out also that high-order derivatives (say, the 50th or the 100th) become numerically available to high precision [22, 84, 183]. In a different approach, applicable only when a function f (x) is analytic and real-valued for x real and one wants only the first derivative f  (x) for such an x-value, Cauchy-Riemann’s equations tell us that one equivalently can evaluate the derivative in the imaginary direction. Regular FD approximations will then not suffer the usual floating point cancellations as h → 0, and machine precision is readily achieved [252]. This approach has later become known as the complex step method. It can also be noted that, for analytic functions that satisfy simple ODEs (thus excluding important classes of functions such as those involving Γ (x), ζ (x), etc.), derivatives of any order can be calculated recursively [4].

8

Chapter 1. Brief Summary of Finite Difference Methods

Downloaded 11/04/15 to 132.239.1.231. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

1.1.5 Generalizations to more than 1-D 1.1.5.1 Cartesian lattices ∂3

On an (x, y) grid, any mixed derivative, such as ∂ x∂ y 2 , is most easily approximated by a stencil that amounts to approximating in the two directions in sequence. Just as in the case of analytic differentiation, the result will not depend on the order in which the partial derivations were carried out. The combined procedure can directly be formulated in terms of a 2-D stencil. Example 1.

Create the second-order centered approximation for

tion of the stencils for

∂2 ∂ y2

and

∂ ∂x

Multiplica-

(from Tables 1.2 and 1.1, respectively) gives

⎡

⎤ 1  ⎣ −2 ⎦ /h 2 × − 1 2 1

Example 2. ator

∂3 . ∂ x∂ y 2

0

1 2



⎡

1

−2 /h = ⎣ 1 1 −2

0 0 0

1 2

⎤

−1 ⎦ /h 3 . 1 2

Create second- and fourth-order approximation of the 2-D Laplace operL=

∂2 ∂2 + . ∂ x2 ∂ y2

Based on the top two lines in Table 1.2, the 1-D formulas (1.7) for and O(h 4 ) can be written

∂2 ∂ x2

of accuracies O(h 2 )

[1 − 2 1] /h 2 ,   5 4 1 1 4 − − − /h 2 . 12 3 2 3 12 Applying these operators in the x- and y-directions and adding the results leads directly to the 2-D counterparts (accurate to orders O(h 2 ) and O(h 4 ), respectively): ⎤ ⎡ 1 − 12 ⎡ ⎤ ⎥ ⎢ 4 ⎥ ⎢ 1 3 ⎢ 4 1 ⎥ ⎥ /h 2 . ⎣ 1 −4 1 ⎦ /h 2 and ⎢ − 1 4 (1.10) −5 − ⎢ 12 3 3 12 ⎥ ⎥ ⎢ 4 1 ⎦ ⎣ 3 1

− 12

PS methods pursue this stencil construction concept in Example 2 to still higher orders. However, in the context of solving PDEs, this approach might not be optimal, as seen in the examples of Section 1.2.2 and again (from a different perspective) in Section 3.3.11.3. 1.1.5.2 Hexagonal grids

With a specified distance h between adjacent nodes in 2-D, this grid type provides the densest possible node arrangement. However, boundary conditions may become more difficult to implement, and it also becomes less clear how to best approximate simple ∂ ∂ derivatives, such as ∂ x and ∂ y (since both of these cannot be aligned with the primary grid directions). Nevertheless, FD approximations can become pleasingly simple. For

Downloaded 11/04/15 to 132.239.1.231. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

1.2. Application of FD formulas to PDEs

example, the Laplacian operator

9 ∂2 ∂ x2

adjacent to (0, 0) located at (±h, 0), ⎡ ⎣ 1

+

∂2 ∂ y2

on a hexagonal node set with the six nodes

1 3 (± 2 h, ± 2 h)

1 1 −6 1 1

can be approximated by

⎤ 3 1 ⎦ / ( 2 h 2 ),

(1.11)

accurate to O(h 2 ). We will study the accuracy of this approximation further in Example 2, Section 1.2.2. Our main interest in hexagonal grids will arise later in the context of RBF and RBFFD methods. Both hexagonal and quasi-random (e.g., “minimal energy” distributed) node layouts then become very easy to work with, and it will transpire that they can offer advantages over Cartesian-based ones.

1.2 Application of FD formulas to PDEs The idea of using FD approximations for numerical solutions of PDEs was first proposed by L. F. Richardson in 1911 [221]. In a revolutionary paper, he notes, “Step-by-step arithmetical methods of solving ordinary differential equations have long been employed.” He then proceeds with generalizing this concept to 2-D. After a flawless equilibrium calculation of the stresses in a cross section of the first Aswan Dam over the Nile, he looks to the future: “The extension to three variables is, however, perfectly obvious. One has only to let the third variable be represented by the number of the page of a book of tracing paper.” Although our computational hardware is now far more powerful than pencil and paper, the basic concept of FD approximations for PDEs remains the same.

1.2.1 Time-dependent PDEs If a PDE is time dependent, ∂u = F (u, u x , uy , . . . , {maybe higher spatial derivatives}), ∂t the most straightforward FD approach is to place a Cartesian grid over the spatial domain. Given an approximation for u at time t , F is approximated at all interior nodes (with boundary information incorporated as needed), and an ODE solver is invoked to advance in time the resulting coupled ODE system (featuring a separate ODE for each node point). This general approach is known as the Method of Lines (MOL). Its main advantages include the following: • It is easy to reach high accuracy (and thereby high computational efficiency) in both space and time. • It is easy to interchange ODE solvers (such as explicit, implicit, different orders, etc.). • It reduces the need for user input with regard to time step selection, error control, etc.

Downloaded 11/04/15 to 132.239.1.231. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

10

Chapter 1. Brief Summary of Finite Difference Methods

Example 1. Create an FD scheme for the 1-D heat equation est possible stencil size. This FD approximation would become

∂u ∂t

=

∂ 2u ∂ x2

with the small-

u(x, t + k) − u(x, t ) u(x − h, t ) − 2u(x, t ) + u(x + h, t ) . = k h2

(1.12)

 . This amounts to an MOL    approximation, using the top line in Table 1.2 for the spatial approximation and then time stepping with the Forward Euler method (also known as AB1, or Adams-Bashforth of first order). The accuracy becomes O(k)+O(h 2 ); i.e., the scheme is first-order accurate in time and second-order accurate in space. This stencil can in the (x, t )-plane be illustrated as

The four key concepts with regard to FD approximations to time-dependent PDEs are convergence, accuracy, consistency, and stability. These are connected by the Lax Equivalence Theorem: For a well-posed linear problem, a consistent approximation is convergent if and only if it is stable. Consistency requires only that the FD formula, when k and h → 0, approximates the PDE; first-order accuracy in both directions suffices. Hence, stability (ensuring that the numerical approximation does not diverge to infinity within finite time) becomes the key property to test for. For an explicit time-stepping scheme, such as in Example 1 above, the numerical approximation at any fixed time t > 0 will diverge to infinity when the step-sized k and h 1 are decreased, unless a certain relation between k and h is satisfied, namely, k/h 2 ≤ 2 . There are three main methods to determine such stability conditions. Two of these will be described below. In cases of nonlinearities, variable coefficients, and nontrivial boundary conditions, a third approach—energy methods—is often the only available option. However, its generality usually comes at the price of significant algebraic complexity. A brief summary (focusing on the same case in Example 1) is given in [90], Appendix H. More extensive treatments can be found in [135, 222]. Even when the assumptions of the two approaches below are not fully satisfied, they still often give good indications when applied to suitably simplified equations (e.g., after linearization of nonlinear terms, assumption of spatial periodicity, etc.). The influence of boundary conditions on stability is discussed in [265]. 1.2.1.1 Von Neumann stability analysis

This approach is strictly applicable only on periodic or infinite domains and with constant coefficients in the PDE. At time level t , let the numerical solution be a combination of modes u(x, t ) = σ t /k e i ωx , where σ is the factor by which this mode grows in amplitude for each time  step. Substitution into (1.12) gives, after some simplifications, σ = 1 − k 2 ωh 4 h 2 sin 2 . For the solution to stay bounded, we need |σ| ≤ 1 to hold for all ω in the range |ω| ≤ π/h (all modes that can be represented on a grid with spacing h). The 1 stability condition thus becomes k/h 2 ≤ 2 (as σ, then, for all the ω-values, will become confined to the interval [−1, 1]). 1.2.1.2 ODE stability domains

Each numerical ODE integration technique has an associated stability domain—the region in a complex ξ -plane, with ξ = λk, for which the ODE method does not have any growing solutions when it is applied to the constant coefficient ODE, y  = λy .

(1.13)

Downloaded 11/04/15 to 132.239.1.231. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

1.2. Application of FD formulas to PDEs

11

Example 2. Find the stability domain for Forward Euler. This scheme, y(t + k) = y(t ) + ky  (t ), becomes in the case of (1.13) y(t + k) = (1 + λk)y(t ). With ξ = λk, the stability domain thus becomes |1 + ξ | ≤ 1, a circle of radius 1 centered at ξ = −1. Example 3. Find the stability domain for the third-order Adams-Bashforth method, as given by (1.9). As an approximation to (1.13), the scheme becomes y(t + k) = y(t ) + λk (23y(t ) − 16y(t − k) + 5y(t − 2k)). With ξ = λk, this linear recursion relation has as 12 its characteristic equation r 3 = (1 + 12r (r −1) . 23r 2 −16r +5 2

23ξ 12

)r 2 −

4ξ 3

r+

5ξ . 12

Next, we solve for ξ to obtain

ξ = The edge of the stability domain is traced out in the complex ξ -plane if we let r move around the edge of the unit circle. The result is seen as the p = 3 curve in Figure 1.2 a. For convenient reference, Figures 1.2–1.5 display the stability domains for a number of well-known ODE solvers. Both schemes in Figure 1.5 are enhancements to the secondorder accurate leapfrog (LF) scheme, which we in slightly abbreviated notation write as yn+1 = yn−1 + 2k f (tn , yn ) . In both subplots, the LF stability domain is shown as the line segment joining +i and −i. The Hyman method [152] includes a corrector step

yˆn+1 = yn−1 + 2k f (tn , yn ), 4 1 2 yn+1 = 5 yn + 5 yn−1 + 5 k ( f (tn+1 , yˆn+1 ) + 2 f (tn , yn )), resulting in third order of accuracy and a greatly enlarged stability domain (extending to 3 3 − 2 and ± 2 i along respective axes). LF with the Robert filter [225] proceeds by repeating the steps  yˆn+1 = yn−1 + 2k f (tn , yˆn ), = yˆn + γ (ˆ yn+1 − 2ˆ yn + yn−1 ), yn where γ is a positive parameter. This procedure requires only one function evaluation per time step. However, the accuracy drops to first order (although the actual degradation is small if γ is small). The approach is best suited for cases when eigenvalues stay close to the imaginary axis rather than extending far into the left half-plane. Time-staggered ODE solvers are discussed in [123, 124, 196, 271]; see also Section 2.1.5. For further discussions of the major classes of ODE solvers, a book specializing on the topic should be consulted, e.g., [136, 137, 170].

1.2.1.3 Stability analysis via ODE stability domains

We give here only one illustrative example. Example 4. Determine the stability condition for (1.12)—the same test case as with von Neumann stability above—but using the ODE stability domain approach. The scheme (1.12) amounts to using Forward Euler (also described as AB1) in time together with the spatial approximation, i.e., to solve the ODE system ⎤ ⎤ ⎡ ⎡ ⎤ ⎡ −2 1 1 u1 u1 ⎥ ⎢ u2 ⎥ ⎢ u2 ⎥ ⎢ 1 −2 1 ⎥ ⎥ ⎢ ⎢ ⎥ ⎢ ⎥ ⎢ . ⎥ .. ⎥ ⎢ . . . d ⎢ .. .. .. ⎥ 1/h 2 ⎢ .. ⎥ . ⎢ . ⎥=⎢ ⎥ ⎥ ⎢ ⎢ ⎥ dt ⎢ ⎢ . ⎥ ⎢ . ⎥ ⎢ .. .. .. ⎥ . ⎦ ⎣ ⎣ .. ⎦ ⎣ . . . . ⎦ un

1

1

−2

un

12

Chapter 1. Brief Summary of Finite Difference Methods Adams−Bashforth

Adams−Moulton

Downloaded 11/04/15 to 132.239.1.231. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

2

6 2

1.5 4 1

3 1

2 2

0.5

3

4

0

1

4

5 6

5 6

0

−0.5 −2 −1 −4 −1.5 −2 −2.5

−2

−1.5

−1

−0.5

0

0.5

−6

−6

−4

−2

0

2

Figure 1.2. Stability domains for (left) Adams-Bashforth (AB) and (right) Adams-Moulton (AM) methods of orders p = 1, 2, . . . , 6. The stability domains in all cases include the regions immediately to the left of the origin; i.e., for AM1, it is the domain |1 − ξ | ≥ 1, and, for AM2, the left half-plane. In all other cases, the regions are bounded. A section along the imaginary axis near the origin is included for AB methods of orders 3, 4, 7, 8, 11, 12, . . . and for AM of orders 1, 2, 5, 6, 9, 10, . . . . Note that the scale differs by a factor of three between the two figures. BD method

Detail near the origin

25 1.2 20

4

15

3

2

1 5

6

10 5

0.8

4

5 3 1 2

0

0.6

1

6

−5 0.4

−10 −15

0.2 −20 −25 −10

0

10

20

30

0 −0.5

0

0.5

Figure 1.3. Stability domains of backward differentiation (BD) methods of orders 1–6. The domains are in all cases outside of the shown boundary curves. (Left) The complete boundary curves. (Right) Detailed boundary structure near the origin.

The matrix is symmetric; i.e., all its eigenvalues are real. By Gershgorin’s theorem, they are located on the interval [−4, 0]. If the number of nodes n is even, inspection shows that [1, 1, . . . , 1]T is an eigenvector with eigenvalue 0 and [1, −1, 1, −1, . . . , −1]T with eigenvalue −4, so both interval bounds are sharp (n odd makes an insignificant difference). When including the 1/h 2 factor, the eigenvalues satisfy λ ∈ [−4, 0]/h 2 . This range has to fall within the ODE solver’s stability domain, which in this case is ξ = λk ∈ [−2, 0] (according to Example 2 above or by inspecting the p = 1 curve in Figure 1.2 a). We thus 1 need [−4, 0]/h 2 ∈ [−2, 0]/k, which simplifies to k/h 2 ≤ 2 .

1.2. Application of FD formulas to PDEs

13

Downloaded 11/04/15 to 132.239.1.231. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

AB(p−1) / AMp predictor / corrector

Runge−Kutta / Taylor series

3

6

2

4 10 9

2 3

1

2

4

8

7

6

5

5

4

3

2

6 0

0

−1

−2

−2

−4

−3 −3

−2

−1

−6 −6

0

−4

1

−2

0

Figure 1.4. Stability domains for (left) AB(p−1)/AMp predictor/corrector methods for p = 2, 3, . . . , 6. A section along the imaginary axis near the origin is included for AB(p − 1)/AMp methods of orders 3, 4, 7, 8, 11, 12, . . . and for ABp/AMp methods of orders 1, 2, 5, 6, 9, 10, . . . [125]. (Right) Solid curves: Runge–Kutta (RK) methods or orders (= number of stages) 1, 2, 3, 4. Solid and dash-dot curves: Taylor series methods of orders p = 1, 2, . . . , 10. Note that the scales differ by a factor of two between the plots.

Hyman’s method

Robert filters

2

1.5

1

1 0.5 γ = 0.2

0.5

0.15 0

0.1

0.05

0

−0.5 −0.5 −1

−1

−1.5

−2 −2

−1.5

−1

−0.5

0

−0.4

−0.3

−0.2

−0.1

0

0.1

Figure 1.5. Stability domains for (left) Hyman’s method, and (right) Leapfrog with Robert filter, for four values of the parameter γ . Note the different scales along the axes in the two subplots.

Downloaded 11/04/15 to 132.239.1.231. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

14

Chapter 1. Brief Summary of Finite Difference Methods

Figure 1.6. Schematic illustration of the CFL analysis for the stencil u(x,t )−u(x−h,t ) h

u(x,t +k)−u(x,t ) k

+

= 0.

In the later contexts of RBF and RBF-FD methods, the generally irregular node layouts will make von Neumann analysis impossible. In contrast, the ODE stability domain approach, together with a numerical calculation of the differentiation matrix’s eigenvalues, will become the outstanding time-stepping analysis tool. Comparison of the eigenvalue distributions with the different ODE methods’ stability domains will immediately give an excellent guide to both what time-stepping method to choose and then to the step sizes that can be used. 1.2.1.4 The Courant-Friedrichs-Lewy (CFL) condition

The two previous stability tests (von Neumann analysis and matrix eigenvalues together with ODE stability domains) have given necessary and (almost) sufficient stability conditions (some subtleties can arise if the DMs are nonnormal, i.e., can’t be diagonalized by a unitary matrix). The CFL condition [50] is an extraordinarily quick test that can show that some schemes are unconditionally unstable and others unstable for certain k-values, but it can never show a scheme to be stable. In heuristic terms, it states that a FD scheme (discrete in both space and time) must be unstable if the stencil shape does not allow information to flow with the speed required by the PDE. We apply next this test to two different schemes for the 1-D one-way wave equation ∂u ∂u + = 0. ∂t ∂x

(1.14)

 . Figure   1.6 illustrates the characteristic speed of the PDE (vP D E = 1; the velocity by which the solution travels in the (x, t )-plane) and the maximal speed by which information can travel sideways in the same direction in the numerical solution (vN U M = h/k). The condition vN U M ≥ vP D E tells us that, for k/h ≤ 1 (the stencil shape in the illustration), the scheme might be stable. For k/h ≥ 1, vN U M is not fast enough, and the scheme must be unstable. Example 5.

Use the CFL test for

u(x,t +k)−u(x,t ) u(x,t )−u(x−h,t ) + k h

= 0;

 . With   this stencil shape, vN U M = 0 for all combinations of k and h, and the scheme must therefore be unconditionally unstable. Example 6.

Use the CFL test for

u(x,t +k)−u(x,t ) u(x+h,t )−u(x,t ) + k h

= 0;

1.2. Application of FD formulas to PDEs

15

Downloaded 11/04/15 to 132.239.1.231. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

1.2.2 Elliptic-type PDEs We restrict our discussion here to the case of Poisson’s equation in 2-D ∂ 2u ∂ 2u + =f . ∂ x2 ∂ y2

(1.15)

Equations in this “elliptic” category arise in numerous situations, such as for a streamfunction in fluid mechanics or from field equations (describing gravitational and electrical fields, featuring potentials that satisfy Laplace’s equation; equation (1.15) with RHS zero). Another source of equations of this type is equilibrium processes. For example, (1.15) ∂u ∂ 2u ∂ 2u arises in the t → ∞ limit of the heat equation ∂ t = ∂ x 2 + ∂ y 2 − f . Example 1. Create the following compact fourth-order accurate approximation for the 2-D Poisson’s equation (a 2-D counterpart to (1.8)): ⎡

1 ⎣ 4 1

⎡ ⎤ 1 1 4 ⎦ u/(6h 2 ) = ⎣ 1 8 1 1

4 −20 4

⎤ 1 ⎦ f /12 + O(h 4 ).

To derive this, we follow Collatz’s Mehrstellenverfahren [49, 114]. Because of f , it also holds that

(1.16) 

∂2 ∂ x2

 ∂2 + ∂ y2 u =

 2 2  ∂ ∂2 ∂2 ∂2 u= f. + + ∂ x2 ∂ y2 ∂ x2 ∂ y2    

∂4 ∂ x4

∂4

∂4

+ 2 ∂ x2∂ y2 + ∂ y4

Approximation of these two relations to fourth and to second order, respectively, gives ⎡

⎤

1

− 12

⎢ ⎢ ⎢ ⎢ −1 ⎢ 12 ⎢ ⎣

4 3

4 3

−5

4 3 1 − 12

4 3

1

− 12

⎥ ⎥ ⎥ ⎥ u/h 2 = [ f ] + O(h 4 ) , ⎥ ⎥ ⎦

(1.17)

and ⎡ ⎢ ⎢ ⎢ 1 ⎢ ⎣

2 −8 2

1 −8 20 −8 1

respectively. Adding (1.16).

⎤

2 −8 2

1 2 h 12

⎡ ⎥ ⎥ 4 ⎣ 1 ⎥ ⎥ u/h = 1 ⎦

1 −4 1

⎤ 1 ⎦ f /h 2 + O(h 2 ) ,

(1.18)

times (1.18) to (1.17) eliminates the “outliers” and produces

Downloaded 11/04/15 to 132.239.1.231. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

16

Chapter 1. Brief Summary of Finite Difference Methods

The formula (1.16) achieves its fourth order only thanks to the stencil for f in the right-hand side (RHS). As an approximation to the Laplace operator, the left-hand side (LHS) of (1.16) is accurate only to second order, as seen by Taylor expanding it around the center point: ⎡

1 ⎣ 4 1

4 −20 4

⎤   2 1 ∂ ∂2 u + 4 ⎦ u/(6h 2 ) = ∂ x2 ∂ y2 1  2 2 1 ∂2 ∂ + h2 + u 12 ∂ x2 ∂ y2   4  ∂ 1 4 ∂2 ∂2 ∂4 ∂4 h u + + +4 + 360 ∂ x2 ∂ y2 ∂ x4 ∂ x 2∂ y 2 ∂ y 4  4    1 ∂ ∂4 ∂4 ∂4 ∂4 ∂4 6 + 3 u +4 + + 16 +3 h 60480 ∂ x4 ∂ x 2∂ y 2 ∂ y 4 ∂ x4 ∂ x2∂ y2 ∂ y4 + O(h 8 ).

For solutions to Laplace’s equation, the first three RHS terms vanish, and the approximation becomes sixth-order accurate. The two key advantages of (1.16) over (1.17) are the following: • The compact stencil is easier to use near boundaries. • The diagonal dominance of coefficient matrix improves numerical stability and speeds up iterative solution methods. Example 2. Analyze the accuracy of the hexagonal grid Laplace operator approximation (1.11). Series expansion in the same style as in the previous example gives ⎡ ⎣ 1

1 1 −6 1 1

⎤ 1 ⎦

 ∂2 ∂2 u + ∂ x2 ∂ y2  2 2 1 ∂ ∂2 + h2 + u 16 ∂ x2 ∂ y2   ∂6 ∂6 ∂6 ∂6 1 4 h 11 u + 15 + 45 +9 + 5760 ∂ x6 ∂ x4∂ y2 ∂ x2∂ y4 ∂ y6 

3 / ( 2 h 2) =

+ O(h 6 ). This expansion confirms that the approximation is only second-order accurate for the Laplace operator but shows that it supports a compact fourth-order approximation for (1.15). However, the accuracy improves no further for solutions to Laplace’s equation since the operator in the h 4 -term does not factorize. It thus falls short in this regard of the Cartesian grid compact 9-point operator analyzed in Example 1. Extending from 2-D to 3-D does not introduce any significant differences. For example, the 3-D counterpart to (1.16) becomes

1.2. Application of FD formulas to PDEs

Downloaded 11/04/15 to 132.239.1.231. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

⎡

⎤ ⎡ 1 0] 1] ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ − − ⎥ ⎢ ⎥ ⎢ 2 1] ⎥ ⎢ 2 ⎥ ⎢ 2] ⎥ u/(6h ) = ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ − − ⎥ ⎢ ⎢ 1 0] ⎥ ⎥ ⎢ ⎦ ⎣ 1]

17

⎤ [0 0 0] [1 [0 1 0] ⎢ ⎥ ⎢ ⎥ ⎢ [0 1 ⎥ [0 0 0] ⎢ ⎥ ⎢ − − − − − − − ⎥ ⎢ ⎥ ⎢ [0 1 0] ⎥ ⎢ ⎥ ⎢ ⎥ f /12 + O(h 4 ), [2 [1 6 1] ⎢ ⎥ ⎢ [1 2 ⎥ [0 1 0] ⎢ ⎥ ⎢ − − − − − − − ⎥ ⎢ ⎥ ⎢ [0 0 0] ⎥ ⎢ ⎥ ⎣ ⎦ [1 [0 1 0] [0 1 [0 0 0] (1.19) again combining fourth-order accuracy with diagonal dominance [305]. In this case, the 19-point stencil in the LHS is an O(h 2 ) accurate approximation to the Laplacian operator, which reaches O(h 4 ) for solutions to Laplace’s equation—as it does for the Poisson’s equation when used with the shown RHS stencil. The last several examples have shown that FD approximations can provide higher orders of accuracy for PDEs than the orders by which they approximate individual derivative operators. This issue will come up again in the context of RBF-FD methods. Numerous generalizations of the compact formulas mentioned above have been described in the literature, including to variable coefficients, inclusion of lower-order terms, extensions to the coupled streamfunction-vorticity system for both steady and time-dependent 2-D Navier-Stokes equations, etc. [62, 175, 134, 176]. [0 2 0] − [1 −24 1] − [0 2 0]

Downloaded 11/04/15 to 142.103.160.110. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

Chapter 2

Brief Summary of Pseudospectral Methods

This chapter provides a brief summary of pseudospectral (PS) methods, again more heuristic than rigorous, and with special emphasis on the aspects that will become important in the subsequent chapters on RBF and RBF-FD methods. Most items that are discussed below are covered in significantly more detail in [90]. Valuable sources for further reading also include [23, 266]. On topics related to Chebyshev-based interpolants, spectral accuracy, the Gibbs and Runge phenomena, etc., the recent book [267] is particularly recommended. There are several options for creating spectral approximations for ODEs and PDEs. Some of these approaches deal explicitly with expansions in basis functions, often chosen to obey certain orthogonality relations. We will here focus on the PS (collocation) approach: Enforcing some differential equation at a set of node points. We will introduce this approach in two quite different ways: (i) via increasing order limits of FD approximations, and (ii) via expansions in terms of basis functions. When viewing the PS method from a basis function perspective, certain operations (such as differentiation) are best done in spectral space, whereas other operations (such as multiplying two functions) are best done in physical space. Hence, when time stepping, one would typically transform the discrete data forwards and backwards between these spaces every time step (hence the name “pseudospectral”). Pseudospectral methods should not be confused with the entirely different concept of pseudospectra—which make eigenvalue-based analysis a key tool for analyzing/utilizing also non-Hermitian matrices (possibly with degenerate eigenspaces) [268, 269].

2.1 Periodic (Fourier) PS methods Sections 2.1.1 and 2.1.2 give two different introductions in the periodic PS case.

2.1.1 Introduction to the periodic PS method via FD approximations The bottom lines in Tables 1.1 and 1.2 indicate that, on an infinite equispaced grid, there exist limiting infinitely wide FD approximations of formally infinite order of accuracy. For the first and second derivatives, the entries can readily be written down explicitly. At 19

Downloaded 11/04/15 to 142.103.160.110. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

20

Chapter 2. Brief Summary of Pseudospectral Methods

Figure 2.1. Schematic illustration of equivalence between FD and periodic PS methods. Reprinted with permission from Cambridge University Press. [90] (1)

position j , the limiting weights w j for the first derivative become  (1) wj

=

and, for the second derivative, (2) wj

(−1) j +1 j

0 

=

if if

j = ±1, ±2, . . . , j = 0,

if

j = ±1, ±2, . . . ,

if

j = 0.

2(−1) j +1 j2 π2 − 3

In general, for the pth derivative, ( p)

wj

= (−1) p



d p sin πx d x p πx

 x= j

(derived in [89, Appendix 2]). When p is odd, the coefficients decay like O(1/ j ) and, when p is even, like O(1/ j 2 ). If a set of equispaced data is periodic, one can imagine extending it over the full real axis. It would then in principle be possible to apply one of the infinitely wide, infinite order difference stencils to obtain derivative approximations. Given the periodicity of the data, one could just as well have periodically collapsed the infinitely wide stencil to one extending only over the original period and applied this to the original (nonextended) data (cf. Figure 2.1). We assume for simplicity that the periodic interval is [−π, π] and that the nodes are located at xi = π(−1 + 2i/n), i = 0, 1, 2, . . . , n (i.e., we do not include separately the node at x = π). The approximations thus obtained become for the pth derivative ⎡ ⎤ ⎡ ⎤⎡ v ⎤ v0 0 p d ⎢ . ⎥ ⎣ ⎢ .. ⎥ ( p) ⎦ . = (2.1) D ⎣ . ⎦, ⎣ . ⎦ dxp vn vn

Downloaded 11/04/15 to 142.103.160.110. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

2.1. Periodic (Fourier) PS methods

21

where the entries for the differentiation matrices can be written down in closed form [131]. For example (with n even), p =1:

0 if i = j , (1) Di , j = 1 (i − j )h i−j (−1) cot otherwise, 2 2 and p = 2 :

 (2) Di , j

=

−

π2 1 − 6 3h 2 i − j +1

(−1)

 (i − j )h 2 / sin 2

if i = j , otherwise.

Similar formulas are readily available also for n odd [276] and for higher derivatives. When n is small, it is fast to carry out the matrix × vector multiplication indicated in ( p) (2.1). However, since the matrices Di , j are circulant, convolution by means of the FFT algorithm becomes a faster option when n is large. An easy way to implement that is described next in Section 2.1.2.

2.1.2 Introduction to the periodic PS method via Fourier expansions The FFT algorithm immediately converts equispaced periodic data to its discrete Fourier transform counterpart. One can then differentiate this trigonometric interpolant analytically and return to physical space with derivative approximations at all the node points. The equivalence between the FD limit approach above (Section 2.1.1) and differentiating the trigonometric interpolant was proven in [89], Appendix 3, and later extended on in [90]. With the FFT approach for calculating derivatives, it should be noted that codes allowing the data to be complex valued become particularly simple. In the usual case when the data is real valued, such a code would not be optimally efficient as it stands since it uses actual computations to arrive at a zero imaginary part vector in the answer. A very easy way to avoid that inefficiency is to run two separate cases together, i.e., place two independent real-valued vectors in the input vector’s real and imaginary parts, respectively. Although mixed while in Fourier space, the two cases become separated again at the end and can be read off as the output vector’s real and imaginary parts.

2.1.3 Convergence properties Most error analysis for PS methods in the literature has been focused on the case of smooth data—when Fourier expansions are quite rapidly convergent and error estimates can be based on the first omitted terms. It transpires that PS methods can also be highly effective when this is not the case. 2.1.3.1 Convergence for smooth functions

For smooth data, the discrete Fourier coefficients will decay rapidly, and error estimates become straightforward—typically of the general form O(e −c n ) for some c > 0. To see how the accuracy improves for FD p when p increases, we first note that a grid with spacing h can represent all Fourier modes e i ωx in the range [−π/h, π/h] (with still higher modes, due to aliasing, on the grid points becoming indistinguishable from a lower mode within this range). If we want to solve a wave-type PDE, we need to consider how

Downloaded 11/04/15 to 142.103.160.110. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

22

Chapter 2. Brief Summary of Pseudospectral Methods

Figure 2.2. Multiplicative factors arising when the pth order FD approximation for d/d x is applied to e i ωx . Reprinted with permission from Cambridge University Press. [90]

the spatial discretization method approximates

d i ωx e . dx

The exact result should be

d i ωx = iωe i ωx . e dx With centered second-order FD (abbreviated FD2), we get instead D (2) e i ωx =

sin ωh i ωx e i ω(x+h) − e i ω(x−h) =i e 2h h

and for FD p D

( p) i ωx

e

(2.2)

sin ωh =i h

 p/2−1  k=0

(k!)2 ωh 2 sin (2k + 1)! 2

!2k "

(2.3)

e i ωx .

(2.4)

Figure 2.2 displays the factors in front of e i ωx in (2.2)–(2.4), omitting the “i.” While the PS method is constructed to be exact for every mode that can be represented on the grid, lower-order FD methods treat well only a rather narrow range surrounding ω = 0. The order of accuracy p of a scheme matches the number of derivatives that are correct at ω = 0. Analysis of these curves, together with knowledge about the spectral content of initial conditions, can therefore provide good estimates for how errors will grow during time integrations [85]. 2.1.3.2 Convergence for nonsmooth functions

The best-known version of the Gibbs phenomenon is the overshoot that arises when a discontinuous function is represented by a truncated set of Fourier expansion terms. A similar situation occurs when a Fourier or a cubic spline interpolant is obtained by means of interpolation on an equispaced grid. Figure 2.3 illustrates these three cases. In the limits of increasingly many terms and of increasingly high node densities, respectively, the formulas for the peak heights are 1 1 ´ sin ξ + π 0π ξ d ξ ≈ 1.0895, a. Truncated Fourier series 2 # % k $ sin πξ ∞ (−1) b. Fourier interpolation max0 0, the replacement for the leading factor (e −||ω|| /(4 ) ) inside the integral in (3.6) will again be positive, and the positive definiteness proof will carry through just as in the GA case. This situation arises, e.g., for many types of compactly supported RBFs. 2

2

3.1. General background on RBFs

45

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

3.1.3.3 Completely monotone functions

A function is described as completely monotone if either of the following two equivalent definitions apply [44, 65, 241, 283]: Definition 3.1. A C ∞ (0, ∞) function ψ(r ), which has a bounded first derivative at the dk

origin, is said to be completely monotone if and only if (−1)k d r k ψ(r ) ≥ 0 for r > 0 and k = 0, 1, . . . Definition 3.2. A function ψ(r ), r ≥ 0 is said to be completely monotone if and only if its ´∞ inverse Laplace transform γ (s) is nonnegative (i.e., γ (s) ≥ 0 when ψ(r ) = 0 γ (s) e −s r d s). ´ Differentiation of ψ(r ) = 0∞ γ (s) e −s r d s immediately shows that Definition 3.2 implies Definition 3.1. The reverse direction is more difficult to show; see, e.g., [281, 283]. Among many theorems relating to completely monotone functions, we note in particular that products as well as nonnegative linear combinations of such functions again become completely monotone. From Definition 3.2 follows Theorem 3.3. Theorem 3.3. If ψ(r ) is completely monotone and not constant, then the RBF matrix A based on the radial function φ(r ) = ψ(r 2 ) will be positive definite. As an example of utilizing the two definitions and the theorem above, we consider 1 1 the generalized MQ case (GMQ) φ(r ) = (1+r 2 )β , i.e., ψ(r ) = (1+r )β . Considering Definition 3.1, one notes that

& k−1 ' . dk ψ(r ) = (β + i) /(1 + r )β+k (−1) drk i =0 k

≥ 0 if β > 0, k = 0, 1, . . . . This will therefore assure that A is positive definite for β > 0. Applying Definition 3.2, one obtains γ (s) = e −s s β−1 /Γ (β), which again is positive when β > 0, leading to the same result. A more direct nonsingularity approach, following the theme of Definition 3.2, pro ceeds as follows: Taking the inverse Laplace transform of φ( r ) for different radial functions φ(r ) gives formulas such as ˆ ∞ 1 2 IQ: = e −s e −s (r ) d s, 2 1 + (r ) ˆ0 ∞ −s 1 e 2 IMQ: ( = e −s (r ) d s. 2 πs 0 1 + (r ) Whenever the factor in front of e −s (r ) inside the integral is positive, we observe (using here IQ as an illustration) 2

αT A α =

n  n  j =1 k=1

ˆ =

∞

e 0

α j αk

−s

+

1 1 + 2 ||x

n  n  j =1 k=1

j

− x k ||2

α j αk e

− s 2 ||x j −x k ||2

, ds .

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

46

Chapter 3. Introduction to Radial Basis Functions

From the nonsingularity proof for GA RBFs, we know that the double sum is positive whenever the vector α = [α1 , α2 , . . . , αn ]T is not identically zero. Therefore, the integral and, with that, the quantity αT A α will also be positive; i.e., A is a positive definite matrix. 3.1.3.4 MQ RBFs

In the MQ case, it transpires that the (real and symmetric) A-matrix will not be positive definite but will instead have one positive and n − 1 negative eigenvalues—therefore again ensuring nonsingularity. This result was conjectured by Hardy already in his original proposal for RBF interpolation [143], and the first proof, by Micchelli in 1986 [195], remains one of the landmarks in the history of RBFs. Powell then (repeatedly) simplified the proof. We follow here [215]. Omitting  (which will have no influence on the following argument), we note that dk φ( r ) = ψ(r ) = 1 + r will obey (−1)k d r k ψ(r ) < 0 for r > 0 and k = 1, 2, . . . . Had the result held also for k = 0, then A would have been negative definite, according to the results and so large that ϕ(r ) < above. We thus consider ϕ(r ) = 1 + r −M , where M is positive/ 0 for all r -values of relevance. The matrix A with entries Ai , j = 1 + ||x i − x j ||2 − M is

therefore negative definite; i.e., v T A v < 0 for any nonzero$vector v. If we limit attention to the (n − 1)-dimensional space of vectors v such that ni=1 vi = 0, it also holds that $ $ v T A v = v T A v; i.e., A has n − 1 negative eigenvalues λi . Since ni=1 λi = ni=1 Ai ,i > 0, the last eigenvalue of A must be positive.

3.1.3.5 Inclusion of constant and/or polynomial terms

A common variation of (3.2) is to also include some low-order polynomial terms combined with certain matching constraints. In the case of including a degree-zero polynomial, we replace (3.2) by s(x) =

n 

λk φ(||x − x k ||) + γ1 , with the constraint

k=1

n 

λk = 0.

(3.7)

k=1

Including also linear terms would in 2-D amount to using s(x) =

n 

λk φ(||x − x k ||) + γ1 + (γ2 x + γ3 y) ,

k=1

with the constraints

n 

λk =

k=1

The counterpart to (3.3) then becomes ⎡  1 .. ⎢ ⎢ A  . ⎢ ⎢  1 ⎢ ⎢ − − − + − ⎢ ⎢ 1 ··· 1  ⎢ ⎣ x ··· x  1 n y1 · · · y n 

n 

λ k xk =

k=1

x1 .. . xn − 0

⎤⎡ y1 .. ⎥ ⎢ ⎢ . ⎥ ⎥⎢ ⎢ yn ⎥ ⎥⎢ ⎥ − ⎥⎢ ⎢ ⎥⎢ ⎥⎢ ⎦⎣

n 

λk yk = 0.

(3.8)

k=1

⎤ ⎡ λ1 .. ⎥ ⎢ ⎢ . ⎥ ⎥ ⎢ ⎢ λn ⎥ ⎥ ⎢ = ⎥ − ⎥ ⎢ ⎢ ⎢ γ1 ⎥ ⎥ ⎢ ⎣ ⎦ γ2 γ3

⎤ f1 .. ⎥ . ⎥ ⎥ fn ⎥ ⎥ , − ⎥ ⎥ ⎥ 0 ⎥ 0 ⎦ 0

(3.9)

where A is the same matrix as in (3.3). This generalizes immediately to higher dimensions and to higher-order polynomial degrees. There are several reasons for pursuing these types of generalizations:

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

3.1. General background on RBFs

47

1. The corresponding linear system matrices in (3.9) may represent positive (or negative) definite operators in the constrained parameter spaces, also when the original A-matrix lacked such properties (or might have been singular). In any number of dimensions, ( just including a constant will create negative definite operators for MQ φ(r ) = 1 + (r )2 . The situation with PHS-type RBFs is partly similar; see, e.g., [214]. However, nonsingularity theorems generally require the nodes not just to be distinct but also to be unisolvent with regard to the appended polynomial space. 2. Practical experience shows that already including a constant tends to improve the accuracy of derivative approximations, in particular, avoiding oscillatory representations of constant data. In the context of RBF-FD approximations, it is shown in Section 5.1.5 that including also some higher-order polynomial terms can greatly increase the resulting accuracy. 3. Including (at least) a constant may lessen stagnation errors (cf. Section 3.3.7). 4. Including constant and linear terms can improve the RBF accuracy at domain boundaries [92]. Elaborating further on the last point: Considering, e.g., cubic RBFs φ(r ) = r 3 , approximations based on (3.2) will typically diverge like O(||x||3 ) for ||x|| → ∞. With (3.7), farfield growth is reduced to O(||x||2 ) and, with (3.8), still further to O(||x||). As discussed further in Section 3.3.4, including low-order polynomial terms is one of the options that RBFs offer to reduce Runge phenomenon–type boundary oscillations without having to resort to node clustering.

3.1.4 Some convergence results 3.1.4.1 Lattices

The earliest convergence results for RBFs were given on infinite lattices [7, 37]. Some more recent results are noted in Sections 3.3.8–3.3.11. It is, however, important to recognize that results on infinite lattices in some cases have limited practical relevance. For example, the MQ convergence rate is sometimes quoted as O(h d +1 ) in d dimensions, contrary to the spectral accuracy that typically would be encountered. Other aspects for which infinite lattices give potentially misleading results include eigenvalue sizes (Section 3.2.1.2), possibilities of MQ divergence as  → 0 on finite lattices [55, 110], etc. Other observations on infinite (but not periodic) lattices give results consistent with the general case, such as the performance similarities between different smooth RBF types [27]. In general, great care has to be taken when interpreting grid/lattice-based RBF analysis since qualitative features also may differ between infinite, finite, and periodic lattice cases, and these in turn may differ from (computationally more relevant) results for nonstructured finite node sets. 3.1.4.2 Arbitrary domains

Theoretical studies demonstrating spectral (or better than algebraic) convergence in cases of smooth RBFs (in the absence of the Runge phenomenon) include [184, 185, 186, 296, 301]. Further discussions can be found in the [34, 65, 281]. RBF convergence in 1-D is studied in [207, 209] by tools closely related to polynomial approximation theory [267].

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

48

Chapter 3. Introduction to Radial Basis Functions

Figure 3.2. (a) A set of 41 scattered nodes in the unit circle. (b) The error in max norm when 1 1 the test function f (x, y) = 59/(67 + (x + 7 )2 + (y − 11 )2 ) is interpolated at these nodes using MQ RBFs, displayed as a function of the shape parameter . Reprinted with permission from Elsevier. [75]

3.2 Near-flat RBFs With  available as a free parameter, a most natural numerical test is to see how interpolation accuracy varies with this parameter. A typical outcome of such an experiment is seen in Figure 3.2. Especially for smooth functions, the interpolation error often decreases to remarkably low levels before the calculations suddenly break down [171, 235, 236, 260]. It is suggested in [236] that a fundamental trade-off is required between accuracy and numerical conditioning (described there as an uncertainty principle). The cause of these breakdowns for low  is readily seen to be numerical ill-conditioning in both of the two steps (3.3) and (3.2). Since the matrix A in (3.3) becomes increasingly ill-conditioned when  → 0, the expansion coefficients λk will become oscillatory and have large magnitude. That in turn will cause numerical cancellations when (3.2) is used to evaluate the interpolant s(x). The trend of the error curve in Figure 3.2 (b), before the breakdown occurs, suggests, however, that if somehow the numerical ill-conditioning could be bypassed, spectacular accuracies might become possible. The errors obviously cannot decrease all the way to zero (since data at a fixed number of node locations cannot contain an exact description of a general smooth function), but they might well decrease further still before some altogether different limiting factor enters. This section explores issues and opportunities that arise in connection with near-flat (or, in the  → 0 limit, of totally flat) basis functions. The crucially important main result in this context is that the uncertainty principle is misleading in the sense that the RBF interpolation problem does not itself become illconditioned in the flat basis function limit. However, the numerical steps of (3.3) followed by (3.2), often denoted “RBF-Direct,” amount then to an ill-conditioned numerical procedure for a completely well-conditioned problem. Several well-conditioned numerical algorithms will be described below.

3.2.1 The ill-conditioning of the A-matrix 3.2.1.1 Heuristic discussion

RBFs based on odd powers of r , e.g., φ(r ) = r or φ(r ) = r 3 , feature irregularities (discontinuities in a low-order derivative) at the center of their corresponding RBFs φ(||x||). Although this limits the accuracy that can be reached by the interpolant s(x), it can contribute to avoiding singular A-matrices. Even powers, such as φ(r ) = r 2 , φ(r ) = r 4 , etc., fail in this regard already in 1-D. Using, e.g., φ(r ) = r 2 , the interpolant (3.2) will take

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

3.2. Near-flat RBFs

49

Figure 3.3. (a) Legendre and Chebyshev polynomials of increasing orders. (b) Eleven equispaced translates of GA RBFs for four different values of . From top to bottom,  = 10, 1, 0.1, and 0.01. Reprinted with permission from Elsevier. [75]

$ the form s(x) = nk=1 λk (x − x k )2 . This is one single parabola, no matter how large n is, making it impossible to interpolate more than n = 3 points. Similarly, one finds that φ(r ) = a0 + a1 (r )2 + a2 (r )4 + · · · + a m (r )2m

(3.10)

2m+d

) points [55]. In the case of m = d = 1, in d -D can interpolate at most n = m+d ( m+d d we recover the previous result of n = 3. If we, for example, consider n = 300 nodes in d = 2 dimensions, RBFs based on (3.10) will thus produce singular A-matrices up through m = 16. If we, in place of (3.10), use, say, MQ, IQ, GA, etc., it thus falls on the higherorder terms a17 (r )34 + a18 (r )36 + · · · to “save” the A-matrix from being singular; i.e., O(34 )-sized changes in the O(1)-sized A-matrix entries suffice to make it singular. This estimate confirms the ill-conditioning, but it fails to fully demonstrate its actual severity. As we will see next, O(46 )-sized changes in the entries of the A-matrix suffice in this case to make it singular. 3.2.1.2 Eigenvalue structure

Results in [237] imply that the eigenvalues of the A-matrix will scale with specific powers of . The numerical calculations in [112] provide a more detailed picture. For example, with n = 51 scattered nodes in 2-D, the eigenvalues vary with  as seen in Figure 3.4(a) (computed using extended precision arithmetic). Regardless of the choice of RBF type (IQ, MQ, or GA), the eigenvalues form distinct groups, following the specific pattern {O(1)}, {O(2 ), O(2 )}, {O(4 ), O(4 ), O(4 )}, {O(6 ), O(6 ), O(6 ), O(6 )}, . . . (3.11) until the last eigenvalue is reached (causing the last group to possibly contain fewer eigenvalues than the general pattern would suggest). Different choices of scattered node locations x k make no difference in this regard. More concisely, we can write the eigenvalue pattern above as 1, 2, 3, 4, 5, 6, . . . , (3.12) indicating how many eigenvalues there are of orders 0 , 2 , 4 , 6 , 8 , 10 , etc. Given such 0n a pattern, one can immediately calculate the orders of both cond(A) and det(A) = λ . Doing so shows that, for n = 300, d = 2, the smallest eigenvalues are of size k=1 k O(46 ), i.e., cond(A) = O(−46 ). Corresponding results for different geometry types are

50

Chapter 3. Introduction to Radial Basis Functions 1{

0

1{

0

10

2{

3{

−20

−20

10

10 3{

5{

−40

−40

10

|Eigenvalues|

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

10

10

4{

5{

−60

7{

9{

−60

10

10 6{

11 {

−80

−80

10

10 7{

−100

10

13 { −100

10

8{

9{

−120

2{

−120

10

10 6{ −8

10

(a)

−6

10

−4

10

ε

−2

10

0

10

−8

10

(b)

−6

10

−4

10

ε

−2

10

0

10

Figure 3.4. Eigenvalues of the MQ RBF A-matrix in the 2-D n = 51 scattered node case, as functions of , computed using extended precision arithmetic. The number of eigenvalues in each of the different groups are also shown (easiest counted when displayed numerically rather than graphically). (a)  constant. (b) k is variable. Reprinted with permission from Elsevier. [112]

shown in Table 3.2 on the lines labeled “ constant.” Because cond(A) = O(1/{smallest eigenvalue}), we can readily convert the information in Table 3.2 to obtain cond(A) as a function of n, as shown in some typical cases in Table 3.3. For fixed n, conditioning is also seen to improve rapidly with increasing number of dimensions. Instead of using the same  at all the node points x k , one can consider using different values k at the nodes x k . Figure 3.4(b) shows that choosing k =  · {random numbers on [0,1]} and letting  → 0 for the same random n = 51 node set in 2-D—again qualitatively independent of node distribution details as long as lattices (nonunisolvency) are not involved—creates a different but equally distinct and clear eigenvalue pattern 1, 3, 5, 7, 9, 11, . . . .

(3.13)

In the n = 300 case discussed above, we thus find that with spatially variable k , cond(A) = O(−34 ), which we recognize from the heuristic discussion above. The results above have been verified for IQ, MQ, and GA RBFs in calculations extending to still higher values of n and also for numerous cases of different scattered node sets and random k distributions. The BE class of RBFs (cf. Section 3.2.3) shows, however, partly different patterns [112]. The patterns seen in Table 3.2 show that “k variable” is more favorable than “ constant” with regard to conditioning in the nonperiodic cases. Since there is no difference between the two eigenvalue sequences in the periodic (scattered node) cases (nodes on the periphery of a circle and on the surface of a sphere), it would seem that boundary effects

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

3.2. Near-flat RBFs

51

Table 3.2. Numbers of eigenvalues of different sizes (powers of ) for different geometries and types of shape parameter.

Geometry

Shape param.

Power of  = 0 2 4 6

8

10

12

14

...

1-D nonperiodic

 constant k variable  constant k variable  constant k variable  constant k variable  constant k variable

1 1 1 1 1 1 1 1 1 1

1 2 2 2 5 9 9 9 15 25

1 2 2 2 6 11 11 11 21 36

1 2 2 2 7 13 13 13 28 49

1 2 2 2 8 15 15 15 36 64

... ... ... ... ... ... ... ... ... ...

1-D on circle periph. (embedded in 2-D)

2-D nonperiodic On spherical surface (embedded in 3-D)

3-D nonperiodic

1 2 2 2 2 3 3 3 3 4

1 2 2 2 3 5 5 5 6 9

1 2 2 2 4 7 7 7 10 16

Table 3.3. Condition number c ond (A) = O(−α(n) ) with α(n) displayed for various values of n in all the  constant cases of Table 3.2.

Geometry

Number of nodes n = 1 10 100 1000 10000

100000

...

1-D nonperiodic 1-D on circle periph.

0 0

18 10

198 100

1998 1000

19998 10000

199998 100000

... ...

0 0

6 6

26 18

88 62

280 198

892 632

... ...

0

4

14

34

76

166

...

(embedded in 2-D)

2-D nonperiodic On spherical surface (embedded in 3-D)

3-D nonperiodic

are involved. Use of randomly variable k was later advocated for practical computing in [234]. The only use of spatially variable k that will be discussed in the present work arises in the context of local node refinement, in which case it greatly reduces spurious oscillations (cf. Section 3.3.3). The data in Table 3.2 showed that, even with spatially variable k -values, clearly defined eigenvalue patterns hold. One might have expected that irregular variations in the shape parameters k would lead to irregular variations in the eigenvalues of the A-matrix (compared to the constant  situation) and therefore that some of the extremely small eigenvalues might have become perturbed enough to change sign (with the possibility of becoming zero). Although examples have been given of that actually happening [112], the fact that even the very smallest eigenvalues show no irregular tendencies in the present type of display might suggest that singular systems still are not very likely to arise. There is much that remains to be fully understood in this area. As we will come across repeatedly in this monograph, placing nodes on lattices often degrades RBF performance with regard to both conditioning and accuracy. For example, for the three finite node layouts shown in Figure 3.5, the number sequences matching (3.12) become Halton Hexagonal Cartesian

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |15 1 2 3 4 5 6 7 8 9 10 11 12 13 14 |14 14 14 14 | 7 1 2 3 4 5 6 7 8 9 10 11 12 13 |12 11 10 9 8 7 6 5 4 3 2 1

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

52

Chapter 3. Introduction to Radial Basis Functions

Figure 3.5. Three examples of finite node layouts.

leading to condition numbers O(−34 ), O(−36 ), and O(−48 ), respectively. A short vertical line is inserted at each place where the patterns get broken. We can note that the conditioning in the hexagonal case is much better than for the Cartesian case and, although lattice based, comes close to the optimal rate. Chebyshev-type clustering in the xand y-directions of the Cartesian nodes makes no difference in the present context. For a description of the Halton nodes, see Appendix C.

3.2.2 Flat RBF limit 3.2.2.1 Limit theorem in 1-D

With a radial function φ(r ) = a0 + a1 (r )2 + a2 (r )4 + · · ·

(3.14)

and using n = 2 nodes x1 , x2 in 1-D, with associated data values f1 , f2 , one can readily solve the system Aλ = f to obtain ⎡

⎤

⎡

⎣ λ1 ⎦ = ⎣ λ2

f2 − f1 1 2 2(x1 −x2 )2 a1

+4

f1 − f2 1 2 2(x1 −x2 )2 a1

+4

1 1

1 1

f1 + f2 a0

+

f1 + f2 a0

+

2

2( f1 − f2 )a2 + O(2 ) a12 2 2( f1 − f2 )a2 + O(2 ) a12

⎤ ⎦.

1

The O( 2 )-terms have opposite sign, and the λ1,2 coefficients diverge to ±∞ when  → 0. Nevertheless, when one continues to calculate the interpolant, the divergent terms cancel out, giving s(x) =

2  i =1

λi φ(x − xi ) =

(x − x2 ) f1 + (x − x1 ) f2 + O(2 ). x1 − x2

(3.15)

This calculation required the simplifications a0 /a0 = 1 and a1 /a1 = 1, i.e., the assumptions a0 = 0, a1 = 0. With that the case, the RBF  → 0 limit becomes the standard linear interpolant. Trying the same thing with n = 3 nodes in 1-D gives, on the additional condition 6a0 a2 − a12 = 0, the interpolating parabola, etc. These results were generalized in [55] to the following. Theorem 3.4. Let n distinct data nodes be given in 1-D. Suppose that the radial function (3.14) is such that the RBF matrix A is nonsingular for all  > 0. For integer k, define the

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

3.2. Near-flat RBFs

53

symmetric matrices G2k−1 and G2k by ⎡

304

⎢ ⎢ ⎢ G2k−1 = ⎢ ⎢ ⎣  ⎡

324

a 3 02 4 0 a1 0

2k−2 0

.. .

a 3 24 4 1 a2 2 2k 2

ak−1

324

ak

344

a 3 14 4 1 a2 1

⎢ ⎢ ⎢ G2k = ⎢ .. ⎢ ⎣  . 2k ak−1 1

..  .

a 3 36 4 2 a3 3 

2k+2 3

.. .

ak+1



 2k−2 a 2k−2  k−1 2k ··· ak 2k−2 .. .  4k−4 a2k−2 ··· 2k−2 ···

··· ··· ···



  

2k a 2k−1 k 2k+2 ak+1 2k−1

4k−2 2k−1

.. .

⎤ ⎥ ⎥ ⎥ ⎥ ⎥ ⎦

, n×n

⎤ ⎥ ⎥ ⎥ ⎥ ⎥ ⎦

a2k−1

. n×n

If Gn−1 and Gn are nonsingular, then the RBF interpolant s(x) satisfies lim s(x) = Ln (x), →0

where Ln (x) is the Lagrange interpolating polynomial to the given data. The G-matrix nonsingularity conditions may look algebraically complicated, but they are satisfied for all our standard RBF choices. In the case of GA, they have been rigorously proven to hold [35] (also shown by T. Hrycak, personal communication, 2004). In fact, one then finds det(G1 ) = 1, det(G2 ) = −2, and det(Gk+1 ) = ((−2)k /k!) det(Gk−1 ), k = 2, 3, . . . . Theorem 3.4 has some very important consequences: • Each 1-D PS method can equivalently be obtained by using RBF interpolants and taking the  → 0 limit. Hence, every PS method (Fourier, Chebyshev, etc.) can be seen as a special case of an RBF method. • RBF methods generalize PS methods in fundamental ways, such as providing guaranteed nonsingularity also in multidimensional scattered node cases (thereby simplifying local node refinement, etc.). Furthermore, the  → 0 limit is usually not the best choice, nor do lattice type node layouts (required by PS methods) usually give the best accuracy. • The fact that Lagrange interpolants depend in a well-conditioned way on the input data f means that 1-D RBF interpolants will do the same as  → 0. The illconditioning of the two steps (3.3) and (3.2) is therefore merely a manifestation of RBF-Direct being an ill-conditioned numerical algorithm for a wellconditioned problem. From the last observation, it is clear that there ought to exist numerically stable algorithms for RBFs, also in their flat basis function limit. This observation prompted the development of such algorithms, described in Section 3.2.4. 3.2.2.2 Some higher-D results

It turns out that one can write down the RBF interpolant in closed form [111]. We consider first the case of cardinal data—taking the value one node point and zero at the other

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

54

Chapter 3. Introduction to Radial Basis Functions

nodes. Just like how Lagrange’s interpolation polynomial is created, the case with general data becomes then a linear superposition of such cardinal data cases. Theorem 3.5. Given cardinal data  1, i = k, fi = at nodes x i , i = 1, 2, . . . , n, 0 otherwise the RBF interpolant can be written sk (x) =

det(A(k) ) , det(A)

(3.16)

where A is the standard RBF matrix (with elements Ai , j = φ(||x i − x j ||)) and A(k) is the same (k)

matrix but with row k replaced by Ak, j = φ(||x − x j ||), j = 1, . . . , n. Proof. Expansion of det(A(k) ) along row k shows that sk (x) indeed becomes of the form (3.2). Also, sk (x k ) = 1 since the numerator and denominator in (3.16) then are the same, and sk (x i ) = 0 for i = k since det(A(k) ) then has two rows that are equal. An immediate consequence is the following. Theorem 3.6. If lim→0 s(x) exists, it will be a multivariate polynomial in x. Proof. Write x = {x1 , . . . , xd } and nodes x k = {x1,k , . . . , xd ,k }. Then substitute (3.14) together with r 2 = (x1 − x1,k )2 + · · · + (xd − xd ,k )2 into the determinants in (3.16) and expand in powers of . Given that the limit exists, both expansions will start with the same power of . The leading coefficient in the denominator will be a constant (dependent on the node locations), whereas the leading coefficient in the numerator will be a finite degree polynomial in the components of x = {x1 , . . . , xd }. For example, in the 1-D case that introduced Section 3.2.2.1, we thus obtain s(x) as  det s(x) =

 det

φ(|x − x1 |) φ(|x2 − x1 |)

φ(|x − x2 |) φ(|x2 − x2 |)

φ(|x1 − x1 |) φ(|x2 − x1 |)

φ(|x1 − x2 |) φ(|x2 − x2 |)



 det  f1 +

 det

φ(|x1 − x1 |) φ(|x1 − x2 |) φ(|x − x1 |) φ(|x − x2 |) φ(|x1 − x1 |) φ(|x1 − x2 |) φ(|x2 − x1 |) φ(|x2 − x2 |)

Substituting φ(r ) = a0 + a1 (r )2 + a2 (r )4 + O(6 ) into (3.17) extends (3.15) to s(x) =

  f2 . (3.17)

5 6 (x − x2 ) f1 + (x − x1 ) f2 2 a f − f2 a + (x−x1 )(x−x2 ) 2 1 (2x − x1 − x2 ) + 1 ( f1 + f2 ) +O(4 ). x1 − x2 a1 x1 − x2 2a0

All terms following the leading one will vanish at all the node points.

3.2.3 Nondivergence for GA and Bessel (BE) RBFs in the flat limit The clause “if lim→0 s(x) exists” in Theorem 3.6 raises the issue of whether convergence is to be expected. Placing n nodes along a straight line in d -D will typically result in the Lagrange interpolation polynomial along this line. We will next consider what will

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

3.2. Near-flat RBFs

55

happen off that line. For example, with n = 3 nodes located at x1 , x2 , x3 (along the x-axis in 2-D) with cardinal data values of {1, 0, 0}, the interpolant becomes s(x, y) =

a 2 − 2a0 a2 (x − x2 )(x − x3 ) y2 + 12 + O(2 ) (x1 − x2 )(x1 − x3 ) a1 − 6a0 a 2 (x1 − x2 )(x1 − x3 )

= {Lagrange} + O(y 2 ) + O(2 ) . The result for n = 4 nodes is similar, but with n = 5 nodes, one obtains instead ! 1 s(x, y) = y 2 O + {Lagrange} + O(2 ) , 2 1

where the numerator in the O( 2 ) term contains the factor a1 a22 − 3a12 a3 + 3a0 a2 a3 . Unless this expression is zero, the RBF interpolant diverges to infinity away from the xaxis when  → 0. Similar evaluations using still more points introduce terms of type 1 1 O( 4 ), O( 6 ), . . . and give a sequence of conditions for all these to vanish: n = 5, n = 6, n = 7, n = 8,

3

3

a1 a22 − 2 · 2 a12 a3 + 1 a0 a2 a3 = 0, 4 4 a2 a32 − 2 · 3 a22 a4 + 2 a1 a3 a4 = 0, 5 5 a3 a42 − 2 · 4 a32 a5 + 3 a2 a4 a5 = 0, 6 2 6 2 a4 a5 − 2 · 5 a4 a6 + 4 a3 a4 a6 = 0.

Assuming that the same pattern continues indefinitely, it was shown in [102] that this nonlinear recursion implies that (3.14) sums up to a BE radial function of the form φk (r ) =

Jk/2−1 (r ) (r )k/2−1

,

(3.18)

where k is arbitrary and Jα (·) denotes the J -Bessel function of order α (see Figure 3.6). In view of the relation Jδ (2 δ r ) 2 δ lim 2 δ! = e− r , δ→∞ (2 δ r ) the BE RBF class φk (r ) contains GA RBFs as a special (limiting) case. It can readily be shown directly that RBFs of this special BE/GA class will never diverge off a line if all data is on a line. For all other RBF types, nodes on a finite lattice (or on other nonunisolvent node sets) can lead to divergence when  → 0. It was shown in [237] that such divergence can never occur when using GA RBFs (no matter how distinct nodes are distributed). We conjectured in [102] that the BE class (3.18) is the largest class possible of RBFs for which such divergence cannot occur. The nonsingularity of the A-matrices for the BE class (noted already in [220, 241]) can be shown in the same way as for the GA case in Section 3.1.3.1 when using ˆ 1 e i x·ω d ω (3.19) φk (||x||) = (2π)k/2 ||ω||=1 in place of (3.5). The integration is here over the surface of the unit sphere in Rk (causing us in [102] to consider φk (r ) only for k = 1, 2, 3, . . .). The step in going from A semidefinite to A definite turns out to require k ≥ min(2, d ), where d as usual denotes the

56

Chapter 3. Introduction to Radial Basis Functions

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

0

−1

k=1

−2

−3 k=2 −4

−5 k=3 −6

−7 k=4 −8

−9 −50

−40

−30

−20

−10

0 ε*r

10

20

Figure 3.6. Illustration of the BE radial functions φk (r ) =

30 Jk/2−1 (r ) (r )k/2−1

40

50

, k = 1, 2, 3, 4.

dimensionality of the approximation space. It furthermore turns out that using k = d leads to some strange complications, leaving the practical choices to k = d + 1, d + 2, . . . . There are several additional situations in RBF theory where the BE class turns out to have unique properties [72, 100, 238], some related to the fact that they have compact support in Fourier space (as follows from (3.19)). Although BE RBFs have been recommended over other RBF types in certain applications [156, 245], their overall numerical performance remains unclear [248]. Given the interest that φ(r ) = sinc r = (sin r )/(r ) basis functions have attracted in (mostly equispaced) 1-D contexts [255, 256], it may be of interest to note that φ3 (r ) = sinc r ; i.e., some properties of sinc basis functions follow from the general properties of the BE RBF class. Figure 3.7 illustrates how various RBF types can act very differently in the  → 0 limit. Apart from what is noted above, only a few theoretical results are available [35, 36, 174]. One can also consider flat limits of nonsmooth radial functions. It is shown in [173, 250] that interpolants then converge to (also nonsmooth) polyharmonic spline functions. 3.2.3.1 Brief remarks on oscillatory radial functions

For any fixed number of dimensions d , it is trivial to find examples of oscillatory radial functions φ(r ) such that nonsingularity is assured for interpolation. Based on the observation in Section 3.1.3.2, it suffices to choose any nonnegative function g (||ω||) and then select φ(r ) as the radial variation in its inverse Fourier transform (i.e., its Hankel transform). A noteworthy result is that if φ(r ) for any r > 0 differs in sign from φ(0), then this radial function cannot feature guaranteed nonsingularity in arbitrarily high dimensions (in contrast to the situation with the standard RBF choices, such as GA, MQ, IMQ, IQ, SH, etc.; cf. Table 3.1). This can either be shown directly [102, Theorem 2.3] or be deduced from more general results [65, Theorem 3.9]. The BE class appears to be the only class of oscillatory radial functions that has been shown to possess especially noteworthy features.

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

3.2. Near-flat RBFs

57

Figure 3.7. Illustrations of some simple 2-D node layouts, with a summary of the qualitative properties of interpolants when using increasingly flat RBFs. J0 denotes the Bessel RBF φ2 (r ) = J0 (r ).

3.2.4 Computational approaches for near-flat basis functions As noted above, (i) near-flat basis functions often provide especially accurate results, and (ii) RBF-Direct (immediate use of (3.3) followed by (3.2)) in that case becomes a numerically ill-conditioned approach for solving a genuinely well-conditioned problem. The obvious question therefore becomes how to devise algorithms that, for small , are both numerically well-conditioned and computationally efficient. The next several subsections first provide an overview of some concepts and then focus on the main present types of stable algorithms. 3.2.4.1 Overview of some computational options

The most straightforward and, in terms of coding, also the simplest approach to address the numerical ill-conditioning with RBF-Direct is to resort to extended precision arithmetic [45, 148, 233]. The only drawback with this is computational cost (in terms of both computer time and memory usage). Given the results in Section 3.2.1, one can readily determine in advance just how many digits of precision would be needed as function of n and  in various geometrical settings. Although extended precision arithmetic can never reach the actual limit of  → 0, already quadruple (quad) precision (128 bit) floating point might suffice in certain nonchallenging cases (such as for the fairly low N -values typical for RBF-FD stencils). Several decades ago, quad precision hardware was included with many scientific mainframe computers (such as CDC and IBM System 370 machines), giving a cost penalty factor only around 4 to 8. When extended precision is available only by means of emulation software, the cost penalty already for stepping past the hardwareprovided double precision (64 bit) is more likely to be a factor in the 40 to 200 range (and growing still further with increased precision requirements). For MATLAB users, the presently fastest option is provided by an add-on package developed by Advanpix, including specially optimized code for quad precision. The sole purpose of stable algorithms is to reach accurate final results (RBF interpolation values, derivative approximations, RBF-FD weights, etc.) faster than what extended precision offers. This is achieved via computational paths that are genuinely numerically well-conditioned all the way into the  → 0 limit (and therefore require only standard dou-

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

58

Chapter 3. Introduction to Radial Basis Functions

ble precision arithmetic, no matter how small  is). We will require of a stable algorithm that its results match those produced by extended precision arithmetic. Before describing some stable algorithms, we note that certain types of preconditioning/SVD variations of the RBF-Direct approach have been suggested in the literature. Their greatest benefits may, however, be in speeding up certain iterative procedures [10, 58, 59]; see also Chapter 34 in [65]. Approaches suggested in [166, 61] can have stabilizing effects in the case of small , but it is not clear how they resolve the issue that significant information/digits then becomes lost already when the coefficient matrix A is formed (with all A-entries virtually the same), making the recovery of such missing information challenging or impossible. Formulated differently [38], “this algorithm cannot be applied in general to large scale problems, because the technique that eliminates the singular values simultaneously removes their respective singular eigenvectors, degrading the basis of the space.” Still another stabilization-type approach is based on Riley’s algorithm [223]. The theoretically positive definite A-matrix is then changed to a numerically “safely” positive definite one, A + μI , before proceeding with Cholesky factorization. To some extent, the error caused by using μ > 0 can be compensated for by an iterative improvement process. Some benefits of this approach were reported in [5]. A related regularization approach is provided by Tikhonov’s method [140]. In this case, the system Aλ = f (3.3) is replaced by minimizing ||Aλ − f ||2 + μ||λ||2 , where μ is a small parameter. This can be conveniently carried out by using backslash in MATLAB to solve the overdetermined linear system ⎤ ⎤ ⎡ ⎡   f A ⎣ −−− ⎦ λ = ⎣ − ⎦. μI 0 So far, two main classes of stable algorithms have been proposed. The first realizations of these were denoted Contour-Padé [110] and RBF-QR [107], respectively, and will be described next. 3.2.4.2 Concept behind the Contour-Padé algorithm

Figure 3.8 shows a very small test problem that we will use to illustrate the Contour-Padé concept. Focusing at first on the solid RBF-Direct curve, cond(A) is seen to “explode” in size for  → 0, thus making the RBF-Direct approach unacceptable. Although the shape parameter  has so far been considered to be a real-valued quantity, nothing stops us from extending it also to complex values. When considering (3.16), it becomes apparent that a GA RBF interpolant s(x, ), for any fixed evaluation point x = (x, y), then becomes a meromorphic function of  (analytic, with poles as its only singularities across the finite complex -plane). This is because, in (3.16), both the numerator and the denominator can be expanded out as sums of exponentials—both becoming entire functions of . The only singularities in the ratio of numerator/denominator will thus be poles at locations in the -plane that come from zeros in the denominator. These will depend only on the node locations (and not on the evaluation point x). Figure 3.9(left) extends Figure 3.8 to complex ; the front edge exactly matches the solid curve in Figure 3.8(right). We next recall that s(x, 0) for this GA case is known to be finite even as  → 0. The origin  = 0 must therefore be a removable singularity of s(x, ) (as it cannot be a pole, and no other singularity types are possible for meromorphic functions). The massive growth seen in Figure 3.9(left) near  = 0 for cond(A) is thus a manifestation of the illconditioning of RBF-Direct and not an irregularity of the RBF interpolant s(x, ). We also see a large number of very thin spikes. These represent actual poles for s(x, ). As

3.2. Near-flat RBFs

59 GA A−matrix condition number

log10(cond(A))

15

0.5 y

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

Halton nodes 1

0 −0.5 −1 −1

0 x

1

RBF−Direct RBF−GA

10 5 0

0

0.5

1 ε

1.5

2

Figure 3.8. (Left) A set of N = 21 Halton nodes distributed across the square [−1, 1]×[−1, 1]. (Right) log10 cond(A) as a function of  when using GA RBFs; Solid curve RBF-Direct, dotted curve RBFGA.

Figure 3.9. (Left) log 10 cond(A) for the same test case as in Figure 3.8(left) but shown for  in the first quadrant in the complex -plane rather than only along the real -axis. (Right) Same contour curves as in the left figure, but displayed from straight above.

noted above, the locations of these depend only on the A-matrix, i.e., on the data locations x i , i = 1, 2, . . . , N , and not on the evaluation point x. By the nonsingularity result in Section 3.1.3.1 and the knowledge that s(x, 0) is finite, none of these poles can be on the real -axis. Any algorithm that matches an extended precision implementation of RBFDirect will therefore feature exactly the same distribution of the poles in the complex plane. Figure 3.10 (computed with the RBF-GA algorithm [104], described in Section 3.2.4.8) should be compared to Figure 3.9. No visible trace remains of the ill-conditioning around  = 0. However, all the additional singularities indeed remain unchanged (since these represent genuine singularities of the RBF interpolation problem). The front edge of Figure 3.10(left) is shown as the dotted curve in Figure 3.8(right). 3.2.4.3 Some technical aspects of the Contour-Padé algorithm

If Figure 3.9 is extended from the first quadrant to the full complex -plane, it becomes symmetric with respect to both axes. If we similarly display the magnitude of the (true) GA interpolant s(x, ), these symmetries will again hold. That picture would be very similar, but with the fundamental difference that the massive growth at the origin would

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

60

Chapter 3. Introduction to Radial Basis Functions

Figure 3.10. (Left) log10 cond(A) for the same test case as in Figures 3.8 and 3.9, having used the RBF-GA approach to first form a different linear combination of the original GA basis functions. (Right) Same contour curves as in the left figure but displayed from straight above. All genuine singularities of the original A-matrix remain unchanged, but the ill-conditioning for small  is entirely removed.

be gone (however, with all the thin spikes remaining, i.e., reminiscent of Figure 3.10). To obtain s(x, ) very close to (or right at)  = 0, we thus first compute s(x, ) around the periphery of a circle centered at  = 0, which goes sufficiently far from the ill-conditioned central region that RBF-Direct can be safely used. In the simplest case with no poles inside this circle, it then suffices to note that the value of an analytic function at the center of a circle is exactly the average of the values around its periphery. Figure 3.11 illustrates the concept further. Attempting to accurately interpolate a value for s(x, )|=0 from data only along the -axis (Figure 3.11(a)) is hopeless since the central section lacking any reliable data is wide. In total contrast, the mean value result for analytic functions (Figure 3.11(b)) is exact no matter the circle size. The main issues that a practical algorithm based on this observation has to address are the following: 1. The possible appearance of genuine poles inside the chosen contour (schematically illustrated in Figure 3.11(c); in Figures 3.9 and 3.10, we see one pole somewhat near the origin, at  ≈ 0.7i). 2. Evaluation of s(x, ) not just at  = 0 but for arbitrary -values inside the circle. 3. Improving efficiency/stability by utilizing the fact that the pole picture is identically the same for all evaluation points x. 4. Algorithmic robustness if the chosen path by chance would happen to go right through a pole. In turn, we have the following issues: Issues 1 and 2: Instead of just taking the average of the equally spaced s(x, ) values around the circle, we apply the DFT (FFT) to them, obtaining approximations to the Laurent expansion in a circular annulus surrounding the evaluation path (due to Cauchy’s integral formula). This is followed by converting the Laurent coefficients associated with negative powers to rational form by means of a Padé conversion. The end result of these

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

3.2. Near-flat RBFs

61

Figure 3.11. Conceptual idea behind the Contour-Padé and its RBF-RA follow-up. (a) Situation along the real -axis. (b) Idealized situation in complex -plane. (c) Typical situation in complex -plane, with black dots representing poles.

two steps will take the form

  Rational approximation descriTaylor expansion centered at the s(x, ) = + . bing the poles inside the circle origin and valid past the circle (3.20) This form is well suited for numerical evaluation of s(x, ) throughout the entire illconditioned region surrounding  = 0. Issue 3: The fact that the denominator of the rational part of (3.20) depends only on the node locations x i and not on the evaluation locations x allows for it to be reused, for some cost savings. Issue 4: “Trial-and-error” testing can be used to provide a circular path that avoided the poles; this path can then be reused for any number of evaluation locations. 3.2.4.4 Further developments on the Contour-Padé concept

The original Contour-Padé algorithm is today mainly of historical interest, having established the feasibility of stable algorithms. There are two main directions in which it has recently been improved. Common to them is to replace the FFT, followed by the subsequent steps leading to (3.20) by more directly obtained rational approximations. While this offers several advantages, it does not overcome the rather severe restrictions on N (to about 80 in 2-D, higher in 3-D). When N is increased further, the central area (with disastrous ill-conditioning) will grow in size and force the RBF-Direct evaluation points further out, where they will run into the mess of poles we see the beginning of toward the upper right corners in Figures 3.9 and 3.10. Gonnet et al. [129] advocate directly approximating s(x, ) by a rational function s(x, ) ≈ pn ()/q m (), where typically n  m. The (x-dependent) coefficients are determined by enforcing equality at equispaced locations along similar circles as used in the Contour-Padé version. It was found that FFT-based aliasing inaccuracies (very high frequencies at the two ends of the spectrum becoming indistinguishable from each other)

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

62

Chapter 3. Introduction to Radial Basis Functions

Figure 3.12. Two examples of “good” approximation spaces, showing for each both a good and a bad set of basis functions.

could be much reduced, offering similar accuracies with fewer evaluation points. A difficulty arises, however, in the emergence of Froissart doublets, near-equal factors in pn () and q m () that mostly cancel each other but not exactly. These factors not only are computationally wasteful (as they lower the degrees of the parts of pn () and q m () that represent actual information) but also produce very narrow spikes in the rational approximations when their cancellation is not completely perfect. A strategy is given in [129] that greatly reduces their presence. In the RBF-RA approach [293], approximations again are sought in the form of s(x, ) ≈ pn ()/q m (). However, this algorithm couples the calculations for a large number of evaluation points x in order to form a large, sparse, and highly structured overdetermined linear system in which the x-independence of q m () allows its coefficients to enter into it only once. This was found to increase “numerical robustness” and also to mostly eliminate Froissart doublets. Scaling different equations in a least squares system can be used to increase or decrease the significance of individual equations. This observation is readily turned into a procedure that “scales away” equations corresponding to evaluations near or even right at complex -plane poles, making it unnecessary to take any additional precautions with regard to Issue 4. 3.2.4.5 RBF-QR concept

As a conceptual background to this approach, Figure 3.12 illustrates that one must separate carefully between a space and a basis that spans it. In both of the displayed examples, it is easy to design both a bad (ill-conditioned) and a good (well-conditioned) basis. Turning to RBFs, the translates of near-flat RBFs clearly form a basis that is exceedingly ill suited for immediate numerical use. This naturally raises the question of whether the conditioning issues can be resolved by finding an alternate good basis in exactly the same space. This turns out to hold true, leading to the follow-up issue of how one can carry out the basis conversion so that no numerical cancellations will arise in the process. The RBF-QR method offers a systematic approach for this. 3.2.4.6 RBF-QR for nodes on the surface of a sphere

The first case for which the RBF-QR concept was successfully implemented was for nodes scattered over the surface of a sphere [107], still the main geometry for which it is directly applicable with all the standard RBF choices (includes also, e.g., periodic cases, implemented according to (3.34)). The cost for n nodes scales as O(n 3 ) (like for RBF-Direct), easily allowing n-values in the thousands.

3.2. Near-flat RBFs

63

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

Table 3.4. SPH expansion coefficients corresponding to some different choices of smooth RBFs.

RBF MQ

Definition ( 1 + (r )2

IMQ

(

1 + (r )2 1 1 + (r )2

IQ GA

1

e −(r )

2

Expansion coefficients cμ,  −2π(22 +1+(μ+1/2) 1+42 ) (μ+3/2)(μ+1/2)(μ−1/2)

4π (μ+1/2)



2μ+1 2 2 1+ 4 +1

2μ+1 2 1+ 42 +1

4 π3/2 μ! 3 Γ (μ+ 2 )(1+42 )μ+1

42 2 F1 (μ + 1, μ + 1; 2μ + 2; 1+42 )

4π3/2 −22 e Iμ+1/2 (22 ) 2μ+1

A SPH expansion of an arbitrary function defined over the surface of the unit sphere takes the form μ ∞   s(x) = dμ,ν Yμν (x) μ=0 ν=−μ

(cf. Appendix B). Truncated SPH expansions (μ ≤ μmax ) feature a completely uniform resolution over the surface of the sphere. Explicit formulas are available for the coefficients when expanding a RBF centered at an arbitrary location x i on the surface of the unit sphere: μ ∞    φ(- x − x i -) = {cμ, 2μ Yμν (x i )} Yμν (x), (3.21) μ=0 ν=−μ

$ where the symbol  implies halving the ν = 0 term of the sum [116, 150]. The resulting coefficients cμ, in the cases of MQ, IMQ, IQ, and GA are shown in Table 3.4. The equation (3.21) is not quite a Taylor expansion in  since the coefficients cμ, also depend on . A key feature of these formulas is that, even for  vanishingly small, all coefficients cμ, can be calculated without any danger of losing significant digits (assuming in the GA case that the removable singularity of Iμ+1/2 (22 )/2μ+1 at  = 0 is handled appropriately). The result when (3.21) is applied in turn to the n RBFs, centered on the surface of the sphere, can be written in matrix × vector form: ⎡ ⎢ ⎢ ⎢ ⎣

- x − x -) φ(1φ(- x − x 2 -) .. . φ(- x − x -)

⎤ ⎥ ⎥ ⎥ ⎦

⎡

n

⎡

c0, Y00 (x 1 ) 2 c0, Y00 (x 2 ) 2

⎢ ⎢ ⎢ =⎢ ⎢ ... ⎣

c0, Y00 (x n ) 2

= B ·Y.

2 c1, Y1−1 (x 1 ) 1 2 c1, Y1−1 (x 2 ) 1

2 c1, Y10 (x 1 ) 2 2 c1, Y10 (x 2 ) 2

2 c1, Y11 (x 1 ) 1 2 c1,Y 1 (x ) 1

2

4 {}.. ...

n)

4 {}..

1

...

...

...

2 c1, Y1−1 (x n ) 1

2 c1, Y10 (x n ) 2

2 c1,Y 1 (x 1

1

4 {}..

⎤ ⎥ ⎥ ⎥ ⎥ ⎥ ⎦

⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎣

Y00 (x) Y1−1 (x) Y10 (x) Y11 (x) Y2−2 (x) Y2−1 (x) Y20 (x) Y21 (x) Y22 (x) .. .

⎤ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎦

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

64

Chapter 3. Introduction to Radial Basis Functions

Following [107], we next note that all powers of  in B can be factored out in the form of a diagonal matrix E to the right of B. Next, a QR factorization produces B = QR1 E = Q(E E −1 )R1 E = QE(E −1 R1 E) = QE R, i.e., ⎡ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎣

- x − x -) φ(1φ(- x − x 2 -) φ( x − x 3 -) .. . φ(- x − x n -)

⎤

⎡

⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎥ = [Q] [E] × ⎢ ⎢ ⎥ ⎢ ⎥ ⎣ ⎦

∗

. ∗

. ∗ ∗

. ∗ ∗ ∗

. . . . ∗

. . R . ∗ .

.

.

. . . . ∗

. . . . ∗

... ... ... ... ...

...

...

...

⎤⎡ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎦⎢ ⎣

.. . .. .

⎤

⎥ ⎥ ⎥ ⎥ Y ⎥ ⎥. .. ⎥ . ⎥ ⎦ .. .

Here Q is a unitary and E is a diagonal matrix (with powers of  in the diagonal). The entries marked as “*” in the matrix R are of size O(0 ). All the other nonzero entries of R are of size O(2 ) or beyond. The new basis is given by the elements of R · Y (x). It differs from the original basis only by having omitted the nonsingular matrix Q · E from its left side; i.e., it forms a different basis for the same space. The key feature of the algorithm is that the E-matrix, which contains all the ill-conditioning, disappeared analytically from the problem and has not in any way damaged the accuracy of the equivalent (but wellconditioned) base R · Y (x). A MATLAB implementation of this algorithm is given in the Appendix of [107]. A subtle point that is essential for the algorithm to succeed is that the number of terms of successive powers O(1), O(2 ), O(4 ), O(6 ), . . . in (3.21) follows the pattern {1, 3, 5, 7, . . .}—and that this exactly matches the sequence labeled “On spherical surface” in Table 3.2. Had there been any discrepancy between the two sequences, the procedure would not have eliminated all the ill-conditioning in the  → 0 limit. We denote this requirement as the counting condition. The main difficulty when generalizing RBF-QR to other geometries and to other RBF types is to get this counting condition to hold. Figure 3.13 illustrates a typical comparison between RBF-QR and RBF-Direct for interpolating scattered data over the unit sphere. The RBF-QR error that is seen here for low  reflects the genuine difference between the test function and its RBF interpolant and is not a reflection of the 10−16 accuracy of the used double precision arithmetic. In contrast, the RBF-Direct curve deviates at low  due to the fact that this arithmetic precision then becomes insufficient in view of the fact that, for n = 1849, cond(A) = O(−84 ) (cf. Section 3.2.1.2). Lowering  by a factor of 100 (about what is needed to “safely” reach the optimal -range) would here increase cond(A) by a factor of 10084 = 10168 ; i.e., the arithmetic precision would, for RBF-Direct, need to be raised from 16 to about 180 digits. Quad precision (about 34 digits) is thus entirely insufficient.

3.2.4.7 RBF-QR for general domains

Implementations of RBF-QR for general node sets in 1-D, 2-D, 3-D, etc. proved to be more difficult, but some variations are now available [66, 101] and will be discussed below. We will follow [101] and focus the description on 2-D. The key issue becomes to find a

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

3.2. Near-flat RBFs

65

1

1

1

−7(x+ )2 −8(y+ )2 −9(z− )2

2 2 2 . (b) n = 1849 ME (miniFigure 3.13. (a) Test function f (x) = e mal energy) nodes. (c) MQ interpolation errors when using RBF-Direct vs. RBF-QR. Reprinted with permission from Cambridge University Press. [96]

counterpart to (3.21) that again satisfies the counting condition. So far, such expansions have been found in explicit form only for GA and BE RBFs. In the case of GA, the radial 2 function φ(r ) = e − ( r ) centered at the point (xi , yi ) becomes ψ(x, y, xi , yi ) = e − ((x−xi ) +(y−yi ) ) 2

= e −

2

2

2

(xi2 +yi2 )

· e − (x 2

2

(3.22) +y ) 2

· e 2 (x xi + y yi ) . 2

Only the last factor above mixes (x, y) and (xi , yi ) values. It has the Taylor expansion e 2 (x xi + y yi ) = 1 + 22 (x xi + y yi ) + 2

22 4 (x xi + y yi )2 + . . . . 2!

(3.23)

Thanks to having factored out e − (x +y ) (a “harmless” factor as  → 0), the degrees of the polynomials in the subsequent Taylor expansion increase by just one order at a time. The expansion functions thus become 2

e − (x 2

2

+y 2 )

2

2

· {{1}, {x, y}, {x 2 , xy, y 2 }, {x 3 , x 2 y, xy 2 , y 3 }, . . .}

(3.24)

with associated powers of  matching (3.11). While a RBF-QR implementation based on (3.24) will resolve issues related to  → 0, another problem arises in that high-order monomials tend to become linearly dependent as their degrees increase. This difficulty can be resolved by means of two further changes of basis. Conversion to polar coordinates: With ψ(r, θ, ri , θi ) denoting a e − r radial function centered at the polar coordinate location (ri , θi ), its value at the location (r, θ) follows 2 2 2 2 2 from rewriting (3.22) as e − ri · e − r · e 2 ri r (cos θi cos θ+sin θi sin θ) : 2 2

ψ(r, θ, ri , θi ) = 2 · e − ri · e − r    1 1 · 0 ri0 r 0 · 2 0!0!   1 1 1 · + Ψ2 + 4 ri2 r 2 2 1!1! 2!0! 2 2

2 2

(3.25)

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

66

Chapter 3. Introduction to Radial Basis Functions

+ 8 ri4 r 4 + ···+



1 1 1 1 · + Ψ2 + Ψ 2 2!2! 3!1! 4!0! 4



 1 Ψ1 1!0!   1 1 + 6 ri3 r 3 Ψ1 + Ψ3 2!1! 3!0!   1 1 1 10 5 5 +  ri r Ψ1 + Ψ3 + Ψ5 3!2! 4!1! 5!0!  + ... .

2 ri1 r 1



Here Ψk abbreviates (cos kθi cos kθ + sin kθi sin kθ). Since their patterns are slightly different, the terms have been split in two groups, containing the powers {0 , 4 , 8 , . . .} and {2 , 6 , 10 , . . .}, respectively. In place of (3.24), we have now e −

2 2

r

{

{1}, r {cos θ, sin θ}, r 2 {1, cos 2θ, sin 2θ}, cos 3θ, sin 3θ}, r 3 {cos θ, sin θ, . . . },

(3.26)

with the counting condition still satisfied. In the BE case, there is no initial factorization available (like (3.22) for the case of GA), but a counterpart to (3.25) can nevertheless be found [100]. Changing from monomials to Chebyshev polynomials in the radial direction: This change can also be carried out analytically (i.e., free from numerical cancellations), finally leading to a 2-D basis function set that is consistent with (3.11) and that features good numerical independence across 2-D domains. Implementation details and a MATLAB code is provided in [101]. If the node distribution on a bounded domain is fairly uniform, errors tend to be large at boundaries. To some extent, this is unavoidable [211]. If one weakly concentrates RBF nodes toward boundaries, tests in [101] show the RBF-QR method to give excellent accuracy without any falloff even with thousands of nodes, in 1-D as well as in higher-D (more discussion about RBFs at boundaries is given in Section 3.3.4). Generalizations from 2-D to 1-D and to 3-D are described in [101]. In 3-D, spherical harmonics replace trigonometric functions in the angular directions. Just like with RBF-Direct, the computer time for RBF-QR scales as O(n 3 ), with a ratio that varies somewhat with . The QR decomposition is more costly than a linear system solve, and larger -values necessitate longer internal series expansions. As a rough rule of thumb, the 2-D RBF-QR implementation just described is about 10 to 20 times slower than RBF-Direct (to be compared with a factor close to 1,000 times slower for a 2-D Mercer expansion–based implementation, as reported in [66]). 3.2.4.8 The RBF-GA algorithm

It is possible to effectively utilize the fact that the remainder term after any truncation of (3.23) can be expressed in closed form as an incomplete gamma function [104]. This leads

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

3.2. Near-flat RBFs

67

to a stable algorithm that is entirely free from any infinite expansions or numerical approximations, and it directly obeys the counting condition in any number of dimensions. We describe here this RBF-GA algorithm in 2-D and start again by considering the GA basis function set (1)

ψi (x, x i ) = e − ((x−xi ) +(y−yi ) ) = e − (xi +yi ) · e − (x 2

2

2

2

2

2

2

2

+y 2 )

· e 2 (x xi +y yi ) , i = 1, . . . , n 2

(cf. (3.22)). Omitting the scalar multipliers e − (xi +yi ) leads to a second basis function set in exactly the same space: 2

(2)

ψi (x, x i ) = e − (x 2

= e − (x 2

2

2

+y 2 )

where and

+y 2 )

5

2

2

· e 2 (x xi +y yi ) 2

· 1 + 22 (x xi + y yi ) + . . . +

1 (k − 1)!

(3.27) 6 2 k−1 (2 (x xi + y yi )) + Gk (z) ,

z = 22 (x xi + y yi ) = 22 x x i ˆ z k−1  zj ez Gk (z) = e − e −t t k−1 d t = e z · gammai nc(z, k) , = (k − 1)! 0 j =0 j ! z

with gammai nc(z, k) denoting the incomplete gamma function (as it is defined, e.g., in MATLAB). For each i, the nonnegative integer k can be chosen arbitrarily. The goal (2) now becomes to combine the ψi (x, x i ) functions with each other (i.e., again staying in the same space) so that the polynomial part of (3.27) vanishes analytically. This can be achieved as follows: (2) Write the first one, first three, first six, etc. ψi (x, x i ) functions, arranged as column vectors, as 8 7 (2)    2 2 2 + G0 (22 x x 1 ) , ψ1 (x, x 1 ) = e − (x +y ) · [0] ⎤ ⎡ (2) ⎧⎡ ⎤ ⎡ ⎤⎫ ψ1 (x, x 1 ) G1 (22 x x 1 ) ⎬ ⎨ 1 2 2 2 ⎥ ⎢ (2) − (x +y ) + ⎣ G1 (22 x x 2 ) ⎦ , · ⎣ 1 ⎦ [1] ⎣ ψ2 (x, x 2 ) ⎦ = e ⎩ (2) 1 G1 (22 x x 3 ) ⎭ ψ3 (x, x 3 ) ⎤ ⎡ (2) ⎧⎡ ⎤⎫ ⎤ ⎡ ψ1 (x, x 1 ) 1 x1 y1 ⎡ G2 (22 x x 1 ) ⎪ ⎤ ⎪ ⎪ ⎪ ⎪ ⎥ ⎢ (2) 1 ⎢ G2 (22 x x 2 ) ⎥⎪ ⎬ ⎨⎢ 1 x2 y2 ⎥ ⎢ ψ2 (x, x 2 ) ⎥ 2 2 2 ⎥ ⎥ ⎢ ⎢ − (x +y ) ⎥=e ⎢ ⎣ 22 x ⎦ + ⎢ · ⎢ . . , ⎥ ⎥ . . .. ⎥ ⎢ .. .. ⎦ ⎦⎪ ⎣ .. .. ⎣ ⎪ 2 ⎦ ⎣ ⎪ ⎪ . 2 y ⎪ ⎪ ⎭ ⎩ (2) G2 (22 x x 6 ) 1 x6 y6 ψ6 (x, x 6 ) etc. Next, let B0 = 1, let B1 be the 2 × 3 matrix spanning the left nullspace of ⎡ ⎤ 1 ⎣ 1 ⎦, 1 let B2 be the similar 3 × 6 matrix spanning the left nullspace of ⎡ ⎤ 1 x1 y1 ⎢ 1 x2 y2 ⎥ ⎢ ⎥ ⎢ .. .. .. ⎥ , ⎣ . . . ⎦ 1

x6

y6

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

68

Chapter 3. Introduction to Radial Basis Functions

etc. Multiplying the expressions above from the left with B0 , B1 , B2 , . . . , respectively, gives the new sequence of basis functions 1  · G0 (22 x x 1 ) 0 ⎡   G1 (22 x x 1 ) 1 2 2 2 ψ2 (x) − (x +y ) ⎣ · B1 · · G1 (22 x x 2 ) =e ψ3 (x) 2 G (22 x x ) 

⎡

ψ1 (x)



⎤

= e − (x 2

2

+y 2 )

· B0 ·

ψ4 (x) ⎣ ψ5 (x) ⎦ = e −2 (x 2 +y 2 ) · B2 · 1 4 ψ6 (x)

⎡

⎢ ⎢ ·⎢ ⎣

1

3

G2 (22 x x 1 ) G2 (22 x x 2 ) .. .



,

⎤ ⎦, ⎤ ⎥ ⎥ ⎥, ⎦

G2 (22 x x 6 )

etc. All the potentially dangerous cancellations for small  have now occurred analytically since we know that the multiplications with the null matrices will have eliminated the first matrix in each RHS, and we therefore can just omit this result instead of actually carrying it out numerically. The scaling factors 1/0 , 1/2 , 1/4 , etc. have been introduced so that the new basis functions remain of size O(1) as  → 0. In a test case with 15 Halton nodes over the unit circle, with  = 10−2 , Figure 3.14 shows the original GA RBFs and Figure 3.15 the new basis functions, spanning exactly the same space, as produced by the RBF-GA algorithm. For some further implementation details (such as other numbers of nodes than contained in the sequence n = 1, 3, 6, 10, 15, . . . , issues in cases the nodes are lattice based, etc.), see [104]. This RBF-GA algorithm is at present the fastest available stable algorithm. Figure 3.16 shows it, in 3-D, to be about 10 times slower than (the unstable) RBF-Direct approach. It holds a clear speed (and simplicity) advantage over RBF-QR, which in turn is far faster than any available extended precision option. As seen in Figure 3.15, the new basis functions are reminiscent of polynomials of increasing orders ({1}, {x, y}, {x 2 , xy, y 2 }, etc.). While this can lead to Runge phenomenon–type edge effects, this is of little of no concern in case of RBF-FD approximations, when good accuracy is needed only very near the center of a stencil. Still another reason for the RBF-GA approach to be particularly well suited for RBF-FD applications is the relation dp G (z) = Gma x(0,k− p) (z), p = 0, 1, 2, . . . , dzp k which makes it trivial to calculate any number of derivatives of the new basis functions. A MATLAB implementation of the RBF-GA algorithm for creating RBF-FD weights (cf. (5.3)) is available from MATLAB Central.

3.3 Some additional issues with regard to RBF approximations We have in this section collected a number of numerical RBF topics, in no particular order, that can contribute toward an overall understanding and applicability of the RBFs.

3.3.1 The Runge phenomenon (RP) for RBFs Considering again the polynomial RP test case from Section 2.2.1, Figure 3.17 illustrates (in an equispaced case) how the RBF interpolation error at first decreases and then increases when  goes from very large to very small. For  large, each GA basis function

3.3. Some additional issues with regard to RBF approximations (1)

(1)

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

ψ1 (x)

(1)

ψ2 (x)

69 (1)

ψ3 (x)

(1)

ψ4 (x)

ψ5 (x)

1

1

1

1

1

0.5

0.5

0.5

0.5

0.5

0 1

0

−1 −1 (1) ψ6 (x)

0

0 1 1 0

−1 −1 (1) ψ7 (x)

0

0 1 1 0

−1 −1 (1) ψ8 (x)

0

0 1 1 0

−1 −1 (1) ψ9 (x)

0

0 1 1 0

1

1

1

1

1

0.5

0.5

0.5

0.5

0.5

0 1

0

−1 −1 (1) ψ11 (x)

0

0 1 1 0

−1 −1 (1) ψ12 (x)

0

0 1 1 0

−1 −1 (1) ψ13 (x)

0

0 1 1 0

−1 −1 (1) ψ14 (x)

0

0 1 1 0

1

1

1

1

1

0.5

0.5

0.5

0.5

0.5

0 1

0

−1 −1

0

0 1 1 0

−1 −1

0

0 1 1 0

−1 −1

0

0 1 1 0

−1 −1

0

0 1 1 0

−1 −1 (1) ψ10 (x)

−1 −1 (1) ψ15 (x)

−1 −1

0

0

0

1

1

1

Figure 3.14. Original RBF basis function set in the RBF-GA example. Reprinted with permission from Elsevier. [104]

consists of a sharp spike, with a height such that it just reaches up to the corresponding function value. While this type of error disappears for decreasing , it gets replaced by another error type for very small —the RP (as an immediate consequence of the fact that the interpolant then approaches the polynomial one). Figure 3.18 illustrates how the smoothness of the interpolant influences when the trend reversal occurs (in error vs. ) and how strong this reversal will be. The third case (α = 16) uses the same test function as is shown in Figure 3.17. The RP enters in all cases once  is sufficiently small, and its level at  = 0 matches exactly the polynomial case. The brief discussion above followed [112]; a more theoretical discussion can be found in [172]. The degree by which errors may increase the last way into the  → 0 limit reflects how well boundary issues have been handled. Some opportunities in this regard are discussed in Section 3.3.4; see also [26].

3.3.2 Choice of constant shape parameter Figures 3.2(b) and 3.13(c) illustrate the reason for considering values of  as low or even lower than the point of encountering dangerous conditioning with the RBF-Direct approach. Figures 3.17 and 3.18 show that, even when stable algorithms are available, there can be good reasons not to lower  too far. Early discussions on the subject can be found in [39] (where Cray 1 double precision calculations, with about 29 decimal digit precision, revealed some of these complexities) and in [272]. An interesting approach for finding a suitable value for , even without a detailed knowledge about the causes for error behavior, is offered by the LOOCV (Leave One Out Cross Validation). This was introduced in [224] and is summarized in [67, 240] and in [65, Section 17.1.3]. The basic idea is to leave some data point(s) out and then vary  so as to recover as well as possible the omitted

70

Chapter 3. Introduction to Radial Basis Functions

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

ψ1 (x) 1 0.5 0 1

0.5 0 −0.5 1

0.2 0 −0.2 1

0.1 0 −0.1 1

0.05 0 −0.05 1

0

0

0

0

0

−1 −1 0 ψ2 (x)

−1 −1 0 ψ4 (x)

−1 −1 0 ψ7 (x)

−1 −1 0 ψ11 (x)

−1 −1 0

1 ψ3 (x) 0.5 0 −0.5 1 1

0.2 0 −0.2 1 1

0.1 0 −0.1 1 1

0.05 0 −0.05 1 1

0

0

0

0

−1 −1 0 ψ5 (x)

−1 −1 0 ψ8 (x)

−1 −1 0 ψ12 (x)

−1 −1 0

1 ψ6 (x) 0.2 0 −0.2 1 1

0.1 0 −0.1 1 1

0.05 0 −0.05 1 1

0

0

0

−1 −1 0 ψ9 (x)

−1 −1 0 ψ13 (x)

−1 −1 0

1 ψ10 (x) 0.1 0 −0.1 1 1

0.05 0 −0.05 1 1

0

0

−1 −1 0 ψ14 (x)

−1 −1 0

1 ψ15 (x) 0.05 0 −0.05 1 1

0

−1 −1 0

1

Figure 3.15. New basis function set in the RBF-GA test example. Reprinted with permission from Elsevier. [104]

value(s). While a “brute force” implementation of the concept can be very costly, effective enhancements are available.

3.3.3 Spatially variable shape parameters While boundaries are a common trigger of oscillations, it is not the only one. They will also arise if one attempts to improve local accuracy by clustering nodes in select areas. This effect—and an easy remedy against it—is described in [77, 112]. As an example, consider Figure 3.19(a). There is no RP visible, but the equispaced RBF approximation is relatively inaccurate near the center, where the data features a very sharp gradient. In part b, two extra nodes are inserted in the critical area, and in part c, still two more nodes are inserted. The most striking result of this local refinement attempt is disastrous RPlike oscillations. An easy approach for bringing it under control is to make the shape parameter  spatially variable, e.g., making it proportional to the local node density (e.g., inversely proportional to typical distances between adjacent nodes). This idea has a long history [30, 56, 162, 165, 282]. Figure 3.20 shows that one can obtain excellent accuracy in the f (x) = arctan(20x) test case already with very few nodes if one just uses good choices both for their locations x j and for the local shape parameter values  j . In this particular example, a multivariate optimizer was used for obtaining both the x j and the  j values (in order to explore for good strategies without the danger of imposing any

3.3. Some additional issues with regard to RBF approximations

71

3

VPA (MuPad) VPA (Advanpix)

2

10

RBF-QR 1

10

Runtime (s)

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

10

RBF-GA Direct

0

10

−1

10

−2

10

−3

10

−4

10

1

2

10

3

10

10

n

Figure 3.16. Comparison in run time using different RBF algorithms for 3-D test cases as a function of n. The calculations were done on a dual core PC running at 1.3 GHz. The VPA (variable precision arithmetic) calculations used RBF-Direct in 100 decimal digit precision with MATLAB’s symbolic toolbox (MuPad) and the MATLAB-compatible Advanpix software package (version 3.3.8), respectively. Reprinted with permission from Elsevier. [104] 1.5

1.5

1.5

1

1

1

0.5

−1

0.5

1

−1

−0.5

(a) ε = 30

0.5

1

−1

−0.5

(b) ε = 3.5

1 −0.5

(c) ε = 0.5 1

Figure 3.17. GA interpolants of f (x) = 1+16x 2 for a wide range of -values:  = 30, 3.5, and 0.5, respectively. Reprinted with permission from Elsevier. [112]

preconceived ideas). Numerous experiments like this support the general strategy just mentioned—cluster nodes where solution changes most rapidly and then correspondingly increase  locally. The error levels that can be reached are quite spectacular in comparison with what can be achieved with, say, polynomial interpolation at the Chebyshev nodes (corresponding to a typical nonperiodic PS method). As Figure 3.21 shows, n = 170 nodes are needed to match the max norm accuracy of 2.5 · 10−5 that RBF achieved using only n = 10 interpolation nodes. Section 4.4.2 describes the use of spatially variable  j in the context of node refinement for solving transport-type PDEs.

72

Chapter 3. Introduction to Radial Basis Functions

α=1

1.5

1 0.5

−1

x

1

0.5

−1

1

x

−0.5

0

0

10

0 −4

10

−8

10

−8

10

−8

10

−12

10

−12

10

−12

10

−16

10

−16

10

10

10

1

10

−16

10

ε

−1

10

0

0

ε

1

10

10

0

10

−4

−4

−8

0

10

1

10

−16

10

ε

ε

−12

10

−16

−1

10

1

−8

10

−16

ε

10

−12

10

1

10

−4

−8

10

0

10

10

10

−12

−1

10 10

10

10

10

0

10

10

1

10

−4

0

x −0.5

10

−4

−1

−1

−0.5

10

GA |Error|

1.5

1 0.5

fα(x)

α = 16

1.5

1

MQ |Error|

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

α = 0.1

−1

10

0

ε

1

10

10

10

−1

10

0

10

10

1

Figure 3.18. Top row: The function fα (x) = 1+αx 2 for three values of α in the case of N = 21 equispaced nodes over [−1, 1]. The next two rows of subplots show how the error varies with  in the case of GA and MQ RBFs, respectively . Reprinted with permission from Elsevier. [112] 4

4

4

2

2

2

0

0

0

−2

−2

−2

−4 −1 (a)

0 14 nodes.

1

−4 −1 (b)

0 16 nodes.

1

−4 −1 (c)

0 18 nodes.

1

Figure 3.19. MQ RBF  = 2 interpolants (dashed curves) of f (x) = arctan(20x) (dotted curves) over [−1, 1]. (a) 14 equispaced points. (b) Two extra points inserted near the center. (c) Still two more points inserted near the center. Reprinted with permission from Elsevier. [112]

3.3.4 Boundary techniques Interpolation is generally a much more stable process than extrapolation. Similarly, errors at boundaries are likely to be higher than in domain interiors. A rather extreme manifestation is the polynomial RP (Section 2.2.1), related to the fact that high-order polynomials pn (x) always diverge rapidly to infinity when x → ±∞. Several ideas for improving RBF

73

Figure 3.20. (a) The function f (x) = arctan(20x), with nodes clustered around the “step.” (b) Spatially variable -values, approximately proportional to the local node density. (c) Resulting nearuniform error across the domain of interpolation. Reprinted with permission from Elsevier. [112]

n = 20 points

Interpolant

Error

2

0.2

1

0.1

0

0

−1

−0.1

−2 −1

−0.5

0 x

0.5

1

−0.2 −1

−0.5

0 x

0.5

1

−0.5

0 x

0.5

1

−5

2

3

x 10

2 n = 170 points

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

3.3. Some additional issues with regard to RBF approximations

1 1 0

0 −1

−1 −2 −2 −1

−0.5

0 x

0.5

1

−3 −1

Figure 3.21. 20-point and 170-point Chebyshev interpolants for arctan(20x) over [−1, 1]; displays of the interpolants and of their errors. In this 1-D case, Chebyshev interpolation would have been helped by a domain decomposition of [−1, 1] into [−1, 0] and [0, 1].

edge behavior have been tested. One early suggestion (in the case of solving elliptic PDEs) [70] is to employ, at a boundary, not only the boundary condition but also the PDE— and then locate extra centers outside the domain in order to still have the same number of equations as free RBF parameters. In the context of interpolation (with no extra governing equations available), some different ideas were compared in [92]. Figure 3.22 a shows a test example from that study, interpolation of f (x, y) =

1 1

25 + (x − 5 )2 + 2y 2

(3.28)

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

74

Chapter 3. Introduction to Radial Basis Functions

Figure 3.22. Test problem: (a) The function f (x, y) given by (3.28), displayed over the unit circle. (b)–(e) Different distributions of RBF centers (circles) and collocation points (dots) when using four different boundary control strategies: Regular, NaK (Not-a-Knot), SNaK (Super-Not-a-Knot), and boundary clustering, respectively. Reprinted with permission from Elsevier. [92]

over the unit circle, using n = 200 nodes. The subplots (b)–(e) illustrate four plausible strategies, as follows: b.

With the fairly uniform distribution of nodes (RBF centers agreeing with the collocation points) as shown in subplot (b), we will compare regular RBF interpolation against adding a constant and terms, i.e., $1, x, y, to the basis $ linear$ set and then enforcing the constraints λi = λi xi = λi yi = 0 (inspired by the natural spline RBF interpretation in Section A.1.2; it is seen in Section 5.1.5 that continuing to higher-degree polynomials can be very beneficial).

c.

We here generalize the 1-D cubic spline Not-a-Knot (NaK) approach of moving out one “row” of RBF centers from just inside to just outside the boundary (again, cf. Section A.1.2).

d.

This illustrates SNaK—pushing the NaK idea a bit further by also shifting out RBF centers from the collocation points on the boundary.

e.

Chebyshev-inspired node clustering.

The results in Figure 3.23 are by no means conclusive but give indications that regular RBF interpolation (labeled “Regular”) is not a good strategy. Clustering nodes toward the boundary can be beneficial but must be used with care since it is associated with a depletion of nodes near the center. All the other three approaches, “Const. + Linear,” “NaK,” and “SNaK,” are clearly beneficial in the present case, possibly with an edge for “SNaK.” More explorations are needed on the topic of boundary effects. Most likely, the best choices will depend also on factors not focused on here, such as choice of , maybe the use of spatially variable  as boundaries are approached, etc. An important observation is that, compared to high-degree 1-D global polynomials, for which Chebyshev-type edge clustering is almost the only option, RBFs offer more opportunities.

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

3.3. Some additional issues with regard to RBF approximations

75

Figure 3.23. Comparison of the (absolute) error of different RBFs with varying boundary treatments for the function (3.28) inside the unit circle. The black dots mark the locations of the data values. Errors range from 0 (white) to 2.0 · 10−5 (black). The labels (b)–(e) to the left match the labels for the subplots (b)–(e) in Figure 3.22. Reprinted with permission from Elsevier. [92]

3.3.5 Fast algorithms Many “fast” algorithms are focused on rapid evaluation of (3.2), both since this is important in its own right and also since sums of this form become the dominant task within many iterative schemes for solving (3.3). Fast evaluations were until recently viewed as crucial for “scaling” RBF implementations to massive problem sizes. The recent advances in RBF-FD methods reduce or altogether eliminate this need in PDE contexts. As a result, our discussion of “fast algorithms” will be very brief. Before quoting a few concepts for fast summations of (3.2), we note that several studies have used a severely flawed test problem for this task. 3.3.5.1 Flawed test problem

It has been surprisingly common to choose as a test problem a large number of random expansion coefficients λi , i = 1, 2, . . . , n (or even setting all the λi to a constant), and then see how fast one can evaluate (3.2) to some specified level of accuracy. Figure 3.24 shows typical 1-D RBF interpolants s(x) that have been created in this manner, with the xi randomly distributed over [0, 1] and the λi similarly uniformly randomized over [−1, 1], i = 1, . . . , n, n = 10k , k = 0, 1, . . . , 5. As can clearly be seen, these functions s(x)

76

Chapter 3. Introduction to Radial Basis Functions

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

n=1

n = 10

0.8

0.4

0.6

0.2

0.4

0

0.2

−0.2

0

0

0.2

0.4

0.6

0.8

1

−0.4

0

0.2

n = 100 3

10

2

0

1

−10

0

0

0.2

0.4

0.6

0.8

1

−20

0

0.2

n = 10000 300

60

200

40

100

0

0.2

0.4

0.6

0.8

1

0.6

0.4

0.6

0.8

1

0.8

1

n = 100000

80

20

0.4 n = 1000

0.8

1

0

0

0.2

0.4

0.6

$ Figure 3.24. Display of s(x) = ni=1 λi φ(|x − xi |) for n = 10k , k = 0, . . . , 5, with φ(r ) = 1 ,  = 4, and xi , λi randomly distributed over [0, 1] and [−1, 1], respectively. 1+(r )2

do not gain any more structural detail no matter how large n is chosen. In the present example, a 10-node cubic spline or some even briefer Chebyshev expansion will always give excellent approximations to this particular sequence of “test functions,” independently of n. Numerous “fast algorithms” have been published that, behind a sequence of superficially plausible steps, often accompanied by extensive error analysis, hide what essentially amounts to utilizing this high level of n-independent smoothness to re-represent very large-n representations more economically and then, based on high measured speeds for large n-values, make the claim of the discovery of a “fast RBF algorithm.” It is well known that pseudospectral (PS) methods require around 2–3 nodes per wavelength that they resolve (technically 2 for Fourier-PS and π for Chebyshev-PS [277]). RBF approximations (assuming smooth basis functions) are again spectrally accurate and perform similarly. For a fast algorithm to be relevant, it is the data values fi = s(xi ) (rather than the expansion coefficients λi ) that need to feature correspondingly high-frequency content when n is increasing. 3.3.5.2 Examples of fast algorithms / approaches

In spite of the comments above, it should however be noted that, in other application areas than PDEs (such as interpolation and surface rendering), requirements can be different, and “fast algorithms” can indeed lead to significant savings. Primary approaches include (i) Krylov iteration–based algorithms, in particular the BFGP (Beatson-Faul-GoodsellPowell) algorithm using local Lagrange (cardinal) bases [68, 69], (ii) multipole algorithms, used either directly [11, 13, 46, 299] or to speed up Krylov iterations [133], (iii) fast Gauss transform algorithms [229] (based on the transform originally given in [132]), and (iv) fast tree code algorithms [54, 169] (based on concepts originating in computational fluid mechanics/chemistry). Some fast algorithm applications to surface rendering are described

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

3.3. Some additional issues with regard to RBF approximations

77

Table 3.5. Degrees of polynomials exactly reproduced by different RBF types on infinite d -D node layouts.

Type of basis function

Radial function φ(r ) Jd /2−1 (r )/(r ) r3 r 2 log r r ( 1 + (r )2 ( 1/ 1 + (r )2 1/(1 + (r )2 ) 2 e −(r ) -

BE Cubic TPS Linear MQ IMQ IQ GA Compact RBFs

d /2−1

Reproduced degree all d +2 d +1 d d d −2 d −3 none none

in [281]. Still another approach for speeding up RBF calculations is to utilize parallel computing hardware [300]. Note also comments at the end of Section 3.2.4.1.

3.3.6 Polynomial reproduction This feature was originally observed on infinite lattices but was soon afterward found to remain essentially unchanged also if nodes are scattered irregularly over an infinite domain [60, 213]. Table 3.5 summarizes, for some different RBFs, the degree polynomials in d -D that will be reproduced exactly by their RBF interpolant (no matter the choice of , if present). Apart from in the BE case, the ability to reproduce polynomials is typically directly related to the rate by which the radial functions grow/decay as r increases. This ability of many smooth RBF types to exactly reproduce polynomials up to some finite degree in the infinite node number limit has given rise to confusing statements about RBF approximations having only algebraic O(h p )-type accuracy (with h being a “typical” node distance), when convergence generally is spectral, of type O(e −c/h ). For classical FD methods, convergence rates can be found by inspecting the highest degree polynomial for which an approximation is exact. In the RBF case, this immediate connection between polynomial reproduction and actual convergence rate is absent. 3.3.6.1 Heuristic explanation of reproduction of a constant in 1-D

We consider first φ(r ) = r on a$unit-spaced infinite grid in 1-D (i.e., with xi = i∈ ) and want the interpolant s(x) = ∞ i =−∞ λi |x − i| to take the value 1 at a wider and wider set of nodes surrounding the origin. Clearly, this cannot be achieved by letting the expansion coefficients λi all converge to the same finite value, as s(x) would then diverge. Instead, for k → ∞, choosing Linear RBFs

⎧ 1 ⎨ λ−k−1 = 2 , 1 λ = −2, ⎩ k λi = 0

1

λ−k = − 2 , 1 λk+1 = 2 , otherwise

(3.29)

will achieve it. A constant is “reproduced” not by making all expansion coefficients the same but by creating two tiny groups of nonzero coefficients (here, by two entries each) and letting these groups move apart.

78

Chapter 3. Introduction to Radial Basis Functions

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

Table 3.6. Examples of RBF weights and corresponding interpolant values.

Linear φ(r ) = r λi s(xi )

MQ φ(r ) = 1 + r 2 λi s(xi )

MQ φ(r ) = 1 + r 2 λi s(xi )

.. . 0 0 0 0 0 1 0 0 0 0 0 .. .

.. . 0 0 0 0 0 1 0 0 0 0 0 .. .

.. . −0.0366 0.0932 −0.2895 0.7852 −2.3104 4.5115 −2.3104 0.7852 −0.2895 0.0932 −0.0366 .. .

.. . 5 4 3 2 1 0 1 2 3 4 5 .. .

.. . 5.0990 4.1231 3.1623 2.2361 1.4142 1.0000 1.4142 2.2361 3.1623 4.1231 5.0990 .. .

.. . 5 4 3 2 1 0 1 2 3 4 5 .. .

GA RBFs Using Gaussians changes this picture completely. The interpolant s(x) = $∞ −2 (x−i )2 converges when all λi are the same, λi = λ, and s(xi ) = 1 is achieved i =−∞ λi e by just choosing this scalar constant λ appropriately. The resulting interpolant s(x) = $ −2 (x−i )2 will become an oscillatory function of x, obviously not reproducing λ ∞ i =−∞ e the constant one identically.

Simplifying by letting  = 1, the MQ radial function becomes φ(r ) = 1 + r 2 . Table 3.6 relates some expansion coefficient sets λi and resulting interpolant values s(xi ) in cases of linear and MQ RBFs. The rightmost pair of columns show that using a (exponentially converging) group of MQ coefficients produces the identical result at all node points as would a single linear RBF. In view of (3.29), we can then achieve s(xi ) = 1 for increasingly wide intervals around the origin by using two local groups of MQ coefficients that gradually are shifted further out from the origin. Since MQ become increasingly linear at larger distances, it must then converge to one also in between the node points. This is described as “reproducing a constant.”

MQ RBFs

3.3.6.2 Extensions to d-D

As the GA and MQ cases above have illustrated, reproduction of a constant entirely hinged on whether using RBF expansion coefficients proportional to the data itself caused convergence or divergence. This concept generalizes to higher dimensions (and does not rely on lattice-type node layouts). In case of convergence (as will always be the case with GA RBFs), we do not have any polynomial reproduction. In case of divergence, interpolation is instead achieved by expansion coefficients that decay to zero in an increasingly wide region surrounding the origin and with the essential contributions coming from small coefficient groups increasingly far out. Their contribution will be smooth enough to achieve the reproduction. In conclusion, we note that BE RBFs, being band limited in Fourier space in any number of dimensions, lead to some unusual reproduction features [72].

3.3. Some additional issues with regard to RBF approximations

79

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

(a) GA, ε = 1/h

(b) Wendland, ε = 1/(3h)

1

1

0.9999

0.998

0.9998

0.996

0.9997

0.994

0.9996

0.992

0.9995 1

0.99 1 1 0.8

0.5

0.6

1 0.8

0.5

0.6

0.4 y

0

0.4

0.2 0

x

y

0

0.2 0

x

Figure 3.25. The variation in SE over the 2-D “period box” for two choices of radial function.

3.3.7 Stagnation errors With most numerical methods, one can improve accuracy by using increasingly fine grids. When using a stable RBF algorithm, one can similarly increase the node density and achieve very much improved accuracies. However, if one uses RBF-Direct, cond(A) will then grow disastrously. The “remedy” of simultaneously increasing  in order to keep cond(A) unchanged may lead to stagnation errors (also known as stationary or saturation errors; we abbreviate these by SE). Keeping cond(A) constant usually requires  to be increased as O(1/h), where h is a typical distance between neighboring nodes. Not surprisingly, this increase in  severely degrades the resulting accuracy. It depends on the RBF type, the node distribution, the distance from boundaries, etc. whether the resulting errors will slowly go to zero or fail to decrease past some finite SE level. The limited analysis that is available is based mainly on equispaced 1-D lattices [25]. The SE concept can be understood from our discussion in Section 3.3.6. If we, on an infinite 1-D equispaced lattice, decrease h and increase  as  = α/h for some α > 0, the (infinite) A-matrix remains unchanged, and the only difference is that the data gets locally flatter and flatter—thus producing the error described above when using RBFs to approximate a constant. The error as h → 0 will stagnate at some finite value (dependent on α) for some RBF types (such as GA and compact RBFs) but will slowly decay to zero for others (e.g., as O(h 2 ) for MQ). When interpolation errors stagnate at O(1) for h → 0, first-derivative errors will grow like O(1/h), second-derivative errors like O(1/h 2 ), etc. As an illustration of SE, consider an infinite h-spaced 2-D lattice and using (a) Gaus2 1 sians φ(r ) = e −( r ) with  = h , and (b) the Wendland function φ3,1 (r ) = (1−r )4+ (4r + 1 1) with  = 3h (with the difference in -values motivated by the latter being more peaked for the same ). With all the node data equal to one, the variation within each h × h-sized “period box” becomes as shown in Figure 3.25, independent of h (and thus shown here with h = 1). As h → 0, only the box size shrinks, but all the values remain unchanged. One option to eliminate SE is to choose a stable algorithm in place of RBF-Direct and then refrain from increasing  like O(1/h). The SE issue arises not only with global RBF approximations but also when using RBF-FD approximations. Even when basing these on PHS-type RBFs (which do not have any shape parameter ), SE can arise but can then be reduced or eliminated by including polynomial terms in the RBF approximation (cf. Sections 5.1.5–5.1.7).

3.3.8 RBFs on infinite lattices We recall from Section 3.1.4.1 that results derived on lattices often can be misleading. When interpreted with great care, such analysis can, however, still sometimes be informative.

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

80

Chapter 3. Introduction to Radial Basis Functions Table 3.7. Some common choices for smooth radial functions together with their Fourier (Hankel) transforms in d dimensions. MN and TPS denote monomial and thin plate spline, respectively (combined as “PHS” in Table 3.1). In these cases, j ∈ , with j = 1 a common choice. In this table, it is assumed that r ≥ 0 and ρ ≥ 0.

Type

RBF φ(r )

MN

r 2 j +1

TPS

r 2 j log r

GA MQ

e −(r ) ( 1 + (r )2

IMQ

1/

IQ

1/(1 + (r )2 )

BE

2

Jδ 2

(

−1

1 + (r )2

(r ) δ

(r ) 2 −1

) in 2-D FT φ(ρ)

) in d -D FT φ(ρ)

(−1) j +1 ((2 j +1)!!)2 ρ 2 j +3 (−1) j +1 22 j ( j !)2 ρ 2 j +2

(−1) j +1 22 j + 2 +1 ( j + 2 )Γ ( j + 2 )Γ ( j + 2 ) 1 π ρ2 j +d +1 d d 1 (−1) j +1 22 j + 2 −1 j !Γ ( j + 2 ) ρ2 j +d

2

e −ρ /(4 2 2

d

2)

2



2/π

K d +1 (  ) ρ

1 2 δ −2 ρ2 2 1− 2 δ  2 2δ/2−1 Γ 2 −1 π

0

2



2

d +1  2

d −1 ρ 2 ρ Kd −1  1 2 d d  2 +1 ρ 2 −1

K(ρ/) 2

⎪ ⎩

2/π

− d −1 d +1  2 ρ 2 ρ K d −1 ( )

e −ρ/ ρ

⎧ ⎪ ⎨

d +1

1

2

e −ρ /(4 ) ( 2)d

e −ρ/ (ρ+) − ρ3

1

if ρ ≤  otherwise

⎧ ⎪ ⎨ ⎪ ⎩

( )

1

ρ2

1− 2

2 δ−d −1 2

 δ−d  d 2δ/2−1 Γ 2 π d /2

if ρ ≤ 

0

otherwise

In most analysis of RBF approximations on infinite lattices, the Fourier transforms of the RBF will be of interest. Table 3.7 shows these for some standard smooth radial functions (or their generalized Fourier transform if the integral that defines the regular version would be divergent; cf. [159, 177]). We use here the 1-D convention u(x) = 1 ´∞ 1 ´∞ u(x) e −i ωx d x, and its generalization to higher u)(ω) e i ωx d ω; u)(ω) = 2π −∞ 2π −∞ dimensions. When a function in d dimensions is radially symmetric (i.e., depends on / r = x12 + x22 + · · · + xd2 only), the Fourier transform will similarly depend only on ρ = / ω12 + ω22 + · · · + ωd2 . The computation of the d -dimensional transforms is then known as a Hankel transform, ) = 1 ´ ∞ . . . ´ ∞ φ(x) e −i ω · x d x φ(ρ) −∞ (2π)d /2 −∞ =

1 ρ(d −2)/2

´∞ 0

φ(r ) r d /2 J(d −2)/2 (r ρ) d r,

which is identical to its own inverse. Alternatively, the Hankel transform can be computed by the formulas < ) = (−2) m 2 d m ´ ∞ φ(r ) cos(r ρ) d r , d = 2m + 1 odd: φ(ρ) π d (ρ2 )m 0 ´∞ m m d ) d = 2m + 2 even: φ(ρ) = (−2) φ(r ) r J (r ρ) d r , d (ρ2 )m

0

0

involving derivatives but no Bessel functions of higher than zeroth order.

3.3.9 Expansion coefficient “locality” Figure 3.26 illustrates the locality concept on equispaced nonperiodic data over [−1, 1]. No matter whether the radial function is infinitely smooth or whether it is growing or

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

3.3. Some additional issues with regard to RBF approximations

81

Figure 3.26. Top row: Three cases of cardinal data on a nonperiodic equispaced grid over the interval [−1, 1]. Each of the bottom three rows: At left, the radial function when centered at r = 0. To the right of this: The numerical values of the RBF expansion coefficients in the cases shown straight above (in the top row of subplots).

decaying for increasing r , a nonzero node point data value fi will mostly influence expansion coefficients λ j when j is close to i. This “locality” can be a very desirable feature. It is typical for wavelet expansions but is entirely absent for PS methods—changing only one data value typically changes all Fourier and Chebyshev expansion coefficients by about the same amount. This “locality” property was analyzed in detail in [97] and will be briefly summarized here. 3.3.9.1 Some basic results on a 1-D unit-spaced infinite lattice

The cardinal condition implies ∞ 

 λk φ(n − k) =

k=−∞

1, n = 0, 0 otherwise.

The LHS is a discrete convolution, which in Fourier space takes the form Λ(ξ ) · Ξ(ξ ) = 1, where Λ(ξ ) =

∞  k=−∞

λk e i kξ and Ξ(ξ ) =

∞ 

φ(k)e i kξ .

k=−∞

Given a radial function φ(r ), we can therefore calculate first Ξ(ξ ) and then Λ(ξ ). Since it frequently happens that direct calculation above of Ξ(ξ ) diverges, one can instead form

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

82

Chapter 3. Introduction to Radial Basis Functions

Figure 3.27. The steps for calculating the cardinal data expansion coefficients on an infinite equispaced 1-D lattice, illustrated in the case of the cubic radial function φ(r ) = |r |3 .

) ) of φ(r ) [159, 177] and evaluate the generalized Fourier transform φ(ξ Ξ(ξ ) =



2π

∞ 

) + 2π j ). φ(ξ

j =−∞

For example, in the case of cubic RBFs φ(r ) = |r |3 , one can first note that ) ) = lim 1 φ(ξ α0 2π

ˆ

∞

−∞

12(ξ 4 − 6ξ 2 α2 + α4 ) 12 1 = α0 2π(ξ 2 + α2 )4 2π ξ 4

|r |3 e −α|r | e −i ξ r d r = lim

and then proceed as illustrated in Figure 3.27. Returning to the task of studying cardinal data, one obtains 1 λk = 2π

ˆ

2π

0

Λ(ξ )e

−i kξ

1 dξ = (2π)3/2

ˆ 0

2π

e i kξ dξ , $∞ ) j =−∞ φ(|ξ + 2π j |)

(3.30)

and the cardinal interpolant becomes s(x) =

1 2π

ˆ

∞

−∞

$∞

) ) e i xξ φ(ξ dξ . ) + 2π j |) φ(|ξ

(3.31)

j =−∞

Closed-form expressions for the coefficients λk (not containing integrals) are available also in some other cases than cubics, e.g., for GA and SH (φ(r ) = sech r ). However, in all cases, a change of integration path together with steepest descent asymptotics applied to (3.30) give excellent approximations. For some radial functions (such as cubics, GA, and SH), the exponential decay of |λk | will be found to continue indefinitely, whereas for other types, algebraic decay becomes dominant for |k| large. For example, in the case of MQ with  = 1, one thus finds (for k = 0, 1, 2, . . .) λk ≈

17.433(−1)k e −1.0566k + . . .    leading exponential term (from pole nearest to real axis)

3 − + ... 5 k   

. (3.32)

leading algebraic term (from branch points at 0 and 2π)

Figure 3.28 compares (3.32) (using only the two displayed terms; solid curve) with exact coefficients λk (dots). The two displays differ in the k-axis being linear or logarithmic, causing one or the other of the two parts to display as straight lines.

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

3.3. Some additional issues with regard to RBF approximations

83

Figure 3.28. Comparison between exact MQ expansion coefficients λk and the asymptotic values (3.32) based on the leading term only in the exponential and algebraic parts. Reprinted with permission from Oxford University Press. [97]

3.3.9.2 Coefficient locality on a d-D unit-spaced lattice

Similarly to the situation in 1-D, the exponential coefficient decay will be supplemented ˆ is not analytic at the origin of a complex ρby an algebraic regime far out in case 1/φ(ρ) plane. While this always happens for, say, MQ and IMQ and never for GA, it will happen for MN when d is even and for TPS when d is odd. Figure 3.29 illustrates some different cases in 2-D.

3.3.10 Gibbs phenomenon for RBFs Figure 2.3 in the PS chapter introduced some cases of the Gibbs phenomenon. Corresponding RBF interpolation cases for step functions were analyzed in [94]. We limit ourselves here to showing two related figures. Figure 3.30 compares the Gibbs oscillations for three different RBF types, showing in all cases initially an oscillatory decay that is exponentially fast. In two of the cases (TPS and IQ), there appears a sudden transition to one-sided oscillations that decay at a much slower algebraic rate. In the GA case, no such transition will ever occur. Figure 3.31 shows how the transition point in the MQ case rapidly moves toward infinity when  decreases, leading for  → 0 to the same limit as we saw in Figure 2.3(b). In this limit process, the exponential decay rate of the oscillations will gradually diminish and become slow algebraic. An application of the Gibbs phenomenon for RBFs arises in edge detection [160].

3.3.11 RBFs on manifolds and on periodic domains One of the most striking (and useful) strengths of RBFs arise when the given data is located on some low-dimensional manifold that is embedded in a higher-dimensional space. As was highlighted by M. J. D. Powell already in 2001 [214], the cost/accuracy then becomes essentially independent of the higher-dimensional space. In its simplest form, this provides a convenient way to approximate 1-D periodic data, as described in the next Section 3.3.11.1. Examples of solving PDEs on curved 2-D surfaces in 3-D space are given in Section 4.1, and the same concept is again used throughout the remaining sections whenever we consider computations on spherical (or other) surfaces.

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

84

Chapter 3. Introduction to Radial Basis Functions

Figure 3.29. Decay (in magnitude) of expansion coefficients λk1 ,k2 for 2-D cardinal data, displayed using linear scale in the k1 , k2 -plane and logarithmic scale vertically. In the cases in the left column of subplots, the exponential decay persists indefinitely. Reprinted with permission from Oxford University Press. [97]

3.3.11.1 Periodic implementations of RBFs

An attractive option is to place nodes θi that belong to a periodic interval θ ∈ [−π, π] instead around the periphery of the unit circle in 2-D, i.e., use {xi = c os(θi ), yi = s i n(θi )}, and then consider the obtained RBF 2-D results only around this same unit circle. Numerically, this can be achieved by staying in 1-D but replacing a radial function φ(r ) by 1 r2 φ1 (r ) = φ 2 sin . 2

(3.33)

A natural generalization of this to a 2-D periodic case would be to use as basis function centered at (xc , yc ) & = φ2 (x − xc , y − yc ) = φ 2

sin

2

1x −x 2 c

2

+ sin

2

1 y − y 2' c

2

(3.34)

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

3.3. Some additional issues with regard to RBF approximations

85

Figure 3.30. Comparisons of Gibbs oscillations in the cases of TPS, IQ, and GA interpolants.

(no longer strictly radial). This last formulation generalizes immediately to d -D. Another option in the 2-D periodic case is to map to a torus (cf. Figure 5.2). The analysis that is referred to in the next Section 3.3.11.2 uses a different approach to periodizing RBFs. Although convenient for some types of analysis, it is less well suited for computations. It is shown in Section 2 of [93] that, with grid spacing h, the trigonometric data f (nh) = e i ωn h gives expansion coefficients λk =

2π $∞ h

e i ωhk ,  2π j φˆ ω +

j =−∞

(3.35)

h

where φˆ is the regular or generalized Fourier transform of the radial function (see also $ 2 2 [297]). Using periodized radial functions, such as φ(x) = ∞ e − (x+2πk) over x ∈ k=−∞ [−π, π], furthermore changes the eigenvalue patterns described in Section 3.2.1.2 from powers to exponentials in . Although this worsens the conditioning when  is decreased, stable algorithms (such as RBF-QR) can be applied.

3.3.11.2 Analysis on a periodic lattice in 1-D

Based on (3.31), the convergence of s(x) to f (x) = e i ωx can readily be analyzed either for  → 0 or for h → 0. As described in [93], this reveals in the h → 0 case three distinct rates—“algebraic,” “spectral,” or “super-spectral”—exemplified by, e.g., TPS, MQ, 2 and GA, of types O(h con s t ), O(e −con s t /h ), and O(e −con s t /h ), respectively. For derivative approximations in the case of  → 0, one can similarly verify a rapid convergence toward the approximations that the Fourier-PS method would provide.

86

Chapter 3. Introduction to Radial Basis Functions

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

a. MQ, ε = 10

−5

x 10

1.5

0 ← 1.0328

1

−2

0.5

−4

0

−6

−0.5 −4

−2

0

2

4

3

b. MQ, ε = 1

7

8

9

10

16

17

18

19

20

21

22

−2

0

−4 −2

0

2

4

15

c. MQ, ε = 0.1

−13

x 10

1.5

4 ← 1.1406

1

2

0.5

0

0 −0.5 −4

6

0

← 1.1173

0.5

−0.5 −4

5

x 10

1.5 1

4 −8

−2 −2

0

2

4

170

172

174

176

178

180

Figure 3.31. The Gibbs oscillations for MQ around the origin (left column) and further out (right column) (a)  = 10, (b)  = 1, and (c)  = 0.1.

3.3.11.3 RBFs on a 2-D periodic lattice—Comparison against Fourier-PS

In 1-D, the equispaced lattice analysis above did not indicate any advantage of the RBF approach over Fourier-PS approximations—it merely showed the former to converge to the latter in either of the  → 0 and h → 0 limits. The situation in 2-D (and higher) is distinctly different. Following the discussion in [98], Figure 3.32(left) illustrates that there is something ∂ clearly nonoptimal about how Fourier-PS approximates the operator L = 1 ∂ + ∂ y on a 2-D periodic lattice. Considering the approximation at the origin, the ∂x 2

∂

∂

approximations for ∂ x and ∂ y would use values at locations marked by squares and circles, respectively, and L would combine these two approximations. However, L is analytically a directional derivative along the main diagonal, marked by crosses. A derivative is a local property of a function, and it is not plausible that the best approximation should ignore local information in the direction of actual interest while using information far out in other directions. A highly heuristic argument suggests that it would be preferable to weigh ∂ together information based on a 2-D pattern, as indicated for ∂ x in Figure 3.32(right),   x 1 described by the function 8 0 F1 3, − 4 (x 2 + y 2 ) . We next inspect how RBF derivative approximations “pick up” their information. Figure 3.33 shows three cases. On a Cartesian lattice and in the limit of  → 0, RBFs converge to the PS case, and only information along the x-axis is used. However, even with nodes on a lattice, most smooth RBF types will, for  > 0, pick up information from a full 2-D neighborhood (although preferentially in the direction of the derivative). This is more noticeable still if the nodes are on a hexagonal lattice, or generally “scattered.” It is shown in Figure 19 of [98] that, for a fixed number of nodes n, RBF approximations,

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

3.3. Some additional issues with regard to RBF approximations

87

Figure 3.32. (Left) Illustration of how Fourier-PS approximations pick up derivative information (reprinted with permission from Elsevier [75]). (Right) A heuristically motivated pattern of how to better weigh together functional information when approximating ∂ /∂ x (reprinted with permission from Oxford University Press [98]).

Figure 3.33. Magnitude of weights (when these exceed 0.03) for approximating ∂ /∂ x on lattices with node spacing h = 1. (a) Cartesian lattice, RBFs in the  → 0 limit. (b) Cartesian lattice, MQ with  = 1. (c) Hexagonal lattice, RBFs in the  → 0 limit. Reprinted with permission from Oxford University Press. [98]

even using highly irregular Halton nodes, have a small edge in accuracy over Fourier-PS on a Cartesian lattice. This advantage increases both with n and when going from 2-D to 3-D, etc.

3.3.12 A collection of additional RBF issues and methods, with very brief descriptions only We have collected below a number of RBF topics that, while of unquestionable importance, do not so far appear to have played any central role in the geoscience focus area of the present book. 3.3.12.1 Domain decomposition; partition of unity

When using global RBFs over n nodes and with the resulting linear algebra handled in the most straightforward manner possible, the cost becomes O(n 3 ) operations for inverting the A-matrix and O(n 2 ) operations for each application of an N-node differentiation matrix (DM). There are clearly major savings feasible already if one splits the domain into two, with n/2 nodes each. Typically, one would split it in many more partitions than this. Significant parts of the already extensive domain decomposition literature applies fairly

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

88

Chapter 3. Introduction to Radial Basis Functions

Figure 3.34. Example of partition of unity on the sphere. Illustration copied from [290].

directly also in RBF contexts. With domain decomposition, one distinguishes between nonoverlapping and overlapping procedures. For some recent RBF domain decomposition literature, see, e.g., [12, 47, 130, 179, 201]. The partition of unity idea can be seen as a special case of overlapping domain decomposition—with smooth partition functions having local support and also adding up to identically one across the complete domain. The topic is discussed in [65, Chapter 29]. A key strength is that, for each partition, the weight on the RBF interpolant gets gradually reduced to zero as the partition edge is approached, greatly helping there against edge accuracy falloff. A recent application concerns interpolating large scattered data sets on the sphere [40]. Figure 3.34 illustrates a case where N = 16, 384 nodes have been partitioned into groups with n = 100 nodes in each. Strengths include good accuracy together with a high level of sparsity for the linear systems that relate function values with, say, derivative values at the nodes (in the illustrated case, this matrix is only 1.47% full). Preliminary tests for using this approach to solve convective PDEs on the sphere are reported in [290]. 3.3.12.2 Greedy algorithms for controlling resolution

The concept of “greedy approximation” applies to most situations where a function has been expanded in an infinite set of basis functions; see [262] for a general overview. A standard approach for reducing these to finite expansions would be to truncate after some specified number of terms—a strategy that would not depend on the function under consideration. In general terms, a “greedy” approximation includes instead only terms that contribute above some specified threshold. This strategy becomes data dependent and might omit many low-order terms while including higher ones. In RBF contexts, this can amount to taking out and putting in nodes according to how much doing so will affect the overall approximation accuracy. Since the resulting node distributions can become very nonuniform, this is a situation for which it is important to combine with a good spatially variable shape parameter strategy (cf. Section 3.3.3). Automated greedy procedures may require difficult trade-offs between accuracy gains and algorithmic complexity/computational cost; see, e.g., [71, 147, 178, 180, 189]. 3.3.12.3 Least squares and multilevel techniques

Both of these concepts have received prominent treatment in the recent monographs [65, 154, 281]. They can improve computational stability significantly, and they are easy to

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

3.3. Some additional issues with regard to RBF approximations

89

motivate heuristically. Least squares approximation is usually more stable than outright interpolation and can be highly effective in suppressing spurious numerical “noise.” Its application to RBF interpolation forms, in these regards, is no exception. Most data that needs to be interpolated contains significant information across a wide range of scales—high modes superimposed on low ones. Peaked RBFs are inaccurate for low modes, and flat RBFs feature conditioning issues when having to deal with highly oscillatory data (making it then tempting to use larger , only to encounter saturation issues; cf. Section 3.3.7). The “obvious” concept to resolve this is to approximate the data at multiple “levels”—use small  RBFs (maybe on a coarser node set to avoid conditioning issues) to capture low modes and superpose on that other levels with, for each level, RBFs appropriate for successively higher mode ranges. Once such a composite RBF-based interpolant is available, it can the be differentiated analytically in case the task is to solve PDEs (or else used as needed). References include [64, 71, 126]. 3.3.12.4 Divergence and curl-free RBF interpolation

Mathematical models involving vector-valued quantities, such as velocity, magnetic, or electric fields, often contain additional constraints for these variables. For example, magnetic fields and also velocity fields of incompressible fluids must have zero divergence, while electric fields may have zero curl. It was shown in [199] that RBF expansions can be “customized” to obey such constraints. The basic idea is to construct a matrix-valued kernel Φ from a standard (scalar-valued) RBF φ so that its columns satisfy the imposed constraint. For example, the following two kernels are divergence free and curl free in 2 , respectively: Φd i v (||x||) = (−∇2 I − ∇∇T )φ(||x||)

and

Φc u r l (||x||) = −∇∇T φ(||x||),

where I is the 2 × 2 identity matrix and ∇∇T is the Hessian matrix. If φ is positive definite and has at least two continuous derivatives, then both of these kernels will also be positive definite. A numerical implementation is given in [119], showing how it can be used to compute the Helmholtz-Hodge decomposition of a vector field on a sphere (cf. Figure 3.35). This can provide useful information about a flow field. For example, in the atmosphere, the divergence-free part of the horizontal wind field gives details about cyclonic storms, while the curl-free part gives details on high- and low-pressure systems. The study [192] considers RBF-FD based divergence-free interpolation of vector fields arising in MHD (magnetohydrodynamics) and electrodynamics simulations (however, leaving it unclear to what extent zero divergence is maintained). 3.3.12.5 Lagrange functions

We recall that “cardinal data” takes the value 1 at one node point and 0 at the remaining ones. In analogy to how the Lagrange’s interpolation polynomial is formed, the cardinal RBF interpolants sk (x) satisfy  sk (x i ) =

1, 0

i = k, otherwise.

Although available analytically (3.16), they are normally computed numerically (requiring a stable algorithm in case  is small). The set of Lagrange functions functions sk (x) spans the same function space as the original RBFs but with the difference that the associated A-matrix becomes the identity matrix I .

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

90

Chapter 3. Introduction to Radial Basis Functions

Figure 3.35. Illustration of the Helmholtz-Hodge decomposition. (a) Velocity field. (b)– (c) Contours of the stream function and the velocity potential, respectively, computed by the RBF field decomposition method. Illustration provided by Grady Wright.

Since the Lagrange functions sk (x) for large  and for piecewise smooth RBFs (such as PHS) often decay exponentially fast away from the node x k [139], local Lagrange functions (including only some small number of nearest neighbors to the node x k ) can serve as excellent preconditioners for iterative solvers of linear systems having (the global) A as coefficient matrix, as noted earlier in Section 3.3.5.2. A recent application is for finding quadrature weights for integration over the surface of a sphere [118]. A GMRES-based iterative procedure leads to an operation count of O(N 2 ), accuracy O(h 4 ), and allows N values up to around N = 106 . This can be compared to O(N 3 ), spectral accuracy, and N -values up to around N = 104 for the SPH-based approach described in [105]. A still more recent RBF-FD based algorithm (with code available in MATLAB Central) features O(N log N ) operations and O(h 7 ) accuracy [219].

Downloaded 11/04/15 to 142.103.160.110. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

Chapter 4

Global RBFs for Solving PDEs

Kansa proposed in 1990 to use the analytic derivatives of an RBF interpolant to approximate the spatial derivatives of a PDE [162, 163]. For time-dependent equations, this procedure can be combined with Method of Lines (MOL)-type time stepping. RBFs feature a number of unique strengths that make this general approach very attractive. In particular, they combine spectral accuracy with • complete geometric flexibility, • opportunities for local refinements, and • freedom from the need to cluster nodes merely to fit peculiarities of special coordinate systems (e.g., as occurring by the poles in spherical coordinates). In view of these special strengths, we will here consider only RBF implementations for PDEs that • include two or more space dimensions, • are of high-order (or of spectral) accuracy, and • are not based on lattice (grid)-type node layouts. We will thus not survey the literature on issues such as RBFs for ODEs, RBFs for scalar nonlinear wave equations, or “indirect RBF methods” (where high accuracy in more than 1-D may be challenging). Our focus will also be less on small test problems (often showing just that the approach can be made to work) and more on studies in which direct comparisons have been made against the very best of previously available numerical approaches. Furthermore, our focus will be on collocation approaches, enforcing the PDE pointwise. Other formulations (such as Galerkin based [15, 280]) are also possible. The chapter is among the shorter in this book since a number of issues relevant to RBFs for PDEs will instead be covered in the following chapters, focusing on RBF-FD approximations and on geoscience applications.

91

Downloaded 11/04/15 to 142.103.160.110. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

92

Chapter 4. Global RBFs for Solving PDEs

4.1 A few larger-scale applications One of the earliest RBF applications was to model the water flow in a part of Hong Kong harbor, in the process also demonstrating the effectiveness of domain decomposition with RBFs [287]. A problem of 2-D flame front motion in turbulent combustion is considered in [164], reporting computer times of 2 minutes against 14 hours for a FD level set approach for the same task. An application to radiative transport is discussed in [167]. There exists a fairly extensive RBF literature on elasticity applications, such as vibrating plates, etc., both using global RBFs and using RBF-FD discretizations (cf. [228] for some comparisons). For global RBFs, see in particular [226] (which also provides an example of using a stable algorithm, RBF-QR, for application work). Some references to elasticity calculations with RBF-FD are given in Section 5.2.2.3. The subject of solving PDEs over curved surfaces has a substantial history both in terms of application areas and with regard to numerical approaches. Some different methods (including RBFs) are discussed in [243]. RBFs are particularly well suited to the task, as they avoid the singularities that are intrinsic to any surface-bound coordinate system, exemplified already for a sphere with the two poles in spherical coordinates. Another key advantage is that spectral accuracy readily becomes available (in contrast to, e.g., surface triangularization–based FEM discretizations). An RBF-based approach for computeraided design (CAD) mesh repair is described in [188]. The solution of PDEs over biological surfaces was pioneered in 1952 by A. M. Turing [270] in the context of pattern formation on animals. Both this topic and also other processes occurring on cell surfaces and on other types of biological membranes (2-D manifolds in 3-D space) have since received extensive mathematical and numerical attention. The solutions presented in [205] use global RBFs, in combination with the orthogonal gradient method (OGr). The surface can be described with the same cloud of points that is also used for the RBF-based PDE discretization. Figure 4.1 illustrates an N = 560 node set defining the shape of a frog and two RBF-generated solutions to the brusselator equations over this surface. This nonlinear reaction-diffusion system closely models actual formation of skin patterns on animals (for which the time evolution gets frozen at some embryonic stage). The article [120] describes solutions to another reaction-diffusion type PDE (the Barkley model [2]), again over surfaces of biological objects (cf. Figure 4.2). The global RBF approach was in this case somewhat different (a “projection” approach, for which the surfaces were given in the form of level surfaces of specified 3-D functions). An RBF-FD implementation is described in [244].

4.2 Time-independent PDEs The solutions to time-independent PDEs often represent equilibrium states in science and engineering applications. Many RBF implementations for PDEs in the literature address these type of equations, e.g., the Poisson equation, steady states for the Navier-Stokes equations, or equilibria in elasticity. RBFs have been applied very successfully in all these (and many other) steady-state cases.

4.2.1 Kansa’s formulation vs. symmetric formulations Recalling our focus on collocation approaches, there are two main ways to use RBFs to discretize elliptic PDEs. To keep the notation simple, we consider Poisson’s equation on

Downloaded 11/04/15 to 142.103.160.110. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

4.2. Time-independent PDEs

93

Figure 4.1. Top row: Patterns produced by RBF solutions of the Brusselator reaction-diffusion equations for two different parameter settings. Bottom row: The skin patterns on two frog species (Panamanian golden frog and Poison dart frog, respectively). Illustrations provided by Cécil Piret. Top row images: Reprinted with permission of Elsevier. Bottom left: Reprinted with permission of Brian Gratwicke. c Arco Images GmbH/Alamy. Reprinted with permission. Bottom right: 

Figure 4.2. Snapshots in time of numerical solutions to the Barkley model. Illustration provided by Grady Wright.

a domain Ω, with a Dirichlet boundary condition on ∂ Ω:  u(x) = g (x) on boundary ∂ Ω, Δu(x) = f (x) in interior Ω discretized at node locations x 1 , . . . , x N on ∂ Ω and x N B

B +1

(4.1)

, . . . , x N in Ω. The two formu-

lations can be summarized as follows: Kansa’s formulation: Let the solution be of the form u(x) =

N  j =1

λ j φ(||x − x j ||) .

(4.2)

Downloaded 11/04/15 to 142.103.160.110. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

94

Chapter 4. Global RBFs for Solving PDEs

Enforcing (4.1) at all nodes gives a linear system for the λ j of the following structure: ⎤ ⎡ ⎤⎡ ⎤ ⎡ ⎢ φ(||x − x j ||)| x=x i ⎢ ⎢ ⎢ ⎢ −−−−−−−−− ⎢ ⎣ φ(||x − x ||)| x=x j i

g ⎥ ⎥⎢ ⎥ ⎢ ⎥ ⎥⎢ ⎥ ⎢ ⎥ ⎥⎢ λ ⎥ ⎢ ⎢ ⎥ ⎥⎢ ⎥=⎢ . ⎥⎢ ⎥ ⎢ − ⎥ ⎥ ⎥⎢ ⎥ ⎢ ⎥ ⎦⎣ ⎦ ⎣ f ⎦

(4.3)

This approach has proven to be widely successful, even if rare possibilities for singularities have been noted [146]. Symmetric formulation: The assumed form of the solution is now changed from (4.2) to NB N   u(x) = λ j φ(||x − x j ||) + λ j φ(||x − x j ||) ; j =1

j =NB +1

i.e., we use φ(||x − x j ||) rather than φ(||x − x j ||) as the RBF at the interior nodes. The counterpart to (4.3) becomes (in abbreviated notation) ⎤ ⎤ ⎡ ⎡ ⎤⎡ | g ⎥ ⎥ ⎢ ⎢ ⎢ φ | φ ⎥ ⎥ ⎥ ⎢ ⎢ ⎥⎢ ⎢ ⎥ ⎥ ⎢ ⎥ ⎢ λ | ⎢ ⎥ ⎥=⎢ ⎢ ⎥⎢ , ⎥ ⎢ − ⎥ ⎢ −−−−− + −−− ⎥⎢ ⎥ ⎥ ⎢ ⎢ ⎥⎢ ⎥ ⎦ ⎣ f ⎦ ⎣ φ | 2 φ ⎦ ⎣ | with (for the standard RBF choices) an ensured symmetric and positive definite coefficient matrix [63, 294]. Although this is an obvious advantage, actual numerical performance of the two approaches seems relatively comparable (with different comparisons suggesting slight advantages either way, e.g., [171, 216]). Generalizations to other linear (or nonlinear) operators is straightforward. If Newton’s method is used, the cost per iteration becomes (as usual) comparable to that of solving a linear case, as either will require the solution of a full N × N linear system.

4.2.2 Boundary conditions 4.2.2.1 Runge phenomenon–related issues

Like for interpolation, as described in Section 3.3.4, PDE discretization errors also tend to be the largest at boundaries. In many PDE cases, these errors will not remain local but will “contaminate” the solution across the full domain, making it particularly important to keep them small in the first place. An early suggestion in the case of elliptic PDEs is to apply, at domain boundaries, not just the boundary condition but also the PDE itself [70]. The idea has some similarities with the Not-a-Knot and Super Not-a-Knot approaches for interpolation, introduced independently around the same time [92] (cf. Section 3.3.4). In both cases, there arises an imbalance between the number of equations and number of unknowns, which then is remedied by centering a matching number of additional RBFs just outside the boundary. Still other options for increasing boundary accuracy include local node clustering (essentially the only option in case of polynomialbased spectral methods) and local reduction at the boundary in the value of the shape parameter  [112, 282].

Downloaded 11/04/15 to 142.103.160.110. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

4.3. Time-dependent PDEs

95

4.2.2.2 Approximating actual boundary singularities

Entirely separate from the fact that derivative approximations often are less accurate at boundaries (as just discussed), some PDEs will feature actual solution singularities at boundaries without this always being immediately obvious from their formulation. In the case of elliptic equations, this can happen along a smooth boundary if there is a change from Dirichlet-to-Neumann-type BCs and also at domain corners (with or without a change in the type of the BC). For specialized RBF treatments in some such cases, see [17, 208]. For time-dependent PDEs, time-space corner singularities will arise where initial and boundary conditions meet, unless an infinity of compatibility conditions are satisfied [73, 81]. The errors that these singularities cause are usually smaller than the truncation errors of low-order methods, with the result that they have received relatively little attention [28]. For PS methods, time-space corner singularities have been handled successfully by including specially constructed corner singularity functions [74]. No corresponding results have been reported when using RBFs.

4.2.3 Poisson’s equation in a circle and first PDE use of a stable algorithm Naturally, the earliest implementations of RBFs for PDEs were focused on showing that the approach is viable for very simple test problems. We summarize here the study [171] since this also introduced the novelty of applying a stable algorithm to the task of solving PDEs by RBFs (using Contour-Padé, the only available such choice in 2003). In order to allow easy comparisons of RBFs against FD2 and PS methods (which require simple domain shapes), the domain was chosen as the unit circle. All the node sets had NB = 16 nodes on the boundary ∂ Ω and NI = 48 nodes in the interior of Ω. For FD2, the nodes were equispaced in both angle and radius and for PS again equispaced in angle but of Chebyshev type radially. For RBF, the nodes were somewhat irregularly scattered (cf. Figure 4.3). Figure 4.4 shows a typical result. Kansa’s approach is here applied to (4.1) with g (x) and f (x) selected in such way that the equation has as its solution u(x) = 100/(100 + (x − 0.2)2 + 2y 2 ). Even when using RBF-Direct, the RBF approach is seen to be the most accurate option (if the optimal  is used). The use of a stable algorithm not only improves the accuracy further still but also makes the choice of “optimal”  very much less critical. It can be noted that a second-order method (such as FD2, or second-order finite elements) gains a factor of four in accuracy when step sizes are halved, i.e., in 2-D when four times as many nodes are used. The error is thus inversely proportional to the number of nodes. In the present test case, the errors for MQ and IQ RBFs are roughly 10−6 times those for FD2, implying that, in order to match this accuracy, FD2 would need the node count N = 64 to be increased by a factor of about 1 million.

4.3 Time-dependent PDEs We will focus our discussion on two main varieties of time-dependent PDEs, (i) with a diffusive operator present and (ii) without one, describing a convective process. The RBF literature for time-dependent PDEs is so far dominated by implementations in the former category, which includes much of fluid mechanics (e.g., the Navier-Stokes equations). The latter type is, however, the more challenging one due to time-stability issues caused by the lack of any “natural” viscous damping of high/spurious modes. The rest of this chapter will be focused on the convective case.

Downloaded 11/04/15 to 142.103.160.110. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

96

Chapter 4. Global RBFs for Solving PDEs

Figure 4.3. Illustration of how polar type grids create highly nonuniform resolutions in different directions near the origin. (a) Polar: Equispaced in radius. (b) Polar: Chebyshev along each diameter. (c) Irregular (but avoiding clustering)—as typically used in RBF contexts.

Figure 4.4. Max norm errors, as functions of , when solving a 2-D Poisson test problem using three choices of RBFs: GA, IQ, MQ. (a) Using RBF-Direct. (b) Using Contour-Padé . The dashed lines across both subplots compares the accuracies reached by FD2, PS (both independent of ) and by the best result reached by use of RBFs. Reprinted with permission from Elsevier. [75]

While spatial derivatives virtually always are of integer orders, there are also applications where fractional order diffusion operators arise. Since such derivatives are global in nature (in contrast to integer order derivatives, which characterize a function only in the immediate vicinity of a point), global RBFs offer for such cases an intriguing computational opportunity [206].

4.3.1 Purely convective PDEs In order to study the RBF discretization of a purely convective operator separately from boundary-related issues, we consider next PDEs on the surface of a sphere. This is, of course, also a geometry of major relevance in the geosciences. The most important aspect of numerical methods for such PDEs is that they advect solution features intact, without trailing waves trains or diffusion. Two common test cases in the numerical literature are the following: 1. Solid body rotation, linear advection of the initial condition at an arbitrary angle α to the polar axis, as discussed previously in Section 2.1.6.1.

Downloaded 11/04/15 to 142.103.160.110. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

4.3. Time-dependent PDEs

97

2. Vortex roll-up (also known as deformational flow or cyclogenesis), where an angular velocity is applied to the initial condition, spinning it up around an axis of rotation. The linear advection equation (2.8) becomes on a unit sphere, with latitude-longitude coordinates denoted (θ, λ)   ∂h u ∂ ∂ + +v h =0. ∂t cos θ ∂ λ ∂θ

(4.4)

Depending on how the velocity, (u, v), is prescribed, the height field h will simply translate around the sphere at some angle α, as demonstrated in Section 2.1.6.1; spin up in a vortical pattern [79]; or both [77]. To discretize (4.4) using RBFs, we first need the partial derivatives with respect to λ and θ acting on an RBF φk = φ(r = ||x − xk ||) centered at the node xk and then evaluated at the node x, where in spherical coordinates ( r (λ, θ) = 2(1 − cos θ cos θk cos(λ − λk ) − sin θ sin θk ),   1 ∂ φk 1 ∂ φk (r (λ, θ)) ∂ r ∂ φk = = cos θk sin(λ − λk ) , (4.5) cos θ ∂λ ∂λ ∂r r ∂r ∂ φk (r (λ, θ)) ∂ r ∂ φk = = (sin θ cos θk cos(λ − λk ) − cos θ sin θk ) ∂θ ∂θ ∂r   1 ∂ φk · . (4.6) r ∂r   >  N  1 d φk 1 ∂ h(λ, θ) >> = c cos θ sin(λ − λ ) |{θ j ,λ j }N , k k j k > j =1 cos θ ∂ λ r dr {θ j ,λ j }Nj=1 k=1    B λj ,k D λ h = cB λ = (B λ A−1 )h.

(4.7)

Here A is the standard global RBF interpolation matrix given in (3.3), and the vector h is the field evaluated at the node locations. In order to derive the differentiation matrix ∂ D λ that approximates ∂ λ , we have used the fact that c = A−1 h. D θ is derived in the same manner. The general semidiscrete equation for linear advection on a sphere can now be written as ⎤ ⎡ ∂h ∂t The discretization of

⎥ ⎢ + ⎣ u(α, θ, λ, t )D λ + v(α, θ, λ, t )D θ ⎦ h = 0.   

(4.8)

DN

∂ π has canceled out the apparent singularity at the poles θ = ± 2 , ∂λ 1 in (4.4). This singularity was an artifact of the spherical coordicos θ

implied by the factor nate system itself being singular at the poles and had nothing to do with the physical problem that the equation describes. Hence, the singularity was bound to disappear once the equation was reformulated into RBFs since the basic x, y, z space is free of any artificial singularities. This advantage of RBFs (not depending on any surface-bound coordinate system) is not limited to the present case of the sphere but applies to PDEs on any curved surfaces (cf. Section 4.1).

Downloaded 11/04/15 to 142.103.160.110. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

98

Chapter 4. Global RBFs for Solving PDEs

In the two cases just mentioned, the discrete governing equations become the following: 1. Solid body rotation: With u and v defined as in Section 2.1.6.1, the motion becomes a solid body rotation around an axis that is inclined the angle α to the polar axis, with discrete governing equation ∂h ∂t

= −DN h.

2. Vortex roll-up: If u = ω(θ ) cos θ and v = 0, where (λ , θ ) represents a coordinate system rotated an angle α such that the new north pole is at (λ p , θ p ), two diametrically opposed vortices will spin up with an angular velocity ω (see p. 1076 in [79] for coordinate transformation from (λ, θ) to (λ , θ )); the discrete governing equation becomes ∂h   = −ω(θ j )DN h. ∂t Notice that, whether the field h is rotated an angle α relative to the coordinate system (case 1) or instead the coordinate system and thus the nodes are rotated (case 2), we arrive at the same RBF discrete advection operator DN , except for a coordinate rotation  from (λ, θ) to (λ , θ ) and hence the notation DN (see Appendix A of [79] for the proof of the rotational invariance of DN ). In other words, although the governing equation originally was formulated in a spherical coordinate system, the RBF advective operator DN is not only free from pole singularities but also completely independent of how the original spherical coordinate system was oriented in space. In particular, there is no trace left in it of the original coordinate system’s polar axis. No other spectral method has this property—complete independence from the underlying coordinate system and its orientation. In the subsections that follow, we will study both convergence and time stability of the RBF method and give comparisons against other commonly used spectral methods.

4.3.2 Solution and convergence Figures 4.5 (a), (c), (e) display the solution, the magnitude of the error over the domain, and the convergence of the cosine bell test case using 4096 Minimal Energy (ME) nodes on a sphere. About 30 lines of MATLAB code suffice for a complete implementation of this test (cf. the code listed in Appendix B of [79]). Figures 4.5 (b), (d), (f) display similar results for a vortex roll-up test case that has the analytical solution   ρ(θ )     (4.9) h(λ , θ , t ) = 1 − tanh sin(λ − ω(θ )t ) . γ Here, the angular wind velocity ω(θ ) is defined by ⎧ ⎨ v0 3 3 sech2 (ρ(θ )) tanh(ρ(θ )) 2ρ ω(θ ) = ⎩ 0

if ρ = 0, if ρ = 0,

where ρ = 3 cos θ , γ = 5, v0 = 2πR/T = 1, and R = 1 (representing a unit sphere). Figure 4.5(c) shows that the dominant error is a ring at the base of the bell where the function is

Downloaded 11/04/15 to 142.103.160.110. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

4.3. Time-dependent PDEs

99

Table 4.1. Performance comparison between commonly used high-order methods for the cosine bell test case. Time-stepping schemes used: RBF→ RK4, SPH and DF→ Leapfrog, SE → third order Adams-Bashforth bootstrapped with RK4, DG → TVD RK3. Both SE and DG used a cubed sphere mesh. The column labeled “2 error” shows relative errors. The cosine bell had initially a height of 1,000 m, and time is in dimensional units, with one full revolution corresponding to 12 days.

Method RBF [79] Spherical Harmonics (SPH) [251] Double Fourier (DF) [251] Spectral Element (SE) [261] Discontinuous Galerkin (DG) [198]

2 error 0.006 0.005 0.005 0.005 0.005

Time step 1/2 hour 90 seconds 90 seconds 45 seconds 6 minutes

Number of node points 4096 32768 32768 24576 7776

only C 1 (featuring a discontinuity in its second derivative), leading to low-order algebraic convergence, as shown in Figure 4.5(e). Furthermore, there is no evidence of any trailing dispersive wave trains as were seen in Figure 2.10 (c, d), showing all deviations in the solution of less than 1 m (or an error < 0.001) in white. In contrast, the solution for the vortex roll-up case is C ∞ , giving spectral convergence, as seen in Figure 4.5(f). As with other spectral methods, the rate of convergence for global RBFs is determined by the smoothness of the underlying function.

4.3.3 Time stability The discrete RBF linear advection operator DN on the surface of a sphere is the product of the positive definite matrix A and the antisymmetric matrix B (as given in Section 4.3.1). As a result, the eigenvalues are purely imaginary [79, 210], as shown in Figure 4.6(a). The only difference between the two test cases is that vortex roll-up has a diagonal matrix   W = ω(θ j )|{θ j }N multiplying DN . This causes a slight scatter of the eigenvalues off the j =1

imaginary axis, as seen in Figure 4.6(b). However, most fall well within the RK4 stability domain, and those that do not turn out to have no adverse effects on the solution until past the time when genuine solution features have become too fine (i.e., too highly oscillatory) in space to be resolvable by the current number of nodes. In case that eigenvalues just to the right of the stability domain had interfered with accurate solution during the relevant integration time, hyperviscosity stabilization would likely have resolved this (cf. Section 5.3.2.1). The present eigenvalue structure (with an absence of “spurious” eigenvalues far out in the complex plane) leads to very favorable time stability properties for the RBF discretization, allowing the use of remarkably large steps during explicit time stepping. This is seen in Table 4.1 for the cosine bell test case. The conclusion is that RBF can take much longer time steps with less degrees of freedom in space (i.e., number of grid points) for the same accuracy. The same holds for the vortex wrap-up case. A table of comparison to other methods is given in Section 4.4, where local node refinement has also been considered.

4.3.4 Error with respect to time and : Comparisons between different RBFs To study this topic, it suffices to consider the cosine bell test case. The test case is here nondimensionalized, so that t = 2π corresponds to one full revolution of the bell around the sphere. Over different periods of time, first up to time t = 10 and then up to time t = 10,000, Figure 4.7 demonstrates that, for nonsmooth RBFs such as thin plate splines (TPS)

Downloaded 11/04/15 to 142.103.160.110. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

100

Chapter 4. Global RBFs for Solving PDEs

(a)

(b)

(c)

(d)

(e)

(f)

Figure 4.5. (a) Numerical solution of the cosine bell test case after 1 revolution (t = 12 days) on an “unrolled” sphere for N = 4096 ME node points. (b) Numerical solution of the vortex roll-up test case at t = 3 or 6 days. (c) The error (exact, numerical) for (a). Solid line marks the base of the cosine bell where the function is C 1 , while the dashed line outlines the sphere. (d) The error (exact, numerical) for the vortex roll-up case in (b). (e) The ∞ and 2 error after one revolution for the cosine bell test case as a function of the spacing of the ME node sets (h ∼ N −1/2 ) on a log-log plot. The dashed line is a plot of N −3/2 , showing that the convergence is cubic. (f) The 1 and 2 error for vortex roll-up test case at t = 3 (6 days) on a log-linear plot, showing spectral convergence. Results are for the GA RBF, RK4 for the time integration with a 30-minute time step (spatial errors dominate), and the node sets used in (e) and (f) N = 529, 1024, 1849, 3136, 4096. Reprinted with permission from Elsevier. [79]

or sixth-order Wendland functions (W6), the error degrades severely in time, whereas smooth RBF give excellent accuracy even up through thousands of revolutions. Using both RBF-Direct and RBF-QR, Figure 4.8 shows the error throughout the full range of 

101

25 20 15 10 Im(λ)

Downloaded 11/04/15 to 142.103.160.110. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

4.4. Static local node refinement for time-dependent PDEs

5 0 −5 −10 −15 −20 −25 −1

0 Re(λ)

1 −8

x 10

(a)

(b)

Figure 4.6. (a) Eigenvalues of −DN for the cosine bell test case with N = 1849 ME nodes  and the GA RBF with  = 6. (b) Eigenvalues of −W DN for the vortex wrap-up scaled by a time step of 3/15 shown with the RK4 stability domain for N = 3136 ME nodes and the GA RBF with  = 6.45. Reprinted with permission from Elsevier. [79]

from zero and upward. There are several observations to be made: • The difference in performance between the smooth RBF choices GA, IQ, IMQ, and MQ is very minor and is indistinguishable for small . • For smooth RBFs and small  (using the RBF-QR method), there is almost no loss in accuracy between t = 10 and t = 10,000. With the RBF-Direct, there is at least a factor-of-five loss in accuracy even for the best  choice possible. • The rapid loss of accuracy for piecewise smooth RBF is caused by their lack of smoothness. Smoother ones, such as W4 and W6, give more accurate results than do TPS, cubics (CU), or W2. Just as for FD and PS methods, it is highly beneficial to use smooth basis functions even when the solution that is convected does not possess matching smoothness. • In order to offer some accuracy over longer time integrations, also the Wendland functions need to be scaled so that they become relatively flat ( small), causing their A-matrices to lose their sparseness (however, their differentiation matrices are full matrices in all cases). • In the case of GA, the ill-conditioning for RBF-Direct occurs earlier than for the other smooth RBF types, leaving a small gap between the  ranges of RBF-Direct and RBF-QR (see Figure 4.8(b)). More details on this test calculation can be found in [108].

4.4 Static local node refinement for time-dependent PDEs When refining nodes, the best accuracy is typically obtained when there is a gradual rather than an abrupt transition in node density between different regions. If the change in the

Downloaded 11/04/15 to 142.103.160.110. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

102

Chapter 4. Global RBFs for Solving PDEs

Figure 4.7. The time evolution of the ∞ error for the cosine bell test case for a variety of RBF types: (a) From t = 0 to t = 10. (b) From t = 0 to t = 10,000. All results in this figure and in Figure 4.8 used N = 1849 ME nodes. Reprinted with permission from Elsevier. [108]

Figure 4.8. The ∞ error as a function of  (a) at t = 10 and (b) at t = 10,000. Solid lines indicate how far down in  it was possible to use RBF-Direct (inverting the interpolation matrix A, etc.) when calculating the RBF differentiation matrices. The dashed lines show results obtained by using the RBF-QR algorithm. Reprinted with permission from Elsevier. [108]

node density is too sudden, then there will also be sudden changes in phase speeds of waves and in the wavelengths that can be supported, potentially causing artificial wave interferences or other unphysical effects. Consequently, we will focus below on strategies for smooth clustering. Another issue to be considered is that spatially variable node densities can cause Runge-type phenomena (RP) to occur, as was discussed in Section 3.3.3, in the context of interpolation. To counteract this, the shape parameter of the RBFs need to vary over the domain [77, 112].

Downloaded 11/04/15 to 142.103.160.110. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

4.4. Static local node refinement for time-dependent PDEs

103

4.4.1 Smooth node clustering The node refinement scheme should reflect the physics of the PDE(s) while also producing smooth node distribution. A simple and (sometimes) effective strategy is to simulate electrostatic repulsion between freely moving particles. If every node is assigned the same charge, a random initial node distribution will result in a ME-type node set. However, by applying different charges to the nodes by means of a charge distribution function (or by making the proportionality factor spatially variable in the formula for the force between particles), one can obtain node sets that vary in density but that still locally everywhere have an ME-like structure (i.e., hexagonal-like in 2-D). One effective strategy is to, for each node, determine only the direction of the total force acting on it (ignoring the magnitude) and then move the node some fixed distance in that direction. During successive cycles, the lengths of all the moves are gradually decreased to some small fraction of the typical node separation distance. Where the nodes have high charges, the node density will then be low and vice versa. A wider selection of strategies for generating node sets is discussed in Appendix C.

4.4.2 Varying the shape parameter  When clustering nodes, the shape parameter of the RBF must vary across the domain in order to avoid Runge phenomena in areas more sparsely populated by nodes, as was shown in Figure 3.19. Scaling  according to the inverse of the Euclidean distance to the nearest neighbor node gives nearly optimal results with respect to the interpolation error as well as approximating derivative operators when solving PDEs [77]. The nearest neighbor rule for an RBF φk centered at the node xk is implemented by letting & ' maxk dk,min , k = min dk,min where dk,min is the Euclidean distance between the node xk and its closest neighbor node. The numerator is the maximum separation distance between two neighboring nodes in the entire node set. This representation is chosen so that the quantity in brackets never falls below 1. Here min is a scaling parameter so that k is generally O(1), depending on the density distribution of the nodes. Where the nodes are dense (i.e., tightly clustered), k will be larger, leading to peaked RBF. Where the nodes are sparser, the RBF will be flatter or wider (smaller k ). Figure 4.9(a) shows a smoothly clustered node set on the sphere, and Figure 4.9(b) shows the corresponding variation in the shape parameter. Notice that  varies from about 11 in the center, where the nodes are tightly clustered, to just below 2 near the perimeter.

4.4.3 Condition number of the RBF interpolation matrix when using spatially variable  Since the RBF differentiation matrices depend on inverting the global RBF interpolation matrix A, it is important to consider how varying both node density and  across the domain effects the condition number of this matrix. Figure 4.10 shows that the condition number of A for ME nodes grows much more rapidly as function of the number of nodes N than the refined (clustered) nodes. Normally, larger N is detrimental to good conditioning. However, by varying the shape parameter over the domain, the basis functions become more linearly independent; i.e., they look different from one another and are no longer translates of an identical function. This offsets much of the ill-conditioning that

Chapter 4. Global RBFs for Solving PDEs

(a)

(b)

Figure 4.9. (a) 1849 smoothly refined(clustered) nodes on a sphere obtained from electrostatic repulsion. (b) -values associated with the clustered node set in (a). Reprinted with permission from Elsevier. [77] 12

10 Condition number of matrix A

Downloaded 11/04/15 to 142.103.160.110. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

104

ME nodes 10

10

Refined nodes

8

10

6

10

4

10

2

10

0

1000

2000 3000 4000 Number of nodes (N)

5000

Figure 4.10. The condition number of the interpolation matrix A as a function of the total number of nodes N for both ME nodes and refined nodes. Reprinted with permission from Elsevier. [77]

occurs as N grows large. Hence, for the refined nodes, the condition number grows by only about two orders of magnitude, while for the ME nodes, it grows by about seven orders of magnitude.

4.4.4 An advective PDE test utilizing spatially variable  The moving vortices test case is a combination of the deformational flow test with solid body rotation [197]. The form of the analytical solution is given by (4.9), with v0 = 2π/12. However, since the vortices are no longer stationary, as in Section 4.3.2, a series of rotations must be performed to calculate the analytic solution at a given point. The details can be found in Appendix A of [197]. In addition, when applying node refinement, the node set must also rotate as the vortices translate across the sphere. This does not affect the differentiation matrices, but the solution must be interpolated to the new node locations at every rotation of the node set. The total computational cost of  (N 2 ) operations per time step is still the same. For the numerical results presented here, the node set was rotated at every time step. RK4 was implemented to advance the system in time with

4.4. Static local node refinement for time-dependent PDEs

105

Latitude 0 90 Longitude

180

−90

0 90 Longitude

180

−90

0 90 Longitude

180

0 90 Longitude

180

Latitude

Latitude

−90

Latitude

Latitude Latitude Latitude Latitude

Downloaded 11/04/15 to 142.103.160.110. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

−6

90 45 0 −45 −90 −180 90 45 0 −45 −90 −180 90 45 0 −45 −90 −180 90 45 0 −45 −90 −180

−90

90 45 0 −45 −90 −180 90 45 0 −45 −90 −180 90 45 0 −45 −90 −180 90 45 0 −45 −90 −180

x 10 3 2 1 −90

0 90 Longitude

180 x 10−6 15 10 5

−90

0 90 Longitude

180 x 10−5 10 8 6 4 2

−90

0 90 Longitude

180 x 10−4

0 90 Longitude

180

−90

5 4 3 2 1

Figure 4.11. Numerical solution and magnitude of the error at t = 3, 6, 9, and 12 days for the moving vortex case with N = 3136 refined nodes and Δt = 18 minutes. For the plots of the solution, contours with an interval of 0.05 are shown. The dashed lines correspond to values of the height field h < 1, while solid lines are h > 1. Reprinted with permission from Elsevier. [77]

inverse quadratic (IQ) RBFs, φ(r ) = 1/(1+(r )2), being used for the spatial discretization. The test case is run to t = 12. Figure 4.11 shows contour plots of the solution and the magnitude of the errors at different times with a refined node set of N = 3136 nodes. 4.4.4.1 Numerical results: Near-uniform nodes versus refined nodes

When doing local refinement with any type of numerical method, one expects a reduction in computational cost for a given accuracy. RBFs are no exception. For example, to achieve a relative 2 error of about 2·10−3 , N = 4096 nodes are needed in the near-uniform ME case compared to N = 900 nodes with node refinement, as shown in Figure 4.12(left). At O(N 2 ) arithmetic operations per time step, this results in computational savings by a factor of 20. Furthermore, the memory requirement for each full matrix is also reduced by a factor of 4.5. Although there is not a strict CFL definition for scattered nodes, to ensure stability, the time step cannot be larger than the physical velocities the node spacing can support. Thus, with node refinement, a more restrictive time step is required, as seen in Figure 4.12(right) (however, costwise, far outweighed by the reduced cost of the spatial approximations). Although a time step of Δt = 45 minutes is stable with the refined node set, temporal errors will dominate for time steps larger than Δt =20 minutes, seen in the figure as where the refined error begins to grow. Table 4.2 gives a comparison for this test case to other state-of-the-art methods. Finite volume (FV), on a latitude-longitude grid (lat-lon), was run both without (0.625◦ ×0.625◦ ) and with adaptive mesh refinement (AMR), the base level being N = 2592 (5◦ × 5◦ ) with three adaptive refinement levels. On a cubed sphere, FV and discontinuous Galerkin (DG)

Downloaded 11/04/15 to 142.103.160.110. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

106

Chapter 4. Global RBFs for Solving PDEs

Figure 4.12. The normalized 2 error at day 12 with respect to (left) Reprinted with permission from Elsevier. [77]



N and (right) Δt .

Table 4.2. Results using RBF, DG, and FV for the moving vortices test case run for 12 days. The FV AMR has a base level of 5◦ × 5◦ , corresponding to N = 2592 if a point is centered in each control volume.

Method

RBF, refined

RBF, ME

RBF, MD DG [197] FV (lat-lon) [197] FV (cubed-sphere) [217]

N

Δt (min.)

900 60 1849 30 3136 18 900 180 1849 120 3136 60 900 180 1849 120 3136 60 9600 6 AMR (2592-base, 3 levels) Variable 165888 (0.625◦ × 0.625◦ ) 10 38400 30

Normalized error 1 2 4.0 · 10−3 2.6 · 10−4 4.6 · 10−5 1.6 · 10−2 6.5 · 10−3 2.0 · 10−3 1.6 · 10−2 6.9 · 10−3 2.9 · 10−3 2.1 · 10−3 1.4 · 10−3 5 · 10−4 5 · 10−3

5.4 · 10−3 3.9 · 10−4 7.2 · 10−5 2.8 · 10−2 1.5 · 10−2 4.1 · 10−3 3.1 · 10−2 1.5 · 10−2 4.7 · 10−3 7.1 · 10−3 2.2 · 10−3 2.0 · 10−3 2 · 10−3

were run without AMR. The DG method was run suboptimally with respect to the CFL limit (personal communication with Ram Nair). The conclusion is: For the same accuracy, RBF use significantly fewer nodes with much larger time steps. In Figure 4.13(left), the runtime per time step is shown, growing quadratically with the number of nodes N . Since the RBF method requires O(N 2 ) operations per time step, this result is expected. When using refined nodes, the computational cost is slightly higher since an extra matrix-vector multiply is needed to interpolate the solution to the new rotated node locations at each time step. The total runtime as a function of the normalized

4.4. Static local node refinement for time-dependent PDEs

107

10 ME

0.1

0.01 500

ME Refined

Total runtime (minutes)

Runtime per time step (s)

Downloaded 11/04/15 to 142.103.160.110. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

0.5

Refined N = 3136 N = 1849 N = 3136

1 N = 1849

N = 900

N 2 /(2.5 · 107 ) 1000

N

2000

3000 4000

0.1 −5 10

N = 900 −4

10

−3

−2

10 10 Normalized 2 error

−1

10

Figure 4.13. For refined and ME nodes: (Left) Runtime per time step as a function of N . (Right) Total runtime (i.e., wall clock time) as a function of the normalized 2 error. Reprinted with permission from Elsevier. [77]

2 error is plotted in Figure 4.13(right), showing the large benefits of using node refinement. Reaching an 2 error of 4·10−3 requires a runtime of about 12 seconds with refined nodes (N = 900) compared to 3 minutes with near-uniform nodes (N = 3136). Another important performance aspect is memory usage, which due to the full differentiation matrices grows rapidly with increasing N . In comparison, less than N = 1000 refined nodes are needed to achieve a comparable error, which corresponds to a reduction of the memory cost for each full matrix by a factor of six. The lowest error reached with refined nodes is also almost two orders of magnitude lower than for the largest ME node set. The important outcome of the present tests on this benchmark problem is to note that global RBFs—when used with local refinement—is able to solve it over an order of magnitude more accurately than any of the main methods previously reported in the literature (cf. Table 4.2), and it did so using only about 6 minutes of computer time on a standard PC.

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

Chapter 5

RBF-Generated FD (RBF-FD) Methods

The earliest reference to RBF-FD methods seems to be a conference presentation by Tolstykh in 2000 [263]. It was independently noted (in passing only) in [55] that the convergence of RBF interpolants to polynomial form when  → 0 would suggest RBF-generated FD methods. Also in 2002, some RBF-FD applications were suggested in [273]. Two works from 2003 put RBF-FD methods firmly “on the map” [246, 264]. That year, they were also considered in [289]. The RBF-FD approach is still in rapid development, with many issues only partially resolved or in some cases not even systematically addressed yet. Computational cost was an obvious concern with global RBFs and the key driving force behind the development of RBF-FD methods. When using RBF-Direct, finding an interpolant or calculating a differentiation matrix (DM) each cost O(N 3 ) operations for N nodes, with an additional O(N 2 ) operations each time a DM is applied (e.g., during time stepping). In parallel with the successful applications of global RBFs, as described in the previous chapter, efforts were under way on several fronts to dramatically reduce these costs. Of several potentially viable approaches (such as “fast algorithms” based on multipole ideas, innovative preconditioners, domain decomposition, partition of unity, etc.), RBF-FD is at present the leading option. With RBFs being a rapidly developing field, directions may of course change again in the future. However, in the context of solving PDEs, O(N 3 ) (or even O(N 2 )) cost “barriers,” preventing scalings to massive problem sizes, belong to history.

5.1 Basic concepts In the case of grid-based regular FD approximations, as described in Chapter 1, the same stencil shape and weights can be (re-)used at all node points, possibly with minor modifications at boundaries. In the present scattered node case, each stencil becomes different. With N nodes across the full domain, a common strategy is to center an RBF-FD stencil at each of these and let it extend over its n − 1 nearest neighbors, for a total of n nodes in each stencil. Until recently, numerical stability issues has limited n mostly to relatively small values, such as n between 5 and 15 in 2-D. The “hyperviscosity” approach (Section 5.3.2.2) allows this in some cases to be increased by at least an order of magnitude, allowing approximations of high accuracies. Conceptually, RBF-FD can be seen as an extreme case of overlapping domain decomposition, with a separate domain surrounding each node point. 109

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

110

Chapter 5. RBF-Generated FD (RBF-FD) Methods

Figure 5.1. Schematic illustrations of static node layout following a curved material interface but with regular lattice structure away from it and a snapshot of a node set that is updated intermittently to economically resolve an evolving vortex pair.

5.1.1 Node layouts Appendix C summarizes some different node distribution procedures. While regular grids can be problematic in connection with global RBFs, this might be less so in local RBF-FD contexts. One might consider using grid-like node layouts (and regular FD approximations) over extended regions for which there is no need to gradually vary node densities in order to accommodate irregular boundaries or material interfaces. Figure 5.1 illustrates two typical scenarios. The left subplot shows an example of nodes straddling a curved material interface and then transitioning to become a regular FD lattice for the main part of the domain (discussed further in Section 8.2). The right subplot illustrates dynamic local refinement with a node density that matches the solution features that are to be resolved. Many variations are possible. As an example, [52] considers Poisson’s equation at an indented corner and employs extensive local refinement to improve accuracy. An alternate approach is to then instead consider specialized basis functions obeying the local asymptotic features [17, 208].

5.1.2 Calculation of nearest neighbors nodes A key step in all RBF-FD calculations is to determine, for each of N nodes, some number n − 1 of its nearest neighbors (for a total of n nodes in each stencil; it is assumed that n  N ). A “brute force” approach would be to find all pairwise distances between nodes and then sort all columns in the resulting distance table—for a total cost of O(N 2 ) operations. In contrast, a “k-d tree” (short for k-dimensional tree) algorithm first rearranges the node data in O(N log N ) operations, after which a further O(N log N ) operations suffice for finding some fixed number of nearest neighbors to all the nodes. Many effective numerical implementations are available, such as knnsearch in MATLAB’s Statistics toolbox and kdtree in MATLAB Central. Typical input for these routines would be a list of N scattered nodes x i , and the output would be a matrix in which the n entries in row i would be the indices j for the nearest neighbors x j to x i (starting with a pointer to the element x i itself).

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

5.1. Basic concepts

111

Figure 5.2. The conformal mapping from a periodic square 2-D domain λ, θ ∈ [−π, π] to the surface of a torus, as described by (5.1). Reprinted with permission from Elsevier. [95]

The cost increase associated with increases in k (say, going from 2-D to 3-D) is usually very small, making the k-d tree approach very well suited also for point sets on manifolds, such as for scattered nodes on the surface of a sphere. Another application of nodes on manifolds occurs if one wants to search for nearest neighbors in a periodic 2-D domain, such as λ, θ ∈ [−π, π] (a search type typically not directly supported by k-d tree implementations). Since the conformal (angle preserving) mapping ⎧ f = 1/( 2 − cos θ), ⎪ ⎪ ⎪ ⎨ x = f cos λ, ⎪ y = f sin λ, ⎪ ⎪ ⎩ z = f sin θ

(5.1)

scales all distances the same way within each local area, we can alternatively conduct the search for nearest neighbors on the resulting 3-D torus, illustrated in Figure 5.2.

5.1.3 Node ordering The sparsity pattern of a RBF-FD DM depends on how the nodes are ordered in the physical domain. For example, a standard N = 6400 MD node set on the surface of a sphere typically features no particular node ordering, leading to n = 50 DM structures of the type seen in Figure 5.3(a). Such scatter of the N · n = 320000 DM nonzero entries degrades computational efficiency, especially for memory access in parallelized RBF-FD codes, and increases the cost of direct sparse solvers. In order to modify the node ordering and obtain DMs with tight diagonalized structure, bandwidth reduction algorithms, such as reverse Cuthill-McKee (MATLAB command symrcm), and locality-sensitive hashing algorithms are used [21]. The latter decomposes the physical domain into coarse blocks; within each block, the nodes are reordered using a space filling curve (e.g., raster, Hilbert, Peano) such that nodes nearby in physical space are nearby in index space (memory location). The result of applying reverse Cuthill-McKee on the matrix in Figure 5.3(a) is seen in part (b) and of applying a locality sensitive hashing algorithm (with a raster indexing by column) in part (c). In some cases, nodes might originally be ordered advantageously, e.g., according to their spatial closeness. The right panel of Figure 7.1 shows such a case, a N = 25600 MD node set on sphere, directly producing tight diagonal structure.

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

112

Chapter 5. RBF-Generated FD (RBF-FD) Methods

Figure 5.3. (a) Sparsity of the RBF-FD DM for the advective operator on a sphere (the operator inside the brackets in (4.4)) using a 50-node stencil and a k-d tree algorithm to find the nearest 49 neighbors when no attention has been paid to the node ordering. (b) Sparsity pattern of the RBF-FD DM in (a) after using a reverse Cuthill-McKee algorithm. (c) Sparsity pattern of the RBF-FD DM in (a) after using a locality sensitive hashing algorithm.

5.1.4 Calculation of RBF-FD weights Traditional FD formulas are usually derived only in 1-D, and with generalizations to more dimensions limited mostly to Cartesian type grids since this allows such 1-D formulas to be applied separately in each spatial direction. The simplest approach for finding the weights for 1-D FD formulas (although not the most effective one; cf. Section 1.1.1) is to consider the FD weights to be unknowns and then enforcing that the resulting FD formula becomes exact for the monomials 1, x, x 2 , . . . , up to as high degree as possible. As long as the nodes xi are distinct, the resulting Vandermonde matrix in (1.6) is guaranteed to be nonsingular. This unisolvency property is no longer ensured if the approach is generalized to scattered nodes and multivariate polynomials in 2-D or higher. If we replace the 1-D polynomial test functions {1, x, x 2 , . . .} with d -D functions ϕi (x), i = 1, 2, . . . , n, the counterpart to (1.6) for finding the weights to approximate a linear operator L at the location x c becomes ⎤ ⎤ ⎡ ⎤⎡ ⎡ Lϕ1 (x)| x=x w1 ϕ1 (x 1 ) ϕ1 (x 2 ) · · · ϕ1 (x n ) c ⎥ ⎢ ϕ (x ) ϕ (x ) · · · ϕ (x ) ⎥ ⎢ w ⎥ ⎢ ⎢ Lϕ2 (x)| x=x ⎥ ⎢ 2 1 2 2 2 n ⎥⎢ 2 ⎥ ⎢ c ⎥ ⎢ ⎥ ⎥ ⎢ (5.2) ⎥. .. .. .. .. ⎥ ⎢ .. ⎥ = ⎢ ⎢ ⎥ ⎦⎣ . ⎦ ⎢ ⎣ . . . . ⎣ ⎦ wn ϕn (x 1 ) ϕn (x 2 ) · · · ϕn (x n ) Lϕn (x)| x=x c

The key idea for RBF-FD is to choose as test functions ϕi (x) = φ(||x − x i ||), i.e., radial functions centered at the nodes x i , i = 1, 2, . . . , n. Equation (5.2) then becomes ⎤⎡ ⎤ ⎡ w1 φ(||x 1 − x 1 ||) φ(||x 2 − x 1 ||) · · · φ(||x n − x 1 ||) ⎢ φ(||x − x ||) φ(||x − x ||) · · · φ(||x − x ||) ⎥ ⎢ w ⎥ ⎥⎢ 2 ⎥ ⎢ 1 2 2 2 n 2 ⎥⎢ . ⎥ ⎢ .. .. .. ⎥⎢ . ⎥ ⎢ ⎦⎣ . ⎦ ⎣ . . . wn φ(||x 1 − x n ||) φ(||x 2 − x n ||) · · · φ(||x n − x n ||) ⎡ > ⎤ Lφ(||x − x 1 ||)> x=x c > ⎢ ⎥ ⎢ Lφ(||x − x 2 ||)> x=x ⎥ ⎢ c ⎥ =⎢ ⎥ . (5.3) .. ⎢ ⎥ . ⎣ ⎦ > Lφ(||x − x n ||)> x=x c

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

5.1. Basic concepts

113

The matrix A in the linear system above is the same as the matrix that arises in the RBFDirect approach for finding the coefficients λi for the standard RBF interpolant s(x) = $n λ replaces this form by s(x) = i φ(||x − x i ||). 4As noted in Section 3.1.3.5, one often$ 3$i =1 n n λ φ(||x − x ||) + λ together with the constraint n+1 i =1 i i =1 λi = 0. When using this i alternate procedure, the same type of derivation that led to (5.3) will instead lead to the relation ⎤ ⎤ ⎡ Lφ(||x − x ||)>> ⎡ ⎤⎡ | 1 w1 1 x=x c ⎥ ⎥ ⎢ ⎢ ⎢ .. .. ⎥ ⎥ ⎥ ⎢ ⎢ ⎥ ⎢ ... . ⎥ ⎢ A | . ⎥ ⎢ ⎢ ⎥⎢ > ⎥ ⎥ ⎢ ⎥⎢ (5.4) > ⎥. − x ||) Lφ(||x | 1 ⎥ ⎢ wn ⎥ = ⎢ ⎢ n ⎥ ⎢ x=x c ⎥ ⎢ ⎢ ⎥⎢ ⎥ ⎦ ⎣ ⎣ − − − + − ⎦⎣ − ⎦ − wn+1 1 ··· 1 | 0 L 1 | x=x c

The A-matrix in (5.4) is the same as the matrix in (5.3), and the last entry wn+1 in the solution vector should be ignored. The relation (5.4) can immediately be extended to include also linear and higher-order terms. For instance, in the linear 2-D case, (5.3) becomes replaced by ⎤ ⎤ ⎡ Lφ(||x − x ||)>> ⎡ ⎤⎡ 1  1 x1 y1 w1 x=x c ⎥ .. ⎥ ⎢ ⎢ ⎢ .. .. .. ⎥ ⎥ ⎢ ⎥ ⎢ ... ⎥ ⎢ . ⎥ ⎢ A  . . . ⎥ ⎢ ⎢ ⎥⎢ ⎥ ⎥ ⎢ Lφ(||x − x ||)>> ⎢ ⎥⎢ ⎥ ⎢  1 xn y n ⎥ ⎢ w n ⎥ ⎢ n x=x c ⎥ ⎥ ⎢ ⎢ ⎥⎢ ⎥ , (5.5) = ⎢ − − − + − − − ⎥⎢ − ⎥ ⎢ ⎥ − ⎥ ⎢ ⎢ ⎥⎢ ⎥ ⎥ ⎢ ⎢ ⎥⎢ ⎥  L 1 | x=x ⎢ 1 ··· 1 ⎥ ⎢ wn+1 ⎥ ⎢ ⎥ c ⎥ ⎢ ⎢ ⎥⎢ ⎥ ⎣ x1 · · · xn  ⎦ ⎣ wn+1 ⎦ ⎣ 0 ⎦ L x | x=x c y1 · · · y n  wn+3 L y | x=x c

where again only the weights w1 , w2 , . . . , wn should be used. The relation (5.5) can be derived as follows: Denoting the (n + 3) × (n + 3) matrix in ˆ it follows from (3.8) and (3.9) that the LHS by A, s(x) = [φ(||x − x 1 ||) . . . φ(||x − x n ||) 1 x y] [λ1 . . . λn γ1 γ2 γ3 ]T = [φ(||x − x 1 ||) . . . φ(||x − x n ||) 1 x y] Aˆ−1 [ f1 . . . fn 0 0 0]T . Applying a linear operator L and evaluating the result at location x = x c gives Ls(x)| x=x = [Lφ(||x − x 1 ||)| x=x . . . c

c

Lφ(||x − x n ||)| x=x L1| x=x Lx| x=x Ly| x=x ] Aˆ−1 [ f1 . . . fn 0 0 0]T . (5.6) c

c

c

c

If fi = 1 and f j = 0, j = i (cardinal data), it holds that Ls(x)| x=x = wi , i = 1, . . . , n. c These n relations represent different special cases of (5.6). Placing these relations side by side gives ⎤

⎡

[w1 , . . . , wn ] = [Lφ(||x − x 1 ||)| x=x . . . Lφ(||x − x n ||)| x=x L1| x=x Lx| x=x Ly| x=x ] Aˆ−1 c

c

c

c

c

1 ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ − ⎢ ⎢ ⎢ 0 ⎢ ⎣ 0 0

..

.

− ··· ··· ···

1 − 0 0 0

⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥. ⎥ ⎥ ⎥ ⎥ ⎦

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

114

Chapter 5. RBF-Generated FD (RBF-FD) Methods

Adding three extra columns to the rightmost matrix so that it becomes the identity matrix allows it to be ignored. This adds three “nonsense” entries [wn+1 , wn+2 , wn+3 ] to the LHS. Then multiplying by Aˆ from the right and transposing gives (5.5). The pattern of (5.5) continues to arbitrary orders and numbers of dimensions. While the operator L needs to be linear, there is no requirement that it has constant coefficients. The formulas (5.4) and (5.5) apply immediately also for approximating variable coefficient convection-diffusion operators, such as  + α(x, y)∂ /∂ x + β(x, y)∂ /∂ y (as an alternative to generating separate weight sets for the individual derivative operators). As will be seen in Section 6.1, this is crucially important in cases where coordinate systems feature “unphysical” singularities (such as by the poles in spherical coordinates; the separate terms may then be singular while the combination is not). The operator L above need not be of differential type. For example, if the nodes x i are scattered over a sphere, (5.2) with all entries in the RHS the same, set to the integral over the sphere of a single RBF, the wi will form an associated quadrature weight set. A problem with this approach—and a remedy for it—are discussed in [105].

5.1.5 Interpolation properties of PHS-based RBF-FD stencils In the remaining parts of Section 5.1, we focus on the accuracy that RBF-FD approximations provide near the center of their stencils for the task of interpolation. While this is very relevant also to the use of RBF-FD approximations for solving PDEs, other considerations then enter, such as the eigenvalue spectrum of the corresponding differentiation matrices (DMs). As will be noted in subsequent chapters, some of the present observations will then need to be partly revised. One such case is the almost unconditional benefits seen here (for interpolation) when increasing the degree of the supplementary polynomials as far as the stencil size permits. For PDE solutions, there is often some optimal degree past which they no longer are equally beneficial. In the present context of creating RBF-FD stencils, it was found in [3, 76] that PHS basis functions have several attractive features. Following the discussion in these references, we consider a stencil with n = 56 ME-like nodes, as shown within the unit circle (R = 1) in Figure 5.4(a). In most RBF-FD cases, one is interested only in the accuracy near the stencil center, although near-one-sided approximations will become needed in connections with boundaries and interfaces. Comparing interpolation errors at the two sets of evaluation locations seen in Figure 5.4(b, c), it transpires that the errors in the latter case generally become about 100 times larger than in the former case. It therefore suffices to consider the former (near-center) case in the following displays in Figures 5.6 and 5.7. We choose as a test function f (x, y) = 1 + sin(4x) + cos(3x) + sin(2y),

(5.7)

shown over R = 1 in Figure 5.5(a). In two steps of refinement, we then reduce R to 0.3 and to 0.1. The displays in Figure 5.4 remain unchanged apart from being shrunk in proportion to R. Subplots (b) and (c) of Figure 5.5 show how the test function, when zoomed in accordingly, appears gradually more like a plane (as in the case of any smooth functions). The top two rows of subplots in Figure 5.6 gives an overview how log10 (|max error|) changes with the RBF degree (here 3, 5, 7, . . . , 15) and the degree of (2-D) polynomial terms (with degree −1 corresponding to RBF only, i.e., to (5.3), degree 0 to (5.4), degree 1 to (5.5), etc.). A number of observations can now be made:

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

5.1. Basic concepts

115

Figure 5.4. (a) RBF-FD stencil (n = 56 nodes) in the case of R = 1. (b)–(c) Two sets of evaluation points at which the approximation’s interpolation accuracy is evaluated.

Figure 5.5. Test function (5.7) when displayed over x 2 + y 2 ≤ R2 , with R = 1, R = 0.3, and R = 0.1, respectively.

• The top two rows of subplots (computed using quadruple vs. double precision) are virtually identical, showing standard double precision to be entirely adequate in all the cases. This may be surprising in view of the very high condition numbers that arise in some of them (cf. the bottom row of subplots). However, as noted previously in [153], these reflect only a limitation in the standard definition of condition number (which is sensitive to scaling issues for the rows and columns of a matrix and also in situations where this has no adverse effect on the computed interpolant). • The right edges of the subplots correspond to including 2-D polynomials of degree 9. Accuracy (and nonsingularity) collapses if this is increased one more degree since there are 55 linearly independent 2-D polynomials of degree 9 and 66 of degree 10. With n = 56 nodes in the stencil, the last 66 rows in the degree 10 counterpart to (5.5) will thus become linearly dependent, making the matrix singular. • Increasing the degree of the RBF does little to increase accuracy. However, it is seen to always be beneficial to include polynomials, and the resulting accuracy increases rapidly until the breakdown just mentioned. There are a couple of ways to explain why the improvements should be expected (cf. Section 5.1.7). • Looking down the left edge in the three subplots in the middle row, we see very little, if any, error improvement as R (i.e., h) is decreased. This is a typical example of stagnation error (SE, previously discussed for global RBFs in Section 3.3.7). As also discussed in Section 5.1.7, including increasing order polynomials causes this to disappear, in turn for interpolation, then for first derivatives, etc.

Chapter 5. RBF-Generated FD (RBF-FD) Methods

R = 0.3

−3 0

−1 −2 −4 −6 −7 −8 −9 −10 −11

−3

−10 −11 −12

−5

−3

−10 −11 −12

0

−1 −2 −4 −6 −7 −8 −9 −10 −11

−3

−5

−3 0

−1 −2 −4 −5 −6 −7

0 2 4 6 8 Degree of polynomial terms

1

22

2220 18 16

1412

3

8

14 12

8

8

−5

−3 0

−1 −2 −4 −5 −6 −7 −3

−9

−3

−2

0 −2 −3 10

RBF degree m ; φ(r) = r

m

log10 | error | , double precision

−3 4

10

4

1

5

6

0 2 4 6 8 Degree of polynomial terms

3

14

7

5

4

16

12

7

6

1

10

(condition number)

9

18

8

9

3

13

8

10

10

10

11

8

11

5

−14 −13

2018 121416

12 10

15 12

11

6

24

17

13

7

0 2 4 6 8 Degree of polynomial terms R = 0.1

19

13

9

−9 −8 −7 −6

1

21 14

17 15

3

−4

−10 −9

−8 −7

−6 −5

0 2 4 6 8 Degree of polynomial terms R = 0.3

12

19

−14

5 −4

−4 −3

21

19

−15 −14 −13

7

3

21

−13 −12

13

9

1

−9 −8 −7 −6

−9

−10

15

11

5

15

21

9 7

17

0 2 4 6 8 Degree of polynomial terms R = 0.1

11

5

0 2 4 6 8 Degree of polynomial terms R=1

1

17

−8

13

7

−2

3

19

15

3

−4

0 2 4 6 8 Degree of polynomial terms R = 0.3

19

−4

9

−10 −9

−8 −7

1

17

13 11

3

21

−1

15

5

−6 −5

17

7

−4

−4 −3

0 2 4 6 8 Degree of polynomial terms R=1

19

1

log

9

5

21

RBF degree m ; φ(r) = r m

9

5 −2

13 11

7

3

15

11

7

1

−9

−3

−2

0 −2 −3

m

RBF degree m ; φ(r) = r

9

−9

13

−8

11

17

−15 −14 −13 −12

15

13

21 19

17

−10

15

19

−4

log10 | error | , quad. precision

19 17

R = 0.1

21

−5

R=1 21

−1

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

116

0 2 4 6 8 Degree of polynomial terms

Figure 5.6. Top two rows: Accuracy (log 10 |error|) in PHS-based RBF-FD approximations, as functions of the RBF degree and the degree of included polynomial terms, calculated using 34 decimal digits quad precision vs. in standard 16-digit double precision, Bottom row: log10 of the condition number for the system (5.5) (calculated using quad precision).

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

5.1. Basic concepts

117

• Using even order RBFs of PHS type, i.e., φ(r ) = r p log r for p = 2, 4, . . . leads to an almost identical counterpart to Figure 5.6. While some theory suggests using φ(r ) = |r | p = 1, 3, 5, . . . in odd dimensions and φ(r ) = r p log r , p = 2, 4, 6, . . . in even dimensions, the distinction may not be necessary.

5.1.6 Interpolation properties of GA-based RBF-FD stencils Changing from PHS to GA RBFs will leave certain features unchanged, such as the error for evaluation nodes near the boundary being a couple of orders of magnitude larger than for evaluation nodes near the center. Based on Figure 5.7 [76], we can again make a number of observations: • The top two rows of subplots differ this time significantly in terms of the accuracy that is reached when using extended vs. double precision. In contrast to the PHS case, the condition number when using GA (or other smooth RBFs) controls closely the accuracy that can be reached by RBF-Direct. With standard double precision, errors generally decrease until the condition number reaches around 1015 and then grows rapidly (cf. the dashed horizontal lines in the bottom two rows of subplots). In the R = 1 case, the 10−5 error curve is thus not quite reachable, and quad precision will barely suffice to reach the best error in the R = 0.3 case. Given the high cost of software-implemented extended precision, the use of a stable algorithm is in these cases a more attractive option. The conditioning issue is then bypassed and at a cost of about 10 times RBF-Direct for RBF-GA and only slightly more for RBF-QR. Using high-degree polynomial terms close to doubles the number of equations in (5.5), increasing the cost by a factor of about eight; i.e., there is about a breakeven in both cost and accuracy between using RBF-Direct with close to the maximal degree polynomials and RBF-GA without polynomials. • Looking along the left edges of the subplots in the top row (the case of RBFs only— no supporting polynomials), we see how decreasing  (as for many cases of global RBFs) causes the error to first fall deeply and then bounce back up slightly again. The location of this dip remains roughly unchanged as R is decreased (equivalent to refining the node set, keeping the stencil size fixed)—here occurring around  = 0.5. This invariance of dip location under refinement was noted previously in [52]. • Increasing the degree of the polynomial terms immediately improved the accuracy in the PHS case. With GA RBFs, this trend is much delayed. When it finally occurs, as noted above, it offers no clear advantage over having used instead a stable algorithm and no polynomials. • Stagnation errors arise when not using a stable algorithm or appending with highorder polynomials. Maintaining an acceptable condition number while the typical node separation h is decreased requires  to be increased, eliminating convergence. If we choose to use double precision and RBF-Direct (i.e., not to use a stable algorithm), comparison between Figures 5.6 and 5.7 shows that PHS and GA can produce similar accuracy levels (possibly with a slight edge in favor of PHS). Advantages with PHS are that (i) they achieve this without requiring computations right on the “edge” of severe ill-conditioning, and (ii) they do not require any difficult precise choice of a parameter ( in case of smooth RBFs).

Chapter 5. RBF-Generated FD (RBF-FD) Methods

0.1

−10

20

20

10

10

8

1

15 20 25

0.5 0.2 0.1

30

0 2 4 6 8 Degree of polynomial terms

2 1 0.5 0.2 0.1

4

15

8

2

5

−12 −13 −15 −14

−7 −8.5 −10 −10.5 −11

0 −1 −3−2 −4 −5

8

−10

−6 −7 −8

−12 −14

−11

−13 −11

15

4

50

20

4

−7 −8 .5

−10 −8.5

−7

0 −1

0

50

15 20 25 30 35

40 0 2 4 6 8 Degree of polynomial terms

10 5 2 1 0.5 0.2 0.1

20

100

50

25

100

2

8

200

100

5

0 2 4 6 8 Degree of polynomial terms R = 0.1

500

8

4

200

−4 −3

0.2

1000 15

8

500

−7

−14

0 2 4 6 8 Degree of polynomial terms R = 0.3

1000

2

−9

−3 −2

−3

0.1

−8

1 −9 −6

0.2

5

0.5

2

−4 2 −−3 1 − 0 0.1 0 2 4 6 8 Degree of polynomial terms R=1 1000

0.2

0.5

10

−6−5

1

−5

200

−−45 −−67 −7 −6 −5 4 −

−8

0.5

20

2

−3 −4

1

50

−9 −10

−2

200

−12 3

5

500

−1

−1

1000

4

10

0 2 4 6 8 Degree of polynomial terms R = 0.1

0 −1 −3−2 −4 −5

10

−1

−7−6

20 0

20

0.1

100

−1

50

−3

50

100

5.5

0.2

−10

−2 −3

200

0 .5 −−1−1110 −12 −13 4 −1 5 −1

−9

100

0.5

−9 −8

200

2

−3 −2

500

5 1

0 2 4 6 8 Degree of polynomial terms R = 0.3

1000

10

0.5

−8

0 2 4 6 8 Degree of polynomial terms R=1

−4

| error | , double precision

.5

−1

0.2 0.1

100

2

−4 −5

−5

2

(condition number)

−8

0.5

500

10

−10

1

0.5

5

−−45 −7

2

−1

10

−3 4 −

200

2

1

5

−10

2

−2

−8

−1

5

500

20

−9

10

−6 −7

10

1000

50

−3

−3

20

500

Shape parameter ε (GA RBFs)

50

20

1000

log

−4

50

100

R = 0.1

−3 −4 −5

−1

0

100

0.2

Shape parameter ε (GA RBFs)

200

0

| error | , extended precision 10

log

500

200

−2

500

0.1

log

R = 0.3 1000

−2

Shape parameter ε (GA RBFs)

R=1 1000

0

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

118

8 15

20 25 30 35 40 45

50 0 2 4 6 8 Degree of polynomial terms

Figure 5.7. Top two rows: Accuracy (log10 |error|) in GA-based RBF-FD approximations, as functions of shape parameter  and degree of included polynomial terms, calculated using 80 decimal digits extended precision vs. in standard 16-digit double precision, Bottom row: log10 of the condition number for the system (5.5) (calculated using 80 decimal digits extended precision). Dashed horizontal lines mark -values for which the condition number (not using polynomials) are around 1015 .

5.1. Basic concepts

119

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

5.1.7 Stagnation errors in the context of PHS-based RBF-FD stencils The critical quantity in the stagnation error (SE) discussion in Section 3.3.7, with smooth RBFs on infinite domains, was the growth/decay rates of the radial functions φ(r ) for r → ∞. SE, meaning lack of convergence when typical node distances h → 0, was again visible along the left boundaries (i.e., with no added polynomials) in the top rows of subplots of Figure 5.6. In these PHS cases, there is no shape parameter  present. The lack of convergence has in this case an entirely different cause, namely, errors penetrating in from the RBF-FD stencil’s outer edge. The 1-D case of the present RBF-FD discretization can provide some qualitative (if maybe not quantitative) insights into the relevant error sources, as indicated in Figure 5.8. With φ(r ) = |r |3 , (A.5) implies that the boundary errors become very severe already for constant data, and these oscillatory errors then decay from both sides in toward the center according to the relatively fast cubic spline decay rate given as the p = 3 case in (2.5). Increasing the RBF degree offers little (if any) boundary improvement, and these errors then decay slower toward the interval center (cf. p = 5, 7, . . . cases in (2.5)), with the net effect that the error near the center might not have improved at all. In view of this observation, it is not surprising that, as seen along the left edges of the subplots (especially in the R = 0.3 and R = 0.1 cases), changing the RBF degree has relatively little effect on the resulting accuracy. Edge errors are quite large, and these then penetrate inward throughout the domain of the stencil, even there becoming the dominant error source. When including a constant and linear terms, the 1-D φ(r ) = |r |3 approximation becomes identical to the natural cubic spline (as shown in Section A.1.2), exact, of course, for all constant and linear data. With still higher-order polynomial terms included, these gradually “take over” and provide the corresponding levels of accuracy under RBF-FD refinement, while the RBF terms continue to provide other benefits, such as safety against singularities due to particular node layouts. Independent of the dimension, one might from the argument above guess that, when using φ(r ) = r p , p odd, together with polynomials of degree , errors under refinement will be of size O(h −k+1 ) when approximating a kth derivative. The two parts of Figure 5.9 show how the error when approximating the Laplacian varies with h when using φ(r ) = r 3 and φ(r ) = r 7 , respectively, in both cases together with polynomials of different orders. For each choice of h, the worst case was recorded over 1000 Halton distributed locations of stencil centers within the unit circle. The log-log plots are in excellent agreement with all the observations above—in particular with the errors being of size O(h −k+1 ) with here k = 2. The thick dashed lines mark E r r o r = 10−15 /h 2 , which is the accuracy barrier in double precision that arises if weights with magnitudes adding up to 10/h 2 are combined with function values that are uncertain to 10−16 . For example, the weights for the standard second-order FD approximation  Δu =



⎡

∂ ∂ ⎢ u =⎣ 1 + 2 2 ∂x ∂y 2

2

1 −4 1

⎤ ⎥ 1 ⎦ u/h 2 + O(h 2 )

(5.8)

add in magnitude up to 8/h 2 , with larger constants times 1/h 2 inevitable for higher-order approximations. The curves in Figure 5.9 reach right down to this barrier, showing that no significant digits have been lost due to any RBF-FD-related conditioning issues. This “barrier” appears in just the same position for regular lattice-based FD approximations, for which the weights are analytically known.

120

Chapter 5. RBF-Generated FD (RBF-FD) Methods

−2

(b) abs(error) halfway between nodes

10

φ(r) = r 3

Data − identically one 3 φ(r) = r φ(r) = r

1.0015

φ(r) = r 7

−3

10

7

−4

10 1.001

−5

10 1.0005

−6

10

1

−7

10

−8

0.9995 −1

−0.5

0 x

0.5

10

1

−1

−0.5

0 x

0.5

1

Figure 5.8. (a) The RBF interpolants for n = 18 node constant equispaced data. (b) The errors at the halfway points between the nodes, corresponding roughly to the extrema of the oscillations in the interpolants. The errors decrease toward the interval center closely following the theoretical rates (2.5) for the two corresponding spline cases, shown by the short straight-line segments at the bottom left. (a) RBF φ(r) = r3

10

(b) RBF φ(r) = r7

10

10

10

5

5

no poly

10

no poly

10

constant

constant

0

0

10

10

poly 2

−5

poly 2

−5

10

10

−10

−10

poly 8

10

poly 8

10

−15

10

poly 1

Error

poly 1

Error

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

(a) Data and RBF interpolants 1.002

−15

−1

10

−2

10

−3

10 h

−4

10

−5

10

10

−1

10

−2

10

−3

10 h

−4

10

−5

10

Figure 5.9. The worst error encountered when approximating the Laplacian using the n = 54 node PHS-based RBF-FD stencils, as described in the text. The curve identifiers (the text strings “no poly” etc.) are in exactly the same locations in the two subplots, illustrating that the changes when going from φ(r ) = r 3 to φ(r ) = r 7 indeed are quite insignificant. Likewise, changing the value of n has little effect on the shown curves. The slopes of the different curves perfectly match the predictions (including the thick dashed lines, which mark rounding errors in double precision of 10−15 /h 2 ).

5.2 RBF-FD for elliptic equations 5.2.1 Implicit RBF-FD stencils Just like for regular FD methods, one can consider both explicit or implicit (compact/Hermite-type) approximations. For example, recalling from Section 1.2.2, we can approxi-

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

5.2. RBF-FD for elliptic equations

121

Figure 5.10. Graphical illustrations of three different stencil shapes: (a) Standard FD2 approximation to Δu. (b) Implicit FD4 approximation to Δu. (c) Implicit size {10,5} RBF-FD approximation. Reprinted with permission from Elsevier. [292] ∂ 2u

∂ 2u

mate the Laplacian Δu = ∂ x 2 + ∂ y 2 either explicitly as in (5.8) or implicitly by rewriting (1.16): ⎤ ⎤ ⎡ 1 ⎡ 1 1 −8 1 4 4 ⎥ ⎢ ⎢ 1 ⎥ [1] Δu = ⎣ 1 −5 1 ⎦ u /h 2 + ⎣ − 18 (5.9) − 8 ⎦ Δu + O(h 4 ). 1 1 1 1 − 4 4 8 The approximations for Δu associated with (5.8) and (5.9) can be illustrated graphically, as shown in Figure 5.10(a, b), respectively. The lines from the x 1 -node remind us that we are approximating Δu at that particular location. Single circles indicate locations where we use weights for u only and double circles where we use weights for both u and Δu. In view of the number of entries in the RHS stencils in (5.8) and (5.9), we describe them as {n = 5, m = 0} and {n = 9, m = 4} approximations, respectively. Figure 5.10(c) similarly illustrates a scattered node {n = 10, m = 5} stencil. In analogy with (5.9), we can write this scattered node approximation as ⎡

⎤

a9

⎡

⎢ ⎥ b5 ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ a7 ⎥ ⎢ ⎥ ⎢ ⎥ ⎥ u +⎢ ⎢ ⎥ ⎢ ⎥ a6 ⎢ ⎥ ⎢ b ⎥ ⎢ 4 ⎥ ⎢ ⎥ ⎣ ⎦

⎢ a5 ⎢ ⎢ ⎢ a2 ⎢ ⎢a ⎢ 8 ⎢ [1] Δu ≈ ⎢ a1 ⎢ ⎢ ⎢ ⎢ a4 ⎢ ⎢ ⎣ a3 a10

⎤ b2

b6

⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ Δu, ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎦

b10

where it remains to determine the weights, here {a1 , a2 , . . . , a10 } and {b2 , b4 , b5 , b6 , b10 }. This can be done similarly to the explicit case in Section 5.1.4, enforcing that the result becomes exact for an RBF centered at each of the n = 10 nodes but now also for the Laplacian of the RBF centered at the m = 5 nodes for which the stencil’s RHS includes Δu entries. It is again usually preferable to include at least a constant and its matching constraint. Like for the explicit case in Section 5.1.4, the implicit (compact) RBF stencil approach generalizes directly from the case of the Laplacian, used for illustration above, to arbitrary order variable coefficient operator L in any number of dimensions, to inclusion of polynomial terms, etc. The linear system for the a and b coefficients will in general

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

122

Chapter 5. RBF-Generated FD (RBF-FD) Methods

be nonsingular. In order to employ a stencil of this kind for solving Lu = f , it is first rewritten in analogy with how (1.16) follows from (5.9). For more information about implicit RBF-FD approximations, see [52, 292]. In applications of RBF-FD methods, it is common to use explicit RBF-FD stencils when approximating convective terms (thus avoiding having to solve a linear system each time step) but implicit ones for elliptic equations. Reasons for the latter include (i) when solving an equation Lu = f , applying a stencil also to the f-values barely adds any cost, as these typically are known values; (ii) implicit stencils offer higher accuracy for the same stencil size; (iii) implicit stencils tend to preserve diagonal dominance better at increasing accuracies; and (iv) for a given accuracy, their smaller stencil sizes simplify the implementation of boundary conditions.

5.2.2 Some tests in the literature Like earlier for global RBF methods, we will again focus on cases where the RBF-FD strengths are most pronounced, and we will thus not cover applications in which RBFs are applied in only 1-D and also not implementations that are based entirely on Cartesian lattice-type node layouts. 5.2.2.1 Poisson’s equation in 2-D circular geometry

The following test problem has been considered in several independent studies, employing different numerical techniques. Consider Poisson’s equation Δu ≡

∂ 2u ∂ 2u + = f (x, y) ∂ x2 ∂ y2

over the unit circle, with both the RHS f (x, y) and the Dirichlet boundary data defined to be consistent with the solution u(x, y) = 25/(25 + (x − 0.2)2 + 2y 2 ). RBF-FD solution: All three subplots of Figure 5.11 are taken from [292]. All the schemes that are referred to (FD2, compact FD4, and three RBF-FD schemes) feature diagonal dominance. Subplot (c) shows that, for a total of N = 200 nodes), an n = 9, m = 5 implicit RBF-FD scheme reaches a max norm error of about 5·10−7 . However, the very small increase in stencil size to n = 10 and making it maximally implicit (m = 9) improves the error by a factor of about 10, down to about 5 ·10−8 . Regular FD schemes, based on polar grid–type mesh (as illustrated in subplot (a)) are seen to give much lower accuracy as a consequence of the highly uneven node densities in radial and angular directions near the center. In contrast to the RBF-FD discretizations above, neither of the two approaches below leads to diagonally dominant matrices or to an easily usable sparsity structure for the resulting linear systems. Global RBF solution: The same test problem was among those considered in [171]. With Kansa’s approach and N = 50 MQ RBFs (centered in a scattered manner, reminiscent of the N = 200 illustration in Figure 5.11(b)), the error was found to be 4.6 · 10−9 . PS (Fourier-Chebyshev) solution: The same test problem was recently again revisited by polar grid–based PS discretization in [29]. This study compared a large number of different such variations (see the title of the paper) and reported for up to N = 50 nodes a best error of 3.2 · 10−6 . This falls several orders of magnitude short of the N = 50 global RBF result, for the same reason that FD solutions were less accurate that RBF-FD ones— the very nonuniform discretization near the origin in polar-type grids.

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

5.2. RBF-FD for elliptic equations

123

Figure 5.11. (a) Structured 201 node set for use with explicit FD2 and compact FD4 approximations. (b) Unstructured 200 node set used with the RBF-FD approximations. (c) Max norm errors for the RBF-FD methods as functions of . Reprinted with permission from Elsevier. [292]

In conclusion for this test problem, we can note that the RBF-type discretizations (whether global or of RBF-FD type) apply equally well to irregular domain shapes, whereas polar coordinate–based discretizations are far more limited in that regard. 5.2.2.2 Nonlinear Poisson equation in a doubly connected domain

We consider the nonlinear Poisson equation Δu = e −2x u 3 , with analytic solution y u(x, y) = e x tanh , over the domain shown in Figure 5.12(a). This figure also illustrates 2 a hybrid node set—using a lattice and implicit FD4 approximations where the geometry is sufficiently simple to allow this, and then {n = 10, m = 9} RBF-FD approximations (again diagonally dominant) over the area with irregular nodes (in order to accommodate the curved inner boundary). Figure 5.12(b) shows the resulting error as a function of . It reaches very low levels as  → 0, with the small final increase possibly related to the fact that no particular precaution was taken about controlling the Runge phenomenon at the inner boundary. It has been noted that the location of these error curve “dips” are quite independent on N [9, 53]. Nevertheless, it may be difficult to exploit their high accuracy since they appear to be highly problem dependent, suggesting that problems featuring nontrivial spatial structures may lack these “dips” altogether. 5.2.2.3 Some RBF-FD steady-state applications

The study of the eigenmodes of vibrating membranes [228] shows RBF-FD to compare favorably against both FD and global RBF methods. In [227], the authors consider both square and L-shaped composite plates and compute displacements and stresses using fully irregular, fully Cartesian, and slightly deformed Cartesian node layouts (cf. Figure 5.13). While none of three node distributions represent what the present authors would recommend for either global RBF or for RBF-FD usage (which is non-lattice-based node sets, with ME-like local regularity), the good results presented offer testimony to the high “robustness” of these numerical approaches.

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

124

Chapter 5. RBF-Generated FD (RBF-FD) Methods

Figure 5.12. (a) Domain and node distribution for nonlinear Poisson equation test problem. (b) Max norm error as function of . Reprinted with permission from Elsevier. [292]

Figure 5.13. Three illustrations from [227], showing (a) a fully irregular set of n = 121 nodes, (b), (c) examples of a purely Cartesian grid and a weakly deformed Cartesian grid. Reprinted with permission from Elsevier. [227]

Steady viscous flow past a sphere is a classical problem in fluid mechanics for which a very large number of calculations (and experiments) have been reported, often with different areas of emphasis, such as onset of instabilities,vortex shedding, steady flows in unstable high-Reynolds-number regimes (up to Re = 5000 in [88]); for a survey, see [158]. The study [242] applies the RBF-FD approach to steady 3-D flows up to Re = 270 and iterate the resulting fully 3-D primitive variable discretizations to convergence with successive over relaxation (SOR). The node distributions that are used (cf. Figure 5.14) are conceptually reminiscent of the one shown in Figure 5.12(a) although far denser and fully 3-D. Figure 5.15 illustrates some obtained axisymmetric flow fields. The RBFFD implementation was capable of also obtaining nonaxisymmetric steady solutions that emerge above Re ≈ 210.

5.3 RBF-FD for time-dependent convection-type PDEs 5.3.1 RBF-FD implementations for time-dependent convection-diffusion Many implementation variations are possible, with several tested in [257]. These include the inclusion of stabilizing “upwinding” (which has a long FD history; suggested for RBFFD in [41, 232]). The literature in this area is growing rapidly.

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

5.3. RBF-FD for time-dependent convection-type PDEs

125

Figure 5.14. RBF-FD node distribution within the z = 0 slice of the 3-D, N = 510768 discretization surrounding the sphere. Reproduced from [242], reprinted with permission from Tech Science Press.

Figure 5.15. Some 3-D steady flow fields, shown in a plane of symmetry. Reproduced from [242], reprinted with permission from Tech Science Press.

2-D driven cavity: This is a classical benchmark problem in computational fluid mechanics for which many different numerical implementations have been reported over the years (allowing for good comparisons between methods of accuracy vs. resolution but unfortunately seldom with actual computer costs reported). An RBF-FD implementation is described in [48] (following up on a global RBF version in [47]). Figure 5.16 illustrates that already n = 9, N = 612 can produce results in excellent agreement with earlier literature. The boundaries were (in both studies) handled by irregularly placed nodes on the inside but with one layer of “ghost centers” immediately outside the boundary. The RBFFD code was time stepped with a combination of Forward Euler and Crank-Nicolson. The former is usually quite inaccurate, but that is less of a concern when the goal is only to

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

126

Chapter 5. RBF-Generated FD (RBF-FD) Methods

Figure 5.16. Square driven cavity, Re=1000, using an irregular distribution of N = 612 nodes; image adapted from Figure 8 in [48], reprinted with permission from Sage Publications. (a) Stream function. (b) Vorticity.

reach a steady state. Both 2-D-driven cavity and time-dependent flow past a cylinder were considered in [157]. The shape parameter was varied between stencils to keep the condition number manageable in response to differences in RBF-FD stencil properties/sizes. This study reported first-order accuracy in time for a split-step explicit-implicit approach and somewhat less than third order of accuracy in space (presumably before refining to the level at which saturation errors arise). Further RBF-FD solutions of the driven cavity problem as well as for temperature-driven convection are presented in [274].

5.3.2 RBF-FD implementations for purely convective PDEs A key issue becomes how to secure the stability of explicit time-stepping methods. For diffusive and convective-diffusive PDEs, DM eigenvalues tend to lie well into the left halfplane, and RBF-FD-caused “jitter” will not be strong enough to shift eigenvalues into the right half-plane. However, the purely convective case should preserve energy for all times. For linear PDEs, this means that all eigenvalues of the DM should be purely imaginary. While FD schemes on regular grids often can satisfy this (e.g., by featuring antisymmetric DMs), node location irregularities in RBF-FD cases will invariably cause a scatter, sending the DM eigenvalues off from the imaginary axis and into both the left and the right halfplanes. The only strategy presented so far for controlling eigenvalue “jitter” is based on hyperviscosity [103], which is adapted from turbulence simulations. This technique leaves the physically relevant modes essentially intact while it nudges the eigenvalues for spurious modes over to the left half-plane. It also achieves a second major goal in permitting the use of much larger (and therefore much more accurate) RBF-FD stencils. Without this enhancement, stencils can rarely exceeded around n = 10 − 15 nodes, whereas with it, n-values up to around 100 were instrumental for obtaining the high accuracies reported in [21, 78, 103]. The study [103] discusses two stabilization approaches, the first one of which is best suitable for global RBFs and the second one for RBF-FD approximations.

5.3. RBF-FD for time-dependent convection-type PDEs

127

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

5.3.2.1 Stabilization of explicit time stepping: The A−1 method

We consider here RBF types with the property that the A-matrix is positive definite (e.g., GA, IQ, IMQ, etc. but not MQ). According to the analysis in Section 3.2.1.2, the Amatrix eigenvalues will decrease very rapidly to zero if  is small. One can show that the corresponding eigenvectors at the same time become increasingly oscillatory. The matrix A−1 will have the same eigenvectors, but its eigenvalues are the inverses for those of A; i.e., they will start out O(1) and then rapidly become very large (and again all positive). Hence, adding a term γ A−1 u with a very small constant γ > 0 to the RHS of an MOL d discretization of a convective PDE d t u = Lu will leave all the physically relevant (reasonably smooth) modes essentially intact but will rapidly damp out all highly oscillatory (spurious noise) modes. This A−1 method appears to work particularly well for global RBF discretizations of PDEs. 5.3.2.2 Stabilization of explicit time stepping: Powers of the Laplacian

This is the hyperviscosity version that is best suited for stabilizing RBF-FD discretizations of PDEs, such as pure convection, the shallow water equations, or the NavierStokes/Euler equations. The general concept is again to leave smooth modes intact (in order not to damage accuracy) but to rapidly damp out only such high ones that are certain to be spurious. Details for its application in the setting of solving nonlinear systems of PDEs for fluid flows are in Sections 7.1.3 and 7.2. The approach can be heuristically understood from considering a 1-D situation as follows: ∂ Figure 2.2 illustrated the effect of the ∂ x operator (and of approximations to it) when applied to the Fourier modes that are present on a uniform grid. Figure 5.17 shows sim∂2 ∂4 ∂8 ∂ 12 ilarly the effect caused by the operators ∂ x 2 , ∂ x 4 , ∂ x 8 , ∂ x 12 when these are normalized π

ωh

so that each curve takes the value 1 at ω = ± h ; i.e., it displays the curves (sin 2 )2k , k = 1, 2, 4, 6. The effect of subtracting such a scaled derivative operator from a convective PDE’s RHS will then be negligible for a wide range of low wave numbers, but high ones will be damped out. The transition location above which damping occurs will depend on the choice of k. Analysis in [103] showed it to be suitable to use the same size RBF-FD stencils for the spatial discretization and hyperviscosity. However, it is not at all necessary to use the same type of RBF for the two tasks, as the role of the hyperviscosity is simply to move the eigenvalues of the RHS of the linearized PDE system to the left half of the complex plane, as seen in Figure 7.2. The stencil weights for convection and hyperviscosity are simply added (see (7.2) as an example), so there is no additional cost per time step. There are two RBF cases in which applying the hyperviscosity operator repeatedly is 2 particularly simple—GA and PHS. If GA RBFs (φ(r ) = e −(r ) ) are used for creating the hyperviscosity stencils, the entries for Δk φ(r ) that are required in the RHSs of (5.4)–(5.5) are available explicitly, thanks to the relation Δk φ(r ) = 2k pk (r )φ(r ) ,

(5.10)

where pk (r ) are the regular Laguerre polynomials in case of 2-D and the generalized Laguerre polynomials (also immediately available recursively) in d dimensions, as given in the MATLAB code below. If the symbolic toolbox is available, the command “laguerreL” can be used but is slower. Similarly, for odd-powered P H S, φ(r ) = r m , m ≥ 3 in dimension d , Δφ(r ) is simply given by Δφ(r ) = m(m + d − 2)r m−2 .

(5.11)

128

Chapter 5. RBF-Generated FD (RBF-FD) Methods

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

1 k=1 k=2 k=4 k=6

0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 −3

−2

−1

0

1

2

3 ωh

Figure 5.17. Illustration of how much d 2k /d x 2k -based damping affects different Fourier π π modes across the frequency range − h ≤ ω ≤ h (which is the range present on a grid with spacing h).

In other words, applying the Laplacian to a PHS RBF of degree m gives a new PHS RBF of degree m − 2. To derive higher-order Laplacian operators (i.e., k > 1), the formula for the Laplacian operator of a radially symmetric function, Δ = ∂ 2 /∂ r 2 + ((d − 1)/r ) ∂ /∂ r , is applied repeatedly. function p = Laguerre(ep2r2,d,k) % Laplacian to the power k in dimension d for calculating (5.10) % ep2r2 = ep^2*r^2 n = length(ep2r2); P = zeros(n,k+1); P(:,1) = 1; P(:,2) = 4*ep2r2-2*d; for j=3:k+1 P(:,j) = 4*(ep2r2-2*j-d/2+4).*P(:,j-1) - 8*(j-2)*(2*j+d-6)*P(:,j-2); end p = P(:,k+1);

5.3.2.3 Two test problems

The stabilization approach just outlined was used for two test cases of convective flow over a sphere in [103]. The results are briefly summarized next. In both cases, the time stepping was carried out by the regular RK4 method. Convection of a cosine bell: For the standard cosine bell test case (described previously in Sections 2.1.6.1 and 4.3.1–4.3.4), we get with N = 25600 MD nodes, stencil size n = 74,  = 8, and Δ8 -type hyperviscosity the long-term evolution illustrated in Figure 5.18. In spite of the very long integration time (1000 revolutions around the sphere), we see no hints of instabilities or even any loss in peak height. The only visible errors remain right at the base of the bell (where there is a jump in the second derivative). Figure 5.19 shows how the 2 error at time t = 10 varies with total number of nodes N and stencil size n (with lower  and hyperviscosity powers preferable for lower values of N and n; see [103] for details). The straight-line slopes in this log-log plot indicate cubic convergence with N —as to be expected given that the test function has a discontinuous second derivative. Increasing the stencil size nevertheless significantly improves the accuracy.

129

Figure 5.18. The numerical solution and the magnitude of the errors for the solid body rotation test case, using the stabilized RBF-FD approach. Reprinted with permission from Elsevier. [103] 0

10

Normalized 2 error

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

5.3. RBF-FD for time-dependent convection-type PDEs

−1

10

−2

10

n= n= n= n= n=

−3

10

17 31 50 74 101

3

10

4

N

10

Figure 5.19. The 2 error in the cosine bell convection test case, displayed after 10 revolutions, as function of N and n. Reprinted with permission from Elsevier. [103]

Figure 5.20 contrasts the scattered node calculation in Figure 5.18 with a correspondingly resolved 1-D equispaced node calculation. The errors in the n = 74 RBF-FD case are slightly lower than in the 1-D FD8 calculation, which on a 2-D equispaced Cartesian

Chapter 5. RBF-Generated FD (RBF-FD) Methods

1.5

1

u(x,1000)

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

130

FD10

0.5

FD8 0

FD6 FD4

−0.5

10

FD2 −1 −3

8 −2

6 −1

0

4

1

2

3

2

Order of scheme

x

Figure 5.20. Numerical solution for 1-D 2π-periodic test problem, with a cosine bell of the same width as initial condition and 1000-revolution integration time as for test case in Figure 5.18, using equispaced centered FD approximations of increasing orders in space (together with analytic MOL in time). N = 265 nodes give h ≈ 0.0238, matching the average node spacing in the N = 25600 ME node set on the unit sphere.

lattice would require a stencil size of n = 81. The greatly improved geometric flexibility achieved by the (stabilized) RBF-FD approach thus has not had any adverse influence on the computational accuracy that is achieved. Vortex roll-up test case: This is the same test case that was also previously described in Sections 4.3.1–4.3.4. Figure 5.21 shows the results, at times t = 3, 6, 9, of a calculation that uses using N = 25,600 MD nodes and a stencil size of n = 50. Although the solution rapidly develops increasingly fine structures as time increases, it remains infinitely differentiable at all times. Hence, we see in Figure 5.22 steeper slopes for larger n, reflecting increases in the effective order of accuracy. Figure 5.23 provides computational cost comparisons of this RBF-FD implementation against global RBF methods. For a given level of accuracy, a global RBF method typically will require fewer nodes than an RBF-FD method, but more work will be needed per node point. Hence, the appropriate way to compare is to specify a required accuracy level and then measure how costly it will be to reach this goal with the alternative approaches in terms of both computer wall clock time and memory usage. Figure 5.23 shows the RBFFD to compete very favorably against the global RBF case (which in turn was compared against different approaches in Sections 4.3–4.4). The advantage is close to a factor of 10 in both of the categories (and increasing with higher accuracy requirements).

131

Figure 5.21. The numerical solution and the magnitude of the errors for the vortex roll-up test case, using the stabilized RBF-FD approach. Reprinted with permission from Elsevier. [103]

−2

10

−3

Normalized 2 error

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

5.3. RBF-FD for time-dependent convection-type PDEs

10

−4

10

−5

10

n= n= n= n= n=

−6

10

−7

10

17 31 50 74 101

3

10

4

N

10

Figure 5.22. The 2 error in the vortex roll-up test case, displayed at t = 3, as function of N and n. Reprinted with permission from Elsevier. [103]

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

132

Chapter 5. RBF-Generated FD (RBF-FD) Methods

Figure 5.23. The computational costs of the (stabilized) RBF-FD method vs. the global RBF method for the vortex roll-up test case, for reaching time t = 3 with different accuracy requirements; computations carried out on a single core 1.83 GHz PC. (a) Run time. (b) Memory requirement. Reprinted with permission from Elsevier. [103]

Downloaded 11/04/15 to 142.103.160.110. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

Chapter 6

Global RBF Applications to Geo-Modeling: Spherical Domains The test cases in this chapter include flows on a sphere (2-D) and in a spherical shell (3-D). Extensive comparisons will be provided in terms of accuracy, time stability, and computational cost against “competing” numerical approaches that have been presented in the literature. It should be noted that many of the concepts developed in this chapter are directly applicable to the RBF-FD applications discussed in the next chapter.1

6.1 Vector versus scalar variables and operators on a sphere When modeling with a numerical method that is dependent on a coordinate system, one must be careful since the coordinate system can introduce singularities into the problem although the operators are smooth. For example, the gradient operator, by its very nature, has no singularities on the surface of a sphere. However, if we choose to represent it in 1 ∂ ∂ ˆ spherical coordinates, ∇s = cos θ ∂ λ λˆ + ∂ θ θ, singularities are introduced at the poles θ = ±π/2. Since RBFs are completely independent of the orientation and type of coordinate system in which the operator is posed, the application of ∇s to an RBF, centered at the node (θ j , λ j ) on the surface of a sphere, results in a singularity-free expression, as already noticed in Section 4.3.1: 7 ∇s φ(r ) = cos θ j sin(λ − λ j )λˆ & ' 8 1 dφj ˆ + (cos θ j sin θ cos(λ − λ j ) − sin θ j cos θ)θ . (6.1) r dr At first glance, it might seem that there are two singularities in this expression: (i) ˆ However, in any the fact that λ is undefined at the poles and (ii) the flip in sign of θ. infinitesimal region about the poles, (6.1) is approximated by [sin(λ − λ j )λˆ + cos(λ − ˆ cos θ ( 1 d φ j ), where (λ, ˆ θ) ˆ define a rotating coordinate system. Notice that the exλ )θ] j

j r dr

pression in the square brackets is an orthogonal rotation in the tangential plane formed ˆ This rotation will cancel any corresponding rotation in the (λ, ˆ θ) ˆ coordinate by λˆ and θ. system, leaving the gradient invariant. With regard to the latter point (ii), cos(λ − λ j ) flips sign since λ changes by π when it crosses the poles. Thus, for PDEs with only scalar 1 To be consistent with the notation in the geoscience literature, boldface is used to represent vectors (underlining was used in previous chapters).

133

Downloaded 11/04/15 to 142.103.160.110. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

134

Chapter 6. Global RBF Applications to Geo-Modeling: Spherical Domains

variables acted on by smooth spatial operators, the equations can be posed in a coordinate system with singularities if that is more convenient and then easily apply RBFs for spatial discretization. However, this is not the case for vector fields. For example, consider a smooth velocity field over a sphere. If a spherical coordinate system were to be used, the component of the latitudinal velocity vector will inherently carry a singularity in its solution in order to cancel the corresponding discontinuity in the unit vector θˆ at the poles. As a pole is approached from 0◦ longitude and from 180◦ longitude, the θˆ unit vector flips direction. Since most equations in the geosciences involve vector fields, such as velocity or magnetic fields, the PDEs are best expressed in Cartesian form since such a coordinate system has no singularities. This in turn requires projection operators to confine the flow to the domain. Appendix D gives a variety of projection operators, such as grad, div, and curl, for the surface of a sphere.

6.1.1 An RBF spherical surface Laplacian formulation The derivation below was originally given in [291]. The surface Laplacian in spherical coordinates is Δs =

∂2 ∂ 1 ∂2 − tan θ . + ∂ θ2 ∂ θ cos2 θ ∂ λ2

(6.2)

Since an RBF is invariant to coordinate rotations, the surface Laplacian can be derived as if the RBF is centered at the north pole, eliminating the λ dependence and simplifying the derivation. Furthermore, any node location on the sphere can act as the north pole (notice the nondirectionality of ME or MD node sets). This reduces (6.2) to Δs =

∂2 ∂ − tan θ . ∂ θ2 ∂θ

(6.3)

Applying (6.3) to an RBF φ(r = ||x − x j ||2 ) centered at the node j gives 5 6 2 1 4 − 3r 2 ∂ φ 2 ∂ φ (4 − r ) . Δs φ(r ) = + 4 ∂ r2 r ∂r

(6.4)

Notice that (6.4) depends only on the Euclidean distance between nodes. This is an example of how a basic operator of physics can be easily implemented through RBFs in a way that is free of any connection to surface-based coordinates.

6.2 Shallow-water equations on a sphere The shallow-water equations (SWE) are a good place to start when testing the performance and viability of a novel numerical method for application to systems of nonlinear purely hyperbolic equations. Discussed in the following subsections is (i) RBF discretization of the SWE in Cartesian form, (ii) the linearized equations needed to perform an eigenvalue analysis, (iii) accuracy and stability on two different test cases (steady and unsteady nonlinear flow), (iv) comparison against commonly used spectral methods for the test cases,

Downloaded 11/04/15 to 142.103.160.110. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

6.2. Shallow-water equations on a sphere

135

and (v) an eigenvalue analysis to understand unique time stability properties of the global RBF method as well as the  → 0 limit effect.

6.2.1 Global RBF discretization of the SWE on the sphere in Cartesian form As explained in Section 6.1, when computing vector fields on a sphere, Cartesian coordinates must be used. The SWE in a 3-D Cartesian coordinate system for a rotating fluid are ∂u = − (u · ∇)u − f (x × u) − g ∇h, ∂t ∂h = − ∇ · (hu) , ∂t

(6.5) (6.6)

ˆ u = uˆi+vˆj+w k ˆ is the velocity vector, h where f is the Coriolis force, ∇ = ∂ xˆi+∂yˆj+∂ z k, T is the geopotential height and x = {x, y, z} represents the position vector. Confining the motion to the surface of a unit sphere requires two steps. First, the projection operator defined by D.1 in Appendix D is applied to the gradient and divergence operator appearing in (6.5) and (6.6), i.e., ∇ → P∇ = [p x · ∇, py · ∇, p z · ∇]. Second, the entire right-hand side (RHS) of (6.5) needs to be projected, with the modified differential operators, onto the corresponding x, y, and z directions. For example, in the case of the u momentum equation (corresponding to the velocity in the x-direction), this results in ∂u = ∂t ⎡

⎤ ⎡ ⎡ ⎤ ⎤ u(p x · ∇)u + v(py · ∇)u + w(p z · ∇)u (p x · ∇) y w − zv ⎥ ⎢ ⎢ ⎢ ⎥ ⎥ − p x · ⎣ u(p x · ∇)v + v(py · ∇)v + w(p z · ∇)v + f ⎣ z u − xw ⎦ + g ⎣(py · ∇)⎦ h ⎦ . xv − y u u(p x · ∇)w + v(py · ∇)w + w(p z · ∇)w (p z · ∇)    RHS (6.7) Similarly, ∂v = −py · RHS, ∂t

∂w = −p z · RHS, ∂t

∂h = −(P∇) · (hu) . ∂t

(6.8)

Now that the flow is confined to the sphere, the same methodology as discussed in Section 4.3.1 to derive the RBF DM is used, except that r = ||x − xk || needs to (be expressed ( in Cartesian coordinates, r (x) = (x − xk )2 + (y − yk )2 + (z − zk )2 = 2(1 − xT xk ). Thus, when the projected gradient operator, P∇, is applied to an RBF φk centered at (xk , yk , zk ) and then evaluated at the node locations x j = {x j , y j , z j }Nj=1 , the derivative y evaluation matrices, B jx,k , B j ,k , B jz,k , are obtained: ⎤ ⎡ x ⎤ > > B j ,k  x xT xk − xk > ⎥ φk (r (x)) > ⎢ T ⎢B y ⎥ = ⎣ y x xk − y k ⎦ = ⎣ j ,k ⎦ , > r (x) >> z xT xk − zk B jz,k x=x ⎡

|P∇φk (r (x))|x=x

j

j

(6.9)

136

Chapter 6. Global RBF Applications to Geo-Modeling: Spherical Domains

Downloaded 11/04/15 to 142.103.160.110. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

y

where  denotes differentiation with respect to r . To derive the global RBF DM, DNx , DN , DNz that will approximate p x · ∇, py · ∇, p z · ∇ in (6.7) and (6.8), the B matrices are simply multiplied by A−1 as in Section 4.3.1. For a detailed derivation of the RBF-discretized SWE on a sphere, see pp. 1955–1957 in [80].

6.2.2 Eigenvalue stability: The linearized equations In the following test cases, an eigenvalue stability analysis is performed on (6.5) and (6.6) to understand (i) why the RBF method is so stable, allowing for very long time steps when using explicit ODE solvers, and (ii) how stability is affected as  → 0. To perform the analysis, the SWE are linearized about the initial condition since this is the solution for all time. For both the steady-state and the unsteady flow cases, the shape of the initial condition is advected intact. ˜ and ˜h are approximate solutions to (6.5) and (6.6), they can be written as Assuming u ˜ = u0 + δu1 + O(δ 2 ), u ˜h = h + δ h + O(δ 2 ), 0

1

where u0 and h0 are the initial conditions. Substituting this expansion into (6.7) and (6.8) and considering only perturbations of O(δ) gives the linearized equations on which the eigenvalue analysis is based: ⎤ u1 · P∇u0 + u0 · P∇u1 + f (y w1 − zv1 ) + g px · ∇h1 ∂ u1 ⎥ ⎢ = −p x · ⎣ u1 · P∇v0 + u0 · P∇v1 + f (z u1 − xw1 ) + g py · ∇h1 ⎦, ∂t u1 · P∇w0 + u0 · P∇w1 + f (xv1 − y u1 ) + g pz · ∇h1    RHS1 ⎡

∂ v1 = −py · RHS1 , ∂t ∂ w1 = −p z · RHS1 , ∂t ∂ h1 = −(u1 · P∇h0 + u0 · P∇h1 + h1 P∇ · u0 + h0 P∇ · u1 ). ∂t

(6.10a)

(6.10b) (6.10c) (6.10d)

6.2.3 Numerical studies for global RBF In order to evaluate both the high accuracy and the time stability of the global RBF method when compared to other spectral methods, such as spherical harmonics (SPH), double Fourier (DF), and spectral elements (SE), two test cases for the SWE are addressed: (i) global steady-state flow where the initial (and for all time) velocity field is compactly supported (i.e., nonzero in a limited band region), admitting a solution that is represented as an infinite SPH expansion, and (ii) a forced translating low-pressure/geopotential height center (i.e., trough) that is superimposed on a westerly jet stream. In the latter test case, the trough is analytically kept intact by adding nonlinear forcing terms to the RHSs of (6.7) and (6.8), with the challenge to the numerical method being to minimize dispersive effects. The first two subsections address the solution and convergence of the method, followed by a section on stability and then comparisons to other methods. For all RBF calculations, ME node sets [286] are used.

137

3000m

(a)

(b)

Figure 6.1. Analytical solution for the steady-state test case plotted as orthographic projections centered at 60◦ S (−π/3) and 0◦ E, which is what the solution would look like in nonrotated coordinates. (a) The velocity field sampled at the N = 3136 ME nodes. (b) The contours of the height field h from 2100 m to 3000 m in increments of 100 m. Reprinted with permission from Royal Society Publishing. [80] −9

Error (exact − numerical), N=3136, t=5 days

2

x 10 −6

1.5

x 10

1.8

6

1.6

1

4 2

0

0

−0.5

−2

−1

−4

∞

1.4 Relative error

0.5

θ

Downloaded 11/04/15 to 142.103.160.110. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

6.2. Shallow-water equations on a sphere

1.2 1 0.8 0.6 0.4

2

0.2

1

−1.5 −3

−2

−1

0

λ

1

2

3

0 0

1

2

(a)

3

4

5

Time (days)

(b)

Figure 6.2. (a) The error (exact, numerical) for the height field from the steady-state test case at t = 5 days on an “unrolled” sphere for N = 3136 nodes with the contour lines representing the exact solution in 100-m intervals. (b) Relative 1 , 2 , and ∞ errors in the height field for the steady-state test case as a function of time for N = 3136 ,  = 3.25, Δt = 10 min. Reprinted with permission from Royal Society Publishing. [80]

6.2.3.1 Steady-state nonlinear flow: Compactly supported wind field

This test is a steady-state solution to the full nonlinear SWE with a balanced height field (∇ · (hu) = 0) and a compactly supported but infinitely differentiable wind field. The solution is a mid-latitude jet with the associated height field having a single low over the region encircled by the jet. For a description of obtaining the analytic solution, see [80, 155]. The initial condition and analytic solution for all time is plotted in Figure 6.1, with the numerical solution after 5 days given in Figure 6.2(a) and its error as a function of time given in Figure 6.2(b). For these latter two figures, a node set of N = 3136 was chosen since it gave a relative 2 error of O(10−10 ), the lowest 2 error reported by many of the other spectral methods (see Table 6.1). Figure 6.2(a) shows that the maximum error is O(10−6 ) m, with deviations of less than that being displayed in white. As expected, the dominant error is concentrated in the area with the steepest gradients, requiring higher resolution. Figure 6.2(b) displays the stability of the method, showing that the solution barely grows in time. In fact, the method could easily be run for a month without any

Downloaded 11/04/15 to 142.103.160.110. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

138

Chapter 6. Global RBF Applications to Geo-Modeling: Spherical Domains Table 6.1. Comparison of commonly used spectral methods for the steady-state and unsteady flow test cases, given in Sections 6.2.3.1 and 6.2.3.2. The number in (·) in the RBF section corresponds to the square root of N, which is inversely proportional to the node spacing. The number in (·) in the SPH section corresponds to the number of SPH coefficients updated in time. RBF and DF use the same time-stepping scheme. SE uses a third-order Adams-Bashforth. SPH uses a semi-implicit time-stepping scheme denoted by the *. For the SPH 1849 case, [251] gives t = 3 minutes when using a leapfrog scheme with a Robert’s filter (γ = 0.07), as is done in RBF and DF for the steady-state case. RBF use RK4 for the unsteady flow case. Method

No. of nodes (N )

Time step, t steady-state flow

Rel. 2 error in h

Time step, t unsteady flow

Rel. 2 error in h

RBF [80]

784 (28) 1849 (43) 3136 (56) 4096 (64) 5041 (71) 8192 (1849) 18432 (4096) 2048 8192 32768 6144 24576

20 min 12 min 10 min 8 min 6 min 20min∗ (3) 15 min∗ 6 min 3 min 90 s 90 s 45 s

6.32 × 10−6 1.97 × 10−8 3.65 × 10−10 4.72 × 10−11 6.88 × 10−12 7 × 10−10 2.5 × 10−10 2 × 10−6 4 × 10−10 2 × 10−13 8 × 10−7 1 × 10−10

40 min 24 min 15 min 8 min 6 min 20min∗ (3)

4.88 × 10−1 3.46 × 10−3 8.83 × 10−6 2.57 × 10−7 1.02 × 10−8 2 × 10−3

6 min 3 min 90 s 90 s 45 s

3.9 × 10−1 8.2 × 10−4 4.0 × 10−4 6.5 × 10−3 4 × 10−5

SPH [155] DF [251]

SE [261]

visual change in the solution and the error only slightly increasing. Spectral convergence is easily achieved, as shown in Table 6.1. 6.2.3.2 Unsteady nonlinear flow: Forced translating low-pressure system

This case tests the dispersive effects of a numerical discretization method. It models a low-pressure system, initially centered at (λ, θ) = (0, π/4), superimposed on a westerly jet stream. Forcing terms are added to the SWE, (6.7) and (6.8), to constrain the motion of the system so that the initial condition is nonlinearly advected intact. For a complete derivation of the analytic solution in Cartesian coordinates, see [80]. Classic Runge-Kutta fourth-order (RK4) is used to advance the RBF method in time. This is chosen over a leapfrog with a Robert filter scheme used in the steady-state test case since the unsteady flow case is much more sensitive to time truncation errors. Figure 6.3 shows the initial velocity and height field and the error in the latter for N = 3136 after 5 days of simulation. It is clear that the largest errors are located predominantly where the gradients in the solution are the highest, as would be expected. The time traces of the relative 1 , 2, , and ∞ error in h for Figure 6.3(c) are given in Figure 6.4(a), showing that error barely grows except in the 1 norm. The same behavior was observed with the SE method. The other spectral methods did not report times traces. A 15-minute time step was used with N = 3136 RBF so that spatial errors matched temporal ones, not to obtain time stability. Figure 6.4(b) shows that for N = 4096, an 8-minute time step achieves this purpose (as reported in Table 6.1), but larger time steps could have easily been taken at the sacrifice of accuracy. Additionally, spectral convergence is easily achieved, as shown in Table 6.1. 6.2.3.3 Comparative results for steady-state and unsteady nonlinear flow test cases

The results for the test cases of the previous two sections are compared against the following spectral methods:

Downloaded 11/04/15 to 142.103.160.110. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

6.2. Shallow-water equations on a sphere

139

10600

(a)

(c)

(b)

Figure 6.3. Initial (a) velocity field and (b) height field with N = 3136 for the unsteady flow test case plotted as orthographic projections centered at 45◦ N and 0◦ E. The contours in (a) range from 10600 m to 10100 m in intervals of 50 m. (c) The error (exact, numerical) in the solution after 5 days for N = 3136,  = 3.25, Δt = 15 min. Reprinted with permission from Royal Society Publishing. [80]

(a)

(b)

Figure 6.4. (a) Relative 1 , 2 , and ∞ errors in the height field for the unsteady-state test case as a function of time for N = 3136 ,  = 3.25, Δt = 15 min. (b) The relative 2 error as a function of time step, t , in minutes. Reprinted with permission from Royal Society Publishing. [80]

1. SPH requires twice as many grid points as basis functions when used on a lat-lon grid. Due to a severe CFL condition, a semi-implicit time stepping is commonly used [155]. Also, they are notorious for incorrectly increasing the energy in low modes through aliasing of the high frequencies. As a result, SPH are never run without dealiasing, normally using Orszag’s 2/3 rule [131]. Figure C.8 in Appendix C demonstrates how roughly the upper 1/3 of the spectrum is corrupted on a latlon grid. So, for example, a SPH method that uses 1849 basis functions is in fact using 4096 basis functions with a grid of 8192 nodes. However, only 1849 bases are updated in time in spectral space due to dealiasing. Since there are no effective “fast” Legendre transforms (for use in the latitudinal direction) and FFTs can be used in the longitudinal direction, the resulting operation count is O(N 3/2 ) per time step. 2. DF, unlike SPH, has a one-to-one correspondence between grid points and spectral coefficients. For comparison, a SPH method that uses 1849 bases with the necessary 8192 grid points is approximately equivalent to a DF using 8192 grid points (see Table 6.1). DF, for the same reasons as SPH, also requires the use of some type of

Chapter 6. Global RBF Applications to Geo-Modeling: Spherical Domains N = 484,

= 0.0, Δ = 1080 s

0.8

0.8

0.6

0.6

0.4

0.4

0.2

0.2 Im( )

Im( )

N = 484,

0

0

0.1

0.2

0

0.1

0.2

Re( )

(a)  = 0

N = 484,

= 0.2, Δ = 1080 s

0

Re( )

(b)  = 0.2

= 0.7, Δ = 1080 s

0.8 0.6 0.4 0.2 Im( )

Downloaded 11/04/15 to 142.103.160.110. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

140

0

0

0.1

0.2

Re( )

(c)  = 0.7

(d)

Figure 6.5. (a)–(c) Eigenvalue spectrums of the global RBF approximation operator of the RHS of the linearized system, (6.10), for different values of , linearized about the steady flow case with N = 484. The spectrums have been scaled by Δt = 1080 seconds with ξ = Δt × eigenvalues. The stability domain of the leapfrog time-stepping scheme with a Robert’s filter of γ = 0.15, is plotted as light solid line (γ = 0.07 was used for the steady-state case results). (d) The eigenvector associated with the eigenvalue that has the largest positive real part in (a). To compute (a), the RBF-QR algorithm (Section 3.2.4.6) was needed. Reprinted with permission from Royal Society Publishing. [80]

dealiasing filter. Due to FFTs, DF has an operation count of only O(N log N ) per time step. 3. The SE method considered here (formulated on a cubed sphere [261]) uses a tensor product of Legendre polynomials on each element. This model became the basis on which the current default climate model at NCAR, HOMME (High-Order Methods Modeling Environment), is built. Due to the clustering of Legendre grid points near the ends of each element boundary, the CFL restriction is severe and increases as the order of polynomials used increases. SE models generally use some type of filtering, or instability will set in. The operation count for SE is O(k p 2 ), where k is the number of elements and p the order of the polynomial expansion on each element. 6.2.3.4 Time stability and the effect of the shape parameter 

In order to understand the accuracy and time stability illustrated by the RBF method in Table 6.1, it is necessary to observe the eigenvalue spectrum of the full linearized RHS

Downloaded 11/04/15 to 142.103.160.110. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

6.3. Thermal convection in a 3-D spherical shell

141

operator that is time stepped, i.e., (6.10a)–(6.10d). Consider the steady-state test case (the following conclusions also hold for the unsteady flow case; see pp. 1969–1972 in [80]). Performing a linearized eigenvalue stability analysis, as described in Section 6.2.2, about the steady-state solution, eigenvalues are noticed to spread off the imaginary axis and into the right half-plane, as shown in Figure 6.5. In the limit as  → 0, time instability sets in. In this limit, the RBF interpolant approaches the SPH interpolant [107], and special filtering would be needed. As shown in Section C.4, this is especially true for ME nodes, given their near singular nature for representing spherical harmonics. However, Figure 6.5(d) shows that the eigenvector associated with the eigenvalue having the largest real part in Figure 6.5(a) is completely spurious and not physical, as oscillations are on a scale smaller than the grid spacing. Imposing a filter, such as the A−1 -based hyperviscosity (Section 5.3.2.1), can easily take care of such eigenmodes. Away from this limit, eigenvalues are tightly clustered around the imaginary axis and near the origin, as it should be for a hyperbolic system. This allows for the strong time stability exhibited by the global RBF method.

6.3 Thermal convection in a 3-D spherical shell This section demonstrates both the outstanding performance of the global RBF method and how it can easily be hybridized with other numerical methods, in this case a Chebyshev PS discretization. An isoviscous model of thermal convection in the Earth’s mantle is presented. The physical model is ∇·u = 0 (continuity),  3 4 ∇ · η ∇u + {∇u}T + Ra T ˆr = ∇ p (momentum), ∂T + u · ∇T = ∇2 T (energy), ∂t

(6.11) (6.12) (6.13)

where u = (u r , uθ , uλ ) is the velocity field in spherical coordinates (θ = latitude, λ = longitude), p is pressure, T is temperature, ˆr is the unit vector in the radial direction, η is the viscosity, and Ra is the Rayleigh number. The boundary conditions on the velocity of the fluid at the inner and outer surfaces of the spherical shell are

u r | r =R ,R = 0  i o  impermeable

and

> > ∂ 1 uλ 2>> ∂ 1 uθ 2>> = r = 0, ∂ r r > r =Ri ,Ro ∂ r r > r =Ri ,Ro    shear-stress free (slip) r

(6.14)

where Ri is the radius of the inner surface of the shell and Ro is the radius of the outer surface as measured from the center of the Earth. The boundary conditions on the temperature are T (Ri , θ, λ) = 1 and T (Ro , θ, λ) = 0. As shown in [1, 42], any divergence-free field can be expressed in terms of a poloidal Φ and toroidal Ψ potential, u = ∇ × ∇ × ((Φr )ˆr) + ∇ × (Ψˆr). If the fluid is isoviscous and η = constant (or the viscosity stress tensor is spherically symmetric) and satisfies (6.14), then the field is purely poloidal (i.e., Ψ ≡ 0). As a result, the 3-D continuity and momentum equations (6.11) and (6.12) can be alternatively written as ∇4 Φ = Ra r T .

(6.15)

Downloaded 11/04/15 to 142.103.160.110. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

142

Chapter 6. Global RBF Applications to Geo-Modeling: Spherical Domains

Equation (6.15) can be reduced to a system of two coupled Poisson equations, resulting in the nonlinear thermal convection model ! ∂Ω ∂ Δs Ω + r2 = Ra r T , (6.16) ∂r ∂r ! ∂ ∂Φ (6.17) r2 = r 2 Ω, Δs Φ + ∂r ∂r ! 1 ∂ ∂T ∂T 1∂T 1 ∂T ∂T 1 + ur + uθ + uλ = Δs T + r2 , (6.18) ∂t ∂r r ∂θ r cos θ ∂ λ r2 r2 ∂ r ∂r where θ ∈ [−π/2, π/2], λ ∈ [−π, π), and Δs is the spherical surface Laplacian operator defined in (6.2). The velocity boundary conditions (6.14) in terms of Φ are > ∂ 2 Φ >> Φ| r =Ri ,Ro = 0 and = 0. (6.19) > ∂ r2 > r =Ri ,Ro

The components of the velocity u = (u r , uθ , uλ ) are given by     1 1 ∂2 1 ∂2 u = ∇ × ∇ × (Φr )ˆr = Δ Φ, (Φr ), (Φr ) . r s r ∂ r∂ θ r cos θ ∂ r ∂ λ

6.3.1 Node layout, discretizations, and solvers An operator-splitting method is used in space such that all angular operators in (θ, λ) are discretized using N global RBF on a sphere and the radial operators are discretized using M Chebyshev polynomials, resulting in M N nodes in the 3-D spherical shell. In the modeling literature, this is often known as a “2+1” layering approach. The node layout is illustrated in Figure 6.6. As noted in the caption of Figure 6.6, the nodes in the radial direction are the extrema of the Chebyshev polynomials. This is reviewed in Section 2.2.2. RBF discretization of 1 ∂

1

∂

uθ r ∂ θ + uλ r cos θ ∂ λ is done by using (4.6) and (4.5), respectively, and Δs is defined in (6.4). Since we have four boundary conditions on Φ, given by (6.19), and none on Ω, we cannot directly solve (6.16) and (6.17). Therefore, the influence matrix method [204] is used to find the unknown boundary values on Ω such that all four boundary conditions on Φ are satisfied and the momentum equation is solved. This methodology is quite involved. However, it is explained step-by-step, with all details included, in Section 5 and Appendix A of [291]. A semi-implicit time-stepping method is used due to the Chebyshev discretization of the radial component of the diffusion operator, which would result in a CFL condition (Section 1.2.1.4) on the time step that is proportional to O(1/M 4 ), making an explicit scheme ineffective. As a result, a third-order Adams-Bashforth method is used to time step all terms discretized with RBF, while a Crank-Nicolson (i.e., Adams-Moulton second-order) method is used for time-stepping the radial component of the diffusion operator in the energy equation (see Section 6 in [291]). For a detailed discussion and the pseudoalgorithm on how to computationally implement this RBF-Chebyshev methodology to solve such a system of PDEs, from the steps involved in preprocessing to execution, see Appendix B of [291]. The algorithm in its greatest generality is given by the following: ∂

1

∂

1. Discretize ∂ θ , cos θ ∂ λ , and Δs using collocation with N global RBFs or RBF-FD with stencil size n = 50 (for the results shown in [82]).

Downloaded 11/04/15 to 142.103.160.110. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

6.3. Thermal convection in a 3-D spherical shell

143

Figure 6.6. Node layout for 3-D thermal/mantle convection. (a) N ME nodes on a sphere. (b) 3-D view of the discretization of the spherical shell used in the hybrid RBF-PS calculation. The lighted ball is the inner boundary, and black circles display the computational nodes, which are distributed in the radial direction along the extrema of the Chebyshev polynomials. Reprinted with permission from the American Geophysical Union. [291]

2. Discretize

∂ ∂r

,

∂ ∂r



 ∂ r2 ∂ r ,

Chebyshev polynomials.

>

∂2 > ∂ r 2 > r =R

, and i

>

∂2 > ∂ r 2 > r =R

using collocation with M + 2 o

3. With the given initial condition on temperature and using (i) the influence matrix method to address the boundary conditions on Ω (see Section 5 in [291]) and (ii) matrix diagonalization (i.e., eigenvector decomposition) on the RBF and Chebyshev DMs for the Laplacian, Ω and Φ are solved for in O(M 2 N )+O(M N 2 ) operations and require O(M 2 ) + O(N 2 ) memory storage. This is significant savings over a direct solve of the equations, which would require O(M 2 N 2 ) operations and O(M 2 N 2 ) storage. 4. Compute the velocity field u = ∇ × ∇ × ((Φr )ˆr, which is represented discretely by u = (L s ΦR−1 , Dθ ΦRD r R−1 , Dλ ΦRD r R−1 ),         ur uθ uλ

(6.20)

where the operators in (6.20) are given in Table 6.2 and R is a diagonal matrix containing the M interior Chebyshev nodes. 5. Time step the energy equation, treating all terms explicitly with a third-order AdamsBashforth method, except the radial component of the diffusion term, which is treated implicitly using a Crank-Nicolson scheme. Return to step 3 with an updated temperature profile.

6.3.2 Comparison with community benchmarks: Ra = 7000 and Ra = 100,000 One of the most common benchmarks for computational models of mantle convection in a spherical shell is the steady-state cubic test case (where downwelling occurs at the

Downloaded 11/04/15 to 142.103.160.110. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

144

Chapter 6. Global RBF Applications to Geo-Modeling: Spherical Domains Table 6.2. Notation and size for the various differentiation matrices used in the hybrid RBFChebyshev method. Note that each dependent variable is a matrix of size N × M . Matrix Dλ Dθ Ls Dr Lr

Operator 1 ∂ cos θ ∂ λ ∂ ∂θ Δs ∂ ∂r ! ∂ ∂ r2 ∂r ∂r

Discretization

Dimension

RBF

N ×N

RBF

N ×N

RBF

N ×N

Chebyshev

M ×M

Chebyshev

M ×M

corners of a cube and upwelling on the faces). The fluid is treated as isoviscous, and Ra is set to 7000. The initial condition for the temperature is specified as @  r −R  ? Ri (r − Ro ) 5 4 i 0 , + 0.01 Y4 (θ, λ) + Y4 (θ, λ) sin π T (r, θ, λ) = r (Ri − Ro ) 7 Ro − Ri

(6.21)

where Ynm denotes the normalized spherical harmonic of degree n and order m in Appendix B. The first term in (6.21) represents a purely conductive temperature profile, while the second term is a perturbation to this profile and determines the final steadystate solution. The θ − λ temperature dependence of (6.21) on a spherical shell surface can be seen in Figure 6.7(a). For this test case, N = 1600 nodes were used on each spherical surface, and 23 Chebyshev nodes were used in the radial direction (i.e., M = 21 interior nodes), giving a total of 36,800 nodes. A time step of 10−4 was used, or 10,000 time steps were taken to reach steady state at the nondimensionalized time of t = 1, corresponding to roughly 58 times the age of the Earth. Figure 6.7(b) displays the final RBF-PS steady-state solution for the cubic test case in terms of the residual temperature δT = T (r, θ, λ) − 〈T (r )〉, where 〈 〉 denotes averaging over a spherical surface. Since no analytical solutions exist, validation is done via comparison to other published results in the literature with respect to scalar global quantities, such as the Nusselt number at the inner and outer boundaries (Nui and Nuo ), and both the averaged root mean square velocity 〈Vrms 〉 and temperature over the volume 〈T 〉. Table 6.3 contains such a comparison for the RBF-PS method with respect to popular methods used in the mantle convection literature. This simulation was also run with RBF-FD in place of global RBFs [82], with results included in the table. The following observations can be made: 1. The only method that is spectral in at least one direction is the spherical harmonicfinite difference method of [141]. In [254], Harder’s method was used with Romberg extrapolation to obtain the results to at least four digits of accuracy. With regard to almost all quantities for both test cases, the results of the RBF-PS method match exactly with Harder’s extrapolated results. 2. The number of nodes (degrees of freedom) needed to accomplish the results is an order of magnitude lower than what was used with the community CitcomS model reported by [307], approximately one and a half orders of magnitude lower than either the finite volume method by [254] or the method by [141] and three orders of magnitude less than the Yin-Yang, multigrid method by [161].

Downloaded 11/04/15 to 142.103.160.110. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

6.3. Thermal convection in a 3-D spherical shell

145

Figure 6.7. (a) θ − λ dependence of the initial condition for the cubic mantle convection test case. (b) Steady-state isosurface of the residual temperature, δT = T (r, λ, θ) − 〈T (r )〉, at t = 1 for the isoviscous cubic mantle convection test cases at Ra = 7000 computed with the RBF-PS model. Light gray plumes correspond to δT = 0.15 and denote upwelling relative to the average temperature at each radial level, while dark gray corresponds to δT = −0.15 and denotes downwelling. The solid sphere shows the inner boundary of the 3-D shell corresponding to the core. Reprinted with permission from the American Geophysical Union. [291] Table 6.3. Comparison between computational methods for the cubic mantle convection test cases with Ra = 7000. Nuo and Nui denote the respective Nusselt number at the outer and inner spherical surfaces, 〈Vrms 〉 the volume-averaged rms velocity over the 3-D shell, and 〈T 〉 the mean temperature of the 3-D shell. Extrap. indicates that the results were obtained using Romberg extrapolation. Solid lines indicate that numbers were not reported. Abbreviations: FE = finite element, FD = finite difference, FV = finite volume, SP-FD = hybrid spectral and finite difference, and SP = purely spectral. For the RBF-PS method, the standard deviation of all the quantities from the last 1000 time steps was less than 5 · 10−5 , which is a standard measure for indicating that the model has reached numerical steady state. Model

Type

Nodes

r × (θ × λ)

Nuo

Nui

〈Vrms 〉

〈T 〉

3.6254 3.5554 3.6083 3.5806 3.5983 3.6090 3.6086 3.6096 3.6096 3.6095

3.6016 — — — 3.5984 — — — 3.6096 3.6096

31.09 30.5197 31.0741 30.87 31.0226 31.0709 31.0765 31.0821 31.0820 31.0819

0.2176 — 0.21639 — 0.21594 0.21583 0.21582 0.21578 0.21577 0.21577

Cubic test case, Ra = 7000 Zhong(CitcomS) [307] Yoshida [302] Kameyama [161] Ratcliff [218] Stemmer [254] Stemmer [254] Harder [141, 254] Harder [141, 254] RBF-PS [291] RBF-FD/PS (n = 50) [82]

FE FD FD FV FV FV SP-FD SP-FD SP SP-FD

393216 2122416 12582912 200000 663552 Extrap. 552960 Extrap. 36800 59823

32 × (12 × 32 × 32) 102 × (102 × 204) 128 × (2 × 128 × 384) 40 × (50 × 100) 48 × (6 × 48 × 48) Extrap. 120 × (48 × 96) Extrap. 23 × (1600) 23 × (2601)

3. The Nusselt number is nondimensional, measuring the ratio of convective to conductive heat transfer across a boundary. Thus, if there are no sources or sinks in the domain, energy should be conserved and Nui = Nuo . Notice this is the case with the RBF-PS method, even with such a low number of nodes. This results from the spectral accuracy of the method, which will inherently dissipate physical quantities less. For the isoviscous Ra = 105 case, there are only two published studies for comparison, the CitcomS study [307] and a FV one [218], the latter being underresolved and therefore not desirable for comparison. The second-order CitcomS FE model decomposes the sphere into 12 caps, with each cap having N × N elements and then stacked M times in

Chapter 6. Global RBF Applications to Geo-Modeling: Spherical Domains

800 CitcomS

700

RMS velocity

600 500 400 300 200 100 0 0

0.05

0.1

(a)

0.15 time

0.2

0.25

0.3

(b)

0.32

35 CitcomS

0.3

CitcomS

30

0.28

25

0.26 20 Nuo

Avg. temperature

Downloaded 11/04/15 to 142.103.160.110. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

146

0.24

15 0.22 10

0.2

5

0.18 0.16 0

0.05

0.1

0.15 time

(c)

0.2

0.25

0.3

0 0

0.05

0.1

0.15 time

0.2

0.25

0.3

(d)

Figure 6.8. (a) Same as 6.7(b) but for Ra = 105 and at t = 0.3. Time plots of the (b) 〈vrms 〉, (c) 〈T 〉, and (d) Nuo , comparing the results (obtained through Computational Infrastructure for Geodynamics (http://www.geodynamics.org/cig/workinggroups/mc/workarea/ benchmark/3dconvention/) ) of the CitcomS and RBF-PS models for isoviscous mantle convection at Ra = 105 . Reprinted with permission from the American Geophysical Union. [291]

the radial direction, forming 3-D brick-like elements. The cubic test case initial condition, given by (6.21), is used, and the model is integrated to t = 0.3 . Since Ra = 105 is a more convective regime, resulting in thinner plumes, as seen in Figure 6.8(a), larger resolution is needed. Thus, 43 Chebyshev nodes are used in the radial direction and 4096 ME nodes on each spherical surface. Since the time step is purely restricted by the Chebyshev discretization, the increase in Chebyshev nodes results in a more severe CFL criterion that causes a necessary decrease in the time step. For this case, 50,000 time steps are needed to reach t = 0.3 as opposed to the 35,000 time steps needed in [307]. Comparative results are given in Table 6.4. The following four points should be noted: 1. The difference between Nuo and Nui is 0.14%, showing that the RBF-PS method is close to conservation of energy. 2. The RBF-PS requires approximately an order of magnitude less degrees of freedom than the study [307]. 3. Once steady state has been reached, differences between the two models are within 0.4% for Nuo and Nui , 0.2% for 〈V r m s 〉, and 0.9% for 〈T 〉 .

Downloaded 11/04/15 to 142.103.160.110. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

6.3. Thermal convection in a 3-D spherical shell

147

Table 6.4. Comparison between computational methods for the isoviscous cubic mantle convection test case with Ra = 105 . Model CitcomS [307] RBF-PS [291] RBF-FD/PS [82]

Nodes 1,327,104 176,128 282,123

r × (θ × λ) 48 × (12 × 48 × 48) 43 × (4096) 43 × (6561)

Nuo 7.8495 7.8120 7.8072

Nui 7.7701 7.8005 7.8030

〈Vrms 〉 154.8 154.49 154.43

〈T 〉 0.1728 0.1712 0.1707

4. As seen in Figure 6.8(b)–(d), the curves for Nuo , 〈V r m s 〉, and 〈T 〉 as a function of time are almost indistinguishable from those of CitcomS, validating the RBF-PS method.

6.3.3 Timing results for Ra = 7000 and Ra = 100,000 benchmarks All test cases were conducted on a work station with one Intel i7 940 2.93-GHz quadcore processor. The code was written in MATLAB 2009b with BLAS multithreading enabled. Less than 2 GB of memory was used. The results under “Total runtime” in Table 6.5 include all preprocessing steps, such as setting up the differentiation matrices and diagonalizing them. Table 6.5. Runtime results for the RBF-PS method for the Ra = 7000 and Ra = 100,000 cases on a single 2.93-GHz Intel i7 940 quad-core processor. Test case

Total number of nodes

Runtime per time step

Total runtime

Total time steps

Ra = 7000 Ra = 100,000

36,800 176,128

0.0516 sec. 0.44 sec.

8 min. 16 sec. 6 hours 27 min.

10,000 (to t=1) 58x age of Earth 50,000 (to t=0.3) 19x age of Earth

6.3.4 Ra = 1,000,000 RBF-PS simulation In the literature, the RBF-PS model is the only purely spectral one in 3-D spherical geometry that has been run at Ra = 106 . Common practice at such high Ra is to start the simulation with an initial condition taken from a simulation run at a lower Ra. This is to avoid the extremely high velocity values that occur during the initial redistribution of the temperature from a conductive profile to a convective one, which severely restricts the time step. Here, the initial condition was taken from a simulation that was started at Ra = 105 with an initial condition consisting of a purely conductive temperature profile plus a small perturbation in the lateral direction of randomly combined spherical harmonics up to degree 10 multiplied by the same sine term in the radial direction as in (6.21). Due to the very fine features that develop in the flow as well as the strong boundary layers, high resolution is needed. As a result, 81 Chebyshev nodes in the radial direction and 6561 ME nodes on each spherical surface were used, for a total of 531,441 nodes. Since this is a purely convective regime, the choice of ending time is arbitrary, and the model was run until the average temperature had decreased to an acceptable level, t = 0.08 (approximately four and half times the age of the Earth). Figure 6.9 displays the isosurfaces of the residual temperature at t = 0.08, clearly showing the mantle in a purely convective turbulent regime.

Downloaded 11/04/15 to 142.103.160.110. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

148

Chapter 6. Global RBF Applications to Geo-Modeling: Spherical Domains

Figure 6.9. Residual temperature, δT , for the Ra = 106 simulation at t = 0.08, where light corresponds to δT = 0.1 (upwelling), dark corresponds to δT = −0.1 (downwelling), and the solid sphere shows the inner boundary of the 3-D shell corresponding to the core. Reprinted with permission from the American Geophysical Union. [291]

Downloaded 11/04/15 to 142.103.160.110. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

Chapter 7

RBF-FD Applications to Geo-Modeling: Spherical Domains This chapter will consider two examples of applying RBF-FD in spherical geometries: (i) the shallow-water equations on the surface of a sphere (a system of hyperbolic PDEs) and (ii) a model for the global electric circuit (an elliptic PDE in a 3-D spherical shell). Furthermore, Section 7.2 gives some heuristic guidelines for choosing the necessary parameters (e.g., stencil size, amount and degree of hyperviscosity, shape parameter).

7.1 Shallow-water equations on a sphere The system of PDEs considered in this section are the same as given by (6.7)–(6.8). The topics of the following sections are discussed in much greater detail in [78].

7.1.1 Stencils on a sphere, RBF-FD weights, and DMs The left panel of Figure 7.1 gives an example of four different stencils used to calculate the RBF-FD weights for approximating the convective operator L at a location denoted by the square box, xc . The differentiation weights, $ wi , are calculated by enforcing that a linear combination of the function values, i.e., ni=1 wi ui = (Lu(x))|x=xc , should be exact for RBFs, φ(x − xi )ni=1 , centered at each of the nearest n node locations and evaluated at the node xc . The nearest neighbors can be found in different ways, as noted in Section 5.1.2. It has also been shown through experience and studies [78, 92, 103, 292] that better accuracy is gained by the interpolant being able to reproduce a constant, guaranteeing $ that constant data gives a zero result for derivative operators. Hence, the constraint ni=1 ai = L1|x=xc = 0 is added. This is thoroughly covered in Section 5.1.4 and leads to the (n + 1) × (n + 1) linear system given in (5.4), where wn+1 is ignored after the system is solved. Thus, if there are N nodes on the sphere, then there will be N separate (n + 1) × (n + 1) systems to solve, resulting in a preprocessing cost of O(n 3 N ). However, the cost for each time step of the RBF-FD method becomes only O(nN ). Its sparsity structure is seen in the right panel of Figure 7.1, with the nodes ordered according to their spatial closeness (see Section 5.1.3). This results in a significant speedup from global RBFs that require O(N 3 ) operations to create the DMs and O(N 2 ) to time step. 149

Downloaded 11/04/15 to 142.103.160.110. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

150

Chapter 7. RBF-FD Applications to Geo-Modeling: Spherical Domains 4

0

x 10

0.5

1

1.5

2

2.5 0

0.5

1 1.5 nz = 793600

(a)

2

2.5 4

x 10

(b)

Figure 7.1. (a) An example of four n = 75 node RBF-FD stencils for approximating the convective operator, at the square on a sphere of N = 900 nodes. Black corresponds to negative differentiation weights and medium gray to positive. The marker sizes reflect the magnitude of the weights. Light gray nodes are not included in the stencils. (b) Sparsity of the RBF-FD DM for the advective operator (in Cartesian coordinates) on a sphere using a 31-node stencil, the most commonly used in the following benchmarks, and N = 25,600 MD nodes. A k d -tree algorithm was used to find the nearest neighbors in a stencil [16]. Reprinted with permission from Elsevier. [78]

7.1.2 RBF-FD discretization of the shallow-water equations on the sphere in Cartesian form The RBF-FD formulation is similar to the global case given in Section 6.2.1. However, the projected grandient operator P∇ that confines the flow to the sphere is now calculated for each RBF-FD stencil of size n, and, as noted just above (in Section 7.1.1), a constant is added to the A matrix, as seen in (5.4). As an example, consider the x component p x · ∇ acting on an RBF centered at xc ; the differentiation weights are found by solving the following linear system, remembering that r (x) = ||x − xi ||2 for i = 1, . . . , n: ⎡ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎣ − 1

| A − ···

− 1

| | + |

1 .. . 1 − 0

⎤⎡ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎦⎣

w1 .. . wn − wn+1

⎤

⎡

[ xc xTc x1 −x1 ]

r (xc ) ⎢ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ = ⎢ [ x xT x −x ] ⎥ ⎢ c c n n ⎥ ⎢ r (xc ) ⎦ ⎢ ⎢ ⎣

∂ ∂r

> > φ1 (r (x))>>

.. . ∂ ∂r

⎤ x=x c

> > φn (r (x))>

− 0

x=x c

⎥ ⎥ ⎥ ⎥ ⎥ ⎥. ⎥ ⎥ ⎥ ⎥ ⎦

(7.1)

As noted in Section 7.1.1, (7.1) is solved N times for each node on the sphere, via a “for” y loop, with each solve forming one row of the DM, denoted by DNx . Similarly, DN and z DN are obtained, the discrete RBF-FD approximations to the y and z components, respectively, of the projected gradient operator. Note that these matrices are never actually stored as full matrices but only as sparse matrices, containing the nonzero entries together with pointers to their row and column indices. An example of a MATLAB code to calculate DNx using GA RBFs is given below.

Downloaded 11/04/15 to 142.103.160.110. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

7.1. Shallow-water equations on a sphere

151

function P_Dx = RBFFD_DM(x,y,z,n,N,ep) % Computes RBF-FD weights for the x-component of the projected gradient % on the sphere, p_x dot grad % x,y,z are the N Cartesian nodes on the sphere, ep is the shape % parameter, n is the stencil size idx A RHS rbf drbf

= = = = =

knnsearch([x y z],[x y z],’k’,n); ones(n+1,n+1); A(end,end) = 0; zeros(n+1,1); @(ep,r2) exp(-ep^2*r2); @(ep,r2) -2*ep^2*exp(-ep^2*r2);

for k=1:N r2 = max(0,2*(1 - x(idx(k,:))*x(idx(k,:)).’ - ... y(idx(k,:))*y(idx(k,:)).’ - z(idx(k,:))*z(idx(k,:)).’)); A(1:n,1:n) = rbf(ep,r2); xc_x

= x(k)*x(idx(k,:),1) + y(k)*y(idx(k,:)) + ... z(k)*z(idx(k,:)); RHS(1:n,1) = (x(k,1)*xc_x - x(idx(k,:))).*drbf(ep,r2(:,1)); w_temp = A\RHS; w ( (k-1)*n+1 :

k*n) = w_temp(1:n);

ind_i((k-1)*n+1 : ind_j((k-1)*n+1 :

% omit weight n+1

k*n) = k; % nonzero row entries k*n) = idx(k,:); % nonzero column entries

end P_Dx = sparse(ind_i,ind_j,w,N,N);

7.1.3 Hyperviscosity Stability issues for RBF-FD emanate from the fact that the natural intrinsic irregularity of the RBF-FD stencils causes eigenvalues of the DM to scatter into the right half of the complex plane, as shown in Figure 7.2(a). This becomes an issue with the RBF-FD method when (i) solving naturally dissipation-free PDEs, such that even a very mild numerical scatter of the eigenvalues into the right half complex plane can cause severe instability, and (ii) using large RBF-FD stencils since as the stencil size increases, so does the scatter of eigenvalues. This latter point is even an issue for systems with dissipation, in which case the scatter is likely to be too large for the natural dissipation to control. To address the issue above, hyperviscosity (Section 5.3.2.2) is added as a filter to the right-hand side (RHS) of the evaluation. For example, ∂F = RHS + H F, ∂t

(7.2)

where F is the vector of unknowns, RHS is the right-hand side of the system of PDEs as given in (6.7)–(6.8) but now discretized using RBF-FD DMs (see Section 7.1.2), and H is the hyperviscosity operator. Applying hyperviscosity shifts all the eigenvalues of RHS to the left half of the complex plane. This shift is controlled by k, the order of the Laplacian, and a scaling parameter γc , defined by H = γ Δk = γc N −k Δk . Having chosen N , n, and  (assuming C ∞ RBFs are used), it was found that the relation γ = γc N −k provides stability and good accuracy, also ensuring that the viscosity vanishes

Downloaded 11/04/15 to 142.103.160.110. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

152

Chapter 7. RBF-FD Applications to Geo-Modeling: Spherical Domains

Figure 7.2. Plots of the eigenvalue spectrum with the RK4 stability domain (solid line) for the shallow-water equations linearized about an initial state (a) without hyperviscosity and (b) with a Δ4 -type hyperviscosity added. Reprinted with permission from Elsevier. [78]

as N → ∞. In general, the larger the stencil size n, the higher the order of the Laplacian k. This is attributed to the fact that, for convective operators, larger stencils treat a wider range of modes accurately (see Figure 5.17). As a result, the hyperviscosity operator should preserve as much of that range as possible. The parameter γc is generally O(1) to O(10−2 ), and its sign depends on k to ensure that diffusion and not antidiffusion is being added (for k even, γc will be negative, and for k odd, it will be positive). If γc is too large, the eigenvalues move outside the stability domain of the time-stepping scheme, and/or eigenvalues corresponding to lower physical modes are not left intact, reducing the accuracy of the approximation. If γc is too small, eigenvalues remain in the right half-plane [78, 103]. However, notice that the quantity γc N −k will vary from about O(10−20 ) to O(10−45 ) for values of N from O(103 ) to O(105 ) that are considered in the following sections. In other words, only a very minute amount of hyperviscosity is added to stabilize the RBF-FD method.

7.1.4 On the choice of the shape parameter  The choice of  when using RBF-FD in large simulations becomes slightly more complicated since  is now a function of both stencil size n and the total number of nodes N . For RBF-FD, the node spacing may vary by orders of magnitude as the total number of nodes is increased. In order to scale to large node sets,  is chosen such that the mean 1 $ ¯A = N N condition number of the RBF-FD interpolation matrices κ i =1 (κA)i is kept constant as N increases ((κA)i is the condition number of the interpolation matrix in (5.4), representing the ith stencil). While this scaling does introduce the danger of stagnation errors, convergence only for very large N , on the order of 105 (i.e., approximately 50 km resolution), showed to be at risk. Currently, there is no effective stable algorithm as  → 0 for RBF-FD on a sphere, although progress is being made in that direction. A mean con¯ A, on the order of 108 to 1013 (see bottom row of Figure 5.7 comparing to dition number, κ accuracies in the middle row of the same figure) was found to give RBF-FD a competitive edge with regard to the accuracies reported by other high-order methods in the literature.

7.1.5 Numerical studies for RBF-FD Two test cases are considered: (i) flow over an isolated mountain that is C 0 to study the effects of Gibbs phenomena [155] and (ii) evolution of a highly nonlinear wave with rapid

Downloaded 11/04/15 to 142.103.160.110. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

7.1. Shallow-water equations on a sphere

153

energy transfer from large to small scales, resulting in complex vortical dynamics [121]. In both cases, there are no analytical solutions. For the first test, three high-resolution reference models are considered as “truth” based on high-order DG, spherical harmonics (DWD model), and RBF-FD. They are described below. The second test is compared against the DG method as well as a SE and FV method described in [253]. 1. DG: The DG results are computed using MUSE, a high-order discontinuous Galerkin (DG) model then used at NCAR. The simulations used as references herein have been performed on a cubed sphere grid (see Figure 7.5) made up of 6144 elements (each side of the cube has 1024 elements). Each element contains 12 × 12 nodes to represent the solution, which results in a total of 884,736 degrees of freedom and an average resolution around 26 km over the globe. For computing these reference solutions, no dissipation mechanism was found to be needed. However, for the run time versus error computations in Figure 7.6, the 2-D exponential filter described in [145] was applied. For more details on this DG model, see [19]. 2. DWD-SPH: The DWD (Deutscher Wetterdienst, German National Weather Service) spectral transform shallow-water model is an updated derivative of the NCAR spectral transform model. It is based on spherical harmonics implemented with de-aliasing, using Orszag’s 2/3 rule [131], and has become the standard reference solution in the community. For the flow over a mountain test, it has a spectral truncation of T426; i.e., it uses 182,329 spherical harmonic bases. 3. RBF-FD: This is a high-resolution RBF-FD model based on N = 163,824 icosahedral nodes on the sphere, representing a 60-km resolution. It uses a stencil size of n = 31 (see Section 7.2 for stencil size choice). 7.1.5.1 Flow over an C 0 isolated mountain

To simulate an undifferentiable C 0 mountain, the forcing term, h m t n = h ma x (1 − d /R),

(7.3)

where h ma x = 2000 m, R = π/9, and d 2 = min[R2 , (λ − λc )2 + (θ − θc )2 ], with (θc = 30o N, λc = −90o W) being the center of the mountain, is added to the RHS of the geopotential height h equation in (6.8). To differentiate between errors due to a nonsmooth forcing, which causes Gibbs phenomena in any high-order method, and those inherent in the RBF-FD method, the results for convergence and accuracy are compared against test runs that use an exceptionally steep C ∞ Gaussian profile given by d 2

h(λ, θ) = h ma x e −(2.8 R ) ,

(7.4)

where h ma x , d , and R are the same as for the cone mountain. The initial conditions are given by   u02 2 1 aΩu0 + z , u = u0 {−y, x, 0}, (7.5) h = h0 − g 2 where h0 = 5400 m (mean reference height), g = 9.80616 m, u0 = 20 m/s, a = 6,371,220 m (mean radius of the Earth), and Ω = 7.292(10)−5 s−1 (rotation rate of the Earth). The simulation is run for 15 days using a classic RK4 time-stepping scheme. The left column of Figure 7.3 shows the profile of the conical mountain, the solution in h at day 15, and the magnitude of the error between the RBF-FD solution for

Downloaded 11/04/15 to 142.103.160.110. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

154

Chapter 7. RBF-FD Applications to Geo-Modeling: Spherical Domains

Figure 7.3. Left column: Cone mountain results: (1) profile of mountain; (2) RBF-FD solution for h at day 15, N = 25,600, and n = 31 with contour intervals at 50 m; (3) magnitude in the error between the RBF-FD solution and DG reference solution; (4) 2 error as function of the resolution N for varying stencil sizes. Right column: Same as left but for the Gaussian mountain forcing. Dashed circles show the base of the mountain. Reprinted with permission from Cambridge University Press. [96]

N = 25,600 and n = 31 and the DG reference solution. The same holds for the right column but with the Gaussian mountain profile. With the solutions looking identically the same, the key difference to notice is that even though the C ∞ Gaussian mountain is slightly steeper than the C 0 mountain, no Gibbs phenomena is observed in the former. With the C 0 mountain, there are high-frequency waves emanating throughout the domain (i.e., Gibbs phenomena), illustrating the sensitivity of high-order methods to nonsmooth forcing. Another consequence is that the accuracy of the RBF-FD method does not indefinitely increase with stencil size n, as shown in the bottom left panel of Figure 7.3. After n = 31, stencil size has no bearing on accuracy when a nonsmooth mountain forcing is present. Hence, this is the stencil size chosen for this test case. In contrast, the bottom right panel of Figure 7.3 demonstrates how the accuracy of the RBF-FD method for a C ∞ solution does increase as n increases, that is, as the derivative approximations become more global. However, even with a smooth forcing, the rate of convergence is not much greater than for the cone case since both the Gaussian and the cone mountains are so steep, leading to underresolution even for very large nodes sets. To overcome this, adaptive node refinement in the area of the mountain needs to be used, as was done in [253].

Downloaded 11/04/15 to 142.103.160.110. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

7.1. Shallow-water equations on a sphere

155

Figure 7.4. The normalized 2 error in the height field h as a function of N for flow over a conical mountain at day 15 for RBF-FD (left) and global RBFs (right). The different markers correspond to different reference solutions. Reprinted with permission from Elsevier. [78]

Next, the RBF-FD method is compared against the DG and DWD-SPH models. The left panel of Figure 7.4 shows that the normalized 2 error is an order of magnitude larger when the DWD-SPH reference solution is used as opposed to DG or the RBF-FD (N = 163,842) reference solutions. Furthermore, when these latter reference solutions are used, the normalized 2 errors are almost identical (notice the  overlay the ). This same trend is also seen in the right panel of Figure 7.4 with global RBFs, a different approach than both RBF-FD and DG that does not require hyperviscosity. That DG, RBF-FD, and global RBFs are vastly different numerical methods strongly indicates that the DWD-SPH T426 spectral simulation is providing a less accurate solution. This is further supported by the few articles that do report 2 errors for this test case [251, 253, 261], all of which use either the NCAR or the DWD SPH reference solution and obtain errors on the order of 10−4 , an order of magnitude larger than that obtained by DG or RBF-FD. Finally, for this test case, time benchmarking is considered, where comparison is done against the DG model. Benchmarking was done on a MacBook Pro laptop with an Intel i7 2.2-GHz quad-core processor, using only a single core and 8 GB of memory. This configuration was chosen to compare the vastly different codes on the most basic level. The RBF-FD code was written in MATLAB and the DG code in C++. The RBF-FD reference solution of N = 163,842 and n = 31 (i.e., 60-km resolution) was used for calculating the 2 error versus runtime (i.e., wall clock time) for both methods in Figure 7.6. The RBF-FD resolutions with corresponding time steps is given in Table 7.1, while the DG resolutions are given in Table 7.2 and Figure 7.5. The RBF-FD method was computationally faster than the DG method, from about an order and a half of magnitude for coarser resolutions to four times faster for the finest resolutions. 7.1.5.2 Evolution of a highly nonlinear wave

This test case is more challenging, describing the evolution of a highly nonlinear wave with rapid energy transfer from large to small scales over a short time period. The adjustment of the background flow to the initial perturbation results first in high-frequency gravity waves propagating around the sphere followed by complex vortical dynamics, displaying very sharp gradients. The details of how to set up the test can be found in [78, 121]. Essentially, the background flow is only a function of latitude, represented by

Downloaded 11/04/15 to 142.103.160.110. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

156

Chapter 7. RBF-FD Applications to Geo-Modeling: Spherical Domains Table 7.1. Time steps used for the cone mountain case with respective spatial resolutions for the MD nodes based on the mesh norm, maxx∈2 min1≤i ≤N dist(x, xi ) (see http://web.maths.unsw.edu.au/∼rsw/Sphere/Extremal/New/ for discussion and tabulation) .

N

Resolution (km)

4096 6400 12100 25600 40962 163842

550 420 330 220 120 60

Node type

MD

Icosahedral

Δt (minutes) 20 15 12 5 3 1

Table 7.2. Samples of mesh specifications for the DG model. “d.o.f.” stands for degrees of freedom. See also Figure 7.5.

Resolution (km) Elements per face Polynomial order Number of d.o.f.

R1N4

R1N6

R1N8

R2N6

R2N8

R3N6

R3N8

1251 4 3 384

834 4 5 864

625 4 7 1536

417 16 5 3456

313 16 7 6144

208 64 5 13824

156 64 7 24576

Figure 7.5. Mesh resolution for the DG model. The meshes used for the different simulations are obtained by splitting recursively the elements of the initial cube mesh R0. Reprinted with permission from Elsevier. [78]

an exponential profile that is zero everywhere except in the latitudinal band π/7 ≤ θ ≤ 5π/14. To generate the instability, the height field is perturbed by Gaussians, in longitude and latitude, multiplied by a cosine to force the perturbation to go to zero at the poles θ = ±π/2. The test case is run for six days. There are two main concerns with this test case: (i) how well the sharp gradients are resolved and (ii) the effect of Gibbs phenomena. For short time integration periods, as here, these two numerical issues become a balancing act. As both the resolution N and the order of the method n are increased, the gradients are resolved better. However, as with classical FD, the higher the order of the method, i.e., n, the more prominent becomes the Gibbs phenomena. Here, n = 101 would correspond approximately to a ninth-order method. Notice in Figure 7.7 that when an n = 101 stencil size is used (enlargements to the right), the contour lines are more jagged with a slight “ringing,” characteristic of Gibbs phenomena. Yet the gradients and vortices are much better defined than for n = 31,

7.2. Practical guidelines to solving hyperbolic PDEs on a sphere with C ∞ RBF-FD

157

DG −1

10

2 Error

Downloaded 11/04/15 to 142.103.160.110. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

0

10

R1N4

−2

R1N6

10

R1N8

RBF−FD

−3

10

420 km

R2N6 417 km R2N8

−4

10

R3N6 R3N8 156 km

120 km −5

10

0

10

1

10

2

3

10 10 Runtime in seconds

4

10

Figure 7.6. The error as a function of runtime (defined by wall clock time) for the flow over the cone mountain test case. Reprinted with permission from Elsevier. [78]

roughly analogous to a fifth- to sixth-order method. It this latter case, the features appear more smoothed out and less developed. Increasing the resolution N and keeping n = 31 fixed gives the best results, as seen in Figure 7.8, for N = 163,842 (60 km, or 0.54◦ ×0.54◦ ). The solution is extremely similar to that given by the high-order DG solution with a resolution of 39 km, or 0.35◦ × 0.35◦ . Another interesting feature to note is that the RBF-FD method is able to produce the basic wave pattern structure even at very coarse resolutions, such as N = 4096 (5◦ ×5◦ ), displayed in Figure 7.9. This is not the case with the DG method (a slightly higher resolution of 4◦ ×4◦ is displayed in the third panel of Figure 7.9), a spectral element method, or a finite volume method. At such coarse resolutions, the DG and spectral element [253] methods instead produce features of the grid, such as an artificial wavenumber four pattern for the cubed sphere. The finite volume method is noted to be so dissipative that no spatial structures are even seen at a resolution of 5◦ × 5◦ in Figure 7.9.

7.2 Practical guidelines to solving hyperbolic PDEs on a sphere with C ∞ RBF-FD The steps below provide a guideline on how to choose the parameters n, , k, and γ involved in modeling hyperbolic PDEs with RBF-FD when using infintely smooth RBFs: 1. n: If the node layout is hexagonal-like, as in MD, ME, or icosahedral nodes and N  n, such that the stencil can reasonably be approximated by a 2-D planar one, then good choices for n would be the same as those given by the symmetric stencils shown in Figure 8.2(b). As n increases, so does the accuracy of the RBF-FD method. The larger n is, the more physical modes are represented correctly. However, there is a trade-off between time stability and cost and accuracy with increasing n. Thus, it is generally best to settle for a fourth- to sixth-order method, e.g., n ≈ 30–45 in 2-D and around n = 100–150 in 3-D.

Downloaded 11/04/15 to 142.103.160.110. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

158

Chapter 7. RBF-FD Applications to Geo-Modeling: Spherical Domains

Figure 7.7. The relative vorticity at day 6 for the evolution of the highy unstable wave case as function of resolution N and stencil size n. Contour interval is 2 · 10−5 s −1 . The panels to the right show enlargements of the region between −175 and 0 in longitude—the region where the most exiciting dynamics is occurring. Notice that the power of the hyperviscosity depends on n, as discussed in Section 7.1.3.

Figure 7.8. Top: RBF-FD solution for N = 163,842 and n = 31. Bottom: DG solution at 39-km resolution. Reprinted with permission from Elsevier. [78]

Downloaded 11/04/15 to 142.103.160.110. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

7.2. Practical guidelines to solving hyperbolic PDEs on a sphere with C ∞ RBF-FD

159

Figure 7.9. The relative vorticity at day 6 for the evolution of the highy unstable wave test case with resolution of 5◦ × 5◦ for a finite volume [253], spectral element (SEM) [253], discontinous Galerkin, and the RBF-FD models. The DG model is actually closer to 4◦ ×4◦ resolution, corresponding to column R2N6 in Table 7.2. In the SEM model, there are nine elements per face of the cubed sphere. In each element, 8 × 8 Gauss-Legendre-Lobatto nodes are used. The RBF-FD model uses a total of 4028 nodes, a stencil size of 31, and a Δ4 hyperviscosity. Reprinted with permission from Springer Science + Business Media. [82]

2. : The shape parameter  is a function of both N and n. It is typically chosen so that the condition number κA of the RBF-FD interpolation matrix A remains relatively constant as N increases for a set stencil size n (κA from any single stencil will suffice for roughly evenly spaced node sets. However, a slightly better estimate, taking 1 $ ¯A = N N little time, is using the mean condition number κ i =1 (κA)i , where (κA)i represents the condition number of the ith stencil). Optimal values for accuracy

Chapter 7. RBF-FD Applications to Geo-Modeling: Spherical Domains

log10 κ ¯A

6 5

6

8

ε

Downloaded 11/04/15 to 142.103.160.110. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

160

4 10

3

12

14

2 40



50

60

√70 N

80

90

100

¯ A, as a function of Figure 7.10. The mean condition number of the interpolation matrix, κ N and  for a stencil size of n = 31. Reprinted with permission from Elsevier. [78]

tend to correspond to κA = 108 to 1013 , as seen in the bottom and middle rows of Figure 5.7. For further discussion, see also [76]. Constructing a plot of log10 κA as a function of  and N for a given n is very useful. This requires little calculation N → c N and  → c  leaves the A matrices and is simply done by noting that invariant, and therefore κA = f (/ N ); i.e., κA is constant along straight lines through the origin in the (, N ) plane. Then the formula for κA = f (/ N , n) for the 2-D nonperiodic case on p. 389 in [112] can be used, κA = c(/ N )− 8n−7−1 , as a stencil on a sphere can locally be viewed as a 2-D plane. Note that [112] also gives such formulas for other types of domains. As an example, for a stencil size of n = 31, − 8n − 7 − 1 ∼ = −14.5. Therefore, increasing the slope by a factor of α, decreases log10 κA by 14.5 log10 α, and thus log10 κA needs to be computed only for a single case of n, N ,  to construct the entire plot as given in Figure 7.10. 3. k: Next, the order of the Laplacian, k, for the hyperviscosity, Δk , is chosen— dependent on n. As noted in Figure 5.17, as k increases, more physical modes are preserved correctly when the hyperviscosity is applied. Thus, the larger n is the larger k sould be. 4. γ : The damping coefficient fine-tunes the hyperviscosity. Given that n, , and k are selected by the steps above, the eigenvalues of the linearized discrete RHS operator of the PDE system are plotted for small N , e.g., N = 3000. The amount the eigenvalues are pushed into the left half complex (as in Figure 7.2) then becomes proportional to γ = γc N −k , where γc is a small fine-tuning parameter whose magnitude increases with n, generally being between O(10−2 ) andO(1). Care should be taken that the sign of γ ensures diffusion and not antidiffusion (for k even, γ will negative and for k odd, positive). The final result should be that the eigenvalues lie within the time-stepping stability domain being used but still as close as possible to the imaginary axis. If the RBF-FD method is unstable for much larger node sets N , increase the γc until stability is ensured. The artificial dissipation typically requires a small safety factor to account for variability in the node sets.

Downloaded 11/04/15 to 142.103.160.110. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

7.3. A 3-D ellipitic PDE with irregular boundaries: Global electric circuit

161

7.3 A 3-D ellipitic PDE with irregular boundaries: Global electric circuit The work overviewed in this section is detailed in [8]. The global electric circuit (GEC) is a system of currents within Earth’s atmosphere. The system is defined by the volume between two highly conductive spherical shells, one the topographical surface of the Earth and the other a sphere that defines the lower boundary of the ionosphere at 90 km. The basis of the model is that highly electrified clouds, such as thunderclouds, produce a source current (the RHS of the PDE) that maintains the ionosphere at a fixed electric potential of 240 kV± 40 kV relative to the Earth’s surface, with the strongest returning currents present at the higher elevations. Thus, topography plays a major role in the GEC. This quasi-static system is modeled by the 3-D elliptic PDE with Dirchlet boundary conditions −∇ · (σ(r, θ, λ)∇Φ) = S(r, θ, λ) or S ΔΦ + (∇ log σ) · ∇Φ = − , σ

(7.6)

where σ is the conductivity, Φ is the electric potential, and S is the source current distribution. Since Φ is a scalar and recalling the discussion in Section 6.1, the PDE can be defined in spherical coordinates without any poles singularities as −90◦ ≤ θ ≤ 90◦ (latitude), −180◦ ≤ λ < 180◦ (longitude), k(θ, λ) ≤ r ≤ 60 km, where k(θ, λ) is the Earth’s surface (i.e., topography) and the top boundary is the sphere at 60 km above sea level. In true modeling (cf. [8]), σ is given by data output from a climate community model and S by satellite obervations. However, in order to demonstrate how to implement RBF-FD in spherical geometries with irregular boundaries, a much simpler test case is considered. To a reasonable first approximation, without any cloud layers, the σ is an exponential function of altitude. Assuming σ(r ) = σ0 e r /c , where c = 6 km, then (7.6) reduces to the 3-D problem ⎧ 1∂Φ ⎪ ⎪ = f (r, θ, λ), ⎨ ΔΦ + c ∂ r (7.7) Φ(r = k(θ, λ)) = g (θ, λ), ⎪ ⎪ ⎩ Φ(r = 60 ) = 0. For this test case, the RHS f (r, θ, λ) of the PDE and the boundary conditions at r = k(θ, λ) and r = 60 km are chosen such that the exact solution is given by     r − k(θ, λ) 0 4 , (7.8) Φ(r, θ, λ) = 0.8 Y7 (θ, λ) + 0.5 Y5 (θ, λ) sin π 60 − k(θ, λ) whereY lm (θ, λ) denotes a spherical harmonic of degree l and order m.

7.3.1 Numerical implementation with topography In contrast to assuming that the surface of the Earth is a simple sphere, the needed inclusion of topography has drastic consequences on discretization, stencil shape, the resulting matrix structure, and thus how the solver converges. 7.3.1.1 Exponential stretching of topography and the radial coordinate

If the actual radial scaling of the problem r ∼ rEa r t h 6400 km were used, all the topographical features on the Earth would appear flat (as Mount Everest is about 8.6 km

Downloaded 11/04/15 to 142.103.160.110. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

162

Chapter 7. RBF-FD Applications to Geo-Modeling: Spherical Domains

(a)

(b)

(c)

(d)

Figure 7.11. Earth’s topography: (a) Actual scale of the problem (the color bar is the gridaveraged altitude above sea level in kilometers). (b) Result of the change of variable (7.9) on the topography with β = 0.05. (c) Node set for a spatial discretization of the Earth’s topography at an ≈ 400 km resolution at sea level, under the change of variable (7.9) with β = 1. (d) Same as (c) but with β = 0.05 and ≈ 150-km resolution at sea level. In both Figures (c) and (d), there are twice as many nodes on land as on water to maintain quasi-uniform distributions in the presence of steeper gradients.

above sea level). In order to increase the topographical resolution of the model, a change of variable is considered: (7.9) r (ξ ) = Ae β(ξ −ξ0 ) + B, where A and B are constants determined by enforcing the conditions r (ξ0 ) = r0 = 6400 km

and

r (ξ t o p ) = r t o p , = 6460 km

(7.10)

and β is a parameter that controls the topography stretching. Under this change of variable, the topogrpahy of the Earth is mapped over a sphere of mean radius ξ0 , and the radial coordinate r is exponentially stretched. For details, see Section 4.1 in [8]. Examples are given for two values of β and different angular resolutions in Figure 7.11. 7.3.1.2 Spatial discretization

The majority of the domain is discretized horizontally by using a spherical shell formed by NH icosahedral nodes and radially by repeating this spherical shell from sea level to the top boundary (60 km) with a spacing of h r in ξ , remembering that all computations are done in the evenly spaced radial ξ domain. This results in N r radially aligned spherical

Downloaded 11/04/15 to 142.103.160.110. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

7.3. A 3-D ellipitic PDE with irregular boundaries: Global electric circuit

163

shells of NH nodes. However, to incorporate topography, the following alterations need to be made: 1. To create the node layouts shown in Figure 7.11, an algorithm was developed to distribute nodes along the topography, with approximately twice as many nodes on land as over the oceans to accommodate the steeper gradients of the orography. 2. When part of a spherical shell intersects land, the nodes that fall under the Earth’s topography are discarded. 3. Finally, a rearrangement of the nodes are needed where a shell intersects land in order to have a quasi-uniform distribution of nodes so as to have more well-balanced stencils and thus increase the stability of the elliptic solver. Therefore, the nodes on each shell are repelled only in the lat-lon direction (using a charge-type repulsion algorithm) while holding the nodes on the topography fixed; this allows the nodes near the Earth’s surface to follow the topography more closely yet keeping the radial distance between shells fixed and thus preserving conditioning of the matrix system to be solved. As a result of incorporating topography, there are two very different and distinct shapes of stencils used to approximate the differential operators in the domain, using Gaussian RBFs. A near-surface region formed by the nodes close to the topography (< 8 km), where the stencil to approximate the differential operators at a node is composed of the closest 56 nodes in 3-D space (found via a k-d tree search) and thus forming a true 3-D RBF-FD stencil, with  = 3.25. In contrast, above all topography (i.e., > 8 km), where radially 2-D + 1D stencil formation is present, as shown in Figure 6.6(b). A hybrid RBF-FD/FD approach is implemented, where a classical 5-node FD stencil is used to approximate operators in the radial direction (1-D), combined with a 21-node stencil for RBF-FD approximations of the angular derivatives (2D), i.e., the surface Laplacian Δs . For ease of computation, the weights for the Δs are computed once using (6.4) on a unitary sphere, with  = 2 and then scaled by 1/r (ξ )2 , where r (ξ ) is the radius of each spherical shell above 8 km. The difference in the structure of the stencils between the regions with and without topography is strikingly evident when the sparsity pattern of the matrix representing the discretized elliptic PDE is considered, as shown in Figure 7.12(a). The broader-bandwidth, denser, unstructured region in the upper left corner area of the matrix in Figure 7.12(a) corresponds to where 3-D RBF-FD stencils are used for discretization of the elliptic operator, i.e., in the region where topography is present. In constrast, the rest of the matrix, with its pentadiagonal-type pattern, corresponds to the 21+5 node hybrid RBF-FD/FD stencil in the structured region above 8 km. 7.3.1.3 Ghost nodes to increase stencil symmetry near boundaries

This topic will be covered more in depth in Section 8.1.2. Stencils near boundaries will be more one-sided and might have skew shapes, especially when incorporating topography. As a result, the weights that approximate the differential operators on those stencils can adversely affect the stability of the numerical solver. In order to avoid this issue, the concept of “ghost nodes” is implemented. The name comes from the fact that these nodes are used to approximate the differential operator on near-boundary/boundary nodes, making the stencils more symmetric, but no equations are ever enforced at these nodes, as they are outside the domain. For most boundary nodes, a ghost node is introduced directly outside the domain, i.e., under the topography or directly above about 60 km (the only caveat to this is when the terrain becomes to steep, as in the Andes or Himalayas,

Downloaded 11/04/15 to 142.103.160.110. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

164

Chapter 7. RBF-FD Applications to Geo-Modeling: Spherical Domains

(a)

(b)

Figure 7.12. Sparsity pattern of the differentiation matrix that approximates 7.7 with 4◦ × 1 km resolution (a) before ordering and (b) after applying sparse reverse Cuthill-McKee ordering.

making the ghost nodes close to overlapping; in these cases, a smaller one-sided stencil is used to maintain stability of the solver). To determine the value of the function at the ghost nodes, the PDE is enforced on the boundary, in addition to the Dirichlet boundary conditions. Hence, the resulting linear system has as many equations as unknowns. The result is that the interior stencils near the irregular boundary recover a more symmetrical shape, and the stability of the solver improves. This procedure is also used at the top boundary to enable the use of five-node stencils in the radial direction.

7.3.2 Test case results Drastic changes in the sparsity pattern of the matrix, as seen in Figure 7.12(a) from a much denser and unstructured pattern to a pentadiagonal-type pattern, generally impede iterative solvers, making for much poorer and slower convergence. As a result, a reverse Cuthill-McKee reordering is applied, giving a sparsity pattern with an overall narrower, more consistent bandwidth, as seen in Figure 7.12(b). The linear system is then solved using GMRES with an incomplete LU factorization as a preconditioner. A residual tolerance of 10−9 is used in the iterative solver. The subplots in Figure 7.13 show the relative ∞ error and the number of iterations required, respectively, as a function of the total number of nodes N . Table 7.3 gives the exact specifications of the node sets used for the results in Figure 7.13. Table 7.3. The specifications for the results shown in Figure 7.13.

Nr

NH

N

52 74 102 142 201 285 401

252 362 492 812 1002 1442 2562

17369 34729 65632 145786 261560 526853 1241164

l∞ error

1.1 · 10−2 4.0 · 10−3 1.9 · 10−3 5.9 · 10−4 2.6 · 10−4 1.4 · 10−4 5.4 · 10−5

# iterations 34 41 50 71 206 505 730

Downloaded 11/04/15 to 142.103.160.110. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

7.3. A 3-D ellipitic PDE with irregular boundaries: Global electric circuit

165

Figure 7.13. (Left) Relative l∞ error and GMRES iterations (right) as a function of the total number of node N .

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

Chapter 8

RBF-FD Applications to Geo-Modeling: Limited-Area Domains 8.1 Solving the Navier-Stokes equations on a rectangular domain with RBF-FD The 2-D compressible Navier-Stokes (NS) equations for modeling dynamics in a stratified dry atmosphere at low Mach number (M≈ 0.1) are given by ∂P  ∂u ∂u ∂u =−u −w − cp θ + θ + μΔu, ∂t ∂x ∂z ∂x

(8.1)

∂P  gθ ∂w ∂w ∂w =−u −w − cp θ + θ + + μΔw, ∂t ∂x ∂z ∂z θ

(8.2)

∂θ ∂θ ∂θ =−u −w + μΔθ, ∂t ∂x ∂z ' &  ∂ u ∂ w! R  ∂P ∂P dP ∂ P − d P +P =−u −w + + , ∂t ∂x dz ∂z cv ∂x ∂z

(8.3) (8.4)

where u and w are the flow velocities in the horizontal and vertical directions, respec3 4 p tively; P = ( p )Rd /c p is the nondimensional Exner pressure p0 = 1 × 105 Pa that allows 0 for the elimination of the equation of state p = ρRd T (notice that conservation of mass T (8.4) is then written in terms of P instead of ρ); and θ = P is the potential temperature, a conserved quantity in dry air with no heating/cooling sources. The constants c p = 1004 and cv = 717 are the specific heat at constant pressure and the specific heat at constant volume, respectively, with the gas constant for dry air being Rd = c p − cv = 287. The gravitational constant is g = 9.81 m/s2 , and μ is the viscosity. All quantities to be solved for, [u, w, θ, P ]T , are perturbations to the background state denoted by barred quantities [θ = 300K, P ]T that are assumed to be in hydrostatic balance, i.e.,

dP dz

=−

g cp θ

, meaning

the background Exner pressure is a linear function of height z. With regard to the velocities, the background is at a state of rest u = w = 0. Before a case study is presented, the following sections discuss the node sets that will be considered and a method for increasing the accuracy of the RBF-FD approximation near boundaries.

167

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

168

Chapter 8. RBF-FD Applications to Geo-Modeling: Limited-Area Domains

Figure 8.1. Three different node distributions considered for solving the Navier-Stokes equations.

8.1.1 Node sets and stencils Unlike traditional methods, RBF-FD has the advantage of being equally simple to apply on any node layout. Figure 8.1 shows the three types of node layouts that will be constrasted in terms of how well they capture the physical features of the fluid flow. However, since hexagonal layouts provide the optimal packing of points in a 2-D plane, the total number of nodes N would be larger for a fixed spacing h as compared to Cartesian and scattered. To do a fair comparison, N must be keep constant between the nodes sets. This will result in hexagonal layouts having a slightly coarser resolution. When considering stencil sizes for node sets that are lattice based, such as Cartesian and hexagonal, best accuracy is achieved for stencil sizes that provide a symmetric layout, as shown in Figure 8.2 for a 2-D plane. The reason for this is that, away from boundaries, information for approximating the derivatives will be picked up in a uniform manner, with as many directions as possible being treated equally given the lattice structure. Hence in 2-D, hexagonal lattices have an advantage over Cartesian ones in providing better anisotropy, with three directions being treated equally as opposed to two. In higher dimensions, hexagonal node layouts generalize to ME-type ones (equilibria of particles repelling each other), increasing this advantage still further. Scattered quasi-uniform node layouts (right panel of Figure 8.1) often give RBF-FD performance somewhere in between the hexagonal and the Cartesian cases. Good anisotropy may then be partly offset by some low level of “noise” from node set irregularities. However, the big advantage of these node sets versus any form of lattice-based node sets lies in their geometric flexibility with regard to treating irregular boundaries/interfaces and for local refinement.

8.1.2 Increasing accuracy near boundaries: Ghost nodes As with any numerical method, approximations near boundaries exhibit Runge phenomenon due to the one-sided nature of the stencils. One method for forming a stencil where information is coming from all sides is to place a layer of nodes just outside the boundary of the domain. These nodes are often called “ghost” nodes, as no equations are enforced there, but the weighted function values at these nodes are part of stencils for approximations at boundary and near-boundary interior nodes where equations (boundary conditions and/or the PDEs) are enforced. The function values at the ghost node locations are solved for by enforcing non-Dirichlet conditions or the PDE itself at the boundary nodes . For example, if the upper boundary of a rectangular domain is free-slip, then ∂ u/∂ z = 0. This condition can be used to solve for values of u at the ghost nodes on the top boundary as follows:

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

8.1. Solving the Navier-Stokes equations on a rectangular domain with RBF-FD 1 node

5 nodes

9 nodes

13 nodes

21 nodes

25 nodes

29 nodes

37 nodes

45 nodes

49 nodes

57 nodes

61 nodes

69 nodes

81 nodes

89 nodes

97 nodes

169

(a) 1 node

7 nodes

13 nodes

19 nodes

31 nodes

37 nodes

43 nodes

55 nodes

61 nodes

73 nodes

85 nodes

91 nodes

97 nodes

109 nodes

121 nodes

127 nodes

(b)

Figure 8.2. Symmetric stencils in 2-D: (a) Cartesian stencils. (b) Hexagonal stencils. Note that n = 13, 37, 61, 97 are stencil sizes held in common between the two sets.

1. For each node on the upper boundary, one ghost node is placed just outside the boundary, as shown in Figure 8.3. In this case, ghost nodes are not placed outside the corners of the domain. 2. Next, for a given boundary node, the n − 1 nearest neighbors (n being the stencil size) to that node are located. This forms the stencil, which is a combination of ghost nodes x G , boundary nodes x B , and interior nodes x I , also shown in Figure 8.3. A boundary node i s placed on the corner of the domain and the upper boundary condition enforced there. 3. Given these node locations, the differentiation weights approximating ∂ /∂ z at that boundary node are found, as discussed in Section 5.1.4. This forms one row of the

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

170

Chapter 8. RBF-FD Applications to Geo-Modeling: Limited-Area Domains

Figure 8.3. An example of a boundary stencil using ghost nodes to alleviate any one-sidedness that might lead to Runge phenomena.

differentiation matrix D z . The process is then repeated for all nodes along the upper boundary, resulting in a rectangular sparse matrix of the size NB × (NB + NG + NI ). It will have the block structure of Dz =



WI (NB × NI ) WB (NB × NI ) WG (NB × NG )



,

where W has been used to denote the submatrices of weights that compose D z and the size of each is given in parentheses. Due to its extreme sparsity, the matrix is never actually formed, only the entries are saved with pointers to those entries (easily done by using the s pa r s e command in MATLAB). 4. Now that the weights have been found, the boundary condition ∂ u/∂ z = 0 → D z [u I u B u G ]T = 0 is enforced, WI u I + WB u B + WG u G =0,

(8.5)

and the function values at the ghost nodes u G are solved for by 3 4 u G = −WG−1 WI u I + WB u B .

(8.6)

It should be noted that u B is known since the system of PDEs is also time stepped on the boundary. Of course, the elliptic system (8.6) has to be solved at each time step. However, this is trivial since WG is an exceptionally sparse m-diagonal matrix (assuming the ghost nodes have been ordered sequentially), where m is the number of ghost nodes that have been used per stencil. For example, if three ghost nodes are used as in Figure 8.3, then WG will be a tridiagonal matrix.

8.1.3 A numerical case study for the NS equations on different node layouts To test the dynamics in a numerical weather prediction model, an often-used study is known as the Straka cold density current [258]. A bubble of cold air, centered at x = 0 m, z = 3000 m, falls to the ground and develops three smooth and distinct rotors due to shear instability (i.e., Kelvin-Helmholtz instability) as it spreads sideways. Figure 8.4 shows the behavior of the numerical solution in time from t = 0 s until the final time

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

8.1. Solving the Navier-Stokes equations on a rectangular domain with RBF-FD

171

Figure 8.4. The time evolution of the potential temperature θ using a hexagonal node layout at 100 m resolution. Reprinted with permission from Cambridge University Press. [96]

t = 900 s. The computational domain is [−25, 600, 25, 600] m in x with periodic boundary conditions and [0, 6400] m in z with no-flux (w = 0) and free-slip (∂ u/∂ z = 0) boundary conditions on the velocity and Neumann on the temperature and pressure (see [3] for exact relations). The viscosity is μ = 75 m2 /s. PHS RBFs, r 7 , together with polynomials up to fourth degree, are used to approximate all spatial derivatives locally by the RBF-FD approach (see Section 5.1.4) with a stencil size of n = 37, a size in common between Cartesian and hexagonal node layouts. A Δ3 hyperviscosity was used, with the weights calculated by PHS with polynomials up to fourth order, as given in (5.11). The remaining system of first-order ODEs is time stepped with RK4. In Figure 8.5, the performance of each of the three different node layouts given in Section 8.1.1 is considered. In the highest-resolution displays (bottom row of subplots), it can be seen that all node layouts converge to the same solution. However, in numerical weather prediction, the ability to run 100 m models for real-time prediction is not currently a practical reality. The fact that observational data that initializes models is generally observed on an order of a kilometer or more makes the degree to which the physics is captured at coarser resolutions more important. In the coarsest case shown, using only 720 nodes in the domain (top row of subplots), the calculations using hexagonal and scattered nodes give more clear evidence of the first

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

172

Chapter 8. RBF-FD Applications to Geo-Modeling: Limited-Area Domains

(largest) rotor being formed, with hexagonal performing slightly better. Furthermore, the center of this rotor is closer to the correct location (compare to 100-m resolution). Although the first rotor for the scattered case is not quite as nicely formed as in the hexagonal case, it has entrenched more cold air, having a −4.5K contour (teardrop shape). Notice that at 100 m the −4.5K contour is the coldest that appears. At the next higher resolution (2700 nodes in the entire domain; about 350 m resolution), these two node layouts provide a clearer picture of the beginning formation of the second rotor. In contrast, Cartesian nodes give solutions that are more prone to spurious numerical oscillations, as can be seen by the “wiggles” in the contour lines. Also, the overshoots in white present in the N = 720 case are of 2.4o K as opposed to 1o K for hexagonal and scattered nodes. Not until 90 m are these spurious oscillations no longer visible in the Cartesian case. In terms of front location, at coarser resolutions (200 m and above), scattered nodes performed best. The most visibly striking difference is at low resolution (N = 720). The current traveled slower than appropriate, undershooting the correct position (compare to 90-m resolution) but not to the degree of hexagonal nodes and not overshooting it as in the Cartesian node case. For further details, refer to [3]. The differences between the columns of subplots reflect only the intrinsic resolution capabilities of the different node layouts for capturing the physics. The traditional Cartesian choice is the least effective one. If using a fixed node separation, a hexagonal layout provides the optimal packing of nodes in a fixed region. Conversely, with fixed node numbers, as in Figure 8.5, the separation distance then becomes somewhat larger. Even so, at every resolution level, the hexagonal choice gives better accuracy than the Cartesian one. The big advantage of generalizing further, from hexagonal to quasi-uniformly scattered nodes, is that it then becomes trivial to implement spatially variable node densities, i.e., to do local refinement in select critical areas. It is very important to note that this major increase in geometric flexibility (from hexagonal to quasi-uniformly scattered) hardly has any negative effect at all on the accuracy that is achieved or on the algorithmic complexity of the code. To place Figure 8.5 in context with the results of other numerical methods, a comparison is done with DG, spectral element (SE), finite volume (FV), and finite difference upwinding schemes in Figure 8.6. As can be seen, when no filtering is used in the RBF-FD method, there is a trade-off between capturing features at low resolutions and preserving monotonicity. Only the FV and upwind schemes do not exhibit Gibbs oscillations and have solutions with monotonic properties. However, the price to be paid is that the solution is smoothed out with regard to both rotor formation and the amount of cold air that has been entrenched. The DG and SE solutions have more structure, but the beginning formation of the second rotor is still not seen as well as in the RBF-FD model. Lowering the viscosity to 2 ·10−5 m2 /s (approximately that of dry air at 300K at mean sea level pressure), the solution enters the turbulent regime. In such regimes, there is no convergence to any solution, as energy cascades to smaller and smaller scales, eventually entering the subgrid scale domain. Nevertheless, it is interesting to observe whether the model remains stable in this regime. Figure 8.7 shows the solution at 100 m, 50 m, and 25 m resolutions on the three different node layouts. For any given resolution, the solution looks completely different depending on the node layout. This is to be expected as changing the node layout in a simulation with such low viscosity is equivalent to introducing slight perturbations in the solution. The fact that the viscosity μ is very low does not affect the time stability, and the time step did not have to be altered from the μ = 75 m2 /s case. Stability is governed solely by the fact that the time step could not exceed the speed of sound in air. Also, the same amount of hyperviscosity was used in this case as in the above case of μ = 75 m2 /s.

8.2. Forward seismic modeling

173

N ≈ 38500 N ≈ 10000 N ≈ 2700 N ≈ 720

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

Cartesian

Hexagonal

Scattered

700 m

800 m

700 m

350 m

400 m

350 m

180 m

200 m

180 m

90 m

100 m

90 m

−12.5 −10.5 −8.5 −6.5 −4.5 −2.5 −0.5

−12.5 −10.5 −8.5 −6.5 −4.5 −2.5 −0.5

−12.5 −10.5 −8.5 −6.5 −4.5 −2.5 −0.5

Figure 8.5. The solution at the final time for the density current test case solved on different node layouts for approximately 800 m, 400 m, 200 m, and 100 m resolutions. Only half the solution is shown to enlarge details. Contours for the density current begin at −0.5K and are in intervals of 1K. The white areas are enclosed by a contour of 0.5K.

RBF – FD 37 node stencil

5th – order upwind advecon

8th – order DG and SE

4th – order finite volume 400 m

Figure 8.6. Comparison at 400 m between four different numerical methods. Contour intervals: RBF-FD 1o K [3], DG and SE 0.25o K [127], FV 1o K [200], fifth-order upwind 1o K [249].

8.2 Forward seismic modeling Seismic exploration is the primary tool used for finding and then mapping out hydrocarbon deposits. In forward modeling, subsurface structures are assumed to be known, and the task is to simulate elastic wave propagation through the medium. Inversion programs then update subsurface assumptions to reconcile the model response with actual measurements. Typically, hundreds of irregularly curved interfaces are present, often interrupted by fracture lines with associated translations between the strata on the two sides. Throughout geologic time, the vast majority of all hydrocarbons (such as natural gas and oil, lighter than water) have migrated up to the surface and then biodegraded. As a result, today’s deposits of hydrocarbons are mostly small pockets of where hard layers have

174

Chapter 8. RBF-FD Applications to Geo-Modeling: Limited-Area Domains

Hexagonal

Scattered

N ≈ 526593 N ≈ 152650 N ≈ 38500

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

Cartesian

−21

−17

−13

−9

−5

−1

−21

−17

−13

−9

−5

−1

−21

−17

−13

−9

−5

−1

Figure 8.7. The potential temperature θ at t = 900 s for the low-viscosity 2·10−5 m 2 /s density current using approximately 100 m (N ≈ 38500), 50 m (N ≈ 152650), and 25 m (N ≈ 526593) resolution on the three different node layouts.

formed traps due to their curvature or the presence of corners resulting from fractures. With drilling being far more expensive (and environmentally damaging) than seismic exploration, the latter is constantly being pushed to its limits, leading to some of the largest computational tasks in any field. Figure 8.8 shows a extremely simplified model for the Marmousi test case, itself a highly simplified 2-D vertical slice off the coast of Madagascar (cf. [191], in particular its Figure 2). The governing elastic wave equations in 2-D spatially varying media are ⎧ ρu t = f x + gy , ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎨ ρv t = gx + hy , (8.7) ft = (λ + 2μ) u x + λ vy , ⎪ ⎪ ⎪ g t = μ (u x + vy ), ⎪ ⎪ ⎪ ⎩ h = (λ + 2μ) vy + λ u x . t The dependent variables are u, v (horizontal and vertical velocities) and f , g , h (components of the symmetric stress tensor), and the material is specified by ρ (density) and λ, μ (Lamé parameters for compression and shear). Away from interfaces, these ( equations support two types of waves: P-waves (pressure or primary) with speed c p = (λ + 2μ)/ρ and ( S-waves (shear or secondary) with speed c s = λ/ρ. Each incoming wave that encounters an interface usually results in four main outgoing waves—reflected and transmitted P- and S-waves (and sometimes also waves following interfaces). With typically hundreds of interfaces, wave patterns become extremely complicated. Simulated return signals at the surface need to accurately represent (i) wave propagation over long distances through regions with smoothly varying material properties as well as (ii) reflection/transmissions at interfaces (with respect to amplitudes, phase angles, and directions). In the smoothly varying regions, the dominant error source is numerical dispersion. The only practical remedy against this is to use high-order approximations [85]. Industry standard advanced from second to fourth order in the 1980s, and FD approximations of extremely high (around 20th) order are in common use today. It has proven much

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

8.2. Forward seismic modeling

175

Figure 8.8. Subsurface acoustic velocities in a “micro-Marmousi” test case. Reprinted with permission from Cambridge University Press. [96]

more difficult to achieve accurate interface treatments [181, 182, 259]. While closed-form expressions are available in simplified cases (such as straight interfaces between constant media), incorporating related approximations in full production codes has so far not been cost effective. Foregoing special treatments at interfaces (beyond possibly making the material properties rapidly varying rather than outright discontinuous) has surprisingly remained the industry standard, accepting typically first-order convergence for reflected waves. The RBF-FD/AC method (with AC standing for “analytic correction”) achieves third-order accuracy both in smooth regions and across curved interfaces, making it very competitive. For simplicity, the AC concept is described only in 1-D below, but we show results for 2-D and 3-D test calculation in Sections 8.2.2 and 8.2.3, respectively. More information on the RBF-FD/AC approach is given in [190].

8.2.1 RBF-FD/AC approach Figure 8.9 illustrates a typical node layout and also the different stencil types used in a still more simplified test case, with just one curved interface in 2-D. The nodes are distributed to straddle the interface (following observations in [87]) but then smoothly transition to become lattice based a short distance away from it. Three stencil types are used: (i) regular FD when the whole stencil is lattice based, (ii) standard RBF-FD when part of the stencil has irregularly nodes away from the interface, and (iii) RBF-FD/AC when an interface intersects a stencil. For simplicity, the RBF-FD/AC discretization is described here in 1-D. Equation (8.7) can then be reduced to 6 5 6 A 1 ∂ B5 0 ∂ u u ρ∂x . (8.8) = ∂ f ∂t f ρc 2 0 ∂x

While both ρ and c typically jump at an interface, continuity of motion and traction require u and f to be continuous (in the 2-D case, there will similarly be four continuity relations linking the five variables in (8.7)). Denoting left and right sides of the interface by subscripts L and R, respectively, it will then hold that 5 6 5 6 5 6 uL uR 0 − = , 0 fL fR which implies ∂k ∂ tk

5

uL fL

6

5 −

uR fR

6

5 =

0 0

6 , k = 0, 1, 2, . . . .

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

176

Chapter 8. RBF-FD Applications to Geo-Modeling: Limited-Area Domains

Figure 8.9. The three stencil types (a), (b), (c) used in a hybrid approach that combines FD with RBF-FD/AC. Reprinted with permission from Cambridge University Press. [96]

With use of (8.8), these time identities will translate to relations between spatial derivatives for u and f on the two sides. The primary idea is to embed these relations in the supplementing polynomials for the approximation RBF-FD differentiation weights (with the option to also embed them in the RBFs themselves; cf. [303], where a similar approach was considered in the context of Maxwell’s equations). Figure 8.10 illustrates how one thus arrives at “interface aware” supplementary polynomials (with their changes across the interface dependent on the material properties on the two sides).

8.2.2 2-D test case Based on the geometry shown in Figure 8.8, part (a) of Figure 8.11 shows the vertical velocity v associated with an underground explosive source and part (b) a very accurate calculation of the solution at a later time t = 0.3. Parts (c) and (d) display the error at this same later time for RBF-FD/AC solutions when using N = 38400 and N = 153600 nodes, respectively (using IMQ-type RBFs; stencils of type (b) (as shown in Figure 8.9): n = 19, polynomials degree 3; stencils of type (c): n = 38, polynomials degree 2). Since the color bars for these latter two cases ((c) and (d) in Figure 8.11) are identical, it can readily be noted that halving the typical node separation h has reduced the error by more than a factor of 10. For a scheme that is third-order accurate everywhere, the expected error reduction would have been a factor of eight.

8.2.3 3-D test case Figure 8.12 illustrates a simple acoustic 3-D test case that seismic modeling codes routinely are applied to before proceeding to more complicated situations with many more (and intersecting) interfaces. In this case with a single inclined planar interface, the availability of an analytic solution simplifies error comparisons. One can thus for different methods readily compare the errors at the receiver location against different numerical parameters, such as a method’s order of accuracy, the total number of grid points used, the operation count per grid point or per “element,” etc. The most meaningful comparisons are, however, obtained by simply comparing error vs. elapsed (wall clock) time on equivalent computer systems, when each approach is implemented as effectively as possible. Part (a) of

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

8.2. Forward seismic modeling

177

1

0.5

y

0

1 1

x

−0.5

2

x

x3 x4

−1 −1

(a)

y

−0.5

0 x

0.5

1

1

1

0.5

0.5

y

0

0

(1|1)

(1|1)

1

1

2

2

(4x |x )

−0.5

1

(4x2|x2)

(4x |x ) 3

3

4

4

3

(16x |x )

(b)

−1

−0.5

0 x

0.5

3

(4x |x ) 4

(16x |x )

−1

1

(x |x )

−0.5

1

(c)

4

(16x |x )

−1 −1

−0.5

0 x

0.5

1

Figure 8.10. (a) The “naïve” supporting monomials up through degree 4 (appropriate away from interfaces). (b), (c) The interface specific supporting polynomials for the variables u and f, respectively, in the special case of cL = 1, cR = 2, ρL = ρR = 1. Reprinted with permission from Cambridge University Press. [96]

Figure 8.11. Test calculation for the “micro-Marmousi” example using the RBF-FD/AC approach, showing better than a factor of 10 reduction in error when the number of nodes is doubled.

Figure 8.13 shows a solution snapshot, while part (b) shows how errors depend on elapsed computer time when using RBF-FD vs. very high order FD implemented on Cartesian lattices. The CPU is here a 12-core Intel Xeon processor and the GPU a NVidia Tesla K20 card. The RBF-FD curves are obtained as described above, with a 3-D AC (analytic correction) approach used at the interface, while the FD implementation (like present

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

178

Chapter 8. RBF-FD Applications to Geo-Modeling: Limited-Area Domains

Figure 8.12. Standard 3-D acoustic test problem with a single inclined interface.

Figure 8.13. (a) Snapshot of the solution to the 3-D acoustic test problem. (b) Error at receiver location for RBF-FD vs. FD discretizations, displayed against elapsed computer time (CPU and GPU implementations).

production codes in the industry) relies on highly refined grids and very high order approximations. We note in particular (i) the advantage by around an order of magnitude in cost effectiveness provided by RBF-FD over FD and (ii) how well the RBF-FD approach can be implemented on GPUs. Performance of state-of-the-art finite element codes are somewhat comparable to the FD case [306]. Their better accuracy at interfaces (than regular FD) becomes offset by increased complexity and higher operation counts not only when creating 3-D elements but also during the subsequent time marching.

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

Appendix A

Introduction to RBFs via Cubic Splines

Chapter 3 introduced RBFs from the perspective of generalizing PS methods in order to allow for scattered nodes in more than 1-D. This appendix provides a different initial motivation, namely, seeking to overcome some limitations of cubic splines, again with regard to node distrubutions in multiple dimensions but now also to raise the order of accuracy.

A.1 Cubic splines in 1-D Figure A.1 a shows a smooth function that is sampled at equispaced nodes over [−1, 1] and also an interpolating cubic spline (obtained using MATLAB’s default Not-a-Knot end conditions). The lower figure displays the error of this interpolation. A standard cubic spline is made up of a different cubic polynomial between each pair of adjacent node points, and it may at these points feature a jump in the third derivative (the function and its first two derivatives are continuous everywhere). The standard approach for computing the coefficients of the different cubics that form the spline requires only the solution of a (diagonally dominant) tridiagonal linear system [212]. If the spacing between the sample points is h, the size of the error will decrease like O(h 4 ). It makes almost no difference in the algorithm if the nodes are equally spaced. However, generalizations to more space dimensions have in the past been practical mainly if the nodes are lined up in the coordinate directions.

A.1.1 RBF representation: Interval interior Another way to approach the problem of finding the 1-D cubic spline (for now omitting to address the issue of end conditions) is the following: At each data location x j , j = 1, . . . , n, place a translate of the function φ(r ) = |r |3 , (A.1) i.e., at location x j , the function φ(x − x j ) = |x − x j |3 . We then ask if it is possible to form a linear combination of all these functions n  λ j φ(x − x j ) (A.2) s(x) = j =1

such that this combination takes the desired function values f j at the data locations x j , j = 1, . . . , n., i.e., enforcing s(x j ) = f j . This amounts to asking for the coefficients λ j to 179

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

180

Appendix A. Introduction to RBFs via Cubic Splines

Figure A.1. (Top) The function arctan(10x) and its Not-a-Knot cubic spline interpolant. (Bottom) The interpolation error.

satisfy the linear system of equations ⎡ φ(x1 − x1 ) φ(x1 − x2 ) · · · ⎢ φ(x − x ) φ(x − x ) · · · ⎢ 2 1 2 2 ⎢ .. .. .. ⎢ . ⎣ . . φ(xn − x1 ) φ(xn − x2 ) · · ·

φ(x1 − xn ) φ(x2 − xn ) .. . φ(xn − xn )

⎤⎡ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎦⎣

λ1 λ2 .. . λn

⎤

⎡

⎥ ⎢ ⎥ ⎢ ⎥=⎢ ⎥ ⎢ ⎦ ⎣

f1 f2 .. . fn

⎤ ⎥ ⎥ ⎥. ⎥ ⎦

(A.3)

Assuming that this system is nonsingular, it can be solved for the coefficients λ j . The interpolant s(x), as given by (A.2), will then become a cubic function between the nodes and, at the nodes, have a jump in the third derivative. We have thus found another way to create an interpolating cubic spline. At first glance, there are several reasons why using translates of (A.1) might come across as a bad idea: 1. We have replaced a diagonally dominant tridiagonal linear system with a full system, lacking diagonal dominance. 2. The overall numerical conditioning has become doubtful. The basis functions become very big at large distances, and large amounts of numerical cancellation will need to occur in order to represent functions that are not rapidly growing. 3. Strategies for implementing various spline end conditions (essential for good performance) are not immediately obvious. There are, however, two fundamental advantages in the φ(r ) = |r |3 basis function approach for cubic splines, as described in Sections A.2 and A.3, respectively. Before addressing that, we note that the standard cubic spline end conditions actually can be readily implemented also in the φ(r ) = |r |3 formulation.

A.1. Cubic splines in 1-D

181

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

A.1.2 RBF representation: Boundary conditions The cubic radial function φ(r ) = r 3 applied to 1-D data produces an interpolant, s(x) =

n 

λk |x − xk |3 ,

(A.4)

k=1

that is a cubic in x between the nodes. The functions s(x), s  (x), and s  (x) are everywhere continuous, but there will typically be a jump in s  (x) at each node point. It is well known that cubic splines need an extra condition at each end (as for natural splines s  (x1 ) = s  (xn ) = 0) or two freedoms removed (as for Not-a-Knot splines—no jumps in the third derivative at x2 and xn−1 —the nodes one step in from each boundary). Since the former condition degrades the typical spline accuracy from O(h 4 ) to O(h 3 ) near the ends, the latter is usually preferred (and is the default option in MATLAB). The standard RBF representation needs no such additional conditions. Hence, it is natural to ask what end conditions it does correspond to. Following [92], we note that, for x > xn , every term |x − xk |3 in (A.4) becomes equal to (x − xk )3 , whereas if x < x1 , every term |x − xk |3 becomes equal to −(x −xk )3 . If we for simplicity assume that −1 = x1 < x2 < · · · < xn = 1, then if s(x) = ax 3 + b x 2 + c x + d for x ≥ 1, it must hold that s(x) = −ax 3 − b x 2 − c x − d for x ≤ −1. From s(1) = a + b + c + d , s  (1) = 3a + 2b + c, s  (1) = 6a + 2b ,

s(−1) = a − b + c − d , s  (−1) = −3a + 2b − c, s  (−1) = 6a − 2b

follow, after eliminating a, b , c, d between the six equations, the two end conditions 3

s  (1) = 2s  (1) − s  (−1)

− 2 (s(1) + s(−1)),

s  (−1) = s  (1) − 2s  (−1)

− 2 (s(1) + s(−1)).

3

(A.5)

These conditions make little logical sense. The coupling of the two sides to each other is very unnatural, and it also forces cubic RBFs, implemented by (A.4), to feature severe end oscillations even for constant (nonzero) data. It transpires, however, that minor variations of (A.4) will make it match either natural or Not-a-Knot splines. Carrying these ideas over to other RBF types and to more space dimensions will become possible approaches toward controlling RBF boundary issues. RBF natural Following the idea in$ Section $ 3.1.3.5, we replace (A.4) by s(x) = $ splines a + b x + λk |x − xk |3 with the constraints λk = λk xk = 0. For x ≥ xn , it then $ $ d2 $ holds that s  (x) = d x 2 λk (x − xk )3 = 6x λk − 6 λk xk = 0, and, similarly, s  (x) = 0 also for x ≤ x1 . This RBF approximation therefore satisfies all the requirements of the unique natural spline. RBF Not-a-Knot splines Conventionally, one centers a basis function at each node point. If we shift the two basis functions normally centered at x2 and xn−1 to somewhere outside the left and right end points, respectively, we can still fit all the data, but s  (x) will not jump at the x2 and xn−1 locations; i.e., we have reproduced the (again unique) Not-a-Knot spline. We will build on these results from cubic splines when discussing RBF boundary improvement ideas in Section 3.3.4.

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

182

Appendix A. Introduction to RBFs via Cubic Splines

Figure A.2. Illustration of the RBF concept in 1-D and in 2-D.

A.2 Generalization to multiple dimensions Figure A.2 a illustrates the RBF idea in 1-D. At each data location x j , we centered a translate of our symmetric function φ(x). In 2-D, as illustrated in Figure A.2(b), we instead use a rotated version of the same radial function. In d dimensions, we can write these rotated basis functions as φ(||x − x j ||), where · denotes the standard Euclidean norm. The form of the RBF interpolant and of the linear system that is to be solved has hardly changed from the 1-D case. Instead of (A.2) and (A.3), we now use as interpolant s(x) =

n  j =1

λ j φ(x − x j )

(A.6)

with the collocation conditions ⎡ ⎢ ⎢ ⎢ ⎢ ⎣

φ(- x 1 − x 1 -) φ(- x 2 − x 1 -) .. . φ(- x n − x 1 -)

φ(- x 1 − x 2 -) φ(- x 2 − x 2 -) .. . φ(- x n − x 2 -)

··· ··· .. . ···

φ(- x 1 − x n -) φ(- x 2 − x n -) .. . φ(- x n − x n -)

⎤⎡ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎦⎣

λ1 λ2 .. . λn

⎤

⎡

⎥ ⎢ ⎥ ⎢ ⎥=⎢ ⎥ ⎢ ⎦ ⎣

f1 f2 .. . fn

⎤ ⎥ ⎥ ⎥. ⎥ ⎦

(A.7)

In particular, we note that the algebraic complexity of the interpolation problem has not increased with the number of dimensions—we will always end up with a square symmetric system of the same size as the number of data points. Cubic splines have thus been generalized to apply also to scattered data in any number of dimensions (noting, however, again the extensions discussed in Section 3.1.3.5).

A.3. Improvement in accuracy from algebraic to spectral

183

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

A.3 Improvement in accuracy from algebraic to spectral The error O(h 4 ) for cubic splines in 1-D will become O(h 6 ) in the case of quintic splines, and it falls to O(h 2 ) for linear splines (which correspond to φ(r ) = |r |). In general, if we take the RBF approach as outlined above and use φ(r ) = |r |2m−1 , m ∈ , the error will become O(h 2m ) (even powers in φ(r ) will not work; for example, if φ(r ) = r 2 , the interpolant (A.2) will then reduce to a single quadratic polynomial, no matter the value of n, and attempting to interpolate more than three points will have to give rise to a singular system). The orders of these errors correspond directly to which derivative of φ(r ) it is that features a jump. This leads to the “obvious” question: Why not 2 choose a φ(r ) that is infinitely differentiable everywhere, such as φ(r ) = 1 + r , φ(r ) = 2 e − r , φ(r ) = 1/(1 + r 2 ), etc.? This idea is an excellent one and can be applied to good advantage in any number of dimensions. If we still ignore boundary issues (possibly leading to some counterpart for RBF of the Runge phenomenon (RP) for polynomials), the accuracy will become spectral: better than any polynomial order, and generally of the form O(e −c/h ), where c > 0 and h is a “typical” distance between adjacent node points. Some precise statements and proofs in this regard have been given in [186, 301]. The monomial basis functions φ(r ) = |r |2m−1 have the property that φ(r ) = |r |2m−1 = ||2m−1 · |r |2m−1 ; i.e., scaling r will just lead to a corresponding scaling of φ, which in turn will scale the λi , with no resulting influence whatsoever on the interpolant s(x). Once we introduce smooth RBFs, this will no longer be the case. In exchange for spectral accuracy, we thus need to be concerned with the “shape parameter” . Numerous issues surrounding the choice of  (such as its influence on accuracy, on numerical conditioning, etc.) are discussed throughout this book.

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

Appendix B

Spherical Harmonics

Spherical harmonics (SPH) expansions can be seen as genealizations of 1-D Fourier expansions. Both types can in turn be viewed as special cases of Taylor expansions in which many of the terms have been omitted. This latter viewpoint can be very helpful in conveying an intuitive feeling for certain of the features of SPH, such as the ability of truncated SPH expansions to provide a completely uniform resolution over a sphere. The more traditional approach to SPH (via expansions in eigenfunctions of the Laplace operator) is, however, usually preferred for both numerical and analytical work.

B.1 Introduction of Fourier series via an eigenvalue problem Laplace’s equation remains invariant under a rotation of the (x, y)-coordinate system. In polar (r, θ)-coordinates, this equation takes the form r2

∂ u ∂ 2u ∂ 2u +r = 0. + ∂ r2 ∂r ∂ θ2

(B.1)

Since we are interested in the 1-D periodic domain θ ∈ [−π, π], we separate variables and focus on the θ-dependent part. This leads us to consider the following eigenvalue problem for the angular part of Laplace’s operator: ∂ 2u = λu. ∂ θ2 With periodicity assumed in θ, the eigenvalues become λ = −n 2 and the eigenfunctions {cos nθ, sin nθ}, n = 0, 1, 2, . . . . If we allow complex functions, we can write the eigenfunctions even briefer as e i nθ , n ∈ . General theory assures us that eigenfunctions obtained in this manner will form an orthogonal basis.

B.2 Introduction of Fourier expansion via 2-D Taylor expansions A periodic interval −π ≤ θ ≤ π can be graphically represented as the periphery of the unit circle, where θ is the polar coordinate angle. We start by considering smooth functions that are defined not just on the periphery but also throughout the interior of this 185

186

Appendix B. Spherical Harmonics

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

Table B.1. The five Platonic bodies. Type Tetrahedron Octahedron Cube Icosahedron Dodecahedron

Vertices 4 6 8 12 20

Faces 4 8 6 20 12

unit circle. The most natural functional representation is probably a 2-D Taylor expansion: a0 + {a1,1 x + a1,2 y} + {a2,1 x 2 + 2a2,2 xy + a2,3 y 2 } + . . . . The basis functions are in this case {1}, {x, y}, {x 2 , 2xy, y 2 }, . . . . Inside the circle, all these functions are independent. However, on the periphery (described by x 2 + y 2 = 1), this is not the case. Using this relation to eliminate one of the functions in the last bracket, we can, for example, choose as basis functions {1}, {x, y}, {x 2 − y 2 , 2xy}, . . . . On the periphery, we then have {1}, {x = cos θ, y = sin θ}, {x 2 − y 2 = cos 2θ, 2xy = sin 2θ}, . . . . Although arrived at in an unusual manner, this again produces the standard 1-D Fourier basis. To pursue this approach further, we need a rule for how to select which polynomials to keep of each order so that the resulting expansion, on the periphery of the unit circle, forms an orthogonal expansion. Leaving the details out, the answer for order k turns out to be to pick the real and imaginary parts of (x + i y)k (both parts becoming polynomials of degree k). Since they are the real and imaginary parts of analytic functions, they will satisfy Laplace’s equation u x x + uyy = 0. The following two properties are very familiar from trigonometric expansions but follow also directly from the procedure above: 1. The basis functions are everywhere smooth around the circle. 2. We obtain uniform resolution everywhere. This follows from the fact that rotation of a truncated Taylor expansion corresponds to a linear change in x and y. Such a change can alter existing coefficients but cannot introduce any new terms of higher degree.

B.3 Uniform point sets on the unit sphere It is not obvious that there will exist any functional representation that features a completely uniform resolution over the surface of the unit sphere since, in this case, it is no longer possible to place an arbitrary number of node points uniformly over the domain. The only such possibilities are given by the five Platonic bodies summarized in Table B.1. In particular, there is no possibility of placing more than 20 points uniformly over a sphere. However, as the following construction will show, this limitation is solely a problem of discrete gridding and not a problem of functional representation.

B.4 Introduction of SPH via an eigenvalue problem With this approach, the first step typically starts by introducing a spherical polar coordinate system: ⎧ ⎨ x = r sin θ cos ϕ, y = r sin θ sin ϕ, ⎩ z = r cos θ.

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

B.4. Introduction of SPH via an eigenvalue problem

187

Note that we here use co-latitude (measuring from the north pole) in order to follow conventions for spherical coordinates rather than latitude, as in other sections of this book. Laplace’s equation ∇2 u ≡ u x x + uyy + u z z = 0 now becomes r2

1 ∂ 2u ∂ u ∂ 2u ∂u ∂ 2u + + 2 + 2r − (cot θ) = 0. ∂ r2 ∂r ∂ θ2 ∂ θ sin θ ∂ ϕ 2

Separation of variables leads this time, in the two angular directions, to the eigenvalue problem (B.2) ∇2 u = λu, where ∇2 =

∂2 ∂ 1 ∂2 − (cot θ) . + ∂ θ2 ∂ θ sin2 θ ∂ ϕ 2

Separation of variables can now again be applied. Setting u(θ, ϕ) = P (z) · Φ(ϕ), where z = cos θ, and substituting into (B.2) leads to the pair of two 1-D eigenvalue problems Φ + m 2 Φ = 0, 6 d 2P dP m2 (1 − z 2 ) P = 0. − 2z − λ+ d z2 dz 1 − z2 5

(B.3) (B.4)

Equation (B.3) gives, just as before, trigonometric eigenfunctions

cos mϕ, m Φ (ϕ) = m = 0, 1, 2, . . . . sin mϕ, The eigenvalue problem (B.4) is more complicated. For every integer value of m ≥ 0, it is an eigenvalue problem in the z = cos θ direction. Requiring as boundary conditions that the solutions be bounded at the poles (z = ±1) turns out to give eigenvalues λ = −n(n+1) with eigenfunctions that are known as associated Legendre functions: Pnm (z) = (1 − z 2 ) m/2

dm P (z), n = m, m + 1, m + 2, . . . . d zm n

Here, Pn (z) denotes the Legendre polynomial of degree n. Some of the lowest-orderassociated Legendre functions are summarized in Table B.2. The customary notation ( Snm (z) = (2n + 1)(n − m)!/(4π(n + m)!)Pnm (z) ´1 m 1 gives −1 [Sn (z)]2 d z = 2π for m ≥ n ≥ 0. We furthermore simplify our notation by writing the basis functions in complex notation Ynm (θ, ϕ) = Snm (cos θ)e i mϕ . We then obtain the orthogonality relations ˆ πˆ π ˆ m l Ynm (θ, ϕ)Ykl (θ, ϕ) sin(θ)d ϕd θ Yn (θ, ϕ)Yk (θ, ϕ)d s = −π

0

=

1 0

if n = k and m = l , otherwise,

´ where the first integral ( . . . d s) is taken over the surface of the unit sphere.

Downloaded 11/04/15 to 132.239.1.230. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

188

Appendix B. Spherical Harmonics Table B.2. The polynomials (d m /d z m )Pn (z) for n = 0, 1, . . . , 4 and m = 0, 1, . . . , n. When multiplied with (1 − z 2 )m/2 , these form the associated Legendre functions Pnm (z).

n= 0 1 2

m=0 1 z 3 2 1 z −2 2

1

2

1 3z

3

3

5 3 3 z − 2z 2 35 4 15 3 z − 4 z2 + 8 8

15 2 3 z −2 2 15 35 3 z − 2z 2

4

3

15z 105 2 z 2

−

4

15 15 2

105z

105

Table B.3. Some low-order SPH basis functions, expressed in x-, y-, z-coordinates.

n= 0 1

m=0

1

1 2