Deep Learning for Fluid Simulation and Animation: Fundamentals, Modeling, and Case Studies 9783031423321, 9783031423338

Table of contents :
Preface
Contents
Acronyms
1 Introductory Material to Animation and Learning
1.1 Introduction
2 Fluids and Deep Learning: A Brief Review
2.1 Introduction
2.2 Fluid Modeling and Neural Networks
2.3 Fluid Rendering
3 Fluid Modeling Through Navier–Stokes Equations and Numerical Methods
3.1 Introduction
3.2 Navier–Stokes Equations
3.2.1 Conservation of Mass
3.2.2 Conservation of Linear Momentum
3.3 Smoothed Particle Hydrodynamics (SPH)
3.3.1 Standard Approach
3.3.1.1 Gaussian Function
3.3.1.2 Spline Function
3.3.2 Volume Maps
3.3.3 Divergence-Free Smoothed Particle Hydrodynamics
3.3.4 Versatile Rigid-Fluid Coupling for Incompressible SPH
3.3.4.1 Boundary-Fluid Pressure Force
3.3.4.2 Boundary-Fluid Friction Force
3.4 Stable Fluids
4 Neural Networks Universality, Geometric, and Dynamical System Elements for Fluid Animation
4.1 Introduction
4.2 Neural Networks and Universality
4.2.1 Boolean Expressions
4.2.2 Circuit Theory
4.2.3 Multilayer Perceptron Model
4.3 Activation Functions
4.4 Neural Networks as Universal Computing Model
4.5 Regular Surfaces and Differentiable Manifolds
4.6 Data Model and Learning
4.7 Hamiltonian Systems and Fluids
5 Data and Learning Theory
5.1 Introduction
5.2 Learning From Examples: Mathematical Perspective
5.3 Optimizing the Loss Function
5.4 Train, Validation, and Test Stages
6 Modeling Fluids Through Neural Networks
6.1 Introduction
6.2 Deep Architectures and Fluid Simulation
6.2.1 Convolutional Neural Networks: CNNs
6.2.2 Generative Adversarial Network: GAN
7 Fluid Rendering
7.1 Introduction
7.1.1 The Rendering Process in Fluid Simulation
7.2 Traditional Techniques
7.2.1 Surface Extraction
7.2.2 Screen-Space Approaches
7.2.3 Volume Rendering
7.3 Advanced Techniques
7.3.1 Optical Effects
7.3.2 Bubble, Spray, Foam, and Droplets
7.3.3 Optimizations
7.4 Machine Learning in Rendering
7.4.1 Neural Rendering
7.4.2 Differentiable Rendering
7.5 Machine Learning in Fluid Rendering
8 Case Studies
8.1 Introduction
8.2 Lagrangian Fluid Simulation with Continuous Convolutions
8.2.1 Comparison with SPlisHSPlasH Simulation
8.2.2 Rendering Liquids
8.3 TempoGAN: GAN for Super-resolution Fluid Flow
8.3.1 TempoGAN Results
9 Perspectives and Final Remarks
9.1 Introduction
9.2 Physics-Informed Neural Networks for Fluid Animation
9.3 Explainable Artificial Intelligence for Fluid Simulation
9.4 Machine Learning and Fluid Rendering
9.5 Final Remarks
References
Index

Citation preview

SpringerBriefs in Mathematics Gilson Antonio Giraldi Liliane Rodrigues de Almeida Antonio Lopes Apolinário Jr. Leandro Tavares da Silva

Deep Learning for Fluid Simulation and Animation Fundamentals, Modeling, and Case Studies

SpringerBriefs in Mathematics Series Editors Nicola Bellomo, Torino, Italy Michele Benzi, Pisa, Italy Palle Jorgensen, Iowa, USA Tatsien Li, Shanghai, China Roderick Melnik, Waterloo, Canada Otmar Scherzer, Linz, Austria Benjamin Steinberg, New York, USA Lothar Reichel, Kent, USA Yuri Tschinkel, New York, USA George Yin, Detroit, USA Ping Zhang, Kalamazoo, USA

SpringerBriefs present concise summaries of cutting-edge research and practical applications across a wide spectrum of fields. Featuring compact volumes of 50 to 125 pages, the series covers a range of content from professional to academic. Briefs are characterized by fast, global electronic dissemination, standard publishing contracts, standardized manuscript preparation and formatting guidelines, and expedited production schedules. Typical topics might include: • A timely report of state-of-the art techniques • A bridge between new research results, as published in journal articles, and a contextual literature review • A snapshot of a hot or emerging topic • An in-depth case study • A presentation of core concepts that students must understand in order to make independent contributions SpringerBriefs in Mathematics showcases expositions in all areas of mathematics and applied mathematics. Manuscripts presenting new results or a single new result in a classical field, new field, or an emerging topic, applications, or bridges between new results and already published works, are encouraged. The series is intended for mathematicians and applied mathematicians. All works are peer-reviewed to meet the highest standards of scientific literature. Titles from this series are indexed by Scopus, Web of Science, Mathematical Reviews, and zbMATH.

SBMAC SpringerBriefs

Editorial Board Celina M. H. de Figueiredo Federal University of Rio de Janeiro (UFRJ) The Alberto Luiz Coimbra Institute for Graduate Studies and Research in Engineering (COPPE) Program in Systems Engineering and Computing (PESC) Rio de Janeiro, Brazil Paulo J. S. Silva University of Campinas (UNICAMP) Institute of Mathematics, Statistics and Scientific Computing (IMECC) Department of Applied Mathematics Campinas, Brazil The SBMAC SpringerBriefs series publishes relevant contributions in the fields of applied and computational mathematics, mathematics, scientific computing, and related areas. Featuring compact volumes of 50 to 125 pages, the series covers a range of content from professional to academic. The Sociedade Brasileira de Matemática Aplicada e Computacional (Brazilian Society of Computational and Applied Mathematics, SBMAC) is a professional association focused on computational and industrial applied mathematics. The society is active in furthering the development of mathematics and its applications in scientific, technological, and industrial fields. The SBMAC has helped to develop the applications of mathematics in science, technology, and industry, to encourage the development and implementation of effective methods and mathematical techniques for the benefit of science and technology, and to promote the exchange of ideas and information between the diverse areas of application.

http://www.sbmac.org.br/

Gilson Antonio Giraldi • Liliane Rodrigues de Almeida • Antonio Lopes Apolinário Jr. • Leandro Tavares da Silva

Deep Learning for Fluid Simulation and Animation Fundamentals, Modeling, and Case Studies

Gilson Antonio Giraldi National Laboratory for Scientific Computing Petrópolis, Rio de Janeiro, Brazil

Liliane Rodrigues de Almeida National Laboratory for Scientific Computing Petrópolis, Rio de Janeiro, Brazil

Antonio Lopes Apolinário Jr. Institute of Computing Federal University of Bahia Salvador, Bahia, Brazil

Leandro Tavares da Silva Campus Petrópolis Federal Center for Technological Education - RJ Petrópolis, Rio de Janeiro, Brazil

ISSN 2191-8198 ISSN 2191-8201 (electronic) SpringerBriefs in Mathematics ISBN 978-3-031-42332-1 ISBN 978-3-031-42333-8 (eBook) https://doi.org/10.1007/978-3-031-42333-8 Mathematics Subject Classification: 35Q30, 68T07, 76-10 This work was supported by Conselho Nacional de Desenvolvimento Científico e Tecnológico (307769/2020-8). © The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 This work is subject to copyright. All rights are solely and exclusively licensed by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors, and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. This Springer imprint is published by the registered company Springer Nature Switzerland AG The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland Paper in this product is recyclable.

To our Families

Preface

Since 1990s, physically based approaches have become an important tool that enabled animators to control the behavior of the objects using laws of physics. Fluid animation can be viewed as part of this approach that can be implemented through a complex pipeline that includes fluid simulation and rendering. The former has been executed through techniques and models in continuum mechanics (Navier–Stokes equations) and numerical approaches. Once the fluid equations are numerically solved, we move on to the rendering phase that encompasses elements in transport theory in its traditional implementation. Recently, with the diffusion of deep learning techniques, new approaches have emerged using machine learning algorithms for modeling and animation of complex systems like fluids. This book serves as an introduction to the exciting area of machine learning approaches for fluid animation. Machine learning technology provides sophisticated tools that allow performing fluid simulation with a low computational cost if compared with traditional computational modeling techniques based on partial differential equations and numerical methods. On the other hand, deep neural networks for style transfer can be used to give the artists and animators in games and movies more control over the final appearance of the fluid. The book is intended to graduate and advanced undergraduate students in the field of fluid simulation, or computer science interested in deep learning and computer graphics. Moreover, researchers in traditional areas of mathematical modeling and computer graphics interested in neural networks for fluid simulation or animation may be part of the audience. However, we must offer the reader some background in computability theory to convince him (or her) about the power of neural computing. Hence, the material starts with fundamental points related to the universality of the computational model of neural networks. Once it is developed, it is possible to walk in the avenue of fluid modeling through modern neural network architectures. Aspects related to data preparation, data augmentation, and neural network training are discussed, aiming to clarify the importance of the quality and the representativeness of the training data. Regarding data generation for neural network training, we branch our presentation into particle-based and grid-based numerical methods to solve Navier–

vii

viii

Preface

Stokes equations. Moreover, another distinguishing aspect of our proposal is to discuss data-driven fluid simulation methods for computer graphics. Petrópolis, Brazil Petrópolis, Brazil Salvador, Brazil Petrópolis, Brazil June, 2023

Gilson Antonio Giraldi Liliane Rodrigues de Almeida Antonio Lopes Apolinário Jr. Leandro Tavares da Silva

Contents

1

Introductory Material to Animation and Learning . . . . . . . . . . . . . . . . . . . . . . 1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1 1

2

Fluids and Deep Learning: A Brief Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 Fluid Modeling and Neural Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3 Fluid Rendering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

5 5 6 9

3

Fluid Modeling Through Navier–Stokes Equations and Numerical Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 Navier–Stokes Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.1 Conservation of Mass. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.2 Conservation of Linear Momentum. . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3 Smoothed Particle Hydrodynamics (SPH) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.1 Standard Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.2 Volume Maps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.3 Divergence-Free Smoothed Particle Hydrodynamics . . . . . . . . 3.3.4 Versatile Rigid-Fluid Coupling for Incompressible SPH . . . . . 3.4 Stable Fluids. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

11 11 12 12 14 16 17 23 25 27 30

Neural Networks Universality, Geometric, and Dynamical System Elements for Fluid Animation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 Neural Networks and Universality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.1 Boolean Expressions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.2 Circuit Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.3 Multilayer Perceptron Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3 Activation Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4 Neural Networks as Universal Computing Model . . . . . . . . . . . . . . . . . . . . . 4.5 Regular Surfaces and Differentiable Manifolds . . . . . . . . . . . . . . . . . . . . . . . 4.6 Data Model and Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.7 Hamiltonian Systems and Fluids . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

33 33 34 34 36 39 42 43 46 47 50

4

ix

x

Contents

5

Data and Learning Theory. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2 Learning From Examples: Mathematical Perspective . . . . . . . . . . . . . . . . 5.3 Optimizing the Loss Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.4 Train, Validation, and Test Stages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

55 55 56 58 61

6

Modeling Fluids Through Neural Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.2 Deep Architectures and Fluid Simulation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.2.1 Convolutional Neural Networks: CNNs . . . . . . . . . . . . . . . . . . . . . . 6.2.2 Generative Adversarial Network: GAN . . . . . . . . . . . . . . . . . . . . . . .

65 65 68 68 75

7

Fluid Rendering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.1.1 The Rendering Process in Fluid Simulation . . . . . . . . . . . . . . . . . . . 7.2 Traditional Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.2.1 Surface Extraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.2.2 Screen-Space Approaches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.2.3 Volume Rendering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.3 Advanced Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.3.1 Optical Effects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.3.2 Bubble, Spray, Foam, and Droplets . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.3.3 Optimizations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.4 Machine Learning in Rendering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.4.1 Neural Rendering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.4.2 Differentiable Rendering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.5 Machine Learning in Fluid Rendering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

79 79 82 83 84 86 88 90 91 93 95 96 97 98 99

8

Case Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.2 Lagrangian Fluid Simulation with Continuous Convolutions. . . . . . . . . 8.2.1 Comparison with SPlisHSPlasH Simulation . . . . . . . . . . . . . . . . . . 8.2.2 Rendering Liquids . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.3 TempoGAN: GAN for Super-resolution Fluid Flow . . . . . . . . . . . . . . . . . . 8.3.1 TempoGAN Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

101 101 102 107 121 121 127

9

Perspectives and Final Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.2 Physics-Informed Neural Networks for Fluid Animation . . . . . . . . . . . . . 9.3 Explainable Artificial Intelligence for Fluid Simulation . . . . . . . . . . . . . . 9.4 Machine Learning and Fluid Rendering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.5 Final Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

137 137 138 141 143 144

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147 Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161

Acronyms

BRDF CConv CFD CG CNN ConvNet .Dte .Dtr .Dval DFSPH DMD FC FD FE FLIP GAN GPU LES LOD LSTM MC ML MLP MPM NeRF PDEs PDF PINN POD RANS ReLU

Bidirectional Reflectance Distribution Function Continuous Convolutions Computational Fluid Dynamics Computer Graphics Convolutional Neural Network Continuous Convolutions Neural Network Test Dataset Training Dataset Validation Dataset Divergence-Free Smoothed Particle Hydrodynamics Dynamic Mode Decomposition Fully Connected Neural Network Finite Difference Finite Element Fluid-Implicit-Particle Generative Adversarial Network Graphics Processing Units Large Eddy Simulation Level Of Detail Long Short Term Memory Neural Network Marching Cubes Machine Learning Multilayer Perceptron Material Point Method Neural Radiance Field Partial Differential Equations Probability Density Function Physics-Informed Neural Network Proper Orthogonal Decomposition Reynolds-Averaged Navier–Stokes Rectified Linear Unit xi

xii

RGB SHAP SPH SPlisHSPlasH SPNets TempoGAN VRE XAI XOR

Acronyms

Red-Green-Blue Shapley Additive Explanation Smoothed Particle Hydrodynamics Software for Smoothed Particle Hydrodynamics simulation Smooth Particle Networks GAN for Super-Resolution of Fluids Volume Rendering Equation Explainable Artificial Intelligence Exclusive or

Chapter 1

Introductory Material to Animation and Learning

Abstract In this chapter, we introduce concepts in computer animation, starting with physics-based animation. We revise the main steps of the pipeline involved: (1) geometric modeling of the scene; (2) the physical model describing the dynamics and interaction of objects therein; (3) computer simulation of the model; (4) rendering. Then, machine learning concepts regarding training and learning taxonomy (supervised, unsupervised, semi-supervised, and reinforcement learning) are discussed, aiming to prepare the way to introduce the main goal of this work: fluid animation based on deep learning techniques, which is motivated by the universality of the computational model of neural networks besides the possibility of generating surrogate models with the capability of producing predictions faster than traditional approaches, from the knowledge gained in the training process. Finally, the organization of the following sections is outlined. Keywords Physics-based animation · Geometric modeling · Physical model · Computer simulation · Rendering · Machine learning

1.1 Introduction Computer animation is the process of using computers to generate a sequence of images representing the evolution of some scene in time. The images are produced in such a frequency that, to human eyes, the sensation of a continuous movement is achieved. In the computational animation of scenarios involving fluids, the use of methods in computational fluid dynamics (CFD) has proved to be a valuable tool, always taking into account the fact that the final objective is the generation of visual effects and not the description of natural phenomena [103]. Techniques and models in continuum mechanics have been successfully applied for generating animations for entertainment [34, 72, 149, 193, 258] and for the video game industry [121, 181, 197, 218], showing how the areas of mathematics, computer graphics, and continuous mechanics can be integrated, generating multidisciplinary research topics, with technological applications.

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 G. A. Giraldi et al., Deep Learning for Fluid Simulation and Animation, SpringerBriefs in Mathematics, https://doi.org/10.1007/978-3-031-42333-8_1

1

2

1 Introductory Material to Animation and Learning

We are in the context of physics-based animation, a sub-area of computer graphics. Regardless of the application in focus, the ultimate goal in this area is to reproduce the dynamics of objects in a scene with the necessary realism from the viewpoint of the visual effects [103]. The complete pipeline involves: (1) geometric modeling of the scene; (2) the physical model describing the dynamics and interaction of objects therein; (3) computer simulation of the model; (4) rendering. Geometric modeling techniques may involve polygonal meshes to represent the scene, parametric surfaces and textures to represent the visible surface of objects, and regular or irregular 3D grids for domain discretization and numerical solution of differential equations [126]. Particle models may also be used where the objects and scene boundaries are represented through point clouds. The physical model is generally described by partial differential equations, which, in the case of classical mechanics, are derived from principles of momentum and/or energy conservation [102]. In general, these equations do not have an analytical solution, which motivates the use of numerical methods to simulate the model [102]. The discretization approaches applied in physics-based simulations range from Eulerian (grid-based), Lagrangian methodologies that represent the continuous medium as a particle system, and hybrid methods that combine characteristics of both Eulerian and Lagrangian categories [129]. In modern simulators, a continuous fluid model based on Navier–Stokes equations is mapped onto those structures generating linear systems that must be solved to approximate the solution. The data generated in those simulations should receive a graphical treatment via computer graphics methods that constitute the rendering phase [3]. Rendering is a computationally expensive and challenging process. It involves taking a complete scene 3D model as input, describing its objects, its light sources, and the camera from which one captures the scene, and processing all these data to produce a photorealistic image of this scene. The rendering process should analyze the light sources’ characteristics to produce the light rays illuminating the scene. Then, calculate how those light rays interact with the surface of each object. Considering only reflection interactions, the reflected rays can go directly to the camera or continue their path to interact with another scene object. When the reflected rays reach the camera, they produce the color information of one pixel of the digital image. The more sophisticated the model used to represent the light interactions in the scene, the more close to the photorealism of the image will be. Fluids bring extra challenges to this process. As their shape changes over time, the surface properties of fluids must be evaluated in each time step. Also, the material of fluids usually promotes more physical interactions with light than simple reflections. Light in liquids usually can refract and be transmitted by the medium. If the fluid is a gas, the light can interact with it as a participant media, which means that light interacts with the particles of the fluid along its volume. Nowadays, the idea of learning from data is present in almost every aspect of modern technology. In these applications, and everywhere in this text, learning theory means methodologies that enable computational systems to solve complex problems by adaptively improving their performance using data samples of the

1.1 Introduction

3

involved spaces. Learning processes are at the heart of the machine learning field, which contains neural networks as a subarea. These methods fulfill the everincreasing requirement for automated methods for data analysis and inference, which is a demand due to the enormous amounts of data in digital form. The machines learn through a process named training. Specifically, the machine has internal parameters, and training involves providing a learning algorithm and data to automatically adapt the machine parameters to solve the focused problem (classification, regression, etc.). Mathematically, the machine parameters are also degrees of freedom of an objective (Loss) function , which is iteratively optimized during the training process. In each iteration, the learning rule inside the learning algorithm updates the machine parameters toward a critical point of the objective function (point where the gradient vanishes). In this way, machine learning methods can be divided into four main types: supervised, unsupervised, semi-supervised, and reinforcement learning. In the supervised or predictive learning approaches, the data are composed of a set of labeled samples, and the goal is to learn a map from the sample space to the corresponding labels. In this text, the samples are represented by points in n .R . The set of pairs formed by the samples and corresponding labels is called the training set. The Loss function incorporates an error term to measure how close the machine response is to the target labels. The labels can be categorical or nominal variable that takes values from some finite set. In this case, the problem is known as classification or pattern recognition. Moreover, if the desired output is a real-valued scalar variable, then we are in face with a regression problem [173]. The second category of machine learning algorithms, the descriptive or unsupervised learning algorithms only receive data samples, and the aim is to perform knowledge discovery. This is an ill-defined problem since we do not know the target patterns. The construction of the Loss function and its error term are also more tricks because, unlike supervised learning, it is not given the desired prediction to be compared with the machine output. Semi-supervised learning, the third class, encompasses machine learning techniques that use labeled and unlabeled data for training [173]. Reinforcement learning [10], the last type of machine learning in our taxonomy, has as its essence to learn through interaction. It is easier to think with agents instead of algorithms. So, the agent interacts with its environment and learns how to alter its behavior in response to rewards received [10]. Recently, with the diffusion of deep learning techniques (neural networks with deep architectures), new approaches have emerged using machine learning algorithms for modeling complex systems [61, 74]. On the other hand, neural networks and physical models have been integrated, generating new methods whose precision can meet the needs of the animation and computational modeling of complex systems [117, 272]. These new models open up the prospect of taking advantage of the ability to learn from data of neural networks along with the rigor and mathematical properties of the differential equations [41]. An analogous scenario has been observed for rendering [288]. New research areas like Differential Rendering [119] and Neural Rendering [231] have emerged in

4

1 Introductory Material to Animation and Learning

the last few years. The main idea behind these new areas is to add learnable elements to the rendering process. The main challenge is to transform unstructured data into differentiable data used in the backpropagation phase of the learning process [93, 97, 259, 288]. Our proposal focuses on deep learning techniques for fluid animation, leaving the integration between neural networks and physical models as a perspective in the area. The fundamental objectives are to disseminate and promote the combination of methods in machine learning and computational modeling for fluid simulation, as well as appropriate methods for generating visual effects (rendering) from the simulation’s results. The application of neural networks models for fluid simulation and rendering is motivated by the universality of the computational model of neural networks [98] besides the possibility of generating surrogate models with the capability of producing predictions faster than traditional approaches from the knowledge gained in the training process [30]. In this sense, we offer a review in Chap. 2 oriented to the main tasks in fluid animation that have been addressed using deep neural networks. However, the generation of the input data for neural network training depends on simulating fluid dynamics using traditional methods to produce the training data. Hence, Chap. 3 presents the Navier–Stokes equations and numerical techniques based on smoothed particle hydrodynamics (SPH) [145] and finite difference schemes [220]. Following, the universality of neural networks as a computation model and some geometric and dynamical system elements are considered in Chap. 4. Next, Chap. 5 formalizes the concept of learning based on examples. Specific elements and architectures regarding neural networks for fluid simulation are presented in Chap. 6. Rendering approaches based on traditional and deep learning models are discussed in Chap. 7. Then, Chap. 8 describes two case studies using neural networks for Lagrangian simulation and super-resolution of fluids. The material is ended with perspectives in the area and final remarks in Chap. 9.

Chapter 2

Fluids and Deep Learning: A Brief Review

Abstract This chapter surveys deep learning models for fluid simulation and rendering. Regarding the former, we can group the works into small and large-scale fluid simulation, the interaction of the flow with other objects, and fluid control. Steered by this fact, we review works in turbulence, fluid solvers improvement, super-resolution, and generation of small-scale details for visual effects and video game applications. Also, applications of physics-based deep learning to produce fluid simulations with desirable behaviors for movies and the game industry are considered. Concerning rendering, we will highlight the most recent works applying machine learning techniques to generate photorealistic images. Keywords Deep learning · Fluid simulation · Rendering · Turbulence · Super-resolution · Visual effects · Photorealistic images

2.1 Introduction Machine learning technology provides sophisticated tools that allow performing fluid simulation with a low computational cost if compared with traditional computational modeling techniques based on partial differential equations and numerical methods [30]. Among other steps, the pipeline for fluid animation involves: (a) Simulation of fluid dynamics and the interaction of the fluid with objects in the scene; (b) Rendering. The methods found in the literature for fluid animation, based on computational fluid dynamics models, are based on Navier–Stokes equations, with finite difference discretization techniques [224, 229], grid-independent Lagrangian methods such as Smoothed Particle Hydrodynamics (SPH) [145] and hybrid approaches, such as fluid-implicit-particle (FLIP) [26]. Inspired by the rapid developments obtained with the application of deep learning methods in computer vision and natural language processing, the fluid mechanics community has realized the potential of data-driven methods that rely on big numerical data to yield surrogate models with the capability of producing predictions from the knowledge gained in the training process for both the simulation and rendering © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 G. A. Giraldi et al., Deep Learning for Fluid Simulation and Animation, SpringerBriefs in Mathematics, https://doi.org/10.1007/978-3-031-42333-8_2

5

6

2 Fluids and Deep Learning: A Brief Review

stages [30, 277]. In this line, we can group the fluid simulation works into smalland large-scale fluid simulation [38], interaction of the flow with other objects [211], and fluid control [171, 196]. Regarding fluid rendering, machine learning techniques are typically applied to enhance the visual quality of low-resolution simulations, adding fine visual details [46, 204]. CNNs can be used to learn how to transfer detailed information from high-resolution datasets to simulations with similar behavior or parameters but have lower resolution [12, 43, 210]. Hence, in the following, we survey related works steered by the main tasks of the categories addressed through deep learning models for fluid simulation/analysis and fluid rendering [29, 38].

2.2 Fluid Modeling and Neural Networks The study of turbulence is a fluid mechanics task that has received a great impact from deep learning because deep architectures can efficiently extract multiscale features [134]. In fact, defining meaningful features allows us to incorporate prior physical knowledge into turbulence models [287]. Applications of deep learning models for turbulence have focused on closure modeling [16], parameter estimation [79], high-fidelity modeling of three dimensional turbulence [162], super-resolution reconstruction [127], reduced order modeling [285], among others. Regarding the closure problem in turbulence, the velocity and pressure are decomposed into average and fluctuating parts, with the former being governed by the ReynoldsAveraged Navier–Stokes (RANS) equations. However, velocity fluctuations still occur in the RANS equations due to the nonlinearities in the Navier–Stokes equations. The closure problem refers to the challenge of obtaining governing equations involving only average velocity and pressure [268]. Such development has a fundamental influence in turbulence modeling, as we can check in [287], where deep neural networks are applied to study flows past airfoils. Moreover, turbulence models, like RANS, are formulated as transport equations that depend on parameters and coefficients that must be set in advance. Such tasks can be addressed by deep learning techniques. For instance, in [209], authors apply a multilayer perceptron (MLP) to build a deep neural network to represent a scale similarity model when addressing the problem of viscosity coefficient learning. High-fidelity three-dimensional computational fluid dynamics, like large eddy simulation (LES), have become an important resource for studying phenomena in turbulence. However, its computational cost remains prohibitive for transient flows and extremely large domains. These facts have motivated the application of deep learning models to design data-driven approaches to generate high-fidelity data in turbulence as observed in [162], where a combination of convolutional autoencoder and the convolutional long short-term memory (LSTM) neural network is applied to learn the attractor of the three-dimensional flow that is used to generate further steps of the simulation. Super-resolution approaches could be classified as high-fidelity

2.2 Fluid Modeling and Neural Networks

7

modeling in the sense that they learn to produce high-resolution data from lowresolution samples [73]. Specifically, the neural network is trained over a sequence of low-resolution simulations, with corresponding ground truth in high resolution, aiming to obtain a deep learning model that receives low-resolution fields as input and generates temporally coherent, high-resolution versions of them as output. In turbulence context, dimensionality reduction for reduced order modeling is an important tool. For instance, in [285], a hybrid approach is proposed that treats the underlying governing equations using standard proper orthogonal decomposition (POD) Galerkin to generate the model in the reduced space but computes the POD coefficients for the eddy viscosity using a MLP. Deep learning approaches can help researchers to surpass the limitation of linear projection techniques, like POD and dynamic mode decomposition (DMD). Also, reduced-order modeling has been used to accelerate simulations and underlying physics extraction out of the turbulence area [131]. One challenge when applying neural networks for fluid dynamics is the necessity of a huge amount of data for training. To address this issue, similarity theory and invariant properties related to symmetry groups of Navier–Stokes equations can be used for data augmentation in fluid dynamics [1, 278]. Moreover, we expect that machine learning models trained with those augmented data can predict results that respect such physical restrictions. Regarding invariance, in [144], a novel deep architecture is presented, which incorporates a multiplicative layer to embed Galilean invariance into the predicted results. In fact, the designing of machine learning models that encompass elements to incorporate physical constraints is another challenge in fluid simulation [163]. In this line, the generation of divergence-free velocities can be obtained by finite-difference schemes compatible with the back-propagation algorithm and by designing a convolutional neural network (CNN) model that incorporates hard physical constraints (like boundary conditions) in the neural network, as performed in [163]. In another line, deep neural networks have been applied to improve fluid solvers [251]. For instance, the papers [14, 221] describe approaches based on deep learning to compute spatial derivatives. The work [14] surpasses traditional finite-difference methods in low-resolution grids, while the paper [221] improves fifth-order finite-difference schemes for shock-capturing simulations. Also, to alleviate the computational load of the pressure projection to compute divergencefree velocity fields for incompressible fluid simulation, some works have applied data-driven approaches based on deep architectures, like in [263], which identifies multigrid methods as linear convolutional encoder–decoder networks. Active control has also been addressed using deep learning strategies. For instance, in [196], a novel methodology is applied where a deep MLP is trained using a reinforcement learning strategy to perform active flow control in a 2D simulation of two jets positioned on the sides of a cylinder that is immersed in a constant flow. Considering the developments of deep learning methods mentioned above, it is possible to generate animations augmented with useful attributes through models learned from data along the training stage [31]. In this context, new requirements

8

2 Fluids and Deep Learning: A Brief Review

such as motion control [45], artistic stylization [124], and small-scale details [241], demand intricate simulation methods that, in turn, need high-performance platforms to host the simulation framework [72, 241]. Hence, data-driven strategies are playing an important role in the animation of fluids. In videogames and movie industries, where visual realism is usually more important than physical behavior precision, data-driven approaches can predict high-resolution details for the fluid given a low-resolution simulation, bringing to the animation process the capability to produce high-quality images with less computational effort. Following [38], we can categorize data-driven approaches for fluid animation considering the model that the neural network is going to simulate. In this way, we have Eulerian, Lagrangian, and hybrid data-driven techniques. The Eulerian class encompasses data-driven approaches whose training data are generated by grid-based simulators. This is the case of the generative adversarial network (GAN) approach proposed in [278], for super-resolution of fluids (see Sect. 8.3). Other works can be found in the line of improving the resolution of precomputed fields, like [46] that takes a large database containing space-time fluid regions, simulated in low-resolution, and generate new high-resolution volumes. In this process, a CNN is used for the computation of descriptors that will be used for numerical frames retrieval in the database. Also, the pressure projection step in incompressible fluid simulation has been addressed through neural network methodologies to reduce the computational cost for fluid animation. For instance, the work [281] applies an MLP trained with local feature vectors composed of pressure, velocity gradient, and boundary conditions gathered at time step .n − 1 to compute the projected pressure at the time step n. The computation is performed cell by cell in order to advance the fluid over time. It is worthwhile to emphasize that the structural similarity between the Eulerian field and image motivates the application of CNN against MLP approaches for fluid simulation because, in this way, we can avoid ambiguities when converting grid-represented data into one-dimensional arrays, like performed in [281], to allow the application of MLP architectures. Moreover, in [264], a combination of autoencoders and LSTM allows to predict the temporal evolution of pressure fields given a sequence of previous states. In this case, a convolutional autoencoder is pretrained with grid-based simulated data. Then, the encoder–decoder is detached, and the LSTM is inserted between them. In this way, the LSTM processes sequences of time steps projected in the latent space of the encoder. The LSTM output is reconstructed in the simulation space through the decoder to generate the new time step of the simulation. Data-driven approaches in the Lagrangian class must deal with the processing of unstructured data (particle system), which is a challenging task for neural network architectures like MLP because the particle coordinates must be converted into an array before passing it to the network, a process that can be executed in too many ways. So, the network must operate invariant with respect to the permutation group. Fortunately, such a task can be implemented by surrogate models based on a CNN adaptation for particle system processing. They can be developed and trained with data generated by SPH [243]. In this way, we can simulate Lagrangian systems more efficiently for visual effects purposes (see Sect. 8.2). In this line, the approach [243]

2.3 Fluid Rendering

9

allows us to compute convolutions using input points different from the output ones, which is an important characteristic for fluid simulation that is not incorporated in traditional CNNs. On the other hand, the work [211] proposes the Smooth Particle Networks (SPNets), which use smoothing kernels to implement two new neural network layers, named ConvSP and ConvSDF. Owing to these developments, combined with standard deep learning layers, it is possible to implement algorithms that simulate fluids based on smoothing kernels and particle systems, like SPH. The authors explore the methodology to learn fluid parameters from data and propose a method to manage a liquid starting close to its target state. Hybrid data-driven techniques have focused on FLIP and material point method (MPM). Specifically, the data-based model described in [241] incorporates splash effects in low-resolution FLIP simulations. The approach learns to yield smallscale splash details using training data generated through high-resolution FLIP simulations. The whole methodology incorporates two MLP neural networks: one to classify small portions of fluid as splash or non-splash and another MLP to calculate the velocity modification to be applied to droplets classified as splashes to improve the visual appearance of the fluid. The MPM method was presented in [226] as a variation of FLIP and further adapted to simulate fluids in [283]. The MPM approach involves the projection of density, mass, and momentum from particles to the grid, the processing of grid nodes to calculate acceleration and the projection of this acceleration to particles. In [62], a neural network is applied to predict the internal acceleration of particles. The neural network is trained using MPM simulations. The implementation is restricted to 2D, and the input of the neural network for each grid node is composed of 45 entries that hold its density, gradients, and velocities for the nine grid node nearby. The output is the internal acceleration for each particle. Naturally, data-driven approaches can be used to learn how the physical parameters can be used to generate the kind of animation behavior the user wants. Physics-based deep learning (PBDL) [234] is an attempt to use physics-informed deep neural networks [41, 117] to produce complex simulations based on learning techniques, complementary to traditional physically based approaches. Extending this approach makes it possible to learn new parameters/constraints that can produce desirable behaviors [38, 45, 171] like those needed in the movie and game industries.

2.3 Fluid Rendering Rendering is a computationally complex process that takes as input the description of the shape of all objects in a scene and, based on the characteristics of the light sources and position and parameters of the virtual camera, calculates the light interactions that reach the observer. The number of light interactions can be significant. Research in rendering is always in the cut of the edge between improving models that compute the light phenomena [190] and reducing the computational cost of the algorithms [4].

10

2 Fluids and Deep Learning: A Brief Review

Research on machine learning for Computer Graphics has had a boom in the last few years. As a new field of research, many techniques must be mature enough to be considered the final solution for modeling and rendering [277]. From the rendering perspective, the representation of the objects is an important issue. Machine learning techniques for extracting the geometric representation of models from point clouds [93, 256], polygonal meshes [97, 200], and volumetric data [146] have been studied in the last years. A common difficulty in geometric processing using machine learning is the lack of a natural way to differentiate the data, as they are not necessarily regular (like point clouds and meshes) and how loss functions can be back-propagated. Nowadays, a popular approach to reconstructing geometric information is the Neural Radiance Field – NeRF [159]. Based on a database of images from the scene, a volumetric radiance field is constructed and can be rendered using Differentiable Rendering techniques [259]. Rendering fluids brings additional challenges. The fluid’s shape evolves along the simulation, and its surface depends on evaluating some field [172]. Besides, its surface could have complex structures, such as splashes and liquid drops, which change the topological properties of the surface during the simulation [56]. The light interactions with fluids are more sophisticated than with solid objects. When the fluid is gas, it must be treated as participant media, where light interacts with the particles of the fluid, producing scattering effects [183]. In this scenario, rendering researchers have been investigating how data-driven approaches, especially machine learning techniques, could help reduce the complexity of rendering fluids. A straightforward way to reduce computational complexity is to reduce the resolution of the rendering image. Using high-resolution simulation datasets, the fluid’s visual characteristics can be learned and reintroduced in a postprocessing step to improve the low-resolution image [204]. Regarding the rendering of smoke simulation, an exciting way to produce more artistic effects is the technique called Style Transfer in the image processing literature. Originally proposed as a stylization to mimic artistic styles in photographs [78], Neural Style Transfer can be used to transfer high details to smoke simulations [210]. The same technique can change the color [12] or a structural pattern [122] that can add specific details in the fluid rendering. GANs can also be used to enhance details on smoke simulations [124].

Chapter 3

Fluid Modeling Through Navier–Stokes Equations and Numerical Methods

Abstract Fluid dynamics is a field of physics that focuses on the study of the behavior of liquids and gases in motion. It explores the laws governing fluid movement and the forces they exert on objects they come into contact with. The motion of fluids is described by mathematical equations that consider factors such as velocity, pressure, and applied forces. These equations are based on the principles of mass and momentum conservation. This chapter introduces the Smoothed Particle Hydrodynamics (SPH) method and some variations of the traditional approach for simulating fluids. SPH is a Lagrangian method based on particles.

3.1 Introduction Fluid dynamics is the study of the behavior of fluids (liquids and gases) in motion. It is a branch of physics that deals with the laws that govern the movement of fluids and the forces that fluids exert on objects in contact with them. In fluid dynamics, the motion of fluids is described by mathematical equations that take into account various factors, such as the velocity of the fluid, its pressure, and the forces acting on it. The equations used to describe fluid flow are based on the principles of mass and momentum conservation. The fundamental setting behind the fluid theory is the microscale, which takes into consideration the motion of individual fluid molecules, and the system behavior is ultimately ruled by the kinetic theory and Boltzmann equation [55]. Traditional fluid dynamics starts to exist as soon as we can make the approximation of considering the system of interacting particles as a continuum. That is the macroscale, defined roughly as the scale of classical mechanics. In the context of fluid models, the bridge between the microscale and macroscale is given by the lattice Boltzmann models, which consider distribution functions (related to particle density) and a simplification of the Boltzmann equation to compute the time evolution of the system [161]. We focus on macroscale models for fluid description, which can be derived through mass and momentum conservation laws. The local differential forms of these conservation laws give the continuity and Navier–Stokes equations, respec© The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 G. A. Giraldi et al., Deep Learning for Fluid Simulation and Animation, SpringerBriefs in Mathematics, https://doi.org/10.1007/978-3-031-42333-8_3

11

12

3 Fluid Modeling Through Navier–Stokes Equations and Numerical Methods

tively. These are partial differential equations (PDEs) for which no analytical solutions exist in general. Computational fluid dynamics (CFD), on the other hand, combines continuum and discrete theories for fluid modeling with computational algorithms for fluid simulation. It is an important research area since a wide range of natural phenomena can be modeled through fluid theory. Some common engineering examples are pumps, fans, turbines, airplanes, ships, rivers, windmills, pipes, and more recently, the hemodynamics of the arterial system [102, 240]. Therefore, approximate numerical solutions will be sought through some discretization approach. The majority of fluid simulation methods use 2D/3D meshbased discretization frameworks that are mathematically motivated by the Eulerian methods of Finite Element (FE) and Finite Difference (FD), in conjunction with Navier–Stokes equations of fluids [102]. More recently, mesh-free methods like Smoothed Particle Hydrodynamics (SPH) [145], Moving Particle [129], and Moving-Particle Semi-Implicit [11] have been applied. Among these techniques, the SPH has the advantage of being a general one due to its capabilities to simulate a wide range of natural phenomena [166].

3.2 Navier–Stokes Equations The Navier–Stokes equations are a set of partial differential equations that describe the motion of fluid, such as liquids and gases. They were named after French physicist Claude-Louis Navier and British physicist George Gabriel Stokes, who independently derived the equations in the nineteenth century. The Navier–Stokes equations have two main parts: the conservation of mass and the conservation of momentum. The conservation of mass requires that the amount of fluid in any given volume does not change over time. The conservation of momentum requires that the net force on any small volume of fluid is equal to the change in momentum of the fluid in that volume. We have two ways to describe a fluid: using the Eulerian or Lagrangian formulation. In Eulerian formulation, flow quantities are specified as functions of position and time. In Lagrangian formulation, the idea is to follow the motion of each fluid particle.

3.2.1 Conservation of Mass The law of conservation of mass is a fundamental principle in physics and chemistry that states that the mass of a closed system must remain constant over time as long as no mass is added to or removed from the system. Now, let an arbitrary region in a continuous medium .B be denoted by W , and the bounding closed surface of this region be continuous and denoted by .∂W . Let .x ∈ B.

3.2 Navier–Stokes Equations

13

Relative to standard Euclidean coordinates in space, we assign to each particle of the fluid the coordinates x, y, and z. Therefore, we write .x = (x, y, z) and let .v(x, t) denote the velocity of the particle of fluid that movies through .x at time t [42]. Let .ρ(x, t) denote the mass density, thus the mass of fluid in W at time t is given by { m(W, t) =

ρ(x, t)dV ,

.

(3.1)

W

where dV is the volume element in the plane or in space. Let’s assume that .v and .ρ are smooth. Then the rate of change of mass in W is .

d d m(W, t) = dt dt

{ ρ(x, t)dV .

(3.2)

W

We can show that the time derivative of the volume integral over .ρ(x, t) is given by d . dt

{

{ ρ(x, t)dV = − W

ρv · ndA,

(3.3)

∂W

where .n denote the unit outward normal defined at points of .∂W and dA denote the area element on .∂W . By converting the surface integral to a volume by means of the divergence theorem and since { { ∂ρ d (x, t)dV , . ρ(x, t)dV = dt W W ∂t we obtain { [ .

W

] ∂ρ + div(ρu) dV = 0 ∂t

(3.4)

As true for every W , we obtain the following local form: .

∂ρ + div(ρu) = 0. ∂t

(3.5)

The equation above is known as the continuity equation. It is the differential form of the law of conservation of mass. Knowing that: .

div(ρv) = ∇ · (ρv) = ρ∇ · v + v · ∇ρ,

and defining the operator .D/Dt = ∂/∂t + v · ∇, called material derivative, we can rewrite the equation as:

14

3 Fluid Modeling Through Navier–Stokes Equations and Numerical Methods

.

Dρ + ρ∇ · v = 0 Dt

(3.6)

The equation (3.5) is the Eulerian formulation, and the equation (3.6) is the Lagrangian formulation of the continuity equation.

3.2.2 Conservation of Linear Momentum The law of conservation of momentum, also known as the balance of momentum, is used to describe the motion of fluids, such as gases and liquids. The principle states that the total momentum of a fluid in a control volume remains constant unless acted upon by external forces. To understand the balance of momentum in fluid dynamics, it is important to consider the different types of forces that can act on a fluid. These include pressure forces, viscous forces, and gravitational forces. As previously stated, we have two ways to describe a fluid: the Eulerian formulation and the Lagrangian formulation. Using the Lagrangian formulation, we assign coordinates x, y, and z to each particle of the fluid and specify each of these coordinates as a function of time. Thus, we describe the fluid’s motion through the functions .x(x0 , y0 , z0 , t), .y(x0 , y0 , z0 , t), and .z(x0 , y0 , z0 , t), which determine the coordinates x, y, and z at time t of the fluid particle that, at time .t0 , was at .x0 , .y0 , and .z0 [227]. Thus, using the Lagrangian formulation, let us consider .x(t) = (x(t), y(t), z(t)) as the path followed by a particle of the fluid whose velocity field along the path is given by: v(t) = (x(t), ˙ y(t), ˙ z˙ (t)).

.

(3.7)

Thus, using the fact that in the Eulerian formulation, the fluid velocity field is a function of spatial coordinates and time, that is, .v = v(x, t), we can write: v(x(t), t) =

.

dx (t). dt

(3.8)

That is, the equation above gives us a connection between the Eulerian (left-hand side) and Lagrangian (right-hand side) formulations. On the other hand, the acceleration of the fluid particle is given by: a(t) =

.

by the chain rule:

d 2x d (t) = v(x(t), y(t), z(t), t), 2 dt dt

(3.9)

3.2 Navier–Stokes Equations

15

a(t) =

.

∂v ∂v ∂v ∂v x˙ + y˙ + z˙ + . ∂x ∂y ∂z ∂t

(3.10)

Let .v(x, y, z, t) = (u(x, y, z, t), v(x, y, z, t), w(x, y, z, t)), and using the notation: ∂v ∂v ∂v ∂v , vy = , vz = , vt = , ∂x ∂y ∂z ∂t

vx =

.

(3.11)

we obtain: a(t) = uvx + vvy + wvz + vt ,

(3.12)

.

which we can write as: a(t) = vt + v · ∇v ≡

.

Dv . Dt

(3.13)

A fluid is ideal when it is not subject to shear stress [42], even when in motion. In this case, if W is a region of the fluid with boundary .∂W , then there exists a function .p(x, t) called pressure, such that if .n is the exterior normal to .∂W , then the force of stress exerted along the surface per unit area at .x ∈ S at time t is .p(x, t). Thus, if W is a region of the fluid at a particular time t, the force exerted on the ideal fluid inside W , generated by the force of stress on .∂W , is: { S∂W = −

(3.14)

pndA,

.

∂W

where the negative sign appears because .n points outward from W . If .e is any constant vector in space, then applying the divergence theorem, we have: {

{

e · S∂W = −

{

pe · ndA = −

.

∂W

div(pe)dV = − W

∇p · edV .

(3.15)

W

Therefore, { S∂W = −

∇pdV .

.

(3.16)

W

If .b(x, t) denotes the external force applied on a body per unit of mass, then the total force is given by: { B=

ρbdV .

.

W

Thus, in any part of the fluid,

(3.17)

16

3 Fluid Modeling Through Navier–Stokes Equations and Numerical Methods

{force per unit volume} = −∇p + ρb.

.

(3.18)

By the second law of Newton, and equations (3.13) and (3.18), we obtain the conservation of momentum law: ρ

.

Dv = −∇p + ρb. Dt

(3.19)

In the simulation of real fluids, the effect of viscosity must be taken into account. Thus, the linear momentum conservation law, analogous to equation (3.19), for viscous fluids is given by: ρ

.

Dv = −∇p + ρb + ∇ · τ , Dt

(3.20)

where .τ represents the tensor of viscous stresses [94]. In our work, we will only use Newtonian fluids, and therefore, in this case, the force of viscous stress is proportional to the rate of deformation [94], which can be represented as: ) 2 ( τ = μ ∇v + ∇vT − μ(∇ · v)I. 3

.

(3.21)

By introducing the above equation into (3.20), we obtain: ρ

.

[ ( ] ) 2 Dv = −∇p + ρb + ∇ · μ ∇v + ∇vT − μ(∇ · v)I . Dt 3

(3.22)

The equation (3.22) is called the Navier–Stokes equation. For incompressible fluids, that is, fluids with .∇ · v = 0, and with constant viscosity, equation (3.22) reduces to: ρ

.

Dv = −∇p + ρb + μ∇ 2 v. Dt

(3.23)

For the equation in the Eulerian formulation, considering incompressible fluids, Dv simply replace . as in the equation (3.13) to obtain Dt ρ

.

∂v + ρv · ∇v = −∇p + ρb + μ∇ 2 v. ∂t

(3.24)

3.3 Smoothed Particle Hydrodynamics (SPH) The SPH method was originally invented to solve astrophysical problems in threedimensional open space [81, 164]. It is a mesh-free Lagrangian method that

3.3 Smoothed Particle Hydrodynamics (SPH)

17

represents the fluid as a particle system and computes fluid flow fields using interpolation functions named kernels. Basically, field quantities are only defined at the particle locations, but they can be evaluated anywhere in space using radial symmetrical smoothing kernels. Numerical and error analysis were performed in order to construct analytical kernels to obtain n-th order approximation schemes [145]. Also, variational formulations of SPH have been presented to provide useful frameworks to address conservation properties, like linear and angular momentum preservation [24, 52]. Since its invention, SPH has been extensively studied and extended to address scientific and engineering problems in material science, free surface flows, explosion phenomena, heat transfer, mass flow, and many other applications [166].

3.3.1 Standard Approach The starting point for studying SPH is the representation of a function as an integral. Let .O ⊂ Rn be the domain of the problem, and let .f : O → Rm be any function defined on .O. The integral representation of the function f is defined as the convolution of the function f with a smooth function .W : Rn → R { :=

f (x)W (u − x, h)dx.

.

(3.25)

O

The function W is called the interpolation kernel, and the parameter h is used to define the support of W . The definition above is motivated by the Dirac delta, which can be seen as a distribution .δ(x) such that δ(x) = 0, if x /= 0,

.

(3.26)

and satisfies: { .

R

δ(x)dx = 1.

(3.27)

Several functions can be used as interpolation kernels, but in the literature, the most common ones are the normalized Gaussian function and spline functions [167]. Below we will present some examples of kernel functions. To do this, we define the relative distance between two particles as: R=

.

|x − x' | . h

(3.28)

18

3 Fluid Modeling Through Navier–Stokes Equations and Numerical Methods

3.3.1.1

Gaussian Function

The Gaussian function below was initially used by [167] to simulate problems with non-spherical stars: W (R) = αd exp(−R 2 ),

.

(3.29)

where R is defined in Equation (3.28). The constant .αd is a normalization constant, where d is the dimension of the problem. For instance, if we consider problems of 15 dimension 2 then we use .α2 = [145]. 7π h2 3.3.1.2

Spline Function

The degree-five spline, introduced by [170], approximates the Gaussian function with the advantage of being more numerically stable [186]. ⎧ (3 − R)5 − 6(2 − R)5 − 15(1 − R)5 , 0 ≤ R < 1, ⎪ ⎪ ⎨ 1 ≤ R < 2, (3 − R)5 − 6(2 − R)5 , .W (R) = αd 5, ⎪ 2 ≤ R < 3, (3 − R) ⎪ ⎩ 0, R ≥ 3,

(3.30)

where R is defined in Equation (3.28). The constant .αd is given by:

αd =

.

⎧ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎨

120 , h

d = 1,

7 , d = 2, ⎪ 478π h2 ⎪ ⎪ ⎪ ⎪ ⎪ 3 ⎪ ⎩ , d = 3. 359π h3

(3.31)

The two fundamental elements in the SPH method are an interpolation kernel W : R3 → R+ , which is a symmetric function with respect to the origin .(0, 0, 0), limited, with compact support, and a particle system .xi = (xi1 , xi2 , xi3 ) ∈ R3 , .i = 1, 2, · · · , M, which represents a discrete version (sampling) of the fluid. The kernel estimate of a scalar quantity A and its gradient in a point .xi ∈ R3 are given by [145]: .

.

=

M E mj A(xj )W (xi − xj , h), ρ(xj ) j =1

(3.32)

3.3 Smoothed Particle Hydrodynamics (SPH)

.

=

M E mj A(xj )∇i W (xi − xj , h), ρ(xj )

19

(3.33)

j =1

where .∇i W (xi − xj , h) means .∇x W (x − xj , h) evaluated at .x = x(i , h) is the smoothing length which determines the support of the kernel and .ρ xj is the density at the particle position .xj [145]. Therefore, the kernel estimate of the density at the position .xi is:

.

=

M E

mj W (xi − xj , h),

(3.34)

j =1

Besides, we can show that the divergent for a vector field .v can be computed as [145]:

.

=

M E ) mj ( ( ) v xj − v (xi ) ∇i W (xi − xj , h). ρ(xj )

(3.35)

j =1

An analogous expression can be obtained for the Laplacian operator. From equations (3.33) and (3.35), we can observe that there is no need for a mesh to compute spatial derivatives. With equations (3.32), (3.33), (3.34), and (3.35), we are ready to write the discrete version of the fluid equations of Sect. 3.2. For simplicity, we will remove the brackets and place a subscript i on the denoted value; that is: . ≡ Ai and . ≡ ∇Ai . We also use the notations .Wij ≡ W (xi − xj , h) and .ρ(xj ) ≡ ρj . By using the material derivative for the density (.dρ/dt = ∂ρ/∂t + ∇ρ · v) as well as expressions (3.33), (3.34), and (3.35) we can show that the SPH version of the time derivative of the density becomes: E dρi mj vij ∇i Wij , = dt M

.

(3.36)

j =1

where .vij ≡ vi − vj . The mass density of a particle i can be directly approximated, as in equation (3.34). Another way to approximate the specific mass is to use the continuity equation (3.6), which we rewrite here: .

Dρ = −ρ∇ · v. Dt

(3.37)

Thus, we use the expression (3.35) to compute the divergence on the right-hand side of the above equation, obtaining:

20

3 Fluid Modeling Through Navier–Stokes Equations and Numerical Methods

.

E mj Dρi = vij · ∇i Wij , Dt ρj

(3.38)

j ∈Vi

where .vij = vi − vj . Next, we use a time integrator to calculate the specific mass: ρik+At = ρik + At

.

Dρik , Dt

(3.39)

where .ρik is the specific mass of particle i at simulation time .tk , .At is the simulation Dρ k time step, and . i is given by Equation (3.38). Dt In SPH, fluid pressure is calculated through an equation of state, where the pressure is given as an explicit function of the fluid’s density. In the case of a free surface, the equation of state suggested by Batchelor [165] is used, given below: [( pi = B

.

ρi ρ0

)γ

] −1 ,

(3.40)

with .γ = 7, where .ρ0 is the reference density. The parameter B is given by .B = c2 ρ0 , where c is the velocity of sound in the medium. γ Another possibility to model incompressible fluids, using a quasi-incompressible equation of state, is to utilize the equation introduced in [169], given by: p = c2 ρ,

(3.41)

.

where c is the velocity of sound in the medium. Now, we shall obtain the discrete version of the Navier–Stokes equations given by expression (3.23). Firstly, we will consider each term separately before to put all together. The pressure gradient could be directly computed using equation (3.33), but the obtained expression does not conserve linear momentum [169]. To address this we shall rewrite the pressure gradient as: .(∇p/ρ) = ∇ (p/ρ) + ) ( problem, p/ρ 2 ∇ρ. Then, we can use the equation (3.33) to calculate the gradients to obtain the force that particle i feels due to the pressure only [169]: E Dvi mj . =− Dt j ∈Vi

(

pj + ( )2 2 (ρi ) ρj pi

) ∇i Wij +

xij · ∇i Wij 2μ E mj vij ( )2 + F, ρi ρj rij j ∈Vi

(3.42) where .F is an external force, .vij = vi − vj , .xij = xi − xj , .rij = ||xij ||, with .xi and xj being points in Euclidean space, which define the positions of particles i and j , respectively. Due to numerical stability issues, we follow [145] and use the artificial viscosity computed by:

.

3.3 Smoothed Particle Hydrodynamics (SPH)

21

⎧ 2 ⎪ ⎨ −2(aϕij c + bϕij ) , vij · xij < 0 .||ij = ρi + ρj ⎪ ⎩ 0, vij · xij ≥ 0

(3.43)

with: (vij · xij )h ϕij = ( )2 , rij + 0.01h2

(3.44)

.

where a, b are parameters, .vij = vi − vj , .xij = xi − xj , .rij = ||xij ||. Besides, we shall include repulsive boundary forces to prevent the interior particle from penetrating the boundary of the domain. This can be implemented using boundary particles that do not move but interact with fluid particles [145]. Specifically, if a boundary particle .xg is the neighboring particle of a real particle .xi that is approaching the boundary, then the force:

Sig =

.

⎧ [( )n1 ( )n2 ] r0 r0 ⎪ ⎪ ⎪ ζ , − ⎪ ⎨ rig rig

if

r0 ≤ 1, rig

if

r0 > 1, rig

(3.45)

⎪ ⎪ ⎪ ⎪ ⎩

0

,

is applied pairwise along the centerline of these two particles, where .n1 = 12, n2 = 4, .rig = ||xi − xg ||, and .r0 is usually selected close to the initial particle spacing. The parameter .ζ is problem dependent, and its value should be chosen with the same order as the square of the largest velocity. Moreover, external forces can be added to the system like the gravitational one given by:

.

Fi = mi g,

.

(3.46)

where .g the gravitational field. Once the force terms are computed, the acceleration of particle i can be obtained by: ai = −

M E

.

j =1

( mj

pj

pi + 2 ρj2 ρi

) ∇ i Wij +

) 1 ( ||ij + Sig + Fi . ρi

(3.47)

Thus, an integration scheme, the Leap-Frog one, is used in order to update the velocity and the position of particles as follows : vt+At = vti + i

.

At t a, 2 i

xt+At = xti + Atvt+At . i i

.

(3.48) (3.49)

22

3 Fluid Modeling Through Navier–Stokes Equations and Numerical Methods

In the simulation pipeline, the SPH particles are processed as follows: (a) Compute the pressure field using expression (3.40) and the density at previous time; (b) Update the density field by integrating equation (3.36); (c) Compute the accelerations by expression (3.47); (d) Update the velocity and the position of SPH particles through equations(3.48)–(3.49), respectively. The SPH equations require the computation of sums over the set of particles. However, due to the compact support of the kernel in expression (3.30), it is clear that we need to search only neighboring particles, as we can see in Fig. 3.1. This process can be accelerated by using spatial data structures that should be properly generated and queried. To perform this task, the space is subdivided into cubic cells, each one having a size equal to the kernel support. At the initialization, each particle is associated with one cell, and after the SPH iteration, we should update this structure. Consequently, given a particle position, we find the corresponding cell, and then only nearby cells should be visited in order to find all the neighboring particles that we need to compute the SPH expressions above. For simulations of fluids with free surfaces, there are several ways to treat this surface, one of which is the use of an appropriate surface tension force for fluids with a single phase. This force has a well-defined physical meaning [35]. Additionally, in the case of SPH, fluid particles are subject to forces resulting from interaction with neighboring particles. Within the fluid, these interparticle forces are added together to generate the fluid motion. However, the forces acting on particles at the free surface can produce a resultant force in the direction of the exterior normal to the fluid surface, which is a known issue in the SPH literature [143]. An example to address this problem is the method proposed in [17], which is based on cohesive forces, which are attractive forces between molecules of a liquid. In this formulation, surface tension is given by:

.

κ E dva =− mj Wij xij , mi dt j ∈Vi

Fig. 3.1 Highlighted are the particles in the center of the circle with radius kh, their neighbors, and the cells where the search for their neighbors is performed (From [50])

(3.50)

3.3 Smoothed Particle Hydrodynamics (SPH)

23

where .xij = xi − xj , .κ is a surface tension constant. One advantage of this approach compared to others in the literature [172], for incompressible fluids, is the possibility of applying this force to all particles and not just those located on the free surface of the fluid, facilitating computational implementation. This is possible since this force tends to cancel out within the fluid due to the distribution of particles.

3.3.2 Volume Maps Over the past years, several research studies on the SPH method have been conducted, creating variants of the traditional method presented in the previous section. In this section, we will summarize the SPH approach introduced by the paper [21], which is used in the SPLisHSPLash library [18]. This method is inspired by the density maps approach and is also based on an implicit boundary representation. It uses the same data structures as described in [20] to store and query function values. However, instead of precomputing the density contribution of the boundary, it determines the boundary volume that overlaps with the support domain of the smoothing kernel [21]. The proposed method initially distinguishes between the fluid domain and the boundary domain. Thus, the first step is to write the density as { ρ≈ .

=

{N(x)

ρ(x' )Wij dx '

{N(x)∩F

ρ(x' )Wij dx ' + '

ρ(x )Wij dx N(x)∩B = ρF (x) + ρB (x),

(3.51)

'

where .F is the fluid domain, .B is the boundary domain and .N(x) is the support domain. Particle-based methods discretize the geometric boundary with particles, thus the portion of the boundary .ρB can be discretized similarly as a fluid portion using the sum: E E .ρi = ρ (xi ) + ρ (xi ) ≈ m W + m ˜ k W˜ ik , (3.52) j ij B F j

k

where j and k denote the indices of particle i’s fluid and boundary particle neighbors, respectively. The values of .m ˜ and the kernel function .W˜ are chosen ˜ is treated as a pseudo depending on the approach. Often, we set .W˜ = W and .m mass; that is, its value is independent of the boundary object’s material properties. The function .ρB is discretized and precomputed on a spatial grid to solve the computational cost problem, and the discontinuities are removed by extending the boundary density outside the boundary.

24

3 Fluid Modeling Through Navier–Stokes Equations and Numerical Methods

Unlike particle-based approaches, the density maps concept is based on an implicit representation of the boundary obtained by constructing a distance function 3 .O : R → R, given by: { O(x) =

.

−d(x, ∂B) if x ∈ B d(x, ∂B) otherwise.

(3.53)

The density field is extended inside the boundary using the extension function .γ : R → R+ : { ( x) ρ0 1 − if x < r .γ (x) = (3.54) r 0 otherwise. Therefore, we are able to compile these equations with the SPH formalism and write the density in the boundary domain as: { ρB (x) =

.

N(x)

γ (O(x' ))Wij dx ' .

(3.55)

The main contribution of [21] is the new volume maps approach, which was inspired by the concept of density maps and also by the implicit boundary representation technique, with some of the advantages of this method being improved computational performance and avoidance of discontinuities. To do so, the intersection volume of the sphere around the fluid particle, which is defined by the support radius h, is calculated. The most important advantage is that the boundary volume can be directly used in the general SPH formulation, which enables a consistent computation of the density gradient. Volume maps only determine the intersection volume between the kernel support domain and boundaries and use the SPH formulation to compute the density gradient. In contrast, density maps determine the gradient by differentiating the cubic shape functions of Serendipity type of their spatial grid discretization [21]. To generate a volume map, the boundary volume was determined as { VB (x) =

.

N(x)

γ ∗ (O(x' ))dx,

(3.56)

where .O is the signed distance function and .γ ∗ is given by: ⎧ C(x) ⎪ ⎪ if 0 < x < r, ⎨ C(0) ∗ .γ (x) = 1 if x ≤ 0, ⎪ ⎪ ⎩ 0 otherwise,

(3.57)

3.3 Smoothed Particle Hydrodynamics (SPH)

25

where C denotes the cubic spline kernel. To compute the density at each fluid particle, the traditional SPH formulation was used in Equation (3.34), and to determine the density at the boundary .ρB , volume maps were utilized: ρB (x) = VB (x)ρ0 W (||x − x∗ ||),

.

(3.58)

where .x∗ is the closest point to .x on the boundary; this point can be determined by .O. The symmetric pressure force for each particle i is calculated as FiP .

) pj pi + 2 ∇Wij = −mi j mj ρi2 ρj ( ) pi p˜ −mi Vj ρ0 + 2 ∇W (||x − x∗ ||) ρ˜ ρi2 = FP

i←−F

where .F P

(

E

(3.59)

+ FP

i←−B

is part of the pressure force that is exerted by the neighboring fluid

i←−F particles, .F P i←−B

is the part that is exerted by the boundary, .p˜ and .ρ˜ are the pressure and the density at the boundary, respectively.

3.3.3 Divergence-Free Smoothed Particle Hydrodynamics In this section, we will briefly summarize the paper [19], which is one of the main references for SPLisHSPLash library [18]. This paper introduces a stable and efficient implicit method for SPH to simulate incompressible fluids, which utilizes the combination of two pressure solvers. What makes this method, divergence-free SPH (DFSPH), faster than many of the current SPH methods? A fluid is said to be incompressible when .div v = 0. Based on this fact, the paper Dρ = 0, proposes the following: the divergence-free solver enforces the condition . Dt which means that the density does not change over time. This is equivalent to the divergence-free condition and therefore leads to a divergence-free velocity field. When the zero divergence is not satisfied for a particle, the method calculates a set of pressure forces for particles and their neighboring particles to correct the divergence error. The pressure force for a particle is given by: p

Fi = −

.

mi ∇pi ρi

(3.60)

where the pressure gradiente is computed by differentiating equation .pi = k(ρi − ρ0 ), where k is stiffness parameter. Therefore, the DFSPH method uses the SPH

26

3 Fluid Modeling Through Navier–Stokes Equations and Numerical Methods

formulation ∇pi = kiv ∇ρi = kiv

E

.

mj ∇Wij ,

(3.61)

j

where .kiv is the stiffness parameter to be determined. In order to conserve momenp p p tum, the pressure forces .Fi and .Fj ←−i are considered, with .Fj ←−i acting on the neighboring particles,Ethus obtaining a set of symmetric pressure forces that satisfies p p p the condition .Fi + j Fj ←−i = 0. The forces .Fj ←−i are computed analogously p to .Fi : p

Fj ←−i = −

.

mi v mi ∂pi = k mj ∇Wij . ρi i ρi ∂xj

(3.62)

The density ratio of particle i is calculated as: .

E Dρi = mj (vi − vj )∇Wij , Dt

(3.63)

j

which should be zero after applying the symmetric pressure forces on particle i. The p p Fi←−j F pressure forces cause the velocity changes .Avi = At i and .Avj = At . mi mi Inserting these terms in Equation (3.63) yields: E Dρi . mj = −Δt Dt j

p

(

p

p Fj ←−i Fi − mi mi

) ∇Wij .

(3.64)

p

Using the equations of .Fi and .Fi←−j in Equation (3.64) gives us an equation for the stiffness parameter .kiv : ⎛| ⎞ |2 | |E E | | | | Δt ⎜| Dρi |mj ∇Wij |2 ⎟ = kiv mj ∇Wij || + . ⎝ ⎠. ρi || Dt | j j

(3.65)

Solving for .kiv yields: kiv =

.

ρi 1 Dρi ·| |2 E | |2 . Δt Dt |E | | | j mj ∇Wij | + j mj ∇Wij |

(3.66)

The pressure forces computed with the stiffness parameter .kiv exactly fulfill the Dρi condition . = 0, which means that the velocity field in the neighborhood of Dt

3.3 Smoothed Particle Hydrodynamics (SPH)

27 p

particle i is divergence-free. Finally, the total force .Fi,total is determined for particle i including the forces from neighboring particle j as p .F i,total

=

p Fi

+

E

p Fi←−j

= −mi

j

E

( mj

j

kjv kiv + ρj ρi

) ∇Wij .

(3.67)

Note that this pressure force is equivalent to the symmetric pressure force introduced by [164].

3.3.4 Versatile Rigid-Fluid Coupling for Incompressible SPH The reference [8] presents a two-way coupling method for SPH fluids and arbitrary rigid objects based on hydrodynamic forces. The proposed method samples the surface of rigid bodies with boundary particles that interact with the fluid, which prevents deficiencies and discontinuities. The traditional formulation of SPH for density, given by Equation (3.39), accurately approximates the density if each particle is surrounded by other particles forming a sphere with the particle being calculated at its center. However, particles near the boundary are not well calculated since we do not have the formation of a sphere of neighboring particles, creating discontinuities for the pressure gradient. To solve this situation, the authors applied the Shepard filter, which significantly improved the situation. In order to model fluids interacting with non-deformable rigid bodies, the authors suggest that each boundary particle .bi represents a volume .Vbi at the surface of a rigid body: Vbi =

.

m bi mbi , =E ρbi k mbk Wik

(3.68)

where k denotes boundary particle neighbors. The density of a fluid particle can be written as E E .ρfi = mfi Wij + mfi Wik , (3.69) j

k

where j denotes fluid particle neighbors. For fluid particles near the boundary, the authors add the contribution of the boundary particle, taking into account its volume: wbi (ρ0 ) = ρ0 Vbi ,

.

(3.70)

where .ρ0 denotes the rest density of the fluid. Now, the corrected density can be written in the form

28

3 Fluid Modeling Through Navier–Stokes Equations and Numerical Methods

ρfi = mfi

E

.

Wij +

E

j

wbk (ρ0 i )Wik ,

(3.71)

k

which computes the densities correctly regardless of the boundary particle sampling. It should be noted that in a uniformly sampled scenario, .wb enhances the involvement of particles located at the boundary by approximately 1.4 times the volume proportion of fluid particles.

3.3.4.1

Boundary-Fluid Pressure Force

The pressure between two particles can also be derived in SPH as follows: ( p .F i←j

= −mi mj

pj ρi ρj

) ∇Wij .

(3.72)

For weakly compressible fluids, we can assume that .ρi ≈ ρj and .pi ≈ pj . Therefore, we can rewrite equation (3.72) as: ( p .F i←j

= −mi mj

px ρx2

) ∇Wij ,

(3.73)

where x can be either i or j . In practical terms, the pressure exerted by the fluid onto a specific region of the rigid body does not affect the motion of neighboring fluid particles. Based on this fact, we write the pressure force applied from a boundary particle .bj to a fluid particle .fi as ( p .F fi ←bj

= −mfi wbj (ρ0i )

p fi ρf2i

) ∇Wij ,

(3.74)

by substituting .mj with .wbj (ρ0i ) as done in (3.71). The symmetric pressure force from a fluid particle to a boundary particle is p

Fbj ←fi = −Ffi ←bj .

.

(3.75)

In (3.74) and (3.75), the idea is making use of a fluid particle’s own pressure when computing the boundary force. The size of the forces acting on the boundaries relies on the pressure of the fluid particle. As the particle approaches the boundary, its pressure increases, resulting in a force pushing against the boundary. To counteract this force and prevent the fluid particle from sticking or penetrating the boundary, a proportional force is applied. This approach avoids the use of additional forces or position adjustments, effectively eliminating sticking issues. Furthermore, it

3.3 Smoothed Particle Hydrodynamics (SPH)

29

removes the necessity of having normal information for our boundary particles and does not require density or pressure information for these particles.

3.3.4.2

Boundary-Fluid Friction Force

Friction between interacting fluid and boundary particles is generated by employing the laminar artificial viscosity model (3.43). It is written as ( ||ij = −ν

.

min(vij · xij , 0) |xij |2 + eh2

) (3.76)

2αhcs , where .α is the viscosity constant, .cs denotes ρi + ρj the speed of the numerical propagation, .vij = vi − vj , .xij = xi − xj , and .e = 0.01 is used to avoid singularities for .|xij | = 0. Now, the viscosity force between two particles can be defined as

and the viscous factor .ν =

v Fi←j = −mi mj ||ij ∇Wij .

.

(3.77)

Based on the above equation, we can define the viscosity force from a boundary particle to a fluid particle as Ffvi ←bj = −mfi wbj (ρ0i )||ij ∇Wij ,

.

(3.78)

with the reformulated viscous factor ν=

.

σ hcs , 2ρfi

(3.79)

where .σ is the viscosity coefficient between fluid and rigid bodies. Before calculating any relevant forces, we modify the boundary particles using the position and orientation provided by the rigid solver. Once we have determined the forces involved, we can express the overall force acting on a fluid particle due to the boundary, as well as the total force exerted on a boundary particle by its neighboring fluid particles: Fftotal = i

) E( p Ffi ←bj + Ffvi ←bj , j

.

Fbtotal = i

) E( p Fbi ←fj + Fbvi ←fj . j

(3.80)

30

3 Fluid Modeling Through Navier–Stokes Equations and Numerical Methods

Considering that the paired forces between the particles are symmetrical, i.e., p p Ffi ←bj + Fbi ←fj = 0 and .Ffvi ←bj + Fbvi ←fj = 0, the viscosity and total boundary E E forces are symmetric as well: . Fftotal = Fbtotal . i i

.

i

i

Subsequently, for every boundary particle i that is part of a dynamic rigid body and has neighboring fluid particles, F is converted to total force and torque for the rigid body as Frigid = .

E

Fbtotal , i

i

(3.81)

τrigid = (xi − x cm ) × Fbtotal , i where .x cm is the center of mass of the rigid body, and .xi is the position of a boundary particle. Finally, .Frigid and .τrigid are applied to the rigid body.

3.4 Stable Fluids The paper [220], by Jos Stam, presents a numerical method for simulating fluid dynamics that is stable and efficient. The key idea is to discretize the equations on a grid and then use a technique called “projection” to enforce the incompressibility constraint of the fluid. This results in a stable and accurate simulation of fluid flow. In his paper, Jos Stam uses the Navier–Stokes equations for incompressible fluids, that is, the condition .∇ · u = 0 is enforced, and for the construction of the projection, he uses the Helmholtz–Hodge decomposition, according to which a vector field .w can be decomposed as: w = u + ∇q,

.

(3.82)

where q is a scalar field and .u is divergence-free. Applying the divergence operator to both sides of the previous equation: ∇ · w = ∇ · u + ∇ 2 q,

(3.83)

∇ · w = ∇ 2 q.

(3.84)

.

but as .∇ · u = 0, we have .

Thus, we obtain a Poisson equation for the scalar field q, whose solution allows us to define a projection operator given by: u = Pw = w − ∇q.

.

(3.85)

3.4 Stable Fluids

31

Applying this operator to the momentum equation assuming specific mass .ρ = 1, we obtain: .

( ) ∂u = P −(u · ∇)u + ν∇ 2 u + f , ∂t

(3.86)

where: ( P

.

∂w ∂t

) =

∂ ∂u (P w) = and P (∇p) = 0. ∂t ∂t

By the equation (3.86), we realize that the contribution of pressure is zero for the calculation of the .u component of expression (3.82), which satisfies the condition of incompressibility .∇ · u = 0. The method presented for integrating equation (3.86) is the main contribution of Stam’s work. In short, the Navier–Stokes equations are solved without considering the contribution of the term involving pressure.

Chapter 4

Neural Networks Universality, Geometric, and Dynamical System Elements for Fluid Animation

Abstract In this chapter, we start with basic elements of Boolean expressions and circuit theory to define a background to study neural networks as computational models. We present the truth tables and algebra of propositions. Then, we consider the circuit theory since neural networks can be cast into the computational model formalized by circuit families. Specifically, the neural computation process can be organized in a graph or circuit whose gates (neurons) are arranged in layers. In this way, owing to the equivalence between circuits and Boolean expression approaches, we can discuss the universality of neural computing. We end this chapter with a discussion about elements from differential geometry and dynamical systems that are part of the mathematical background for our presentation. Specifically, we formalize a data model based on the concept of differentiable manifolds. Hence, the database samples and the associated probability density distribution occupy the natural place in the proposed data model. Dynamical systems foundations are necessary to visualize such data model in the context of Hamiltonian formalism when representing fluids as particle systems. Keywords Boolean expressions · Circuit theory · Universality of neural computing · Data model · Particle systems · Hamiltonian formalism

4.1 Introduction The field of neural networks is a mature research area being a branch of machine learning. Since the first artificial neuron model, named perceptron [98], the paradigm of neural computing has been applied almost everywhere in science and technology, attracting researchers from a wide variety of backgrounds. This field involves aspects of computational theory and data science. The former offers elements to demonstrate that neural networks can compute a class of functions with enough generality for fluid simulation. However, to achieve such a goal, practical aspects come into place related to the data necessary for training. Specifically, what are the basic assumptions about data we make when applying

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 G. A. Giraldi et al., Deep Learning for Fluid Simulation and Animation, SpringerBriefs in Mathematics, https://doi.org/10.1007/978-3-031-42333-8_4

33

34

4 Neural Networks Universality, Geometric, and Dynamical System Elements for. . .

neural networks to build surrogate models in fluid dynamics? What about data representation and preparation? From such questions arise a data model that involves geometry and data elements. Specifically, the basic hypothesis is that simulation samples belong to a differentiable manifold, named data manifold in this text. Hence, geometric features like tangent space and metric should be considered. Then, dynamical systems foundations are used to realize such a data model in the context of Hamiltonian formulation when representing fluids as particle systems.

4.2 Neural Networks and Universality The neural network technology provides sophisticated tools that allow performing fluid simulation with a low computational cost if compared with traditional computational modeling techniques based on partial differential equations and numerical methods [30]. However, before entering the topic of machine learning in computational fluid dynamics, we must offer some background in computability theory to set the power of neural computing. Hence, the Sects. 4.2.1, 4.2.2, 4.2.3, 4.3, and 4.4 develops fundamental points necessary to demonstrate the universality of the computational model of neural networks [98].

4.2.1 Boolean Expressions A variable x such that .x ∈ {0, 1} (true 1 and false 0) is called a Boolean variable. Given Boolean variables .x, y, . . ., they can be put together to form Boolean expressions through the operators of conjunction .(∧) , disjunction .(∨) , negation .(∼) , exclusive or (XOR) , implication .(⇒), and bi-implication .(⇔) [9]. These operators are defined by the standard Truth Tables 4.1. Besides, the complete syntax includes parentheses to solve ambiguities. Moreover, as a common convention, it is assumed that the operators bind according to their relative priority. The priorities are, with the highest first: .∼, ∧, ∨, ⇔, ⇒ . So, for example,

Table 4.1 (a) Truth table for negation. (b) Truth table for conjunction (AN D operator). (c) Truth table for disjunction (OR operator). (d) Truth table for XOR operator. (e) Truth table for implication. (f) Bi-implication and its truth table

4.2 Neural Networks and Universality .

35

∼ x1 ∧ x2 ∨ x3 ⇒ x4 = (((∼ x1 ) ∧ x2 ) ∨ x3 ) ⇒ x4 .

(4.1)

A Boolean expression with variables .x1 , x2 , . . . , xn produces for each assignment of truth values (.0, 1 values) to the variables itself a truth value according to the standard Truth Table 4.1a–f. If we denote the set of truth values by .B = {0, 1}, then we can think of a Boolean expression with variables .x1 , x2 , . . . , xn as a function: F : B n → B,

.

where .B n = B × B × B . . . × B (n-times). As an example, let us rewrite expression (4.1) as: F : B 4 → B,

.

F (x1 , x2 , x3 , x4 ) =∼ x1 ∧ x2 ∨ x3 ⇒ x4 .

.

(4.2)

Two Boolean expressions .F1 , F2 : B n → B are said to be equal if: F1 (x1 , x2 , . . . , xn ) = F2 (x1 , x2 , . . . , xn ) ,

.

∀ (x1 , x2 , . . . , xn ) ∈ B n .

(4.3)

Given the Truth Table 4.1a–f, it is possible to establish some algebraic laws that Boolean expressions obey, some of which are analogous to laws satisfied by the real numbers [54, 77]. The corresponding propositional algebra is very much useful to prove properties related to Boolean expressions. Specifically, we have the following laws [54, 77]: (1) Idempotent laws p ∧ p = p,

.

p ∨ p = p.

.

(2) Commutative p ∧ q = q ∧ p,

.

p ∨ q = q ∨ p,

.

p ⇔ q = q ⇔ p.

.

(3) Associative .

(p ∧ q) ∧ r = p ∧ (q ∧ r) ,

.

(p ∨ q) ∨ r = p ∨ (q ∨ r) .

36

4 Neural Networks Universality, Geometric, and Dynamical System Elements for. . .

(4) Absorption laws p ∧ (p ∨ q) = p,

.

p ∨ (p ∧ q) = p.

.

(5) Distributive p ∧ (q ∨ r) = (p ∧ q) ∨ (p ∧ r) ,

.

p ∨ (q ∧ r) = (p ∨ q) ∧ (p ∨ r) .

.

(6) Involution law .

∼ (∼ p) = p.

(7) De Morgan ’s Laws .

∼ (p ∨ q) = (∼ p) ∧ (∼ q) ,

.

∼ (p ∧ q) = (∼ p) ∨ (∼ q) .

(8) Complement Laws p∨ ∼ p = 1.

.

4.2.2 Circuit Theory Formally, neural networks can be cast into the computational model formalized by the circuit theory [180]. We say that a circuit is made up of wires and gates. The wires carry information around, and the gates (functions) perform simple computational tasks. The simplest gates are the logic operators presented in Table 4.1a–f. Figure 4.1a shows the elementary gates and their graphical representation. Also, we can add to the set of elementary gates the identity gate (just a wire) and the FANOUT one, which replaces a bit with two copies of itself. In Fig. 4.1b, it is pictured an example of a circuit that outputs the result of the Boolean expression .x ⊕ y = XOR (x, y) (see Table 4.1d) together with a carry bit set to 1 if x and y are both 1, or 0 otherwise (check it). Formally, let G be a set of gates that map several bits to one bit. For each .n, m ≥ G 0, a circuit .C(n,m) , or simply .Cn,m , if there is no ambiguity over the set G, is a directed, acyclic graph with a list of n input nodes (with no incoming edges), a list of m output nodes (with no outgoing edges), and a gate in G computing each non-input/non-output node.

4.2 Neural Networks and Universality

37

Fig. 4.1 (a) Basic Gates. (b) Circuit example. (Adapted from [180])

Given a binary input string .(x1 , x2 , . . . , xn ) ∈ {0, 1}n , we label each input node .xi or .∼ xi (.N OT (xi )). For every other node v with n predecessors .(y1 , y2 , . . . , yn ), we recursively assign a value .g (y1 , y2 , . . . , yn ), where g is the gate that calculates G G node v. Given an input .x ∈ {0, 1}n , the circuit .C(n,m) outputs the value .C(n,m) (x) ∈ m {0, 1} given by the list of output nodes. The size of a circuit .Cn,m is its number of nodes and the depth of .Cn is the length of the longest path from any input node to an output node. G We say that a circuit .C(n,m) computes a Boolean function .F : {0, 1}n −→ {0, 1}m G if .C(n,m) (x) = F (x), for all .x ∈ {0, 1}n . A circuit family is a list of circuits ( ) .C = C1,m , C2,m , . . . , Cn,m , . . . , where .Ci,m has i binary inputs and m binary outputs. The family C computes a family of Boolean functions .(f1 , f2 , . . . , fn , . . .), where .fi is the Boolean function computed by circuit .Ci,m . We say that the family C has size complexity .s (n) and depth complexity .d (n) if for all .n ≥ 0 the circuit .Cn has size at most .s (n) and depth at most .d (n). Size and depth are important complexity descriptions of circuits that respectively characterize the computational resources and the number of steps needed to compute a family of Boolean functions. Let G be the set of elementary gates represented in Fig. 4.1 and .CG (n, m), the class of functions these { circuits can compute. }We also consider the set of Boolean functions .F (n, m) = F : {0, 1}n −→ {0, 1}m . Theorem 1 Given a Boolean expression .F : {0, 1}n −→ {0, 1}m , there is G G a circuit .C(n,m) such that .C(n,m) (x1 , x2 , . . . , xn ) = F (x1 , x2 , . . . , xn ) for all n .(x1 , x2 , . . . , xn ) ∈ B . As a consequence, it is straightforward that .CG (n, m) = F (n, m). Proof We are going to show the demonstration for .m = 1. From this result, the general case can be straightforwardly addressed. The demonstration is performed using induction in n. Therefore, following the main steps of mathematical induction: • Show that: (a) The property is true for .n = 1; • Next, assume that:

38

4 Neural Networks Universality, Geometric, and Dynamical System Elements for. . .

(b.1) Hypothesis: Property is true for .n = k; • Then, we should demonstrate: (b.2) Thesis: Property is true for .n = k + 1. Using the basic gates of Fig. 4.1 it is more or less straightforward to show that CG (1, 1) = F (1, 1), which solves the item (a). To prove the thesis in item (b.2), the trick is to notice that any function .F : {0, 1}k+1 −→ {0, 1} can be written as:

.

F (x0 , x1 , . . . , xk ) = (F0 ∧ ∼ x0 ) XOR (F1 ∧ x0 ) ,

(4.4)

F0 (x1 , x2 , . . . , xk ) = F (0, x1 , x2 , . . . , xk ) ,

(4.5)

F1 (x1 , x2 , . . . , xk ) = F (1, x1 , x2 , . . . , xk ) .

(4.6)

.

where: .

.

-G Now, using the induction hypothesis, we can affirm that there are circuits .C (k,1) G and .C(k,1) that computes .F0 and .F1 , respectively. By using this fact, to complete the demonstration of the thesis (item (b.2)), it is just a matter of noticing that G expression (4.4) can be converted into a circuit .C(k+1,1) , as shown in Fig. 4.2. Therefore, from the fact that the property is true for .n = 1 and items (b.1) and (b.2), we can affirm that, given a Boolean function .F : {0, 1}n −→ {0, 1}, G G there is a circuit .C(n,1) such that .C(n,1) (x1 , x2 , . . . , xn ) = F (x1 , x2 , . . . , xn ) for n all .(x1 , x2 , . . . , xn ) ∈ B . Therefore .F (n, 1) ⊂ CG (n, 1). Conversely, by the definition of circuits in the set .CG (n, 1) it is obvious that .CG (n, 1) ⊂ F (n, 1). So, .CG (n, 1) = F (n, 1) which completes the proof for .m = 1. A weighted threshold gate (or simply, a threshold gate) with threshold .A ∈ Q and a vector of weights .w = (w1 , w2 , . . . , wn ) ⊂ Qn is a Boolean function denoted by .T hn,A w and defined as follows [201]:

G representing expression (4.4) to compute an arbitrary function .F : Fig. 4.2 Circuit .C(n+1,1) k+1 {0, 1} −→ {0, 1}. (Adapted from [180])

4.2 Neural Networks and Universality

39

n T hn,A w : {0, 1} → {0, 1} ,

.

{ n,A .T hw (x1 , x2 , . . . , xn )

=

En

1,

if

0,

otherwise.

i=1 wi xi

≥ A,

(4.7)

n,A A threshold circuit, denoted by .T C(w,m) , is a circuit over the set of threshold n gates, with input .x ∈ B and output .y ∈ B m . Hence, from the definition of threshold n,A gates (expression (4.7)) a threshold circuit is also a function .T C(w,m) : {0, 1}n −→ n,A {0, 1}m . The class of functions that can be computed by threshold circuits .T C(w,m)

will be denoted as .TFn,A (w,m) . An important theorem about threshold circuits is the following one: Theorem 2 Given a Boolean expression .F : B n −→ B m , where .B = {0, 1} , n,A n,A such that .T C(w,m) there is a weighted threshold circuit .T C(w,m) (x) = F (x) for all n .x ∈ B . Proof To demonstrate this theorem, it is just a matter of showing that the basic gates of Fig. 4.1 can be computed by threshold circuits. Hence, the proof is a straightforward consequence of Theorem 1.

4.2.3 Multilayer Perceptron Model The first logical neuron was developed by W. S. McCulloch and W.A. Pitts in 1943 [15]. It describes the fundamental functions and structures of a neural cell, reporting that a neuron will fire an impulse only if a threshold value is exceeded. Figure 4.3 shows the basic elements of McCulloch–Pitts model: .xi ∈ R, .i = 1, 2, . . . , m are the inputs, that can be arranged in the input vector .x = (x1 , x2 , . . . , xm ) ∈ Rm . The coefficients .wi ∈ R and the bias b are machine parameters, usually named weights in the neural network literature. The combinaE tion . m w i=1 i xi +b is named the induced local field . The function .σ is the activation function that computes the output .y ∈ R. A simple choice for .σ is the signal function .σ (.) = sgn(.) defined by: σ

( m E

.

i=1

) w (i) x (i) + b =

{ 1, −1,

if

Em

i=1 w (i) x (i) + b

otherwise.

> 0,

(4.8)

But the McCulloch–Pitts neuron did not have a mechanism for learning. Based on biological evidences, D.O. Hebb suggested a rule to adapt the weights, which is interpreted as the learning rule for the system [15]. This biologically inspired procedure can be expressed in the following manner:

40

4 Neural Networks Universality, Geometric, and Dynamical System Elements for. . .

Fig. 4.3 McCulloch-Pitts neuron model. (Adapted from [15])

winew = wiold + Awi ;

.

Awi = η(y desired − y)xi ,

(4.9)

where .w new and .w old are adapted and previous weights respectively, .η is a real parameter to control the rate of learning and .y desired is the desired (known) output. This learning rule plus the elements of Fig. 4.3 compose the perceptron model for a neuron. Then, the learning occurs through training or exposure to a know set of input/output data: D = {(x1 , l1 ) , (x2 , l2 ) , ·, ·, ·, (xM , lM )} ⊂ Rm × {−1, 1} .

.

(4.10)

The training algorithm iteratively adjusts the connection weights and bias analogous to synapses in the biological nervous. These connection weights and bias store the knowledge necessary to solve specific problems. In terms of dataflow, we can think of a neural network as a directed, acyclic graph, whose nodes are neurons, implemented as perceptrons, arranged in layers, with weights associated with the edges [98]. As an example, let us consider the learning machine .M = M (x; w) represented by a multilayer perceptron (MLP) with .L ≥ 1 hidden layers [98] defined by the following elements: 1. Input Layer: I = {1, 2, . . . , p (0)} ⊂ N.

.

2. First Hidden Layer: H (1) = {p (0) + 1, p (0) + 2, . . . , p (1)} ⊂ N.

.

3. Remaining Hidden Layers: H (n) = {p (n − 1) + 1, p (n − 1) + 2, . . . , p (n)} ⊂ N,

.

2 ≤ n ≤ L.

4.2 Neural Networks and Universality

41

4. Output Layer: Out = {p (L) + 1, p (L) + 2, . . . , p (L + 1)} .

.

5. Synaptic weight connecting the output of neuron i to the input of neuron j : .wij 6. Bias associated to the neuron j : .bj 7. Array of network weights and biases: w=

.

({

wj k

}p(i)+1≤j ≤p(i+1) p(i+1)+1≤k≤p(i+2)

) , {bs }p(0)+1≤s≤p(L+1) , i = −1, 0, . . . , L − 1 , (4.11)

where .p(−1) = 0. In Fig. 4.4, we picture an MLP where we have .p (L + 1) = p (L)+1; that means, only one output node, corresponding to simple regression or binary classification network. Hence, considering the indexation of Fig. 4.4, the parameter space of the machine model has dimension: k=

L−1 E

.

i=0

p (i) p (i + 1) +

L+1 E

p (i) ,

(4.12)

i=1

and the learning machine encapsulates a space composed of functions .f : Rp(0) × Rk −→ R.

Fig. 4.4 Deep MLP scheme for classification or regression with the sample .xk as input (From [82])

42

4 Neural Networks Universality, Geometric, and Dynamical System Elements for. . .

A neural network with only one hidden layer (.L = 1 in Fig. 4.4) is also named a shallow neural network . On the other hand, deep neural networks have more than one hidden layer (.L > 1). The MLP is also named fully connected (FC) neural network in the sense that all the nodes from the layer .i − 1 are connected to the nodes of layer i. We must be careful about this definition because, if a weight .wj k is null, the effect is the same as if there was no connection between neuron j and neuron k. It is important to discuss the equivalence of shallow and deep neural networks from the viewpoint of computational models. In fact, we can show that we always can convert a deep architecture into a shallow one. However, the conversion is not efficient in the sense that the number of gates in the hidden layer may grow exponentially with respect to .p(0).

4.3 Activation Functions The nonlinear capabilities of neural networks are the consequence of the application of nonlinear activation functions. If we desire the output to be in the range .[0, 1], one possibility for activation functions is the Logistic (or sigmoid) function. f (z; α) =

.

1 . 1 + exp (−αz)

(4.13)

However, the sigmoid function suffers some drawbacks due to the fact that its image set is bounded in the interval .(0, 1). Hence, given a regression problem whose target function is .f : Rm −→ [0, ymax ] ⊂ R we shall normalize the image set to .[0, 1] and, consequently, the combination of normalization and sigmoid will bind a large range of input to a small range of output [53]. Such a procedure could generate small gradient values during the training process, resulting in premature convergence to a poor solution or the inability of deep models to learn patterns on a given dataset due to the fact that the machine propagates only small gradient values from the output layer to the input one, a phenomenon known as vanishing gradient problem [98, 130]. To overcome the limitation of sigmoid to output only a small range of positive values, but keeping the benefits of sigmoidal functions, the hyperbolic tangent (tanh) function is introduced: .

tanh (x) =

1 − exp (−x) , 1 + exp (−x)

(4.14)

that can produce zero-centered results. However, .tanh also undergoes the vanishing gradient problem. The rectified linear unit (ReLU) function, proposed in [177] is defined as:

4.4 Neural Networks as Universal Computing Model

f (x) = max {0, x} .

.

43

(4.15)

Since its proposal, the ReLU has been one of the most widely used activation functions in deep architectures [53]. We shall notice that ReLU function is continuous, but its derivative is not defined for .x = 0. Moreover, it is not bounded and not zero-centered. The computational cost of the evaluation of ReLU is cheap due to its definition. Also, it can introduce sparsity because all negative input values are pushed to zero. In deep architectures, the ReLU function is generally used within the hidden layers with other activation functions in the output unities. In this way, the literature has reported better performance and generalization than the sigmoid or .tanh alone [53]. A variation of ReLU is the parametric ReLU function: { f (x; a) =

ax,

if x ≤ 0,

x,

otherwise,

.

(4.16)

where .a > 0. The special case of .a = 0.01 is named Leaky ReLU function. Other activation functions can be found in [266]. In multiclass problems, given an input sample .x, the neural network must outputs a vector .y = (y1 , y2 , . . . , yK ) where K is the number of classes. So, let us assemble the induced local fields of the output layer in the form .z = W h + b. If we consider a probabilistic interpretation of feedforward outputs of neural networks, then the set of outputs should have the form of a probability distribution over class labels, < i = p (c = i|x) [27]. In this case, it is also conditioned on the input; thatEmeans, .Out < i ≥ 0 and . K required that .Out Out i = 1. To enforce such stochastic constraints, i=1 it is suggested in [27] the normalized exponential output: exp (zi ) Outi = EK ( ) , i = 1, 2, . . . , K, j =1 exp zj

.

(4.17)

which can be arranged in the softmax activation function [27, 87]: ( sof tmax (z) =

.

) exp (z1 ) exp (zK ) exp (z2 ) ( ) , . . . , EK ( ) . ( ) , EK EK j =1 exp zj j =1 exp zj j =1 exp zj (4.18)

4.4 Neural Networks as Universal Computing Model The main question in this section is: What kind of functions can be computed by neural networks? To present the solution step by step, we start by considering only Boolean functions .F : B n −→ B m , where .B = {0, 1}. So, we return to Theorem 2,

44

4 Neural Networks Universality, Geometric, and Dynamical System Elements for. . .

which assures that for every Boolean function .F : B n −→ B m , there is a weighted n,A n,A threshold circuit .T C(w,m) such that .T C(w,m) (x) = F (x) for all .x ∈ B n . Moreover, we should notice that a simple change in the neuron model: y=χ

( n E

.

) wi xi − b ,

(4.19)

i=1

where: χ

( n E

.

)

{

wi xi − b =

i=1

En

1, if

i=1 wi xi

> b,

0, otherwise,

(4.20)

allows to identify neurons with threshold gates, defined by expressions (4.7), if we restrict .(x1 , x2 , . . . , xn ) ∈ B n and .(w1 , w2 , . . . , wn ) ∈ Qn . In this way, neurons are threshold gates .Twn,A : B n −→ {0, 1}. Consequently, a neural network n,A becomes a weighted threshold circuit .T C(w,m) . So, a straightforward consequence of Theorem 2 is that n T C(m) ⊃ F (n, m) ,

(4.21)

.

n is the class of functions that can be computed by threshold circuits where .T C(m) and .F (n, m) is the class of Boolean functions .F : B n −→ B m . Therefore, we can affirm that given a Boolean functions .F : B n −→ B m there is a neural network machine .M = M (x; w∗ ), where .w∗ is a fixed parameter vector in k ∗ m .R , such that .M (x; w ) = F (x), for all .x ∈ B . Z ⊂ Z and To generalize this result for functions in .Q, let us consider a finite set .< a bijective function .e : < Z−→ {0, 1}k+1 , with .k ∈ N large enough, such that if .z ∈ < Z then .e (z) = (c0 , c1 , . . . , ck ) is the only binary sequence in .{0, 1}k+1 that satisfies:

⎛ z = s (ck ) ⎝

k E

.

⎞ cj −1 2j −1 ⎠ ,

(4.22)

j =1

where .s(1) = +1 and .s(0) = −1. Also, we shall remember that, given .q ∈ Q, ∃ (a, b) ∈ Z × Z∗ such that:

.

q=

.

a . b

( )n ( )m Z×< Z∗ and . < Z×< Z∗ allow to represent two subsets Therefore, the sets . < < −→ V 0, Initialization: O0 , t = 0,s = 0, p ← O0 , while (t ≤ kT ) do while (s ≤ kD ) do } { Sample minibatch B = x1 , x2 , . . . , xς ⊂ Dtr , } { Sample minibatch from prior distribution pg (z): Z = z1 , z2 , . . . , zς ⊂ Z, Build the objective function by computing the Loss (6.27) over the minibatches B and Z: L (p; B, Z) =

ς 1E (log D (xi ) + log (1 − D (G (zi )))) . ς

(6.34)

i=1

Learning rule. Update parameter vector for discriminator: Os+1 = Os + ηd ∇θd L (p; B, Z) ,

(6.35)

p ← Os+1 , s ← s + 1, end while s = 0, } { Sample minibatch from prior distribution pg (z): Z = z1 , z2 , . . . , zς ⊂ Z, Build the Loss (6.27) over the minibatches B and Z, Learning rule. Update parameter vector of generator: Ot+1 = Ot − ηg ∇θg L (p; B, Z) ,

(6.36)

p ← Ot+1 , t ← t + 1, end while O*opt ← p,, Output: Solution O*opt .

The minibatch stochastic gradient descent training for GANs is shown in { } Algorithm 2, where .O = θ g , θ d .

Chapter 7

Fluid Rendering

Abstract As shown in the previous chapters, fluid simulation deals with complex models and algorithms to approximate the myriad of behaviors fluids may have. Those simulations produce large amounts of data. An essential step in analyzing these data is its visualization. As 3D data, rendering techniques are crucial in the visualization step. Render deals with transforming the simulation data into an image. It is a challenging task per se and is even more complicated when the subject, in this case, the fluid, has a non-rigid form that can evolve in 3D space and interact with light in complex ways. We will present in this chapter the traditional techniques used to render fluids and the new perspectives that Machine Learning can provide to improve this process. Keywords Rendering equation · Path tracing · Rasterization · Ray tracing · Screen-space rendering · Volume rendering · Differentiable rendering · Neural rendering

7.1 Introduction In a broad definition, the rendering process transforms data into an image, usually a digital one. Figure 7.1 depicts this process. It involves calculating the interactions of light and the objects in the scene, determining the scene’s visible elements, given the observer’s location, and registering these intensities in a discrete set: the digital image. Traditionally, Computer Graphics (CG) is the area of Computer Science that studies how a computer can realize the rendering process in a computer. CG explores algorithms and data structures to model the interactions between light and objects to produce realistic images. There are many issues concerning the optimization of the whole process. The details about the rendering process are beyond the scope of this book. Still, we suggest the interested reader consult “Fundamentals of Computer Graphics” [216], which can give you a broad overview of the fundamental of the rendering process.

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 G. A. Giraldi et al., Deep Learning for Fluid Simulation and Animation, SpringerBriefs in Mathematics, https://doi.org/10.1007/978-3-031-42333-8_7

79

80

7 Fluid Rendering

Fig. 7.1 Overview of the image formation process: light source produces light rays that interact with scene elements. Some reflected light rays enter a capture device, passing through its lenses and reaching its sensor, defining the color of one part (pixel) of the image. (Image from [33])

To support the rendering process, some data should be registered and structured. Some of the most important are the geometric properties of the objects (e.g., location, orientation, and scale), the optical properties of its materials, the scene elements, such as the observer, and light sources, locations, and properties. Physics has been studying the interaction of light and objects for centuries. In the context of CG, the most important formulation to represent those interactions is the Rendering Equation, proposed by Kajya [114]. It describes how to calculate the light intensity (radiance) perceived by an observer looking at some point of an object as a function of the radiance emitted by its material and the interaction of this material and the radiance received in all directions around that point, coming from the entire scene. Equation (7.1) shows that relations. { Lo (x, ωo , λ, t) = Le (x, ωo , λ, t) +

fr (x, ωo , ωi , λ, t)(ωi · n)dωi .

.

(7.1)

O

Figure 7.2 illustrates the physical interpretation of each term of equation (7.1). The terms .Lo , .Le , and .Li represent, respectively, the radiance observed, emitted, and incident in a point .x on the surface of an object. The radiance is a function of the position .x, the direction of observation (.ωo ), the light wavelength (.λ), and time (t). The first term, .Le , is related to material emission and usually is the dominant term in light sources. The second term, the Integral, considers the contributions of .Li through the directions along the hemisphere .ω around .x. The terms .fr (x, ωo , ωi , λ, t) and .ωi · n deal with, respectively, material and geometric properties of the object at point .x. The first one, called Bidirectional Reflectance Distribution Function (BRDF), models the relationship between the in (.ωi ) and out (.ωo ) directions, depending on the object’s material. This function allows the model to incorporate the visual characteristics of materials, as reflection is concerned, such as glossy, diffuse, and specular materials. The BRDF can be mathematically constructed or captured from natural materials using proper devices [133].

7.1 Introduction

81

Fig. 7.2 Illustration of the components of the model used in the rendering equation proposed by Kajya [114]: the observer receives some radiance in direction .ωo from point .x over the surface of some object, which is related with all contributions from directions .ωi in every direction along the domain .O. (Image by Timrb – Own work. Used under CC BY-SA 3.0. https://commons.wikimedia. org/w/index.php?curid=4251811)

The geometric component considers the influence of the orientation of the surface, given by its normal .n and the direction of reflection .ωi in point .x. Although the Rendering Equation considers the wavelength .λ in its formulation, the visible spectrum of light is also quantized to three primary wavelengths: Red, Green, and Blue. These three wavelengths are what we call the Color Tristimulus Model [4]. The RGB representation of colors can incorporate another channel related to the material’s opacity: the alpha channel. This channel is used in blending operations and can mimic translucent materials. It is important to notice that the Rendering Equation is intrinsically recursive; to evaluate the radiance in directions .ωi along .O, one should evaluate the Rendering Equation in that direction to calculate its radiance. This recursive aspect is due to what we in CG call indirect or global illumination. The Rendering Equation was the first attempt to model the light interaction in a scene. Nevertheless, this model does not consider many light phenomena, such as transmission, subsurface scattering, polarization, and luminescence. There is no analytical solution to equation (7.1) in a general case, so we use numerical methods or simplified models to solve it. If someone disregards the indirect illumination, the integral can be modeled as a summation along the directions of the light sources, like Phong’s Reflection Model [191]. In the general case, to solve the integral in equation 7.2 numerically, stochastic methods, such as Monte Carlo Integration [248], can be used. The Path Tracing [114] was the algorithm proposed by Kajiya to obtain a numerical solution to the integral of the rendering equation. As Fig. 7.2 indicates, and equation (7.1) states, we have to calculate the radiance in each point .x of the surface of objects in a scene. As we want to produce a digital image of the scene, a sampling process should evaluate, for each pixel, how much light it receives from the scene. Two main approaches can perform this sampling process: Rasterization [4] and Ray Tracing [96]. Figure 7.3 depicts these two approaches. Rasterization is the most commonly used approach in real-time graphics applications. The objects in the scene are projected in the image plane, and then samples

82

7 Fluid Rendering Sh P o Pr lyg oje on ct s s to urfa vie ce wp or t

oo thr t ray ou s in gh vie to sc wp ene ort

Viewport 2D image

Camera

Image Light Source View Ray

3D scene

Shadow Ray

Viewport Scene Object

Viewpoint

Fig. 7.3 Rasterization (left) and Ray Tracing (right) are the main approaches to rendering a scene. The first projects each object in the image plane (viewport), while the second traces rays from the observer to the scene identifying the closest object visible and registering its color. (Image by Henrik – Own work. Used under CC BY-SA 3.0)

are generated. Most Graphics Processing Units (GPUs) have native support for this approach. Its limitations appear when more complex interactions between the light and the objects, such as transparency, should be considered. On the other hand, Ray Tracing takes each image pixel as the start point of a search process that traces a ray of light back to the scene to discover which object emitted light in that direction and then calculates the intensity of the light reaching the image. Ray Tracing is the most used algorithm in offline rendering applications, such as movies, which require more photorealism. Its process can be applied recursively to deal with more complex light interactions in the scene. The interested reader, who wants a more deep discussion about the rendering process, could consult classical Computer Graphics rendering books, such as [4, 190] and [96, 154].

7.1.1 The Rendering Process in Fluid Simulation As briefly summarized in the last section, the rendering process is challenging, considering the complex phenomena of light propagation and object interaction. When the object is fluid, the rendering process is even more complex. In Fig. 7.4, we can see a fluid’s two most common visual appearances: liquid and gas. They all represent one first difficulty in rendering fluids: the lack of a fixed geometry. Rigid objects do not change their geometric properties over time, so it is easy to characterize their surface and its properties, such as curvature. So, the most used data structure representing rigid objects is a polygon mesh [25]. It models the object surface as two sets, one of the vertices, sample locations over the surface, and another of faces, representing vertices’ connection (the surface’s topology). Faces

7.2 Traditional Techniques

83

Fig. 7.4 Fluids can be mostly (a) liquids or (b) gases, leading to different visual characteristics that must be considered in rendering. ((a) Image by Davide Restivo – Own work. Used under CC 2.0. (b) Image by Sushumna Rao Tadinada – Own work. Used under CC 2.0)

are typically triangular (a set of three vertices), which implicitly defines a planar surface in its interior. The surface approximation in this way is called a Piecewise Linear Surface [85]. Conversely, fluid is usually represented as a particle system, a set of positions in 3D space with properties such as position, pressure, and velocity. In such representation, there is no predefined surface. The boundary of a fluid may vary in time in its geometry and topology. As shown in Fig. 7.4a, the liquid can change its topology when drops are produced for some time. So, to render a fluid using traditional techniques, we have to determine the location of the particles along the fluid’s boundary to calculate the light interaction with it. We also must estimate the curvature of the fluid’s surface, which can change substantially over time. But it can be even more complicated if we consider that light can interact with the fluid’s material more sophisticatedly than in rigid solid opaque objects. When the fluid is a gas, light can cross the medium and interact with it, as in Fig. 7.4b. In a gas, particles are distributed more sparsely along the medium (usually the air). Then, there is no surface in a gas to be determined. The light passes through the fluid and interacts with its particles. While light transpasses the fluid, it can be absorbed, reflected, or even scattered by the gas particles. In this case, the Rendering Equation is not applicable as long as there is no boundary to interact with light. In the following sections, we will go deeper into that discussion, showing how to adapt traditional rendering techniques to render fluids, particularly liquids, and gases.

7.2 Traditional Techniques For the context of this section, what we call traditional techniques are the two main approaches presented earlier: Rasterization and Ray Tracing. More than addressing

84

7 Fluid Rendering

the methods themselves, we will focus on the adaptations needed to render fluids correctly. Moreover, depending on the type of fluid we are dealing with, one or another will be more adherent to dealing with rendering. Again, we recommend the interested reader look for more details about those rendering techniques in books like [4] for Rasterization and the trilogy from Peter Shirley [213–215] for a pragmatic approach to Ray Tracing.

7.2.1 Surface Extraction Let us consider that a particle system represents the fluid, where each particle stores at least its position in 3D space and is subjected to fields like velocity and pressure. As these fields evolve during the simulation, the particle’s positions are updated, and the fluid can assume a new configuration and geometric appearance. A particle system does not define a shape that can be directly rendered. Even if we consider that particles have a size, like a marble, they do not describe a continuous surface as a polygonal mesh does. A particle system is an intrinsically discrete representation of the fluid. So, to use the traditional rendering techniques, we must produce a continuous model to sample to create the digital image. In Computer Graphics, using points to represent shapes is not new. First proposed by James Blinn [23] in the early 1980s, named Blobs. This representation was revisited in the 2010s with the name Point-Based Models [91], as the development of capture devices and Computer Vision techniques capable of producing large Point Clouds from objects and scenes. For rendering proposes, point-based objects can be modeled as implicit objects. An implicit function can be built based on each point’s location .ri = (xi , yi , zi ). As proposed by Blinn [23], in its seminal work, an implicit function .F (r) can be created as: F (r) =

n E

.

bi exp(−ai .di2 ),

(7.2)

i=0

where n is the number of points, .di is the Euclidean distance from location r to the point’s location .ri , and .ai and .bi are parameters that can enhance the user’s control of the final shape. This simple approach is, conceptually speaking, similar to the SPH model. Müller et al. [172] proposed using the color field, the base field to simulate surface traction, to identify surface particles in an SPH simulation. The color field was defined as cS (r) =

E

.

j

mj

1 W (r − rj , h), ρj

(7.3)

7.2 Traditional Techniques

85

where .mj and .ρj are, respectively, the particle’s mass and density at its location, and W is the kernel that correlates the influence of particles located at r and .rj with the threshold h. Particles in the surface of the fluid satisfy [172]: |n(ri )| > 1,

(7.4)

n = ∇cS .

(7.5)

.

where: .

Although this kind of implicit object is mathematically interesting, it is not a good solution from the rendering point of view. The implicit formulation did not describe the surface as an explicit surface, that is, a set of points .x where we know they are on the surface. Instead, the implicit formulation gives us a tool to verify whether a point .x belongs to the surface. So, we will need an algorithm to search for those points .x over the 3D domain where the fluid surface is. The easiest way to do such a search is to sample the domain regularly and evaluate the implicit function (the color field) over those samples. The samples are organized topologically as a grid composed of cubic cells. Then, for each cell, the algorithm verifies, based on the values on the cell’s vertices, if the surface passes through the cell. If it is the case, the intersections of this surface along the edges of the cell are calculated using linear interpolation. The intersection points are then connected to form a polygonal approximation of the surface along the cell. By combining all the polygons, the algorithm builds a piecewise linear approximation or a polygon mesh [25] that represents the surface of the fluid. This process was first proposed by Lorensen and Cline [148], who called the algorithm Marching Cubes. Nevertheless, Marching Cubes (MC) presents some issues that are especially critical in objects like fluids. First, the surface details that MC can recover have a strong relationship with the grid resolution used to sample the domain. Fluids may have small details like droplets and sprays that require a small cell size to capture these elements. A high-resolution grid increases the computational cost of the algorithm, not only in processing but also in memory occupation. To reduce this cost, solutions based on an adaptive subdivision of the space [275] or using a narrow band to look for the iso-surface [39, 282] can mitigate this problem. The quality of the piecewise approximation is also an essential issue for rendering. MC algorithm produces a polygon mesh using a simple linear interpolation over the edges of the intersecting cells. The triangles obtained in this process have different shapes, mostly with an aspect ratio not balanced, which means far from an equilateral shape. When rendered, such an irregular polygon mesh produces a bumpy surface, which is not expected in a fluid surface. A postprocessing stage can be applied to rearrange the triangles and create a more smooth surface formed by triangles with a better aspect ratio. Akinci and colleagues [6] address this problem by applying in sequence a decimation [76] and a subdivision [36] steps.

86

7 Fluid Rendering

The visual quality of the surface generated in this scheme is also related to the kind of implicit function associated with each particle used. As equation (7.3) shows, the kernel W is based on the Euclidean distance. The implicit function induced by this kind of kernel generates a “blobby” surface. On the surface, we can still see the shape of the particles as little rounding spheres. Alternative ways to construct these kernels have been proposed in the literature, such as [289]. Using anisotropic kernels [39, 284] is also an attractive alternative, as it can produce smooth implicit surfaces, preserving the regions of the fluid where “sharp edges” can appear, like on the crest of a wave. Surface Extraction was a popular approach to render fluids for a long time since MC is a simple and easy-to-implement algorithm in CPUs. However, when GPUs became popular, they brought new possibilities for parallel algorithms and the use of the graphics pipeline to accelerate the rendering process. It is possible to implement the MC algorithm in GPU [63], but GPU opened new possibilities to render fluids with more sophisticated light phenomena, such as transmission and reflections.

7.2.2 Screen-Space Approaches In a complex scene, it is plausible that many geometric details or even entire objects could be hidden from the observer and consequently not contribute to the final image. The same happens when the geometric models are massive, like a fluid simulation. To resolve objects’ visibility, raster-based rendering algorithms usually rely on z-buffer algorithm [223]. The z-buffer stores the distance of the closest fragment of a scene object to the camera, the visible fragment. During the rendering process, for each new fragment, the z-buffer allows detection if this new fragment is visible. If comparing the closest distance stored in the z-buffer and the new fragment distance, the new fragment has a smaller distance, the z-buffer and the framebuffer are updated, as the new fragment is closer and so visible; if not, the fragment is discarded as the current closest fragment occludes it. Of course, this solution is valid just for materials without transparency. Although simple and fairly used in the rendering process, the z-buffer algorithm has a drawback for complex scenes. The fragment is only discarded after the render step is done; then, much of the rendering effort is lost. A common approach to accelerate the rendering process in such cases is to use G-Buffers [205]. From Geometric Buffers, the main objective of G-Buffers is to store information about the objects visible from a scene in 2D buffers, which will be used to render the scene as a postprocessing step. So, the inspiration of G-Buffers is to postpone the lighting/shading calculations, which is a costly step in the rendering process, just when the visibility was already resolved for the entire scene.

7.2 Traditional Techniques

87

This process of using G-Buffers to render the scene receives the name of Deferred Shading [195]. It is characterized as a multipass rendering approach. In the first rendering pass, the entire scene is processed to resolve visibility and produce the GBuffers used in the next steps. The lightning/shading algorithms are applied in the following steps, but now the scene is no more processed, just the information stored in the G-Buffers. Then, Deferred Shading algorithms became cheaper as long as they were almost independent of the scene’s complexity [140]. Some algorithms used the same idea of multipass rendering and G-Buffers in a more restricted way to solve a specific issue in the rendering process. Such approaches are usually called Screen-Space, as the process involved in their task is based on data stored in a 2D matrix/image. Shadow is an example of a process that can be done using extra buffers [270]. In fluid rendering, Screen-Space approaches are very popular and efficient in achieving real-time visualization of massive simulations. The main advantage of screen-space rendering is not relying on surface reconstruction algorithms. Each particle can be treated as a small sphere that can be projected into the screen and produce G-Buffers that store depth, normal, among other information. These G-Buffers can be used, in a deferred shading approach, to produce the image of the fluid surface from the observer’s point of view. This is enough to render the fluid without reconstructing its whole surface. This process is visually presented in Fig. 7.5. The surface rendering using this simple idea will have visual inconsistencies as the fluid surface is induced by combining all particles’ influence. But it is possible to reproduce this kind of summation of contributions of nearby particles in screen space. Van de Laan and colleagues [245] proposed to apply an image-based filtering process to produce a smooth version of the depth buffer. The choice of filter is

Fig. 7.5 An overview of a typical screen-space fluid rendering pipeline: the fluid data are processed in a deferred-shading-like process, generating depth and thickness buffers (2D), filtered and used to render the scene image. (Image by Kitware – https://www.kitware.com/screen-spacefluid-rendering-vtk/)

88

7 Fluid Rendering

crucial. As the depth buffer presents a view projection of the surface, we are supposed to have discontinuities as the fluid can produce wave crusts and valleys. So the filter must preserve these discontinuities. Filters like Curvature Flow [245]; Bilateral Gaussian [60] or Narrow-Range [237] can be used to improve the visual quality of the final image. Other buffers can be produced during the first rendering pass to improve the visual effects that the Screen-Space approach can achieve. The normal buffer can be generated from the depth buffer. It is important for lighting calculation and specular reflections on the surface of the fluid. A thickness buffer that estimates the fluid volume can be used to reproduce transparency and color attenuation during the rendering process [60]. Like the depth buffer, this buffer must be filtered to produce a smooth result. To identify foam and droplets, the density of the particles can be used to classify, based on a threshold, if particles belong to fluid, foam, or droplets [60]. A mask buffer can be generated, identifying this kind of particle that can be rendered with different visual properties. Optimizations such as using anisotropic particles [276] and narrow-bands [56] can also be used in screen-space approaches to improve the visual quality and performance of these algorithms. Screen-Space approaches are a common solution for fluid rendering as it is simple, well adapted to the traditional rendering pipeline, and achieves real time. Nevertheless, it still relies on the fact that the fluid has a distinguished surface, and this is not the case when the fluid is a gas.

7.2.3 Volume Rendering While rendering liquid fluids, the challenges are dealing with the dynamic geometry and properties like transparency and reflection; gaseous fluids elevate them even more. Gases are fluids where large distances separate the particles compared to liquids. Therefore, the forces between these particles are fragile. Their particles can move more freely. Those characteristics imply that gases are more suitable to interact with the support medium where they are immersed. So, visually speaking, gases interact more diverse and sophisticatedly with air. Figure 7.6 shows the three most important light interactions that can occur in a gas fluid: the smoke looks black as its particles absorbs most of the light; clouds are particles of water and dust, that scatters the light; and fire emits light as the particles participate in the combustion chemical reaction. In this context, the Rendering Equation [114] discussed in Sect. 7.1 cannot represent the light interactions that occur in gases. The Rendering Equation deals with light interactions on the surface of opaque materials, where reflection is the primary phenomenon. Gases, on the other hand, are intrinsic volumetric particles that occupy a volume, and the light interacts with its particles as it travels through

7.2 Traditional Techniques

89

Fig. 7.6 Three different kinds of volumetric light interactions can produce distinct visual effects: (a) absorption, (b) scattering, and (c) emission. (Image by Jan Novák [184] – https://cs.dartmouth. edu/wjarosz/publications/novak18monte-sig-slides-2-fundamentals.pdf)

Fig. 7.7 An overview of the volume rendering equation, which calculates the contributions along a ray that passes through the volume in the direction .ω and reaches point .x, considering the radiance from the background .Ld (xd , ω), the radiance emitted .Le (x, ω) from any voxel in the path, and the radiance received in any voxel by scattering .Ls (x, ω). (Image from [70])

this volume. So what we will need to render a gas fluid is a Volume Rendering Equation (VRE) [273]. Figure 7.7 depicts the main components of the VRE. Considering a light ray defined by the parametric form .xt = x − tω. Starting from the location .xd at the background, which radiance is .Ld (xd , ω), as this it passes through the fluid, its particles can absorb the radiance. This absorption is modeled as the transmittance function .T (t) that increases as the ray passes through the fluid. During this path, some particle’s fluid can produce radiance by emission .Le (x, ω) or receive radiance from scattering .Ls (x, ω). These last two components are also subject to absorption during their path, so the transmittance .T (t) is also applied to them. Formalizing the process illustrated in Fig. 7.7, the Volume Rendering Equation can be stated as:

90

7 Fluid Rendering

{ L(x, ω) =

d

.

T (t) [σa (x)Le (xt , ω) + σs (x)Ls (xt , ω)] dt + T (d)Ld (xd , ω).

t=0

(7.6) Like the Rendering Equation, VRE has no algebraic solution in a general case, so numerical methods must be used to solve it. The discretization of equation (7.6) leads to a discretization of the whole volume in voxel (volume elements), where we assume that the emission, absorption, and scattering are constant. Usually, this discretization is a 3D regular grid. The discretization of equation (7.6) leads to an algorithm similar to Ray Tracing, presented in Sect. 7.1, called Ray Marching. Ray Tracing is concerned with reflections and presumes that nothing happens to the radiance while the light travels through the medium where the objects are embedded. On the other hand, Ray Marching is concerned with the interaction between light and participant media: the fluid. A more profound discussion of the VRE and Ray Marching goes beyond the scope of this chapter. We recommend that the interested reader consult the book of Magnus Wrenning [273] or the SIGGRAPH 2017 Course Notes from Fong and colleagues [70] for a more profound discussion of the theory and implementation of Volume Rendering techniques. The computational cost of volume rendering techniques is more significant than traditional surface-based rendering for time and memory metrics. The volumetric grid storage may consume a considerable amount of memory. The resolution of that grid is directly related to the visual quality of the rendering. But more voxels demand more processing time. Many attempts have been made to reduce those costs, mainly related to adaptation. The work of Fraedrich et al. [71] proposed a hierarchical particle representation that allows data resampling during the rendering process. To deal with the memory limitation, they use a grid partitioning scheme where the volume is cut into slabs and processed in front-to-back order to render the fluid. Although not focused on volume rendering, Zirr and Dachsbacher [290] presented an adaptive voxelization data structure that can accelerate the ray marching algorithm. Nowadays, neural network techniques can be used to enhance volumetric rendering by applying techniques like super-resolution and style transfer, which will be discussed in Sect. 7.5.

7.3 Advanced Techniques The rendering techniques presented in the previous sections can provide visual feedback for scientists interested in evaluating fluid behavior. However, additional techniques must be used if the application requires a more realistic visualization.

7.3 Advanced Techniques

91

In the movie industry, fluids can be used as part of the postproduction [108], or in animations like Disney Frozen [222], and Moana [32, 72, 187, 217]. Realism is very important, but the time required to produce each frame is not critical. Rendering farms are a common solution to parallelize image generation. The game industry is another area that can use fluids and requires more sophisticated rendering effects. With an additional requirement: real-time rendering. The players and non-players characters can interact with fluid that must respond plausibly but, most important, must look real. This section presents more advanced techniques to increase visual quality and performance in fluid rendering.

7.3.1 Optical Effects For the rendering of a fluid like water to become realistic, some optical effects must be present in the image. Those effects include surface specular reflection, refraction, and opacity due to the depth of the fluid. Figure 7.8 depicts those three effects. Specular Reflection enables one to see the image of the surrounding environment reflected over the fluid’s surface. Environment Mapping [89] is a well-known rendering technique that can simulate this reflection. It is based on Texture Mapping [101], where an image of a material can be projected on the object’s surface. A mapping function defines a correspondence between points in the object and a location in the texture space, a 2D space directly related to the texture image. Then, the color of each fragment in the surface object can be associated with the color stored in the texture image. Environment mapping extended this idea by defining a set of images representing the views of the environment around the object and calculating, based on the specular reflection law (incident and reflection rays have

Fig. 7.8 Surface specular reflection, refraction, and opacity due to deep are the three most perceived optical effects in a fluid like water. https://www.disneyanimation.com/data-sets/?drawer=/ resources/moana-island-scene/

92

7 Fluid Rendering

Fig. 7.9 Snell’s Law of refraction. (Image by Oleg Alexandrov. Public Domain)

the same angle considering the surface normal), which view should be used for texture mapping. Texture mapping and, on your turn, environment mapping can be accommodated into both rendering schemes: Rasterization and Ray Tracing. So surface reflection can be achieved easily with this kind of technique. In shallow water, the refraction effect is significant. In Fig. 7.8, the rocks near the beach and the sand can be seen under the waterline. As we know, the refraction effect occurs when light passes from one medium to another. The difference in the densities promotes a change of light direction. Snell’s Law model that effect by equation: .

sin θ1 n1 v1 = . = n2 v2 sin θ2

(7.7)

Figure 7.9 depicts the Equation (7.7) terms. It states that the ratio of the two refractive indices, .n1 and .n2 , is equal to the ratio of the phase velocity in the two mediums, .v1 and .v2 . As the velocity changes, the light direction gets closer or farther from the surface’s normal, represented by the angle of incidence .θ1 and the angle of refraction .θ2 . So, Equation (7.7) can be used to determine the light direction and, like in surface specular reflection, use this direction to define what the observer sees through the water. Refraction can be incorporated into the rendering process depending on the scheme used. Rasterization can use the direction of refraction to define a texture/environment mapping the fragment’s color [49]. As refraction occurs at some point on the surface of the fluid, a composition scheme should be applied to add reflection and refraction components. Although simple, texture mapping can only “see” the bottom of the fluid. Transparency must be considered whether there are objects inside the liquid, like the rocks in Fig. 7.8. The surveys from Maule et al. [156] and Vasilakis and colleagues

7.3 Advanced Techniques

93

[247] give a broad overview of algorithms used to incorporate transparency into rasterization rendering. The most used techniques involve z-buffer extension, where multiple fragments’ depths and colors are stored in an ordered data structure. A final blending step combines all those fragments to produce the last pixel. Transparency algorithms allow to render of underwater objects but do not take into account the distortions promoted by refraction. Deformation in partially submerged objects, the difference in depth of objects, and deformation of the image due to ripples on the water surface are not supported directly by rasterization algorithms, as they did not compute the redirection of the light ray due to refraction. Ray Tracing rendering scheme incorporates refraction naturally, as the refraction defines another ray of light that can be traced in the same way as a reflection does. Usually, a decay term is used in the composition related to the path of the rays in the scene. When dealing with fluids, this decay must be a function of the depth of the liquid. An estimation of the depth of the fluid can be obtained using a two-pass render of the fluid registering the closest and farthest distance along the ray [279]. Both rasterization and ray tracing schemes can support the optical effects described in this section. Ray Tracing deals more inherently with it than rasterization, which must use additional algorithms to produce almost realistic results. Specific fluids, such as water or seawater, require additional effects for a realistic rendering. The following section will discuss how to enhance splashes, add foam, and produce bubbles in water rendering.

7.3.2 Bubble, Spray, Foam, and Droplets Aside from optics, other visual effects are important in fluid rendering, particularly in water visualization. Figure 7.10 depicts some common visual effects related to water in nature. In the waterfall shown in Fig. 7.10a, the mixture of water and air during the fall produces the spray. As Fig. 7.10b shows, air bubbles produce foam. Small quantities of liquid can accumulate over the surface of objects or form droplets due to surface tension, as in Fig. 7.10c.

Fig. 7.10 Liquids can produce many visual artifacts when interacting with other mediums and objects: (a) spray, (b) foam, and (c) droplets. ((a) Image by JoKerozen – Own work. Used under CC 2.5. (b) Image by Sean O’Flaherty – Own work. Used under CC 2.0. (c) Image by Suparnp – Own work. Used under CC 4.0)

94

7 Fluid Rendering

All these phenomena are important clues that produce convincing fluid rendering in gaming and movies. The physical modeling of those effects is complex and involves sophisticated multiphase models [83, 128, 139] in the case of bubbles, foam, and spray. Bubbles occur when air and water particles mix themselves. Heat or strong movements in the liquid can cause large-scale bubbles, where much air is trapped inside the fluid. Due to the density difference between air and water, bubbles are forced into the fluid surface, which elevates the liquid surface, then breaks, causing a shallow wave. Some smaller bubbles do not break but accumulate with other small bubbles forming foam. Conversely, water can form drops when small quantities become isolated, surrounded by air. This can happen by condensation or accumulation on a surface. Surface tension acts to hold the water in rounded geometry. When these drops have a small scale and are mixed with air particles, it forms the spray effect. As rendering is concerned, bubbles and droplets bring more challenges. In both cases, their boundaries are dynamic, and their visualization must consider refraction to become realistic. Foam and sprays are even more complicated to render as the fluid surface in those areas is fuzzy. Spray occurs in waterfalls and waves. The simulation must consider the interaction of air and water particles in the volume where the spray occurs. Kin et al. [128] describe a narrow-band approach to determine where air particles are introduced. Bubbles and foam can be represented as special particles in an SPH simulation [236]. It is possible to use surface extraction [106] and Ray Tracing to render the refractions that occurred inside the bubbles. One of the difficulties in this approach is the need for a higher grid resolution to get the bubbles’ structures. Screen-space techniques are particularly interesting for rendering small-scale phenomena directly related to particles. Particles can be classified as part of the foam based on criteria like density or fluid surface curvature [105]. Those particles can produce new layers in the G-Buffer with additional information such as foam intensity, shadow, and irradiance [7]. Different intensity distributions could also be created related to spray particles, surface foam, and air bubbles, as Fig. 7.11 depicts. This can produce more realistic results in the final composition step [7].

Fig. 7.11 The different intensities patterns induced by particles: spray (left), foam (center), and bubbles (right). (Image from [7])

7.3 Advanced Techniques

95

The screen-space solution can be applied for spray rendering. Based on a multiphase simulation, Kim and colleagues [128] identify a narrow band where air particles are introduced. The fluid particles identified on this narrow band can be rendered as a layer where visual properties are calculated considering its density and diffuse properties. Rendering water drops over surfaces is a relatively easier task. Using the same screen-space approach, drops can be mapped into G-Buffers representing height distribution, and normal maps, composed with environment maps, can produce a layer that can be rendered over the object [225]. Heuristics can be used to simulate the physics related to the movement of the drops and how they can merge [37]. One drawback of screen-space approaches is that they require multiple rendering passes to produce each layer of information. As the number of particles in the simulation grows, these multiple passes can become prohibitive. Optimization strategies can overcome this cost, reducing the number of particles that need to be evaluated for rendering. That is the subject of the next section.

7.3.3 Optimizations Fluid simulation and rendering are computer-intensive tasks. So, it is natural that optimization techniques are investigated and proposed to accelerate those processes. In computer graphics research, much effort is made to reduce the cost of the rendering process, eliminating objects that are not visible from the camera or reducing the geometric details of objects far away. Culling techniques [4] can avoid the rendering of objects or part of them that is not visible by the camera. Level Of Detail (LOD) control [150] uses data structures and algorithms capable of reducing or increasing geometric details of an object based on a visual metric that depends on their proximity to the camera. In a broad sense, both techniques try to adaptive control the processing workload due to some metric, like visibility for culling or distance from the observer for LOD. The keyword here is adaptive. In fluids, similar ideas can be applied. Relatively stable fluid regions require less attention than where vortexes, ripples, or sprays occur. This attention, numerically speaking, is closely related to sampling. In a particle fluid simulation, the number of particles used or even the size of the particles could vary based on this adaptive metric. As rendering is concerned, the surface of the fluid/liquid is the place where the details make a great difference in the final image. Adaptive approaches use the concept of split and merge particles [2, 271] to apply some LOD control approach. The split operator can produce small particle sizes in areas of the fluid where fine details are required, as splashes and crests occur. The merge operation generates bigger particles, whereas the fluid requires fewer details to be rendered. If carefully managed, these operations can be used for rendering and simulation [271].

96

7 Fluid Rendering

An exciting use of this split-and-merge approach in the context of the fluid simulation was proposed by Truong and colleagues [238] to deal with particle collision in solid–fluid coupling. The adaptive approach can benefit rendering as more particles can induce a more detailed surface sampling and, consequently, a better surface extraction. Screenspace schemes can also take advantage of adaptation as the total number of particles can be reduced. The relative location and the particle’s neighborhood must be determined quickly to identify particle candidates to split and merge operations. Hierarchical and adaptive data structures are used for localization problems. The Octree is one of the most commonly used data structures [67] for this purpose. Hierarchical data structures can also be applied to accelerate the rendering process on the Ray Tracing approach [174–176]. The search for possible objects the ray can intercept is a critical step in Ray Tracing and is optimized by the spatial data structure [189]. It is important to note that nowadays, all effort in optimizations uses the parallelism of the GPUs intensively to reduce processing time. The work of Yang and Gao [282] is an example of a surface reconstruction approach entirely done in parallel on GPUs. Such algorithms must pay attention to workload balance [271], memory layout, and footprint [176], among other concerns.

7.4 Machine Learning in Rendering Machine Learning (ML) techniques are spreading through many areas of knowledge, including Rendering [288]. An issue about rendering is that it is a timeconsuming process, considering realistic rendering. Rendering algorithm such as Path Tracing requires a vast number of rays to achieve a continuous image, which is computationally costly and time-consuming. To reduce its cost, researchers try two main strategies: reduce the number of rays traced or improve the selection of rays to trace [291]. The former enhances the noise in the final image. In postprocessing, ML techniques can denoise the image when low samples per pixel are used [104]. The choice of how to sample directions to trace during Path Tracing can use ML techniques to learn the best strategies of sampling that can reduce the noise effect [104]. Rendering complex objects requires a lot of information related to their geometry and material. In the 1990s and 2000s, researchers tried to reduce the complexity of rendering using a set of images to recover information about shape, material, and light, the so-called Image-Based Techniques [57]: Image-Based Modeling, ImageBased Rendering and Image-Based Lighting, respectively. The main idea of such techniques was to produce geometric data, lighting conditions, and objects’ material based on analyzing the set of images from a scene using regular algorithms. A lot

7.4 Machine Learning in Rendering

97

of progress has been made since then, but this approach has some drawbacks, such as its computational cost and its difficulty to be generalized [57]. Nowadays, ML techniques can be used to reconstruct objects [93] from discrete samples (point clouds), to recover material characteristics [66], and light conditions [280]. Similar to Image-Based techniques, ML relies on Deep Learning techniques that usually require sets of images to train the neural network to produce missing information on the geometry of a model, change the visual characteristics of a material, or even change the lighting conditions of a scene. The training phase is one of the drawbacks of this kind of technique. Large datasets are required, but they are only sometimes available. To mitigate this drawback, new methods to synthesize data are under development [239].

7.4.1 Neural Rendering Neural Rendering main goal is to generate synthetic photorealistic images of a scene from 3D world data. The data can be directly captured from the real-world or computer generated. The data are depth maps, normal maps, albedo, etc., organized as 1D vectors or even 2D images [231]. A deep neural network is trained from the scene data and learns how to generate a model of this scene. One of the most popular approaches to this new technique is the NeRF [159]. It uses a multilayer perceptron to approximate two volumetric fields of a 3D scene: radiance and density. We can render the scene from different viewpoints using traditional Direct Volume Rendering techniques from this volumetric data. A drawback of the original NeRF approach is the time to render and the number of data required to produce high-quality images. Works like Neural Sparse Voxel Fields [146] and KiloNeRF [202] proposed strategies to speed up the volumetric rendering of MLP-based representation, using adaptive volumetric data structures, such as octree, to enhance empty space skipping and early ray termination. Hadadan and colleagues [95] proposed what they called Neural Radiosity, a Neural Rendering approach to solving the rendering equation using neural networks. They used a neural network-based to represent the four-dimensional radiance field. Considering that .θ represents the weights and bias of the neural network, solving the rendering equation is stated as the minimization of the norm of the residual difference .$(θ ) =|| rθ (x, ωo ) ||22 , where this residual is repeatedly estimated using Monte Carlo integration, and gradients of the radiance field, computed with automatic differentiation. The training time is the main issue, varying from minutes to hours, depending on the scene’s complexity. The quality of the results to render new views of the scene is visually competitive with path tracing. Neural Lumigraph Rendering (NLR) [120] is an approach that claims to accelerate neural volume rendering, given a set of multiview images of a scene. The paper focus on facial models acquired using a camera array. They used seven multiview

98

7 Fluid Rendering

captures of a person performing different expressions. Using two neural networks to represent shape (Signed Distance Function – SDF) and appearance, NLR can export a view-dependent mesh and texture for conventional graphics pipeline rendering. Although the results are relevant for shape reconstruction and rendering time, the authors conclude that Neural Rendering cannot yet surpass real-time rendering rates. As a new and challenging field of research, Neural Rendering has had an impressive number of papers in the last few years. We recommend the reader to see [230, 231] for a comprehensive view of the field.

7.4.2 Differentiable Rendering The traditional rendering process uses geometry, material, lights, and camera properties to generate images. Integrating this process into a neural network pipeline is a challenging task as it requires that its parameters can be differentiated, which means the gradients of the rendering parameters in a scene must be computed. Differentiable Rendering (DR) techniques [119] try to build a bridge between the rendering process and the neural network pipeline, providing proper gradients from rendering. So, neural networks could optimize 3D scenes operating on their 2D projections. Considering the rendering function R that takes as input parameters .O: shape, camera, material, and lighting, and produces as output an image I containing the RGB color and the depth of each pixel. To optimize the loss function, the differential rendering process must compute the gradients .∂I /∂O. Based on these gradients, a Self-Supervison Pipeline with Differentiable Rendering takes an image as input from a neural network that produces scene parameters used by a renderer to create an RGB image. The neural network will be updated using backpropagation of the error between the image and the rendering output. The methods and techniques DR uses strongly depend on the scene’s data representation: meshes [286], voxels [259], point clouds [256], or implicit function [249]. For example, voxel-based models are almost extensions of 2D images, so differentiating such a representation is easy, although memory consumption can be problematic. Point cloud representation has a drawback that it lacks a surface, but at the same time, it is easy to render and differentiate. Differential Rendering is available in libraries such as Mitsuba 2 [182], PyTorch3D [200], Kaolin [111], and TensorFlow Graphics [244]. While Neural Rendering can generate new images from real-world data, Differentiable Rendering can modify scene parameters to produce new images to improve NR training. Works like [147, 179] bring some ideas in that direction. We recommend the reader to see [119] for an exciting survey of this new area of research.

7.5 Machine Learning in Fluid Rendering

99

7.5 Machine Learning in Fluid Rendering A common way to reduce the computational cost of fluid simulations is to diminish the number of samples or the resolution used to discretize the domain. Another way is to increase the time step used on the simulation and then apply an interpolation scheme to produce the in-between behavior of the fluid [233]. The results can be visually poor in both cases and may lose important information. In this scenario, ML techniques can improve the rendered image’s quality by increasing its resolution or transferring details from a learned visual behavior of fluids in the same conditions [124]. It also can improve the coherence of the time evolution of the fluid even with more significant time steps. Inspired by image super-resolution research [13], ML techniques can enhance the details in a simulation, producing high-resolution images based on low-resolution simulations [46, 204]. The work of Roy and Thuerey [46] was one of the first attempts to use CNNs to synthesize high-resolution smoke simulation. Based on a fluid flow volumetric dataset, they used a CNN to generate descriptors from fluid velocities and densities. A search in a high-resolution dataset is made using the learned descriptor as a key. The rendering process is a two-pass algorithm, wherein the low-resolution data are generated in the first pass. In the second pass, the high-resolution volumetric is generated and rendered. The volumetric data store only normalized densities, so the final rendering is obtained by blending the contributions of the normalized patches. A drawback of this approach is the need to store a 3D repository of 5.1 GB on secondary memory. When the fluid rendering application is not concerned with physical but only visual accuracy, approaches based on Style Transfer [113] can be used. Style Transfer was initially used to produce a painting-like image of any photograph by applying famous artistic styles [78]. Style Transfer can be used in fluid simulation to create more detailed images from low-resolution simulations. In [210], the Style Transfer approach is used to, in a postprocessing step, apply turbulency details in a low-resolution smoke simulation. The authors considered the velocity fields in the texture synthesis process. Given the low-resolution simulation target and high-resolution source simulations, the method creates a high-resolution field, using a two-step search to find a global match in the first search and the local one in the second. An optimization-based synthesis is used to generate the final field and to remove discontinuities between patches. The results are visually consistent, but the main issue of this approach is that the source animation should be at least as long as the target one. Extensions to the traditional Neural Style Transfer were proposed to include more parameters beyond the density of the fluid. Doing so makes it possible to control the smoke’s or gas’s appearance and structure, which is particularly interesting to games, movies, and art applications. Kim et al. [122] proposed what they claim to be the first Transport-based Neural Style Transfer (TNST) algorithm for volumetric smoke data. TNST introduced a semantic transfer that can produce

100

7 Fluid Rendering

new patterns and more complex motifs using a texture-based-like system. The final product of the TNST is a set of grayscale images representing the densities in the volume where the smoke is contained. A lightweight differential rendering algorithm with a single directional light and an orthographic camera was used to render the volumetric data. Although the results were visually impressive, this work did not support color information to enable appearance effects. Christen and colleagues [43] proposed an extension of the TNST that can incorporate color style transfer. Aurand et al. [12] enhance the performance of TNST by using more lightweight algorithms for advection and temporal smoothing, although keeping the restriction not to support color style transfer. The Neural UpFlow [204] uses a neural network to enhance the details of a low-resolution Lagrangian fluid simulation. This approach is based on a training phase where pairs of low- and high-resolution data are presented. The deformation between the two resolutions is captured in a 4D optical flow, which is learned by their fluidFlowNet (FFNet). The predicted displacements are used to up-resing a low-resolution particle-based simulation. The results can improve the visual quality of low-resolution liquid simulations, adding details near the surface, but effects such as sprays are poorly captured. Another way to improve rendering is the use of Generative Adversarial Networks (GANs) (Sect. 6.2.2) [110] to produce images based on some input data that, in the case of fluids, could be physical parameters captured from the simulation, such as position, velocity, or density [262]. With a few numbers of samples, the GAN can synthesize a higher-resolution version of the simulation to be rendered. GANs can also generate control parameters that modify the simulation [45] and produce more possible results for animation in games or movies.

Chapter 8

Case Studies

Abstract This chapter describes a work involving the Lagrangian simulation of fluids using convolutional networks and another work whose goal is the generation of new numerical frames in higher resolution from simulations in lower spatial resolution. The former is an adaptation of the CNN architecture to obtain a convolutional neural network capable of processing Smoothed Particle Hydrodynamics (SPH) data. The network input is composed of dynamic particle positions and velocities together with static particles that model the boundary of the fluid domain. The convolutional neural network output is a displacement array that must be used to update the positions and velocities of the fluid particles. The other focused approach is motivated by the success of generative adversarial networks (GANs) for data generation. The approach can be trained in 2D and 3D data produced using traditional solvers of the Navier–Stokes equations simulated using grid-based numerical methods. It is implemented through CNNs, using residual networks. The methodology includes a data augmentation step implemented through invariance properties (symmetry groups) of the Navier–Stokes equations. The chapter includes simulations generated with both techniques besides comparisons with numerical (ground-truth) results. Keywords Smoothed particle hydrodynamics · Convolutional neural network · Generative adversarial network · Fluid simulation · Smoke simulation · SPlisHSPlasH solver · MantaFlow software

8.1 Introduction This section describes and offers computational experiments for two works. The first one involves SPH simulation of fluids using convolutional networks [242]. The second work, presented in [278], aims at the generation of new frames in high resolution from simulations in low spatial resolution. It is motivated by the success of generative adversarial network (GAN) approaches [110]. The SPH is a grid-independent technique used for fluid simulation that was revised in Sect. 3.3. From the point of view of computer graphics, the fact that SPH © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 G. A. Giraldi et al., Deep Learning for Fluid Simulation and Animation, SpringerBriefs in Mathematics, https://doi.org/10.1007/978-3-031-42333-8_8

101

102

8 Case Studies

does not depend on a previously defined grid is an advantage over Eulerian (gridbased) methods. Scenes in a film are usually dynamic, with variable boundaries as new objects enter the scene as it evolves. In this way, methods that do not use grids can be advantageous by avoiding the extra cost of regenerating the grid whenever such changes occur. Considering these aspects, we decided to describe in Sect. 8.2 a convolutional neural network, called ConvNet, adapted to predict SPH simulations [242]. However, from the point of view of fluid simulation via neural networks, the scalar and vector fields generated by grid-based methods can be directly processed by traditional CNN architectures, which use convolution kernels directly adaptable to this type of data. In this way, the processing of numerical data by the network respects the spatial coherence of the data, avoiding ambiguities inherent to the vectorization of particle systems or even the extra cost of mapping particle neighborhoods to a regular grid in order to use networks based on convolutional methods, as performed in [242]. This fact, together with the success of generative adversarial networks (GANs) for super-resolution images [110], motivated the development of TempoGAN [278], a GAN methodology implemented through CNNs that is trained to receive numerical frames in low spatial resolution and generate high-resolution versions of these frames. Thus, the computational cost of simulating fluids using very refined grids is avoided, with acceptable results from the point of view of visual effects. The training data for TempoGAN are composed of a series of numerical simulations performed using grid-based techniques besides new numerical frames obtained through a data augmentation step, implemented using invariance properties (symmetry groups) of the Navier–Stokes equations. Also, in [278], some artistic control is obtained by scaling the initial velocity and through modification of the loss hyperparameters. The TempoGAN approach and the corresponding computational experiments are described in Sect. 8.3.

8.2 Lagrangian Fluid Simulation with Continuous Convolutions This section describes a work involving Lagrangian simulation of fluids using convolutional networks [242]. It is an adaptation of the convolution approach to obtain a neural network capable of processing SPH data. In this approach, fluids are formulated using the SPH framework reviewed in Sect. 3.3. The boundary of the fluid domain is modeled using static particles that interact with the dynamic particles representing the fluid. So, considering the discussion of Sect. 6.1, the simulator is a function .g : Rm −→ Rm that receives the configuration of the particles, given by their positions and velocities, arranged in an array .x(i) , at simulation time i, and outputs the new configuration .x(i+1) . Hence, the problem is to build a surrogate model for the function g.

8.2 Lagrangian Fluid Simulation with Continuous Convolutions

103

Fig. 8.1 Process of transforming a ball onto a cube (Adapted from [242])

We start the presentation in .R2 in order to simplify the explanation. The neural network architecture is projected to generalize discrete convolutions to the continuous domain to allow processing of SPH data. The key idea for performing this task is to apply the map .A described in [90] to transform a ball with radius R and center in .x ∈ R2 , denoted by .N (x, R), into the normalized square .[0, 1]2 . Then, following the sketch of the methodology represented in Fig. 8.1, given a point .xi ∈ N (x, R), the value .b = A (xi − x) defines a cell in the unity square. So, given a mask .h defining the filter in Fig. 8.1, we can take the corresponding element (.h (1, 1) in the filter mask of Fig. 8.1) and apply linear interpolation using the mask values nearby to compute the weight, denoted by .h (A (xi − x)) that will be used in the next steps. So, following the SPH setting, we have a point cloud .S = {x1 , x2 , . . . , xN } ⊂ R2 and a function .f : R2 −→ R with known values .fi = f (xi ), .i = 1, 2, . . . , N. Then, the SPH convolution at a position .x, given by expression (3.32), is generalized as: W ( ) h (x) = . f *a (xi , x) fi (8.1) h (A (xi − x)) , xi ∈N(x,R) where: a (xi , x) =

.

⎧( ⎪ ⎨ 1− ⎪ ⎩0,

||xi −x||22 R2

)3 , if ||xi − x||2 < R, otherwise,

that is introduced to improve the smoothing of the convolution result against different values of the radius R.

104

8 Case Studies

In the case that we have a general map .f : R3 −→ RD and consequently .fi = (fi (1) , fi (2) , . . . , fi (D)) ∈ RD , with the dimension .D > 1, the generalization of expression (8.1) is straightforward given by:

.

( ) f *h (x) =

D W

W

a (xi , x) fi (t) h (A (xi − x)) ,

(8.2)

xi ∈N(x,R) t=1

where in this case .A is a map from .N (x, R) ⊂ R3 to the unity square .[0, 1]3 and the filter h has dimensions .4 × 4 × 4 since we are considering only square/cubic filters (see Fig. 8.1). Likewise in the CNN, the filter values at the regular grid are the learnable parameters. The whole architecture is named ConvNet in [242]. Part of its input in n∗ step n is the intermediate positions .xn∗ i and velocities .vi of fluid particles computed by: n vn∗ i = vi + Ataext ,

(8.3)

.

( n∗ .xi

=

xni

+ At

vni + vn∗ i 2

) ,

(8.4)

where .aext is the acceleration that accounts for external and internal forces and .vni is the actual velocity of the particle i. Consequently, the intermediate positions and velocities do not consider the interactions particles–particles or particles–scene. So, in the simulation step n, ConvNet receives the intermediate of ]} { n∗ [ configuration n∗ . Besides, each fluid particle of the SPH model arranged as .pn∗ = x , 1, v i i i static particles used to model the {domain} boundary compose also the input. These particles are represented by .sj = yj , nj where .yj is the position of static particle j and .nj is the normal that defines the local geometry. The ConvNet output is the corrections that must be added to the intermediate positions .xn∗ i to account for the collision dynamics: .

) [ n ] ({ } n∗ n∗ Ax1 , Axn2 , . . . , AxnN = ConvNet pn∗ 1 , p2 , . . . , pN , {s1 , s2 , . . . , sM } . (8.5)

Hence, the positions and velocities for simulation step .n + 1 are computed out of the neural network as: n xn+1 = xn∗ i + Axi , i

(8.6)

− xni xn+1 i , At

(8.7)

.

vn+1 = i

.

for .i = 1, 2, . . . , N. The ConvNet architecture is composed of a sequence of calls to the continuous convolutions (CConv) operator which, for 3D data, computes

8.2 Lagrangian Fluid Simulation with Continuous Convolutions

105

expression (8.2) at intermediate positions .xn∗ i , .i = 1, 2, . . . , N . Each call of CConv operator is specified as follows: OutF eatures =

.

.

= CConv(I nF eatures, I npP ositions, OutP ositions, f ilters, KernelSize), (8.8)

with the} where .I nF eatures = {f1 , f2 , . . .} denotes the array of features associated { n∗ , . . . xn∗ points in the array I npP ositions. The array .OutP ositions = xn∗ , x N 1 2 encompasses the points where expression (8.2) is evaluated. The f ilters is the number of convolution kernels .h1 , h2 , . . . , hf ilters , each one with .KernelSize = 4 × 4 × 4. The ConvNet presented in [242], is depicted in Fig. 8.2. The first level of the network is composed of two independent CConv calls, the first one processes the static particles with .fi = ni , .I npP ositions = {y1 , y2 , . . . , yM }, .f ilters = 32, and .KernelSize = (4, 4, 4). Also, it is used the ReLU activation function applied element-wise to generate the OutF eatures array. So, following expression (8.2): OutFeatures1,1 (j, l) = ReLU ⎛

.

⎜ ×⎝

⎞ W

3 W

) ( ( ))⎟ ( ni (t) a yi , xn∗ hj A yi − xn∗ ⎠, l l

t=1 yi ∈N(xn∗ l ,R )

(8.9) where .j = 1, 2, . . . , 32, l = 1, 2, . . . , N, with .hj being the weight array obtained through the procedure described in Fig. 8.1, and the sub-indices ‘.1, 1’ means the first feature map of the first level. The second CConv call, which processes the[ dynamic particles, is analo] gous to expression (8.9), but in this case .fi = 1, vn∗ and .I npP ositions = i OutP ositions, generating the feature map .OutF eatures1,2 . The last component [of then∗first ] level is a fully-connected (FC) neural network that receives as input .fi = 1, vi , .i = 1, 2, . . . , N , and outputs N feature vectors in .R32 that are also processed by ReLU to generate a feature map .OutF eatures1,3 . The column-wise concatenation .OutF eatures1 = OutF eatures1,1 |OutF eatures1,2 |OutF eatures1,3 generates the feature vector with size 96 for each particle that is processed by the next level, as indicated in Fig. 8.2. From Fig. 8.2, we notice that the second level of ConvNet encompasses a CConv and an FC neural network. Returning from expression (8.9), the former receives as .I nF eatures = OutF eatures1 , .I npP ositions = OutP ositions = {arguments } n∗ , . . . xn∗ , and .f ilters = 64. The KernelSize is kept the same from xn∗ , x N 1 2 the first level. So, expression (8.9) generates a feature vector .OutF eatures2,1 ∈ R64 and the FC generates .OutF eatures2,2 ∈ R64 . Following Fig. 8.2, the

106

8 Case Studies

Fig. 8.2 Fed-forward information processing in ConvNet architecture with CConv arguments indicating the I nF eatures and I npP ositions (Adapted from [242])

output (of the second level is the element-wise ) sum followed by ReLU activation: ReLU OutF eatures2,1 + OutF eatures2,2 = OutF eatures2 . Besides CConv and FC, the third level involves also a skip connection such that ( the feature map .OutF eatures3) is given by: .OutF eatures3 = ReLU OutF eatures3,1 + OutF eatures3,2 + OutF eatures2 . The last level is formed by a CConv and an FC, with element-wise addition, to output the corrections n .Ax as stated in expression (8.5). i The loss function compares the ground-truth particle’s positions at time steps ˆ n+1 .n + 1 and .n + 2, denoted by .x and .xˆ n+2 , with the predicted positions calculated i i by ConvNet and expression (8.6): .

L=

N W

.

i=1

N || ||γ W || ||γ || || || || φin+1 ||xn+1 − xˆ n+1 φin+2 ||xn+2 − xˆ n+2 || + || , i i i i 2

i=1

2

(8.10)

where .γ is set to .0.5 in the original implementation of ConvNet and .φin+1 is used to emphasize terms that correspond to particles with fewer fluid neighbors analogous for .φin+2 . The idea is to properly weigh particles nearby the boundary of the domain since the boundary conditions are very determinant of the flow patterns. For instance: ) ( 1 || ( k )|| k .φi = exp − |N xi , R | , c where .k = n + 1, n + 2, .|·| means the number of particles inside the argument. In this case, the neighborhood .N with radius R, that is set as .R = 0.225 m in [242]. Besides, .c = 40, that is, the mean number of particle neighbors across the experiments performed in [242]. The training data are a set of simulations performed using SPH and Navier– Stokes equations. Solvers like SPlisHSPlasH [219], PySPH [199] and DualSPHysics [59], among others, can be used in this task. A simple data augmentation based on global rotations is implemented to avoid the necessity of huge datasets. Specifically, each simulation is subdivided into blocks of three consecutive numerical frames .((0, 1, 2), (1, 2, 3), . . .) in order to fit the way the loss function in expression (8.10) is computed; that means, when the network receives a block .(k, k + 1, k + 2) it takes the numerical frame k, processes it to predict the

8.2 Lagrangian Fluid Simulation with Continuous Convolutions

107

displacements .Axki that are used to update the positions and velocities of dynamic particles at simulation time .k + 1 through expressions (8.6)–(8.7). Next, the combination of ConvNet and expressions (8.6)–(8.7) takes the SPH data in .k + 1 and outputs the particles’ positions and velocities at .k + 2. Now, the loss function in equation (8.10) can be calculated using the ground-truth .xˆ k+1 and .xˆ k+2 , and the i i k+1 k+2 predicted positions .xi and .xi . During the above feed-forward computation, ConvNet processes dynamic as well as static particles augmented with local geometry of the boundary domain encompassed in the normal direction. In this way, during the training, the model can learn how to simulate fluids in arbitrary domain geometries.

8.2.1 Comparison with SPlisHSPlasH Simulation This section aims to compare ConvNet simulations and SPH results obtained through the SPlisHSPlasH software [219]. The SPlisHSPlasH was used to generate the training set of simulations applied to generate the ConvNet model available in GitHub1 that is used in the tests of this section. We take this pretrained model without performing any fine-tuning. The simulation method used in the SPlisHSPlasH implementation is the DFSPH, described in Sect. 3.3.3. For the first example, we decided to simulate a fountain whose setup is pictured in Fig. 8.3a. The domain is a box with dimensions .10 × 10 × 3 m in the x, y, and z directions, respectively, as represented in Fig. 8.3a. The fountain has an emitter that is placed in the center of the domain bottom (Fig. 8.3b). The emitter can be defined as a shape that adds fluid particles to the simulation, with a given velocity and direction, at a certain flow rate. In this example, the emitter is a cylinder with a height .0.225 m and a diameter .0.9 m. In the SPlisHSPlasH implementation, an emitter is a physical object that particles interact with and can have two shapes: cylinder and square. The particles are created inside the emitter, as pictured in Fig. 8.3c–d, and launched forward in their normal direction. The SPlisHSPlasH simulation parameters are given in Table 8.1 that divides the information into Model and Numerical parameters. The former are discussed in the Sect. 3.3 while the latter are internal parameters of the numerical solver. The SPlisHSPlasH package version 2.9.2, allows SPH simulation with boundary handling using static particles (Fig. 8.3e), following the methodology described in [8], which is revised in Sect. 3.3.4, as well as the volume maps concept [21], revised in Sect. 3.3.2. For other possibilities see [18]. Since the training set of ConvNet was generated through SPlisHSPlasH with boundaries represented as particles, we first decided to perform the fountain simulation with the interaction between static (boundary) and dynamic particles modeled 1 https://github.com/isl-org/DeepLagrangianFluids

108

8 Case Studies

Table 8.1 Parameters of SPlisHSPlasH simulations Model Parameters Simulation Method Particle Radius Rest density of the fluid Surface Tension Coefficient for the viscosity force computation Stiffness coefficient of the equation of state Exponent in the equation of state Numerical Parameters Max. allowed time step size Maximal number of iterations of the pressure solver (default: 100) Maximal density error in percent which the pressure solver tolerates (default: 0.01) Maximal number of iterations of the divergence solver Maximal divergence error in percent which the pressure solver tolerates Velocity Update Method (Implicit solvers) Max. iterations of the viscosity solver (Implicit solvers) Max. error of the viscosity solver

DFSPH 0.025 m 1000 g/cm.3 0.2 0.01 50.000 7 0.005 100 0.01 100 0.1 First Order Update 200 0.05

by [8]. Next, we also apply volume maps aiming to compare the performance of the ConvNet against fountain simulation with another boundary handling. In the case of volume maps, the density map and signed distance field resolutions are given by .40 × 30 × 15 and .64 × 64 × 64, respectively. Considering these resolutions and the maximum number of iterations of pressure and divergent solvers reported in Table 8.1, the computational cost of SPlisHSPlasH may increase for larger numbers of particles, as we will see later. Besides, SPlisHSPlasH uses an adaptive time step scheme steered by the CFL condition [48]. Despite this, numerical data are exported with a constant rate of 50 frames per second along the simulation execution. The obtained sequence is composed of 1250 frames that will be used for comparison with the ConvNet simulation that is performed with .At = 0.02s in expression (8.7). Here, the velocity, the Diameter/Height of the cylinder, and the time step will dictate how many particles will be generated at each frame. Table 8.2 shows the influence of the cylinder parameters in the total number of particles generated at each frame. The SPlisHSPlasH setup exports 50 frames per second, and the time step (.At) is variable along the simulations. Hence, all the statistics of this section are performed considering the exported frames. Likewise for SPlisHSPlasH, in the ConvNet implementation , the boundary of the domain is modeled through static particles. Figure 8.3d–e show the fountain domain with boundaries sampled using .310.266 particles. The domain dimensions are the same presented in Fig. 8.3a. The emitter can assume any shape represented by a given mesh, which also is converted to static particles, and the velocity can be in any direction. In our ConvNet simulation, the emitter geometry is also a cylinder like

8.2 Lagrangian Fluid Simulation with Continuous Convolutions

109

Table 8.2 Relation of the emitter’s dimensions and particle velocity to the particles generated per frame by SPlisHSPlasH and the total number of particles emitted considering 10s of emission Shape Cylinder

Diameter/Height 0.848/0.225 m

0.9/0.225 m

0.956/0.225 m

Fig. 8.3 Fountain setup: (a) Domain visualization, where .a = 10 m, .b = 3 m, .c = 10 m, .d = 4.55 m, .e = 0.225 m, .f = 0.9 m. (b) Fluid domain with particles being emitted in the center bottom. (c) Zoom in the emitter region. (d) Boundary represented through static particles and emitter filled with fluid particles. (e) Zoom in the corresponding emitter region (From Author)

Velocity (m/s) 4 5 6 4 5 6 4 5 6

Particles per frame 137–274 274 274–411 156–312 312 312–468 177–354 354 354–531

Total Particles 109.600 137.000 164.400 124.800 156.000 187.200 141.600 177.000 212.400

110

8 Case Studies

Table 8.3 Influence of the emitter’s dimensions to the number of particles generated per frame by the ConvNet (fourth column). The total numbers of particles in the last column, counted after 10 s of emission, are variable due to ConvNet errors to reproduce boundary conditions, generating particle loss Shape Cylinder

Diameter/Height 0.848/0.225 m

0.9/0.225 m

0.956/0.225 m

Velocity (m/s) 4 5 6 4 5 6 4 5 6

Particles per frame 274

312

354

Total Particles 136.998 136.999 137.000 155.995 155.997 156.000 176.997 176.996 177.000

in Fig. 8.3. The particles are generated inside the volume of the emitter as pictured in Fig. 8.3c–e, where we see the cylinder filled with particles, similarly placed as in SPlisHSPlasH. Here, particles are added once per frame. Hence, the flow rate does not depend on the particle velocity, as we can see in column Particles per frame of Table 8.3. In fact, the flow rate in the emitter depends on the particle’s size and the emitter’s shape. Since the particle radius is fixed in .0.025 m, we show in Table 8.3 just the influence of the dimensions of the cylinder emitter for the ConvNet in the number of particles generated per frame. In the SPlisHSPlasH simulations, each particle is emitted with velocity in the z-direction; so .v0 = (0, 0, v0 ). The software exports one simulation frame for each .0.02 s of simulation. For instance, in the case .v0 = (0, 0, 5) and Diameter/Height equal to .0.9/0.225 m, reported in Table 8.2, the number of particles per frame exported is 312. Hence, at the end of 10s of emission, the total number of dynamic particles emitted by the fountain is .156.000 particles in that setup. However, considering that the time step .At is variable, the particle dimensions, and the emitter dimensions, the number of particles per frame sent in the domain may vary when changing the velocity as reported in Table 8.2. In fact, as we can see in Table 8.2, when the emitter has diameter = .0.9 m and height = .0.225 m, the amount of emission is 312 per frame for velocity 5 m/s while when the initial particle velocity is 4 m/s, the minimum amount of emission is 156 particles in a frame, and the maximum is 312, which gives us an average of .249.60 particles per frame. For ConvNet simulation, particles are emitted with the same velocity applied for SPlisHSPlasH. However, the time step is fixed in .At = 1/50 = 0.02 s and, consequently, the number of particles that the emitter sends to the domain at each step of the simulation depends only on the emitter geometry and particle size. As the particle size is fixed, the emitter size determines the number of particles that the emitter sends to the domain per time step, as shown in Table 8.3, which reports in the last column the total number of particles counted after 10 s of simulation. We

8.2 Lagrangian Fluid Simulation with Continuous Convolutions

111

shall emphasize that this number is variable due to ConvNet errors to reproduce boundary conditions generating a loss of particles. Figure 8.4 shows some snapshots of both ConvNet and SPlisHSPlasH simulations for emission velocity .v0 = (0, 0, 5) and emitter dimensions .0.9/0.225 m. We have noticed that the fluid evolution in the ConvNet is slower than in the SPlisHSPlasH simulation. So, the snapshots for ConvNet correspond to larger time steps than the corresponding ones in the SPlisHSPlasH. The visual comparison shows some asymmetries in the ConvNet flow that are more perceived at the beginning of the simulation. However, we must take into account that the training set of ConvNet does not include a setup similar to the considered fountain. Despite the mentioned problems, the neural network reproduced important visual features of the flow. A quantitative comparison is possible through the expression: dn =

.

N || n || 1 W ||xi − xn ||2 , min n n x ∈X N

(8.11)

i=1

n n .x is the position of particle i of the SPlisHSPlasH simulation and .X is the where i array of particles positions predicted by ConvNet. Moreover, we also apply SPlisHSPlasH with boundary handling based on density maps [21], revised in Sect. 3.3.2, for performance comparison with ConvNet although, depending on the influence of the boundary handling methodology for the flow patterns, it is expected that ConvNet simulation is closer to SPlisHSPlasH with boundary represented with static particles. Figure 8.5 shows the evolution of the distance (8.11) along the simulation steps for .v0 = (0, 0, 5) and Diameter/Height of emitter equal to .0.9/0.225 m (Tables 8.2–8.3). As expected, the difference between ConvVet and SPlisHSPlasH simulations is more intense for the blue plot that shows the evolution of distance (8.11) when considering the volume maps for boundary interaction. Despite this, the profiles of the plots are similar for most of the numerical frames generated in the simulation. Considering the orange plot of Fig. 8.5, we take the two peaks in the figure that occur in frames 62 and 89 and show the corresponding snapshots for SPlisHSPlasH and ConvNet in Fig. 8.6a–d, respectively. Regarding the blue plot, we noticed two peaks in frames 63 and 111. The corresponding frames are visualized in Fig. 8.6e–h. The comparison between these figures emphasizes the fact that the ConvNet flow is slower than the SPlisHSPlasH one, with patterns less symmetrical for the former. Despite this, the neural network is capable of reproducing important visual features of the fluid motion. From the viewpoint of fluid dynamics, two important parameters for this case study are the flow rate per frame in the emitter and the velocity of the particles when entering the fluid domain, denoted by .v0 . Tables 8.2 and 8.3 show statistics for some experiments that use the same geometric setup of Fig. 8.3 but vary those parameters. Figure 8.7 shows some snapshots from those simulations. We notice

112

8 Case Studies

Fig. 8.4 Snapshots of fountain simulation: (a), (c), (e), (g) SPH simulation through SPlisHSPlasH at .t = 51, 67, 123, 192, respectively. (b), (d), (f), (h) ConvNet simulation at time steps .t = 65, 86, 176, 251, respectively (From Author)

8.2 Lagrangian Fluid Simulation with Continuous Convolutions

113

that the neural network can deal with the variation of those parameters generating visual patterns close to the ground truth obtained by SPlisHSPlasH. Table 8.4 shows the average simulation time (in bold) per frame required for 18 simulations with duration 5 and 10 s, generated using different emitter dimensions and emission velocities, as indicated in the second and third columns of the table. The results of Table 8.4a indicate that SPlisHSPlasH is much more computationally involved than ConvNet, whose simulation times are presented in Table 8.4b. Hence, despite the differences observed between the SPlisHSPlasH and ConvNet simulations, the fact that the ConvNet needs very much less computational time can be explored for visual effects, mainly in situations where the scale of the imperfections does not influence the perception of the scene. The next example uses the geometric setup shown in Fig. 8.8. We have a box with two floors and a hole, as pictured in Fig. 8.8a, that allows the fluid to flow from the second floor to the first one . The initialization is performed by using two identical fluid columns on the first floor, a double dam breaking, whose dimensions are indicated in Fig. 8.8b. The double dam breaking starts evolution with zero as the initial velocity. Each column of dynamic particles in Fig. 8.8b has .102.396 particles comprising a total of .204.792 dynamic particles. The boundary handling is implemented through the method presented in [8], which uses static particles for modeling the domain walls (Fig. 8.8c). We have used .301.919 static particles to model the boundaries of the domain. The SPlisHSPlasH parameters are the same reported in Table 8.1. Likewise in the fountain example, SPlisHSPlasH uses an adaptive time step scheme steered by the CFL condition [48], and numerical data are exported with a constant rate of 50 frames per second along the simulation execution generating a sequence of

Fig. 8.5 Evolution of distance (equation 8.11) along the SPlisHSPlasH simulation with boundary interaction computed using static particles following [8] and with boundary interaction calculated using volume maps [21] (From Author)

114

8 Case Studies

Fig. 8.6 Results obtained using static particles [8]: (a)–(b) Frame 62 from SPlisHSPlasH and ConvNet simulations, respectively. (c)–(d) Snapshots from frame 89 from SPlisHSPlasH and ConvNet simulations, respectively. Results obtained using volume maps [21]: (e)–(f) Frame 63 from SPlisHSPlasH and ConvNet simulations, respectively. (g)–(h) Snapshots from frames 111 from SPlisHSPlasH and ConvNet simulations, respectively (From Author)

8.2 Lagrangian Fluid Simulation with Continuous Convolutions

115

Fig. 8.7 (a)–(b) Frames 43 and 54 from SPlisHSPlasH and ConvNet simulations, respectively, with velocity .v0 = (0, 0, 4) and emitter dimensions .0.848/0.225 m. (c)–(d) Analogous setup but considering frames 254 from SPlisHSPlasH and ConvNet. (e)–(f) Frames 61 and 80 from SPlisHSPlasH and ConvNet simulations, respectively, with velocity .v0 = (0, 0, 6) and emitter dimensions .0.956/0.225 m. (g)–(h) Analogous setup but considering frames 168 and 231 from SPlisHSPlasH and ConvNet (From Author)

10 s

Fountain simulation 5s

Emitter diameter/height 0.848/0.225 m 0.9/0.225 m 0.956/0.225 m 0.848/0.225 m 0.9/0.225 m 0.956/0.225 m

Velocity 4 m/s 1468 ms 1669 ms 1981 ms 2407 ms 2861 ms 3290 ms (a) 5 m/s 1981 ms 1956 ms 2299 ms 3380 ms 3425 ms 4330 ms

6 m/s 2157 ms 2434 ms 2795 ms 3640 ms 5287 ms 6780 ms 10 s

Fountain simulation 5s

Emitter diameter/height 0.848/0.225 m 0.9/0.225 m 0.956/0.225 m 0.848/0.225 m 0.9/0.225 m 0.956/0.225 m

Velocity 4 m/s 963 ms 108 ms 123 ms 166 ms 200 ms 239 ms (b)

5 m/s 938 ms 112 ms 120 ms 174 ms 197 ms 240 ms

6 m/s 935 ms 110 ms 124 ms 166 ms 206 ms 225 ms

Table 8.4 (a) Average simulation time (in bold) per frame, in milliseconds (ms), required using SPlisHSPlasH with boundary interaction implemented through [8]. (b) Average simulation time (in bold) per frame, in milliseconds (ms), required for ConvNet

116 8 Case Studies

8.2 Lagrangian Fluid Simulation with Continuous Convolutions

117

Fig. 8.8 Double dam breaking setup considering .204.792 dynamic particles: (a) Domain visualization (no fluid particles), where .a = 4 m, b = 8 m, c = 8 m, d = 2 m, e = 2 m, f = 3.2 m. (b) Fluid domain with two columns of particles positioned on the second floor, where .g = 1.6 m, h = 2 m, i = 4 m. (c) Zoom in the region nearby the hole with the boundary represented through the static particles (From Author)

1250 frames that will be used for comparison with the ConvNet simulation that is performed with .At = 0.2 s in expression (8.7). Figure 8.9 shows some snapshots of both ConvNet and SPlisHSPlasH simulations for the setup described in Fig. 8.8. Likewise in the fountain example, we have noticed that ConvNet flow is slower than the SPlisHSPlasH one. Consequently, the correspondence between frames of both simulations occurs in different time steps. However, the visual comparison shows that despite some differences, the ConvNet simulation is very convincing in this case study. We also present the plot in Fig. 8.10 showing the evolution of distance .d n computed by expression (8.11) when considering the boundary interaction computed using static particles [8] and volume maps [21]. We observe that the differences between the ConvNet and SPlisHSPlasH simulations are more intense between the frames 0 and 400, following the same pattern for both boundary interaction techniques. If compared with the fountain simulation, we notice that the global maximum of the orange plot in Fig. 8.5 is .0.249 m while the global maximum of the corresponding plot in Fig. 8.10 is .0.168 m. In the case of the blue plot, the maximum in Fig. 8.5 is .0.26506 m (frame 111), while in Fig. 8.10 it takes the value .0.17559 m (frame 52). Hence, the neural network is more precise in the double dam breaking simulation. Taking into account the plots of Fig. 8.10, we take the frames 52 and 101 corresponding to the two principal peaks for the blue and orange curves and

118

8 Case Studies

Fig. 8.9 Snapshots of double dam breaking simulation: (a), (c), (e), (g) SPH simulation through SPlisHSPlasH at .t = 35, 60, 80, 130, respectively. (b), (d), (f), (h) ConvNet simulation at time steps .t = 55, 86, 130, 150, respectively respectively (From Author)

present the snapshots for SPlisHSPlasH and ConvNet in Fig. 8.11a–f . Despite the differences highlighted in the plots of Fig. 8.10, we observe in Fig. 8.11a–c that the main reason for the first peak is that the quantity of particles that have entered the first floor is smaller in the case of ConvNet simulation. Despite this, the flow patterns on the second floor are very convincing from the visual perception. The same comments can be given for Fig. 8.11d–f.

8.2 Lagrangian Fluid Simulation with Continuous Convolutions

119

Fig. 8.10 Evolution of distance (8.11) along the SPlisHSPlasH simulation with boundary interaction computed using static particles following [8] and volume maps [21] (From Author)

Fig. 8.11 (a) Frame 52 obtained using SPlisHSPlasH with [8]. (b) Frame 52 from SPlisHSPlasH with volume maps [21]. (c) Snapshots from frame 52 from ConvNet. (d) Snapshot from frame 101 generated using SPlisHSPlasH with [8]. (e) Visualization of frame 101 from SPlisHSPlasH with volume maps [21]. (f) Snapshots from frame 101 simulated with ConvNet (From Author)

Table 8.5 Double dam break simulation. Second column: Average simulation time in seconds required using SPlisHSPlasH with boundary interaction implemented through [8]. Third column: Average simulation time in seconds required for ConvNet Total particles 50.165 100.661 204.792

SPlisHSPlasH 766 ms 1394 ms 2630 ms

ConvNet 785 ms 145 ms 322 ms

120

8 Case Studies

Fig. 8.12 The same snapshots of fountain simulation presented in Fig. 8.4, now using screen-space rendering techniques discussed in Sect. 7.2.2

With respect to the average simulation time per frame required for SPlisHSPlasH and ConvNet when changing the number of particles in the double dam breaking, Table 8.5 shows the obtained results considering the setup analogous to Fig. 8.8. The reported results demonstrate that SPlisHSPlasH is much more computationally involved than ConvNet when the number of particles goes beyond .100.000.

8.3 TempoGAN: GAN for Super-resolution Fluid Flow

121

8.2.2 Rendering Liquids Now, we shall explore visualization effects using one particular rendering technique described in Chap. 7: screen-space rendering Sect. 7.2.2. In particular, we implemented the algorithm described in [237]. Using as input the two simulations shown in Fig. 8.4, where the fluid is a liquid, simulated using SPH and ConvNet, and the time steps used are the same. In Fig. 8.12, a simple Phong Shading model was used, solving the render equation (Sect. 7.1) using only directions that reach a light source. Although the thickness of the water is computed to produce the refraction effect, it is barely noticed as the background is a solid color, and there is no object behind or at the bottom of the fluid. On the other hand, the refraction can be more perceptible in Fig. 8.13, where an environment mapping (a set of six textures mapped over the faces of a cube that surrounds the scene) is applied. Now the change of direction of the light promoted by the liquid is visible, as the grass pattern over the fluid is deformed. As we can see comparing Figs. 8.4 and 8.12, the rendering techniques improve the perception of the shape of the fluid, the regions containing splashes and droplets. On the other hand, if the fluid is part of a more complex scene, like in a movie or game, effects like surface reflection and refraction are essential to integrate the fluid into the scene. As performance is concerned, the render of simulations in Figs. 8.12 and 8.13 achieve an average performance of 60 fps (.16.6 ms/frame) in a middle-cost notebook (AMD Ryzen 6000 series, 16GB RAM, Nvidia RTX 3050 (4GB GDDR6). All rendering steps are executed on the GPU using shaders, using standard algorithms without any optimizations or tuning.

8.3 TempoGAN: GAN for Super-resolution Fluid Flow This section describes a GAN-like approach that processes low-resolution data generated in Eulerian simulation of fluids aiming super-resolution [278]. The GAN architecture, named TempoGAN, is implemented through CNNs to train the generator to output temporally coherent density fields represented in high-resolution grids. In this way, we shall introduce the following notation: • Velocity and density represented at time t in low-resolution: .vtx , ρxt . • Velocity and density at time t represented in hig-resolution: .vty , ρyt . So, considering the discussion of Sect. 6.1, the trained generator of TempoGAN is a CNN model that receives as input low-resolution fields at simulation time t, say density .ρxt and velocity .vtx and outputs the high-resolution version of the density data, denoted by .ρyt .

122

8 Case Studies

Fig. 8.13 The same snapshots of fountain simulation presented in Fig. 8.12, now using screenspace rendering techniques discussed in Sect. 7.2.2 and appling an environment mapping to simulate an external environment

The training data is composed of nonstationary velocity and density fields, represented in a low-resolution grid, with the corresponding high-resolution reference data generated using grid-based approaches. The GAN structure is depicted in Fig. 8.14. In the training stage, besides the usual spatial discriminator, denoted by .Ds , the approach includes a temporal discriminator named .Dt . This is motivated by the fact that the input and output are fields that vary in space and time. So, in the adversarial training, the system receives as input a set of nonstationary density fields, represented in a low-resolution grid, with the corresponding high-resolution reference data, besides nonstationary velocity fields.

8.3 TempoGAN: GAN for Super-resolution Fluid Flow

123

Fig. 8.14 High-level overview of TempoGAN approach with order of millions of trainable parameters. (Adapted from Source [278])

In Fig. 8.14, the black and dashed rectangles indicate different compositions for input data that occur during training. To set the loss function, we first discretize the continuity equation (expression (3.5)) to obtain: ) ( ρ (x, t + At) ≈ ρ (x, t) − At∇ · (ρ (x, t) v (x, t)) ≡ A ρxt , vtx .

.

(8.12)

So, we take the data inside the black rectangle of Fig. 8.14 to define: .

( ) )} { ( t t+1 t+1 - = A ρyt−1 , vt−1 , A ρ , −v , ρ , Y y y y y A

(8.13)

while with the fields inside the red rectangle we can compute: .

( ( ) ) ) )} { ( ( ) ( - = A G ρxt−1 , vt−1 , vt−1 , G ρxt , vtx , A G ρxt+1 , vt+1 , −vt+1 . G x y x y A (8.14)

Using the above notations and definitions, the losses associated to the TempoGAN can be stated as : [ ( )] [ ( ( ))] LDt (Dt , G) = −E log Dt Y A − E log 1 − Dt GA ,

.

(8.15)

[ ( )] ( ( )))] [ ( LDs (Ds , G) = −E log Ds ρxt , ρyt − E log 1 − Ds ρxt , G ρxt , vtx , (8.16) ( t [ ( ( ))] [ ( t t ))] − E log Dt G .LG (Ds , Dt , G) = −E log Ds ρx , G ρx , vx A +

.

124

.

+

8 Case Studies

W j

|| ( || || ( ( )||2 )) ) ( || || || j || Ej λf ||F j ρxt , G ρxt , vtx − F j ρxt , ρyt || + λL1 E ||G ρxt , vtx − ρyt || , 2

1

(8.17) where .F j is the activation function of layer j of the spatial discriminator .Ds and .Ej the expectation at layer j . The data undergoes a physics-aware data augmentation procedure based on symmetry groups of the Navier–Stokes equations. Specifically, given a velocity field .vty ≡ v (y, t) we generate new vector fields through the transformation ( −1 ) .v (y, t) = Av A y, t where A is formed by combining translation, uniform scaling, reflection, and rotation randomly chosen in the range .[0.85, 1.15] for scaling and in the range .[−90◦ , 90◦ ] for rotation. The training set .Dtr encompasses a set of high-resolution simulations performed using the MantaFlow software [235], which applies finite-difference schemes to discretize the Navier-Stokes equations. From those simulations the fields .ρyt , .vty are taken. The low-resolution fields .ρxt and .vtx that also enter the training data are obtained through down-sampling from the references .ρyt and .ρyt , respectively. In this way, taking into account the expressions (8.13)–(8.14), each nonstationary field is partitioned into 3-uplas corresponding to .t − 1, t, t + 1, for .t = 1, 2, . . . to fit the calculation of the expressions (8.13)–(8.14). Then, data augmentation can be performed as explained above and the losses in equations (8.15), (8.16), and (8.17) are calculated generating functions that depend only on the neural network weights and bias (array .z in Algorithm 1). Next, the gradient computation step takes place to update .z. The Algorithm 3 summarizes the whole procedure. It is an adaptation of Algorithm 1 where we have more than one Loss . To save computational time, the training stage is implemented through multiple optimization steps for each loss component stated in expressions (8.15), (8.16), and (8.17), as observed in expressions (8.19), (8.21), and (8.23) of Algorithm 1. Regarding the neural network architecture, the best results were obtained when implementing the generator based on the residual networks (res-nets) [141]. Details about the generator architecture are given in Fig. 8.15. As observed in Fig. 8.15a, there are two types of residual blocks. The traditional one (on the left) is implemented using batch normalization and denoted by .RB − BN . The second one (on the right) is implemented without the batch normalization step and is denoted by .RB −W N . In both cases, considering 3D data, the expression .Ci (mi , ni , ki ) means convolution with filter size .ni × ni × ni , .m3i is the output resolution and .ki denotes the number of output channels of convolutional layer i (analogously for 2D case). In Fig. 8.15b, the generator input has size .16 × 16 × 16 and four channels (.in(16, 4)) since it receives density and velocity fields. Following [185], the first processing layer of the generator applies the nearest-neighbor interpolation over the input. This operation is denoted by .NI (64, 4) in Fig. 8.15b, indicating that, in this example, the output is a tensor with resolution .64 × 64 × 64 and four channels. That

8.3 TempoGAN: GAN for Super-resolution Fluid Flow

125

Algorithm 3 Minibatch Stochastic Gradient Descent Training for TempoGAN Input: Training set Dtr , minibatch sizes ςs , ςt , ςG η, kT , kDs , kDt , kG > 0, Initialization: z0 ∈ Rk , t = 0, p ← z0 ;, while (t ≤( kT ) do ) while i ≤ kDs do Sample minibatch B ⊂ Dtr , with size ςs , Compute the spatial discriminator Loss (8.16) over the minibatch B : LDs (z; B) = −

1 ςs

W

[[

]] ( )] [ log Ds ρx , ρy + log (1 − Ds (ρx , G (ρx , vx )))

(8.18)

(ρx ,ρy ,vx )∈B

Learning rule. Update parameter vector: zt+1 = zt − η∇z LDs (p; B) ,

(8.19)

p ← zt+1 ; i ← i + 1 end while i = 0; ( ) while i ≤ kDt do Sample minibatch B ⊂ Dtr , with size ςt , Compute the temporal discriminator Loss (8.15) over the minibatch B : LDt (z; B) = −

1 ςt

W (

Y

-

[[ )

A ,GA

))]] ( )] [ ( ( log Dt Y A + log 1 − Dt GA

(8.20)

∈B

Learning rule. Update parameter vector: zt+1 = zt − η∇z LDt (p; B) ,

(8.21)

p ← zt+1 ; i ← i + 1 end while i = 0; while (i ≤ kG ) do Sample minibatch B ⊂ Dtr , with size ςG , Computing the generator Loss (8.16) over the minibatch B : LG (z; B) =

+

W ( ρx ,vx ,ρy ,G

A

)

∈B

W (

) ∈B

))]} { [ ] [ ( ( + − log Ds (ρx , G (ρx , vx )) − log Dt G A

⎧ ρx ,vx ,ρy ,G ⎫ A ⎨W || || || ⎬ ( )|| ||2 j || j j || || λf ||F (ρx , G (ρx , vx )) − F ρx , ρy || + λL1 G (ρx , vx ) − ρy 1 . ⎩ ⎭ 2 j

(8.22)

Learning rule. Update parameter vector: zt+1 = zt − p ← zt+1 ; i ← i + 1 end while end while z*opt ← p, Output: Solution z*opt

η ∇z LG (p; B) , ςG

(8.23)

126

8 Case Studies

Fig. 8.15 (a) Residual blocks used in TempoGAN generator. (b) Details of the generator network (From Author)

figure shows that the generator is composed of four residual blocks, the first three ones of type .RB − BN and the last one being a .RB − W N block. The tests have shown that the discriminator’s architectures represented in Fig. 8.16 are less crucial . In that figure .Conv (m, n, k) means a convolution .C (m, n, k) followed by batch normalization and leak ReLU as activation function. So, as described in Fig. 8.16, discriminators are always constructed with four convolutional layers with leaky ReLU as activation function. The last block of the networks is a fully connected layer (FC) to calculate the final score. The spatial discriminator .Ds is depicted in Fig. 8.16a. As it is a conditional discriminator, it receives as input the low-resolution density (.in (16, 1)) as well as high-resolution density fields (.in (64, 1)) to be classified between fake or real data. So, the network processes the concatenation .N I (64, 1) |in (64, 1). On the other hand, the input of the temporal discriminator .Dt pictured in Fig. 8.16b, is .in (64, 3) because it is composed of three high-resolution density fields: three channels each one with resolution .64 × 64 × 64.

8.3 TempoGAN: GAN for Super-resolution Fluid Flow

127

Fig. 8.16 (a) TempoGAN spatial (From Author). (b) Temporal discriminator of TempoGAN (From Author)

8.3.1 TempoGAN Results In this section, we show super-resolution results obtained with the TempoGAN approach described in Sect. 8.3, trained with a dataset composed of high-resolution simulations generated with MantaFlow software [235] besides the corresponding low-resolution data. We apply the TempoGAN model available in GitHub,2 without performing any fine-tuning. Also, TempoGAN results are compared with MantaFlow high resolution fields. The simulation method used in the MantaFlow package is the one proposed in [220], which is revised in Sect. 3.4, with MacCormack advection and MiCpreconditioned conjugate gradient solver [65, 75]. Similarly to Sect. 8.2.1, the first example applies a fountain whose setup is represented in Fig. 8.17. However, in this case, the numerical approach is meshdependent. The domain for the high-resolution simulation is a cube, with dimensions .1 × 1 × 1 in adimensional unities, discretized using a regular mesh with .256 × 256 × 256 voxels. In Fig. 8.17, we observe the domain walls, modeled using 4 voxels as thickness. The fountain has a vertical jet of smoke with the emitter positioned in the center of the domain bottom. The emitter has a cylindrical shape with a radius .0.14 (36 voxels) and a high .0.015625 (4 voxels) placed .0.1 (26 voxels) above the bottom of the domain, injecting density in the domain with velocity .v = (0, 0, 7 × 10−4 ). In the MantaFlow implementation, the density is inserted in the simulation domain by randomly changing the density field inside the emitter. The advection and the buoyancy force drive the fluid upward along the simulation. The boundary condition for the velocity field is set to the Dirichlet one, with null velocity in the boundary cells. In the applied computational setup, MantaFlow uses .At = 0.125 s along 400 time steps of simulation where the emitter is turned on only for the first 160 time steps. However, the implementation exports only numerical frames for iterations .2 ∗ n, with .n = 0, 1, 2, . . ., each one composed of pressure, density,

2 https://github.com/thunil/tempoGAN

128

8 Case Studies

Fig. 8.17 TempoGAN fountain setup: Fluid domain where the boundary is represented by walls of size .a = b = c = 256 voxels, with .d = 4 voxels as thickness. The emitter in the center is modeled as a cylindrical shape (From Author)

and velocity fields. We take only the density and velocity for TempoGAN model. The exported sequence is composed of 200 frames, each frame being organized as a tensor .(256 + 8) × (256 + 8) × (256 + 8) × 4 since the velocity field has 3 components, the density field has only one component, and the boundary thickness is four cells. The low-resolution fields are obtained by downsampling the simulated ones (high-resolution) by a factor of 4. Each low-resolution tensor enters the TempoGAN neural network and suffers the processing represented in Fig. 8.15b where the first stage is a nearest-neighbor interpolation to recover the high-resolution grid by upsampling the input field by a factor of 4. The rendering of the density field is performed with Blender, version 3.43 with the shader composed by operations in the visual pipeline of Fig. 8.18. The Volume Info node provides information about the density field. The Multiply node just scales the input to avoid saturation in the output. The Emission node adds Lambertian emission shader having as input the color and strength of the emitted light. Finally, the Material Output node receives the data and color information and applies volume rendering to generate the final image. Figure 8.19 shows some snapshots showing the low-resolution density field and its high-resolution generated by the TempoGAN, together with the reference field obtained from MantaFlow simulation.

3 https://www.blender.org/download/releases/3-4/

8.3 TempoGAN: GAN for Super-resolution Fluid Flow

129

Fig. 8.18 Shader of Blender for smoke rendering (Adapted from https://youtu.be/_K0XsHht6pg? si=HQIDhTGaz8sS1nVQ)

The visualization of the low-resolution data, shown in Fig. 8.19a, d, g, j, is affected by some smoothing effect due to the resampling involved to generate the images in the same resolution of the TempoGAN and reference data. We shall emphasize that TempoGAN processing just takes a low resolution frame and outputs its high-resolution version. However, the fluid visualization in the images of Fig. 8.19k–l indicates a fluid density generated by TempoGAN smaller than in the reference. Despite this, the TempoGAN reproduced important visual features of the smoke flow. Besides, it is important to emphasize that the training set does not include a setup similar to the fountain one. The problem perceived when comparing images 8.19k–l must be quantified. A quantitative comparison between the reference data and the corresponding TempoGAN result can be obtained through root square error (RSE) computed as follows: | | 256 | |2 ( ) | W | | t | t t .RSE ρy , ρ ˆy = | (8.24) |ρy (i, j, k) − ρˆyt (i, j, k)| , i,j,k=1

where .ρyt and .ρˆyt represent the reference and TempoGAN high-resolution density fields, respectively. Figure 8.20 shows the result of the expression (8.24) for the Fountain experiment. The plot in Fig. 8.20 shows three peaks in frames 128, 160 and 177 So, Fig. 8.21 shows the corresponding snapshots together with the low and reference frames. Likewise in Fig. 8.19, we observe mainly in Fig. 8.21h, i that the density of the TempoGAN result is smaller than in the reference. Despite this, it is noticeable the capability of TempoGAN to reproduce the visual flow features even in the snapshots with a higher error regarding the reference frames. The next example uses a geometric setup similar to the one in Fig. 8.8, with the domain for the high-resolution simulation being a cube, with dimensions .1 × 1 × 1 in dimensionless unities, divided into two subdomains as presented in Fig. 8.22. The thickness of the external walls is 4 voxels, while the floor separating the subdomains

130

8 Case Studies

Fig. 8.19 (a), (d), (g), (j) Visualization of the low-resolution version of the density field at = 60, 118, 148, 188; (b), (e), (h), (k) Corresponding high-resolution density data generated by TempoGAN; (c), (f), (i), (l) Reference frames from MantaFlow simulation at the times .t = 60, 118, 148, 188 (From Author)

.t

8.3 TempoGAN: GAN for Super-resolution Fluid Flow

131

Fig. 8.20 Error computed by expression (8.24) between TempoGAN and reference frames (From Author)

has a thickness of 16 voxels. The region containing the subdomains and the floor is discretized using a regular mesh with .256 × 256 × 256 voxels. The initialization is performed by using two smoke columns on the first floor, whose dimensions are indicated in Fig. 8.22. The boundary condition for the velocity field is set to the Dirichlet one, with null velocity in the boundary cells. The other elements of the MantaFlow simulation setup follow the ones used in the Fountain simulation. The frame sequence exported is composed of 200 frames, each frame being organized as a tensor .(256 + 8) × (256 + 8) × (256 + 8) × 4, which suffers downsampling by a factor of 4 to generate the low-resolution data (density and velocity) that is processed by the TempoGAN neural network. Figure 8.23 shows some snapshots to allow the comparison between the lowresolution density, the TempoGAN results, and the MantaFlow high-resolution simulation (reference frame). The visual analysis shows that despite some differences, the TempoGAN simulation reproduces the most important features of smoke perception. The Fig. 8.23j–l emphasizes the main differences between TempoGAN and the reference frame, which happen mainly near the bottom of the first floor and, likewise in the Fountain example, in some regions where the fluid density generated by TempoGAN seems to be smaller than the reference one (Fig. 8.23k–l, second floor). Regarding the comparison between the low-resolution data, shown in Fig. 8.23a, d, g, j, and the TempoGAN in the middle column of 8.23, we notice that the blurring effects observed in the visualization of the low resolution fields are not presented in the TempoGAN data. It is noticeable the improvement in the smoke visualization achieved by TempoGAN when comparing Fig. 8.23a, d, g, j and b, e, h, k.

132

8 Case Studies

Fig. 8.21 Snapshots corresponding to the main three peaks indicated in Fig. 8.20: (a), (d), (g) Low resolution density field at .t = 128, 160, 177; (b), (c), (h) Corresponding TempoGAN results; (c), (f), (i) Reference frames from MantaFlow at times .t = 128, 160, 177 (From Author)

The plot in Fig. 8.24 presents the evolution of distance computed by expression (8.24) for the two-floor setup for smoke simulation represented in Fig. 8.22. The comparison between plots in Figs. 8.20 and 8.24 shows a different profile after the global maximum in each figure. While for the former (Fountain simulation), we see some stabilization in the error in the latter the error suffers a considerable decrease. The plot in Fig. 8.24 shows a global maximum in frame 76. The visual analysis of the differences between the corresponding TempoGAN and the reference frames can be performed in Fig. 8.25. In this case, the low resolution field is pictured in Fig. 8.25a and we show two visualizations obtained through different viewpoints for

8.3 TempoGAN: GAN for Super-resolution Fluid Flow

133

Fig. 8.22 Two-floors setup for smoke simulation: Domain visualization, where .a = b = c = 256 voxels, the wall thickness of .d = 4 voxels, the floor division is .e = 16 voxels thick. The initialization of MantaFlow simulation with two smoke boxes on the first floor has the dimensions of .f = 81, g = 51, h = 101 voxels (From Author)

the high-resolution density fields in Fig. 8.25b–c and d–e. We emphasize the fact that even for the density distribution with larger differences according to error (8.24), the visual comparison indicates good agreement between the TempoGAN results and the MantaFlow simulation. During the generation of the tests for TempoGAN, some troubles were found due to the code dependencies. The TempoGAN implementation requires the first version of Tensorflow and CUDA 10.0. These two versions are obsolete and not compatible with our available hardware architecture since the library cuBLAS could not allocate any space in memory. Therefore, the solution found was to use Google Colab, with the miniconda virtual environment to enable Tensorflow v1, and with a 40GB VRAM NVidia A100 video card. However, the time computation was too expensive: 45 min per frame. With a performance far slower than the one presented in the original paper, it is not fair to compare this execution time with the mantaflow simulation since the lack of support for older codes can be the culprit of such time cost. Concerning computational complexity reported in [278], the generation of .600 × 400×400 volume from a .150×100×100 field lasts .3.9 min, even with the necessity of using a subdivision scheme to process the volume piece by piece due to memory limitation of the available GPUs at that time. On the other hand, the Mantaflow simulation at that high resolution took an average of .31.5 min per frame. However, we must be careful with such a comparison because the TempoGAN execution was performed using two Nvidia GeForce GTX 1080 Ti GPUs (each with 11GB Ram) while the fluid simulation was hosted by Intel Core i7-6850K CPU (see section 5.5 of [278]).

134

8 Case Studies

Fig. 8.23 Results for setup of Fig. 8.22: (a), (d), (g), (j) Visualization of the low-resolution version of the density field at .t = 45, 100, 150, 199; (b), (c), (h), (k) Corresponding high-resolution density data generated by TempoGAN; (c), (f), (i), (l) Reference frames from MantaFlow simulation at the times .t = 45, 100, 150, 199 (From Author)

8.3 TempoGAN: GAN for Super-resolution Fluid Flow Fig. 8.24 Evolution of distance (8.24) for the two-floors smoke simulation setup (From Author)

Fig. 8.25 Snapshots corresponding to the main peak at .t = 76 indicated in Fig. 8.24: (a) Visualization of the low-resolution density; (b) Corresponding TempoGAN results; (c) Reference frames from MantaFlow. (d) Another viewpoint for frame .t = 76 generated by TempoGAN; (e) Visualization of reference frame through another viewpoint (From Author)

135

Chapter 9

Perspectives and Final Remarks

Abstract In this chapter, we consider perspectives in deep learning for fluid animation. Firstly, the input data generation for neural network training depends on simulating fluid dynamics using traditional methods to produce the training data, which is time-consuming. To address this issue, we describe the basic model for physics-informed neural networks (PINNs) applied to the Navier–Stokes simulation. In this case, the fluid equations and the initial and boundary conditions are included in the model as new terms in the loss function, which can reduce the necessity for large training databases. Secondly, the success of deep architectures happens with the increase in the difficulties of understanding how deep neural networks come to decisions. This fact motivates the development of explainable artificial intelligence (XAI) techniques, like Shapley additive explanations (SHAP), to quantify the importance of hidden layers for model outcomes. From the rendering perspective, machine learning is a new approach that brings many exciting possibilities to improve visual results for the entertainment industry. But at the same time, it still has drawbacks to scientific applications, where visual accuracy is a vital requirement. We end the material by discussing how machine learning techniques improve the rendering algorithms. Keywords First principles · Deep learning · Physics-informed neural network · Navier–Stokes simulation · Explainable artificial intelligence · Shapley additive explanations · Differential rendering · Neural rendering

9.1 Introduction Despite the potential of deep learning models for fluid animation verified in Chap. 8, limitations are also observed when samples out-of-training are presented to the network. Specifically, the computational experiments of Chap. 8 show some problems for ConvNet related to asymmetries in the fountain simulation (see Fig. 8.4) if compared with the SPH simulation. Hence, a straightforward research direction is to integrate first principles and deep learning models to take advantage of scientific knowledge and data in a © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 G. A. Giraldi et al., Deep Learning for Fluid Simulation and Animation, SpringerBriefs in Mathematics, https://doi.org/10.1007/978-3-031-42333-8_9

137

138

9 Perspectives and Final Remarks

synergistic way [198, 269]. Such a goal can be implemented through PINNs, which can also avoid the necessity of simulating fluid dynamics using time-consuming traditional methods to produce large training data, as we are going to discuss in Sect. 9.2. In the previous chapters, we emphasized that recent developments in deep learning have led to impressive results in tasks related to pattern recognition and regression. However, such success of deep architectures happens with the price of increasing model complexity, which also increases the difficulties in understanding how deep neural networks come to decisions. That is the main motivation to research methods whose aim is the explanation and interpretation of machine learning models [142, 203]. The initiatives in such direction originate explainable artificial intelligence methods [208], informed machine learning [252], and intelligible intelligence [260] approaches. Since the core ideas of those fields are related, in our explanation we follow [142] and summarize that taxonomy as explainable artificial intelligence (XAI) techniques. Section 9.3 discusses the foundations of XAI approaches, with focus on fluid simulation, ending with some perspectives for the application of Shapley additive explanations (SHAP), an XAI approach for ranking features to quantify the importance of hidden layers for model outcomes. Similarly, the integration of machine learning into the rendering pipeline is not an easy task. Although the results of works in neural networks [231] and differentiable rendering [119] are astonishing, their application in fluid animation must be made carefully as long as the scientific analysis is concerned. On the other hand, it is already a great tool for animators who want to improve fluid behavior control artistically [123]. We will discuss these aspects in Sect. 9.4.

9.2 Physics-Informed Neural Networks for Fluid Animation When dealing with scientific applications like fluid simulation, we have two avenues to follow. On the one hand, we have scientific knowledge-only approaches formulated through first principles like conservation equations and numerical techniques. On the other hand, we can take into account data-driven models, like deep learning, that can deal with processes whose mechanistic models are partially known if enough data are available. Moreover, in the case of the availability of large databases, deep learning architectures can accelerate or reduce the computational cost of complex simulations depending on the precision requirements of the focused application. In the specific case of fluid animation, the generation of the input data for neural network training depends on simulating fluid dynamics using traditional methods to produce the training data. However, that is a time-consuming task because it relies on computational models composed of complex equations and numerical methods based on high-resolution grids and/or large sets of particles, as observed in the case studies considered in this text (Chap. 8). Moreover, in the actual stage of neural networks theory, they are almost black-box models, and their interpretation from

9.2 Physics-Informed Neural Networks for Fluid Animation

139

the physics viewpoint still remains an open problem. Therefore, it is of significant meaning to design cost-effective deep neural networks that: (a) Could predict fluid simulations without requiring a large number of simulations steps to build up the training data; (b) Incorporate physical laws described by partial differential equations. Physics-informed neural network (PINN) is a class of machine learning approaches that can fulfill those requirements [41, 117]. Basically, the fluid equations, together with the initial and boundary conditions, are included in the model as new terms in the Loss function to force the optimization process to follow a path in the parameter space toward a neural network configuration that solves the boundary and initial value problem. Formally, the Navier–Stokes and continuity equations presented in expressions (3.24) and (3.5), respectively, can be written in the form: .

∂η (x, t) = F (η (x, t)) , x ∈ O ⊂ R3 , t ∈ [0, tmax ] , ∂t

(9.1)

subject to boundary and initial conditions: .

η (x, t) = f (x) , x ∈ ∂O,

(9.2)

) ( η x, t 0 = g (x) , x ∈ O,

(9.3)

.

where .η (x, t), named the latent solution in the machine learning context, encompasses the unknown fields (velocity and density), .O is the fluid domain, .∂O its boundary, and .F is a differential operator that depends also from physical parameters, like viscosity. The functions .f and .g specify the boundary and initial conditions, respectively. Given a specific set of physical parameters and initial/boundary conditions, numerical methods constructed through finite-difference, finite elements, particlebased (gridless) or hybrid techniques, that apply both particles and grid elements, can be used to simulate equation (9.1). Depending on the application, the operator .F can be simplified impacting the complexity and precision of the computational model to be implemented. For PINNs setup, we must specify the initial and boundary {( } ( )) j j 0 and training datasets given by . xO , t , g xO , j = 1, 2, . . . , N0 )) } {( ( j j i x∂O , t , f x∂O , i = 1, 2, . . . , N1 , j = 1, 2, . . . , N2 , respectively. Also, . {( )) ( j j a set of collocation points . xc , t k , η t k , xc , k = 1, . . . , N3 , j = 1, . 2, . . . , Nc } could be assembled through some traditional simulation technique or by experimental measurements. Hence, let .η (x, t; θ ) the function computed by the neural network that depends on the weight vector .θ , and the Loss:

140

L=

.

9 Perspectives and Final Remarks N0 || ( N1 E N2 || ( )||2 ) ( )||2 E ) ( E || || || j j || j j η xO , t 0 ; θ − g xO || + η x∂O , t i ; θ − f x∂O || + ||||j =1

i=1 j =1

(

) || ||2 j k N3 E Nc || ∂))|| ( ( η x , t ; θ E c || || j || .+ −F η xc , t k || || || ∂t || k=1 j =1 || +

N3 E Nc || ( )||2 ) ( E || || j j η xc , t k ; θ − η xc , t k || , ||-

(9.4)

k=1 j =1

that encompasses all the elements already defined. When minimizing the Loss (9.4) with respect (to .θ, during training, we expect to obtain .θ * that allows to compute ) * a function .η x, t, θ that approximates the PDE solution .η (x, t) and satisfies the boundary and initial conditions. Regarding the architecture of the neural network ) ( * that computes .η x, t, θ we are free to choose the one that best fits the fluid representation and with enough flexibility to deal with problem complexity. The idea of integrating scientific principles and deep learning models has attracted attention in earth and climate sciences [118], turbulence theory [274], material science [116], among others [117]. These results have shown the potential of PINNs to accelerate simulations in several fields. Also, the computation of the time and spatial derivatives involved in the Loss function (9.4) is performed through automatic differentiation resources, available in Tensorflow library [153]. Moreover, in ) stage, the network receives a point ( the test * .(x, t) in the space-time and outputs .η x, t; θ . Variations of the Loss model given by expression (9.4) are presented in [44] that receives as input multiview RGB frames and performs dynamic fluids reconstruction through a PINN model that integrates Navier–Stokes equations and scene representation (radiance field) for animation purposes. Another objective where PINN’s paradigm can be pursued in the context of fluid animation is for projection methods. When simulating incompressible fluids, the projection is a fundamental step to guarantee that the velocity field satisfies the incompressibility condition (.∇ · v = 0). This is an expensive procedure for traditional techniques; particularly for recent approaches that modulate the Navier– Stokes solutions by tensor fields with the goal of fluid control [250]. In such case, the application of numerical techniques like finite element or finite difference methods may be prohibitively expensive depending on the scene complexity. In the context of PINNs, one possibility is to incorporate the corresponding incompressibility condition in the Loss expression. The motivation is to use a PINN as a surrogate model to reduce computation time since, after training, the neural network solver computes the approximation of the Navier–Stokes solution using a single forward pass of the input. Moreover, PINNs solutions are naturally differentiable functions with a closed analytic form that can be explored in further computations. Such a feature is a novelty with respect to traditional solvers [112].

9.3 Explainable Artificial Intelligence for Fluid Simulation

141

9.3 Explainable Artificial Intelligence for Fluid Simulation The application of XAI in the context of fluid simulation is restricted to few works [138], none of them with a focus on fluid animation through neural networks, which is an inspiration for future developments in this field. When developing XAI methodologies, it is worthwhile to take into account the elements of Fig. 9.1. In the chain of this figure, the machine learning model and its input–output have a central role. Around that model, three concepts appear and form part of the foundations of XAI [142, 203]. • Transparency: This concept is related to the possibility of expressing in mathematical terms the processes behind training, structure, and input–output relation concerning the model. • Interpretability: In this case, the focus is to explain the meaning of, which can be done by presenting the properties of a model or of its output results in a comprehensible way to a human. • Explainability: In general the process of enhancing control of some process, justify decisions, and knowledge discovery for model improvement depends on seeking explanations. Moreover, following [168], we can say that an explanation is the collection of features of the interpretable domain, that have contributed for the model to produce a decision, in our case, regression.

Fig. 9.1 Major XAI chain that encompasses a machine learning model, which processes input data and sends output results, besides the prerequisites of transparency, interpretability, and explainability. (Adapted from [203])

142

9 Perspectives and Final Remarks

In Fig. 9.1, the machine learning pipeline incorporates a model (deep neural network, for instance) that learns from a target input data and a suitable learning paradigm. That model outputs results in the form of labels (classification) or fields (regression). In this process, we need transparency to explain the model. Then, interpretability takes place to determine and explain cause-and-effect dependencies between inputs and outputs. The model explanation and output interpretability demand explainability that, in turn, is possible only with domain knowledge, another component of the chain in Fig. 9.1. Moreover, information acquired from domain knowledge (invariance properties, conservation laws, etc.) can offer directions for a preprocessing stage to enhance data or even insights to improve some components of the machine learning model. The scientific outcomes compose the main goal of the whole process in Fig. 9.1. They can be related to the application (or problem) domain or to the model itself. In fact, the intuition and knowledge gained when interpreting and explaining the output can also be used to foster improvements in some part(s) of the machine learning model. However, the generation of reliable outcomes has scientific consistency as a fundamental prerequisite. By scientific consistency, we mean that our conclusions are plausible and consistent with the governing laws of the subject in consideration. One important point is that we have, in general, some degree of transparency, interpretability, and explainability. For example, the ConvNet model (Sect. 8.2) is transparent because the input–output relation and training process can be formalized in mathematical terms through the computational machinery of the processing layers. However, the choice of hyperparameters, like kernel size and number of CConv+FC layers (Fig. 8.2), is performed in an ad-hoc fashion in a trial-and-error process. So, it is not transparent regarding its architectural structure. The same is true for neural networks in general. Regarding interpretability, cause-and-effect relationships between the input at time t and the output at time .t + 1 of the ConvNet neural network can be explained and analyzed considering the fluid dynamic domain, its governing laws, and invariance/symmetry properties. However, up to the best of our knowledge about the ConvNet, even with the transparency of the deep learning model used, it is very difficult to interpret the problems reported in Chap. 8.2.1 and determine, without new experiments, which part of the model should be modified to fix them. In order to improve the interpretability for ConvNet, we could explore the Shapley additive explanations (SHAP) proposed in [151]. Given a feature space M .F ≡ R , where a sample .z = (z1 , z2 , . . . , zM ) has M features given by .zi ∈ R, .i = 1, 2, . . . , M, let us consider the set .F − {zi }. Now, given a subset .S ⊂ F − {zi } and two machine models separately trained with the features in S and .S ∪ {zi }, named .gS and .gS∪{zi } , the SHAP values are given by: Oi (g, z) =

E

.

S⊂F −{zi }

( ) ) |S|! (M − |S| − 1)! ( gS∪{zi } zS∪{zi } − gS (zS ) , M!

where .zS∪{zi } and .zS are the projections of .z into .S ∪ {zi } and S, respectively.

(9.5)

9.4 Machine Learning and Fluid Rendering

143

SHAP methodology is applied in [138] and [157] as an XAI technique for providing interpretable values for fluid dynamics. The former applies SHAP when data are obtained from a low-dimensional model in wall-bounded shear flows. SHAP values indicate that the laminar profile, the streamwise vortex, and a specific streak instability are the most important features for classification between samples that do not relaminarise during .tp time steps ahead and those that do relaminarise. In this case, the machine learning model is the XGBoost [40]. Moreover, in [157], four regression models based on Decision Tree, XGBoost, Extra-tree, and LightGBM are applied to compute coefficients for surface-averaged mean pressure, fluctuation pressure, and peak pressure of low-rise gable-roofed buildings. SHAP technique is used to quantify feature importance for regression. In the case of ConvNet, the original feature space has dimension defined by the number of static particles and the number of dynamic particles. Therefore, its dimension is too high to allow the treatment with SHAP methodology. However, we could compute SHAP values (expression (9.5)) using the feature spaces of the hidden layers. In this case, the feature ranking obtained by SHAP could give insights to quantify the importance of each layer for model outcomes.

9.4 Machine Learning and Fluid Rendering From the rendering perspective, we must consider that photorealistic rendering is a complex and costly process, and that fluid rendering brings more challenging issues. The nonrigid aspect of the shape of the fluids and the way they interact with light, as a translucid material, like water, or even as a participant media, like smoke, open new problems. Generally speaking, machine learning can be used quite directly to improve some internal tasks in the rendering pipeline, such as path guiding in the Path Tracing algorithm by learning mixture models [253]; learning distributions for deep scattering in smoke and gas rendering [115]; or to amortize the cost of supersampling anti-aliasing techniques, using a convolutional neural network to produce a temporally coherent image generated with just one sample per pixel [207]. Nevertheless, machine learning techniques can inject some learnable elements into the rendering pipeline and modify this pipeline to produce images in new ways. Differential Rendering [119] sees rendering as an inverse problem, where samples of a complex function (the rendering function) are obtained from real or simulated data. Neural rendering [231], on the other hand, combines concepts from classical computer graphics and machine learning to create algorithms for synthesizing images from real-world observations, producing a hybrid approach. Differential and neural rendering still faces some common issues as a young area, started in 2018 [64]. From a practical point of view, the need for more computational environments and libraries to support the development and testing of new applications, as machine learning already has, is a challenge that makes researchers spend a lot of time building and optimizing algorithms from scratch.

144

9 Perspectives and Final Remarks

As time consumption is concerned, machine learning tasks need highly parallelized algorithms to reduce the time for training models. Coding and optimizing those algorithms require much effort and hardware and software knowledge. Libraries and standard interfaces need to be more mature to support high-level software development in this area. In the same way, debugging and benchmarks support is another practical issue these days. For Fluid Rendering, machine learning can provide more details on lowresolution simulations, enhance the visual complexity of the simulation by transferring color and semantic data, or even generate visualizations from fluid fields such as velocity and density. Such methods are interested in offline applications but can only provide the necessary performance for real-time applications like games. Nowadays, NeRF [159] is a popular method for reconstruction and rendering natural scenes and objects starting with a set of images. As this technique matures, it is possible to foresee this approach capturing actual fluid data from real-life simulations. The issue is that fluid is dynamic, and NeRF do not work well with deformable objects. The reconstruction presents significant artifacts as long as NeRF depend on the photogrammetric consistency of the scene. Works like Nerfies [188] try to minimize this limitation in a very restricted context: selfie videos, where the subject makes some smooth deformations. Unfortunately, this methodology is still inapplicable in the context of a fluid, where the object can have significant changes, even topological ones. The reconstruction needs to be more precise for a training dataset. On the other hand, methods to enhance the visual details in low-resolution simulations [204] and transfer style and patterns to the fluid rendering [43, 122] have achieved interesting results, as long as they do intend to be visually realistic. In this context, works like [125] reconstruct a volumetric density field from a sketch made by an artist using a CNN.

9.5 Final Remarks The material of this book can benefit the machine learning researchers interested in fluid animation considering the revision of fluid simulation techniques and the case studies presented that put together fluid modeling and deep learning elements with a focus on fluid animation in Lagrangian and Eulerian contexts. On the other hand, researches in physics-based modeling can understand the potential of neural networks for fluid simulation. In fact, neural networks models have being applied for fluid simulation and rendering motivated by the universality of the computational model of neural networks besides the advances in deep learning methods that allow the construction of surrogate models for fluid animation with the capability of producing predictions faster than traditional approaches. We envision that this avenue will play an invaluable role in the future of fluid animation, considering the challenges and the complex tasks involved in the generation of special effects with fluids.

9.5 Final Remarks

145

On the other hand, since the 1990s, physics-based approaches have become an important tool that enabled animators to control the objects behavior using laws of physics. So, nowadays, the subject of fluid animation can be addressed by both physicsbased modeling and deep learning approaches, which are rapidly advancing areas. Physics-based deep learning allows combining both subjects to generate fluid animation models that take advantage of physics-based and data-based approaches. For instance, in the specific case of PINNs, the provided material serves as an introduction to methods for adding physics-based terms to the Loss function to enforce physical constraints in predictive models. However, knowledge combination depends on clear lines of communication between researchers in different fields. We hope this book will accelerate the dissemination of ideas between machine learning research communities and traditional fluid simulation methods. This line could be a way to address the problems observed in Sects. 8.2.1 and 8.3.1. As mentioned in previous sections, the generation of training data depends on time-consuming simulations using traditional methods. Moreover, interpreting neural networks from the physics viewpoint is an open problem. Those facts motivate the exploration of bridges between mechanistic and deep learning models to integrate scientific knowledge and data in a synergistic way. The PINN model is a perspective in this avenue, with promising results as discussed in Sect. 9.2. Novel deep architectures models can be born in the cradle of PINNs, bringing new possibilities to add prior information in fluid dynamics leading to better interpretability. Another aspect that comes with the success of deep architectures is the increase in model complexity, which also increases the difficulties for explainability. SHAP technique can be used to get insights about feature importance in regression tasks, like fluid simulation, as pointed out in Sect. 9.3. To the best of our knowledge, an important point that has not been properly addressed in the deep learning literature is the interaction between the fluid and the boundary of the domain. Specifically, such interaction should respect the boundary conditions nearby the frontier of the domain. In general, the main belief is that the neural network weights could be properly adjusted during training in order to satisfy those requirements. Hence, it is worthwhile to compare the ConvNet and SPlisHSPlasH regarding this point to quantify the precision of ConvNet simulation nearby the domain boundary. An analogous study should be done for TempoGAN versus Mantaflow simulation. That research could bring insights to tackle the challenge of enforcing boundary conditions in neural network models through learning. Regarding rendering, at the same time that there are difficulties in applying machine learning techniques for fluid rendering, the emergence of this approach, with many open challenges, is also exciting to think about the many research possibilities it opens.

References

1. Alvaro Abucide-Armas, Koldo Portal-Porras, Unai Fernández-Gámiz, Ekaitz Zulueta, and Adrián Teso-Fz-Betoño. A data augmentation-based technique for deep learning applied to CFD simulations. Mathematics, 2021. 2. Bart Adams, Mark Pauly, Richard Keiser, and Leonidas J. Guibas. Adaptively sampled particle fluids. ACM SIGGRAPH 2007 papers, 2007. 3. Tomas Akenine-Mölle and Naty Hoffman Eric Haine an. Real-Time Rendering. A K Peters/CRC Press, 3 edition, 2008. 4. Tomas Akenine-Moller, Eric Haines, and Naty Hoffman. Real-time rendering. AK Peters/CRC Press, 2019. 5. Nadeem Akhtar and U Ragavendran. Interpretation of intelligence in CNN-pooling processes: A methodological survey. Neural Computing and Applications, pages 1–20, 2020. 6. Gizem Akinci, Nadir Akinci, Markus Ihmsen, and Matthias Teschner. An efficient surface reconstruction pipeline for particle-based fluids. In VRIPHYS, 2012. 7. Nadir Akinci, Alexander Dippel, Gizem Akinci, and Matthias Teschner. Screen space foam rendering. J. WSCG, 21:173–182, 2013. 8. Nadir Akinci, Markus Ihmsen, Gizem Akinci, Barbara Solenthaler, and Matthias Teschner. Versatile rigid-fluid coupling for incompressible SPH. ACM Trans. Graph., 31(4), Jul 2012. 9. Henrik Reif Andersen. An introduction to binary decision diagrams. Lecture notes, available online, IT University of Copenhagen, page 5, 1997. 10. Kai Arulkumaran, Marc Peter Deisenroth, Miles Brundage, and Anil Anthony Bharath. A brief survey of deep reinforcement learning. arXiv preprint arXiv:1708.05866, 2017. 11. B Ataie-Ashtiani and Leila Farhadi. A stable moving-particle semi-implicit method for free surface flows. Fluid Dynamics Research, 38(4):241, 2006. 12. Joshua Aurand, Raphael Ortiz, Silvia Nauer, and Vinicius C. Azevedo. Efficient Neural Style Transfer for Volumetric Simulations. ACM Transactions on Graphics, 41(6), 2022. 13. Simon Baker and Takeo Kanade. Limits on super-resolution and how to break them. Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662), 2:372–379 vol.2, 2000. 14. Yohai Bar-Sinai, Stephan Hoyer, Jason Hickey, and Michael P. Brenner. Learning data-driven discretizations for partial differential equations. Proceedings of the National Academy of Sciences of the United States of America, 116:15344–15349, 2019. 15. R. Beale and T. Jackson. Neural Computing. MIT Press, 1994. 16. Andrea Beck and Marius Kurz. A perspective on machine learning methods in turbulence modeling. GAMM Mitteilungen, 44(1):1–27, 2021.

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 G. A. Giraldi et al., Deep Learning for Fluid Simulation and Animation, SpringerBriefs in Mathematics, https://doi.org/10.1007/978-3-031-42333-8

147

148

References

17. Markus Becker and Matthias Teschner. Weakly compressible SPH for free surface flows. In Proceedings of the 2007 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, SCA ’07, pages 209–217, Aire-la-Ville, Switzerland, Switzerland, 2007. Eurographics Association. 18. Jan Bender. Splish Splash. https://github.com/InteractiveComputerGraphics/SPlisHSPlasH, 2022. 19. Jan Bender and Dan Koschier. Divergence-free smoothed particle hydrodynamics. In Proceedings of the 14th ACM SIGGRAPH / Eurographics Symposium on Computer Animation, SCA ’15, page 147–155. Association for Computing Machinery, 2015. 20. Jan Bender and Dan Koschier. Divergence-free SPH for incompressible and viscous fluids. IEEE Transactions on Visualization and Computer Graphics, 23(3):1193–1206, 2017. 21. Jan Bender, Tassilo Kugelstadt, Marcel Weiler, and Dan Koschier. Volume maps: An implicit boundary representation for SPH. In Proceedings of the 12th ACM SIGGRAPH Conference on Motion, Interaction and Games, pages 1–10, 2019. 22. Imanol Bilbao and Javier Bilbao. Overfitting problem and the over-training in the era of data: Particularly for artificial neural networks. In 2017 Eighth International Conference on Intelligent Computing and Information Systems (ICICIS), pages 173–177, 2017. 23. James F. Blinn. A generalization of algebraic surface drawing. ACM Trans. Graph., 1:235– 256, 1982. 24. J. Bonet and T.-S.L. Lok. Variational and momentum preservation aspects of smooth particle hydrodynamic formulations. Computer Methods in Applied Mechanics and Engineering, 180(1):97–115, 1999. 25. Mario Botsch, Leif Kobbelt, Mark Pauly, Pierre Alliez, and Bruno Lévy. Polygon mesh processing. CRC press, 2010. 26. J.U. Brackbill, D.B. Kothe, and H.M. Ruppel. Flip: A low-dissipation, particle-in-cell method for fluid flow. Computer Physics Communications, 48(1):25–38, 1988. 27. John S. Bridle. Probabilistic interpretation of feedforward classification network outputs, with relationships to statistical pattern recognition. In NATO Neurocomputing, volume 68 of NATO ASI Series, pages 227–236. Springer, 1989. 28. S. L. Brunton and J. N. Kutz. Data-Driven Science and Engineering: Machine Learning, Dynamical Systems, and Control. Cambridge University Press, 2019. 29. Steven L. Brunton. Applying Machine Learning to Study Fluid Mechanics. Acta Mechanica Sinica, pages 1–15, 2021. 30. Steven L. Brunton, Maziar S. Hemati, and Kunihiko Taira. Special issue on machine learning and data-driven methods in fluid dynamics. Theoretical and Computational Fluid Dynamics, 34(4):333–337, 2020. 31. Steven L. Brunton, Bernd R. Noack, and Petros Koumoutsakos. Machine Learning for Fluid Mechanics. Annual Review of Fluid Mechanics, 52(1):477–508, 2020. 32. Marc Bryant, Ian Coony, and Jonathan Garcia. Moana: Foundation of a lava monster. In ACM SIGGRAPH 2017 Talks, pages 1–2, 2017. 33. Mitch Bryson, Matthew Johnson-Roberson, Oscar Pizarro, and Stefan B Williams. Colourconsistent structure-from-motion models using underwater imagery. Robotics: Science and Systems VIII, 33, 2013. 34. Brent Burley, David Adler, Matt Jen-Yuan Chiang, Hank Driskill, Ralf Habel, Patrick Kelly, Peter Kutz, Yining Karl Li, and Daniel Teece. The design and evolution of disney’s hyperion renderer. ACM Trans. Graph., 37(3), Jul 2018. 35. H.J. Butt, K. Graf, and M. Kappl. Physics and Chemistry of Interfaces. Wiley, 2006. 36. Thomas J Cashman. Beyond catmull–clark? a survey of advances in subdivision surface methods. Computer Graphics Forum, 31(1):42–61, 2012. 37. Kai-Chun Chen, Pei-Shan Chen, and Sai-Keung Wong. A heuristic approach to the simulation of water drops and flows on glass panes. Computers & graphics, 37(8):963–973, 2013. 38. Qian Chen, Yue Wang, Hui Wang, and Xubo Yang. Data-driven simulation in fluids animation: A survey. Virtual Reality and Intelligent Hardware, 3(2):87–104, 2021.

References

149

39. Qiaorui Chen, Shuai Zhang, and Yao Zheng. Enhanced narrow band surface reconstruction with anisotropic kernel. Comput. Graph., 102:280–288, 2022. 40. Tianqi Chen and Carlos Guestrin. Xgboost: A scalable tree boosting system. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2016. 41. Chen Cheng and Guang-Tao Zhang. Deep learning method based on physics informed neural network with resnet block for solving fluid flow problems. Water, 13, 02 2021. 42. A.J. Chorin and J.E. Marsden. A Mathematical Introduction to Fluid Mechanics. Texts in Applied Mathematics. Springer New York, 2012. 43. Fabienne Christen, Byungsoo Kim, Vinicius C. Azevedo, and Barbara Solenthaler. Neural Smoke Stylization with Color Transfer. arXiv preprint arXiv:1912.08757, pages 2–5, 2019. 44. Mengyu Chu, Lingjie Liu, Quan Zheng, Erik Franz, Hans-Peter Seidel, Christian Theobalt, and Rhaleb Zayer. Physics informed neural fields for smoke reconstruction with sparse data. ACM Trans. Graph., 41(4), Jul 2022. 45. Mengyu Chu, Nils Thuerey, Hans Peter Seidel, Christian Theobalt, and Rhaleb Zayer. Learning meaningful controls for fluids. ACM Transactions on Graphics, 40(4), 2021. 46. Mengyu Chu and Nils Thürey. Data-driven synthesis of smoke flows with CNN-based feature descriptors. ACM Transactions on Graphics (TOG), 36:1–14, 2017. 47. Richard Connor, Franco Alberto Cardillo, Robert Moss, and Fausto Rabitti. Evaluation of Jensen-Shannon distance over sparse data. In Nieves Brisaboa, Oscar Pedreira, and Pavel Zezula, editors, Similarity Search and Applications, pages 163–168, Berlin, Heidelberg, 2013. Springer Berlin Heidelberg. 48. R. Courant, K. Friedrichs, and H. Lewy. On the partial difference equations of mathematical physics. IBM J. Res. Dev., 11(2):215–234, mar 1967. 49. Keenan Crane, Ignacio Llamas, and Sarah Tariq. Real-time simulation and rendering of 3d fluids. GPU gems, 3(1), 2007. 50. Leandro Tavares da Silva. Simulação de Fluidos via Smoothed Particle Hydrodynamics: Formulação Variacional, Variação de Parâmetros e Extração de Características Visuais, 2016. 51. Leandro Tavares da Silva and Gilson Antonio Giraldi. Fluid flow summarization technique for computational fluid dynamics analysis. In Computational Fluid Dynamics (CFD): Characteristics, Applications and Analysis, pages 1–44. Nova Science Publishers, 2016. Chapter 3. 52. Leandro Tavares da Silva and Gilson Antonio Giraldi. Fixed point implementation of a variational time integrator approach for smoothed particle hydrodynamics simulation of fluids. Computers & Mathematics with Applications, 79(4):1111–1130, 2020. 53. Leonid Datta. A survey on activation functions and their relation with xavier and he normal initialization. CoRR, abs/2004.06632, 2020. 54. M.D. Davis and E.J. Weyuker. Computability, complexity, and languages: fundamentals of theoretical computer science. New York Academic Press, 1983. 55. S.R. De Groot and P. Mazur, Non-Equilibrium Thermodynamics. Dover Books on Physics. Dover Publications. ISBN 9780486153506, https://books.google.com.br/books?id= mfFyG9jfaMYC 56. Felipe Orlandi de Oliveira and Afonso Paiva. Narrow-band screen-space fluid rendering. Computer Graphics Forum, 41, 2022. 57. Paul E. Debevec and Leonard McMillan. Image-based modeling, rendering, and lighting. IEEE Computer Graphics and Applications, 22:24–25, 2002. 58. M. do Carmo and P. Roitman. Geometria diferencial de curvas e superfícies. Textos Universitários. Sociedade Brasileira de Matemática, 2006. 59. Fourtakas G Domínguez JM. Dualsphysics. open source for SPH to study free-surface phenomena. https://dual.sphysics.org/, 2021. 60. Liordino dos Santos Rocha Neto and Antônio Lopes Apolinario. Real-time screen space cartoon water rendering with the iterative separated bilateral filter. SBC Journal on Interactive Systems, 8:20–32, 2017.

150

References

61. Karthik Duraisamy, Gianluca Iaccarino, and Heng Xiao. Turbulence modeling in the age of data. Annual Review of Fluid Mechanics, 51:357–377, 2019. 62. Pandu Akbar Dwikatama, Dody Dharma, and Achmad I. Kistijantoro. Fluid simulation based on material point method with neural network. 2019 International Conference of Artificial Intelligence and Information Technology (ICAIIT), pages 244–249, 2019. 63. Christopher Dyken, Gernot Ziegler, Christian Theobalt, and Hans-Peter Seidel. High-speed marching cubes using histopyramids. Computer Graphics Forum, 27(8):2028–2039, 2008. 64. SM Ali Eslami, Danilo Jimenez Rezende, Frederic Besse, Fabio Viola, Ari S Morcos, Marta Garnelo, Avraham Ruderman, Andrei A Rusu, Ivo Danihelka, Karol Gregor, et al. Neural scene representation and rendering. Science, 360(6394):1204–1210, 2018. 65. D. J. Evans. Preconditioning Methods: Theory and Applications. Gordon and Breach Science Publishers, Inc., USA, 1st edition, 1983. 66. Jiahui Fan, Beibei Wang, Milos Hasan, Jian Yang, and Ling-Qi Yan. Neural layered BRDFs. ACM SIGGRAPH 2022 Conference Proceedings, 2022. 67. José Antonio Fernández-Fernández, Lukas Westhofen, Fabian Löschner, Stefan Rhys Jeske, Andreas Longva, and Jan Bender. Fast octree neighborhood search for SPH simulations. ACM Transactions on Graphics (TOG), 41(6):1–13, 2022. 68. Anselmo Ferreira and Gilson Giraldi. Convolutional neural network approaches to granite tiles classification. Expert Systems with Applications, 84:1–11, 2017. 69. A.T. Fomenko and S.P. Novikov. Modern geometry: Methods and Applications. New York: Springer-Verlag, 1992. 70. Julian Fong, Magnus Wrenninge, Christopher D. Kulla, and Ralf Habel. Production volume rendering: Siggraph 2017 course. ACM SIGGRAPH 2017 Courses, 2017. 71. Roland Fraedrich, Stefan Auer, and Rüdiger Westermann. Efficient high-quality volume rendering of SPH data. IEEE Transactions on Visualization and Computer Graphics, 16:1533–1540, 2010. 72. Ben Frost, Alexey Stomakhin, and Hiroaki Narita. Moana: Performing water. In ACM SIGGRAPH 2017 Talks, SIGGRAPH ’17, New York, NY, USA, 2017. Association for Computing Machinery. 73. Kai Fukami, Koji Fukagata, and Kunihiko Taira. Super-resolution reconstruction of turbulent flows with machine learning. Journal of Fluid Mechanics, 870:106–120, 2019. 74. Kai Fukami, Koji Fukagata, and Kunihiko Taira. Assessment of supervised machine learning methods for fluid flows. Theoretical and Computational Fluid Dynamics, 34(4):497–519, 2020. 75. Jiaquan Gao, Ronghua Liang, and Jun Wang. Research on the conjugate gradient algorithm with a modified incomplete cholesky preconditioner on GPU. Journal of Parallel and Distributed Computing, 74, 01 2013. 76. Michael Garland and Paul S Heckbert. Simplifying surfaces with color and texture using quadric error metrics. In Proceedings Visualization’98 (Cat. No. 98CB36276), pages 263– 269. IEEE, 1998. 77. R. Garnier and J. Taylor. Discrete Mathematics for New Technology. Adam Hilger, 1992. 78. Leon A Gatys, Alexander S Ecker, and Matthias Bethge. A neural algorithm of artistic style. arXiv preprint arXiv:1508.06576, 2015. 79. Tom M George, Georgy E. Manucharyan, and Andrew F. Thompson. Deep learning to infer eddy heat fluxes from sea surface height patterns of mesoscale turbulence. Nature communications, 12 1:800, 2021. 80. A. Géron. Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems. O’Reilly Media, 2019. 81. R. A. Gingold and J. J. Monaghan. Smoothed particle hydrodynamics: theory and application to non-spherical stars. Monthly Notices of the Royal Astronomical Society, 181(3):375–389, 1977. 82. Gilson Antonio Giraldi. Machine Learning and Pattern Recognition, Lecture Notes - Graduate Program in Nanobiosystems, UFRJ-FIOCRUZ-INMETRO-LNCC, 2021.

References

151

83. Christoph Gissler, Stefan Band, Andreas Peer, Markus Ihmsen, and Matthias Teschner. Approximate air-fluid interactions for SPH. In VRIPHYS, pages 29–38, 2017. 84. Xavier Glorot and Yoshua Bengio. Understanding the difficulty of training deep feedforward neural networks. In Yee Whye Teh and Mike Titterington, editors, Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, volume 9 of Proceedings of Machine Learning Research, pages 249–256, Chia Laguna Resort, Sardinia, Italy, 13–15 May 2010. PMLR. 85. Ronald Goldman. An integrated introduction to computer graphics and geometric modeling. CRC Press, 2009. 86. H. Goldstein. Classical Mechanics. Addison-Wesley, 2nd edition, 1981. 87. I. Goodfellow, Y. Bengio, and A. Courville. Deep Learning. Adaptive Computation and Machine Learning series. MIT Press, 2016. 88. Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron C. Courville, and Yoshua Bengio. Generative adversarial nets. In Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014, December 8–13 2014, Montreal, Quebec, Canada, pages 2672– 2680, 2014. 89. Ned Greene. Environment mapping and other applications of world projections. IEEE Computer Graphics and Applications, 6(11):21–29, 1986. 90. Jens A. Griepentrog, Wolfgang Höppner, Hans-Christoph Kaiser, and Joachim Rehberg. A bi-lipschitz continuous, volume preserving map from the unit ball onto a cube. Note di Matematica, 28, 2008. 91. Markus Gross and Hanspeter Pfister. Point-Based Graphics. Elsevier, 2011. 92. Jiuxiang Gu, Zhenhua Wang, Jason Kuen, Lianyang Ma, Amir Shahroudy, Bing Shuai, Ting Liu, Xingxing Wang, and Gang Wang. Recent advances in convolutional neural networks. CoRR, abs/1512.07108, 2015. 93. Yulan Guo, Hanyun Wang, Qingyong Hu, Hao Liu, Li Liu, and Bennamoun. Deep learning for 3d point clouds: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43:4338–4364, 2019. 94. M.E. Gurtin. An Introduction to Continuum Mechanics. ISSN. Elsevier Science, 1982. 95. Saeed Hadadan, Shuhong Chen, and Matthias Zwicker. Neural radiosity. ACM Transactions on Graphics, 40(6):1–13, 2021. 96. Eric Haines and Tomas Akenine-Möller. Ray Tracing Gems: High-Quality and Real-Time Rendering with DXR and Other APIs. Springer, 2019. 97. Rana Hanocka and Hsueh-Ti Derek Liu. An introduction to deep learning on meshes. ACM SIGGRAPH 2021 Courses, 2021. 98. Simon Haykin. Neural Networks - A Comprehensive Foundation, Second Edition. Prentice Hall, 2 edition, 1998. 99. K. He and J. Sun. Convolutional neural networks at constrained time cost. In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 5353–5360, 2015. 100. Xin He, Kaiyong Zhao, and Xiaowen Chu. Automl: A survey of the state-of-the-art. Knowledge-Based Systems, 212:106622, 2021. 101. Paul S Heckbert. Survey of texture mapping. IEEE computer graphics and applications, 6(11):56–67, 1986. 102. C. Hirsch. Numerical Computation of Internal and External Flows: Fundamentals of Numerical Discretization, volume 1. John Wiley & Sons, 1988. 103. D. House and J.C. Keyser. Foundations of Physically Based Modeling and Animation. Taylor & Francis Group, 2020. 104. Yuchi Huo and Sung eui Yoon. A survey on deep learning-based monte carlo denoising. Computational Visual Media, 7:169–185, 2021. 105. Markus Ihmsen, Nadir Akinci, Gizem Akinci, and Matthias Teschner. Unified spray, foam and air bubbles for particle-based fluids. The Visual Computer, 28:669–677, 2012.

152

References

106. Markus Ihmsen, Julian Bader, Gizem Akinci, and Matthias Teschner. Animation of air bubbles with SPH. In International Conference on Computer Graphics Theory and Applications, volume 2, pages 225–234. SCITEPRESS, 2011. 107. Ihab F. Ilyas and Xu Chu. Data Cleaning. ACM, 2019. 108. Jason Iversen and Ryo Sakaguchi. Growing up with fluid simulation on “the day after tomorrow”. In ACM SIGGRAPH 2004 Sketches, SIGGRAPH ’04, page 142, New York, NY, USA, 2004. Association for Computing Machinery. 109. Alan Julian Izenman. Modern Multivariate Statistical Techniques: Regression, Classification, and Manifold Learning. Springer Publishing Company, Incorporated, 1 edition, 2008. 110. Abdul Jabbar, Xi Li, and Bourahla Omar. A Survey on Generative Adversarial Networks: Variants, Applications, and Training. ACM Computing Surveys, 54(8):1–38, 2022. 111. Krishna Murthy Jatavallabhula, Edward James Smith, Jean-Francois Lafleche, Clément Fuji Tsang, Artem Rozantsev, Wenzheng Chen, Tommy Xiang, Rev Lebaredian, and Sanja Fidler. Kaolin: A pytorch library for accelerating 3d deep learning research. ArXiv, abs/1911.05063, 2019. 112. Xiaowei Jin, Shengze Cai, Hui Li, and George Em Karniadakis. Nsfnets (navier-stokes flow nets): Physics-informed neural networks for the incompressible navier-stokes equations. J. Comput. Phys., 426:109951, 2021. 113. Yongcheng Jing, Yezhou Yang, Zunlei Feng, Jingwen Ye, Yizhou Yu, and Mingli Song. Neural Style Transfer: A Review. IEEE Transactions on Visualization and Computer Graphics, 26(11):3365–3385, 2020. 114. James T Kajiya. The rendering equation. In Proceedings of the 13th annual conference on Computer graphics and interactive techniques, pages 143–150, 1986. 115. Simon Kallweit, Thomas Müller, Brian Mcwilliams, Markus Gross, and Jan Novák. Deep scattering: Rendering atmospheric clouds with radiance-predicting neural networks. ACM Transactions on Graphics (TOG), 36(6):1–11, 2017. 116. Ali Kamali, Mohammad Sarabian, and Kaveh Laksari. Elasticity imaging using physicsinformed neural networks: Spatial discovery of elastic modulus and poisson’s ratio. Acta biomaterialia, 2022. 117. George Karniadakis, Yannis Kevrekidis, Lu Lu, Paris Perdikaris, Sifan Wang, and Liu Yang. Physics-informed machine learning. Nature Reviews, pages 1–19, 05 2021. 118. Karthik Kashinath, M Mustafa, Adrian Albert, Jinlong Wu, C Jiang, Soheil Esmaeilzadeh, Kamyar Azizzadenesheli, R Wang, Ashesh Chattopadhyay, A Singh, A Manepalli, D Chirila, R Yu, R Walters, B White, Heng Xiao, Hamdi Tchelepi, P Marcus, Animashree Anandkumar, and Mr Prabhat. Physics-informed machine learning: Case studies for weather and climate modelling. Philosophical transactions. Series A, Mathematical, physical, and engineering sciences, 379:20200093, 04 2021. 119. Hiroharu Kato, Deniz Beker, Mihai Morariu, Takahiro Ando, Toru Matsuoka, Wadim Kehl, and Adrien Gaidon. Differentiable Rendering: A Survey. arXiv preprint arXiv:2006.12057, 14(8):1–20, 2020. 120. Petr Kellnhofer, Lars Jebe, Andrew Jones, Ryan P. Spicer, Kari Pulli, and Gordon Wetzstein. Neural lumigraph rendering. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 4285–4295, 2021. 121. Timo Kellomäki. Fast water simulation methods for games. Comput. Entertain., 16(1), dec 2017. 122. Byungsoo Kim, Vinicius C. Azevedo, Markus Gross, and Barbara Solenthaler. Transportbased neural style transfer for smoke simulations. ACM Transactions on Graphics, 38(6):1– 11, 2019. 123. Byungsoo Kim, Vinicius C. Azevedo, Markus Gross, and Barbara Solenthaler. Lagrangian neural style transfer for fluids. ACM Trans. Graph., 39(4), Jul 2020. 124. Byungsoo Kim, Vinicius C Azevedo, Nils Thuerey, Theodore Kim, Markus Gross, and Barbara Solenthaler. Deep fluids: A generative network for parameterized fluid simulations. In Computer graphics forum, volume 38, pages 59–70. Wiley Online Library, 2019.

References

153

125. Byungsoo Kim, Xingchang Huang, Laura Wuelfroth, Jingwei Tang, Guillaume Cordonnier, Markus Gross, and Barbara Solenthaler. Deep reconstruction of 3d smoke densities from artist sketches. Computer Graphics Forum, 41(2):97–110, 2022. 126. Doyub Kim. Fluid Engine Development. A K Peters/CRC Press, 1st edition, 2017. 127. Hyojin Kim, Junhyuk Kim, Sungjin Won, and Changhoon Lee. Unsupervised deep learning for super-resolution reconstruction of turbulence. Journal of Fluid Mechanics, 910, 2021. 128. Jong-Hyun Kim, Wook Kim, and Jung Lee. Physics-inspired approach to realistic and stable water spray with narrowband air particles. The Visual Computer, 34:461–471, 2018. 129. Kyung Sung Kim, Moo Hyun Kim, and Jong-Chun Park. Development of moving particle simulation method for multiliquid-layer sloshing. Mathematical Problems in Engineering, page 13, 2014. 130. S. Kong and M. Takatsuka. Hexpo: A vanishing-proof activation function. In 2017 International Joint Conference on Neural Networks (IJCNN), pages 2562–2567, 2017. 131. Jiaqing Kou and Weiwei Zhang. Data-driven modeling for unsteady aerodynamics and aeroelasticity. Progress in Aerospace Sciences, 2021. 132. Alexander Kuleshov and Alexander Bernstein. Manifold learning in data mining tasks. In Machine Learning and Data Mining in Pattern Recognition: 10th International Conference, MLDM 2014, St. Petersburg, Russia, July 21–24, 2014. Proceedings 10, pages 119–133. Springer, 2014. 133. Murat Kurt and Dave Edwards. A survey of BRDF models for computer graphics. ACM SIGGRAPH Computer Graphics, 43(2):1–7, 2009. 134. J. Nathan Kutz. Deep learning in fluid dynamics. Journal of Fluid Mechanics, 814:1–4, 2017. 135. B. Laure, B. Angela, and M. Tova. Machine learning to data management: A round trip. In 2018 IEEE 34th International Conference on Data Engineering (ICDE), pages 1735–1738, 2018. 136. Y. Lecun, Y. Bengio, and G Hinton. Deep learning. Nature, 521(7553):436–444, 2015. 137. John A. Lee and Michel Verleysen. Nonlinear Dimensionality Reduction. Springer Publishing Company, Incorporated, 1st edition, 2007. 138. Martin Lellep, Jonathan Prexl, Bruno Eckhardt, and Moritz Linkmann. Interpreted machine learning in fluid dynamics: Explaining relaminarisation events in wall-bounded shear flows. Journal of Fluid Mechanics, 2021. 139. Wei Li, Yihui Ma, Xiaopei Liu, and Mathieu Desbrun. Efficient kinetic simulation of twophase flows. ACM Transactions on Graphics (TOG), 41:1–17, 2022. 140. Gábor Liktor and Carsten Dachsbacher. Decoupled deferred shading for hardware rasterization. In Proceedings of the ACM SIGGRAPH symposium on interactive 3D graphics and games, pages 143–150, 2012. 141. Bee Lim, Sanghyun Son, Heewon Kim, Seungjun Nah, and Kyoung Mu Lee. Enhanced deep residual networks for single image super-resolution. 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pages 1132–1140, 2017. 142. Pantelis Linardatos, Vasilis Papastefanopoulos, and Sotiris Kotsiantis. Explainable ai: A review of machine learning interpretability methods. Entropy, 23(1):1–45, 2021. 143. S. J. Lind, R. Xu, P. K. Stansby, and B. D. Rogers. Incompressible smoothed particle hydrodynamics for free-surface flows: A generalised diffusion-based algorithm for stability and validations for impulsive flows and propagating waves. J. Comput. Phys., 231(4):1499– 1523, February 2012. 144. Julia Ling, Andrew Kurzawski, and Jeremy Templeton. Reynolds averaged turbulence modelling using deep neural networks with embedded invariance. Journal of Fluid Mechanics, 807:155–166, 2016. 145. G.R. Liu and B. Liu. Smoothed Particle Hydrodynamics: A Meshfree Particle Method. World Scientific, 2003. 146. Lingjie Liu, Jiatao Gu, Kyaw Zaw Lin, Tat-Seng Chua, and Christian Theobalt. Neural sparse voxel fields. ArXiv, abs/2007.11571, 2020.

154

References

147. Stephen Lombardi, Tomas Simon, Jason M. Saragih, Gabriel Schwartz, Andreas M. Lehrmann, and Yaser Sheikh. Neural volumes. ACM Transactions on Graphics (TOG), 38:1–14, 2019. 148. William E. Lorensen and Harvey E. Cline. Marching cubes: A high resolution 3d surface construction algorithm. Proceedings of the 14th annual conference on Computer graphics and interactive techniques, 1987. 149. Jia-Ming Lu, Xiao-Song Chen, Xiao Yan, Chen-Feng Li, Ming Lin, and Shi-Min Hu. A rigging-skinning scheme to control fluid simulation. Computer Graphics Forum, 38(7):501– 512, 2019. 150. David Luebke, Martin Reddy, Jonathan D Cohen, Amitabh Varshney, Benjamin Watson, and Robert Huebner. Level of detail for 3D graphics. Morgan Kaufmann, 2003. 151. Scott M. Lundberg and Su-In Lee. A unified approach to interpreting model predictions. ArXiv, abs/1705.07874, 2017. 152. Yunqian Ma and Yun Fu. Manifold Learning Theory and Applications. CRC Press, Inc., Boca Raton, FL, USA, 1st edition, 2011. 153. Charles C Margossian. A review of automatic differentiation and its efficient implementation. Wiley interdisciplinary reviews: data mining and knowledge discovery, 9(4):e1305, 2019. 154. Adam Marrs, Peter Shirley, and Ingo Wald. Ray Tracing Gems II: Next Generation Real-Time Rendering with DXR, Vulkan, and OptiX. Springer Nature, 2021. 155. S. Marsland. Machine Learning: An Algorithmic Perspective. Chapman & Hall/CRC The R Series. CRC Press, 2009. 156. Marilena Maule, João Luiz Dihl Comba, Rafael P. Torchelsen, and Rui Bastos. A survey of raster-based transparency techniques. Comput. Graph., 35:1023–1034, 2011. 157. Pasindu Meddage, Imesh Udara Ekanayake, Udara Sachinthana Perera, Hazi Md. Azamathulla, Md. Azlin Md. Said, and Upaka S. Rathnayake. Interpretation of machine-learning-based (black-box) wind pressure predictions for low-rise gable-roofed buildings using shapley additive explanations (shap). Buildings, 2022. 158. Xuhui Meng and George Em Karniadakis. A composite neural network that learns from multifidelity data: Application to function approximation and inverse PDE problems. J. Comput. Phys., 401, 2020. 159. Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. ArXiv, abs/2003.08934, 2020. 160. Gastão Florencio Miranda, Carlos E. Thomaz, and Gilson A. Giraldi. Geometric data analysis based on manifold learning with applications for image understanding. 2017 30th SIBGRAPI Conference on Graphics, Patterns and Images - Tutorials, pages 42–62, 2017. 161. A.A. Mohamad. Lattice Boltzmann Method: Fundamentals and Engineering Applications with Computer Codes. SpringerLink : Bücher. Springer London, 2011. 162. Arvind T. Mohan, Don Daniel, Michael Chertkov, and Daniel Livescu. Compressed convolutional LSTM: An efficient deep learning framework to model high fidelity 3d turbulence. arXiv: Fluid Dynamics, 2019. 163. Arvind T. Mohan, Nicholas Lubbers, Daniel Livescu, and Michael Chertkov. Embedding hard physical constraints in neural network coarse-graining of 3d turbulence, 2020. 164. J. J. Monaghan. Smoothed particle hydrodynamics. Annual Review of Astronomy and Astrophysics, 30(1):543–574, 1992. 165. J.J. Monaghan. Simulating free surface flows with SPH. Journal of Computational Physics, 110(2):399–406, 1994. 166. J.J. Monaghan. Smoothed particle hydrodynamics and its diverse applications. Annual Review of Fluid Mechanics, 44(1):323–346, 2012. 167. J.J. Monaghan and R.A. Gingold. Shock simulation by the particle method SPH. Journal of Computational Physics, 52:374–384, 1983. 168. Grégoire Montavon, Wojciech Samek, and Klaus-Robert Müller. Methods for interpreting and understanding deep neural networks. Digital Signal Processing, 73:1–15, 2018.

References

155

169. Joseph P. Morris, Patrick J. Fox, and Yi Zhu. Modeling low Reynolds number incompressible flows using SPH. J. Comput. Phys., 136(1):214–226, September 1997. 170. P. Morris. Analysis of Smoothed Particle Hydrodynamics. PhD thesis, Monash University, 1996. 171. Jeremy Morton, Antony Jameson, Mykel J. Kochenderfer, and Freddie D. Witherden. Deep dynamical modeling and control of unsteady fluid flows. In Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicolò Cesa-Bianchi, and Roman Garnett, editors, Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, December 3–8, 2018, Montréal, Canada, pages 9278–9288, 2018. 172. Matthias Müller, David Charypar, and Markus H. Gross. Particle-based fluid simulation for interactive applications. In Symposium on Computer Animation, 2003. 173. Kevin P. Murphy. Machine learning: A Probabilistic Perspective. MIT Press, Cambridge, Mass. [U.A.], 2013. 174. Ken Museth. VDB: High-resolution sparse volumes with dynamic topology. ACM transactions on graphics (TOG), 32(3):1–22, 2013. 175. Ken Museth. Hierarchical digital differential analyzer for efficient ray-marching in OpenVDB. In ACM SIGGRAPH 2014 Talks, 2014. 176. Ken Museth. Nanovdb: A GPU-friendly and portable VDB data structure for real-time rendering and simulation. In ACM SIGGRAPH 2021 Talks, pages 1–2, 2021. 177. Vinod Nair and Geoffrey E. Hinton. Rectified linear units improve restricted boltzmann machines. In ICML, pages 807–814. Omnipress, 2010. 178. KDnuggetsTM News. Six Steps to Master Machine Learning with Data Preparation, 2018. 179. Thu Nguyen-Phuoc, Chuan Li, Lucas Theis, Christian Richardt, and Yong-Liang Yang. Hologan: Unsupervised learning of 3d representations from natural images. 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pages 7587–7596, 2019. 180. M. Nielsen and I. Chuang. Quantum Computation and Quantum Information. Cambridge University Press., December 2000. 181. Michael B. Nielsen and Robert Bridson. Spatially adaptive flip fluid simulations in bifrost. In ACM SIGGRAPH 2016 Talks, SIGGRAPH ’16, New York, NY, USA, 2016. Association for Computing Machinery. 182. Merlin Nimier-David, Delio Vicini, Tizian Zeltner, and Wenzel Jakob. Mitsuba 2: A retargetable forward and inverse renderer. ACM Transactions on Graphics, 38(6):1–17, 2019. 183. Jan Novák, Iliyan Georgiev, Johannes Hanika, and Wojciech Jarosz. Monte Carlo methods for volumetric light transport simulation. In Computer Graphics Forum, volume 37, pages 551–576. Wiley Online Library, 2018. 184. Jan Novák, Iliyan Georgiev, Johannes Hanika, Jaroslav Kˇrivánek, and Wojciech Jarosz. Monte carlo methods for physically based volume rendering. ACM SIGGRAPH 2018 Courses, 2018. 185. Augustus Odena, Vincent Dumoulin, and Chris Olah. Deconvolution and checkerboard artifacts. Distill, 1(10):e3, 2016. 186. Afonso Paiva, Fabiano Petronetto, Thomas Lewiner, and Geovan Tavares. Simulação de Fluidos sem Malha: Uma Introdução ao Método SPH. IMPA, Rio de Janeiro, 2009. 27.◦ Colóquio Brasileiro de Matemática. 187. Sean Palmer, Jonathan Garcia, Sara Drakeley, Patrick Kelly, and Ralf Habel. The ocean and water pipeline of disney’s moana. In ACM SIGGRAPH 2017 Talks, pages 1–2, 2017. 188. Keunhong Park, Utkarsh Sinha, Jonathan T Barron, Sofien Bouaziz, Dan B Goldman, Steven M Seitz, and Ricardo Martin-Brualla. Nerfies: Deformable neural radiance fields. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5865– 5874, 2021. 189. Steven G Parker, Heiko Friedrich, David Luebke, Keith Morley, James Bigler, Jared Hoberock, David McAllister, Austin Robison, Andreas Dietrich, Greg Humphreys, et al. GPU ray tracing. Communications of the ACM, 56(5):93–101, 2013.

156

References

190. Matt Pharr, Wenzel Jakob, and Greg Humphreys. Physically based rendering: From theory to implementation. The MIT Press, 4th edition edition, 2023. 191. Bui Tuong Phong. Illumination for computer generated pictures. Communications of the ACM, 18(6):311–317, 1975. 192. N. Pitelis, C. Russell, and L. Agapito. Learning a manifold as an atlas. In Computer Vision and Pattern Recognition (CVPR), 2013 IEEE Conference on, June 2013. 193. Pixar. Pixar On-Line Library. http://graphics.pixar.com/library/, 2021. 194. I. Pointer. Programming PyTorch for Deep Learning: Creating and Deploying Deep Learning Applications. O’Reilly Media, 2019. 195. Fabio Policarpo, Francisco Fonseca, and CheckMate Games. Deferred shading tutorial. Technical Report 31, Pontifical Catholic University of Rio de Janeiro, 2005. 196. Jean Rabault, Miroslav Kuchta, Atle Jensen, Ulysse Réglade, and Nicolas Cerardi. Artificial neural networks trained through deep reinforcement learning discover control strategies for active flow control. Journal of Fluid Mechanics, 865:281–302, 2019. 197. Amir H. Rabbani and Soufiane Khiat. Fast Eulerian Fluid Simulation In Games Using Poisson Filters. In Daniel Holden, editor, Eurographics/ ACM SIGGRAPH Symposium on Computer Animation - Showcases. The Eurographics Association, 2020. 198. Rahul Rai and Chandan K. Sahu. Driven by data or derived through physics? a review of hybrid physics guided machine learning techniques with cyber-physical system (cps) focus. IEEE Access, 8:71050–71073, 2020. 199. Prabhu Ramachandran and Aditya Bhosale. PySPH. open source framework for SPH. https:// pysph.readthedocs.io/en/latest/, 2018. 200. Nikhila Ravi, Jeremy Reizenstein, David Novotný, Taylor Gordon, Wan-Yen Lo, Justin Johnson, and Georgia Gkioxari. Accelerating 3d deep learning with pytorch3d. SIGGRAPH Asia 2020 Courses, 2019. 201. John H. Reif and Stephen R. Tate. On threshold circuits and polynomial computation. SIAM J. Comput., 21(5):896–908, 1992. 202. Christian Reiser, Songyou Peng, Yiyi Liao, and Andreas Geiger. Kilonerf: Speeding up neural radiance fields with thousands of tiny mlps. 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pages 14315–14325, 2021. 203. Ribana Roscher, Bastian Bohn, Marco F. Duarte, and Jochen Garcke. Explainable Machine Learning for Scientific Insights and Discoveries. IEEE Access, 8:42200–42216, 2020. 204. Bruno Roy, Pierre Poulin, and Eric Paquette. Neural UpFlow: A Scene Flow Learning Approach to Increase the Apparent Resolution of Particle-Based Liquids. ACM Transactions on Graphics, 1(1):1–14, 2021. 205. Takafumi Saito and Tokiichiro Takahashi. Comprehensible rendering of 3-d shapes. In Proceedings of the 17th annual conference on Computer graphics and interactive techniques, pages 197–206, 1990. 206. Karam Sahoo, Ridam Hazra, Muhammad Fazal Ijaz, SeongKi Kim, Pawan Singh, and Mufti Mahmud. MIC_FuzzyNET: Fuzzy integral based ensemble for automatic classification of musical instruments from audio signals. IEEE Access, 9:1–1, 2022. https://doi.org/0.1109/ ACCESS.2022.3208126. 207. Farnood Salehi, Marco Manzi, Gerhard Roethlin, Romann Weber, Christopher Schroers, and Marios Papas. Deep adaptive sampling and reconstruction using analytic distributions. ACM Transactions on Graphics (TOG), 41(6):1–16, 2022. 208. Wojciech Samek, Thomas Wiegand, and Klaus-Robert Müller. Explainable artificial intelligence: Understanding, visualizing and interpreting deep learning models. ArXiv, abs/1708.08296, 2017. 209. Fabrizio Sarghini, Giuseppe de Felice, and Stefania Santini. Neural networks based subgrid scale modeling in large eddy simulations. Computers & Fluids, 32:97–108, 2003. 210. Syuhei Sato, Yoshinori Dobashi, Theodore Kim, and Tomoyuki Nishita. Example-based turbulence style transfer. ACM Transactions on Graphics (TOG), 37:1–9, 2018. 211. Connor Schenck and Dieter Fox. Spnets: Differentiable fluid dynamics for deep neural networks. In Aude Billard, Anca Dragan, Jan Peters, and Jun Morimoto, editors, Proceedings

References

157

of The 2nd Conference on Robot Learning, volume 87 of Proceedings of Machine Learning Research, pages 317–335. PMLR, 29–31 Oct 2018. 212. Pulkit Sharma. A Comprehensive Tutorial to learn Convolutional Neural Networks from Scratch, 2018. 213. Peter Shirley. Ray Tracing in One Weekend, volume 1. Kindle Edition, 1st edition, 2016. 214. Peter Shirley. Ray Tracing: The Next Week, volume 2. Kindle Edition, 1st edition, 2016. 215. Peter Shirley. Ray Tracing: The Rest of Your Life, volume 3. Kindle Edition, 1st edition, 2016. 216. Peter Shirley and Steve Marschner. Fundamentals of computer graphics. AK Peters/CRC Press, 5th edition, 2021. 217. Osnat Shurer, Kyle Odermatt, Hank Driskill, Dale Mayeda, and Adolph Lusinsky. Stories the ocean tells us: the making of" moana". In ACM SIGGRAPH 2017 Production Sessions, pages 9–9, 2017. 218. Peter Sikachev, Martin Palko, and Alexandre Chekroun. Real-time fluid simulation in shadow of the tomb raider. Talk presented at the meeting of 4C Prague 2018, 2018. 219. SPlisHSPlasH. Open-source library for physically-based simulation of fluids using SPH. https://splishsplash.readthedocs.io/en/2.9.0/about.html, 2021. 220. Jos Stam. Stable fluids. ACM SIGGRAPH 99, 1999, 11 2001. 221. Ben Stevens and Tim Colonius. Enhancement of shock-capturing methods via machine learning. Theoretical and Computational Fluid Dynamics, pages 1–14, 2020. 222. Alexey Stomakhin, Craig Schroeder, Lawrence Chai, Joseph Teran, and Andrew Selle. A material point method for snow simulation. ACM Transactions on Graphics (TOG), 32(4):1– 10, 2013. 223. Wolfgang Straßer. Schnelle kurven-und flächendarstellung auf grafischen sichtgeräten. PhD thesis, Universität Berlin, 1974. 224. John C. Strikwerda. Finite difference methods for the stokes and navier–stokes equations. SIAM Journal on Scientific and Statistical Computing, 5(1):56–68, 1984. 225. Ines Stuppacher and Peter Supan. Rendering of water drops in real-time. In Proceedings of the Central European Seminar on Computer Graphics for Students, 2007. 226. Deborah Sulsky, Shi-Jian Zhou, and Howard L. Schreyer. Application of a particle-incell method to solid mechanics. Computer Physics Communications, 87(1):236–252, 1995. Particle Simulation Methods. 227. K. Symon. Mechanics. Texts in Applied Mathematics. Addison-Wesley Publishing Company, 1971. 228. P. Tan, M. Steinbach, and V. Kumar. Introduction to Data Mining. Addison-Wesley, 2005. 229. Roger Temam. Navier—Stokes Equations Theory and Numerical Analysis. Elsevier North Holland, 2 edition, 1979. 230. Ayush Tewari, Ohad Fried, Justus Thies, Vincent Sitzmann, Stephen Lombardi, Kalyan Sunkavalli, Ricardo Martin-Brualla, Tomas Simon, Jason Saragih, Matthias Nießner, and others, State of the art on neural rendering. Computer Graphics Forum, 39(2):701–727, 2020. 231. Anju Tewari, Otto Fried, Justus Thies, Vincent Sitzmann, S. Lombardi, Z Xu, Tanaba Simon, Matthias Nießner, Edgar Tretschk, L. Liu, Ben Mildenhall, Pranatharthi Srinivasan, R. Pandey, Sergio Orts-Escolano, S. Fanello, M. Guang Guo, Gordon Wetzstein, J y Zhu, Christian Theobalt, Manju Agrawala, Donald B. Goldman, and Michael Zollhöfer. Advances in neural rendering. Computer Graphics Forum, 41, 2021. 232. Sergios Theodoridis and Konstantinos Koutroumbas. Pattern recognition. Academic Press, 2003. 233. Nils Thuerey. Interpolations of smoke and liquid simulations. ACM Transactions on Graphics, 36(1):1–16, 2016. 234. Nils Thuerey, Philipp Holl, Maximilian Müller, Patrick Schnell, Felix Trost, and Kiwon Um. Physics-based deep learning. CoRR, abs/2109.05237, 2021. 235. Nils Thuerey and Tobias Pfaff. MantaFlow. http://mantaflow.com, 2018. 236. Nils Thürey, Filip Sadlo, Simon Schirm, Matthias Müller-Fischer, and Markus Gross. Realtime simulations of bubbles and foam within a shallow water framework. In Proceedings

158

References

of the 2007 ACM SIGGRAPH/Eurographics symposium on Computer animation, pages 191– 198, 2007. 237. Nghia Truong and Cem Yuksel. A narrow-range filter for screen-space fluid rendering. Proc. ACM Comput. Graph. Interact. Tech., 1:17:1–17:15, 2018. 238. Nghia Truong, Cem Yuksel, Chakrit Watcharopas, Joshua A Levine, and Robert M Kirby. Particle merging-and-splitting. IEEE Transactions on Visualization and Computer Graphics, 28(12):4546–4557, 2021. 239. A. Tsirikoglou, G. Eilertsen, and J. Unger. A Survey of Image Synthesis Methods for Visual Machine Learning. Computer Graphics Forum, 39(6):426–451, 2020. 240. Jiyuan Tu, Kiao Inthavong, and Kelvin Kian Loong Wong. Computational Hemodynamics Theory, Modelling and Applications. Springer, 2015. 241. Kiwon Um, Xiangyu Hu, and Nils Thuerey. Liquid splash modeling with neural networks. Computer Graphics Forum, 37(8):171–182, 2018. 242. Benjamin Ummenhofer, Lukas Prantl, Nils Thuerey, and Vladlen Koltun. Lagrangian fluid simulation with continuous convolutions. In International Conference on Learning Representations, 2020. 243. Benjamin Ummenhofer, Lukas Prantl, Nils Thuerey, and Vladlen Koltun. Lagrangian fluid simulation with continuous convolutions. In International Conference on Learning Representations, 2019. 244. Julien Valentin, Cem Keskin, Pavel Pidlypenskyi, Ameesh Makadia, Avneesh Sud, and Sofien Bouaziz. Tensorflow graphics: Computer graphics meets deep learning. TensorflowGraphicsIO2019. Available online at: https://github.com/tensorflow/graphics (accessed July 2, 2020), 2019. 245. Wladimir J. van der Laan, Simon Green, and Miguel Sainz. Screen space fluid rendering with curvature flow. In I3D ’09, 2009. 246. Vladimir N. Vapnik. Statistical Learning Theory. John Wiley & Sons, INC., 1998. 247. Andreas Vasilakis, Konstantinos Vardis, and Georgios Papaioannou. A survey of multifragment rendering. Computer Graphics Forum, 39, 2020. 248. Eric Veach and Leonidas J Guibas. Optimally combining sampling techniques for monte carlo rendering. In Proceedings of the 22nd annual conference on Computer graphics and interactive techniques, pages 419–428, 1995. 249. Delio Vicini, Sébastien Speierer, and Wenzel Jakob. Differentiable signed distance function rendering. ACM Transactions on Graphics, 41(4), 2022. 250. Marcelo Bernardes Vieira, Gilson Antonio Giraldi, Allan Carlos Amaral Ribeiro, Marcelo Caniato Renhe, and Claudio Esperança. Anisotropic helmholtz decomposition for controlled fluid simulation. Appl. Math. Comput., 411:126501, 2021. 251. Ricardo Vinuesa and Steven L. Brunton. The Potential of Machine Learning to Enhance Computational Fluid Dynamics. Arxiv, pages 1–13, 2021. 252. Laura von Rueden, Sebastian Mayer, Jochen Garcke, Christian Bauckhage, and Jannis Schücker. Informed machine learning - towards a taxonomy of explicit integration of knowledge into machine learning. ArXiv, abs/1903.12394, 2019. 253. Jiˇrí Vorba, Ondˇrej Karlík, Martin Šik, Tobias Ritschel, and Jaroslav Kˇrivánek. On-line learning of parametric mixture models for light transport simulation. ACM Transactions on Graphics (TOG), 33(4):1–11, 2014. 254. J. Wang. Geometric Structure of High-Dimensional Data and Dimensionality Reduction. Springer Berlin Heidelberg, 2012. 255. Qi Wang, Yue Ma, Kun Zhao, and Yingjie Tian. A comprehensive survey of loss functions in machine learning. Annals of Data Science, 9, 04 2022. 256. Yifan Wang, Felice Serena, Shihao Wu, Cengiz Öztireli, and Olga Sorkine-Hornung. Differentiable surface splatting for point-based geometry processing. ACM Transactions on Graphics (TOG), 38:1–14, 2019. 257. Ying Da Wang, Traiwit Chung, Ryan T. Armstrong, and Peyman Mostaghimi. Ml-lbm: Machine learning aided flow simulation in porous media. ArXiv, abs/2004.11675, 2020.

References

159

258. Mengdi Wang, Yitong Deng, Xiangxin Kong, Aditya H. Prasad, Shiying Xiong, and Bo Zhu. Thin-film smoothed particle hydrodynamics fluid. ACM Transactions on Graphics (TOG), 40:1–16, 2021. https://api.semanticscholar.org/CorpusID:234741780. 259. Sebastian Weiss and Rüdiger Westermann. Differentiable direct volume rendering. IEEE Transactions on Visualization and Computer Graphics, PP:1–1, 2021. 260. Daniel S. Weld and Gagan Bansal. The challenge of crafting intelligible intelligence. Communications of the ACM, 62:70–79, 2019. 261. Qingsong Wen, Liang Sun, Xiaomin Song, Jingkun Gao, Xue Wang, and Huan Xu. Time series data augmentation for deep learning: A survey. CoRR, abs/2002.12478, 2020. 262. Maximilian Werhahn, You Xie, Mengyu Chu, and Nils Thuerey. A multi-pass GaN for fluid flow super-resolution. Proceedings of the ACM on Computer Graphics and Interactive Techniques, 2(2):1–21, 2019. 263. Gabriel D Weymouth. Data-driven multi-grid solver for accelerated pressure projection, 2021. 264. Steffen Wiewel, Moritz Becher, and Nils Thuerey. Latent Space Physics: Towards Learning the Temporal Evolution of Fluid Flow. Computer Graphics Forum, 2019. 265. Wikipedia. Kullback-Leibler divergence. https://en.wikipedia.org/wiki/Kullback-Leibler_ divergence. 266. Wikipedia. Activation Function. https://en.wikipedia.org/wiki/Activation_function, 2020. 267. Wikipedia. Machine learning. https://en.wikipedia.org/wiki/Machine_learning, 2023. 268. Wikipedia. Turbulence modeling. https://en.wikipedia.org/wiki/Turbulence_modeling, 2023. 269. Jared D. Willard, Xiaowei Jia, Shaoming Xu, Michael S. Steinbach, and Vipin Kumar. Integrating physics-based modeling with machine learning: A survey. ArXiv, abs/2003.04919, 2020. 270. Lance Williams. Casting curved shadows on curved surfaces. In Proceedings of the 5th annual conference on Computer graphics and interactive techniques, pages 270–274, 1978. 271. Rene Winchenbach and Andreas Kolb. Optimized refinement for spatially adaptive SPH. ACM Transactions on Graphics (TOG), 40(1):1–15, 2021. 272. Mitchell Woodward, Yifeng Tian, Criston Hyett, Chris L. Fryer, Daniel Livescu, Mikhail Stepanov, and Michael Chertkov. Physics informed machine learning of SPH: Machine learning lagrangian turbulence. ArXiv, abs/2110.13311, 2021. 273. Magnus Wrenninge. Production volume rendering: design and implementation. CRC Press, 2012. 274. Jin-Long Wu, Heng Xiao, and Eric Paterson. Physics-informed machine learning approach for augmenting turbulence models: A comprehensive framework. Phys. Rev. Fluids, 3:074602, Jul 2018. 275. Kui Wu, Nghia Truong, Cem Yuksel, and Rama Hoetzlein. Fast fluid simulations with sparse volumes on the GPU. Computer Graphics Forum, 37, 2018. 276. Xiangyun Xiao, Shuai Zhang, and Xubo Yang. Real-time high-quality surface rendering for large scale particle-based fluids. Proceedings of the 21st ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games, 2017. 277. Yiheng Xie, Towaki Takikawa, Shunsuke Saito, Or Litany, Shiqin Yan, Numair Khan, Federico Tombari, James Tompkin, Vincent Sitzmann, and Srinath Sridhar. Neural Fields in Visual Computing and Beyond. Computer Graphics Forum, 41(2):641–676, 2022. 278. You Xie, Erik Franz, Mengyu Chu, and Nils Thuerey. Tempogan: A temporally coherent, volumetric gan for super-resolution fluid flow. ACM Trans. Graph., 37(4), Jul 2018. 279. Yanrui Xu, Yuanmu Xu, Yuege Xiong, Dou Yin, Xiaojuan Ban, Xiaokun Wang, Jian Chang, and Jian Jun Zhang. Anisotropic screen space rendering for particle-based fluid simulation. Computers & Graphics, 110:118–124, 2023. 280. Zexiang Xu, Kalyan Sunkavalli, Sunil Hadap, and Ravi Ramamoorthi. Deep image-based relighting from optimal sparse samples. ACM Transactions on Graphics (TOG), 37:1–13, 2018. 281. Cheng Yang, Xubo Yang, and Xiangyun Xiao. Data-driven projection method in fluid simulation. Computer Animation and Virtual Worlds, 27(3–4):415–424, 2016.

160

References

282. Wencong Yang and Chengying Gao. A completely parallel surface reconstruction method for particle-based fluids. The Visual Computer, 36:2313–2325, 2020. 283. Allen Rowe York II. Development of modifications to the material point method for the simulation of thin membranes, compressible fluids, and their interactions. The University of New Mexico, 1997. 284. Jihun Yu and Greg Turk. Reconstructing surfaces of particle-based fluids using anisotropic kernels. ACM Trans. Graph., 32:5:1–5:12, 2010. 285. Matteo Zancanaro, Markus Mrosek, Giovanni Stabile, Carsten Othmer, and Gianluigi Rozza. Hybrid neural network reduced order modelling for turbulent flows with geometric parameters. Fluids, 6(8), 2021. 286. Cheng Zhang, Bailey Miller, Kai Yan, Ioannis Gkioulekas, and Shuang Zhao. Path-space differentiable rendering. ACM Transactions on Graphics (TOG), 39:143:1–143:19, 2020. 287. Linyang Zhu, Weiwei Zhang, Xuxiang Sun, Yilang Liu, and Xianxu Yuan. Turbulence closure for high reynolds number airfoil flows by deep neural networks. Aerospace Science and Technology, 110:106452, 2021. 288. Shilin Zhu. Survey: Machine Learning in Production Rendering. arXiv preprint arXiv:2005.12518, 2, 2020. 289. Yongning Zhu and Robert Bridson. Animating sand as a fluid. ACM Trans. Graph., 24:965– 972, 2005. 290. Tobias Zirr and Carsten Dachsbacher. Memory-efficient on-the-fly voxelization and rendering of particle data. IEEE Transactions on Visualization and Computer Graphics, 24:1155–1166, 2018. 291. Matthias Zwicker, Wojciech Jarosz, Jaakko Lehtinen, Bochang Moon, Ravi Ramamoorthi, Fabrice Rousselle, Pradeep Sen, Cyril Soler, and Sung eui Yoon. Recent advances in adaptive sampling and reconstruction for monte carlo rendering. Computer Graphics Forum, 34, 2015.

Index

A Activation function, 42 hyperbolic tangent function, 42 leaky ReLu function, 43 parametric ReLu function, 43 ReLu function, 42 sigmoid function, 42 softmax activation function, 43

B Boolean variable, 34 bi-implication, 34 Boolean expression, 34, 35 Boolean operators, 34 conjuntion, 34 disjunction, 34 implication, 34 negation, 34 truth tables, 34 XOR, 34

C Circuit, 36 circuit Family, 37 depth complexity, 37 elementary gate, 36 F AN OU T , 36 identity, 36 size complexity, 37 threshold circuit, 39 threshold gate, 38 Computer animation, 1

geometric modeling, 2 physics-based animation, 2 Continuous convolutions neural network (ConvNet), 102 CConv InFeatures, 105 OutPositions, 105 CConv operator, 104 data augmentation, 106 discrete convolution, 103, 104 implementation, 105 input, 104 loss, 106 output, 104 training data, 106 Convolutional neural network (CNN), 68 biases, 74 computational complexity, 75 computational cost, 75 convolution over volume, 72 discrete convolution, 69 filters, 74 kernel, 69 padding, 70 pooling, 72 stride, 71

D Data augmentation, 66 Data model, 48 Data preparation, 48 data collection, 48 feature engineering, 48

© The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 G. A. Giraldi et al., Deep Learning for Fluid Simulation and Animation, SpringerBriefs in Mathematics, https://doi.org/10.1007/978-3-031-42333-8

161

162 Differentiable manifold, 46 geodesic, 47 local parametrization, 46 Riemannian manifold, 47 submanifold, 46 tangent bundle, 47 tangent space, 47 tangent vector, 46 Double dam breaking, 113 average simulation time, 120 domain, 113 parameters, 113 snapshots, 117, 118

E Empirical data, 66 Euler-Lagrange equations, 51 Explainable artificial intelligence (EAI), 138, 141 explainability, 141 interpretability, 141 shapley additive explanations, 142 transparency, 141

F Fluid dynamics, 11 Eulerian formulation, 12 Lagrangian formulation, 12 mass density, 13 Navier–Stokes equations, 11, 12, 16 Fountain simulation, 107 Average Simulation Time, 113 ConvNet, 108 emitter, 108 domain, 107 snapshots, 111 SPlisHSPlasH, 107 emitter, 107 parameters, 107 Fountain super-resolution, 127 domain, 127 emitter, 127 low resolution processing, 128 MantaFlow, 127 rendering, 128 snapshots, 128, 129

G Generative adversarial network (GAN), 75 discriminator, 76 Euler–Lagrange equation, 77

Index formulation, 76 generator, 75 latent space, 76 minimax optimization, 76 training, 76 training algorithm, 78

H Hamiltonian formulation, 52 Hamilton equations, 52 Hamiltonian function, 52 Hamilton’s principle, 51 kinetic energy, 51 scalar potential, 51

K K-fold cross-validation, 63 Kullback-Leibler divergence, 57

L Learning Theory, 2 classification problem, 55 cross entropy, 57 error function, 56 learning machine, 56 learning model, 56 learning problem, 55 loss function, 3, 57 maximum likelihood Estimation, 57 regression problem, 56 reinforcement learning, 3 semi-supervised learning, 3, 58 supervised learning, 3, 58 training, 3 unsupervised learning, 3, 58 Loss function hyperparameters, 57 regularization terms, 57

M Manifold learning, 50 MantaFlow, 124 Mathematical induction, 37 Multilayer perceptron, 40 deep neural Network, 42 fully connected neural network, 42 hidden layers, 40 input layer, 40 output layer, 41 shallow neural network, 42

Index N Neural Networks, 40, 65 auto encoder, 68 convolutional neural network, 68 data collection, 66 generative adversarial network, 68 hyperparameters, 67 long short term memory, 68 multilayer perceptron, 68 problem formulation, 65 P Perceptron, 39, 40 activation function, 39 bias, 39 Hebb learning rule, 39 induced local field, 39 McCulloch–Pitts neuron, 39 weights, 39 Physics-informed neural network (PINNs), 67, 138, 139 automatic differentiation, 140 loss, 139 projection, 140 Propositional algebra, 35 Absorption laws, 36 Associative law, 35 Commutative law, 35 Complement laws, 36 De Morgan’s Laws, 36 Distributive law, 36 Idempotent laws, 35 Involution law, 36 Q Quantitative comparison, 111 R Regular surface, 46 Rendering, 2 deferred shading, 87 definition, 79 differentiable rendering, 98, 143 environment mapping, 91 G-Buffers, 86 image-based rendering, 87, 96 image super-resolution, 99 implicit objects, 84 marching cubes, 85 neural radiance field, 97, 144 neural rendering, 97, 143 optical effects, 91

163 path tracing, 81, 96 Phong reflection model, 81 rasterization, 81 ray marching, 90 ray tracing, 82, 93 refraction, 92 rendering equation, 80, 81 screen-space algorithms, 86, 88 soft objects, 84 style transfer, 99 surface extration, 84 visibility, 95 volume rendering, 88 volume rendering equation, 89 Root Square Error (RSE), 129

S Smoke columns super-resolution, 131 boundary condition, 131 domain, 129 down-sampling, 131 snapshots, 131, 135 Smoothed particle hydrodynamics (SPH), 16 artificial viscosity, 20 boundary forces, 21 free surfaces, 22 interpolation kernel, 17 Leap-Frog, 21 SPlisHSPlasH, 107 Super-resolution, 6, 66 Surrogate model, 66

T TempoGAN, 102, 121 algorithm, 124 architecture, 124 data augmentation, 124 discriminators architectures, 126 geneator, 121 losses, 123 overview, 123 results, 127 spatial discriminator, 122 structure, 122 temporal discriminator, 122 training, 124 training data, 122 Test stage, 62, 67 test set, 61 Training process, 58, 61 back-propagation algorithm, 60, 67 cost function, 60

164 Training process (cont.) epoch, 59 loss over the minibatch, 59 minibatch, 59 MLP initialization, 59 overfitting, 61 stochastic gradient descent algorithm, 59 training set, 61 underfitting, 61 validation set, 61

Index Training stage, 67 Turbulence, 6

U Universality of neural networks model, 43

V Vanishing gradient problem, 42