Simulation and Modeling of Systems of Systems [1 ed.] 9781118616659, 9781848212343
Systems engineering is the design of a complex interconnection of many elements (a system) to maximize a specific measur
288 27 6MB
English Pages 394 Year 2011
Polecaj historie
![Guide to modeling and simulation of systems of systems [Second edition]
978-3-319-64134-8, 3319641344, 978-3-319-64133-1](https://dokumen.pub/img/200x200/guide-to-modeling-and-simulation-of-systems-of-systems-second-edition-978-3-319-64134-8-3319641344-978-3-319-64133-1.jpg)
![Agricultural Systems Modeling and Simulation [1 ed.]
9781315214979, 9781351830867, 9781351822176, 9781482269765, 9781420069181, 9780824700416, 9780367400859](https://dokumen.pub/img/200x200/agricultural-systems-modeling-and-simulation-1nbsped-9781315214979-9781351830867-9781351822176-9781482269765-9781420069181-9780824700416-9780367400859.jpg)
![Simulation and Modeling of Systems of Systems [1 ed.]
9781118616659, 9781848212343](https://dokumen.pub/img/200x200/simulation-and-modeling-of-systems-of-systems-1nbsped-9781118616659-9781848212343.jpg)
Citation preview
Simulation and Modeling of Systems of Systems
Simulation and Modeling of Systems of Systems
Edited by Pascal Cantot Dominique Luzeaux
First published 2011 in Great Britain and the United States by ISTE Ltd and John Wiley & Sons, Inc. Adapted and updated from Simulation et modélisation des systèmes de systèmes : vers la maîtrise de la complexité published 2009 in France by Hermes Science/Lavoisier © LAVOISIER 2009 Apart from any fair dealing for the purposes of research or private study, or criticism or review, as permitted under the Copyright, Designs and Patents Act 1988, this publication may only be reproduced, stored or transmitted, in any form or by any means, with the prior permission in writing of the publishers, or in the case of reprographic reproduction in accordance with the terms and licenses issued by the CLA. Enquiries concerning reproduction outside these terms should be sent to the publishers at the undermentioned address: ISTE Ltd 27-37 St George’s Road London SW19 4EU UK
John Wiley & Sons, Inc. 111 River Street Hoboken, NJ 07030 USA
www.iste.co.uk
www.wiley.com
© ISTE Ltd 2011 The rights of Pascal Cantot and Dominique Luzeaux to be identified as the authors of this work have been asserted by them in accordance with the Copyright, Designs and Patents Act 1988. ____________________________________________________________________________________ Library of Congress Cataloging-in-Publication Data Simulation et modelisation des systemes de systemes. English Simulation and modeling of systems of systems / edited by Pascal Cantot, Dominique Luzeaux. p. cm. Includes bibliographical references and index. ISBN 978-1-84821-234-3 1. Systems engineering--Data processing. 2. Computer simulation. I. Cantot, Pascal. II. Luzeaux, Dominique. III. Title. TA168.S522813 2011 003--dc22 2011006656 British Library Cataloguing-in-Publication Data A CIP record for this book is available from the British Library ISBN 978-1-84821-234-3 Printed and bound in Great Britain by CPI Antony Rowe, Chippenham and Eastbourne.
Table of Contents
Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
xi
Chapter 1. Simulation: History, Concepts, and Examples . . . . . . . . . . . Pascal CANTOT
1
1.1. Issues: simulation, a tool for complexity. . . 1.1.1. What is a complex system? . . . . . . . . 1.1.2. Systems of systems . . . . . . . . . . . . . 1.1.3. Why simulate? . . . . . . . . . . . . . . . 1.1.4. Can we do without simulation?. . . . . . 1.2. History of simulation . . . . . . . . . . . . . . 1.2.1. Antiquity: strategy games . . . . . . . . . 1.2.2. The modern era: theoretical bases . . . . 1.2.3. Contemporary era: the IT revolution . . 1.3. Real-world examples of simulation. . . . . . 1.3.1. Airbus. . . . . . . . . . . . . . . . . . . . . 1.3.2. French defense procurement directorate 1.4. Basic principles . . . . . . . . . . . . . . . . . 1.4.1. Definitions . . . . . . . . . . . . . . . . . . 1.4.2. Typology . . . . . . . . . . . . . . . . . . . 1.5. Conclusion . . . . . . . . . . . . . . . . . . . . 1.6. Bibliography . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . .
1 1 3 5 12 14 14 15 18 24 24 26 29 30 37 51 52
Chapter 2. Principles of Modeling . . . . . . . . . . . . . . . . . . . . . . . . . . Pascal CANTOT
57
2.1. Introduction to modeling . . . 2.2. Typology of models . . . . . . 2.2.1. Static/dynamic. . . . . . . 2.2.2. Deterministic/stochastic .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . . . . . . . . . . . . . . .
. . . .
. . . . . . . . . . . . . . . . .
. . . .
. . . . . . . . . . . . . . . . .
. . . .
. . . . . . . . . . . . . . . . .
. . . .
. . . . . . . . . . . . . . . . .
. . . .
. . . . . . . . . . . . . . . . .
. . . .
. . . . . . . . . . . . . . . . .
. . . .
. . . . . . . . . . . . . . . . .
. . . .
. . . . . . . . . . . . . . . . .
. . . .
. . . . . . . . . . . . . . . . .
. . . .
. . . . . . . . . . . . . . . . .
. . . .
. . . . . . . . . . . . . . . . .
. . . .
. . . . . . . . . . . . . . . . .
. . . .
. . . . . . . . . . . . . . . . .
. . . .
. . . . . . . . . . . . . . . . .
. . . .
. . . .
57 58 58 59
vi
Simulation & Modeling of Systems of Systems
2.2.3. Qualities of a model . . . . . . . 2.3. The modeling process. . . . . . . . . 2.3.1. Global process. . . . . . . . . . . 2.3.2. Formulation of the problem . . . 2.3.3. Objectives and organization. . . 2.3.4. Analysis of the system . . . . . . 2.3.5. Modeling . . . . . . . . . . . . . . 2.3.6. Data collection . . . . . . . . . . 2.3.7. Coding/implementation . . . . . 2.3.8. Verification . . . . . . . . . . . . 2.3.9. Validation . . . . . . . . . . . . . 2.3.10. Execution . . . . . . . . . . . . . 2.3.11. Use of results. . . . . . . . . . . 2.3.12. Final report . . . . . . . . . . . . 2.3.13. Commissioning/capitalization. 2.4. Simulation project management. . . 2.5. Conclusion . . . . . . . . . . . . . . . 2.6. Bibliography . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
63 66 67 68 70 71 76 78 82 87 87 87 89 90 90 91 94 94
Chapter 3. Credibility in Modeling and Simulation . . . . . . . . . . . . . . . Roland RABEAU
99
3.1. Technico-operational studies and simulations . . . 3.2. Examples of technico-operational studies based on simulation tools . . . . . . . . . . . . . . . . . . . . . . . . 3.2.1. Suppression of aerial defenses . . . . . . . . . . 3.2.2. Heavy helicopters. . . . . . . . . . . . . . . . . . 3.3. VV&A for technico-operational simulations . . . . 3.3.1. Official definitions . . . . . . . . . . . . . . . . . 3.3.2. Credibility . . . . . . . . . . . . . . . . . . . . . . 3.3.3. Key players in the domain. . . . . . . . . . . . . 3.4. VV&A issues. . . . . . . . . . . . . . . . . . . . . . . 3.4.1. Elements concerned . . . . . . . . . . . . . . . . 3.4.2. Verification and validation techniques . . . . . 3.4.3. VV&A approaches . . . . . . . . . . . . . . . . . 3.4.4. Responsibilities in a VV&A process . . . . . . 3.4.5. Levels of validation . . . . . . . . . . . . . . . . 3.4.6. Accreditation . . . . . . . . . . . . . . . . . . . . 3.5. Conclusions. . . . . . . . . . . . . . . . . . . . . . . . 3.5.1. Validation techniques . . . . . . . . . . . . . . . 3.5.2. Validation approaches . . . . . . . . . . . . . . . 3.5.3. Perspectives . . . . . . . . . . . . . . . . . . . . . 3.6. Bibliography . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . .
99 101 101 101 102 102 103 105 108 108 114 123 141 144 144 145 145 147 150 152
Table of Contents Chapter 4. Modeling Systems and Their Environment . . . . . . . . . . . . . Pascal CANTOT 4.1. Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2. Modeling time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.1. Real-time simulation . . . . . . . . . . . . . . . . . . . . . . . 4.2.2. Step-by-step simulation . . . . . . . . . . . . . . . . . . . . . 4.2.3. Discrete event simulation . . . . . . . . . . . . . . . . . . . . 4.2.4. Which approach? . . . . . . . . . . . . . . . . . . . . . . . . 4.2.5. Distributed simulation . . . . . . . . . . . . . . . . . . . . . . 4.3. Modeling physical laws. . . . . . . . . . . . . . . . . . . . . . . . 4.3.1. Understanding the system . . . . . . . . . . . . . . . . . . . . 4.3.2. Developing a system of equations . . . . . . . . . . . . . . . 4.3.3. Discrete sampling of space . . . . . . . . . . . . . . . . . . . 4.3.4. Solving the problem . . . . . . . . . . . . . . . . . . . . . . . 4.4. Modeling random phenomena . . . . . . . . . . . . . . . . . . . . 4.4.1. Stochastic processes . . . . . . . . . . . . . . . . . . . . . . . 4.4.2. Use of probability. . . . . . . . . . . . . . . . . . . . . . . . . 4.4.3. Use of statistics . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4.4. Random generators . . . . . . . . . . . . . . . . . . . . . . . . 4.4.5. Execution and analysis of results of stochastic simulations 4.5. Modeling the natural environment . . . . . . . . . . . . . . . . . 4.5.1. Natural environment . . . . . . . . . . . . . . . . . . . . . . . 4.5.2. Environment databases . . . . . . . . . . . . . . . . . . . . . 4.5.3. Production of an SEDB . . . . . . . . . . . . . . . . . . . . . 4.5.4. Quality of an SEDB . . . . . . . . . . . . . . . . . . . . . . . 4.5.5. Coordinate systems . . . . . . . . . . . . . . . . . . . . . . . . 4.5.6. Multiplicity of formats . . . . . . . . . . . . . . . . . . . . . . 4.6. Modeling human behavior . . . . . . . . . . . . . . . . . . . . . . 4.6.1. Issues and limitations . . . . . . . . . . . . . . . . . . . . . . 4.6.2. What is human behavior? . . . . . . . . . . . . . . . . . . . . 4.6.3. The decision process . . . . . . . . . . . . . . . . . . . . . . . 4.6.4. Perception of the environment . . . . . . . . . . . . . . . . . 4.6.5. Human factors. . . . . . . . . . . . . . . . . . . . . . . . . . . 4.6.6. Modeling techniques . . . . . . . . . . . . . . . . . . . . . . . 4.6.7. Perspectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.7. Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
vii 159
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
159 160 160 161 162 162 162 163 163 164 165 166 166 166 167 171 173 175 178 178 178 180 182 183 185 193 193 194 196 197 198 199 202 203
Chapter 5. Modeling and Simulation of Complex Systems: Pitfalls and Limitations of Interpretation . . . . . . . . . . . . . . . . . . . . . . . . . . Dominique LUZEAUX
207
5.1. Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2. Complex systems, models, simulations, and their link with reality . . .
207 209
viii
Simulation & Modeling of Systems of Systems
5.2.1. Systems. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2.2. Complexity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2.3. The difficulty of concepts: models, modeling, and simulation 5.3. Main characteristics of complex systems simulation . . . . . . . . 5.3.1. Nonlinearity, the key to complexity . . . . . . . . . . . . . . . . 5.3.2. Limits of computing: countability and computability. . . . . . 5.3.3. Discrete or continuous models . . . . . . . . . . . . . . . . . . . 5.4. Review of families of models . . . . . . . . . . . . . . . . . . . . . . 5.4.1. Equational approaches . . . . . . . . . . . . . . . . . . . . . . . . 5.4.2. Computational approaches . . . . . . . . . . . . . . . . . . . . . 5.4.3. Qualitative phenomenological approaches . . . . . . . . . . . . 5.4.4. Qualitative structuralist approach: application of category theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.5. An example: effect-based and counter-insurgency military operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.6. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.7. Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . .
209 211 215 218 218 223 226 228 229 232 237
. . .
240
. . . . . . . . .
244 246 249
Chapter 6. Simulation Engines and Simulation Frameworks . . . . . . . . . Pascal CANTOT
253
6.1. Introduction. . . . . . . . . . . . . . . . . . . . . . 6.2. Simulation engines . . . . . . . . . . . . . . . . . 6.2.1. Evolution of state variables . . . . . . . . . . 6.2.2. Management of events and causality . . . . 6.2.3. Simulation modes. . . . . . . . . . . . . . . . 6.2.4. Example . . . . . . . . . . . . . . . . . . . . . 6.3. Simulation frameworks . . . . . . . . . . . . . . . 6.3.1. Some basic points on software engineering 6.3.2. Frameworks . . . . . . . . . . . . . . . . . . . 6.3.3. Obstacles to framework use . . . . . . . . . . 6.3.4. Detailed example: ESCADRE . . . . . . . . 6.4. Capitalization of models . . . . . . . . . . . . . . 6.5. Conclusion and perspectives. . . . . . . . . . . . 6.6. Bibliography . . . . . . . . . . . . . . . . . . . . .
253 254 254 255 256 258 260 260 268 270 272 290 291 292
Chapter 7. Distributed Simulation. . . . . . . . . . . . . . . . . . . . . . . . . . Louis IGARZA
295
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . . . . . . . . . . .
. . . . .
. . . . . . . . . . . . . .
. . . . .
. . . . . . . . . . . . . .
. . . . .
. . . . . . . . . . . . . .
. . . . .
. . . . . . . . . . . . . .
. . . . .
. . . . . . . . . . . . . .
. . . . .
. . . . . . . . . . . . . .
. . . . .
. . . . . . . . . . . . . .
. . . . .
. . . . . . . . . . . . . .
. . . . .
. . . . . . . . . . . . . .
. . . . .
. . . . . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . . . . .
7.1. Introduction. . . . . . . . . . . . . . . . 7.1.1. The principle. . . . . . . . . . . . . 7.1.2. History of distributed simulations 7.1.3. Some definitions . . . . . . . . . . 7.1.4. Interoperability in simulation . . .
. . . . . . . . . . . . . .
. . . . . . . . . . .
. . . . .
. . . . . . . . . . . . . .
. . . . .
. . . . .
295 295 297 298 300
Table of Contents 7.1.5. Standardization . . . . . . . . . . . . . . . . . . . . . . . . . . 7.1.6. Advantages and limitations of distributed simulation. . . . 7.1.7. Other considerations . . . . . . . . . . . . . . . . . . . . . . . 7.2. Basic mechanisms of distributed simulation . . . . . . . . . . . 7.2.1. Some key principles . . . . . . . . . . . . . . . . . . . . . . . 7.2.2. Updating attributes . . . . . . . . . . . . . . . . . . . . . . . . 7.2.3. Interactions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.2.4. Time management . . . . . . . . . . . . . . . . . . . . . . . . 7.2.5. Dead reckoning . . . . . . . . . . . . . . . . . . . . . . . . . . 7.2.6. Multi-level modeling. . . . . . . . . . . . . . . . . . . . . . . 7.2.7. Section conclusion . . . . . . . . . . . . . . . . . . . . . . . . 7.3. Main interoperability standards . . . . . . . . . . . . . . . . . . . 7.3.1. History . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.3.2. HLA. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.3.3. DIS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.3.4. TENA. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.3.5. The future of distributed simulation: the LVC AR study. . 7.3.6. Other standards used in distributed simulation. . . . . . . . 7.4. Methodological aspects: engineering processes for distributed simulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.4.1. FEDEP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.4.2. SEDEP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.4.3. DSEEP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.5. Conclusion: the state of the art: toward “substantive” interoperability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.6. Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
ix
. . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
302 303 303 305 305 306 307 308 309 310 311 312 312 313 319 321 324 325
. . . .
. . . .
. . . .
. . . .
. . . .
326 327 329 330
. . . . . . . . . .
331 331
Chapter 8. The Battle Lab Concept . . . . . . . . . . . . . . . . . . . . . . . . . Pascal CANTOT
333
8.1. Introduction. . . . . . . . . . . . . . . . . . . . . . . . . 8.2. France: Laboratoire Technico-Opérationnel (LTO) . 8.2.1. Historical overview. . . . . . . . . . . . . . . . . . 8.2.2. Aims of the LTO . . . . . . . . . . . . . . . . . . . 8.2.3. Principles of the LTO . . . . . . . . . . . . . . . . 8.2.4. Services of the LTO . . . . . . . . . . . . . . . . . 8.2.5. Experimental process . . . . . . . . . . . . . . . . 8.2.6. Presentation of an experiment: PHOENIX 2008 8.2.7. Evaluation and future perspectives of the LTO . 8.3. United Kingdom: the Niteworks project . . . . . . . . 8.4. Conclusion and perspectives. . . . . . . . . . . . . . . 8.5. Bibliography . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . .
. . . . . . . . . . . .
. . . . . . . . . . . .
. . . . . . . . . . . .
. . . . . . . . . . . .
. . . . . . . . . . . .
. . . . . . . . . . . .
. . . . . . . . . . . .
. . . . . . . . . . . .
. . . . . . . . . . . .
. . . . . . . . . . . .
333 336 336 337 338 341 342 345 349 350 351 352
x
Simulation & Modeling of Systems of Systems
Chapter 9. Conclusion: What Return on Investment Can We Expect from Simulation? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
355
Dominique LUZEAUX
9.1. Returns on simulation for acquisition . . . . . . . . . . . . . . . . 9.2. Economic analysis of gains from intelligent use of simulations . 9.2.1. Additional costs of the SBA . . . . . . . . . . . . . . . . . . . 9.2.2. Additional costs from unexpected events or bad planning . . 9.3. Multi-project acquisition . . . . . . . . . . . . . . . . . . . . . . . . 9.4. An (almost) definitive conclusion: conditions for success . . . . 9.5. Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
355 357 358 364 367 368 371
Author Biographies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
373
List of Authors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
375
Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
377
Introduction
In fields such as aeronautics, transport, telecommunications, banking, or defense, systems are becoming increasingly complex, containing more and more components, with significant heterogeneity and disparate lifecycles. The update process that follows the increasingly rapid obsolescence of sub-systems requires a priori mastery of architectures of systems of which we do not know the component configuration. Risk mitigation in the different phases of the project (from the expression of need to implementation, or even withdrawal, considering environmental protection constraints) thus becomes an essential consideration throughout the whole lifecycle of the system. Needs have also evolved. Demands in terms of performance, interoperability, cost, security, and reliability have reached a level where each project becomes a technical and managerial challenge. It is, therefore, necessary to attain new levels of flexibility and reactivity in exploring system concepts. The changeable nature of the environment and the necessary capacity of the system to adapt to different evolutions contribute to its complexity. Moreover, the generalization of information and communication technologies pushes towards the interconnection of systems, leading to systems of systems, reinforced by the context of globalization of exchanges giving rise to shared developments between economic partners: this raises questions of integration, coherency, and interoperability of one system with a higher-level system. Finally, budget, time, profitability, and time to market constraints require mastery of acquisition costs: the utility of reuse becomes evident. With such a wide variety of constraints, particular tools and processes are needed in the area of system engineering and system-of-systems engineering. Among these tools, modeling and simulation have already proven their utility and demonstrated possibilities of reducing cycle time, resource usage, and risks associated with system acquisition, while improving the quality of the systems in question and reducing global possession costs. However, the expected gains can
xii
Simulation & Modeling of Systems of Systems
only be achieved if modeling and simulation are used in the light of suitable processes, which typically take their inspiration from convergent engineering. This work is neither an encyclopedia nor a simulation manual, although it does contain certain didactic aspects. It aims to allow the reader – be they an engineer, project manager, sponsor, or manager – to acquire a basic grounding in the field of simulation and to understand how simulation can help in mastering the challenges of complex systems and systems of systems. This work will also show situations in which simulation does not help; simulation not only has many qualities but also has specific limitations and constraints. Each chapter may be read independently, but we would strongly recommend reading Chapter 1 first. Chapter 1 gives a first cultural overview of simulation. A brief historical summary shows the origins of the discipline, followed by an explanation of the broad basic principles of simulation. Examples taken from areas particularly concerned with complex system problems (e.g. aeronautics and defense) give an initial glimpse of the uses of simulation by large companies Chapter 2 describes a typical model and simulation engineering process, followed by a detailed explanation of each step, illustrated by numerous examples to assist understanding. Chapter 3 is dedicated to the reliability of data, models, and simulations. When decisions (e.g. design choices) are based on the results of a simulation, it is important to know how far the results of the simulation in question can be trusted. This consideration is of fundamental importance and demands particular attention, as much from the developer of the simulation as from the user or purchaser. Chapter 4 deals with techniques used in modeling different aspects of a system. This section is slightly more technical, but it is important to understand what demands can reasonably be made of a simulation (and what we can reasonably expect to pay), and what this implies, for example, in terms of data collection, technology, development time, or processing power. Chapter 5 tackles the specific case of complex systems. Without going into the mathematical detail often found in articles dealing with complexity, the principal characteristics are pointed out, information that we should keep in mind when concerned with the modeling and simulation of systems of this kind. In spite of being aware of precautions necessary when using complex systems, frequent users of models and simulations can easily revert to old habits, ignoring the traps and limitations found when using simulations.
Introduction
xiii
Chapter 6 covers the software engineering aspect of simulations and describes how simulations are built, based on a foundation software infrastructure known as a host structure, which notably provides the core of the simulation, the simulation driver. This chapter provides an understanding of the development and operation of simulations from an IT viewpoint alongside strategies for use when developing and acquiring a simulation system. Chapter 7 is based on Chapter 6 by dealing with a particular architecture, that of distributed simulations, which are particularly well suited to modeling complex systems and systems of systems, but which have their own specific set of problems. A good deal of work is currently being undertaken on these simulation construction techniques, which present numerous possibilities for development as long as they are used correctly. Chapter 8 concerns a new concept in system capacity and system-of-system engineering: “battle labs”, laboratories of engineering, system and system-of-system architecture. Battle labs use a variety of tools and techniques, including simulation, in addition to processes and an organization. Although the battle lab concept is relatively new, their great potential is already evident. Finally, Chapter 9, which serves as a conclusion, goes back over certain cost aspects linked to simulation, used in an integrated manner in the engineering process for increasingly complex systems. We, the editors, hope that this work will assist the reader in understanding simulation in the context of complex system and system-of-systems engineering, a area in which it constitutes a valuable tool, albeit one with its own specificities, pitfalls, and limits, the mastery of which is essential to use the tool to its full potential. We would like to thank those individuals and businesses or organizations, other than the contributing authors, who have helped us in the creation of this work (with a very special mention for Jean-Louis Igarza for the dynamism and energy he communicated to several generations, including some of his managers!), especially Christian Sautereau, Stéphane Chaigneau, Eric Lestrade, Eric Pédo, Jérôme Martinet, the ONERA, THALES Training & Simulation, SOGITEC, EADS, MBDA, and, more generally, the members of the ADIS group (Armées, DGA, Industriesur la 1 Simulation) .
1 Armed Forces, DGA (Defense Procurement Directorate) and Defense Industry on Modeling and Simulation.
xiv
Simulation & Modeling of Systems of Systems
Finally, we would like to dedicate this work to the memory of our colleagues and friends Guy Zanon and Pierre Bouc, early contributors on distributed simulation and the validation of simulation, who actively participated in the development of the concepts presented in Chapters 3 and 7. We hope you enjoy reading this work and gain as much pleasure from reading it as we did in writing it. Pascal CANTOT and Dominique LUZEAU March 2011
Chapter 1
Simulation: History, Concepts, and Examples
1.1. Issues: simulation, a tool for complexity 1.1.1. What is a complex system? The world in which we live is rapidly advancing along a path toward complexity. Effectively, the time when component parts of a system could be dealt with individually has passed: we no longer consider the turret of a tank in isolation but consider the entire weapon system, with chassis, ammunition, fuel, communications, operating crew (who must be trained), maintenance crew, supply chain, and so on. It is also possible to consider the tank from a higher level, taking into account the interactions with other units in the course of a mission. We thus pass from a system where a specific task is carried out to a system of systems accomplishing a wide range of functions. Our tank, for example, is now just one element in a vast mechanism (or “force system”), which aims to control the aeroterrestrial sphere of the battlefield. Therefore we must no longer use purely technical system-oriented logic, but use system-of-systems-oriented capacity-driven logic1 [LUZ 10, MAI 98].
Chapter written by Pascal CANTOT. 1 [LUZ 10] defines a system of systems as follows: “a system of systems is an assembly of systems which could potentially be acquired and/or used independently, for which the developer, buyer and/or the user aim to maximize performance of the global value chain, at a given moment and for a conceivable group of assemblies”. We shall consider another definition, supplied by Maier, later on, which approaches the concept by extension rather than intension.
2
Simulation & Modeling of Systems of Systems
This is also true for a simple personal car, a subtle mixture of mechanics, electronics, and information technology (IT), the conception of which considers manufacturing, marketing, and maintenance (including the development of an adequate logistics system) and even recycling at the end of the vehicle’s useful life, a consideration which is becoming increasingly important with growing awareness of sustainable development and ecodesign. Thus, the Toyota Prius, a hybrid vehicle of which the component parts pollute more than average, has an end-of-life recycling potential, which is not only high, but also organized by the manufacturers who, for example, offer a bonus for retrieval of the NiMH traction battery, the most environmentally damaging of the car’s components. In this way, the manufacturer ensures that the battery does not end up in a standard refuse dump, but instead it follows the recycling process developed at the same time as the vehicle. In spite of this, the manufacturer is able to remain competitive. These constraints place the bar very high in engineering terms. Twenty years ago, systems were complicated, but could be simplified by successive decompositions which separated the system into components that were easy to deal with, for example, a gearbox, a steering, and an ignition. Once these components were developed and validated, they could simply be integrated following the classic V model. Nowadays, engineers are confronted more and more often with complex systems, rendering a large part of the development methods used in previous decades invalid and necessitating a new approach. Thus, what is a complex system? Complex systems are nothing new, even if they have gained an importance in the 21st century. The Semi-Automatic Ground Environment (SAGE) aerial defense systems developed by the United States in the 1950s, or Concorde in the 1960s, are examples of complex systems even if they were not labeled as such. SAGE can even be considered a system of systems. However, the methods involved were barely formalized, leading to errors and omissions in the system development processes. In 1969, Simon [SIM 69] defined a complex system as being “a system made of a large number of elements which interact in a complex manner”. Jean-Louis Le Moigne gave a clearer definition [LEM 90]: “The complexity of a system is characterized by two factors: on the one hand, the number of constituent parts, and on the other, the number of interrelations”. Globally, then, we shall judge the complexity of a system not only by the number of components but also by the relationships and dependencies between components. A product of which a large proportion is software thus becomes complex very rapidly. Other factors of complexity exist, for example, human involvement (i.e. multiple operators) in system components, the implication of random or chaotic phenomena (which make the behavior of the system nondeterministic), the use of very different time scales or trades in sub-systems, or the
Simulation: History, Concepts, and Examples
3
rapid evolution of specifications (changeable exploitation environment). An important property of complex systems is that when the sub-systems are integrated, we are often faced with unpredicted emergences, which can prove beneficial (acquisition of new capacities) or disastrous (a program may crash). A complex system is therefore much more than the sum of its parts and associated processes. Therefore, it can be characterized as non-Cartesian: it cannot be analyzed by a series of decompositions. This is the major (but not the only) challenge of complex system engineering: mastery of these emergent properties. On top of the intrinsic complexity of systems, we find increasingly strong exterior constraints that make the situation even more difficult: − increasing number of specifications to manage; − increasingly short cycles of technological obsolescence; system design is increasingly driven by new technologies; − pressure from costs and delays; − increasing necessity for interoperability between systems; − larger diversity in product ranges; − more diverse human involvement in the engineering process, but with less individual independence, with wide (including international) geographic distribution; − less acceptance of faults: strict reliability constraints, security of individuals and goods, environmental considerations, sustainable development, and so on. To manage the growing issues attached to complexity, we must perfect and adopt new methods and tools: modern global land-air defense systems could not be developed in the same way as SAGE developed in the 1950s (not without problems and major cost and schedule overruns). Observant readers may point out that a complex system loses its complexity if we manage to model and master it. It is effectively possible to see things this way; for this reason, the notion of complexity evolves with time and technological advances. Certain systems considered complex today may not be complex in the future. This work aims to contribute to this process. 1.1.2. Systems of systems In systems engineering, numerous documents, such as the ISO/IEC 15288 norm, define processes that aim to master system complexity. These processes often reach their limits once we reach a situation with systems of systems. If we can, as a first step, decompose a system of systems hierarchically into a group of systems which
4
Simulation & Modeling of Systems of Systems
cooperate to achieve a common goal, the aforementioned processes may be applied individually. However, to stop at this approach is to run considerable risks; by its very nature, a system of systems is often more than the sum of its parts. It would, of course, be naïve to restrict the characterization of systems of systems to this property, but it is the principal source of their appeal and of risks. A system of systems is a higher-level system which is not necessarily a simple “federation” of other systems. Numerous definitions of systems of systems can be found in current literature on this subject: [JAM 05] gives no less than six, and Chapter 1 of [LUZ 10] gives more than 40. In this case, we shall use the most widespread definitions, based on the so-called Maier criteria [MAI 98]: – operational independence of constituent systems (which cooperate to fulfill a common operational mission at a higher level, i.e. capacitive); – functional autonomy of constituent systems (which operate autonomously to fulfill their own operational missions); – managerial independence of constituent systems (acquired, integrated, and maintained independently); – changeable design and configuration of the system (specifications and architectures are not fixed); – emergence of new behaviors exploited to improve the capacities of each constituent system or provide new capacities (new capacities emerge via the cooperation of several systems not initially developed for this purpose); – geographical distribution of the constituent systems (from whence the particular and systematic importance of information systems and communication infrastructures in systems of systems). As a general rule, the main sources of difficulties in mastering a system of systems are as follows: – intrinsic complexity (by definition); – the multi-trade, multi-leader character of the system, which poses problems of communication and coordination between the various individuals involved, who often come from different cultural backgrounds; – uncertainty concerning specifications or even the basic need, as mastery of a system of systems presents major challenges for even the most experienced professionals. This difficulty exists on all levels, for all involved in the acquisition process, including the final user and the overseer, who may have difficulties
Simulation: History, Concepts, and Examples
5
expressing and stabilizing their needs (which may legitimately evolve due to changes in context, e.g. following a dramatic attack on a nation or a major economic crisis), making it necessary to remain agile and responsive regarding specifications; – uncertainty concerning the environment, considering the number of people concerned and the timescale of the acquisition cycle, which may, for example, lead to a component system or technology becoming obsolete even before being used. This is increasingly true due to the growing and inevitable use of technologies and commercial off-the-shelf products, particularly those linked to information and communications technology which leads to products becoming obsolete with increasing speed. To deal with these problems, a suitable approach, culture, and tools must be put into place. Simulation is an essential element of this process but is not sufficient on its own. Currently, there is no fully tested process or “magic program” that is able to deal with all these problems, although the battle lab approach does seem particularly promising as it has been developed specifically to respond to the needs of system-ofsystems engineering. This approach involves battle labs and is described in detail in Chapter 8. 1.1.3. Why simulate? As we shall see, a simulation can be extremely expensive, costing several million euros for a system-of-systems simulation, if not more. The acquisition of the French Rafale simulation centers, now operational, cost around 180 million euros (but generates savings as it removes the need to buy several Rafale training aircraft, with one aircraft costing more than double the price of the centers before even considering the maintenance costs involved in keeping them operational). The American JSIMS inter-army program was another example of this kind, but it was stopped after over one billion dollars of investment. These cases are, admittedly, extreme, but they are not unique. Some of these exceedingly costly simulation systems are themselves complex systems and have been known to fail. If, then, a simulation is so difficult to develop, why simulate? First, a word of reassurance: not all simulation programs are so expensive, and not all are failures. By following rigorous engineering procedures, notably, and a number of the processes described in the present work, it is possible to guarantee the quality of simulations, as with all industrial products. Second, simulation is not obligatory but is often a necessary means to an end. We shall review the principle constraints that can lead to working with simulations to achieve all or part of a goal.
6
Simulation & Modeling of Systems of Systems
1.1.3.1. Simulating complexity Those who have already been faced with system engineering, even briefly, will be aware of just how delicate a matter the specification and development of current systems is. The problem is made even trickier by the fact that current products (military or civilian) have become complex systems (systems of systems). We do not need to look as far as large-scale air traffic control systems or global logistics to find these systems; we encounter them in everyday life. Mobile telephone networks, for example, may be considered to be systems of systems, and a simple modern vehicle is a complex system. For these systems, a rigorous engineering approach is needed to avoid non-attainment of objectives or even complete failure. In this case, simulation can be used on several levels: – simulation can assist in the expression of a need, allowing the client to visualize their demands using a virtual model of the system, providing a precious tool for dialog between the parties involved in the project, who often do not have the same background, and guaranteeing sound mutual understanding of the desired system; – this virtual model can then be enriched and refined throughout the system development process to obtain a virtual prototype of the final system; – during the definition of the system, simulation can be used to validate technological choices and avoid falling into a technological impasse; – during testing of the system, simulations developed during the previous stages can be used to reduce the number of tests needed and/or to extend the field of system validation; – when the system is finally put into operation, simulations can be used to train system operators; – finally, if system developments are planned, existing simulations facilitate analysis of their impact on the performance of the new system. These uses show that simulation is a major tool for system engineering, to the point that nowadays it is difficult to imagine complex system engineering without simulation input. We shall not go any further into this subject at present as we shall discuss it later, but two key ideas are essential for the exploitation of simulation in system engineering: – Simulation should be integrated into the full system acquisition process to achieve its full potential: this engineering concept is known as simulation-based acquisition (SBA, see [KON 01]) in America and synthetic environment-based acquisition (see [BOS 02]) in the United Kingdom (the different national concepts have different nuances, but the general philosophy is similar).
Simulation: History, Concepts, and Examples
7
– The earlier simulation is used in a program; the more efficient it will be in reducing later risks. To understand the issue of risk reduction, we shall provide a few statistics: according to the Standish Group [STA 95], approximately one-third of the projects (in the field of IT) do not succeed. Almost half of all projects overrun in terms of cost by a factor of two or more. Only 16% of projects are achieved within the time and cost limitations established at the outset, and the more complex the project, the further this figure decreases: 9% for large companies, which, moreover, end up deploying systems which lack, on average, more than half of the functionality initially expected. We have, of course, chosen particularly disturbing figures, and recent updates to the report have shown definite improvements, but the 2003 version [STA 95] still shows that only one-third of projects are achieved within the desired cost and time limitations. This represents an improvement of 100%, but there is stillroom for a great deal of progress to be made. 1.1.3.2. Simulation for cost reduction Virtual production is costly, admittedly, but often cheaper than real production. The construction of a prototype aircraft such as the Rafale costs hundreds of millions of euros. Even a model can be very expensive to produce. Once the prototype or model arrives, tests must be carried out, which in turn carry their own costs in terms of fuel and the mobilization of corresponding resources. Finally, if the test is destructive (e.g. as in the case of missile launches), further investment is required for each new test. For this reason, in the context of ever-shrinking budgets, the current tendency is toward fewer tests and their partial replacement by simulation, with the establishment of a “virtuous simulation-test circle” [LUZ 10]. The evolution of numbers of tests during the development of ballistic missiles by the French strategic forces, a particularly complex system, is explained in this respect: − The first missile, the M1, was developed in only 8 years, but at great cost. Modeling was used very little, and a certain level of empiricism was involved in the tests, of which there were 32 (including nine failures). − For the M4, slightly more use was made of simulation, but major progress was essential due to the implementation of a quality control process. Fourteen tests were carried out, including one failure. − During the development of the M51, which represented a major technological leap forward, simulation was brought to the fore to reduce flight and ground tests, with the aim of carrying out less than 10 tests.
8
Simulation & Modeling of Systems of Systems
Nevertheless, it should be highlighted that, contrary to all too widespread beliefs, simulation is not in principle intended to replace testing: the two techniques are complementary. Simulation allows tests to be optimized, allowing better coverage of the area in which the system will be used with, potentially, fewer real tests. Simulation is not, however, possible without tests, as it requires real-world data for parameters for models and for validation. Without this input, simulation results could rapidly lose all credibility. Figure 1.1 illustrates this complementarity: it shows the simulation and a test of armor penetration munitions, carried out with the OURANOS program at the Gramat research center of the Defense Procurement Directorate (Direction générale de l’armement, DGA). The simulation allows a large number of hypotheses to be played out with minimum cost and delay, but these results still need to be validated by tests, even if the number of tests is reduced by the use of simulation. Chapter 3 gives more detail on the principle of validation of models and simulations.
Figure 1.1. Comparison of simulation and real test (DGA/ETBS)
One area in which simulation has been used over the course of several decades is in the training of pilots. The price range of a flight simulator certified by the aviation authorities (Federal Aviation Administration (FAA), USA; Joint Aviation Authorities (JAA), Europe; and so on) is sometimes close to that of a real airplane (from 10 to 20 million euros for an evolved full flight simulator and approximately 5 million euros for a simplified training-type simulator2). The Rafale simulation center in Saint-Dizier, France, costs a trifling 180 million euros for four training simulators. Obviously, this leads to questions about whether this level of investment is justified. To judge, we need a few more figures concerning real systems. To keep to the Rafale example, the catalog price of one airplane is over 40 million euros, meaning it is not possible to have a large number of these aircraft. Moreover, a certain number must be reserved for training. The use of simulation for a large part 2 A very basic trainer for a precise task, for example, learning to use navigation instruments, can cost less (tens of thousands of euros), but for our purposes, we shall refer to complete systems (full flight and/or full mission).
Simulation: History, Concepts, and Examples
9
of the training reduces the number of aircraft unavailable for military operations. Indeed, approximately 50 aircraft would be needed to obtain the flight equivalent of the Rafale Training Center (Centre de simulation Rafale, CSR) training. Moreover, the real cost of an airplane is considerably more than its catalog price. Frequent maintenance and updates are required, alongside fuel, all of which effectively doubles or triples the total cost of the system over its lifespan. An hour’s flight in a Mirage 2000 aircraft costs approximately 10,000 euros, and an hour in a Rafale costs twice that figure3. The military must also consider munitions; a simple modern bomb costs approximately 15,000 euros, or more depending on the sophistication of the guiding system. A tactical air-to-ground missile costs approximately 250,000 euros. An hour in a simulator costs a few hundred euros, during which the user has unlimited access to munitions. We need not continue …. Moreover, the simulator presents other advantages: there is no risk to the student’s life in case of accident, environmental impact (and disturbance for the inhabitants of homes near airfields) is reduced, and the simulator offers unparalleled levels of control (e.g. the instructor can choose the weather, the time of flight, and simulate component failures). In this case, why continue the training using real aircraft? Simulation, as sophisticated as it may be (and in terms of flight simulators, this can go a long way) cannot replace the feelings of real flight. As in the case of tests, simulation can reduce the number of hours of real flight while providing roughly equivalent results in terms of training. For the Rafale, a difficult aircraft to master, between 230 and 250 h of flight per year, are needed for pilots to operate efficiently, but the budget only covers 180 h. Without simulation, the Rafale pilots could not completely master their weapons systems. We are therefore dealing with a question of filling a hole in the budget rather than an attempt to reduce flight hours. Note, though, that in civil aviation, so-called transformation courses exist, which allow pilots already experienced in the use of one aircraft to become qualified on another model of the same range through simulation alone (although the pilot must then complete a period as a co-pilot on the new aircraft). 1.1.3.3. Simulation of dangerous situations Financial constraints are not the only reason for wishing to limit or avoid the use of real systems. Numerous situations exist where tests or training involves significant human, material, or environmental risks. To return to the example of flight simulation, international norms for the qualification of pilots (such as the Joint Aviation Requirements) demand that pupils 3 The cost per hour of flight of the Rafale aircraft is a subject of controversy. From Cour des Comptes [CCO 04], we estimate the figure to be 35,000 euros per hour for the first aircraft, but the figure is certainly lower nowadays.
10
Simulation & Modeling of Systems of Systems
be prepared to react to different component failures during flight. It is hard to see how this kind of training could be carried out in real flight, as it would put the aircraft and crew in grave danger. These breakdowns are therefore reproduced in the simulator to enable the pupil to acquire the necessary reflexes so that the instructor can test the pupil’s reactions. On a different level, consider the case of inter-army and inter-ally training: given the number of platforms involved, the environmental impact would be particularly significant. Thus, the famous NATO exercise, Return of Forces to Germany (REFORGER), training which measured the capacity of NATO to send forces into Europe in the case of conflict with the Soviet Union and its allies involved, in the 1988 edition, approximately 97,000 individuals, 7,000 vehicles, and 1,080 tanks, costing 54 million dollars (at the time), of which almost half was used to compensate Germany for their environmental damage. In 1992, the use of computer simulations allowed the deployment to be limited to 16,500 individuals, 150 armored vehicles, and no tanks, costing a mere 21 million dollars (everything is relative …). The most remarkable part of this is that the environmental damage in 1992 cost no more than 250,000 dollars. 1.1.3.4. Simulation of unpredictable or non-reproducible situations Some of our readers may have seen the film Twister. Setting aside the fictional storyline and Hollywood special effects, the film concerns no more, no less than an attempt by American researchers to create a digital simulation of a tornado, a destructive natural phenomenon which is, alas, frequent in the United States. As impressive as it is, this phenomenon is unpredictable, thus difficult to study, hence the existence of “tornado chaser” scientists, as seen in the film. In this movie, the heroes try to gather the data required to build and validate a digital model. This appears to be very challenging, as tornadoes are unpredictable and highly dangerous, and their own lives are threatened more than once. However, the stakes are high; simulation allows better understanding of the mechanisms which lead to the formation of a tornado, increasing the warning time which can be given. Simulation thus allows us to study activities or phenomena that cannot be observed in nature because of their uniqueness or unpredictability. The dramatic tsunami of December 2004 in the Indian Ocean, for example, has been the subject of a number of simulation-based studies. The document [CEA 05], a summary of works published by the French Alternative Energies and Atomic Energy Commission (Commissariat à l'énergie atomique et aux énergies alternatives, CEA), shows how measurements were carried out, then the construction of digital models, which enabled events to be better understood and, whether we can avoid repetition of the terrible consequences of the tsunami, which left around 200,000 dead, is still in question.
Simulation: History, Concepts, and Examples
11
1.1.3.5. Simulation of the impossible or prohibited In some cases, a phenomenon or activity cannot be reproduced experimentally. A tsunami or a tornado can be reproduced on a small scale, but not, for example, a nuclear explosion. On September 24, 1996, 44 states, including France, signed the Comprehensive Test Ban Treaty (CTBT). By August 2008, almost 200 nations had signed. By signing and then ratifying the treaty 2 years later, France committed itself to not carrying out experiments with nuclear weapons. Nevertheless, the treaty did not rule out nuclear deterrence, nor evolutions in the French arsenal; how, though, can progress be made without tests? For this reason, in 1994, the CEA and, more specifically, the directorate of military applications launched the simulation program. With a duration of 15 years and a total cost of approximately 2 billion euros, the program consists of three main sections: − An X-ray radiography device, Airix, operational since 2000, allows us to obtain images of implosion of a small quantity of fissionable material, an event which allows a nuclear charge to reach critical mass and spark a chain reaction and detonation. Of course, as these experiments are carried out on very small quantities, there is no explosion. To measure these phenomena over a duration of 60 ns (i.e. thousandths of a second), the most powerful X-ray generator in the world (20 MeV) was developed. − The megajoule laser (LMJ) allows thermonuclear micro-reactions to be set off. Under construction near Bordeaux at the time of writing, the LMJ allows the conditions of a miniature thermonuclear explosion to be reproduced by focusing 240 laser beams with a total power of 2 million joules on a deuterium and tritium target. The project is due for completion in 2012. − The TERA supercomputer has been operational since 2001 and is regularly updated. One of the most powerful centers of calculation in Europe, TERA-10, the 2006 version, included more than 10,000 processors, giving a total power of 50,000 teraflops (billions of operations on floating-point numbers per second), and was classed at number 7 on the global list of supercomputers (according to the “Top 500 list”, www.top500.org). Moreover, to provide the necessary data for the simulation program (among other things), President Jacques Chirac took the decision in 1995 to re-launch a program of nuclear tests in the Pacific, a decision which had major consequences for the image of France with other countries.
12
Simulation & Modeling of Systems of Systems
This allows us to measure the scale of the most important French simulation program of all time, which aims to virtualize a now prohibited activity; in this case, simulation is essential, but it is not simple to put into operation, nor is it cheap. 1.1.4. Can we do without simulation? Simulation is a tool of fundamental importance in systems engineering. We should not, however, see it as a “magic solution”, a miraculous program which is the answer to all our problems – far from it. Simulation, while presenting numerous advantages, is not without its problems. What follows is a selection of the principal issues. − Simulation requires the construction of a model, which can be difficult to construct, and the definition of parameters. If the system being modeled does not already exist, the selection of parameters can be tricky and may need to be based on extrapolations of existing systems, with a certain risk of errors. − The risk of errors is considerably reduced by using an adequate process of checks and validation, but processes of this kind are costly (although they do provide considerable benefits, as with any formalized quality control process). − In cases where a system is complex, the model is generally complex too, and its construction, as well as the development of the simulation architecture, requires the intervention of highly specialized experts. Without this expertise, there is a serious risk of failure, for example, in choosing an unstable, invalid, or lowperforming model. − The construction of a model and its implementation in a simulation are expensive and time consuming, factors which may make simulation inappropriate if a rapid response is required, unless a pre-existing library of valid and ready-to-use models is available in the simulation environment. − The implementation of a complex model can require major IT infrastructures: note that the world’s most powerful supercomputers are mainly used for simulations. − The correct use of simulation results often is not an easy task. For example, when simulating a stochastic system – a fairly frequent occurrence when studying a system of systems – results vary from one implementation to another so that data from several dozen implementations of the simulation are sometimes required before reliable results can be obtained. Other methods that can help resolve problems without resorting to simulation exist. These methods are generally specific to certain types of problem but can provide satisfactory results.
Simulation: History, Concepts, and Examples
13
After all, 100 years ago, we got along without simulation. Admittedly, systems were simple (“non-complex”) and designed empirically, for the most part, with large security margins in their specifications. Reliability was considerably lower than it is today. Nevertheless, one technique used is still frequently encountered: real tests on models or prototypes, that is, the construction of a physical model of the system. For example, we could build a scale model of an airplane which could be put in a wind tunnel to study the way the air moves around the model, from which the behavior of the full-scale aircraft in the same circumstances can be deduced. This scaling down is not necessarily without its problems, as physical phenomena do not always evolve in a linear manner with the scale of the model, but the theoretical process is relatively well understood. On the other hand, the uses of this kind of model are limited: they cannot accomplish missions, nor can a wind tunnel reproduce all possible flight situations (without prohibitively expensive investment in the installation). Flying models can be built, for example, NASA’s X43A, a 3.65 m long unpiloted model airplane, equipped with a scramjet motor and able to fly at Mach 10. Of course, numerous simulations were used to reach this level of technological achievement, but the time came when it was necessary to pass to a real model: the results of the simulations needed to be validated, in a little-understood domain, that of aerobic hypersonic flight. Moreover, from a marketing point of view, a model accomplishing real feats (the model in question has a number of entries in the Guinness Book of Records) is much easier to “sell” to the public and to backers than a computer simulation, much less credible to lay people (and even to some experts). Models are therefore useful, even essential, but can also be expensive and cannot reproduce all the capacities of the system. A prototype would go further toward reproducing the abilities of the real system, but it is also too expensive: each of the four prototypes of the Rafale costs much more than the final aircraft, produced in larger numbers. Using models, we are able to study certain aspects of the physical behavior of the system in question. The situation is different if the system under study is an organization, for example, the logistics of the Airbus A380 or the projection of armed forces on a distant, hostile territory. There are still techniques and tools other than simulation which can be used to optimize organizations, for example, constraint programming or expert systems, but their use is limited to certain specific problems. For the qualification of on-board computer systems and communication protocols, mathematical formal systems (formal methods), associated with ad hoc languages (Esterel, B, Lotos, VHDL, and so on), are used to obtain “proof” of validity of system specifications (completeness, coherence, and so on). These methods are effective, but somewhat ungainly and their operation requires a certain level of expertise. These methods also cannot claim to be universal.
14
Simulation & Modeling of Systems of Systems
Finally, simulation does not remove the interest of collaborative working and making use of the competences of those involved in the system throughout its life. This is the case of the technico-operational laboratory (laboratoire technicoopérationnel, LTO) of the French Ministry of Defense, a main actor in the development of systems of systems for defense. We shall discuss this in Chapter 8. When dealing with a requirement, the work of the LTO begins with a period of reflection on the subject by a multi-disciplinary work group of experts in a workgroup laboratory (laboratoire de travail en groupe, LTG). Simulation is used as a basis for reflection and to evaluate different architectural hypotheses. The LTO is a major user of defense simulation, but it has also shown that by channeling the creativity and analytical skills of a group of experts using rigorous scientific methods, high-quality results may be obtained more quickly and at lower cost than by using simulation. Simulation, then, is an essential tool for engineering complex systems and systems of systems, but it is not a “magic wand”; it comes with costs and constraints, and if used intelligently and appropriately, it can produce high-quality results. 1.2. History of simulation Simulation is often presented as a technique which appeared at the same time as computer science. This is not the case; its history does not begin with the appearance of computers. Although the formalization of simulation as a separate discipline is recent and, indeed, unfinished, it has existed for considerably longer, and the establishment of a technical and theoretical corpus took place progressively over several centuries. The chronology presented here does not pretend to be exhaustive; we have had to be selective, but we have tried to illustrate certain important steps in technical evolution and uses of simulation. It is in the past that we find the keys of the present: in the words of William Shakespeare, “what’s past is prologue”. 1.2.1. Antiquity: strategy games The first models can be considered to date from prehistoric times. Cave paintings, for example, were effectively idealized representations of reality. Their aim, however, was seemingly not simulation, so we shall concentrate on the first simulations, developed in a context which was far from peaceful. Even before the golden age of Pericles in Greece and the conquests of Alexander the Great, a sophisticated art of war developed in the East. Primitive
Simulation: History, Concepts, and Examples
15
battle simulations were developed as tools for teaching military strategy and to stimulate the imagination of officers. Thus, in China, in the 6th Century BC, the general and war philosopher Sun Tzu, author of a work on the art of war which remains a work of reference today, supposedly created the first strategy game, Wei Hei. The game consisted of capturing territories, as in the game of go, which appeared around 1200 BC. The game Chaturanga, which appeared in India around 500 BC, evolved in Persia to become the game of chess, in which pieces represent different military categories4 (infantry, cavalry, elephants, and so on). According to the Mahâbhârata, a traditional Indian text written around 2,000 years ago, a largescale battle simulation took place during this period with the aim of evaluating different tactics. In around 30 AD, the first “sand pits” appeared for use in teaching and developing tactics. Using a model of terrain and more or less complicated rules, these tables allowed combat simulations. Intervisibility was calculated using a string. This technique remained in use for two millennia and is still used in a number of military academies and in ever-popular figurine-based war games. 1.2.2. The modern era: theoretical bases The Renaissance in Europe was a time of great scientific advances and a new understanding of the real world. Scholars became aware that the world is governed by physical laws; little by little, philosophy and theology gave way to mathematics and physics in the quest to understand the workings of the universe. Knowledge finally advanced beyond the discoveries of Greek philosophers of the 5th Century BC. Mathematics, in particular, developed rapidly over the course of three centuries. Integral calculus, matrices, algebra, and so on provided the mathematical basis for the construction of a theoretical model of the universe. Copernicus, Galileo, Descartes, Kepler, Newton, Euler, Poincaré, and others provided the essential elements for the development of equations to express physical phenomena, such as the movement of planets, the trajectory of a projectile or the flow of fluids, and chaotic phenomena. Philosophy and theology gave way to mathematics and physics in explaining the workings of the universe, which thus became predictable and could be modeled. In 1855, the French astronomer Urbain le Verrier, already famous for having discovered the existence of the planet Neptune by calculation, demonstrated that, with access to the corresponding meteorological data, the storm which caused the Franco-British naval debacle in the Black Sea on November 14, 1854 (during the Crimean War) could have been predicted. Following this, Le Verrier created the first 4 Note that until recently, the Queen was often known as the General, or by another highranking military title. The capacity to command the best troops is the reason for the power of the corresponding chessman.
16
Simulation & Modeling of Systems of Systems
network of weather stations and thus founded modern meteorology. Today, weather forecasting is one of the most publicly visible uses of simulation. In parallel, the military began to go above and beyond games of chess and foresee more concrete operational applications of simulation. From 1664, the German Christopher Weikhmann modified the game of chess to include around 30 pieces more representative of contemporary troops, thus inventing the Koenigsspiel. In 1780, another German, Helvig, invented a game in which each player would have 120 pieces (infantry, cavalry, and artillery), played on a board with 1,666 squares of various colors, each representing a different type of terrain. The game enjoyed a certain success in Germany, Austria, France, and Italy, where it was used to teach basic tactical principles to young noblemen. In 1811, Baron von Reisswitz, advisor at the Prussian Court, developed a battle simulation game known as Kriegsspiel. Prince Friedrich Wilhelm III was so impressed that he had the game adopted by the Prussian army. Kriegsspiel was played on a table covered with sand, using wooden figurines to represent different units. There were rules governing movements and the effects of terrain, and the results of engagements were calculated using resolution tables. Two centuries later, table-based war-games using figurines operate using the same principles. The computer-based war-games that dominate the field nowadays have rendered the sandpit virtual, but once again, the basic principles are the same. These games allowed different tactical hypotheses to be explored with ease and permitted greater innovation in matters of doctrine. The German Emperor Wilhelm I believed that Kriegsspiel played a significant role in the Prussian victory against France in 1870. During the same period, the game was also used in Italy, Russia, the United States, and Japan, mainly to attempt to predict the result of battles. The results, however, were not always reliable. Thus, before World War I, the Germans used a variant of Kriegsspiel to predict the outcome of a war in Europe, but they did not consider several decisive actions by the allied forces which took place in the “real” war. The game was also used by the Third Reich and the Japanese in World War II. Note, however, that the results of simulations were not always considered by the general staff. Kriegsspiel predicted that the battle of Midway would cost the Japanese two aircraft carriers. The Japanese continued and actually lost four, after which their defeat became inevitable. It is said that, during the simulation, the Japanese player refused an aircraft carrier maneuver from the player representing America, canceling their movements; the real battle was, in fact, played out around the use of aircraft carriers and on-board aircraft. This anecdote illustrates a problem that may be encountered in the development of scenarios: the human factor, where operators may refuse to accept certain hypotheses. In the 1950s and 1960s, the US Navy “played” a number of simulations of conflicts with the Soviet forces and their allies. In these simulations, it was virtually forbidden to sink an American aircraft carrier; this would show their vulnerability and provide arguments for the numerous members of Congress who considered these mastodons of the sea to be too expensive and were reluctant to finance their development as the spearhead of
Simulation: History, Concepts, and Examples
17
US maritime strategy. It is, however, clear that in case of conflict, the NATO aircraft carriers would have been a principal target for the aero-naval forces of the Warsaw Pact. In 1916, the fields of war games and mathematics met when F.W. Lanchester published a dynamic combat theory, which allowed, significantly, the calculation of losses during engagement (attrition). Almost a century later, Lanchester’s laws are still widely used in war games (Figure 1.2).
Figure 1.2. Image from Making and Collecting Military Miniatures by Bob Bard (1957), with an original illustration published in the Illustrated London News [DOR 08]
Before the movement toward computerization, combat simulation became increasingly popular. In 1952, Charles S. Roberts invented a war game based on a terrain divided using a rectangular grid. Cardboard tokens carried the symbols of various units (infantry, artillery, armor, and so on), and movements and combats were controlled by rules. This game, called Tactics, was a great success, and Roberts founded the Avalon Hill games company. From its first appearance, the commercial war board game enjoyed growing success, and large-scale expansion took place in the 1970s. Over 200,000 copies of Panzerblitz, an Avalon Hill game, were sold.
18
Simulation & Modeling of Systems of Systems
1.2.3. Contemporary era: the IT revolution 1.2.3.1. Computers Scientific advances allowed considerable progress in the modeling of natural phenomena in the 17th and 18th Centuries, but a major obstacle remained: the difficulty of calculation, particularly in producing the logarithm and trigonometry tables necessary for scientific work. At the time, the finite difference method was used (convergence to the result using expansion), involving large amounts of addition using decimal numbers. Whole services, each with dozens of workers, were affected to this task in numerous organizations. Furthermore, each calculation was usually carried out twice to detect inevitable errors. Automating calculation would therefore be of considerable scientific and economic interest. The work of Pascal and Leibniz, among others, led to the construction of calculating machines, but these required fastidious manual interventions and remained prone to error. A means of calculating a full algorithm automatically was needed. In 1833, Charles Babbage invented an “analytical engine”, which contained most of the elements present in a modern computer: a calculating unit, memory, registers, control unit, and mass memory (in the form of a perforated card reader). Although attempts at construction failed, it was later proved (a century later) that the invention would work, despite some questionable design choices. Babbage’s collaborator, Ada Lovelace, invented the first programming language. A few years later, George Boole invented the algebra, which carries his name, the basis of modern computer logic. The seeds had been sown, but the technique took around a hundred years to bear fruit. The first calculators were not built until the 1940s, with, for example, John Atanasoff’s (University of Iowa) electronic calculator. Although the calculator contained memory and logic circuits, it could not be programmed. There was also the Colossus, built in the United Kingdom used by cryptologists under the direction of Alan Turing, whose theories predated the foundations of computer programs or the Harvard Mark II by 10 years. The first true modern computer, however, was the ENIAC (Figure 1.3). The ENIAC, with 200,000 tubes, could carry out an addition in 200 µs and a multiplication in 2,800 µs (i.e. approximately 1,000 operations per second). When it was launched in 1946, it broke down every 6 h, but despite these problems, the ENIAC represented a major advance, not only in terms of calculating speed – incomparably higher than that of electromechanical machines – but also by the fact of being easily programmable. This power was initially used to calculate range tables for the US Navy, then rapidly made available for simulation, notably in the development of nuclear warheads, a very demanding activity in terms of repetitive
Simulation: History, Concepts, and Examples
19
complex calculations, based on the work of mathematicians such as Ulam, von Neumann, and Metropolis who, in the course of the Manhattan Project, formalized the statistical techniques for the study of random phenomena by simulation (MonteCarlo method). Note that the Society for Computer Simulation, an international notfor-profit association dedicated to the promotion of simulation techniques, was founded in 1952.
Figure 1.3. The ENIAC computer (US Army)
In the first few years, the future of the computer was unclear. Some had doubts about its potential for commercial success. For simulation, notably the integration of systems of differential equations, another path was opening up, that of the analog computer. Analog computers were used to model physical phenomena by analogy with electric circuits, physical magnitude being represented by voltages. Groups of components could carry out operations such as multiplication by a constant, integration in relation to time, or multiplication or division of one variable by another. Analog computers reached the height of their success in the 1950s and 1960s, but became totally obsolete in the 1970s, pushed out by digital. In France, in the 1950s, the Ordnance Aerodynamics and Ballistics Simulator (Simulateur Aérodynamique et Balistique de l’Armement, SABA) analog computer was used at
20
Simulation & Modeling of Systems of Systems
the Ballistics and Aerodynamics Research Laboratory (Laboratoire de Recherches Balistiques et Aérodynamiques, LRBA), in Vernon, to develop the first French landto-air missile, the self-propelled radio-guided projectile against aircrafts (projectile autopropulsé radioguidé contre avions, PARCA). Digital computers, as we know, emerged victorious, but their beginnings were difficult. On top of their cost and large size, programming them was a delicate matter; at best, a (not particularly user-friendly) teleprinter could be used, and at worst, perforated cards or even, for the oldest computers, switches or selector switches could be used to enter information coded in binary or decimal notation. In the mid-1950s, major progress was made with the appearance of the first high-level languages, including FORTRAN in 1957. Oriented toward scientific calculations, FORTRAN was extremely popular in simulation and remains, 50 years later, widely used for scientific simulations. One year later, in 1958, the first algorithmic language, ALGOL, appeared. This language, and its other versions (Algol60, Algol68), is important as, even if it was never as popular as more basic languages such as COBOL or FORTRAN, it inspired very popular languages such as Pascal and Ada. The latter was widely used by the military and by the aerospace industry from the mid-1980s. In the 1990s, object-oriented languages (Java, C#, and so on) took over from the descendants of Algol, while retaining a number of their characteristics. Smalltalk is often cited as the first object-oriented language, but this is not the case: the very first was Simula, in 1967, an extension of Algol60 designed for discrete event simulation. The 1960s saw a veritable explosion in digital simulation, which continues to this day. 1965 was an important year, with the commercialization of the PDP-8 mini-computer by the Digital Equipment Company. Because of its “reasonable” size and cost, thousands of machines were sold, marking a step toward the democratization of computing and of means of scientific calculation, expanding into universities and businesses. This tendency was reinforced by the unrolling of workstations across research departments in the 1980s, followed by personal computers (PCs). This democratization not only meant an increase in access to means of calculation but also allowed greater numbers of researchers and engineers to work on and advance simulation techniques. These evolutions also took place at application level. In 1979, the first spreadsheet program, Visicalc, was largely responsible for the success of the first micro-computers in businesses. This electronic calculation sheet carried out simple simulations and data analysis (by iteration), notably in the world of finance. Visicalc has long disappeared from the market, but its successors, such as Microsoft Excel, have become basic office tools and, seemingly, the most widespread simulation tool of our time.
Simulation: History, Concepts, and Examples
21
1.2.3.2. Flight simulators and image generation During the 1960s–1970s, although computers became more widespread and more accessible, simulations retained the form of strings of data churned out by programs, lacking concrete physical and visual expression. Two domains were the main drivers behind the visual revolution in computing: piloted simulation, initially, followed by video games (which, incidentally, are essentially simulations in many cases). Learning to fly an airplane has always been difficult and dangerous, especially in the case of single-seat aircraft. How can the pupil have sufficient mastery of piloting before their first flight without risk of crashing? The first flight “simulators” were invented in response to this problem. The “Tonneau” was invented by the Antoinette company in 1910 (Figure 1.4). A cradle on a platform with a few cables allowed students to learn basic principles. In the United States, the Sanders Teacher used an airplane on the ground. In 1928, in the course of his work on pilotage of airplanes without visibility, Lucien Rougerie developed a “ground training bench” for learning to fly using instruments. This system is one of two candidates for the title of first flight simulator. The other invention was developed by the American Edward Link in the late 1920s. Link, an organ manufacturer with a passion for aviation, patented and then commercialized the Link Trainer, a simulator based on electro-pneumatic technology. The main innovation was that the cabin was made to move using four bellows for the roll and an electric motor for the yaw, of which the logic followed the reactions of the aircraft. The instructor could follow the student’s progress using a replica of the control panel and a plotter. The instructor could also set parameters for wind influence. The first functional copies were delivered to the American air force in 1934. The Link Trainer is considered to be the first true piloted simulator. The company still exists and continues to produce training systems. These training techniques using simulation became more important during WWII, a period with constant pressure to train large numbers of pilots. From 1940 to 1945, US Navy pilots trained on Link ANT-18 Blue Box simulators, of which no less than 10,000 copies were produced. Afterwards, these simulators became more realistic due to electronics. In the 1940s, analog computers were used to resolve equations concerning the flight of the airplane. In 1948, a B377 Stratocruiser simulator, built by the company CurtissWright, was delivered to the PanAm company. The first of its kind to be used by a civil aviation company, it allowed a whole team to be trained in a cockpit with fully functional instruments, but two fundamental elements were absent: the reproduction of the movement of the airplane and exterior vision, cockpits of the time still being blind. In the 1950s, mechanical elements were added to certain simulators to make
22
Simulation & Modeling of Systems of Systems
the cabin mobile and thus reproduce, at least to some degree, the sensations of flight. The visual aspect was first resolved by using a video camera flying over a model of the terrain and transmitting the image to a video screen in the cockpit. In France, the company Le Matériel Téléphonique (LMT) commercialized systems derived from the Link Trainer from the 1940s. The first French-designed electronic simulator was produced by the company in 1961 (Mirage III simulator). LMT has continued its activities in the field, currently operating under the name Thales Training & Simulation.
Figure 1.4. The Antoinette “Tonneau”
1.2.3.3. From simulation to synthetic environments The bases of simulation were defined during the period 1940 – 1960. These fundamental principles remain the same today. Nevertheless, the technical resources used have evolved, and not just in terms of brute processing power: new technologies have emerged, creating new possibilities in the field of simulation. Particular examples of this include networks and image generators. The computers in 1950s had nothing like the multi-media capacities of modern computers. At the time, operators communicated with the machine using teleprinters. Video screens and keyboards eventually replaced teleprinters, but until the end of the 1970s, the bulk of man–machine interfaces were purely textual, and data were recreated in the form of lists of figures. Some, however, perceived the visual potential of computers very early on. In 1963, William Fetter, a Boeing employee, created a three-dimensional (3D) wire sculpture of an airplane to study take-off and landings. He also created First Man, a virtual pilot for use in ergonomic studies. Fetter invented the term “computer
Simulation: History, Concepts, and Examples
23
graphics” to denote his work. In 1967, General Electric produced an airplane simulator in color. The graphics and animation were relatively crude, and it required considerable resources to function. Full industrial use of computer-generated images did not really begin until the early 1970s, with David Evans and Ivan Sutherland’s Novaview simulator, which made use of one of the first image generators. The graphics were, admittedly, basic, in monochrome wire, but a visual aspect had finally been added to a simulator. The Novaview company was to become a global leader in the field of image generation and visual simulation. In 1974, one of their collaborators, Ed Catmull, invented Z-buffering, a technique which improves the management of 3D visualization, notably in the calculation of hidden parts. Two years later, this technique was combined with another new technique, texture mapping, invented by Jim Blinn. This enabled considerable progress to be made in terms of realism of computer-generated images by pinning a bitmap image (i.e. a photo) onto the surface of a 3D object, giving the impression that the object is much more detailed than it really is. For example, by pinning a photo of a real building onto a simple parallelepiped, all the windows and architectural features of the building can be seen, but all that needs to be managed is a simple 3D volume. Although processing power remained insufficient and software were very limited, the bases of computer-generated images were already present. At the beginning of the 1980s, NASA developed the first experimental virtual reality systems; interactive visual simulation began to become more widespread. Virtual reality, although not the commercial success anticipated, had a significant impact on research on the man–machine interface. During the 1980s and 1990s, another technology emerged, so revolutionary that some spoke of a “new industrial revolution”: networks. Arpanet, the ancestor of the Internet, had been launched at the end of the 1960s, but it was confined to a limited number of sites. Local Access Networks and the Internet made networks more popular, linking humans and simulation systems. In 1987, a distributed simulation network (SIMNET) was created to respond to a need for collective training (tanks and helicopters) by the US Army. This experimental system rapidly aroused interest, and led, in 1989, to the first drafts of the DIS (Distributed Interactive Simulation) standard protocol for simulation interoperability, which became the IEEE 1278 standard in 1995 (see Chapter 7). Oriented in real time and at low level, DIS was far from universal, but it was extremely successful and is still used today. DIS allowed cooperation between simulations. The community of aggregated constructive simulations, such as, war games, created its own non-real-time standard, ALSP (Aggregate Level Simulation Protocol), developed by the Mitre Corporation. Later, the American Department of Defense revised its strategy and imposed a new standard, HLA (High Level Architecture), in 1995, which became the IEEE 1516 standard in 2000. Distributed simulation found applications rapidly. In 1991, simulation (including distributed
24
Simulation & Modeling of Systems of Systems
simulation through SIMNET) was used for the first time on a massive scale in preparing a military operation: the first Gulf War. France was not far behind: in 1994, two Mirage 2000 flight simulators, one in Orange and the other in Cambrai, were able to work together via DIS and a simple Numeris link (digital link with integrated service (réseau numérique à intégration de service, RNIS). The following year, several constructive Janus simulations were connected via the Internet between France and the United States. 1996 saw the first large multi-national simulation federation, with the Pathfinder experiment, repeated several times since. This capacity to make means of simulation, real materials, and information systems interoperate via networks opens the way for new capacities, used, for example, in battle-labs such as the technico-operational laboratory of the French Ministry of Defense. An entire book could be written on the history of simulation, a fascinating meeting of multiple technologies and innovations, but such is not our plan for this work. The essential points to remember are that simulation is the result of this meeting, that the subject is still in its youth, and that it will certainly evolve a great deal over the coming years and decades. Effectively, we are only just beginning to pass from the era of “artisan” simulations to that of industrial simulation. Simulation, as described in this book, is, therefore, far from having achieved its full potential. 1.3. Real-world examples of simulation 1.3.1. Airbus Airbus was created in 1970 as a consortium. Something of a wild bet in the beginning, on the back of the success (technical, at least) of Concorde, it became a commercial success story, even – something no-one would have dared imagine at the time – overtaking Boeing in 2003. This success is linked to strong political willpower combined with solid technical competence, but these elements do not explain everything. Despite the inherent handicaps of a complex multi-national structure, the consortium’s first airplane, the Airbus A300, entered into service from 1974; its performance and low operating costs attracted large numbers of clients. The A300 incorporated numerous technological innovations, including the following: − just-in-time manufacturing; − revolutionary wing design, with a supercritical profile and particularly accurate flight controls from an aerodynamic point of view, one of the factors responsible for significant reductions in fuel consumption (the A300 consumed 30% less than a Lockheed Tristar, produced during the same period); − auto-pilot possible from the first ascent to the final descent;
Simulation: History, Concepts, and Examples
25
− wind shear protection; − electronically controlled braking. These innovations, to which others were added in later models (including the highly mediatized two-person piloting system, where the tasks of the flight engineer were made automatic) led to the commercial success of the aircraft, but also made the system more complex.
Figure 1.5. Continuous atmospheric wind tunnel from Mach 0.05 to Mach 1, the largest wind tunnel of its type in the world – A380 tests – (ONERA)
The engineers who created the Airbus consortium at that time thus made use of significant simulation, both digital, which was becoming more widespread in spite of the cost of computers, still very high, and by using models in wind tunnels (see Figure 1.5). This use of simulation, to a degree never before seen in commercial aviation, enabled Airbus to develop and build an innovative, competitive, and reliable product, and thus to achieve a decisive advantage over the competition. Without simulation, Airbus would probably not have experienced this level of commercial success. It would be difficult to provide an exhaustive list of all uses of simulation by Airbus and its sub-contractors. Nevertheless, the following list provides some examples:
26
Simulation & Modeling of Systems of Systems
− global performance simulation: aerodynamics, in-flight behavior of a design; − technical simulation of functions: hydraulics, electricity, integrated modular avionics, flight controls; − simulation of mechanical behavior of components: landing gear, fuselage, wings, pylon rigging structure; − simulation of global possession costs; − production chain simulation; − interior layout simulation (for engineering, but also for marketing purposes); − maintenance training simulator; − training simulator for pilots. The use of simulation by Airbus is unlikely to be reduced in the near future, especially in the current context of ever-increasing design constraints. The Advisory Council for Aeronautics Research in Europe (ACARE) defines major strategic objectives for 2020 in [EUR 01], with the aim of providing “cheaper, cleaner, and safer air transport”: − more pleasant and better value journeys for passengers; − reduction in atmospheric emissions; − noise reduction; − improvements in flight safety and security; − increased capacity and efficiency of the air transport system. We clearly see that the aims of these objectives go beyond the engineering of a “simple” aircraft, but necessitate the combined and integrated engineering of a system of systems (that for air transport, [LUZ 10]). To achieve this, large-scale investment will be needed, over 100 billion euros over 20 years (according to [EUR 01]), and modeling and simulation, as well as systems engineering as a whole, play an important role. 1.3.2. French defense procurement directorate The DGA provides an interesting example of simulation use. This organism, part of the French Ministry of Defense, exists principally for the acquisition of systems destined for the armed forces. The DGA’s engineers therefore deal with technical or technico-operational issues throughout the lifecycle of a system. We find examples of simulation use at every step of system acquisition
Simulation: History, Concepts, and Examples
27
(which corresponds to the lifecycle of the system from the point of view of the contracting authority). Furthermore, the DGA uses a very wide variety of simulations (constructive, virtual, hybrid, and so on). Granularity
System of systems
3 1
System
5 11
7 Function
2
9
6
11
8 Physical
4
1
Design
Construction
Program cycle
Preparation
Use
Figure 1.6. Use of simulation in armament programs
Figure 1.6 illustrates the different applications of simulation in the system acquisition cycle: – Determine operational concepts: it allows “customers” and final users (the armed forces) to better express their high-level needs, for example, in terms of capacity. Simulation allows concepts to be illustrated and tested virtually (see Chapter 8). – Provide levels of performance: it would be difficult to issue a contract for a system from a concept; a first level of dimensioning is required to transform an operational need, for example, “resist the impact of a modern military charge”, into “resist an explosion on the surface of the system with a charge of 50 kg of explosives of type SEMTEX”. Simulation can give an idea of the levels of performance necessary to achieve the desired effect. – Measure operational efficiency: the next step is to re-insert these performance levels into the framework of an operational mission by “playing out” operational scenarios in which the new system can be evaluated virtually. – Mitigate risk and specify: the system specifications must be refined, for example, by deciding what material to use for armoring the system to obtain the required resistance. The potential risks involved with these choices can then be evaluated: What will the mass of the system be? Will it be possible to motorize it? Will the system remain within the limits of maneuverability? Note that this activity usually falls under the authority of the system project manager.
28
Simulation & Modeling of Systems of Systems
– Mitigate risk regarding human factors: evaluate the impact of the operators on the performance of the system: Is there a risk of overtaxing the operator in terms of tasks or information? Might the operator reduce the performance of the system? For example, it would be useless to develop an airplane capable of accelerating at 20 g if the pilot can only withstand half of that. – Evaluate the feasibility of technological solutions: this is an essential step in risk mitigation. It ensures that a technology can be integrated and performs well in the system framework. At this stage, we might notice, for example, that a network architecture based on a civil technology, 3G mobile telephony, poses problems due to the existence of zones of non-coverage, gives very variable bandwidth with the possibility of disconnections, and so on, or, on the contrary, check that the technology in question responds well to the expressed need in the envisaged framework of use. – Optimize an architecture or function: once the general principle of an architecture has been validated, we can study means of optimizing it. In the case of “intelligent” munitions, that is, munitions capable of autonomous or assisted direction toward a target, we might ask at what stage in flight target detection should occur. If the sensor is used too early, there may be problems identifying or locking onto the correct target; if it is too late, the necessary margin for correcting the trajectory to hit the target will be absent. – Facilitate the sharing and understanding of results of calculations: the results of studies and simulations can be very difficult to interpret, for example, lists of several million numerical values recording dozens of different variables. Illustrating these results using simulations, for example, a virtual prototype of the system, more visually representative, allows the analyst to better understand the dynamic evolution of the system. – Study system production and maintenance conditions: implemented by the project manager, manufacturing process simulations are used to specify, optimize, and validate manufacturing conditions (organization of the production line, work station design, incident management, and so on). Simulation can be applied to the whole of the production line, the entire logistics system (including supply fluxes for components and materials and delivery of the finished product), or a work station (ergonomic and safety improvements). This activity also includes studies of maintenance operations. In this case, virtual reality is used to check on the accessibility of certain components during maintenance operations. – Prepare tests: tests of complex systems are themselves complicated or even complex to design and costly to implement. Simulation provides precious help in optimizing tests, improving, for example, the determination of the domain of qualification to cover, the finalization and validation of corresponding test scenarios, the test architecture, and the evaluation of potential risks. Simulation can also be
Simulation: History, Concepts, and Examples
29
used during the interpretation of test results, providing a dynamic 3D representation of the behavior of the system, based on the data captured during the test, which is substantial and difficult to analyze. – Supplement tests: the execution of tests allows us to acquire knowledge of a system for a given scenario, for example, a flight envelope. Simulation allows the system to be tested (virtually) over and beyond what can be tested in reality, whether this may be due to budget or environmental constraints (e.g. pollution from a nuclear explosion) or safety (risk to human life) or even confidentiality (emissions from an electronic war system or from a radar can be captured from a distance and analyzed; moreover, they constitute a source of electromagnetic pollution). By carrying out multiple tests in simulation, the domain in which the system has been tested can be extended, or the same scenario can be repeated several times to evaluate reproducibility and make test results more statistically representative, thus increasing confidence in the results of qualification. – Specify user–system interface: the use of piloted study simulators representing the system allows the ergonomics of the user–system interface for the initial system to be analyzed and improvements to be suggested to increase efficiency or adapt it to a system evolution (addition of functionalities). Beyond its role as direct support during the various phases of the program acquisition cycle, simulation is also used by the DGA for the development of technical expertise, particularly in the context of project management, where technical mastery of the operation of a system is indispensable to work on the specifications and qualification of a system. This mastery, however, is difficult to obtain without being the project manager or the final user. Simulation is also a tool used for dialog between experts in the context of collaborative engineering processes using a “board” of multi-disciplinary teams, the idea being, eventually, to maintain and share virtual system prototypes between the various participants in the acquisition process (the SBA principle). Finally, modeling and simulation are seen as the main tools for mastering system complexity; at the DGA, simulation is attached to the “systems of systems” technical department. 1.4. Basic principles In any scientific discipline, a precise taxonomy of the field of work is essential, and precision is critical in the choice of vocabulary. Unfortunately, simulation is spread across several scientific “cultures”, generally considered a tool and not as a separate domain such that the same term will not always mean the same thing depending on the person using it. Furthermore, a large number of typologies of simulation exist. Although simulation is becoming increasingly organized, a certain amount of effort is still required to guarantee coherence between different areas and
30
Simulation & Modeling of Systems of Systems
between different nationalities. In France, the absence of a national norm concerning simulation does not help, as the translation of English terms is not always the same from one person to another. There are, however, a number of documents that can be considered to be references in the field, and upon which the terminology used in this work will be based. Most are in English, as the United States is both the largest market for simulation and the principle source of innovations in the field. This driving role gives the United States a certain predominance in the orientation of the domain. Before going any further, we shall explain what exactly is meant by system simulation. 1.4.1. Definitions 1.4.1.1. System A system is a group of resources and elements, whether hardware or software, natural or artificial, arranged so as to achieve a given objective. Examples of systems include communication networks, cars, weapons, production lines, databases, and so on. A system is characterized by – its component parts; – the relationships and interactions between these components; – its environment; – the constraints to which it is subjected; – its evolution over time. As an example, we shall consider the system of a billiard ball falling in the Earth’s gravitational field. This system is composed of a ball, and its environment is Earth’s atmosphere and gravity. The ball is subjected to a force that pulls it downwards and an opposing force in the form of air resistance. 1.4.1.2. Model First and foremost, simulating, as we shall repeat throughout this work, consists of building a model, that is, an abstract construction that represents reality. This process is known as modeling. Note that, in English, the phrase “modeling and simulation” is encountered more often than the word “simulation” on its own.
Simulation: History, Concepts, and Examples
31
A model can be defined as “any physical, mathematical, or other logical representation (abstraction) of a system, an entity, a process or a physical phenomenon [constructed for a given objective]” [DOD 95]. Note that this definition includes numerous types of models, not only computer ones. A model can take many forms, for example, a system of equations describing the trajectory of a planet, a group of rules governing the treatment of an information flux, or a plastic model of an airplane. An interesting element of the definition is the fact that a model is “constructed for a given objective”: there is no single unique model for a system. A system model that is entirely appropriate for one purpose may be completely useless for another purpose. To illustrate this notion of relevance of the model, to which we will return later, we shall use the example of the billiard ball falling in the Earth’s gravitational field. Suppose that we want to know how long the fall will last, with moderate precision, to the nearest second, that the altitude of release z0 is low (not more than a few hundred meters), and that the ball is subject to no other influence than gravity, g, which we can treat as a constant because of the low variation in altitude and the low level of precision required. The kinematic equations of the center of gravity of the ball give a(t ) = g for the t
acceleration and v(t ) = ∫ a(t )dt = gt + v0 for the velocity, from which we deduce that 0
t
z (t ) = ∫ v(t )dt , so z (t ) = 0
1 2 gt + v0 t + z0 . 2
The equations giving a(t), v(t), and z(t) constitute a mathematical abstraction of the “billiard ball (falling in the Earth’s gravitational field)” system; this is therefore a model of the system. But are there other possible models? Without going too far, let us consider our aims and our hypotheses. Imagine now that we want to obtain the fall time correct to within 50 ms. To arrive at this level of precision, we must consider other factors, notably air resistance, in the form of a term representing fluid friction as a force opposing the fall f(v) = −αv. We thus have a new expression for altitude depending on the time z(t), as follows: z (t ) = A(1 − e−t / τ ) − Bt + z . 0
This expression is clearly more complicated and demanding in terms of calculation power than the former. Moreover, our hypothesis was relatively simple, the forces of friction being closer to f(v) = −αv2, especially at high velocities. Nevertheless, our second expression reproduces the movement of the ball more realistically over long periods of falling, as the velocity is no longer linear, but tends to a limit.
32
Simulation & Modeling of Systems of Systems
If we represent the velocity of the ball as a function of time, following both hypotheses (simple and with fluid friction), we notice that both curves are close at the beginning and then diverge. For short durations starting when the ball is released and assuming that the model with friction is close enough to reality for our purposes, we notice that the simple model is also valid for our needs in this zone, known as the domain of validity of the model. If our needs are such that even the model with friction is not representative enough of reality, other influences can be considered, such as wind. If our ball is released from a high altitude, we must consider the variation in the attraction of weight depending on the altitude, introducing a new term g(z) into the model equations, making matters considerably more complex. We could go even further, considering the Coriolis effect, the non-homogeneity of the ball, and so on. We therefore see that models of the same system can vary greatly depending on our aims. Of the two models we have constructed, we cannot choose the best model to use without knowing what it is to be used for. Some might say that, in case of doubt, the most precise model should be used. In our simple example, this is certainly feasible, but the case would be different in a more complicated system, governed by hundreds of equations, as is often the case with aerodynamic modeling for aeronautics: solving these equations is costly in terms of time and technical resources, and the use of a model too “heavy” for the objective required involves the use of resources over and beyond what would be strictly necessary, leading to cost overrun, a sign of poor quality. 1.4.1.3. Simulation The definition provided by the US Department of Defense, seen above [DOD 95], is interesting as it introduces the notion of subjectivity of a model. However, it concerns models in general; in simulation, we use more specific models. The IEEE, the civil standardization association, gives two other definitions in the document IEEE 610.3-1989: – an approximation, representation, or idealization of certain aspects of the structure, behavior, operation, or other characteristics of a real-world process, concept, or system; – a mathematical or logical representation of a system or the behavior of a system over time. If we add “for a given objective” to the first definition, it corresponds closely to that given by the US DoD. The second definition is interesting, as it introduces an essential element: time. The model under consideration is not fixed: it evolves over time, as in the case of our falling billiard ball model. It is this temporal aspect that
Simulation: History, Concepts, and Examples
33
gives life to a model in a simulation, as simulation is the implementation of a model in time. To simulate, therefore, is to make a model “live” in time. The action of simulation can also refer to the complete process which, from a need for modeling, allows the desired results to be obtained. In general, this process can be split into three major steps, which we will cover in more detail later: – design and development of the model; – execution of the model; – analysis of the execution. Strictly speaking, the definition given above refers to the second step, which is misleading, as the most important part is the design of the model, on which the results and the interpretation of results depend. The last step is not always easy: when the results consist of several thousands of pages of figures which the computer “spits out”, it can take weeks of work to reach the core substance. Use of the full term “modeling and simulation” is therefore helpful, as it emphasizes the importance of the preliminary modeling work. Note that simulation also designates the computer software used to build and execute a model.
SYSTÈME Real System RÉEL
Simulator Simulateur ou or Simulation
Validation
+ Environment Environnement + Scenario Scénario
M od
in
t
en
ed
el by
Validation
ed
em
pl
Im
Modèle(s) Model(s) Figure 1.7. General principle of simulation
Verification V rification
34
Simulation & Modeling of Systems of Systems
Figure 1.7, based on [ZEI 76], resumes the general principle of simulation: we take as point of departure a real system, with its environment and a usage scenario, and we construct an abstraction, the model, which is then made concrete by a simulation application. The quality of the model and simulation in relation to the aims is assured by a process of validation and verification which will be discussed in detail later. To return to the example of the falling billiard ball, the simulation program using the simple model could be as follows, in the language C: CODE : /* falling_ball.c */ #include /* Physical Constants */
const
float g = -9.81; /* acceleration due to gravity (m/s²) */
/* Initiales conditions */
float Vo = 0.0;
/* initial velocity (m/s) */
float Zo = 0.0;
/* initiale altitude (m) */
/* Kinematic Model (simple version) */
float calculate_velocity(float t) {
return g*t + Vo; }
float calculate_altitude(float t) {
return g*t*t/2 + Vo*t + Zo;
Simulation: History, Concepts, and Examples
} /* "simulation engine" et "user interface" */
int main() { /* Parameters for simulation engine execution */
float Step= 1.0;/* time step (s) */ /* External variables */
float t;
/* simulation time */
/* State variables */
float v;
/* velocity */
float z;
/* altitude */
puts("Simulation of falling billiard ball:\n"); printf("Initial altitude and velocity? "); scanf("%f %f", &Zo, &Vo); printf("Integration step? "); scanf("%f", &Step); printf("Initial altitude = %f m/s, Initial velocity = %f m\n", Zo, Vo); t = 0.0;
do { v = calculate_velocity(t); z = calculate_altitude(t);
35
36
Simulation & Modeling of Systems of Systems
printf("t = %7.3f s --> z = %7.2f m, v = %7.2f m/s\n", t, z, v); t = t + Step; }
while (z > 0.0); puts("Collision with the ground!");
return 0; } The execution of the simulation gives the following results for a billiard ball dropped from the top of the Eiffel Tower: EXECUTION: Simulation of falling billiard ball: Initial altitude and velocity? 320.0 0.0 Integration step? 1.0 Initial altitude = 320.000000 m, Initial velocity = 0.000000 m/s t = 0.000 s --> z = 320.00 m, v =
0.00 m/s
t = 1.000 s --> z = 315.10 m, v = -9.81 m/s t = 2.000 s --> z = 300.38 m, v = -19.62 m/s t = 3.000 s --> z = 275.86 m, v = -29.43 m/s t = 4.000 s --> z = 241.52 m, v = -39.24 m/s t = 5.000 s --> z = 197.38 m, v = -49.05 m/s t = 6.000 s --> z = 143.42 m, v = -58.86 m/s t = 7.000 s --> z = 79.65 m, v = -68.67 m/s t = 8.000 s --> z = 6.08 m, v = -78.48 m/s t = 9.000 s --> z = -77.31 m, v = -88.29 m/s Collision with the ground!
Simulation: History, Concepts, and Examples
37
This represents the raw results of the execution of the simulation. Here, we only have a small quantity of data, but imagine a simulation of the flight of a rocket with hundreds of variables and millions of values …; the interpretation and analysis phase should not be underestimated, as it is sometimes the longest of the three phases. Using our example, we can study the “profile” of the fall of the ball and evaluate the exact moment of impact and the speed at that moment. The analytical calculations are simple, but once again, our example is a trivial textbook scenario; the case would be different for a more complex system, including systems of equations with tens, even hundreds, of variables. In summary, we have seen, using the example of the falling ball, a system model, a simulation application, and the execution of a simulation.
1.4.2. Typology Simulations can be classified in various ways. We shall discuss three possibilities, which represent the main divisions in the field of simulation: − levels of granularity; − architecture; − uses. 1.4.2.1. Levels of granularity The level of granularity of a system refers to the physical size of the entities it manipulates; it is an indicator of the scale of the system that can be simulated with the simulation in question. This notion should not be confused with the refinement of a model, that is, its temporal (calculation step) and spatial precision (e.g. resolution of the terrain), as even if granularity and refinement are, clearly, often linked, this is not always the case: we could design a simulation of a system of systems, for example, in which we would model very low-level physical phenomena. Of course, considerable work would be required to construct and validate the simulation, with considerable processing power, but it is possible and, without going as far as our extreme example, can be relevant, particularly when reusing a model which is too refined, but valid and available. In order of increasing granularity, we typically distinguish between the following levels: – Physical phenomena: example, propagation of a radar wave, airflow around a turbine blade, and the constraints on a ship’s hull exposed to swell. Most often, simulations of physical phenomena use modeling based on differential equations. This was the case for our example of the falling billiard ball.
38
Simulation & Modeling of Systems of Systems
– Components: a component carries out a function within the system, but without real autonomy, for example, the homing head on a missile, a gearbox, or a fire detector. – Sub-systems: these are groups of several components, which carry out a function within a system with a certain degree of autonomy, for example, guiding a missile or fire protection on a ship. – System: as defined above. A simulation at system level may contain several systems, examples: aircraft, missile, combat between aircraft and missile. – Multi-systems: groups of significant numbers of systems carrying out a common mission, examples: a factory, an airport, and a car racetrack. – System of systems: see definitions given earlier. Examples: logistics system of the A380 (production, delivery, implementation, and so on), telephone network, international banking system. In defense simulation, the following levels are considered, which correspond to the last two categories above: − Tactical corresponds to combat at multi-system level, for example, an air squadron or a company of infantry. − Operational corresponds to a battle, at regiment level, joint tactical force (groupement tactique interarmes, GTIA) or above, for example, a reconstruction of the Battle of Austerlitz (1805). In the civilian domain, the logistics system of the A380 (production, delivery, implementation, and so on) would be an example at this level. − Strategic (military) refers to a theater of war or a campaign, for example, a reconstruction of Napoleon’s Russian campaign of 1812. The American synthetic theater of war (STOW) deployed 8,000 entities during an exercise in October 1997. An example at this level in the civilian sector would be an economic simulation at planetary level. It is generally fairly easy to define the level of granularity of a simulation very early in the development process as the granularity is linked to what is being simulated. This allows a first approach to be made concerning the level of refinement, reusable models, and even the architecture. Typically, simulations of physical phenomena are based on non-interactive calculation languages (FORTRAN, Matlab, Scilab, and so on), while at tactical and operational level, visual and interactive simulation products are used instead (Janus, JTLS, OneSAF, and so on).
Simulation: History, Concepts, and Examples
39
1.4.2.2. Architecture Taking a step back, we observe that simulation systems usually have the same general architecture: – models of systems, to which environment models can be added, or models of human behavior if the system operators need to be simulated; − parameters giving the dimensions of models (e.g. mass, maximum flight altitude of an airplane, capacity of a fuel tank, maximum acceleration, maximum load supported by a road bridge); − a scenario describing how the simulation progresses (e.g. for a road traffic simulation, the scenario indicates, among other things, the initial state of traffic, the time at which a junction will be closed for road works, and bad weather at the end of the day); – if the simulation is interactive, a user–machine interface allows control input to influence the course of the simulation (e.g. the control stick of a flight simulator); – a simulation engine that allows these components to be animated over time (in the following chapters, we shall discuss this key component of simulations in greater depth); − simulation output: a reproduction of the dynamic behavior of the system (saved state variables and/or statistics for a technical simulation, visualization through an image generator of the state of the system for a flight simulator). Figure 1.8 illustrates the simulation architecture, which we shall discuss on several occasions later in this book.
SIMULATION Model operator parameters
Human behavior model
System parameters
System model
Environmental data
Environment model
Operator controls (interface)
Simulation Motor
Output of dynamic behavior Analysis and interpretation of results Scenario
Figure 1.8. Typical simulation architecture
40
Simulation & Modeling of Systems of Systems
Although, seen from afar, all simulations more or less conform to this model, a closer look shows there is considerable variation in the way it is set out. This differentiation is such that it provokes divisions in the simulation community; it is rare to find a simulation engineer with expertise in working with more than one of these architectures. We can distinguish the following distinct architectures, or forms, of simulation: closed-loop digital simulation, hardware-in-the-loop (hybrid) simulation, constructive simulation, and simulation with instruments. Closed-loop digital (or “scientific”) simulation is a purely computerized form of simulation that operates without human intervention, so it is non-interactive. It provides modeling, often very refined, of a physical phenomenon or a system component, rarely more. It does not operate in real time and sometimes requires considerable processing power. We should highlight, at this point, the fact that the main application of the world’s most powerful supercomputers is simulation, in particular in developing nuclear weapons, meteorological calculations, aerodynamic calculations for aircraft, and simulations of vehicle collisions. The CEA’s TERA-10 computer, built for the “Simulation” program, was the most powerful in Europe when it was launched in 2006, with a calculating power of 50,000 billion operations per second and a storage capacity of 1 million billion bytes. In this type of simulation, the phenomenon under the study is typically modeled using differential equations applied to elementary portions of the system (to make it discrete, i.e. the finite element method). This resolution is obtained using calculations on matrices of tens, even hundreds, of lines/columns (as many as there are variables), hence the need for considerable processing power, especially as calculations are iterated every step in time. Using the TERA-10, the simulation of a nuclear fusion reaction lasting less than one thousandth of a second can require hours of calculations. On a simpler level, our simulation of the billiard ball falling is a closed-loop digital simulation. Figure 1.9 shows an intermediate example of this kind of simulation. Hardware-in-the-loop (hybrid) simulation adds real material into the loop, combing a model (i.e. virtual version) of part of the system with a real part of the system, which is therefore only partially simulated. This type of simulation is particularly important when testing and determining the properties of components. The simulation aims to stimulate the real component or components being tested by generating the environment of the component (entries, reactions to actions, and so on).
Simulation: History, Concepts, and Examples
41
Figure 1.9. Digital simulation: evaluation of damage to a concrete slab caused by a detonation on contact (OURANOS simulation code, by M. Cauret, DGA/CEG)
It means that the system generally needs to work in strict real time. Figure 1.10 gives an example of a hardware-in-the-loop simulation system, the real-time simulator for pulsed Doppler homing heads (simulateur temps réel pour autodirecteurs doppler à impulsions, STRADI) used by the Ballistics and Aerodynamics Research Laboratory of the DGA. To test an air-to-air missile homing head of type Mica IR, the component is mounted on a three-axis table, and then infrared scenes are generated containing targets so that the homing head “believes” it is in use as part of a missile in flight. By stimulating the homing head using simulation, its reactions to different scenarios can be evaluated, determining its performance and its conformity to the specifications. Constructive simulations are purely computer based. They include high-level models, that is, systems and systems of systems, with their operators. The human factor is therefore involved, but in simulated form. These simulations can be interactive, but the operator in the loop only takes general decisions (missions),
42
Simulation & Modeling of Systems of Systems
and the model remains the main source of results. To summarize, a constructive simulation uses simulated human beings, operation simulated systems in a simulated environment. War games and technico-operational simulations fall into this category.
Figure 1.10. STRADI hybrid simulation for missile testing (DGA/LRBA)
Figure 1.11 is a screen capture of the Ductor naval aviation simulator, used by the National Navy to “play” naval operations. Virtual simulations are interactive, real-time simulations at system (platform) level with human operators, using user–machine interfaces which are more or less faithful representations of the real equipment. The user is not only in the loop but constitutes the main source of events and stimuli for the simulation. The bulk of this category is made up of piloted simulations (flight simulators and trainers), but training simulations for information system operators and military commanders also fall into this group. A virtual simulation may use a simple PC to reproduce the interface of a simulated platform, as in Microsoft’s Flight Simulator piloting program. On the other hand, it may use a very realistic and sophisticated model of the platform, or even components of the real system, as in the Rafale simulation centers, where each simulator contains an on-board computer as used in the real aircraft.
Simulation: History, Concepts, and Examples
43
Figure 1.11. Ductor constructive simulation (EMM/ANPROS)
Figure 1.12 shows a “full flight” helicopter simulator, including a reproduction of the cabin of the aircraft (Puma or Cougar), mounted on a three-axis table which allows the crew to feel very realistic movements, and a projection sphere giving immersive vision of the 3D environment. Simulation with instruments uses real material operated by human users in the field. Simulation is used to supply a tactical environment and model the effects of weapons. Simulation with instruments is often referred to as “live simulation”, which designates its field of application. This kind of simulation is used for on-theground training of operators, for example, soldiers, in very realistic conditions; they use their equipment (guns), specially fitted with laser transmitters, data links, recorders, and so on. In the civilian world, the game of paintball could be considered as belonging to the domain of live simulation. In the military sphere, a typical example would be the French Army’s Combat Firing Simulator for Light Weapon (simulateur de tir de combat pour armeslégères, STC/AL), a small weapons combat simulator, used to train operators of the FAMAS, MAS 49/56, AA52, ANF1, and MINIMI weapons (Figure 1.13). Infantry soldiers wear a gilet with infrared and laser sensors, which detect shots hitting the wearer and produce a signal. The STC/AL is not limited to firing devices and also contains a control and arbitration
44
Simulation & Modeling of Systems of Systems
system for starting a combat simulation exercise, studying the exercise for after action analysis and the lessons learned, penalties for “shot” participants, and so on. An equivalent for tank training exists in the form of the CENTAURE system, set up by the French Army at their Mailly camp.
Figure 1.12. SHERPA flight simulator (SOGITEC)
The American Department of Defense uses a similar classification, with three main categories of defense simulation: − constructive simulations: simulated systems implemented by simulated operators (with or without the intervention of a real operator in the course of the simulation), for example, war games; − virtual simulations: simulated systems implemented by real operators. This group includes piloted simulations but also simulations of information and command systems, for example, flight simulators or a simulation of the control center of a nuclear power station; − live simulation: real systems are implemented by real operators, for example, in an infantry combat shooting simulator.
Simulation: History, Concepts, and Examples
45
Digital simulations and hardware-in-the-loop simulations are not included in this classification, which focuses on operational and technico-operational simulations used by the US Department of Defense.
Figure 1.13. STC/AL system on a FAMAS (EADS/GDI Simulation)
1.4.2.3. Uses Simulation has innumerable uses. Within the Ministry of Defense and civilian industry, we can list a number of examples: initial training, continuous training, mission rehearsal, assistance to decision making, justification of choices, concept studies, planning, operation management, forecasting, systems acquisition, marketing, and leisure. Initial training aims to equip a student with basic skills in using a procedure or a system. Gaps in the students’ knowledge and their lack of experience often make immediate use of the real system dangerous; simulation provides a useful halfway step, often using a simplified, targeted version of the system to reduce costs and complexity. In this way, the pupil masters the necessary cognitive processes (e.g. use of a weapons system) and/or psychomotor reflexes (e.g. reaction to a breakdown).
46
Simulation & Modeling of Systems of Systems
Continuous training is used to maintain and develop the capacities of system users by increasing the overall time spent using the system or by exposing them to specific situations (e.g. landing without visibility and system failures). Training activities are the best-recognized use of simulation by the general public, via flight simulators; as spectacular as these are, however, they only represent part of a larger group of training systems. Training simulation systems constitute a whole family which covers a far wider area than learning to pilot a system, including learning procedures, deepening knowledge of the system, maintenance training, and crisis management training. As well as piloted trainers and simulators (not just aircraft simulators but also nuclear power station control center simulators, air traffic control simulators, mechanical excavator simulators, laparoscopic surgical equipment simulators, and so on), we also find constructive simulations (war games, economic strategy games, and so on.). Note that simulation may be the only tool used in training (this is generally the case with piloted simulation) or one of many (as with computer-assisted exercises (CAX), where a simulation provides more or less automatically the environment needed to stimulate those in training, thus reducing the human resources needed for the exercise while maintaining, or even enhancing, realism). Mission rehearsal falls somewhere between training and assistance in carrying out operations; for the military, this consists of playing out a mission in simulation before real-world execution. For example, a pilot may practice bombarding a target in a theater of operations, familiarizing himself or herself with the terrain and the dangers of the zone in question, so that when carrying out the real mission, he or she already possesses the necessary reflexes to achieve his or her objectives and for self-defense. Mission rehearsal by simulation is also used by civilian bodies, for example, in preparing for a sensitive operation (such as repairing a pipe in a nuclear power plant or an innovative surgical intervention). In a world where organizations are becoming more and more complex, simulation has become a key tool for decision-makers, be they engineers, involved with the military, politicians, or financiers. Simulation can provide assistance while making decisions. By simulating the possible consequences of a decision, it becomes possible to choose the best strategy when faced with a given problem. In broad outline, the method usually consists of simulating the evolution of a given system several times, varying certain parameters (those implicated in the choices of the decision-maker) each time, or of validating a choice by simulating the impact of the decision on the situation. This “decision aid” aspect can be found to greater or lesser degree in other domains, such as planning and systems engineering. Choice justification is the next step for the decision maker, or the person recommending the decision: once a decision has been made, simulation can be used to provide certain elements that establish the degree of pertinence of the
Simulation: History, Concepts, and Examples
47
decision. In this way, technico-operational simulations can be used to measure the conformity of an architecture to the initial specification. A concept study consists of establishing the way in which a system should be used: for example, how many main battle tanks should there be in an armed platoon? How should vehicles move along a route where improvised explosive devices (IEDs) may be present, a current problem for troops in Afghanistan? How should tanks react in engagements with an enemy equipped with anti-tank weapons? In the civilian domain, similar questions can be applied to any system; for example, how should an airline use an Airbus A380? How should the time be divided between use and maintenance periods? Planning is an activity which involves scheduling actions and operations to be carried out in a domain, with given aims, means, and duration. It allows the definition of objectives and intermediary steps required for the achievement of the overall aims of an operation or the implementation of a system or system of systems. For the Army, this may involve the development of plans for troop engagements; for civilians, this may supply plans for a factory or company. This planning may be carried out without particular time constraints. In a military context, this could involve, in peacetime, a study of scenarios based on hypotheses laid out in the defense department’s White Paper. Planning may also need to be carried out within a strict time period, to assist in the management of a crisis. In this case, planning and decision-making uses of simulation come together. Simulation allows several action plans to be tested and assists in choosing the optimal solution. The distinction between planning with and without time constraints is essential, as the demands made on tools are very different: with time constraints, the tools must be able to produce the necessary scenarios and environment very quickly, sometimes in a matter of hours. Operation management demands evaluation of a situation and measurement of deviation from what was anticipated in the plans, with a possibility of rapid revision of plans for the operation. Although this activity is essentially based around information and command systems, simulation may be used to visualize the situation and its possible short-term developments. Forecasting involves preparing for the future by evaluating different possible scenarios, most often in the long term. How will the global economy evolve? In 20 years’ time, what will be the effects of the diminution of crude oil reserves? What increase in demand will spark the development of China and India? What might be, over 10 years, the impact of a massive introduction of vehicles running on alternative power sources (hybrid, electric, hydrogen, biofuel, solar, and so on)? Systems acquisition is at the core of the work of the DGA, the main project owner of complex systems in France and a major user of simulations. Simulation
48
Simulation & Modeling of Systems of Systems
effectively accompanies a system throughout the acquisition cycle, that is, at every step from the expression of a need by the client to the commissioning of the system, from a project ownership or project management point of view. We find here the main theme of this study, so the use of simulation will be discussed in detail in later chapters. The appearance of 3D scene and simulation authoring systems has led to a significant development in the use of virtual reality for marketing purposes. It is not always easy to demonstrate the interest of a system or a system of systems via a brochure. It is often possible to give live demonstrations, but these are expensive to organize and will not necessarily be able to show all the performances or operation of the system. Using simulations, the system can be implemented virtually in an environment representative of the needs of the client, making simulation a powerful tool in securing sales, whether for complex systems (e.g. implementation of air defenses) or more standard systems (e.g. a demonstration of the anti-pollution technology in a particular car). We shall finish by touching on a very widespread but little understood use of simulation: leisure activities. This is, in fact, by far and away the main use of simulation. In 2007, video games alone represented a global market of almost 40 billion euros, approximately 10 times the size of the defense simulation market. The contributions of this domain to the progress of simulation are not purely financial; although defense drove technological innovation for simulation until the beginning of the 1990s, this is no longer the case. Video games are responsible for the fact that work stations are now commonplace and therefore cheap and equipped with multi-core processors capable of millions of operations per second, and graphics sub-systems are capable to render, in real time, complex 3D scenes in high definition. We also have access to richer and more user-friendly software, allowing unequalled productivity in the generation of virtual environments. There will always be a place for expensive, specific high-end systems, but their market is shrinking, and visual simulation is becoming much more widespread. Certain parts of the military have even begun to use video games directly, with more or less modifications; these applications have gained the name of “serious games”. The French Army has developed a cheap system, INSTINCT, used for basic infantry combat training, based on Ubisoft’s Ghost Recon game. SCIPIO (Simulation de Combat Interarmes pour la Préparation Interactive des Opérations, interweapon combat simulation for interactive preparation of operations), the constructive simulation used in the training system for command posts in the French Army, was developed specifically for the Ministry of Defense, but the core uses innovative technology developed by MASA SCI, which was initially used for video games. Its efficiency in terms of modeling human behavior, a major technological challenge in the SCIPIO project, is particularly advanced compared with the technology used before.
Simulation: History, Concepts, and Examples
49
We could give a number of other examples, such as therapy (simulation is used in combination with virtual reality to cure phobias, such as fear of flying), but we shall stop here, having already discussed the most common categories of simulation use. The list of uses of simulation given above was general in nature and presented without following a particular structure. In practice, various bodies have attempted to provide a structure for uses of simulation. Figure 1.14 show the typology of simulations used by the French Ministry of Defence, which was first defined in 1999 and underwent a few evolutions since. This typology defines five areas for M&S, based on its purpose: – Préparation des forces (PDF): Training & education of armed forces. – Appui aux opérations (AAO): Support to military operations, that is to say planning, rehearsal, execution and analysis of operations, but also keeping knowhow level of deployed troops on location. – Préparation de l’avenir (PDA): Support to design and evaluation of capabilities models, concepts and doctrines, systems specifications, while taking into account risks and innovations. – Acquisition (ACQ): Support to systems acquisition process in armament programs (see section 1.3.2 and Figure 1.6). – Soutien à la réalisation des outils (SRO): support to tools design. This is a transverse area, contributing to the others through evaluation of technologies and innovations, and definition of common tools, standards and requirements.
Figure 1.14. Typology of defense simulations
50
Simulation & Modeling of Systems of Systems
Analysis simulation to support design of defence capabilities (simulation pour l’aide à la conception de l’outil de defense, SACOD) is simulation used to assist in the design of defense tools. It consists of defining future defense systems in terms of need, means (national and allied, present and future), and threats (also present and future), and it provides a means of arbitrating between different possibilities according to capacitive and economic criteria. In practice, SACOD is the step before a system acquisition process, but it was initially distinguished from acquisition support for practical reasons (different teams using different tools). Nowadays, with the increased importance of capability and system-of-systems approaches, this distinction is less clear cut. In Figure 1.14, the SACQ is the technical and functional simulation used to support the acquisition process after SACOD. In general, simulation for acquisition should also cover SACOD, but the division was made for practical reasons: as a general rule, SACOD and SACQ are not run by the same individuals and do not use the same types of models (SACQ uses lower-level technical and technico-operational simulations than SACOD). Note that, all the same, there is considerable overlap between the two. In passing, note that the limits of each domain are not, in reality, precise; a simulation can be used for studying concepts, training, or defense analysis, with or without adapting the application. Finally, a new domain has appeared in defense simulation in the past few years: Concept Development & Experimentation (CD&E). CD&E is an approach launched by NATO and a number of countries with the aim of implementing processes to study (for forecasting purposes) and evaluate new concepts before trying them. In this way, investment in a concept can be delayed until it has been consolidated, analyzed, and validated by the various partners involved (the military, the project owners, and the industrial project managers) following capacitive logic, looking at doctrinal, organizational, training, financial, and logistical aspects. Of course, the CD&E approach strongly recommends the use of simulation. This approach has structured reflections on capacity and systems of systems in relation to defense and was one of the key factors in the implementation of battle labs in several NATO member states and in parts of the defense industry, such as the technico-operational laboratory of the French Ministry of Defense. We shall return to this subject in more detail in Chapter 8. 1.4.2.4. Other classifications Many different typologies of simulation exist and can be found in various works and articles. The following sections cover some further elements of taxonomy, which can be used to characterize a simulation system.
Simulation: History, Concepts, and Examples
51
− Interactivity: a simulation can be called open, or interactive, if an operator can intervene in the course of the simulation, that is, by generating events. A simulation which does not allow this is referred to as closed or non-interactive − Entity/events: the simulation that operates on the principle of entities or objects that exchange messages among themselves and engage in actions or through events that modify the state of the system and its components. This has a major impact on the architecture: as in the first case, we can, for example, use agent modeling techniques, whereas in the other, we are in the framework of a discrete event simulation engine. − Determinism: the simulation integrating random phenomena is known as a probabilistic or stochastic simulation. This feature is often attached to another factor, that of reproducibility: a stochastic simulation can be reproducible if an execution can be replayed identically, despite the presence of random processes. This characteristic is important for study simulations, to which we will return in greater depth later. − Time management: the time management strategy is a key element in the architecture of a simulation, and so also an item of taxonomy. A simulation may or may not be real time. Real time may be “hard”, that is, it must be imperatively respected, for example, if there is hardware in the loop, or “soft”, if the simulation is required to operate in real time, but respect of timing is not critical for the validity and proper operation of the system. Among non-real time simulations, we find simulations by time step, with discrete and mixed events. There are also simulations that operate in N times real time; this is usually a trick to avoid the time management question, based on the clock of the host computer. − Distribution and parallelism: a simulation may be monolithic and destined for use in a single processor in a single machine. But it may be optimized for implementation on several processors or a processor with multiple cores to improve performance. The architecture of a simulation may also be distributed, with the application being shared across several machines in the same place (local network or cluster) or at a distance (geographic distribution). This very restrictive distribution is a design choice which can be justified by performance requirements or the possibility of reusing simulation components (see Chapter 7).
1.5. Conclusion We have seen that simulation is a fundamental and almost essential tool for complex system and system-of-systems engineering. Widespread in most corresponding activities, at all stages in the life-cycle of the system, the field of simulation is diversified, a fact which can pose coherence and usage problems; we shall come back to this subject later.
52
Simulation & Modeling of Systems of Systems
Simulation can be complex and demands to be used methodically and with prudence to avoid costs spiraling out of control, thus reducing the profitability of the approach, and to avoid producing unusable results, which can happen when a model is invalid. As with any system, simulation requires a rigorous approach and appropriate use. If these conditions are respected, considerable gains can be made, which benefit all involved in the life-cycle of the system.
1.6. Bibliography [ADE 00] ADELANTADO M., BEL G., SIRON P., Cours de modélisation et simulation de systèmes complexes, Société des Amis de Sup’Aero (SAE), Toulouse, 2000. [BAN 98] BANKS J., Handbook of Simulation, Wiley-Interscience, Malden, 1998. [BER 98] BERNSTEIN R., A road map for simulation based acquisition, report of the Joint Simulation Based Acquisition Task Force, US Under Secretary for Acquisition and Technology, Washington DC, 1998. [BOS 02] BOSHER P., “UK MOD update” (on Synthetic Environment Co-ordination Office), presented at the Simulation Interoperability Workshop (02F-SIW-187), SISO, Orlando, 2002. [BRA 00] BRAIN C., “Lies, damn lies and computer simulation”, Proceedings of the 14th European Simulation Multiconference on Simulation and Modelling, Ghent, 2000. [CAN 00] CANTOT P., DROPSY P., “Modélisation et simulation: une nouvelle ère commence”, Revue scientifique et technique de la défense, DGA, Paris, October 2000. [CAN 01] CANTOT P., Cours sur les techniques et technologies de modélisation et simulation, ENSIETA, Brest, 2002. [CAN 08] CANTOT P., Introduction aux techniques et technologies de modélisation et simulation, ENSIETA, Brest, 2008. [CAN 09] CANTOT P., Simulation-Based Acquisition: Systems of Systems Engineering, Classes, Master of Systems Engineering, ISAE, Toulouse, 2009. [CAR 98] CARPENTIER J., “La recherche aéronautique et les progrès de l’aviation”, Revue scientifique et technique de la défense, n 40, DGA, Paris, 1998. [CAS 97] CASTI J.L., Would-Be Worlds: How Simulation Is Changing the Frontiers of Science, John Wiley & Sons, New York, 1997. [CEA 05] COMMISSARIAT CEA, December 2005. [CEA 06] COMMISSARIAT release, January 2006.
À L’ENERGIE
À L’ENERGIE
ATOMIQUE, Le tsunami de Sumatra, press release, ATOMIQUE, Le supercalculateur TERA-10, press
[CHE 02] CHEDMAIL P., CAO et simulation mécanique, Hermès, Paris, 2002.
Simulation: History, Concepts, and Examples
53
[CCO 04] COUR DES COMPTES, Maintien en condition opérationnelle du matériel des armées, The French Government Accounting Office, http://www.ccomptes.fr, Paris, 2004. [DAH 99] DAHMANN J., KUHL F., WEATHERLY R., Creating Computer Simulation Systems, Prentice Hall, Old Tappan, 1999. [DGA 94] DELEGATION GENERALE POUR L’ARMEMENT, L’armement numéro 45, special edition on simulation, DGA, Paris, 1994. [DGA 97] DELEGATION GENERALE POUR L’ARMEMENT, Guide dictionnaire de la simulation de défense, DGA, Paris, 1997. [DOC 93] DOCKERY J.T., WOODCOCK A.E.R., The Military Landscape: Mathematical Models of Combat, Woodhead Publishing Ltd, Cambridge, 1993. [DOD 94] US DEPARTMENT Washington DC, 1994.
OF
DEFENSE, Directive 5000.59-M: DoD M&S Glossary, DoD,
[DOD 95] US DEPARTMENT OF DEFENSE, U.S. Modeling and Simulation Master Plan, Under Secretary of Defense for Acquisition and Technology, Washington DC, 1995. [DOD 96] US DEPARTMENT OF DEFENSE, U.S. Instruction 5000.61:U.S. Modeling and Simulation Verification, Validation and Accreditation, Under Secretary of Defense for Acquisition and Technology, Washington DC, 2003. [DOD 97] US DEPARTMENT OF DEFENSE, U.S. DoD Modeling and Simulation Glossary, Under Secretary of Defense for Acquisition and Technology, Washington DC, 1997. [DOR 08] DOROSH M., The Origins of Wargaming, Calgary, Canada, www.tactical gamer.com, 2008. [EUR 01] EUROPEAN COMMISSION, European Aeronautics: A Vision For 2020 – Report of the Group of Personalities, Office for Official Publications of the European Communities, Luxembourg, 2001. [GIB 74] GIBBS G.I., Handbook of Games and Simulation Exercises, Routledge, Florence, 1974. [GIB 96] GIBBs G.I., Jim Blinn’s Corner: A Trip Down The Graphic Pipeline, Morgan Kaufmann, San Francisco, 1996. [IST 94] INSTITUTE FOR SIMULATION AND TRAINING, The DIS vision: A map to the future of distributed simulation, technical report, IEEE, Orlando, 1994. [IST 98] INSTITUTE FOR SIMULATION AND TRAINING, Standard 610.3: Modeling and Simulation (M&S) Terminology, IEEE, Orlando, 1998. [JAM 05] JAMSHIDI M., “System-of-Systems Engineering - a Definition”, IEEE International Conference on Systems, Man and Cybernetics, Hawaii, October 2005. [JAN 01] JANE’S, JANE’S Simulation & Training Systems, Jane’s Information Group, IHS Global Limited, 2001.
54
Simulation & Modeling of Systems of Systems
[KAM 08] KAM L. LUZEAUX D., “Conception dirigée par les modèles et simulations”, in D. LUZEAUX (ed.) Ingénierie des systèmes de systèmes: méthodes et outils, Hermès, Paris, 2008. [KON 01] KONWIN C., “Simulation based acquisition: the way DoD will do business”, presentation at the National Summit on Defense Policy, Acquisition, Research, Test & Evaluation Conference, Long Beach, March 2001. [KRO 07] KROB D., PRINTZ J., “Tutorial: concepts, terminologie, ingénierie”, presentation, Ecole systèmes de systèmes, SEE, Paris, 2007. [LEM 90] LE MOIGNE J.-L., Modélisation des systèmes complexes, Dunod, Paris, 1990. [LIA 97] LIARDET J.-P., Les wargames commerciaux américains, des années soixante à nos jours, entre histoire militaire et simulation, une contribution à l’étude de la décision, Doctoral Thesis, University of Paul Valéry, Montpellier, 1997. [LUZ 03a] LUZEAUX D., “Cost-efficiency of simulation-based acquisition”, Proceedings of SPIE Aerosense ‘03, Conference on Simulation, Orlando, USA, April 2003. [LUZ 03b] LUZEAUX D., “La complémentarité simulation-essais pour l’acquisition de systèmes complexes”, Revue de l’Electricité et de l’Electronique, vol. 6, pp. 1–6, 2003. [LUZ 10] LUZEAUX D., RUAULT J.-R., Systems of Systems, ISTE, London, John Wiley & Sons, New York, 2010. [MAI 98] MAIER M.W., “Architecturing principles for systems of systems”, Systems Engineering, vol. 1, no. 4 , pp. 267–284, 1998. [MEI 98] MEINADIER J.-P., Ingénierie et intégration des systèmes, Hermès, Paris, 1998. [MON 96] MONSEF Y., Modélisation et simulation des systèmes complexes, Lavoisier, Paris, 1996. [NAT 98] NORTH ATLANTIC TREATY ORGANISATION, NATO Modeling and Simulation Master Plan version 1.0., RTA (NATO Research & Technology Agency), AC/323 (SGMS)D/2, 1998. [NRC 97] NATIONAL RESEARCH COUNCIL, Modeling and simulation: linking entertainment and defense, report of the Committee on Modeling and Simulation: Opportunities for Collaboration Between the Defense and Entertainment Research Communities Computer Science and Telecommunications Board Commission on Physical Sciences, Mathematics, and Applications, NRC, Washington, 1997. [QUE 86] QUÉAU P., Eloge de la simulation, Champ Vallon, Seyssel, 1986. [REC 91] RECHTIN E., Systems Architecting: Creating and Building Complex Systems, Prentice-Hall, Englewood Cliffs, 1991. [RTO 03] NATO RESEARCH AND TECHNOLOGY ORGANISATION, RTO SAS Lecture Series on “Simulation of and for Military Decision Making” (RTO-EN-017), RTA (NATO Research & Technology Agency), The Hague, 2003.
Simulation: History, Concepts, and Examples
55
[SIM 62] SIMON H.A., “The architecture of complexity: hierarchic systems”, Proceedings of the American Philosophical Society, vol. 106, no. 6, pp. 467–482, 1962. [SIM 69] SIMON H.A., The Sciences of the Artificial, MIT Press, Cambridge, 1969. [SIM 76] SIMON H.A., “How complex are complex systems?”, Proceedings of the 1976 Biennial Meeting of the Philosophy of Science Association, American Philosophical Society, Philadelphia, vol. 2, pp. 507–522, 1977. [STA 95] STANDISH GROUP INTERNATIONAL, Chaos, investigation report, West Yarmouth, March 1995. [ZEI 76] ZEIGLER B., Theory of Modeling and Simulation, John Wiley & Sons, Hoboken, 1976.
Chapter 2
Principles of Modeling
2.1. Introduction to modeling Modeling is the activity by which a model is designed. In this chapter, we shall see how a model can be designed and some of the most widespread types of model. Note that modeling for simulation obeys more or less the same rules as modeling in complex systems engineering, albeit with certain specific constraints which, although they are not limited to the field of simulation, are particularly strong in this case; these include the validation, the choice of levels of refinement (which has an impact on the model), and the need to take installation (e.g. the capacities of the target computer platform) into account. We start by laying down some basic principles of modeling: – The model is constructed to solve a specific problem and is, therefore, subjective. A model, although representative of a system, is not necessarily valid or relevant. – The same problems occur again. There is much to be gained by systematically checking whether the model has already developed with similar principles. If so, attempts should be made to modify the existing model, assuring that its validity and availability can be guaranteed. Continuing with the same principle, a model should be able to exist, as hypotheses concerning a system may change (particularly in the case of studies being carried out for a future system) or the system itself may exist (in the case of Chapter written by Pascal CANTOT.
58
Simulation & Modeling of Systems of Systems
a simulation used while the system is already operational, e.g. a flight simulator which must remain coherent with the real platform). – Simplicity and efficiency are essential. Unnecessarily complex setups, which are understood only by the author (if that), must be avoided. The level of refinement of the model should be strictly limited to the minimum. In general, a basic rule of successful engineering should be applied: the KISS method (Keep It Simple, Stupid!). – Simulation is not necessarily the most appropriate tool in all cases. We should always check for alternatives (constraint programming, expert advice, and so on). – The question of validation should always be considered. Validation is a continuous process and a brief diagnostic at the end of each phase of the process is insufficient. – The client should be involved in modeling, for example, to validate the taxonomy used in relation to their sphere of work. Feedback mechanisms should be set up for every step of the modeling and simulation process. – Modeling should be left to experts. The installation of a model requires solid computing abilities, but the system should be modeled by real experts with a perfect command of the system architecture and performances, or, at a push, by people in constant liaison with such experts. Control of a modeling studio is an advantage, but is not sufficient to guarantee the production of a valid and relevant model. Ideally, modeling should be carried out by a trade expert and a computer scientist specialized in simulation working together. – Models and simulations should not only be considered as products in their own right, managed according to the quality control processes of the business (such as configuration management), but also as assets to the business which will be capitalized through future reuse. Awareness of their existence should, therefore, be widespread within the organization and documentation should be readily available. 2.2. Typology of models Models can take multiple forms and have various characteristics. In this section, we shall present a simple typology of the most widespread model types. 2.2.1. Static/dynamic A model is static if it does not evolve over time. For example, both a threedimensional (3D) computer-aided design (CAD) model and a two-dimensional (2D) plan of a house are examples of static models.
Principles of Modeling
59
A model is dynamic, on the other hand, if it evolves over time. The model of the atmosphere used in weather forecasting is dynamic, for example, as the weather changes over time. This dynamism is often expressed in the form of laws of evolution and events, as we shall see later. 2.2.2. Deterministic/stochastic A model is deterministic if, for a given entry, the results are always identical. For example, the model of the falling billiard ball given in Chapter 1 is deterministic, as, if we carry out the corresponding simulation with the same entered data, we always obtain the same results. A model is stochastic or random when its results are apparently determined by one or more random variables, which represent an uncertain process. In other words, the results of the simulation will vary from one execution to another, even if the starting data is identical. The values of these random variables may follow a statistical law (Chapter 4). This type of model allows a random process which should be considered. For example, in Figure 2.1, the impact of a bullet on a target is changed by a random trajectory error (dispersion), following a known law (in this case Gaussian, or normal, distribution). This error is the result of a number of accumulated causes, such as the manufacturing tolerance in munitions (even in the same batch, no two cartridges will be absolutely identical) combined with the absence of total control over initial conditions (it is hard to ensure that the weapon is at the same temperature for each shot, e.g. or that the state of the interior of the gun is identical).
Figure 2.1. Shots on a target
60
Simulation & Modeling of Systems of Systems
Shooting precision could, in fact, be considerably improved by improvements in the munitions manufacturing process and better control of test conditions, but 100% success is impossible; random factors can be reduced, but not completely eliminated. A process of this kind is known as a stochastic process. The term Markov process is also used for a stochastic process in which the probability of an event occurring depends solely on the previous event. If this process is discrete, it is known as a Markov chain. A stochastic model being, by its very nature, non-deterministic, the results produced using the same input data will not necessarily be the same. In practice, when shooting a target, it is practically impossible to deliberately fire two bullets through the exact same hole, even if the weapon is placed in a clamp to prevent it from moving; there will always be uncontrollable variations, however small (vibrations, air movements, irregularities in the manufacture of the cartridges, temperature variations, and so on). The physical reality is not what certain westerns would make us believe! Other examples of random processes include hardware failures, entry of a customer into a shop, the arrival of an electronic message at a server, background noise in a radio communication, or the disintegration of an unstable atomic nucleus. Without going into mathematical detail, we note that these examples are modeled by various statistical laws, which differ from that cited in the previous example. A deterministic system can also be defined as being a system that evolves predictably: starting from state Si, for given conditions, it must evolve towards state Sj (Figure 2.2). Under the same conditions, a stochastic system, on the other hand, could evolve in several different ways.
Sj Si
Sj
Si Sk
Deterministic system
Stochastic system
Figure 2.2. System state evolution
Principles of Modeling
61
The result of a modeling choice depends on whether a model is deterministic or not. Thus, the determination of the line of sight (LOS) of an entity in a simulation can be done following the geometric (deterministic) method, in which a line is drawn between the target point and the entity to ensure that there are no obstacles in the way, or using the probability method, which aims to model the attention of the entity, or perturbations caused by vegetation or fog (Figure 2.3). In the geometric model, vegetation will always or never be an obstacle between the rocket launcher and the tank. In the probability model, the tank has a chance of being able to see the rocket launcher through the vegetation, with a probability based specifically on the density of the vegetation.
Figure 2.3. Two instances of calculation of the line of sight between two objects
If a stochastic model is chosen, certain problems arise concerning whether the results can be used. As results vary from one implementation to the next, we need to be able to draw conclusions. For this, we use statistical techniques applied not to one, but to several executions or replicas. The number of these executions will determine the statistical precision of the results. Developed during the 1940s in the course of US military activities (the Manhattan Project, logistical organization, and so on), the Monte-Carlo method is the method most frequently used to obtain estimation of the precision of results and the number of executions or experiments required. To understand the principle of the Monte-Carlo analysis, we can use a game of darts to calculate π. Let us take a circular target of radius r and surface σ = πr2 drawn in a square with sides of 2r and a surface S of 4r2.
62
Simulation & Modeling of Systems of Systems
Let us throw a large number, N, of darts randomly inside the square (we will not count those which are missed). Among these, the number n will hit the target. The probability P that a dart will hit the target is equal to the relationship between the surface of the circle and that of the square; thus P=
σ S
=
π .r 2 4r 2
=
π 4
Experimentally, the proportion ρ(N) of darts that hit the target is n divided by N. The more the number of experiments increases, the more this value should statistically approach P, that is, lim ρ ( N ) = P . We can thus deduce a value N →+∞
approximating π ⋅ 4n / N (Figure 2.4).
S
σ
Figure 2.4. Calculation of π using a game of darts
In practice, a very large number of experiments are needed to obtain a value correct to even three or four decimal places, and the procedure is very demanding in terms of calculations, but the example illustrates how a stochastic model can be used (this method is sometimes used to calculate certain complex integrals). The MonteCarlo method provides, among other things, an evaluation of error in the value of π, or an estimation of the number of experiments necessary to give the required level of precision, which shows how many times a stochastic simulation must be executed to obtain the desired results (or at least have a high probability of obtaining these results).
Principles of Modeling
63
When designing a simulation that uses stochastic models, it is sometimes necessary to be able to reproduce certain courses of execution; this appears to be contradictory to the fact that, in principle, no two replicas are identical. To obtain the necessary reproducibility, we make use of the fact that the production of random numbers within a simulation is carried out using functions f(x) known as pseudorandom number generators which generate a sequence of seemingly random numbers, u0, u1, …, un so that ui+1 = f(ui). If we know the initial term u0, we can deduce all following values produced by the random generator. This u0 value, called the seed, makes the whole random sequence and thus the results of the simulation (if it is properly constructed!) are reproducible. In the case of a simulation for study, this reproducibility is usually indispensable, as it allows certain situations to be replayed and analyzed to adjust the simulation or to study borderline cases.
2.2.3. Qualities of a model In absolute terms, it is impossible to define an ideal model for a system as the “ideal” model varies according to its intended use. However, a certain number of basic qualities can be used to evaluate the relevance of a model, which should be:
− as simple and as clear as possible, − valid (and able to be validated), − as accurate as possible where the requirements of the simulation are concerned, and − the most efficient and effective in terms of intended goals. 2.2.3.1. Simplicity Simplicity is the property of being simple and not associated with other elements, and is an essential quality in engineering. We expect that the reader of this book may be (we presume) interested in mastering complex systems. If, in attempting to do this, we use simulations and models which are themselves very complex and difficult, if not impossible to master, then progress, if any is made, will be slow. All too many simulation projects have failed due to an inability to master their own complexity. In this case, the principle of Occam’s razor should be applied: “entities must not be multiplied beyond necessity.” This simplicity is a result of the combination of several factors:
− Choice of a minimalist model relative to the intended goal: an example of this kind was given in the previous chapter (the falling billiard ball).
64
Simulation & Modeling of Systems of Systems
− Rigor in constructing the model, avoiding duplicates and ambiguities, and sources of confusion and mistakes. − Use of a suitable meta-model, understandable by the expert, the client, and the computer scientist. Unified Modeling Language (UML1) is admittedly very sophisticated, but its interpretation demands a certain level of experience. Do all involved have the necessary ability to prove the model? This is an important consideration when asking them to validate the model. 2.2.3.2. Accuracy The accuracy of a model is the level of physical and functional similarity in it and the real system it represents. In the case of our falling ball example, this accuracy is shown by the differences between the simulation and measurements taken from a real billiard ball dropped in the atmosphere. Accuracy should not be confused with refinement: a refined model may not be more accurate than a cruder model (although this is usually the case). The refined model might have validity problems, for example, or might not be relevant. To fully understand this distinction, let us consider an example, a digital model of a terrain, representing a space of 30 × 30 km. We shall use the data format, digital terrain elevation data (DTED) as a physical support. DTED gives five possibilities for grid size resolution, from 1 to 100 m, as indicated in Table 2.1. Refinement (post-spacing)
Number of points
Quantity of data
Database size
DTED 1
~ 100 m
90,000
1,442,401
5 Mb
DTED 2
~ 30 m
810,000
12,967,201
54 Mb
DTED 3
~ 10 m
5,000,000
144,024,001
583 Mb!
DTED 4
~3m
21,250,000
1,296,072,001
6,3 Gb!
DTED 5
~1m
506,250,000
11,660,000,001
68 Gb!!!
DTED level
Table 2.1. Terrain database size according to DTED level
At DTED 1 (the least refined level), our space of 900 km2 is represented by 90,000 points. At DTED 3, we have 5 million points and at DTED 5 (the most refined level), we have over 500 million points. We would think that modeling using DTED level 5 would be the most accurate, as it allows variations to be represented from 1 m to the next, for example, holes, rocks, and so on, which would be invisible 1 Unified Modeling Language, description language proposed by the Object Management Group (OMG), an international consortium of scholars and industrialists.
Principles of Modeling
65
at lower levels of refinement. However, for this, we need data for the terrain with the same resolution. Even today, it remains difficult to obtain digital terrains within 1 m over large areas, and there is a high chance that a large part of the DTED 5 terrain would have to be extrapolated; this means there would be no gain in accuracy between DTED 4 and DTED 5. On the other hand, the quantity of data is multiplied by 10 between both these levels, making DTED 5 much more costly, and prone to an increased risk of error. 2.2.3.3. Validity Validity, as we have already seen, is the ability of a model to represent the real world in a sufficiently precise and exact manner for a given purpose. This validity is only valid for a sub-group of possible states of the system, known as the domain of validity. We shall not go into this fundamental concept of modeling and simulation here as an entire chapter is dedicated to the subject later on. We should point out, however, that validity and accuracy are two concepts which can be linked, but remain different; a model is not necessarily valid because it is accurate, and vice versa. To understand this notion, we shall use the example of the modeling of the Earth in the context of an artillery fire simulation. Let us first consider a flat Earth model, as the one found on a travel map. Of course, this model is not “exact” in relation to reality, as we know today that the Earth is not flat. However, if our simulation involves sending a mortar shell over a few kilometers, the model is perfectly valid. If, on the other hand, we shoot further – say 100 km – the curve of the Earth’s surface cannot be ignored. We could possibly “cheat” a little by applying an altitude correction related to the distance, which has the effect of deforming the initially flat terrain model.2 If our simulation involves shooting a tactical ballistic missile with a range of several hundred kilometers, our flat model will no longer suffice and a spherical model of the Earth is required. If we want to simulate firing a strategic ballistic missile (over several thousand kilometers), the spherical model ceases to be valid in turn. As everyone knows, the Earth is not a sphere, but an ellipsoid (like a rugby ball), flattened at the poles by a factor of 1/300 (or, more precisely, of 1/298.257 223 563 according to the reference ellipsoid WGS84). Using this, we obtain another model, which is more valid for our simulation. 2 This application of a nonlinear deformation conforms with the principle of Mercator representations on geographical maps, in which the extent of the Arctic and Antarctic regions are exaggerated in comparison with the rest of the globe.
66
Simulation & Modeling of Systems of Systems
We might think that, by using this last model every time, we would be sure to have a valid model, whatever the aim of the simulation might be. This is probably the case, but it is not necessarily relevant. The use of an ellipsoid model involves managing a more complex system of coordinates than that used by a simple 2D model (with or without altitude correction); in the case of our short-range simulation, use of an excessively precise model would make data and trajectory calculation unnecessarily complex. Note, in passing, that we could go even further, as the ellipsoid model too is just an approximation; the true form of the Earth is even more complex. The planet is, in fact, a geoid, shaped (roughly) like a vaguely ellipsoidal potato. From a geographical coordinates point of view, this would be particularly tricky to deal with. The end story is that both the validity and relevance of a model are relative and depend on the aim of the simulation, as we have already stated and will continue to repeat. 2.2.3.4. Efficiency Effectiveness describes the capacity of a model to attain fixed aims, for example, respond to a given question (comparison of architectures) or allow pilots to train. Efficiency means the fact of achieving an objective using the fewest possible resources. A true efficient model will, therefore, be efficient in terms of operation and cost. The level of accuracy required for the model and the precision needed to provide results satisfying our needs should be given particular attention, allowing us to set the bar in terms of accuracy. To return to our example of a numerical model of a terrain, using the level DTED 3 to model terrain for an Airbus A380 flight simulator (which will mostly fly at high altitude) would not be at all efficient; on the other hand, it might prove efficient for simulating an Earth-bound vehicle.
2.3. The modeling process Lieutenant General William Wallace, of the US Army, made the following comment during the first weeks of the war in Iraq (2003): “The [Iraqi] enemy we’re fighting is different from the one we’d war-gamed against”. By this remark, he expressed his concern that the enemy did not react as it had in simulations, raising doubt as to the effectiveness of the preparation of the Coalition forces. Effectively, the simulations were based on a model of the enemy, which reflected the perception held of the enemy, a vision that proved tragically misguided. Of course, the difficulties of the American-led coalition in Iraq cannot be entirely blamed on this
Principles of Modeling
67
bad approach, but it certainly had a negative impact on the preparation of operations and of the troops. Modeling is the fundamental activity at the base of simulation, and probably the most delicate activity involved, as it determines the validity of the simulation and the credibility of the results. Evidently, the validity of data input into the model is also essential (following the principle of garbage in, garbage out), along with the way the results are analyzed and interpreted, but a simulation with an invalid model will never give correct results, and when these results are the basis for decisions with major consequences (technological choices for a program costing hundreds of millions of euros, specifications of a bridge, military strategy, and so on), the human or financial impact can be drastic. Modeling is a complicated process and cannot be carried out without considerable effort. “Virtual” does not mean “immediate” and “easy to implement”, contrary to what certain Hollywood films and brochures from software editors would have us believe. A well-established process must be followed rigorously, as with any engineering activity.
2.3.1. Global process There are numerous methodological approaches to system modeling and we cannot go into detail about all of them here. Instead, we shall first examine a typical modeling and simulation process, which is relatively generic and universal, followed by a process which is oriented more towards distributed simulation. In most processes, we find the following steps in one form or another: 1. formulate the problem, 2. establish precise goals and organize the project, 3. carry out modeling and collect data, 4. code the model, 5. check the code and the data, 6. validate the model and the data, 7. execute the simulation, 8. go through and analyze results (critically), 9. produce final report, and 10. implement the application and/or capitalize the model.
68
Simulation & Modeling of Systems of Systems
We shall take a simple example and study its modeling through the different steps listed above. We shall take the role of a consultant approached by a client to resolve a problem. The client is the (fictional) Auclerc supermarket in Nantes. The director of the supermarket wishes to optimize the use of staff and increase client satisfaction by rationalizing the use of the store’s 10 checkouts.
2.3.2. Formulation of the problem The element which begins a modeling and a simulation process is, as with all engineering activities, the expression of a need by the client, usually in the form of a problem which he or she hopes to resolve in the case of a system engineering simulation. Here are some examples:
− How can I improve the performance of system X by 30%? − I have three possible architectures for my system X, which should I choose? From this initial step, we shall encounter a number of obstacles. For example, the client often has difficulty in expressing their need precisely: what does “improve performance by 30%” actually mean? What are the criteria that will allow us to compare different architectures? We need a means of measurement to obtain objective and easily exploitable results at the end of the simulation. Moreover, the client and the modeling and simulation specialist generally have different backgrounds. The client may be the director of a pharmaceutical factory wanting to optimize his production chain, and the engineer an employee of a service company with training in computer science, presumably not particularly well versed in the production of acetylsalicylic acid tablets. This sets the scene for dangerous levels of mutual incomprehension. To avoid this, a common vocabulary and perhaps a common taxonomy need to be agreed so that both partners have the same vision of the problem. To illustrate this risk, we shall look at the example given in Figure 2.5 where the modeling of a system involves “vehicle” objects. For the project manager, this clearly refers to cars. A doctor employed by the client thinks of an ambulance, the engineer thinks of a technical object capable of coping with certain technological challenges, and the salesperson, who used to travel long distances, thinks of a train. We should not, therefore, hesitate to provide precise definitions of key terms in the system being modeled, being careful to avoid any ambiguity. This makes the establishment of good communications between the client and the simulation provider crucial.
Principles of Modeling
69
In eliminating ambiguity, it is evidently of primary importance to define “system X”: what are its boundaries, what is part of the system, and what is excluded? This will modify, for example, the parameters which can be modified to optimize the system. In the case of the pharmaceutical factory cited above, is the system limited to the tablet production line? Is the packing chain included? Should we consider the logistics of delivery or the whole distribution circuit?
Figure 2.5. Ambiguity in the interpretation of “vehicle”
This step requires particular attention, as it is at the origin of the process and conditions all that follows. An error, a misunderstanding, or an ambiguity at this stage can render the rest of the process null and void. Without necessarily creating a common document establishing aims (for this, we might follow the FEDEP or DSEEP standard process, see IEEE 1516.3 and IEEE 1730 norms, see Chapter 7), the definition of a common taxonomy between those involved in the project is strongly recommended from the beginning. In our example, the system being considered is a supermarket. The client wants to improve performance following two axes: “optimize staff use” and “increase client satisfaction”. What does this mean? In the case of staff, does “optimizing” involves reducing the staff to the strict minimum, or finding a method to ensure that, with a given number of cashiers, none will be inactive? As for client satisfaction, is this linked to prices, to the decoration of the supermarket, or to the time spent instore? A discussion with the client shows that he would like to minimize the number of cashiers, while keeping the waiting time at the checkouts under 10 minutes, as, according to a recent study, long queues at checkouts are the main cause of client dissatisfaction. Work needs to be done to reformulate the expression of need in association with the client, to obtain an explicit set of precise, measurable requirements.
70
Simulation & Modeling of Systems of Systems
2.3.3. Objectives and organization The problem having been outlined and the system well defined, and communications having been established between the client and the engineer, we now need to define what we wish to obtain at the end of the project, that is, the goals we shall set ourselves: improvements in productivity, cost reduction, measurement of impact, and so on. These goals should have a number of classic qualities as follows:
− Measurable: we must be able to link them to a simulation output which can be evaluated and compared. “Increase the production of boxes of tablets Z by 10% with a maximum cost increase of 5%” is a measurable objective. “Increase the wellbeing of workers” is not, at least without defining an appropriate metric. − Precise: vague formulations are widespread but allow too much room for interpretation. “Improve productivity” is insufficient; the desired improvement must be defined, for example, “by 10%”. − Reasonable in terms of number, time, and ambition: objectives should be attainable with an effort proportional to the desired gains. “Improve production by 500% at no additional costs within three weeks” is not realistic; the challenge is too great in terms of the improvement in system performance and the time margin. Finally, the multiplication of objectives makes the execution and use of the simulation more complex and makes the effort useless. A compromise needs to be found; it is better to concentrate on a limited number of objectives. − Useful: as obvious as this may seem, none of us are immune from taking approaches of doubtful utility. This attribute also includes the fact of not making efforts disproportional to the intended benefits; investing several man-years in the simulation of a complex logistics system to save two delivery lorries is hardly reasonable. It is interesting to begin thinking about the scenario which will be used for the simulation at this stage, as this may shape the simulation somewhat. For example, a simulation including a dozen or so entities may not be designed in the same way as a simulation with a thousand entities; in the latter case, we need to consider available machine resources and, for example, reduce the precision of models or envisage a distribution of the application (without forgetting that the underlying models may then be different, see Chapter 5). In parallel, the project manager organizes and plans the work to be carried out. He or she determines the tasks required and evaluates the resources needed for each task (generally expressed in man-days or man-months). A calendar for the project is
Principles of Modeling
71
set up. So far, this approach is no different to that for any other project with a minimum amount of engineering input. Although this does not appear explicitly in the process, a preliminary study of available human and technical resources should be carried out at this stage. This is required to evaluate feasibility and cost. Do we have the necessary competences for the modeling, validation of models, and exploitation of results? Do the availabilities of experts match up with the plan for the project? If tests need to be carried out, will the necessary resources be available at the right time? It is also absolutely critical to carry out a study of existing knowledge on the subject: has the question already been dealt with? If so, are the results accessible? Are suitable models available which may be re-used, with or without modifications? If so, is the information needed for their validation available, particularly their domain of validity? We shall come back to this question of reusing existing resources, which is an important cost reduction and time-saving factor in any project (not just in simulation), but requires considerable attention, especially where the validity of models is concerned. In our example, the goal is easy to determine using the precisions provided by the director of Auclerc: we must find the minimum number of customers to open in relation to time to keep customer waiting times under 10 minutes. We could possibly request further clarifications, for example, how far exceeding this limit temporarily would be tolerated.
2.3.4. Analysis of the system We can only simulate what we understand. Before constructing a model, we must analyze the system to understand its constituent parts and characteristics. Very roughly, the bulk of modeling often consists of determining the state variables and laws of evolution of the system. Once again, there are many ways to approach the analysis of a system; here, we shall present a simplified version of a classic and widely accepted approach to facilitate understanding of the principles involved. The aim of system analysis is to develop something known as a system view. This view is highly dependent on the goal that we have in mind. For example, the border between a system and its environment may vary. The choice of a diagrambased language, for example, UML, for the representation of the system is highly recommended, as it not only guides the user in describing the system but also gives a certain coherence to the documentation of the different models which will be produced.
72
Simulation & Modeling of Systems of Systems
2.3.4.1. Static system analysis We shall begin by studying the system in a static state, that is, without considering its evolutions over time. Having already determined the perimeter of the system, we must also determine the following:
− input and output (information, objects, and so on), − the entities which make up the system (objects, resources, and so on), − relationships between entities (physical, cognitive, and so on), and − the system environment. The entities of a system are the objects or components by which the system is defined. For example, in a war game, these would be the pawns. Depending on the level of the simulation (tactical, operational), a pawn may be a regiment, a subgroup, or an individual infantry soldier. The size of the entities manipulated by the simulation determines the granularity of the simulation. These entities may be permanent or temporary, depending on whether or not their lifespan is equal to the time of execution of the simulation. For example, in the case of a supermarket queue, the server is a permanent entity (they exist from the beginning to the end of the simulation). Clients, on the other hand, are temporary entities, which appear according to need and disappear once the server has responded to their request. These entities are described by a certain amount of data, known as attributes or variables (if they are dynamic) or parameters (if they are static). For example, for an “airplane” entity, a parameter may be the maximum flight altitude, which is fixed and depends on the type of airplane being modeled; variables include the actual flight altitude, which is constantly being modified throughout the simulation. Note that variables or parameters can apply to the whole of the system; in this case, they are known as global parameters or variables. Time is a typical example of this, being common to all the entities in a simulation. This analysis can be carried out by successive decompositions of the system. Thus, a tank unit will have its variables, and can be broken down into individual tanks, each with its own variables. We can then break down each tank into subsystems: steering, motorization, cannon, and so on. The level of decomposition depends on the aim of the simulation, as it is not useful, and even counterproductive, to take the decomposition process too far. The environment of a system is the group of entities which are outside the system but which have an influence on the evolutions of the system. The border between the system and its environment is not always clear and is often subjective
Principles of Modeling
73
or even arbitrary. The precise identification of this boundary is, however, a pre-requisite for modeling. Take the example of an airplane in flight. If the system being modeled is the airplane itself, then a radar on the ground could be considered as part of the system environment (known, in this case, as the tactical environment). If, on the other hand, the radar is the system being modeled, the airplane will be part of the environment. Finally, if we were to simulate a theatre of aerial operations, then both the airplane and the radar would be included in the system. The identification of the boundary is subjective, but has implications concerning the goals of the simulation; the system is what will be parametered to optimize performance. The environment is determined by the scenario and does not come into the field of study (except by providing external perturbations acting on the system). A system environment includes, notably, the natural domain (terrain, atmosphere, ocean, space, and so on). Arrival (creation) forgotten article created
Shopping
end of shopping
c
checkout free
Wait at checkout checkout free Checkout payment
Departure (end cycle)
Figure 2.6. Lifecycle of a customer in a supermarket
In our example, the “supermarket” system is described by several parameters, for example, the opening hours and the maximum and minimum number of open checkouts. It only contains two types of entities: customers and checkouts. The checkouts are static entities, which are also resources: access to them requires a queue. The checkouts are all identical, so a “number” parameter may be given to each to distinguish them. Customers are temporary entities; the arrival of a customer creates a “customer” entity, which will be destroyed once it has been dealt with a checkout (i.e. when the customer leaves the supermarket). The possible states of each entity can be represented in the form of a state-transition diagram, as in Figure 2.6. We intentionally made the model a little more complex by introducing the possibility that a client, while waiting at the checkout, may become aware that
74
Simulation & Modeling of Systems of Systems
they have forgotten an article. Specification elements of this kind often appear in discussions with clients, who do not know how to express their need in an exhaustive and formalized manner. As our system is very simple, it does not have an environment. 2.3.4.2. Dynamic system analysis The dynamic analysis of a system essentially consists of determining the evolution of its state over time. To do this, the following elements must be characterized:
− state variables of the system and its component entities, − laws governing the evolution of state variables, − events, − laws governing the occurrence of events. The state or state vector of a system is made up of the minimum group of global variables and the attributes of its entities that allow its evolution, in the absence of unpredicted events, to be predicted. We shall see later that this definition may be extended to other elements of the simulation. The corresponding variables are known as state variables. Note that parameters are not state variables: the fact that an airplane has a flight ceiling of 40,000 feet gives no indication as to the state of the system. The speed and the altitude of the airplane, on the other hand, are state variables. The “number of kilometers traveled since take-off” variable, however, does not describe the state of the system, but is needed if, for example, we wish to optimize the autonomy of the airplane; this is a statistical variable. Statistical variables are not necessary when describing the state of the system, but are indispensable in achieving the objectives of the simulation; they are used to obtain the simulation output required to analyze the behavior of the system in relation to objectives. In our supermarket, each checkout has a Boolean state variable (using the two values “true” or “false”), indicating whether or not the checkout is open. The statistical variables must show the waiting time at the checkouts. Several strategies are possible. We could consider the waiting time of each customer and keep this value under 10 minutes, but this would be difficult in practice, as we are dealing with a stochastic system subject to chance; because of this, we will certainly encounter versions of the simulation where this condition would be impossible to meet, even with a large number of checkouts. It is, therefore, preferable to opt for a solution using statistical smoothing, for example by time period, or to carry out a
Principles of Modeling
75
test on the waiting time of each customer and count the failures. In doing this, we have reinterpreted the wishes of the client; we need the client to define the tolerated percentage of “unsatisfied” customers, as analysis of the system has shown that it is impossible to guarantee that no customer will wait more than 10 minutes. We could therefore have, as statistical variables, a counter of the total number of customers and another counter for unsatisfied customers. Once the state variables have been identified, we must identify the physical and behavioral laws governing their evolution (there are not necessarily as many laws as there are variables). If Xi(t) denotes the group of state variables n (with i ∈ {1, …, n}); S(t) (respectively, S (t + δ ) ), the system state at a given instant (respectively following), this boils down to find the p functions Fj so that S (t + δ ) is expressed as a function of Fj (t, Xi(t)), with i ∈ {1,…, n} and j ∈ {1,…, p}. The Fj functions are the laws of evolution of the system. They may be discrete (in this case, δ represents the stage in the simulation) or continuous. A priori, we will look for those representations of the functions Fj that are explicit, or solutions to differential equations (or to the differences, in the discrete case) and which must be integrated to obtain new explicit values for the state variables.
In any case, we must not forget to specify the domain of validity of each function, and check that the system will not leave this domain during a simulation. The intersection of the domains of validity of the laws of evolution (and of potential modeling hypotheses) forms the domain of validity of the model. As the determination of these laws of evolution falls into the technical domain of the system being studied (electromagnetism, thermodynamics, structural mechanics, and so on), we shall not go into any more detail concerning this step. These laws of evolution correspond to a certain invariability of behavior, but a system may be subject to sudden change. Anything that causes such a change in type of behavior is known as an event; examples include the starting of an airplane motor, the explosion of a missile near its target or a flat tire. In a simulation, an event may be triggered by various things. − A condition on the system state: for example, if the speed of a car detected by a speed camera is more than 7 km/h over the speed limit, then the car is flashed. − A time condition in the simulation: for example, a shopping center might open at 9 AM. − A random process: for example, when a gun is fired, there is a 1/500 chance that the charge will not explode.
76
Simulation & Modeling of Systems of Systems
− A stimulus from outside the system: for example, a command entry by the operator (in the case of an interactive simulation) or the arrival of a signal (in the case of a hardware-in-the-loop simulation). The Auclerc supermarket, as we have modeled it, includes a certain number of events. The opening of the supermarket is a determinist event, as it happens at a set time. The arrival of a customer is a random event, which can a priori be modeled by a stochastic process known as the Poisson process. The interval between customers follows a well-known random law, exponential distribution, allowing the phenomenon to be modeled. Using this law, we can randomly generate “customer arrival” events reproducing the real-world phenomenon.
2.3.5. Modeling Now we have defined the system, we can construct an abstraction of it: the model, or more precisely a model, as we would do well to remember that there is no one model which will be “the best” for every possible usage. The modeling of the system will be influenced by the context even more than the analysis is, considering the following factors. − The objective: this determines, for example, the precision required by the model. − Available resources: in a perfect model, the different steps of a sequential engineering process would follow on from each other in chronological order. This being the case, we should specify without pre-conceived ideas on the solution which will be chosen. In practice, however, we must consider the feasibility of the product, and for this, we need a good idea of what may be installed, and thus ideas for possible solutions (without necessarily making a choice at this early stage). It would also be useless to plan a model requiring large amounts of processing power if we do not have the required computing resources. In the same vein, it would be useless to begin modeling without being sure of being able to obtain the data needed to parameter the model. − Availability of reusable components: the existence of valid off-the-shelf models can encourage modelers to reorient their model to maximize re-use of these pre-existing models. A considerable number of works have been published which deal with system modeling, and the corresponding structures are numerous. Here, we shall give an example of a widespread approach to modeling. The object-oriented modeling approach is the most widespread, assisted by current information system design paradigms. It gives a representation of the real
Principles of Modeling
77
world, which is easier for the developer to conceptualize. The intuitive nature of the object-oriented approach for the abstraction of a system makes it a preferred method for modeling; this was, moreover, the reason for the appearance of the first objectoriented language, SIMULA, in the domain of simulation in the 1960s. The success of these techniques has led to large numbers of products, languages, and methods being available on the market. The type of structured data characteristic of object-oriented languages is class. A class is made up of attributes (variables describing the class) and methods (functions which can be applied to this class). The attributes of a class may not be visible from the outside, meaning they can only be manipulated using their methods. This process of grouping methods and data within the same structure is known as encapsulation. In object-oriented programming, objects are instances of a class. For example, we could define a class, Car, of which one instance might be a Peugeot 205. The instantiation relationship is the fact of belonging to a family of objects. Thus, we can say that a Peugeot 205 is a Car. In object-oriented languages, it is possible to derive new classes (derived classes) from an existing class (base or parent class). Each new class inherits methods and attributes from the base class, which it may replace or enrich. This mechanism, known as inheritance, is very powerful for code re-use, the primary objective of object techniques. It also allows classes to be abstracted by regrouping their common characteristics in a higher-level class. Thus, we can define a Vehicle class, from which we can derive new classes, such as Airplane, Car, and Boat. We thus obtain an inheritance tree, as illustrated in Figure 2.7.
Vehicle
Airplane
Car
Boat
Figure 2.7. Inheritance tree of the Vehicle class
If the Vehicle class has a Speed attribute, then all the derived classes will automatically have this attribute. The Airplane class, however, may have a supplementary attribute, Altitude, that the Vehicle class as a whole does not have.
78
Simulation & Modeling of Systems of Systems
We note that if our Peugeot 205 is a Car, then it is also a Vehicle, and so possesses the attributes and methods of this class. This capacity to take several forms (depending on context) is known as polymorphism. A Slow Down method from the Vehicle class, for example, can be applied to an Airplane as well as to a Boat. There are many other object-oriented programming techniques, but it is generally admitted that, in order for a language or a method to be considered as object oriented, the three mechanisms of encapsulation, inheritance, and polymorphism must exist. Object-based modeling is widely used in simulation as it makes the abstraction of the system being studied easier; the system is decomposed into families of objects (classes), each with characteristics and functionalities (i.e. attributes and methods), which interact (method calls). Moreover, a number of standardized modeling languages exist, of which the best known is UML [RUM 04]. An example of UML modeling will be given in Chapter 6.
2.3.6. Data collection 2.3.6.1. Destination of data When developing a simulation, it would be an error to focus solely on the conceptual modeling and the encoding of the model, as the validity of a simulation is not only based on the abstract model constructed but also on the data input. A simulation contains large amounts of data as follows. − System parameters, which give precise usage conditions for the model and allow it to be calibrated. For example, a generic airplane model could be parametered for an Airbus 340 or a Boeing 777. To do this, we modify system performance parameters (cruise speed, fuel quantities, consumption, and so on). This also includes the parameters of laws for stochastic processes, which follow random events in the life of the system. The statistical analysis of measurements in a process allows a law of probability to be obtained, which can then be used to model it. − Input data (and the scenario), which gives initial values for state variables. − Validation data is used to calibrate and validate the model. − Output data, including not only the raw results of the simulation, but also execution parameters (random seed, date, host system characteristics, and so on), saved data, statistical data, execution journals, replay file, and so on.
Principles of Modeling
Tests, experiments
Other simulations
79
Expertise
Modeling
Parameters
Simulation
Data input
Validation Verification
Results
Tests, experiments
Use, analysis
Other simulations Expertise
Figure 2.8. Role of data in simulation
This data is present at every step of the modeling and simulation process, as shown in Figure 2.8. Its use is an activity that must be planned in the same way as other parts of the process, as it can be, in certain (and relatively frequent) cases, the principal difficulty to overcome. 2.3.6.2. Data sources The collection of a sufficient quantity of data of sufficient quality for a simulation is one of the hardest tasks in the modeling and simulation process. In certain cases, data collection may even be impossible (e.g. for a physical phenomenon which cannot be reproduced on Earth) or otherwise unadvisable (too dangerous, too expensive, or concerning industrial or military secrets). As an example, we need to only look at the resumption of French nuclear tests in the Pacific in 1995. On top of the financial cost of the six tests carried out on the islands
80
Simulation & Modeling of Systems of Systems
of Moruroa and Fangataufa, President Chirac’s decision had a considerable political cost for France in Oceania and in the rest of the world. The aim of this considerable investment, besides the qualification of the new TN75 nuclear warhead, was to obtain the data necessary for the construction, calibration, and validation of simulations, and thus, with sufficiently valid models, avoid further real-world or full-scale tests. The CEA’s simulation program, however, does not just include digital methods; it also includes testing hardware such as Airix (a powerful X-ray radiography system) and the Mégajoule laser, which enable nuclear reactions to be studied at a very small scale. It would thus be false to say that the French nuclear weapons development program is now entirely digital, contrary to what the press might made us believe. We thus see the efforts, which can be necessary to provide data for simulations. Where, then, can we find data to parameter, calibrate, and validate a model? Possible data sources are plentiful, but each has its limitations. These sources can be grouped into three major categories: − Experimental data from tests or experiments carried out on the real system or a model: this data has a good level of reliability, but generally only applies to a small part of the domain of validity, which the model aims to cover. Extrapolation is often needed, a practice which is not without its risks. − Data from technical expertise: this includes not only engineering expertise but also technical documentation on the system being modeled (if such documentation exists). − Results from other simulations with a known degree of validity: a typical case would be the use of data from fine simulations to calibrate and validate performance models (e.g. a fine simulation of a homing head can provide information on the behavior of a missile, and allows the construction of technico-operational simulations of a missile–airplane combat). In cases where the system does not yet exist, typically when modeling future systems, which is often the case in systems engineering, knowledge of preceding systems of the same type is used as a base (previous models, systems developed by competitors, or simply systems with similar functionality), alongside scientific theory, tests on real models and/or, once again, finer simulations. Although this is a delicate exercise, it is possible, using a high level of expertise and rigor, to construct a valid model of a system, which does not yet exist as a physical reality. This data collection may require a specific toolset and possibly an infrastructure. For example, imagine we want to model road traffic in a modern city. For this, we need to collect information on real traffic, either from specialized bodies or by collecting data ourselves on the ground. This data collection needs to take place not
Principles of Modeling
81
only throughout a whole day but also on different days of the week, in different weather conditions, at different times of year (climate conditions, school holidays, road works, transport strikes, and so on have an impact). We also need to be able to integrate the impact of factors such as anti-pollution measures, increases in fuel prices, and so on. At the end of the day, thousands, or even millions, of data entries need to be collected and analyzed to find statistical laws that govern traffic circulation. To give another example, Météo France uses a group of satellites, a network of over 1,200 stations, weather balloons, and radars to supply its “ARPEGE” (research project on small and large scales (action de recherche petite échelle grande échelle)) weather forecasting model with data. The data collection process does not end when data is obtained. The data collected may not exactly correspond to the specific requirements of the model in question, and may require certain modifications. Moreover, the validity of data must also be established. 2.3.6.3. Data verification and validation As with models, the data used for a simulation must undergo a verification and validation (V&V) process. The best simulation in the world will not give valid results if the input data contains errors. Chapter 3 is dedicated to the V&V question, so we shall not go any deeper into this subject now. Nevertheless, certain points concerning data demand particular attention as they are frequent sources of error: − Identification and traceability: do we have a list of data needed for the simulation and the desired objectives? Is the origin of the data documented? Where does the data come from? How fresh is it? Has it already been validated? − Availability: what possible sources of data do we have? Does data already exist in a validated form? If this is not the case, can data of sufficient quality be obtained within the required timeframe? − Reliability: how was the data obtained? What is the risk of recording errors? If the data has been obtained from tests, were the test conditions suitable to provide representative data for the domain of use of the system? What degree of error is there in the measurements? Do random phenomena come into play? If so, are the results statistically useable? − Suitability and validity: does the data allow the model to be calibrated and parametered? Specifically, is the data sufficiently precise for the desired objective? − Interpretation: in what conditions can the data be used? What version of the system does it describe? What units are used?
82
Simulation & Modeling of Systems of Systems
The question of units of measurement deserves special attention, as when several simulations of diverse origins are mixed in a system-of-systems-type simulation, this problem frequently rears its head. Sailors have a tendency to measure in knots and not in km/h, aviators in feet rather than meters, and so on. A famous example in the field of systems engineering is the American Mars Climate Orbiter probe, one of two probes used in the Mars Surveyor program, which was supposed to contribute to further understanding of the Martian climate. On September 23, 1999, the mission was cut short: the system was supposed to enter orbit at an altitude of 140 km, but it began to fly around the planet at 57 km, provoking its entry into the planet’s atmosphere and the subsequent destruction of the 900-million-dollar system. Despite a NASA directive demanding the use of metric units, one of the manufacturers had used Imperial units for acceleration (pounds instead of Newton, where 1lb = 4.48 N). A good principle to follow is to systematically store data using the metric units set out in the MKSA system (International System of Units defined by meter, kilogram, second and ampere), specifying the unit used for each value. For example, using an XML file, we can use a unit attribute to remove any ambiguity:
...
10000
As we can see, data requires as much attention as the model in terms of V&V. Besides the basic precautions mentioned above, data V&V techniques re-use part of the V&V techniques used for models, particularly audits and reviews by experts, comparison with technical documents describing the system, the use of data validity tests (assertions) during the installation of the model on a computer, or simply the use of a graphical restitution of system behavior and performance based on this data, making it easier to spot potential errors visually.
2.3.7. Coding/implementation We wish to obtain a digital simulation, that is, a simulation application executed on a computer, called a program. An implementation stage is, therefore, required, during which the abstract model will become a series of instructions in a programming language. This step must follow the same quality control principles used for any program, applied appropriately according to the criticality of the simulation.
Principles of Modeling
83
2.3.7.1. Choice of architecture The choice of architecture has a significant impact on the quality and performance of the application, on its usability, and on its capacity for re-use. This choice should not be left to chance or to the taste of the person responsible for the installation, but must consider the objective technical criteria, with economic and strategic considerations. At this stage, we already know what broad class of architecture (constructive, hardware-in-the-loop, virtual, and so on) the simulation will belong to. The first question to ask is therefore “what simulation environment shall we use”? The whole simulation environment does not need to be encoded from scratch. A number of simulation environments are now available which deal with a large chunk of the work, while offering a level of operational richness well above that provided by any new, specific development. We shall go into more detail concerning these environments in Chapter 6. The choice of the environment will also impose a choice of language (Fortran, C++, Java, Matlab, and DSL) and operating system (Windows, Linux, and so on), removing the need to enter into the eternal debate between partisans of a particular language or of a specific operating system. Arguments of this variety are not particularly significant nowadays and bear more resemblance to religious quarrels than to discussions between experts. Instead, it is important to bear in mind the initial requirement, the resources available, and the policy of the business on simulation use, if such a policy exists. We might, on the other hand, need to consider the internal architecture of the application, if the simulation environment does not impose any: − Monolithic: the code is in a single block on a single machine. In theory, this is the architecture that gives the best performance, but it is complex and provides a reduced capacity to evolve. − Parallel: the code is designed to be executable on several processors and/or cores within a processor. This has a major impact on the choice of processing algorithms, as in this case platforms developed for high-performance calculation are involved (calculation servers, massive parallel processors, clusters, and so on). Sections of code (e.g. calculation loops) can thus be divided across several platforms, but these sections cannot be dissociated from the application. − Distribution: in this case, the application is decomposed into several autonomous “sub-applications”, each dealing with an homogeneous sub-group of calculations. Each of these sub-applications can be executed on a different machine and works with the others using a communications infrastructure. This renders the execution more complex and can pose a number of technical problems, but the method presents certain advantages, notably the capacity to re-use existing resources or applications and thus save time and cut costs during the development process.
84
Simulation & Modeling of Systems of Systems
Moreover, these sub-applications can themselves be re-used later in a different context as each has a certain level of autonomy. Distributed simulation architecture is becoming increasingly widespread for simulations of complex systems and of systems of systems. 2.3.7.2. Non-transparency of material There is a current trend among software engineers to assert that design should be independent of implementation. This is clearly a solid principle, but difficult to maintain in reality. In fact, the target platform on which the simulation is to be installed may have a significant impact on the model, notably when using a particular platform, which imposes strong programming constraints, as with hardware-in-the-loop or parallel systems. This is frequently the case, as supercomputers, of which most are used for simulation, are all multi-processor machines, some with thousands of processors. As the cost of supercomputers is as high as their processing power, computing grids are another popular solution; in this case, the power of several (up to several thousand) geographically widespread computers is brought into play. This architecture provides considerable capacity at a reduced cost. Thus, the SETI@homeprogram is installed in the same way as a screensaver, and uses the spare processing time of simple personal computers to analyze signals captured by the Arecibo radio telescope in the hope of detecting extra-terrestrial intelligence. In situations where parallel and distributed simulation architectures are used, the encoding of the model must follow specific rules to reach a certain level of effectiveness and efficiency. In particular, it must be possible to install the model in such a way as to divide calculations across several processing units. In cases where a continuum is simulated, such as the flow of air around a turbine blade or the deformation of a structure during a crash, this is relatively easy, as the model consists of series of differential equations solved by matrix calculus, using procedures that can be easily parallelized. We obtain quite good results automatically using parallelizing Fortran compilers developed for architectures. In the case of logical or stochastic models, however, the problem is more complex and, if large amounts of processing power are needed for the execution, requires a suitably adapted mode to avoid total or partial re-engineering at the moment of installation. In passing, we highlight once again the necessity of adapting the level of modeling to our needs: the higher the precision, the stronger the constraints in terms of processing power will be, and the cost will consequently be higher. Another typical case where installation can have an effect on modeling is that of real-time simulation, either because hardware is involved (hardware-in-the-loop) or because the simulation involves a real operator (piloted simulation). The real-time treatment constraint on the simulation program demands the model be realistic in
Principles of Modeling
85
terms of processing time; it would be useless to develop a sophisticated flight model for an airplane if we know that the computer will not be able to execute it in real time. A good example of the involvement of installation in the design of a model can be found in distributed simulation of synthetic environments. Let us imagine a joint combat simulation involving naval, terrestrial, and aerial units. For reasons of infrastructure and re-use of existing simulations, a choice of installation is imposed in the form of four simulations: one for environment (land, air, and sea) and one for communications, each simulation being mastered by a team in a different center of competences. As each environment is mastered by a different center, it is simple and natural to install each simulation in the center with the relevant expertise, and to carry out the global simulation as a federation of simulations, distributed following the high-level architecture (HLA) standard (we shall return to this in Chapter 7). However, looking more closely at the situation, we see that this choice is not necessarily ideal: we need to consider the interactions between the objects being manipulated in the simulations. Suppose, then, that the Navy lands units to provide fire support, and comes under attack from aircraft. The interaction with other simulations is limited and so we can also expect exchanges to be limited. The fact that the simulation in question is carried out at a distant site should not cause performance problems. However, terrestrial units need to communicate with each other intensively and will make constant demands on the federated simulation which deals with communications. If this simulation is located at a distant site, this can considerably reduce the performances of the terrestrial simulation to the point where it becomes unusable; it would, therefore, be much better for at least part of the communications to be generated directly by the terrestrial simulation, and we should perhaps reconsider the modeling of terrestrial units to make intercommunication models more efficient. 2.3.7.3. Program quality Simulations are programs, and as such must respect certain rules of software engineering. To fully understand what is at stake, we shall provide a few figures. A study carried out by the Standish Group in 1995 on 8,380 computer applications in 365 businesses reported that only 16.2% of these projects conformed to the initial plans and that more than half of these projects (52.7%) had overrun in terms of cost and/or time by a factor of 2 or more. On average, those projects delivered had only 42% of the functionality initially requested. In principle, of course, a simulation does not put human lives or major financial investment at risk; however, we should not forget that it can have a considerable impact on the choice of architecture for a program involving investments which can run into billions of euros. In the case of a training simulation, a faulty simulation
86
Simulation & Modeling of Systems of Systems
may provide “negative training”, that is, lead to the user acquiring bad reflexes during training, a fact which may, for a pilot, have fatal consequences in reality. Finally, certain simulations involve far from negligible investments in terms of money and resources; thus, the American JSIMS defense simulation swallowed up almost $800 million in investments before it failed.3 The design of a simulation program must not only be integrated in the simulation engineering process but also follow its own software engineering process, to a degree dependent on the objectives, resources, and criticality of the simulation. For a simulation, the following aspects demand special attention: − Verifiability: the code must be set up so as to allow us to understand, by examining the variables used during the execution of the simulation, how specific results were obtained. − Efficiency: in the case of simulations requiring considerable processing power or with real-time constraints, a certain level of optimization of code is required. It is better to plan this before coding by relevant choices of algorithms and technical architecture, as a posteriori optimization is often fiddly and can lead to various errors. − Reusability: the ability to re-use developed and validated code allows substantial savings to be made in terms of time and resources, on the condition that we check that the simulation remains valid in the new context of use each time. − Durability: if we want the simulation to be able to accompany a system throughout its life cycle, the life expectancy of the simulation must be suitably long. In cases where the desired lifespan is particularly long (10 years or more), precautions must be taken so that the simulation will not be affected by changes in technical platforms. A typical example of disregard for these considerations would be a very complex simulation designed, with no consideration for portability, for a proprietary UNIX operating system. The developer decided to change to another version of UNIX, and the simulation in question was no longer able to operate except on obsolete workstations, placing strains on its performance and potential longevity. A quote for the transfer of the simulation to the new system proved prohibitive, and the simulation was simply abandoned in spite of the 10 years of effort that went into developing it. There are many good practices and methods, which can be used to ensure the quality of a simulation program, and it would be difficult to go into depth on each 3 The reasons for this resounding failure in the domain of simulation are multiple. As principal causes, we would pinpoint a very, possibly too, ambitious technical and technological challenge coupled with organizational difficulties between the various people involved in the project.
Principles of Modeling
87
one here. Nevertheless, to give just one, we would repeat that developments based on a suitable simulation environment should be preferred. Thus, at the DGA, a common technical infrastructure for simulation [KAM 02] has been developed to facilitate the task of simulation designers, to obtain functionally rich, interoperable, and reusable simulations at reduced cost.
2.3.8. Verification Verification is the process by which we ensure that the installation of a model conforms to its specifications. In other words, we must prove that the product satisfies the initial demands. This verification process is similar to that used for any computer code. It uses, in particular, techniques borrowed from software engineering.
2.3.9. Validation Validation is a process that aims to ensure that a model or a simulation represents the real world in a sufficiently precise manner for the use to which it will be put. In other words, the validation process determines how adequate the product is for the needs of the user, and measures the trust that can be accorded to the results of the simulation. It also indicates the conditions under which the simulation can be used while remaining valid, that is, the domain of validity. This fundamental notion is described in Chapter 3. Note that V&V are not to be confused: − Verification aims to verify that the problem posed has been resolved following the required rules, for example, code quality rules: did we make a good product? − Validation aims to check that the product responds well to the problem posed: did we make the right product?
2.3.10. Execution Now it is the time for the execution of the simulation. Obviously, the execution of a simulation varies according to the type of model. In the case of a closed-loop digital simulation, the application reads the data from the model as input, carries out calculations for a certain time (or until the end of the phenomenon or observed process in the case of a finite horizon model) and produces result files without human intervention. Execution may be carried out in batch mode, that is, automatically, without graphical visualization or the possibility of human
88
Simulation & Modeling of Systems of Systems
intervention. This model, however, presents the disadvantage of being difficult to follow, which may be a problem if the user is not yet completely sure of the reliability of the application. For this reason, even closed-loop digital simulations often have a graphical mode of execution offering the possibility of controlling the progress of the simulation (including pausing it) and examining, or even modifying, certain variables. In the case of a distributed simulation (Chapter 7), this monitoring interface is particularly important due to the complexity of execution, and may be the object of a separate application (a federation supervisor). One advantage of batch mode, apart from the greater execution speed available in the absence of a graphics interface, is that it allows multiple executions of the same model. It is typically the mode chosen for studying a stochastic model using Monte-Carlo techniques: as each execution of the simulation (known as a replica) gives a different result due to the presence of random processes, a large number of replicas is needed to carry out a statistical analysis which will provide values for average behavior. The Monte-Carlo technique, developed in the 1940s for operational research, allows us to determine (among other things) how many replicas will be required to obtain a certain level of reliability from results, or to evaluate the precision obtained when using a certain number of replicas. Our example of the supermarket is a typical case of a simulation of a stochastic system using discrete events, and lends itself particularly well to this type of analysis. After the execution of a simulation, the results may not correspond to what was expected. This can be the result of various factors. In some cases, the results may reflect a low probability, and be an extreme case: after all, it is possible to win the Lottery, even if the chances of this happening are very small. These extreme results can either be taken into consideration or left to one side to avoid them artificially altering the statistics. Sometimes, however, results may simply be erroneous. In this case, the simulation must be debugged by replaying the replica in a mode that allows interactive control, that is, in graphical mode. The ability to replay a simulation is important in such cases. In the case of a stochastic simulation, we must be able to reproduce the sequence of random events identically; in other words, the stochastic model must be determinist. This apparent paradox can be resolved by the use of a pseudorandom number generator to simulate the effects of chance. In computer science, chance is effectively modeled by a mathematical function generating numbers which appear to be random, but in reality form a recurring sequence of the type un+1 = f(un). Knowledge of the first term u0 of this sequence, called the seed, allows us to find all the other terms of the sequence. A stochastic simulation used as part of a study should ideally have this capacity to reproduce identical replicas given the corresponding seed. Of course, if external elements not controlled by the simulation (user input, other simulations, involvement of a random factor such as unreliable data transmission by a network) are involved in the calculation of results, it is clear that this reproducibility will not be attained.
Principles of Modeling
89
Contrary to closed-loop digital simulations, interactive simulations consider user input. In the extreme case of piloted simulations, the results of user input (action of simulator controls) are turned into output (graphical visualization, modification of instrument indications, mechanical action on the cabin if the simulator is mobile, and so on), with a time delay which constitutes one of the most delicate points of a real-time piloted simulator; any time lapse between the action of the user and the reaction of the display can cause perturbations in the brain and provoke nausea, known as “simulator sickness,” encountered by some, even very experienced, pilots at the controls of a fixed simulator. This real-time restitution of the behavior of the system does not prevent statistical data on the execution to be saved for use in later debriefing.
2.3.11. Use of results Setting apart the case of piloted simulations, the result of a simulation is often a large quantity, sometimes whole gigabytes, of data requiring interpretation and analysis. Various statistical techniques (including Monte-Carlo techniques) and specialized computer programs exist for the analysis of this data. Analysis must be carried out with the original problem in mind. Typically, the analyst attempts to find out how the parameters of the system might be optimized to find, through the simulation process, the “best” values for these parameters. In the case of the supermarket, we might vary the number of cashiers over different time blocks and observe the impact on the waiting time of customers at checkouts, and thus find the minimum values that allow our objectives to be obtained in terms of waiting time and with the desired confidence interval. The presence of stochastic processes does, of course, complicate affairs: for each value of the “number of checkouts open” parameter, several replicas must be executed before carrying out statistical analysis, and the data for each time block must be separated. This rapidly becomes tiresome, and this is a very simple – simplistic, even – case with only one variable parameter. In “real” studies, a whole group of parameters are often varied. We thus require a method to find the most relevant values and avoid unproductive multiplication of executions, which is costly in terms of execution time and the subsequent treatment of data. One example of rationalization of parameter variations is a sensitivity analysis of the model. This operation aims to determine the following: − the input parameters for the simulation which produce the highest variations in output, − possible instabilities (a small variation of one parameter which causes strong variation in results),
90
Simulation & Modeling of Systems of Systems
− parameters with no significant influence on results, − the impact of uncertainty on a parameter (e.g. if some data is missing), − possible influence of one parameter on another (non-independence). This sensitivity analysis also provides indications as to the validity of the model. It is important to think about this rationalization as it allows simulation output to be simplified and made more reliable. Data produced by a simulation can be very difficult to use. In the case of a closed-loop digital simulation of physical phenomena (typically continuum modeling, such as airflow around the wing of an airplane), thousands of state variables are involved which change with every step in time, producing billions of data entries requiring analysis. This can lead to simulations being used to study the results of simulations, by resituating the data output in a graphic and dynamic form, which is easier for a human user to interpret.
2.3.12. Final report The interpretation and analysis of results is a highly technical activity. The final report, given to the client, should express the conclusions supported by the simulation in trade terms (i.e. terms which relate to the domain of expertise of the user of the results, not to that of the simulation operator), that is, the elements that respond to the problem posed by the client.
2.3.13. Commissioning/capitalization As we saw in Chapter 1, simulation is commonly used at different stages of the lifecycle of a system, for different purposes, including analysis of requirements, specifications, qualification, training, and the study of developments. Figure 2.9 summarizes the implications of simulation throughout the system acquisition cycle. We note that, in practice, different teams deal with each step of the cycle, and simulations are rarely re-used from one step to another. Of course, the aims of these simulations will not be the same, but models may be very similar and potentially reusable, either directly or by using their results to validate other models (e.g. a fine model may be used to validate a higher-level model). In the 1990s, a new “doctrine” of global use of simulation throughout the acquisition cycle emerged, with the developed or modified models being capitalized as they were validated: simulation for acquisition (SBA, simulation-based acquisition, in the US and SEBA, synthetic environment-based acquisition in the UK). The aim is not only to optimize the cost of simulations but also to offer,
Principles of Modeling
91
through this process and thanks to distributed simulation standards such as HLA, the capacity to construct virtual prototypes of the system, used in synthetic environments which are gradually enriched over time.
Technicooperational simulations
Digital simulations for study
Quick prototypes (IHM etc.)
Hybrid simulation
Training Decision aid
Figure 2.9. Simulation in the system acquisition cycle
2.4. Simulation project management A simulation project cannot be carried out in exactly the same way as the development project with modeling, as it includes a certain number of specificities that should be considered, such as the verification-validation-accreditation (VV&A) process, the availability of data, or the analysis of results. More than just a “classic” informatics project, a simulation project requires the participation of a multidisciplinary team with a high level of expertise and strong internal communications. Consequently, the manager of a simulation project must be able to pilot a structure including high-level engineers, often with very different cultural backgrounds. He or she must be very open, with a basic knowledge of each team member’s field of expertise and training in simulation techniques, and not just those relating to his or her own background. Those responsible for simulations often have a background that is too specialized, in terms of technology or architecture, means that they may only envisage solutions within their field of expertise. The project manager must, therefore, be well aware of the different ways of developing and using a simulation (reading Chapter 1 of this book would be a good start!) and know what exists and might be reusable (commercial off-the-shelf products, or resources
92
Simulation & Modeling of Systems of Systems
already available within the company), in addition to the processes and standards to apply, to successfully develop and/or validate the most relevant architecture possible for the simulation in terms of costs, time delay, efficiency, validity, and so on. The project manager may even need to identify means other than simulation to respond to the problem posed, for example, if they notice that − Creative meetings between experts may provide results as good as those of a simulation at lesser cost (Chapters 1 and 8). − Common sense may provide a simple response to the question; as strange as this may seem, it does happen. − The cost of the simulation may be more than the expected benefits would justify. − It may not be possible to access the necessary resources (finances, technical resources, personnel, and expertise) to successfully complete the project. − Data may not be available within the timeframe and cost limitations set out by the project. − It will not always be possible to validate models, and so the credibility of results cannot be evaluated (this may not be a major problem, for example, in the case of a video game). − The model will definitely be too complex to construct, or the corresponding modeling techniques are not sufficiently well established (e.g. modeling complex human behaviors, a dynamic natural environment in a distributed simulation, or a structural model with too many links and variables). The fact that simulation is not an all-powerful universal tool capable of resolving all the problems of the universe may shock certain fundamentalists, but a person possessing true expertise in a domain should be aware of, and recognize, its limits. Ignoring these limits and knowingly continuing down the path of error not only shows a lack of professionalism, but is also harmful as every failure contributes to discrediting the domain in question. The project manager must also be able to convince his or her hierarchy of the well-founded nature of their actions, particularly in respecting the simulation processes described in this chapter, or any other equivalent process; it may be tempting to attempt to please hierarchical superiors by cutting corners on certain less visible tasks or those that the client will not understand, for example, validation, to save time and money. Unfortunately, such corner-cutting usually leads to a reduction in the quality of the product and the final results and is detrimental to the interests of the client. We have already observed the consequences of using an invalid model, and we shall return to the subject later.
Principles of Modeling
93
The manager of a simulation project must also be able to see further than the end of the project. Might the simulation be re-used in another project? If this is the case, how can it be capitalized most efficiently? Will the simulation need to be maintained or further developed? For how long? By whom? [BAN 98] gives a description of the activities of the project manager at every step of the modeling and simulation process. We note, nevertheless, that if the manager has a certain level of knowledge of simulation techniques and issues, and consider the formal process used in modeling and simulation, software engineering, and VV&A, most of these activities will be dealt naturally. 1
2
3
4
5
6
7
Define federation objectives
Perform conceptual analysis
Design federation
Develop federation
Plan, integrate and test federation
Execute federation and prepare output
Analyze data and evaluate result
Figure 2.10. IEEE 1516.3 process
Extra steps and activities are added in the case of a distributed simulation project, such as the identification of federated simulations or their integration within the federation. The best practice guide IEEE 1516.3, Federation Development and Execution Process (FEDEP), thus describes a simulation engineering and execution process in seven steps (Figure 2.10): 1. Definition of the objectives of the federation: the requirements of the client are analyzed and the aims of the federation are developed and documented. 2. Conceptual analysis: the global scenario is chosen and a conceptual model, independent of implementation, is developed. 3. Design of the federation: candidates for federation are identified and selected according to the processes and entities to be simulated. 4. Development of the federation: a document, the federation agreement, is produced which identifies the role of each federated part and the exchanges between participating systems, which will be required. These exchanges are documented in the federation object model (FOM). The modifications required for existing systems and the new systems to be developed are identified and the corresponding work is carried out. 5. Planning, integration, and testing of the federation: federated systems are tested and integrated. The final execution of the federation is planned (this may
94
Simulation & Modeling of Systems of Systems
require significant resources, in terms of personnel, networks, installation of gateways, and so on). 6. Execution of the federation and preparation of results: the global federation is executed and the data produced is saved for later use. 7. Data analysis and evaluation of results: data is used and an evaluation of the federation’s success in terms of attainment of objectives is carried out. The first version of the IEEE 1516.3 process was published in 2003, but it was derived from FEDEP HLA, a document linked to the US Department of Defense’s HLA 1.3 standard (later replaced by the international IEEE 1516 standard) and the Synthetic Environment Development and Exploitation Process (SEDEP), a product of the European EUCLID RTP 11.13 project, which involved 13 member nations of the western European Union. A new version of this standard, with a wider field of application, should be published in the near future under the name of Distributed Simulation Engineering and Execution Process (DSEEP). The specificities linked to the use of distributed simulation and international normative references will be discussed in Chapter 7.
2.5. Conclusion We see, then, that modeling and simulation for systems engineering must itself follow an engineering process, derived from systems and software engineering practices, but with modifications necessary to consider specific activities related to simulation, particularly VV&A, data, and the exploitation of results, each of which constitutes a major and time-consuming activity which is often underestimated. These three activities are crucial as they determine the quality of the response provided to the initial question. Simulation is a powerful tool for assisting decisions and implementing systems if it is correctly used, based on suitable competences and management. If not, it can be illusory, or even misleading.
2.6. Bibliography [ADE 00] ADELANTADO M., BEL G., SIRON P., Cours de modélisation et simulation de systèmes complexes, Société des Amis de Sup’Aero (SAE), Toulouse, 2000. [BAN 04] BANK J., CARSON J., NELSON B.L., NICOL D., Discrete-Event Simulation, 4th edition, Prentice Hall, Old Tappan, 2004. [BAN 98] BANKS J., Handbook of Simulation, Wiley-Interscience, Malden, 1998.
Principles of Modeling
95
[BNC 95] BANKS J., NELSON B., CARSON J., Discret-Event System Simulation, Prentice Hall, Old Tappan, 1995. [BOU 02] BOUCHÉ J.-P., Rapport sur la typologie des modèles de comportements humains, Technical Report, Délégation générale pour l’armement, Paris, 2002. [BRA 00] BRAIN C., “Lies, damn lies and computer simulation”, Proc. 14th European Simulation Multiconference on Simulation and Modelling, Ghent, 2000. [BRA 87] BRATLEY P., FOX B.L., SCHRAGE L.E., A Guide to Simulation, Springer-Verlag, New York, 1987. [CAM 06] CAMPBELL S.L., CHANCELIER J.-P., NIKOUKHAH R., Modeling and Simulation in Scilab/Scicos, Springer-Verlag, New York, 2006. [CAN 01] CANTOT P., Cours sur les techniques et technologies de modélisation et simulation, ENSIETA, Brest, 2002. [CAN 08] CANTOT P., Introduction aux techniques et technologies de modélisation et simulation, ENSIETA, Brest, 2008. [CEA 06] COMMISSARIAT release, January 2006.
À L’ÉNERGIE ATOMIQUE,
Le supercalculateur TERA-10, press
[CHE 02] CHEDMAIL P., CAO et simulation mécanique, Hermès-Lavoisier, Paris, 2002. [DAH 99] DAHMANN J., KUHL F., WEATHERLY R., Creating Computer Simulation Systems, Prentice Hall, Old Tappan, 1999. [DGA 94] DÉLÉGATION GÉNÉRALE POUR L’ARMEMENT, L’armement numéro 45, special edition on simulation. ADDIM publication, Paris, December 1994. [DGA 97] DÉLÉGATION GÉNÉRALE POUR L’ARMEMENT, Guide dictionnaire de la simulation de défense, Paris, April 1997. [DOC 93] DOCKERY J.T., WOODCOCK A.E.R., The Military Landscape: Mathematical Models of Combat, Woodhead Publishing Ltd, Cambridge, 1993. [DOD 94] US DEPARTMENT Washington D.C., 1994.
OF
DEFENSE, Directive 5000.59-M: DoD M&S Glossary, DoD,
[DOD 96] DEPARTMENT OF DEFENSE, Instruction 5000.61: U.S. modeling and simulation verification, validation and accreditation, Under Secretary of Defense for Acquisition and Technology, Washington D.C., 2003. [DOD 97] DEPARTMENT OF DEFENSE, DoD Modeling and Simulation Glossary, Under Secretary of Defense for Acquisition and Technology, Washington D.C., 1997. [EZR 99] EZRAN M., MORISIO M., TULLY C., Réutilisation logicielle: guide pratique et retours d’expérience, Eyrolles, Paris, 1999. [FAU 87] FAURE R., KAUFMANN A., Invitation à la recherche opérationnelle, Dunod, Paris, 1987.
96
Simulation & Modeling of Systems of Systems
[FIS 73] FISHMAN G.S., Concepts and Methods in Discrete Event Digital Simulation, John Wiley & Sons, Hoboken, 1973. [FIS 96] FISHMAN G.S., Monte Carlo: Concepts, Algorithms, and Applications, SpringerVerlag, New York, 1996. [FOU 87] FOURASTIÉ J., LASLIER J.-F., Probabilités et statistiques, Dunod, Paris, 1987. [GIB 74] GIBBS G.I., Handbook of Games and Simulation Exercises, Routledge, Florence, USA, 1974. [GIR 98] GIRARDOT D., Guide de développement pour les simulations, Technical Report, Centre d’analyse de défense, Arcueil, 1998. [HIL 93] HILL D., Analyse orientée objets et modélisation par simulation, Addison-Wesley France, Paris, 1993. [IEE 03] IEEE Std 1516.3-2003TM, IEEE Recommended Practice for High Level Architecture (HLA) Federation Development and Execution Process (FEDEP), April 2003. [IST 98] INSTITUTE FOR SIMULATION AND TRAINING, Standard 610.3:Modeling and Simulation (M&S) Terminology, IEEE, Orlando, 1998. [KAM 02] KAM L., LECINQ X., LUZEAUX D., CANTOT P., “ITCS: The technical M&S infrastructure for supporting the simulation-based acquisition process”, NATO Modeling and Simulation Group Conference, Paris, 2002. [KAM 10] KAM L., “Model-driven design and simulation”, Systems of Systems, ISTE, London, John Wiley & Sons, New York, 2010. [KEL 99] KELTON D.W., LAW A.M., Simulation Modeling and Analysis, McGraw-Hill, New York, 2000. [LAW 06] LAW A., Simulation Modeling and Analysis, McGraw-Hill, New York, 2006. [LEM 95] LE MOIGNE J.L., Modélisation des systèmes complexes, Dunod, Paris, 1995. [LUZ 03] LUZEAUX D., “Cost-efficiency of simulation-based acquisition”, Proc. SPIE Aerosense ‘03, Conference on Simulation, Orlando, USA, April 2003. [LUZ 03] LUZEAUX D., “La complémentarité simulation-essais pour l’acquisition de systèmes complexes”, Revue de l’Electricité et de l’Electronique, vol. 6, June 2003. [MAI 02] MAIER M.W., RECHTIN E., The Art of System Architecturing, CRC Press, Oxon, 2002. [MON 96] MONSEF Y., Modélisation et simulation des systèmes complexes, Lavoisier, Paris, 1996. [PER 90] PERLA P., The Art of Wargaming, Naval Institute Press, Annapolis, 1990. [POO 92] POOCH U.W., WALL J.A., Discrete Event Simulation, CRC Press, Oxon, 1992. [PRI 98] PRINZ J., Puissance et limites des systèmes informatiques, Hermès, Paris, 1998.
Principles of Modeling
97
[RAB 06] RABEAU R., Proposition d’une démarche de validation des simulations technicoopérationnelles utilisées comme une aide à la décision, Doctoral thesis, University of Versailles Saint-Quentin – IT specialism, 7th July 2006. [RAB 07] RABEAU R., “Credibility of defense analysis simulations”, 20th International Conference on Software & Systems Engineering and their Applications, ICSSEA07, Paris, December 2007. [RTO 03] NATO RESEARCH AND TECHNOLOGY ORGANISATION, RTO SAS Lecture Series on “Simulation of and for Military Decision Making” (RTO-EN-017), The Hague, 2003. [RUM 04] RUMBAUGH J., JACOBSON I., BOOCH G., The Unified Modeling Language Reference Manual, 2nd edition, Addison Wesley, Old Tappan, 2004. [RUB 81] RUBINSTEIN R.Y., Simulation and the Monte-Carlo Method, John Wiley & Sons, New York, 1981. [SAU 01] SAUTEREAU C., ESCADRE 5.1 documentation, Centre d’Analyse de Défense, Arcueil, 2002. [SIS 08] DSEEP PRODUCT DEVELOPMENT GROUP OF THE SIMULATION INTEROPERABILITY STANDARDS ORGANIZATION (SISO), IEEE P1730/Dv2.0 Draft Recommended Practice for Distributed Simulation Engineering and Execution Process (DSEEP), IEEE, Piscataway, 2008. [STA 03] STANDISH GROUP INTERNATIONAL, Chaos, Investigation Report, West Yarmouth, March 2003. [ZEI 76] ZEIGLER B., Theory of Modeling and Simulation, John Wiley & Sons, Hoboken, 1976. [ZEI 98] ZEIGLER B., BALL G., CHO H., LEE J.S., SARJOUGHIAN H., The DEVS/HLA Distributed Simulation Environment and its Support for Predictive Filtering, University of Arizona, Tucson, 1998. [ZPK 00] ZEIGLER B., PRAEHOFER H., KIM T.G., Theory of Modeling and Simulation, Academic Press, Oxford, 2000.
Chapter 3
Credibility in Modeling and Simulation
Simulation is only useful if the results it provides are representative of the behavior of the system being simulated. This raises issues of verification, validation, and accreditation (VV&A), which allow us to guarantee the reliability and credibility of the results obtained. This chapter is dedicated to these questions and concentrates on simulations used at the beginning of the engineering cycle, during the analysis of concepts and the definition of preliminary architectures; examples of technico-operational simulations in the context of national defense will be used as a framework for illustration. 3.1. Technico-operational studies and simulations In the context of design and preparation of future defense systems, the DGA and General Staff use a strategic approach based on wide reflection and long-term forecasting (up to 30 years). This forecasting work orients future weapons programs from the preparation phase, which precedes the introduction of new products up to the point where the product is retired from service. In this approach, technicooperational studies are widely used to assist in the choice and decision-making process. Their aid is particularly precious when seeking the best compromise between operational characteristics, technical specifications, and associated costs in future systems or in updates to existing systems. Technico-operational studies are best used during the preparation stage before a weapons system is designed, as decisions taken at this level have the strongest Chapter written by Roland RABEAU.
100
Simulation & Modeling of Systems of Systems
impact on final costs and results; at this stage, the level of investment involved is relatively small. The studies carried out focus on evaluating the effectiveness or military vulnerability of force systems (i.e. systems of systems at capacitive level, for example, assuring deterrence or authorizing a capacity for projection according to the level of operational contract given), combat systems (armored vehicles, warships, and so on), weapons systems (cruise missiles, guns), or logistics systems in an operational context. The methods used, traditionally taken from the domain of operational research, often use digital simulations, known in this context as technico-operational simulations, although analytical resolution techniques are used sometimes. As a general rule, the applications of simulation in the context of defense can be categorized according to two different criteria: – the use made of the simulation; – the level of the system being simulated. According to the first criterion, simulation is used for the analysis and design of defense tools; acquisition; training; and planning, preparation, and execution of operations. In the second group, we find simulation being used on several levels: strategic (defense systems), operative (force systems), tactical (combat systems), and dual (weapons systems). This typology can be found in most reflections on the subject of defense simulation, whether in France or abroad. Technico-operational simulations are mainly used in the areas of analysis and design of defense tools and in the planning, preparation, and execution of operations. In more formal terms1, a technico-operational simulation is “a digital simulation2, generally not in real-time, in which we find dynamic3 models4 of real systems and their environment, whether natural (terrain, atmosphere, marine environment, weather etc.) or man-made (roads, towns etc.), possibly5 faced with a hostile environment, known as a threat”. While using the simulation, models representing systems (material, physical, and so on) or living things 1 Definition taken from the Guide-Dictionnaire de la simulation de défense [DGA 97]. 2 Using only the program, as opposed to a hybrid (hardware-in-the-loop) simulation. 3 Models that evolve over time. 4 Mathematical, physical, or logical representation of a system, entity, phenomenon, or process. A representation of reality of this kind is usually called a conceptual model to set it apart from a computerized model, which is the computer installation of the former. 5 In cases where a combat, crisis or similar is being simulated.
Credibility in Modeling and Simulation
101
(system operators, fighters, decision-makers, populations, and so on) evolve according to algorithms, using data related to the characteristics and performances used, according to a pre-selected scenario. A technico-operational simulation can be more or less interactive: some are not (closed-loop simulations), and in this case, if they involve random factors, they are subjected to analysis using Monte-Carlo techniques (with the intention of obtaining statistical results). Other simulations are more open (possibly piloted more or less continually depending on the simulation), such as war games in which two or more camps, commanded by real (not simulated) operators, attack or assist each other in an engagement. 3.2. Examples of technico-operational studies based on simulation tools 3.2.1. Suppression of aerial defenses One of the memorable points of the first Gulf War (1991) was the very low level of attrition experienced during the air campaign (39 coalition aircraft lost in 66,000 sorties). In the months following the end of hostilities, the French Ministry of Defense launched a series of studies that aimed to explain this remarkable fact. The first of these studies consisted of analyzing the course of aerial operations and in particular determining the role of electronic combat in terms of operational effectiveness. The conclusions of this study showed that the explanation for the phenomenon observed seemed to reside in the massive and coordinated use of the whole panoply of electronic combat methods available to coalition forces. The use of these systems allowed the coalition forces to effectively disable the Iraqi aerial defense system without requiring its physical destruction. This is probably a case of “suppression”, a practice already observed on a small scale toward the end of the Vietnam War. Following on from this preliminary analysis, a second study was carried out to determine more precisely how this suppression effect was produced. This study, principally based on using the simulation tools, allowed certain phases of the aerial operations to be replayed, clarifying the mechanisms involved in the suppression effect. 3.2.2. Heavy helicopters Numerous studies were carried out both at national level and in collaboration with other countries to evaluate different concepts of heavy helicopters.
102
Simulation & Modeling of Systems of Systems
The studies carried out consisted of defining scenarios representative of realworld use of aerial forces then analyzing, within these scenarios, the operational interest of the concepts studied. These studies focused on the analysis of different conceivable concepts in terms of transport capacity. They notably attempted to determine, for each of the solutions studied, the number of aircraft needed to complete the required missions and reach certain capacitive goals set by the General Staff. Following on from this initial work, additional studies were carried out to analyze, for a forecasted scenario, the complementarity between existing fleets of medium helicopters and the planned heavy helicopters. Once again, simulation proved to be a precious tool for the comparative analysis of different parameters and their operational influence. 3.3. VV&A for technico-operational simulations 3.3.1. Official definitions Considering the use of the technico-operational simulation results, their quality is the object of particular attention. This attention is expressed by the use of three specific processes: VV&A. The definition of the contents of these processes has been normalized, within the framework of wider modeling and simulation (M&S) activities, by the IEEE, on the impetus of the Defense Modeling and Simulation Office (DMSO, part of the US Department of Defense, DoD): – verification: the process of ensuring that the installation of a model and the associated data represent the description of the subjacent conceptual model and its specifications precisely; – validation: the process of determining the precision with which a model and the associated data represent the real world, considering the planned use of the model; – accreditation: the process of deciding, officially, that a model, a simulation, or a federation of models and simulations and the associated data is acceptable for a given use. In practice, the accreditation decision is pronounced at the end of the verification and validation (V&V) procedures: a decision is made, based on the results of the V&V processes, whether or not to use the models for a given application. Although the person responsible for the study is usually responsible for the accreditation decision, it is important to involve the final user of the results of the study in this process. The decision made has a deciding effect on the level of trust placed on the
Credibility in Modeling and Simulation
103
results of the simulation – that is, their credibility, which we shall discuss next – which is itself a necessary condition for the profitable use of the results. Note that although there is some consensus regarding the definitions of the principal terms used in VV&A6, they are still the subject of more or less fierce debate and frequent reconsideration. Certain elements of reflection on different approaches to the concept of validation, for example, can be found in [GIR 02] and [ZEI 02]. 3.3.2. Credibility A simulation, the data used and the results obtained, become credible when the decision-maker accepts and appropriates them. The support of the decision-makers and their trust in the elements provided have a direct impact on the use of these elements in the decision-making process: the best-run study in the world is of little value if the results obtained are not considered. A high level of credibility is thus crucial to ensure the success of a technico-operational study7. In the field of M&S, credibility is less formally defined than V&V. We do, however, find the following definition [DEF 01a]: “Credibility: the relevance that the user sees in a model and the confidence that the user has that a model or simulation can serve his purpose”. Credibility8 does not, therefore, depend directly on the level and quality of the V&V processes followed but relates to the perception the user has of the suitability of the simulation for his or her needs and the usefulness, in this context, of the results obtained9. We may suppose, then, that this perception does not depend solely on the quality of the arguments developed by those responsible for
6 Mostly established during the first SIMVAL seminar [MOR 90] and reused by the DMSO for the Department of Defense (DoD), as well as in other ministries in the United States (e.g. the Department of Energy, DoE) and abroad. 7 Credibility is not to be confused with validity: a simulation may be credible but not particularly valid and vice versa. A simulation does not necessarily present the qualities required to solve a particular problem just because the decision-maker trusts the results! 8 According to this relatively ambiguous definition, the term “credibility” could be indiscriminately applied to a simulation or to the use to which it is put (i.e. the results obtained): this latter interpretation, closest to the preoccupations of decision-makers and used by several authors, will be used here. 9 Note the distinction between validation of a domain of application and the validation of the domain of a model, to which we shall return later.
104
Simulation & Modeling of Systems of Systems
the V&V process but also on the level of knowledge of the decision-maker in the field of simulation. The awareness of factors with a real influence on credibility and of their impact is, then, essential to obtain a satisfactory level and ensure that the validation process used during the planning of a study will succeed not only in improving its intrinsic quality but also in making its results acceptable to decision-makers. However, although credibility can generally be analyzed by a quantitative evaluation of precision in “classical” applications of scientific simulations [OBE 02], evaluation of this kind is difficult, if not impossible, in the case of models used for technico-operational simulations. This is due, notably, to the fact that these models consider very complex systems and human behavior and, more generally, use high levels of aggregation. Effectively, in technico-operational simulations, the “system” aspect – capacitive or not – cannot be dealt with by one analysis, but requires analysis at various organizational levels, hence the aggregation issue. Note that this apparently simple aggregation approach conceals averaging problems similar to those encountered in physics: this is a difficulty involved in the transformation between microscopic, mesoscopic, and macroscopic levels. As the credibility of results presented in this context cannot be based directly on the results of quantitative evaluations, other factors play a significant role, including: – the understanding and the approbation of the principal hypotheses used by the decision-maker; – the nature of the V&V processes, the perceived performance of the methods used and their sufficiency; – the amount of information known to the decision-maker concerning the V&V procedures used; – knowledge of the modeled systems and their modus operandi; – effective involvement of the decision-maker in the project; – the reputation of the team responsible for the project. However, the real influence of these factors remains relatively little understood, and project teams do not have access to recognized operational procedures which would allow them to choose easily between different possible techniques (other than by reference to their perceived effectiveness in the domain of validation), to determine preferential terms of application, or to adopt a communications strategy truly adapted to the needs of the decision-makers. Nevertheless, [RAB 07] studies the influence of different factors on credibility and their consideration in the development of a suitable validation approach.
Credibility in Modeling and Simulation
105
3.3.3. Key players in the domain 3.3.3.1. DMSO Although the V&V of programs used in the context of decision assistance have been important considerations since they first appeared, the formalization, then standardization, of the vocabulary used only occurred in the 1990s, with the creation of the DMSO within the US DoD in 1991, then the publication of successive Master Plans [DÉL 95] with an emphasis on shared simulation (the High-Level Architecture (HLA) standard10) and VV&A (Recommended Practices Guide, RPG). The initiatives taken by the DMSO in the domain of simulation still federate a very large part of the work carried out in the field of VV&A today; VV&A has been a feature of major programs established by the Master Plan since 199511. The aims of the Master Plan in terms of VV&A are to “develop methodologies, standards and procedures for VV&A of modeling and simulation”. The first visible results of this plan were the DoD’s instruction 5000.61 [DÉL03] (defining prescribed policies, responsibilities, and procedures for VV&A), the VV&A RPG (detailing the philosophy, principles, and methods recommended for VV&A within the DoD), and the organization of Foundations workshops in 2002 and then 2004. The VV&A RPG places a significant number of documents dealing with basic concepts and specific points relating to VV&A, alongside reference texts, at the disposition of the simulation community. The documents included fall into two broad groups: methodological documents, which deal with the organization and implementation of VV&A processes, and technical documents, which deal with methods and tools which may be used in the process. 3.3.3.2. International organizations Two organizations other than the DMSO are also very active in the field of VV&A. The Winter Simulation Conference (WSC) is an annual conference made up of presentations on the latest developments in the domain of digital simulation. Participants come from various backgrounds, whether government (defense or otherwise), industrial, or university. The WSC is organized and managed by a group of eight organizations which are major players in the domain of simulation: 10 See [KUH 99] which presents the HLA. 11 NATO has also taken the issue on board, and published its own M&S Master Plan, which deals with VV&A, in 1998.
106
Simulation & Modeling of Systems of Systems
– American Statistical Association (ASA); – Association for Computing Machinery/Special Interest Group on Simulation (ACM/SIGSIM); – Institute for Operations Research and the Management Sciences–College on Simulation (INFORMS–CS); – Institute of Electrical and Electronics Engineers/Computer Society (IEEE/CS); – Institute of Electrical and Electronics Engineers/Systems, Man, and Cybernetics Society (IEEE/SMCS); – Institute of Industrial Engineers (IIE); – National Institute of Standards and Technology (NIST); – Society for Modeling and Simulation International (SCS)12. The second organization with a major role in the domain of simulation is the Simulation Interoperability Standards Organization (SISO). Created in 1989, the SISO aims to coordinate efforts concerning simulation networks, then develop and promote interoperability standards. The SISO organizes two workshops each year (the Spring and Fall Simulation Interoperability Workshops) in the United States and one per year in Europe (Euro SIW). Although the SISO is more open than the WSC in terms of international participation (with 12 participant nations and a “democratic” organization to elect post-holders), in reality it is essentially American led (by representatives of the DMSO) with some British involvement. In addition to the organizations already cited, we might also consider the Military Operations Research Society (MORS). This society, founded in 1957 and supported by the US DoD, aims to improve the quality and efficiency of military operational research. The increasing use of simulation in the domain of operational research, particularly in the acquisition process, inevitably led the MORS to consider the problem of simulation validation; for this reason, the MORS organized workshops from 1990 onwards (including SIMVAL, SIMVAL II, SIMVAL’ 92, 94, 99, 01, and 02). The concepts and definitions produced during these workshops have been largely adopted by the whole M&S community. 3.3.3.3. Development of VV&A methodology in France In France, the French Armament Directorate (Direction Générale pour l’Armement) has led a certain number of actions in the domain of VV&A, both in a national context and within NATO workgroups. We shall cite some examples:
12 SCS International also organizes the Summer Computer Simulation Conference.
Credibility in Modeling and Simulation
107
– the Franco-British MEVAS (Méthoded’Evaluation, Validation, Accréditation des Simulations) study, a joint project by the CAD (Centre d’Analyse de la Défense, part of the DGA), the ONERA (Office National des Etudes et Recherches Aérospatiales, a public establishment supervised by the French Ministry of Defense), the Defense Science and Technology Laboratory (DSTL, part of the British Ministry of Defence (MOD)), and Cranfield University, within the framework of the Anglo-French Defense Research Group (AFDRG). The chosen perspective centers on the organization of simulation and those involved (promoters, users, and beneficiaries), setting it apart from the DMSO, which uses a more development-oriented philosophy (see the MEVAS reference manual [DÉL 02]); – the NATO Modeling and Simulation Group (NMSG) study 016–TG 019 VV&A Overlay for the Federation Development and Execution Process ((FEDEP) for HLA is the development process used for federations of simulations, inspired by development processes used in software engineering). This study, led by an international group (France, Germany, the United Kingdom, the United States, and Sweden) under the aegis of the NATO Research and Technology Agency, aimed to superpose a VV&A process on the FEDEP HLA. The results of the work of NMSG 016 were considered by a SISO workgroup with very similar objectives (VV&A Overlay to FEDEP Product Development Group); – the activities of the International Test Operations Procedures Working Group of Experts (ITOP WGE 7.2), within the framework of a Memorandum of Understanding between five countries (France, Germany, the United Kingdom, the United States, and Sweden as an observer), which covers reciprocal sales of weapons materials. The WGE 7.2 exists to develop V&V procedures with the aim of facilitating mutual acceptance of simulation results by member states [INT 02]. A first version of the reference document was made available in 2004, and experiment were carried out on a test case; – the work carried out within the Western European Union Armament Group/ Common European Priority Area 11 (WEAG/CEPA 11) on defense research and technology; – European Co-operation for the Long-term In Defense – Research and Technology Program 11.13 (EUCLID RTP 11.13): this project brings together 22 companies in 13 countries with the aim of promoting the use of synthetic environments in Europe by developing a common process and tool set. The project aims to reduce the cost and time delay involved in the development of simulations for training and acquisition. The program of work includes several sub-groups dealing with the validation of simulations and federations; – TecHnological Arrangement for Laboratories for defense European Studies Joint Program 11.20 REVVA (THALESJP 11.20 REVVA (European project to create a reference standard for VV&A)): this project [REV 04] aims to develop a
108
Simulation & Modeling of Systems of Systems
common methodological framework for VV&A of data, models, and simulations. It is led by a multi-national group of five countries (France, Denmark, Italy, Sweden, and the Netherlands). The approach taken considers the results of the work cited above. 3.4. VV&A issues Having described the technico-operational studies framework and provided definitions relating to VV&A and the technico-operational simulations presented, we may now consider the concrete implementation of different VV&A procedures when used as part of a technico-operational studies process. In practice, the application of a VV&A process requires us to respond to five types of questions: – What, precisely, needs to be verified and validated? – How should V&V techniques be used? – What V&V process should be used, and how is it related (in terms of timing and actions) to the development and study processes? – How can we measure the difference between the models and reality (validation metric) and evaluate the adequacy of the simulation used for the problem being dealt with? – What level of resources should be allocated to V&V, that is, how far should the process be taken? These questions are particularly crucial as each comes with associated costs, and the analysis stage of preliminary concepts does not usually attract a large budget, despite the fact that decisions made at this stage impose choices later in the life cycle of the system. The following section gives a detailed overview of these five considerations and of certain specific problems that may be involved in the application of V&V processes. 3.4.1. Elements concerned Recent developments in the domain of VV&A place the emphasis on a widening of current practices, essentially directed toward an evaluation of products used in the simulation, toward the classic trilogy of product/process/project or product/process/ personnel found in software engineering [VOA 99].
Credibility in Modeling and Simulation
Process
Person •
Qualification
•
Independence
•
…
109
Formalism
Products •
Conceptual model
•
Computerized model
•
Simulations
•
Data
•
Scenarios
•
SW-CMM
•
ISO
•
…
VV&Aprocess Figure 3.1. Targets of V&V actions
Following the accepted definition of V&V processes, these essentially concentrate on five types of products created or used during the development and usage phases of simulation (Figure 3.1): – conceptual models; – computerized models; – simulations (and federations of simulations); – data; – scenarios. 3.4.1.1. Conceptual models A conceptual model is a more or less formalized description13 of the algorithms, architectures, hypotheses, and subjacent limitations used in the creation of software components. It is generally a simplified representation of reality, involving a certain level of abstraction, created explicitly or implicitly with a precise use in mind. The validation of a conceptual model consists of ensuring that the theories and hypotheses used are correct and that the planned use is reasonable considering the characteristics of the model [SAR 04] (Figure 3.2).
13 Although the general meaning of the term “conceptual model” as it is used here is widely accepted within the M&S community, there are various official definitions which present minor differences [RPG 01a].
110
Simulation & Modeling of Systems of Systems
Figure 3.2. Sargent’s model and validation of a conceptual model, taken from [SAR 04]
In practice, the formalism used14 to describe conceptual models is frequently ill-adapted to the detailed description of a complex reality. This explains, in part, the general weakness or lack of updates to documentation on conceptual models (e.g. all the connections will not be known to or explained by the designer and will only come to light during operational validation). On occasion, the behavior of the modeled system, although based on simple rules, cannot be predicted (due to the number and complexity of interactions): this is known as emergent behavior. In this case, conceptual validation is tricky as a review, however meticulous, of the conceptual model does not really allow us to pass judgment on new behaviors which will emerge and thus assess the suitability of the model for the planned use. 3.4.1.2. Computerized models Verification operations (Figure 3.3) carried out on computerized components created as part of the project consist of ensuring that the conceptual model has been correctly installed. The methods used are, for the most part, taken from software engineering (formal proof).
14 In general, and particularly in the case of inherited models, the formal aspect is reduced to its weakest expression.
Credibility in Modeling and Simulation
111
Figure 3.3. Sargent’s model and verification operations, taken from [SAR 04]
The validation of the computerized model, known as operational validation (Figure 3.4), consists of determining whether or not the output of the model and its behavior are sufficiently precise for the desired purpose. Most of the validation procedure takes place during this process as the dynamic behavior of the models and that of real systems can be directly compared.
Figure 3.4. Sargent’s model and operational validation, taken from [SAR 04]
112
Simulation & Modeling of Systems of Systems
In cases where this comparison is not possible, for example, when dealing with a future system or a complex system for which we have little observed (real-world) data (e.g. historical data on operations), it is obviously difficult to attain a high level of confidence. One troublesome aspect, which is rarely dealt with, concerns the difference between the subject domain (the domain of the model), the aspect of the real system which is supposed to be represented, and the object domain (the domain of application)15, which relates to the questions for which we hope to find responses using the model. In practice, we observe that in most validation processes, whether operational or conceptual models, the emphasis is placed on the evaluation of the difference between the behavior of the modeled system and the behavior of the real system (which corresponds to the first part of the definition of validation), while reflection on the level of realism actually required for the purpose (the second part of the definition) does not usually attract the attention it demands. Although what follows is something of a caricature, we might say that essentially, we ensure that the model is realistic in what it is supposed to represent without defining precisely what needs to be represented. One of the reasons for the lack of approaches used is probably because it is relatively easy to specify a conceptual model and then compare the output of its computer implementation with that of the real system, but it is much more difficult to specify the characteristics of the necessary conceptual model and ensure its sufficiency for a given problem. 3.4.1.3. Simulations and federations of simulations As a simulation generally involves several elementary models that interact, we need to carry out an operational validation of the system as a whole as well as for individual models. Experience has shown that the procurement of good results during the validation of models does not guarantee the quality of the simulation as a whole once the group of models has been assembled. This separation between the validation of models and the validation of simulations is similar to the sequence between the operation of unitary tests and integration which is characteristic of a cascade or “V” approach to development. Very large simulations and federations of simulations, for example, those constructed using the FEDEP HLA, developed by the DMSO, present, in addition to
15 See subject domain and object domain in [BRA 00] and problem domain in [HON 98].
Credibility in Modeling and Simulation
113
an increased level of complexity16 due simply to the effects of size, a certain number of specific problems: time management, non-repeatability of executions, heterogeneity of material architectures and representations of the environment, conceptual interoperability, and so on17. Validation thus becomes an extremely complex subject, which has led various authors (e.g. [SAR 00, BRA 00]) to doubt that a significant level of confidence is attainable even when costly V&V procedures are used. One focus of research, suggested by Robert Sargent, involves the development of methods of working based on the use of several simulations of manageable size in the place of architectures which do not yet seem to be masterable. The simple validation of these models is still a sensitive subject, so, today, attempting to deal with the validation of federations of simulations is still quite an ambitious objective. 3.4.1.4. Data As the quality of data used has a direct impact on the quality of the results produced by a model, the validation of a simulation must necessarily involve the validation of the associated data. Action can be considered in this area by two groups of people: – The producers of the data, who are guided by the use of rigorous organization in data collection and management. This falls into the domain of Data Quality Activities (DQA). – The users of the data, who carry out validation on data in the course of use for a given model and a given problem. This is part of the VV&A simulation. A particularly acute problem appears when especially reusing the data, as the data collection is always carried out with reference (explicit or otherwise) to a conceptual model of a real system. This reference is generally common to the producer and the user during the initial collection of data. However, if it is not explicit, divergence may occur in later uses, hence the potential risk when reusing data18. One possible solution to this problem is based on the use of metadata associated with the data [DEF 01c]. This metadata would carry information concerning the origin and quality of data and the type of aggregation chosen. The generation and management of this metadata could be facilitated by using specific tools. A second problem is created when there are implicit links between different units of data used in a simulation (e.g. if a terrestrial operational scenario is to be played out in bad weather, we must use data concerning the speed of vehicles on 16 See [MOR 02] and [PAC 02] on this subject. 17 See [TOL 03]. 18 “A data repository is like a drawer full of sharp knives: adult supervision is required …” [MOR 02].
114
Simulation & Modeling of Systems of Systems
muddy terrain and reduced performance of detection hardware), which are not automatically managed; moreover, these links may change depending on the chosen scenario. 3.4.1.5. Scenarios The data used by a simulation is usually divided into two parts: – data specific to different models (technical data, behavioral data, and so on); usually, we find several instances of models using the same data set; – scenario data (initial positions of different objects, choice of technical and operational characteristics, and so on), of which each instance uses (in principle) a different set of data. The validation of technical data is widely accepted nowadays as part of the general VV&A process, but the validation of scenario data does not seem to attract the attention it deserves. In most formalized approaches, the definition of scenarios appears either beforehand in the formulation of the problem, among the user specifications, or at the end of the VV&A process when, after accreditation has been accorded, the user enters the phase of simulation use. However, the choice of scenarios has a major influence on the results which will be obtained during the use of the simulation and on the conclusions which will be reached. Returning to the definition of validation given above, it is clearly necessary to ensure that scenario data sufficiently reflects reality in relation to the problem posed, in the same way as technical data or the models being used. A satisfactory validation process, then, must also contain a phase of scenario definition. 3.4.2. Verification and validation techniques A very large number of V&V techniques have been proposed and described in existing literature: the DMSO [DEF 01d] has collected and described around a hundred of these techniques. A rapid discussion of the choice of techniques depending on the situation may be found in [SAR 84] and in the reference documentation of the DMSO. Concerning the typology of these techniques, two major means of classification are generally used, depending on the following: – the stage of the VV&A process at which they occur [KNE 93]: - validation of the conceptual model, - verification of the computer model,
Credibility in Modeling and Simulation
115
- operational validation, - data validation, - verification of internal security19; – their complexity and the level of the mathematical formalism used [DEF 01d]: - informal techniques, - static techniques, - dynamic techniques, - formal techniques (methods). The following section presents some elements of reflection on the aforementioned techniques in relation to their use in technico-operational simulations, with emphasis on validation techniques. Verification techniques, being taken mainly from software engineering, have consequently been much more thoroughly tested. The typology used is that presented by the DMSO, as it has the advantage of following an order which is not prejudicial to the VV&A process used. Particular attention will be given to the domain of formal methods, still relatively little used in the domain of technico-operational studies, despite the fact that their application is potentially promising. 3.4.2.1. Informal techniques These techniques are so called as they are based essentially on human expertise and judgment with little insistence on the formal aspects of the process. As the DMSO notes, the term “formal” in this context should be understood in its mathematical sense, and even so-called informal techniques respect a certain formalism and demand rigor in their application. The informal techniques most frequently encountered in existing literature20 are audit (and its variants, review, inspection etc.), historical analysis, individual inspection (desk checking/auto-inspection), the Turing test (is an expert capable of telling whether he is faced with a simulation or the real world?), and especially face validation. As an example, let us consider this last technique, which is very widely used in the domain of technico-operational studies. It may be used at two different stages of 19 Internal security verification: an extension to Sargent’s model that covers all verifications which ensure the integrity of the model and data, notably by rigorous configuration management. 20 See [RAB 06a,b] for a critical review.
116
Simulation & Modeling of Systems of Systems
the VV&A process: conceptual validation and operational validation (see Sargent’s model). The first of these uses consists of a review of all the information available on the simulation and the problem being dealt with (including envisaged scenarios) to find potential points on which additional analysis is required. The participants in the process are experts in the different domains concerned. This review has consequences on two phases of a study: – During the design phase, it allows us to identify parts of conceptual models that should be examined in greater depth and possibly modified. It also allows us, if the group of experts is sufficiently large, to ensure that the aspects of reality we have chosen to represent are adapted to the problem and that none has been forgotten; in this case, the sufficiency of the characteristics of the simulation for the problem being dealt with can be evaluated in an empirical manner. – During its usage, it indicates points that should be monitored with particular attention and possible specific procedures to implement. The second use of this technique (Delphi technique variations), during the operational validation phase, consists of confronting, informally, the output of the simulation, the behavior of different models, and so on, with a group of experts with knowledge of the real system, then analyzing possible divergences, and finding their causes. Considering the results of this analysis, the planned use of the simulation, and the risk level the user is willing to take, the analysis may be refined and/or the models concerned adjusted. One advantage of this technique is that it is not particularly costly in terms of time and resources. This, alongside the simplicity of implementation, explains its success in the domain of technico-operational studies where meetings take place between experts in modeling, techniques, and operations. The method is, however, subjective and not systematic21, and the results depend heavily on the level of the experts involved and the information available22. This kind of “high level” analysis is not particularly good at detecting problems outside the field of experience of those involved, or which are brought to light based on the bias (or unsuitability) of models, undetectable without the user of finer analytical methods (e.g. statistical analysis). The popularity of this technique and the numerous criticisms it has received are at the origin of several initiatives to normalize and improve this approach [STE 02]. 21 The points that are examined and to which experts react generally depend more on their sensibility to problems already encountered or to their competences rather than on a formalized and exhaustive approach. 22 Experts are not infallible and may change position or contradict themselves. In the domain of future or unusual systems, expert advice must be considered with a certain degree of care (see [PAC 99]).
Credibility in Modeling and Simulation
117
To summarize, informal techniques are widely used, by the majority, in the domain of technico-operational studies. Two factors explain this: first, technicooperational studies are located within the framework of a little-formalized initial need (and an important part of each study consists of formalizing this need), and second, the exploratory and preliminary character of technico-operational studies means that the issues are sometimes underestimated and the approaches used do not make use of all available techniques, notably those (wrongly) perceived as too limiting or consuming resources. 3.4.2.2. Static techniques Static techniques aim to evaluate the quality of a model and the associated source code alongside the respect of specifications (computerized and conceptual). These techniques are thus named as they do not require actual execution of the simulation or the models. In the domain of technico-operational studies, they are used almost systematically for verification and often for validation. Numerous tools are available for the implementation of these techniques, with varying degrees of automation, and are widely used by the teams in charge of development and/or V&V. Apart from verification techniques which are taken directly from the domain of software engineering, the use of cause-effect graphs, the analysis of control fluxes, state transition analysis, data analysis (checking for dependences, verification of consistency, and so on), and interfaces between models or with users are among the most widespread static verification techniques. 3.4.2.3. Dynamic techniques Dynamic techniques are widely used in technico-operational studies and are based on an evaluation of the behavior of models in the course of their execution. These techniques can be used during the development phase (sub-model/module testing) or during assembly when constructing a simulation or a federation of simulations (product testing), but also during the alpha-test and beta-test phases; we also encounter them in acceptance testing and in testing during the phase of simulation use. The evaluation of model behavior and their comparison with real systems can be done by comparative observation of high-level variables (black-box testing) or less visible internal variables. The domain in which input variables are adjusted can be chosen in different ways: real data sets, borderline data that produces behavioral changes, zones of homogeneous behavior, extreme data values, deliberately incorrect data sets, randomly generated data sets, and so on. A validation process may also be carried out based on the comparison of results obtained by different simulations (cross validation) or by several versions of the same simulation (Nversion programming). This method is used fairly frequently.
118
Simulation & Modeling of Systems of Systems
One significant difficulty in dynamic validation is the precise determination of variables to observe in order to carry out the appropriate comparisons23. To deal with this problem, experts’ advice is often solicited (from those whose knowledge of the domain allows them to satisfactorily predict the significant parameters which should be observed), but systematic techniques may also be used (particularly when dealing with unpredictable behavior or complex assemblies) to identify influential parameters (sensitivity analysis or screening). The means of comparison used in practice are very varied: expert advice (the Delphi technique), graphical comparisons of data, animated visualization, statistical tests (parametric or non-parametric), multiple confidence intervals, analysis of variance, analysis of time series, and so on. Note that for quantitative comparisons to be possible, we need to have one (or more) metric by which the difference between the real world and the models used can be measured. However, the determination of satisfactory metrics, considering the problem being dealt with, and their use (rational judgment of acceptable difference during the accreditation procedure) is still a subject of study24. A connected subject (also open) is the determination of how to propagate, in the hierarchical decomposition of a complex system, the validation information obtained for each element. The visual analysis of model behavior can take place in real time during the execution of a simulation or later using the saved data necessary for graphical replay. This technique is particularly interesting in showing the dynamic behavior of models with complex logic or which include a significant number of interactions. The ability of experts to provide summary from visual information is well used and allows a large part of the anomalies present in a simulation to be detected. Visualizations and graphical replay are very widely used in the domain of technico-operational studies because of their habitual use of complex models which interact and are sufficiently small in number for their individual and collective behavior to be apprehended visually by users. The effectiveness of this technique, the credibility it provides during presentation to decision-makers, and its impact during organized demonstrations have ensured its success ever since work-station performance reached a sufficient level for it to be used. The visualizations used are
23 It is neither necessary nor desirable to compare all the available variables: the more or less systematic use of aggregated models makes divergence between the model and the real world inevitable, at least on certain points. The art of validation, then, consists of determining what aspects of a model must be realistic, and the acceptable divergence from the reality in these aspects, taking into account the question being considered. 24 For an exhaustive presentation of the state of the art in quantification of validation, see [OBE 00].
Credibility in Modeling and Simulation
119
essentially spatial (position of actors), possibly enriched with various interactions (explosions, various detections, and so on). Sometimes, the user chooses to bring out other aspects of the models while remaining, essentially, in the analysis of temporal processes (the observation of evolutions of a selection of variables which have been judged particularly important). Moreover, the use of visualizations presents the advantage of being able to show a large number of experts from different technical and operational fields simultaneously without requiring them to know how to operate the simulations used. One potential problem lies in the visual impact of a simulation being executed: it can easily lead the inexperienced user to place unreasonable trust in the validation actions actually effectuated. Note that, to carry out in a truly effective manner, validation actions based on graphical visualizations should take place in a context where participants have easy access to variables allowing them to follow the internal operation of models and sub-models and to the associated conceptual documentation. We again encounter the problem of a compromise between the objectivity, linked to an exterior viewpoint, and the necessity of an intimate knowledge of the models, as with independent verification and validation (IV&V, see below). We also note that a validation process involving the implementation of a program developed outside the team responsible for the study – for example, by an external sub-contractor – generally encounters difficulties related to the lack of knowledge of the models (conceptual and computerized), hard to combat in the absence of the sub-contracted development team, due to a lack of completely objective and uniquely phenomenological “black box” criteria. In the long term, the perspectives opened up by the use of metamodels (see works on Model Driven Architecture (MDA) [KAM 04, KAM 10]) may allow improvements in the situation on this point. 3.4.2.4. Formal techniques Formal techniques are based on the use of mathematics for the specification, development, and verification of computer programs. These techniques have been studied since the 1960s and are currently the subject of growing interest with a widening field of application which, contrary to popular belief [HAL 90], is not restricted to software used in life-critical or safety-critical systems. Actions carried out within the framework of formal methods focus essentially on the analysis of specifications or of the behavior of the program with regard to the specifications and are not directly part of a process where the real system is confronted with the model. Formal techniques are therefore most often used in the field of verification. They do, however, have potentially interesting applications in the field of validation (e.g. use of lightweight formal methods (LFMs) in the specification phase).
120
Simulation & Modeling of Systems of Systems
3.4.2.4.1. Specification The specification of programs using formal techniques, that is, the description of the behavior and properties of systems using syntax and semantics based on rigorous mathematical bases25, is a domain in which these techniques have proven particularly effective [CLA 96]. The construction of formal specifications allows the demands of the client to be clarified and allows us to develop a precise and comprehensive description of model behavior, free from ambiguity or inconsistency. This description is particularly useful in cases where the model is destined for reuse, currently a significant trend in the programming industry: formal specifications allow precise understanding of the behavior of a software component, something which is currently rare and poses significant problems when using a simulation construction strategy based on the assembly of components. As often happens in modeling activities, the creation of a precise description of a real-world object increases knowledge of this object and improves the quality of dialog between the client, experts, and those responsible for modeling. 3.4.2.4.2. Verification Outside the field of specification, formal methods are used to analyze the properties of formal, then computerized, models. They allow us to ensure that not only the behavior of a model conforms to specifications but also the same model does not present undesirable behaviors. Taking formal specifications as a starting point, formal methods also allow the generation of test scenarios which show that the required properties are effectively present. The formal description of models to which the development team has access after the formal specification phase may be used to implement three main categories of tools: theorem proving tools, model checkers, and tools for the automatic generation and exploitation of tests26. Theorem proving tools aim to demonstrate that a program possesses certain (usually behavioral) properties as set out in conceptual specifications. The tools currently available allow large-scale specifications to be dealt with but do require assistance from a (highly) qualified human operator.
25 Formalization consists of describing the performance of algorithms expressing the behavior of models in a rigorous manner, without going into detail on how the algorithms do this. 26 See [KUH 02], which provides a presentation of this, followed by different examples of operational applications with commentary.
Credibility in Modeling and Simulation
121
The model checking approach consists of exploring, possibly exhaustively, the model state space and ensuring that the specified properties are respected27. These tools are essentially used for finite state models28. They deal poorly with continuum variables due to the size of the state space and the errors generated by the discretization process. The principle advantages of these tools are automatic implementation and traceability (counter examples) of negative conclusions. Automatic test generation and exploitation tools work from a formal specification: unlike analog tools in “classic” software engineering, which are not based on the use of formal specifications, supervision by a human operator is not necessary to ensure that the output of the model conforms to expectations for each test case. Moreover, these “classic” tools are susceptible to another potential difficulty which resides in the quest for a certain level of exhaustivity in the tests carried out, resulting in the explosion of the number of test cases. Once again, several interesting techniques, which profit from formal specifications, are currently being explored in the field of formal methods (such as model reduction and selection tests on input data). These techniques, then, are an effective means of verification which allow us to ensure that the specifications developed, along with the source code – and even the machine code generated by the compiler29 – conform to the descriptions provided by the client and experts of the different models involved in a simulation. 3.4.2.4.3. Validation Although the field of application of formal methods is essentially oriented toward program verification, increased interest has been observed in the use of such techniques in the field of validation. The proposed approaches are very pragmatic and aim to obtain rapid returns on investment without producing unfavorable reactions30 on the part of clients or development teams. These approaches are mainly based on the use of “lightweight” versions of formal methods, better known by the acronym LFMs as a specification aid. In practice, we note that formal methods are not usually implemented across a whole project, but on the most sensitive parts, which may appear to contradict 27 For example, the execution of certain event sequences, the simultaneous occurrence of several events, or the exclusion of simultaneous occurrence of certain events. 28 As an illustration, see the detailed presentation of use of a model-checker in the automatic processing of accident declarations in an insurance company [JAN 99]. 29 This is rarely carried out due to the extent and complexity of the task and the reliability of modern compilers. 30 Although a number of authors support the opposite point of view, it is hard to deny that the image of formal methods remains close to the description provided in two well-known articles dealing with the myths of formal methods: [HAL 90] and [BOW 95a].
122
Simulation & Modeling of Systems of Systems
the affirmation that they are no heavier and no more expensive than classic methods [BOW 95b]. Effectively, it would seem that automatic tools (computerized tools) cannot deal with very large or complex systems; manual methods are not useable in this context either [JAC 01]. In order to profit from the efficiency of formal methods in spite of these problems, the concept of LFMs is based on economy and a more efficient use of resources. The adjective “lightweight”, in this context31, means that these methods may be applied to incomplete specifications (i.e. specifications still under construction) or to a sub-group of existing specifications, possibly limiting ambitions in terms of analysis to a restricted number of properties and by prioritizing the ease of implementation of the language used over its powers of expression. These methods are targeted toward specifications as it is in these cases that the effort to result ratio is highest32. It is a well-known fact that the implementation of an error detection and correction policy as early in the development cycle as possible has an extremely positive effect on the quality and cost of programs. Furthermore, experience has shown that the problem formulation and the delimitation phase, followed by the global specification of the domain of simulation, are especially sensitive and a high percentage of failures observed in the domain of M&S have their origins during this phase of the project [LUT 93]. A natural approach, aiming to profit from the effectiveness of formal methods, consists of using LFMs in a more or less interactive manner to construct high-level formal specifications, then using this formal construction to implement specific tools (theorem proving tools) and explore the properties of these specifications in a rigorous manner (checking for contradictions, incoherence, or incompleteness). This approach of progressive construction of a global formal specification based on the requests of the client also allows improvement on one sensitive point of validation, that of ensuring that the domain of the model to be specified is welladapted to the problems posed (i.e. coherence of the subject and object domains). By providing a rigorous description of the behavior of computerized components, formal specification techniques seem to respond to a certain number of specific problems involved in reuse (development based on a component assembly approach). Moreover, considering the cost of the insertion of these methods in the life cycle of programs and the clear interest of using LFMs in the field of
31 The original elements proposed by Jackson and Wing [JAC 96] are Partiality in Language, Partiality in Modeling, Partial Analysis, and Partiality in Composition. 32 Most of the benefit to be obtained from the use of formal methods resides in the construction of formal specifications (80%). This process only represents 20% of the global effort involved in the use of a formal method [KUH 02].
Credibility in Modeling and Simulation
123
specification, it would be preferable to use them during the development of new components rather than in the use of existing components or simulations. The implementation of formal methods requires particular expertise, in a fairly specialized field, which is generally not part of the field of expertise of the development team33. Training these teams can represent a significant investment which can be hard to justify to the management; the effort generally required to conquer prejudices and resistance when attempting to change established practices should also be considered. Moreover, effectiveness in this domain depends not only on the possession of relevant theoretical knowledge but also on the practical experience of implementing formal methods and an awareness of good practice. In this context, an approach based on IV&V allows the permitted investment in the domain of formal methods (and other validation methods) to be rationalized and also provides a second point of view free from the bias introduced by a too thorough knowledge of the implicit hypotheses of the domain [EAS 98]. Finally, as [ORE 94] reminds us, we should not forget that formal methods relate only to the elements of real systems represented in the models and that, usually, the input data for these models is not itself perfectly known. The “proof” provided by formal methods is thus to be handled with a certain degree of care. 3.4.3. VV&A approaches Overall, we can consider the V&V process as an activity that aims to evaluate and increase confidence in a simulation and the results it produces: confidence assessment methodology [KNE 93]. When this confidence reaches a sufficient level, accreditation is accorded. We need to define on what criteria this confidence will be evaluated, the level of confidence to reach (i.e. the risk to be taken), and the formal process to use. An operational V&V approach should guide the choice of elements considered by V&V actions and the techniques used, but should also specify the organizational setup and the different roles and responsibilities involved. The relationship between the approach and the study and development processes should also be defined: will the V&V process be carried out in parallel (overlay), interlaced with the study and development processes, or designed for a posteriori use (ad hoc method)?
33 We note, however, that formal methods are appearing more and more in computer science training programs [CLA 96].
124
Simulation & Modeling of Systems of Systems
We thus find numerous examples of descriptions of VV&A processes, integrated to a varying degree into the simulation development34 and study process in published literature on the subject. The most representative of these examples is the DMSO’s vision in the case of a new development of a simulation or a model: each stage of development, from the definition of need to installation and testing, via the planning and design stages, is tightly linked to verification and/or validation operations to arrive at an accreditation decision prior to implementation of the simulation. It is, however, interesting to note that the very idealized description of a study process provided by the DMSO, in which the development and V&V phases precede the operational implementation of the simulation, does not really correspond to the reality of work in small teams, characteristic of technico-operational studies, where the general exploratory nature of the studies means that operational use of the simulation is part of the validation so that accreditation may only be pronounced at the end of this process. In the following sections, we shall provide an overview of a number of “classic” approaches or approaches that seem particularly relevant to the context of technicooperational studies. 3.4.3.1. SW-CMM and ISO approaches 3.4.3.1.1. SW-CMM The Capability Maturity Model (CMM), developed in 1987 by the Software Engineering Institute (SEI) at Carnegie Mellon University, is a model of evaluation and evolution of software development processes. It includes five levels of maturity: initial (chaotic), repeatable, defined, managed, and optimizing. The definition of these levels was carried out with reference to good practice observed in businesses with a reputation for good process management. The definitions correspond to the capacity of an organization to develop projects with minimum risk and non-quality costs. For each level, key sectors of activity are identified, alongside conditions that should be fulfilled to fulfill the demands of SEI certification. The success of the CMM led to the rapid emergence of a number of other maturity models based on the same initial approach: SE-CMM (System Engineering), SA-CMM (Software Acquisition), IPD-CMM (Integrated Product Development), and even P-CMM (People: management of human resources). To clarify the situation, the original CMM was renamed SW-CMM (SoftWare), and then the SEI reunited the models as the Capability Maturity Model Integration (CMMI). This latest 34 See the Special Topics of the DMSO/RPG on the relationship between the development paradigm and the VV&A process [RPG 01e].
Credibility in Modeling and Simulation
125
incarnation is compatible with the ISO/IEC 15504 norm for the evaluation of software processes. The CMM was not specifically designed to deal with M&S and credibility problems. It is nevertheless frequently found in this field: certain authors recommend its use to reduce VV&A costs [CON 00], and Balci’s certification process (see below) uses CMM concepts for its indicators. The general idea behind these approaches is that the implementation of development processes which conform to the CMM should lead to an improvement in the quality of the computer programs produced35 which should have a knock-on effect on VV&A. Furthermore, the CMM naturally provides certain essential elements (repeatable procedures, traceability, and so on), which facilitate the installation of an effective VV&A organization. However, the CMM approach lacks theoretical bases. It is based, specifically, on a fundamental hypothesis which turns the correlation observed between certain practices and the quality of products produced into a causal relationship. The CMM allows the quality of a development process to be evaluated, but it does not allow judgment to be passed on specific products. Moreover, certain authors [LEW 92, ART 00] suggest that although the situation in the field of software development seems to have improved, and although SW-CMM may have contributed to this, it does not respond to the essential problem posed by managerial pressure, motivated by cost and delay factors, which are regularly prioritized over crucial technical decisions36. One possible modification to the CMM approach would be to develop, in a similar way to SW-CMM and its clones, a point of reference for good practice in the field of VV&A (a VVA-CMM?). This idea is, however, blocked by two main difficulties: the lack of data on V&V practices and the absence of consensus on criteria to use to compare practices (How can we determine if one practice is better than another?). 3.4.3.1.2. ISO 9000 ISO 9000, a spiritual cousin of the SW-CMM, is a family of norms established by the International Standards Organization, which provides certification for organizations. It does not impose points of reference for good practice (in software development or VV&A), but rather the installation of a clearly identified process 35 SW-CMM level 3 is demanded, in principle, by the DoD for the acquisition of software and notably simulation programs. Higher levels may also be demanded in certain international invitations to tender. 36 The Standish Group attributes the large majority of failures in the field of software to this type of dysfunction.
126
Simulation & Modeling of Systems of Systems
with the ability to evolve progressively toward improvement of the product. ISO 9000, unlike the SW-CMM, is thus used to evaluate the capacity of an organization for progress, whereas SW-CMM is used more to evaluate effective characteristics. The establishment of “quality” processes is cited more rarely than SW-CMM among process-oriented (development or study) VV&A approaches. Similar comments can be made on the two: the interest of an ISO approach is based on the hypothesis that it is possible to correctly evaluate the quality of a simulation (or of a study) by evaluating the production process. 3.4.3.2. RPG on VV&A (DMSO) The DMSO has placed a significant quantity of information at the disposition of the M&S community, freely available on its Website. The RPG document37 brings together a structured group of documents38, which deal at length with the issue of VV&A in simulation. The RPG covers all themes linked to VV&A quite broadly. Among other things, it offers a review of the principal VV&A techniques and guides, definitions, and so on related to the VV&A process and the roles of different participants. However, although the RPG represents a considerable volume of documents, it provides very little information regarding the concrete implementation of VV&A; there is, therefore, a considerable margin for interpretation in its recommendations. In fact, the information provided, while effectively relevant, is usually already well known to the experienced practitioner and, at this level of generality, does not really provide new elements for such individuals. The usual explanation for this is that the RPG exists essentially as a teaching tool, oriented toward those new to the domain of simulation, so it is not designed to respond to the questions of more experienced users. 3.4.3.3. Balci’s evaluation (certification) process Balci proposes a methodology39 that enables operations aiming to certify a complex simulation application to be planned and piloted. This methodology, first proposed in [BAL 01], is essentially based on the use of a hierarchy of indicators (generic or adapted to the problem being considered) developed and implemented by a group of experts. Balci’s approach claims to be essentially pragmatic and aims to
37 First published in 1996 with a second version released in 2000 (the current version). 38 The authors of these documents are key players in the institutions already mentioned. 39 Balcialso proposes a software package, Evaluation Environment, which supports this methodology. See [BAL 02] for a detailed presentation and [BAL 04] for a more recent description.
Credibility in Modeling and Simulation
127
support a certification approach, following evaluation, based on an aggregation of expert judgments. Generally, the approach follows a series of steps: – selection of a group of experts (technical and operational); – creation, in liaison with the group of experts, of a hierarchy of indicators specific to the application and problem under consideration (examples of firstlevel indicators include credibility of the formulation of the problem and global specifications, credibility of the application, the experimental process and the management of the project, credibility of cost analysis, risk analysis, and so on); – use of experts to weigh indicators, for example, via Saaty’s Analytical Hierarchic Process (AHP); – use of experts to fill out a tree view40 and of the AHP to balance expert opinions when several people evaluate the same forms; – production of a report presenting the results of successive aggregations, then accreditation decision based on this report. It is preferable that this approach be carried out in parallel with the project, so the indicators to be used for evaluation should be communicated as soon as possible to those concerned. The hierarchy of indicators is not imposed, but a generic group of 400 elements is proposed, which covers the project/process/product triptych (in which the indicators used for the process aspect are derived from SW-CMM). A guide indicating the techniques to use for each element of the generic tree view is provided, but it remains very general and gives few practical indications as to the effective choice of these techniques and on their aim. This approach, which makes considerable use of expert opinions, is actually very subjective in character despite appearances of objectivity, and for this reason, no consensus has been reached concerning the approach in the M&S community. Although the valuing of each “leaf” of the tree view and the weighing of multiple opinions by the AHP does not pose particular problems, the use of this same AHP for the propagation/combination of evaluations (satisfaction levels) in the tree view raises doubts as to the possible interpretation of values obtained at the highest levels. This is all the more troublesome as the facility of generating aggregated judgments (i.e. scores) present in the tool makes it very tempting to use it to carry out absolute comparisons or evaluations41. 40 The proposed program allows uncertainty concerning evaluations to be considered, which can then be restituted during aggregation. 41 Uses of the type “to be accredited, a study must have a global score above 90” or “study X was better-run than study Y as its score is higher”.
128
Simulation & Modeling of Systems of Systems
3.4.3.4. ITOP WGE 7.2 The activities carried out by the ITOP WGE 7.2 fall into the framework of a Memorandum of Understanding signed between France, United Kingdom, United States, and Germany, with Sweden participating as an observer. The Memorandum of Understanding concerns reciprocal sales of armament materials. The WGE 7.2 has the task of developing V&V procedures to facilitate the mutual acceptance of simulation results by the member nations [INT 02]. The modular structure (V&V Case Concept) designed by the group aims to facilitate and clarify the decision-making process in the field of accreditation. This structure divides information into three sub-groups: – Model case (information on models): arguments concerning the realistic character of the model relative to the planned usage. This information relates to the internal coherence of development products and to the absence of error during successive transformations (e.g. installation specifications). – Data case (information on data): arguments concerning the correct nature of data, including verification (internal consistency, format, and so on) and validation (origin, age, means of collection, and so on). – Use case (information on usage): arguments concerning the correct and appropriate character of the materials, considering the goals being pursued, the conditions of using the model, and the data produced by the model. For each of these three sub-groups, an original structure, linking claims, arguments, and evidence (the Claim–Argument–Evidence (CAE) structure) have been adopted (Figure 3.5): – claims: concern the correct and appropriate character of an element; a claim is generally broken down into sub-claims; – arguments: justification of the reasons for adoption of a particular division of sub-claims or justification of the chosen proofs; – evidence: results of V&V actions which support claims or arguments. One proof can support several claims and/or arguments. To conform to the ITOP, a certain number of claims, arguments, and proofs must be provided. The compulsory elements depend on the nature of the transfer being carried out (models, data, or simulation results). The procedure defines these obligatory minimum sub-groups and provides a suitable workbook template. This minimum hierarchy of indicators may be refined by continuing the breakdown of claims into sub-claims while respecting the model provided. Note that the level of
Credibility in Modeling and Simulation
129
qualification and independence of those responsible for V&V systematically forms part of the information provided in the documentation.
Figure 3.5. CAE structure for the three sub-groups defined by the ITOP [INT 02]42
ITOP documentation also includes a common framework and a group of common definitions for impact levels, V&V levels, and quality level: – Impact levels: these are used to describe the worst possible consequences of misuse of the models, data, and the results produced. These levels, taken from the DERA classification43 [MUE 97], go from “negligible” to “catastrophic”. – V&V levels: these are used to evaluate the effort and the costs required to attain a given level of confidence. – Quality level: this is the global evaluation of the quality of the three dossiers, considering strengths, weaknesses, limitations, and soon. The different phases of the ITOP procedure, with its inputs and outputs, are illustrated in Figure 3.6, taken directly from the official documentation.
42 The encircled “E” is the proof of the sub-claim numbered i.j, which corresponds to argument i, linked to claim i. 43 Defense Evaluation and Research Agency, part of the British Ministry of Defense.
130
Simulation & Modeling of Systems of Systems
Figure 3.6. ITOP V&V procedure [INT 02]
A first version of the reference document, ratified by the various partners involved in the study, was published in 2004, and experiment were carried out on a test-case, the MIC (Modèle Informatisé du Combattant, Computerized Combatant Model, developed by the French DGA in their technical center in Bourges). Other test cases are planned, along with the production of a specific tool to assist in the documentation process. The ITOP approach is based on a structure with variable geometry, which presents the advantage of being flexible; it can be adapted to the problems being studied. However, the problem of propagation of levels of satisfaction in the tree view is not explicitly considered (other than by a logical conjunction of qualifiers of the type satisfied/not satisfied). The ITOP does not allow the level of confidence reached or the convincing character of the available elements to be quantified. In addition, the problem of acceptation is not directly addressed. Furthermore, the minimum decompositions (sometimes compulsory) remain at a very high level, close to the check-list, and the manner of exploring the tree view
Credibility in Modeling and Simulation
131
more deeply or widely is not clearly explained: for example, no questions are posed on the subject of the V&V processes nor on the techniques to be used. The ITOP is a mainly product-centered approach, fundamentally designed for a posteriori work, which allows data to be exchanged in a standardized manner. It is essentially adapted to the specific context of exchanges in the field of qualification tests or Test and Experimentation (T&E). 3.4.3.5. Generalized V&V process (D. Brade) In Brade’s doctoral thesis, A Generalized Process for the Verification and Validation of Models and Simulation Results [BRA 03], he, after providing a critical overview of different approaches to validation, proposed an original V&V process that aims to provide an effective detailed guide for the planning, implantation, and documentation of objectives, activities, and results: the “V&V triangle” (Figure 3.7). The approach is essentially product oriented and does not include the process aspects of the development procedure, except where the hypothesis that the intermediary products of different stages will be effectively available is involved, as the approach is directly based on these intermediary products. In this approach, the study process is divided up, following a classic pattern, into five successive phases. For each of these phases, we consider that two kinds of V&V actions can be carried out: actions that aim to ensure the absence of internal errors and actions aiming to ensure the absence of transformation errors (conformity to output of previous phases).
Figure 3.7. V&V triangle
132
Simulation & Modeling of Systems of Systems
This division aims to enable the mastering of complexity using a judicious break-down of actions and objectives. For each of the 20 sub-phases, Brade analyzed the pre-conditions for use and the inherent interest of the step. He also provided an inventory of potential errors and a list of techniques to use to avoid these problems. As the desired level of credibility is determined in relation to the risk level permitted in the study44, the V&V triangle model allows the intensity of V&V to be adjusted by playing on the redundancies between sub-phases (V&V of transformations) or by accelerating the process by limiting it to the most efficient activities and eliminating redundant actions. Although Brade’s process provides a sufficiently clear typology for the classification of model information and relevant validation actions which may be carried out, it remains very general as far as concrete operation is concerned and gives relatively little information as to the means of identifying elements for validation and the techniques to use. This process, which essentially focuses on verification, is mainly designed for a priori planning of V&V actions and for their implementation in tandem with the development process; this is a configuration quite different from that usually encountered in technico-operational studies, where actions are generally carried out a posteriori or at the same time as development. 3.4.3.6. REVVA The THALESJP 11.20 REVVA project, carried out within the framework of the WEAG/CEPA 11 (Western European Armament Group/Common European Priority Area), aims to “develop a common methodological framework for the VV&A of data, models and simulations”. The project is led by a multi-national group of five countries (France, Denmark, Italy, Sweden, and the Netherlands). The process followed considers the results of previous work and other projects being carried out concurrently (ITOP, SISO, NMSG, DMSO, EUCLID RTP 11.13, and AFDRG/ MEVAS). In developing the REVVA methodology, two main focal points were chosen: the elaboration of an a posteriori VV&A process (a simpler approach than “proactive” methods such as MEVAS) and a focus on the product rather than the process. Approaches that concentrate too heavily on the process45 only give indirect
44 Determined by the decision-maker in relation with the aims and implications of the study. 45 Mainly inspired by software engineering (e.g. SEI CMM).
Credibility in Modeling and Simulation
133
indications as to the level of quality obtained and tend to take responsibility away from those involved46. The REVVA methodology defines an organization, roles, and responsibilities: – it emphasizes the independence of V&V, which should not be the responsibility of the client, nor the provider, but that of a third party, the V&V (or IV&V) agent; – it defines the different roles of those involved in the process and looks at who produces the information used by the V&V agent: - the product supplier, - the end user of the product (the client), - a sponsor or promoter, who finances or finds finance for the simulation as they need the simulation and/or its results, - the V&V agent, independent of the client or supplier, - an accepted authority, independent of the client and the supplier, who has the trust of the client. This methodology also defines a fairly general generic process [VVA 04a, VVA 04b] that can be adapted to the particular needs of specific projects, using the following steps: – based on the planned use of the model, define, by successive decomposition, a group of criteria for acceptance (the Target of Acceptance, ToA) so that the fact of verifying these criteria implies that the model is fit for purpose47; – acquire the necessary information48 on the system being modeled to plan out V&V actions, considering the ToA; – for each ToA criterion, define V&V criteria (Target of V&V, ToVV) considering the information obtained and the existing constraints (budget, time, personnel, and so on), refining the decomposition to reach measurable sizes; –
carry out actions corresponding to the ToVV;
46 The myth that “if the process is correct and the procedures are well-applied, the product must be of good quality” is indeed dubious. 47 A priority level (importance function) should be attributed to each of these criteria. A low priority level would result in summary indicators for the evaluation of achievement of the ToA. 48 Information available, or which may be available, results of prior V&V actions.
134
Simulation & Modeling of Systems of Systems
– evaluate the adequacy (or inadequacy) of the V&V elements obtained in relation to defined criteria; – evaluate the adequacy (or inadequacy) of the V&V elements obtained in relation to the ToA: for each ToA, the person responsible for acceptance considers all linked elements; – evaluate the adequacy (or inadequacy) of the V&V report with regard to the final acceptance decision. REVVA methodology (Figure 3.8) shows a quest for rigor in the logic of the acceptance process, illustrated by the use of ToA and ToVV decompositions to arrive at a decision backed up by solid arguments. This methodology, designed for a posteriori use, is clearly product-oriented, generic, adaptable, and guided by a desire to ensure the independence of evaluations. A large amount of documentation exists to accompany this approach, made up of a number of guides and recommendations.
Figure 3.8. Generic REVVA process [VVA 04b]
Credibility in Modeling and Simulation
135
However, several difficult points can be identified in the practical use of this method: – Construction of the ToA: this process is derived from requirements engineering and is decidedly tricky, even when using ad hoc documentation [TAR 04]. – Construction of the ToVV, an extension of the ToA: REVVA once again suggests taking inspiration from requirements engineering, but indications relating to the choice of techniques to use and to the elements to which they should be applied remain very general [GUI 05, VVA 05]. – The difficult problem of reconstruction/aggregation is not dealt with directly (REVVA suggests the use of multi-criteria methods of analysis). The REVVA approach claims to be pragmatic and offers different, more operational definitions than the classic versions provided by the DMSO for the terms verification, validation49, and acceptance (rather than accreditation). However, although attractive in principle, in practice, its implementation is more suited to simulations for acquisition than to technico-operational studies. The distinctly exploratory character of these studies makes the initial identification of the ToA difficult in cases where important factors are poorly understood. Moreover, this same exploratory characteristic often leads to frequent modifications in the exact definition of the problem, alongside reevaluations of the importance of different factors involved (sometimes calling into question the modeling choices involved). In this context, the use of the REVVA approach naturally leads to delays in reactivity, which would seem to be incompatible with the major time constraints generally encountered in technico-operational studies. 3.4.3.7. V&V guide for the ASCI (Sandia NL): PIRT program 3.4.3.7.1. Origins In the field of operational research, the issue of V&V has, over time, become the object of a certain consensus as to the contours of these domains and common definitions. However, similar preoccupations have only recently emerged in a formalized manner in the field of computational physics (fluid mechanics, solid mechanics, structural dynamics, shock waves, computational chemistry, and so on), where definitions similar to those supplied by the US DoD have been adopted. Here, for example, are the definitions used by the AIAA [AIA 98]: 49 Verification: the process used to build a justified belief in the correctness of a model under constraints (time, cost, knowledge, organization). Validity: the quality of a model which consists of presenting a behavior which cannot be distinguished from that of the system being modeled, for a given experimental framework and with given validation criteria. Validation: the process used to build a justified belief in the validity of a model under constraints (time, cost, knowledge, organization).
136
Simulation & Modeling of Systems of Systems
– Verification: “The process of determining that a model implementation accurately represents the developer’s conceptual description of the model and the solution to the model”50. – Validation: “The process of determining the degree to which a model is an accurate representation of the real world from the perspective of the intended uses of the model”51. It should be noted that in the field of computational physics, concepts and definitions are developed in reference to experimental results and that although the definitions used by the DMSO (and therefore by the AIAA) are fairly widely used, the notion of verification in this context tends to refer more to the quality of the resolution of mathematical equations (most often the digital integration of partial differential equations) than to software engineering52. In 1995, the United States announced their decision to cease designing and testing nuclear weapons. The Accelerated Strategic Computing Initiative (ASCI) program, launched in 1996 [DEP 96], permitted rapid transition from the management of weapons stocks based on tests to a system based on simulation. Within this program, significant consideration was given to V&V processes with the aim of improving the credibility of simulation. This reflection led to the development of an approach, formalized progressively by Oberkampf, Trucano, and Pilch [OBE 00, TRU 02] and presented extensively in [PIL 01]. The approach recommended in this guide is, as far as validation is concerned, essentially based on two key principles: – the creation of a hierarchical decomposition of complex systems into subsystems, for which elementary validation actions will be applied, comparing the predictions of models and experimental data;
50 Very close to the DMSO’s definition: “The process of determining that a model implementation and its associated data accurately represent the developer’s conceptual description and specifications”. 51 Identical to the DMSO’s definition, this definition does, however, present the disadvantage of excluding the activities of researchers comparing model output and experimental data from the field of validation [OBE 02]. Moreover, as Oberkampf points out, the last part of this definition is problematic: the analysts and researchers who compare model output and experimental results do not, then, participate in the validation of this model! Oberkampf therefore proposes a less restrictive definition: “the process which aims to determine the degree of agreement between experimental results and the output of a model”. 52 See the definitions given in [PIL 01]: “Verification is the process of determining that the equations are solved correctly”, and “validation is the process of determining that the equations are correct”.
Credibility in Modeling and Simulation
137
– the execution, using the pre-established hierarchy, of a Phenomena Identification and Ranking Table (PIRT) procedure, which allows the identification of the most important points for validation, thus allowing the prioritization of the relevant experiment. 3.4.3.7.2. Hierarchical decomposition Because of the difficulties encountered in successfully carrying out experiment on complex systems in their entirety and the near impossibility of carrying out such experiment effectively, the recommended approach is based on breaking down complex systems into sub-systems. For each component of the levels being considered, experiment are designed, considering time and resource constraints along with those related to the availability of expertise and the precision required relative to the planned application. The experimental data obtained is then compared to results obtained through the use of models using an appropriate validation metric. For the system as a whole, the data generally relates to a specific real system, available in limited quantities; the real-world experimental conditions are also difficult to determine with precision and may even be unknown for certain parameters. The decomposition of the system into sub-systems generally brings to light those specific components for which the data obtained presents the same limitations as those observed in the system in its entirety. Reference tests (i.e. benchmark cases) are carried out using specific materials representing the principal characteristics of sub-systems. These materials allow a limited sub-group of physical phenomena, acting and interacting in each of the subsystems, to be considered. In this context, experimental conditions can be observed precisely (including estimations of the characteristics of random parameters involved). Unit problem tests are also carried out using special materials: a single, important physical phenomenon is isolated and studied, in which case great precision is available concerning the experimental conditions. 3.4.3.7.3. Identification of important physical phenomena: the PIRT method After carrying out a hierarchical breakdown, we must determine, in the context of limited V&V resources, which phenomena should receive most attention and possibly which experiment should be carried out as a priority. To determine this order of priority, Oberkampf and Trucano suggest the PIRT method, initially developed to evaluate the safety of nuclear reactors [WIL 98]. Used to plan activities, the PIRT method follows five steps, linked to the objectives of the application:
138
Simulation & Modeling of Systems of Systems
– importance of physical phenomena: identification of those physical phenomena which require modeling to attain the objectives of the study, and estimation of their relative importance using a suitable method; – evaluation of conceptual models: evaluation of the suitability of the characteristics used in the conceptual models to model the physical phenomena previously identified in connection with the aims of the study; – evaluation of test sequences: evaluation of the relevance and quality of the tests used for verification operations; – evaluation of the available data: evaluation of the suitability of the experimental data available in connection with the aims of validation; – evaluation of validation actions: identification of the metric to use for these actions and of the level which must be reached for the modeling to be considered satisfactory in relation with the aims of the study. As a whole, the suggested approach (hierarchical breakdown and PIRT) allows the creation of a plan by concentrating efforts on those V&V actions most able to provide the required level of confidence and then provides a framework for piloting the V&V process, evaluating the actions taken in relation to the desired objectives. This approach, presented in some detail in a guide published by the Sandia National Laboratory, directly tackles the problem of priority management, in a context with limited resources, and fully accepts the subjective character of V&V. On this last point, we note that PIRT is certainly a subjective method, but also one which has demonstrated its effectiveness in complex situations with high stakes53. This approach, which applies essentially to simulation in the field of computational physics, remains close to verification in spirit and places the main emphasis on the quality of modeling of physical phenomena. However, although this approach has been very little used outside the fields of nuclear security and the ASCI, its characteristics – particularly its effectiveness and pragmatic qualities – mean that, with certain adaptations necessary to cope with the specificities of technico-operational studies54, the basic principles of the approach may be reused in the validation of technico-operational studies, particularly in cases with strict constraints in terms of time and resources.
53 See, for example, the five following references, cited by [PIL 01]: [SHA 88, HAN 92, ROH 97, WIL 97a, KRO 98]. 54 Complexity of systems, large numbers of people involved, consideration of human factors, chaotic evolution of systems, aggregated models, scarcity of data, and so on.
Credibility in Modeling and Simulation
139
3.4.3.8. Hierarchical validation process: MIRT The major steps of this approach, inspired by the PIRT method55, are the identification of different models to be used in the chosen study scenarios and an inventory of actions already carried out (construction then completion of a Model Identification and Ranking Table, MIRT), then the definition of an order of importance for these models (based on an evaluation of their potential influence on results, considering the aims of the study), and finally the creation, under time and resource constraints, of an action plan for V&V (Figures 3.9 and 3.10). Preliminary Analysis
Simulation objectives Scenarios Stakes
V &V Actions
Choice/Creation of conceptual models
Group conceptual model MOE / MOP
Installation of computerizedmodels
Identification of models
MIRT V&V Plan
Data
Simulation Use
Figure 3.9. Overview and insertion in the study process
Four key principles were used in the creation of this approach: – Acceptance of the intrinsically subjective nature of V&V operations: this subjectivity is seen in the choice of V&V actions (the domain of intervention) and in the way they are carried out (expert advice, experience in the relevant field, intuition, conviction, and so on), as well as in the evaluation of the level of adequacy of the results obtained56, with reference to the problem being studied.
55 For a more detailed presentation, see [RAB 06a] or [RAB 06b]. 56 In spite of numerous attempts to objectivize aggregation/reconstruction processes, these processes are fundamentally subjective in nature, a fact which, given the state of the art, should be considered and managed accordingly.
140
Simulation & Modeling of Systems of Systems
Figure 3.10. V&V action planning loop
– Refocus of the approach on the product rather than on the “product/process/ project” trilogy: the textbook case most frequently encountered by project teams today is the a posteriori use of VV&A processes on inherited products or products developed by a third person, independent of the entity in charge of the study (and not necessarily totally cooperative). Placing the emphasis on the product as a priority allows a direct appreciation of its characteristics, whereas quality-oriented approaches only provide an indirect vision via different indicators, usually formal, and we can only hope that they correlate strongly with credibility. – Gaining the support of the project team and of those managing the study leads to two objectives: the first is to avoid designing a process, which, although effective
Credibility in Modeling and Simulation
141
from a theoretical viewpoint, would be so “heavy” that it would be ineffective in practice as it would be hardly, if at all, followed by the project team. The second point is to aim for high returns on investment (see [DAB 03] on this subject). The approach should thus be relatively simple (low workload, reactivity to evolutions in the problem), explicitly set out (with provision of concrete operational indications), and flexible in methods of application (particularly during the phase of global adequacy judgment) and should cause minimal interference with the development and study processes. – Using available knowledge on the effectiveness of V&V techniques and on factors of credibility. The procurement of a satisfactory level of credibility is a necessary condition for the effort involved in carrying out a study not to be wasted (due to a lack of confidence on the part of the decision-maker). This knowledge is used to select factors to consider in the MIRT, during the evaluation of previous V&V activities as well as when planning what other actions to take. Although this approach provides a formal framework and a procedural guide which facilitate the work involved, several potential problems or points of weakness should be pointed out: – the need for experts in the technical and operational domains within the technico-operational study team, and the requirement for members of the V&V team to be competent in the domains of M&S; – management of relationship between the V&V team and the teams responsible for the initial V&V actions. We need to know if an integral IV&V approach is required, including at the level of implementation of V&V actions, or if a certain level of independent assessment will be tolerated, with acceptance of results produced by a third person; this is not easy to decide; – the credibility obtained reposes in part on the activity of work-groups involving the decision-makers. It is not always easy to ensure this participation. 3.4.4. Responsibilities in a VV&A process In general, there are four ways of carrying out validation actions and attributing the associated responsibilities [SAR 04]: – The most widely observed approach is to give responsibility for VV&A to the development team. – If the development team is small, then another solution would be to involve the final user of the simulation in the VV&A process, making him or her responsible for validation decisions. This strategy has the advantage of increasing the credibility of the simulation.
142
Simulation & Modeling of Systems of Systems
– Using an approach based on the use of scoring systems, in which points are attributed to each validation action carried out. The level of confidence thus obtained depends on the overall score at the end of the process. This kind of approach, rarely used in practice57, has the drawback of presenting a group of intrinsically subjective evaluations in a manner which makes them appear objective. This may lead to excessive confidence in a model presenting a high score (even though nothing guarantees the model is free from major faults) or to comparison of models on the basis of scores obtained to determine which is best. – IV&V: a team that is independent from the development team and from the user (of the simulation and/or the study) is responsible for VV&A. This team works either at the end of the development process or in liaison with the development process (the latter solution is preferable). One proposed approach, which attracts fairly widespread consensus, is based on the use of IV&V [BOE 84]. The potential benefits of this approach include a certain objectivity, early detection of errors, and thus the minimization of costs involved in their correction (in cases where IV&V is carried out in parallel to the development process), a mode of operation which better conforms to expectations and, somewhat unexpectedly, a reduction in variability in the steps of the development process [ART 00]. IV&V requires independence on three fronts for optimal effect: technical, managerial, and financial (a detailed presentation of these rules can be found in [LEW 92]): – technical independence: the use of different personnel and competence profiles in the IV&V and development teams, which ensures that the IV&V process is not directly (or indirectly) subject to pressure from the development process and also provides the benefit of a second approach to the problem being studied and to the proposed solutions, enabling the detection of (even subtle) bias; – managerial independence: IV&V activity must be carried out without interference from the development team or from those directing the project, particularly in the choice of methods and matters of timing. It is important to make sure that the development team is not in a position to influence or filter the conclusions of the IV&V team, which should be transmitted directly to the project management, from which the IV&V team must also be distinct and independent; – financial independence: to avoid all pressure or influence on the course of IV&V activities, the budget accorded to V&V from the outset must remain under the control of those responsible for IV&V or at least be managed by an entity 57 An accreditation methodology, based on this principle, is presented by Balci in [BAL 01]. It uses Saaty’s hierarchy (AHP) and a list of rules.
Credibility in Modeling and Simulation
143
independent of the development team. The global cost of an IV&V approach from the specification phase to the launch of the product is estimated [ART 00] at approximately 10% of the total budget (this is presented as a pessimistic estimation). The added value from this approach in terms of validity and credibility, difficult to quantify, is not included in this estimation. We find various estimations of return on investment (ROI), based uniquely on development costs, which range from 1.2 to 4.958, meaning that the use of IV&V, at least in the cases studied, leads to a global reduction in the cost of projects [DAB 03]. One underlying reason for the use of IV&V is the difficulty for a development team of passing from a modeling approach, where they seek to develop a system close to the real thing and in which success is measured in terms of similarity, to a validation approach where they must seek discordances between the real and modeled systems, where success is based on the capacity to show the weaknesses of the model59. However, although IV&V can combat possible biases which may appear when the same group of people is responsible for development and validation, the approach does have two major difficulties. First, the acquisition of knowledge relevant to the model and the context of the study by the IV&V team represent a considerable amount of work. Second, a good relationship of confidence and cooperation must be established with those responsible for development60. The balance between the increase in objectivity when the system is viewed from the exterior and these two obstacles must guide the choice of a level of externalization for this activity (whether to use a truly external organization, or just another team, distinct from the development team, but still part of the same organization).
58 ROI = (Cost of project with IV&V − cost without IV&V)/Cost of IV&V. In the case cited, we assumed that all problems identified during IV&V would also have been identified in the process without IV&V, but at a later stage. 59 Since the work of Popper [POP 59, POP 69], scientific method has developed progressively from a proof approach (search for an accumulation of facts to back up a theory) to an approach based on refutation (search for facts to disprove a theory). Although it is widely admitted that it is impossible to demonstrate the absolute truth of a scientific theory, but that it is often possible to demonstrate that a theory is wrong, techniques in the domain of validation (and especially the spirit with which they are used) are often based more on the search for concordance between the model and the reality than on a search for discordances and their explanations (see also [WIL 97a]). 60 The development team sometimes resent IV&V as an evaluation of their work, which may, in the long term, damage their reputation and reduce future work opportunities.
144
Simulation & Modeling of Systems of Systems
3.4.5. Levels of validation The determination of the required level of confidence should, in principle, be based on an analysis of possible consequences of errors in the results provided by a simulation or of the misinterpretation of these results: what would be the impact of decisions taken based on this information, and what role does the simulation play in the decision process? Unfortunately, although numerous efforts have been directed toward this issue, there is still no clearly established approach in the domain of technico-operational studies which allows the level of effort required to be quantified in relation to the chosen level of risk. We do, however, find an attempt at methodical classification of impact (from negligible to catastrophic) by domain (human safety, risk to property, environmental impact, financial losses, and so on) in the ITOP WGE 7.2 documentation [INT 02]61 and in [MUE 97] or [FEA 02]. In practice, the level of V&V efforts in the case of technico-operational studies is generally determined by a subjective compromise between the cost and the duration of the actions carried out on the one hand and the evaluation of risks involved and the potential impact of errors on the other hand. It is thus extremely important to involve the final user of the results in this process, as he or she must not only supply a certain amount of essential information but also be able to approve the decisions taken in this area in an informed manner. 3.4.6. Accreditation Accreditation is a decision taken by a clearly identified authority that explicitly recognizes that a simulation (and the data to be used) is acceptable for use in regard to a specific given problem. Concerning formal definitions, note that the usage of the term accreditation occasionally causes problems62, as the corresponding activity is known as certification in software engineering and in the field of quality control in general, and it is part of a well-documented process [BAL 03]. Certain authors, moreover, think that the current definition of the term “accreditation” in the field of simulation has led to a VV&A approach, which concentrates too much on evaluating the fidelity of simulations (product quality). A certain number of recommendations on future evolutions of accreditation focus, therefore, on the necessity to consider, besides the product (i.e. the simulation) itself, the process involved in its production (the software life cycle process), the characteristics of the available documentation, 61 The solicited group of experts also presents a classification of validation actions by level and a correspondence between impact levels and validation levels, which should allow a list of actions to take to be determined in relation to the perceived risks associated with the use of the simulation results. 62 Accreditation may also be referred to as “qualification”, but the former term is more frequently encountered.
Credibility in Modeling and Simulation
145
and the qualification of the individuals involved (technical experts, operators, project managers, and so on). To this end, those responsible for accreditation should be trained not only in V&V techniques but also in Software Quality Assurance (SQA) practices. From a practical point of view, the accreditation decision is based on the results of V&V actions and presupposes, in ideal circumstances, the availability of a metric which would allow the difference between model output and reality to be measured63, along with objective criteria to assess whether or not this difference is compatible with the problem being dealt with64. It is, however, rare for such a metric and/or objective accreditation criteria to exist in the context of technico-operational studies; accreditation decisions therefore remain eminently subjective in this field. 3.5. Conclusions Interest in the issue of validation, which grew rapidly in the mid-1990s, has not waned since, as demonstrated by the number of publications on this theme and the willing participation and leading role taken by the official organisms in charge of M&S65 (notably the launch of several important projects to create operational validation approaches). However, despite the efforts made, no one theory formalizing the validation issue has become standard within the M&S community. On the contrary, a multitude of validation techniques and approaches currently coexist, both on a conceptual level and in actual practice. 3.5.1. Validation techniques In actual fact, although a very large number of validation techniques have been observed and although their characteristics are well-known, there is still no formalized approach to use in making a rational choice, depending on cost of use and performance66, and the information available is essentially an accumulation
63 In concrete terms, for a given level of risk, how can the group of variables to observe and the degree of fidelity to respect be determined? 64 From a decision theory standpoint, the models chosen should lead us to make the same decisions as those we would have reached by observing the real system. In other words, for the chosen experimental dispositive, it should be impossible to distinguish the system from the model [ZEI 02]. In the framework of a “projective” model of the simulation, this comes down to asking if, at the end of the projection, the score or position of different solutions are maintained (in spite of projections restricted to certain observed variables and to the possible imprecision of the projection operation). 65 VV&A is, for example, one of the objectives of the DMSO’s latest M&S master plan. 66 See [KLE 95], but also [SAR 00, STE 02]: 6 years on, this assessment remains unchanged.
146
Simulation & Modeling of Systems of Systems
of subjective recommendations rarely based on a genuine scientific approach (see Glasow’s contribution in [SAR 00]). A certain number of remarks and observations may, however, be made in the light of evaluations of validation techniques that have been carried out and the results of experiment on the use of these techniques in practice (both within and outside the field of technico-operational studies). Concerning the crucial problem of validity of choices relating to conceptual models, and notably their adequacy for the problem being considered (creation and validation of the domain of the models in liaison with the domain of application), it is clear that, in practice, there is an absence of systematic approaches and, from a conceptual point of view, a lack of formalized methods. We currently only have access to informal methods, using the specifications provided by the client, the advice of various experts (technical, operational, and in the field of modeling), and the opinions of the project team. This problem, always sensitive, clearly requires deeper formalization (e.g. in extending the work carried out by the REVVA team67), and more rigorous processes allowing the rational development of specifications (in a similar vein to functional analysis-type approaches or the Triz method) still need to be established. For the second main aspect of validation, validation in the model domain, none of the techniques currently described in the literature on the subject seems to have taken the lead over the others, whether from a theoretical point of view or in observed practice. The DMSO/RPG itself is limited to an essentially descriptive inventory, without providing clear recommendations as to the use of particular approaches or methods depending on the case being considered. In practice, however, a certain number of points for improvement, subjects of consensus within the M&S community, can be identified. These are described as follows: – As the most commonly used approaches are largely informal and based on the subjection of models to expert opinions, the means of choosing these experts, their methods of working, and their evaluation must follow more rigorous rules than those currently used68. In the same broad theme, the means of visualization and more or less global approaches used in this context must be perfected and adapted to this means of functioning, often qualitative and synthetic.
67 In particular, the problem of identification of the relevant aspects of each model and the determination of a “suitable” level of modeling is not dealt with in an entirely satisfactory manner. The choices made at this level are extremely arbitrary and are not based on a formalized approach (this problem is also encountered in the construction of the REVVA ToA–ToVV tree view). 68 See [STE 02, PAC 02].
Credibility in Modeling and Simulation
147
– In quantitative methods, which are more objective, statistical techniques are still considerably underused. These techniques may, however, be used directly in the context of validation of technico-operational studies69, for example, in the identification of influential variables, the study of presumed characteristics of the surface of response (influence of factors and possible interactions), the creation of suitable experimental plans, the evaluation of the quality of explanation provided by the model of observed variables70, and the tests of hypotheses. These techniques have undergone promising developments in recent times for this sort of application (see, e.g. current work in the field of screening). To fill the gap71 between habitual practice in the field of validation and demands in terms of statistics, a reorientation of the competences of simulation practitioners is required, avoiding excessive specialization in the field of software engineering. – Formal methods in general, and LFMs in particular, are the subject of growing interest in the field of M&S (see [STE 02, KUH 02]). The use of LFMs, particularly during the creation of specifications in association with the client72 and the user, seems to be a promising path to follow. Moreover, studies currently in progress on the UML language, and research on convergences between UML and formal languages in particular, also reveal interesting practical perspectives for applications in the field of validation. 3.5.2. Validation approaches Although a significant number of validation approaches are referred to in published works on VV&A, none of the approach has been the subject of general consensus or even been applied to a significant extent73. The DMSO’s RPG/VVA [RPG 01d], which remains the document of reference in the domain, is itself too generalized (because of its pedagogic focus) to effectively guide operational applications. Furthermore, although a good deal of work has been carried out on VV&A, information on the real practices of VV&A teams remains difficult to obtain. 69 For an example, see the case study led by Kleijnen [KLE 95], one of the rare examples of this kind of approach published in open access literature. 70 If the quality of the explanation (adjustment) of variables observed by explanatory variables is judged insufficient, a number of methods can be used to improve the situation, notably by transforming variables. 71 Kleijnen calls this an “abyss” [SAR 00]. 72 A significant part of the difficulties encountered in the field of modeling is due to the difference between the way things are “supposed” to happen and the way they actually work in reality. A lack of credibility comes more often from the difference between the conceptual model and the user’s view of reality than from the difference between the computerized model and the real world (as we see it): “things do not happen that way in real life!” 73 With the exception of approaches limited to a very general presentation of good practice.
148
Simulation & Modeling of Systems of Systems
In the context of extremely variable opinions on techniques and approaches to validation, even the question of defining operational effectiveness criteria remains open. There is no recognized metric that allows different techniques and processes to be compared, and data relating to their effectiveness is virtually inexistent74. No approach similar to that used by the SEI (the originators of CMM-type approaches) in the field of software engineering has been initiated75. The orientations that seem to emerge are essentially pragmatic: selection of effective methods and their use where they have most chance of obtaining significant results (in the context of high demand for ROI). The analysis of approaches used, or recommended, brings up several points, which will be discussed in the following paragraphs. VV&A approaches centered on the product/process/project trilogy in the field of simulation have become widespread under influence from the fields of software engineering and quality assurance76 (see [BAL 00] or [DÉL 02]). The disadvantages of these approaches, used outside of their real domain of effectiveness, have led a number of authors to recommend a more product-oriented approach, while still considering the process aspects of these activities on a secondary level. Although there is currently a strong tendency to entrust the main work of developing simulation tools to an external team and then, possibly, to carry out minor modifications following evolutions in the study, validation processes are, in practice, usually the responsibility of the project team. For these processes, frequently based on the results of validation actions carried out by the development team and on complementary actions carried out by the project team, independent evaluation – as recommended by almost all authors on the subject – is far from being at a satisfactory level. Moreover, the increasing level of technicity required of a simulation software developer on the one hand and by validation techniques on the other means that it seems more and more difficult – even unreasonable – to expect the same individuals to possess the required levels of expertise in both fields. A distinct separation between project teams and those in charge of V&V thus presents considerable advantages and is strongly advised.
74 Essentially due to the classified character of the studies concerned and the absence of obvious economic interest in the collection of data. 75 See [SAR 00, MOR 02, STE 02]. 76 It is easier to ensure that a team respects a procedure than to evaluate the real quality of the work produced. It is also easier to provide a global (aggregated) judgment on the quality of work if we can refer to a well-defined production process.
Credibility in Modeling and Simulation
149
Most recent process-based approaches place an emphasis on the capitalization and reuse of information produced in the course of the V&V process. A large number of studies, in particular, are being carried out with the aim of standardizing the documentation produced and the argumentation used. However, in practice, the results obtained for accreditation, and particularly the traceability expected when making decisions, still show considerable room for improvement. The principles followed and choices made are often based on preconceived ideas or on consensus as to what constitutes good practice77, without the use of objective arguments to prop up these convictions and especially without considering the problem of subjectivity in the perception of credibility of results by the final user. On this last point, note that very few studies have been carried out on factors influencing credibility, despite the fact that this plays a deciding role in the production of effective studies, that is, studies of quality which will also be used by decision-makers. Overall, the proposed approaches are based on fairly extensive planning and usually present characteristics which are mostly incompatible with the intrinsic exploratory character of technico-operational studies, particularly in terms of reactivity and workload78. The life cycle of these studies very often involves rapid evolutions of the problem specification which impose modifications of models (conceptual and/or computerized) or of data and hypotheses (scenarios, doctrines, performances, and so on) in consequence; this in turn leads to frequent reconsideration of the results of initial validation actions, as the relevance of these actions is essentially based on a precise definition of the problem being considered. In this context, the classic sequence of events from the formulation of a question by the decision-maker to the proposition of an answer by the project team is often disturbed. This lack of consideration for issues specific to technico-operational studies is not surprising, as the bulk of the simulation work carried out in the sphere of defense does not fall into this domain, but rather into that of technical simulations, simulations for acquisition, and training simulations, which generally follow a more “traditional” life cycle. In practice, these approaches are often presented in fairly general terms and supply few concrete indications as to their effective use. The approach suggested by the Sandia National Laboratory, while in a different domain, shows a certain level of pragmatism, and some of the principles involved can be used, with a few
77 See Glasow’s presentation on this point in the WSC 2000 VV&A panel [SAR 00]. 78 Faced with intermediary results presented by the study teams while exploring the problem and with the multiple evolutions triggered by the results, decision-makers, themselves involved in fairly reactive processes (even for long-term projects), expect rapid responses from the teams (a few weeks at most).
150
Simulation & Modeling of Systems of Systems
adaptations, in the context of a hierarchical validation process: MIRT. This process, based on an evaluation of VV&A techniques and approaches and a study of credibility factors, was thus developed specifically for use in the domain of technico-operational studies. 3.5.3. Perspectives During the initial phase of increased interest in simulation, the question of VV&A for technico-operational simulations did not receive the attention it deserved (this was generally the case, in fact, for complex simulations for decision making), notably in relation to the difficulty of the problems being considered and the level of effort required. More recently, awareness in this area has increased. The situation is still far from satisfactory79, particularly in the fields of validation and accreditation, but this increased attention and a noticeable increase in efforts made in these domains allow us to hope for significant medium-term improvements. However, the general willingness shown in considering VV&A issues should not make us lose sight of several remaining points of difficulty and of the fact that the challenges encountered when attempting to define a problem and provide solutions are due to the intrinsic nature of certain obstacles, of which we shall cite some of the most problematic. Validation actions require a critical evaluation of the differences between the modeled system and the real system. However, technico-operational studies often apply to future systems or to evolutions of existing systems used in a hypothetical environment and under hypothetical conditions. The evaluation of the distance between a well-defined object, such as the modeled system, and an object which is, at best, “blurry” in these conditions, such as the “real” system (a real system which does not actually exist and which, in the majority of cases, will never exist), is obviously extremely difficult. Furthermore, the reference environment in which the comparison is set is itself virtual and uncertain80. One characteristic of technico-operational simulations, which sets them apart from technical simulations, is the relatively large scale of the scenarios studied. This size leads to the involvement of a large number of entities, themselves made up of more or less elaborate sub-systems. To master the multiplicity of behaviors and the 79 It is clear that although there is general consensus on the importance of VV&A, what has been observed in practice often contradicts declarations of principle. VV&A processes are often considered constraints on the study process rather than assets. 80 What will the future theater of combat be like, how will the adversary react to the introduction of a new system, and how will they respond? Will they develop new strategies and/or modify their own combat systems in turn?
Credibility in Modeling and Simulation
151
complexity of interactions between systems, it is very often necessary to use aggregated models representing, in a simplified manner, several entities or several sub-systems of an entity. In this context, identifying what is the “correct” behavior of (or expected from) an aggregated system and deciding whether or not the simplifications used are suitable (with regard to the theme of the study) are not an easy task. Furthermore, the determination of points of reference for the behavior of entities in a scenario becomes more and more difficult with increases in the number and complexity of models; in this case, expert opinions and intuition should be handled with care (especially when dealing with emergent behaviors in poorly understood systems). The large majority of technico-operational simulations use models to represent human operators (e.g. decision-makers). The use of this type of model poses a number of specific problems, both in the creation of the models themselves81 and the acquisition of the corresponding data and in their validation [DEF 01e]. Effectively, although techniques used to model human factors have advanced considerably over the last few years, their validation remains difficult, notably due to the complexity of the relationships considered and the insufficiency of the data available. Combat situations, as observed in the most common scenarios, are chaotic by their very nature as each combatant attempts to destroy or seriously disturb the normal operation of enemy systems or forces, and modes of command and operation present discontinuities depending on the combat conditions. This non-linearity, combined with high sensitivity to initial conditions, has certain consequences on the capacity of models to predict the outcome of an engagement, as well as on the validation of these models. Very generally, then, technico-operational simulations are used in a context where uncertainty and imprecision rise as the complexity and the number of interconnected systems involved increases. In these conditions, the solution lies not in a leap forward toward increasingly detailed models or in an illusory search for fundamentally inaccessible data, but in a return to a more reasoned use of these tools. The preferential field of application of technico-operational simulations is not, in fact, the prediction of system behavior, but essentially the improvement of understanding of their modus operandi and their interactions with their environment82. A significant proportion of the difficulties we encounter are due 81 Intrinsic complexity, non-linear relationships, and explicit or implicit use of a database on which a perception–decision–action loop is based. 82 In these conditions, it is less interesting to know the results of a simulation than to know why these results were obtained (What hypotheses were used on the operation of real systems and for the different necessary parameters? Were they really justified?) and to adopt a strategy that focuses validation efforts on the explanatory factors thus identified (a posteriori validation of the causality tree).
152
Simulation & Modeling of Systems of Systems
to forgetting this basic principle. The return to a more reasoned use of technicooperational simulations will, of course, involve greater implication of simulation users. In particular, decision-makers must cease to wait for responses from users when making decisions, but must take on their responsibilities in the light of knowledge acquired during their use. 3.6. Bibliography [AIA 98] AIAA, Guide for the Verification and Validation of Computational Fluid Dynamics Simulations, AIAA-G-O77-1998, Reston, Virginia, 1998. [ART 00] ARTHUR J., NANCE R., “Verification and validation without independence: a recipe for failure”, Proceedings of the 2000 Winter Simulation Conference, Orlando, Florida, pp. 859–865, 2000. [BAL 00] BALCI O., ORMSBY W., CARR J., SAADI S., “Planning for verification, validation and accreditation of modeling and simulation applications”, Proceedings of the 2000 Winter Simulation Conference, Orlando, Florida, pp. 829–839, 2000. [BAL 01] BALCI O., “A methodology for certification of modeling and simulation applications”, ACM Transactions on Modeling and Computer Simulation, vol. 11, no. 4, pp. 352–377, 2001. [BAL 02] BALCI O., ADAMS R., MYERS D., NANCE R., “A collaborative evaluation environment for credibility assessment of modeling and simulation applications”, Proceedings of the 2002 Winter Simulation Conference, San Diego, California, pp. 214– 220 (software/modelware tutorials), 2002. [BAL 03] BALCI O., “Verification, validation and certification of modelling and simulation applications”, Proceedings of the 2003 Winter Simulation Conference, New Orleans, Louisiana, pp. 150–158, 2003. [BAL 04] BALCI O., “Quality assessment, verification, and validation of modeling and simulation applications”, Proceedings of the 2004 Winter Simulation Conference, Washington DC, pp. 122–129, 2004. [BOE 84] BOEHM B., “Verifying and validating software requirements and design specifications”, IEEE Software, vol. 1, no. 1, pp. 75–88, 1984. [BOW 95a] BOWEN J.P., HINCHEY M.G., “Seven more myths of formal methods”, IEEE Software, vol. 12, no. 4, pp. 34–41, 1995. [BOW 95b] BOWEN J.P., HINCHEY M.G., “10 commandments of formal methods”, IEEE Computer, vol. 28, no. 4, pp. 56–63, 1995. [BRA 03] BRADE D., A generalized process for the verification and validation of models and simulation results, Doctoral Thesis, University of Bundeswehr, Neubiberg, October 2003.
Credibility in Modeling and Simulation
153
[BRA 00] BRAIN C., “Lies, damn lies and computer simulation”, Proceedings of the 14th European Simulation Multiconference on Simulation and Modelling, Ghent, Belgium, pp. 72–76, 2000. [CLA 96] CLARKE E., WING J., Formal methods: state of the art and future directions, CMU Computer Science Technical Report CMU-CS-96-178, Carnegie Mellon University, Pittsburgh, August 1996. [CON 00] CONWELL C., ENRIGHT R., STUTZMAN M., “Capability maturity models support of modeling and simulation verification, validation, and accreditation”, Proceedings of the 2000 Winter Simulation Conference, Orlando, Florida, pp. 819–828, 2000. [DAB 03] DABNEY J.B., BARBER G., Computing direct return on investment of independent verification and validation, Report for NASA Contract no. NAS2-96024, Titan Corporation, Fairmont, 2003. [DEF 00] Defense Modeling and Simulation Office Recommended Practice Guide (DMSO RPG): Special Topics document – Paradigms for M&S developments, November 2000, http://vva.dmso.mil. [DEF 01a] Defense Modeling and Simulation Office Recommended Practice Guide (DMSO RPG): Reference tools. Glossary, October 2001, http://vva.dmso.mil. [DEF 01b] Defense Modeling and Simulation Office Recommended Practice Guide (DMSO RPG): Core document – The V&V Agent’s Role in the VV&A of New Simulations, May 2001, http://vva.dmso.mil. [DEF 01c] Defense Modeling and Simulation Office Recommended Practice Guide (DMSO RPG): A Discussion of Data Quality for Verification, Validation, and Certification (VV&C) of Data to be Used in Modeling, August 2001, http://vva.dmso.mil. [DEF 01d] Defense Modeling and Simulation Office Recommended Practice Guide (DMSO RPG): V&V Techniques, August 2001, http://vva.dmso.mil. [DEF 01e] Defense Modeling and Simulation Office Recommended Practice Guide (DMSO RPG): Validation of human behaviour representations, September 2001, http:// vva.dmso.mil. [DÉL 97] DÉLÉGATION GÉNÉRALE POUR L’ARMEMENT, La simulation de défense: guidedictionnaire, Ministère de la Défense, Paris, April 1997. [DÉL 02] DÉLÉGATION GÉNÉRALE POUR L’ARMEMENT, Méthode d’Evaluation, de Validation et d’Accréditation des Simulations: manuel de référence version 1.1., Ministère de la Défense, Arcueil, December 2002. [DOD 95] DEPARTMENT OF DEFENSE, U.S. Modeling and Simulation Master Plan, DoD5000.59-P, Under Secretary of Defense for Acquisition, Technology and Logistics, Washington DC, October 1, 1995, https://handle.dtic.mil/100.2/ADA301787. [DOD 03] DEPARTMENT OF DEFENSE, U.S. Modeling and Simulation VV&A, Instruction 5000.61, Under Secretary of Defense for Acquisition, Technology and Logistics, Washington DC, May 2003.
154
Simulation & Modeling of Systems of Systems
[DEP 96] DEPARTMENT OF ENERGY, U.S., Accelerated Strategic Program Initiative – Program Plan, Lawrence Livermore National Laboratory, Los Alamos, 1996, www.llnl.gov. [EAS 98] EASTERBROOK S., LUTZ R., COVINGTON R., KELLY J., AMPO Y., HAMILTON D., “Experiences using lightweight formal methods for requirements modeling”, IEEE Transactions on Software Engineering, vol. 24, no. 1, pp. 4–14, 1998. [FEA 02] FEATHER M., “Infusing and selecting V&V activities”, Proceedings of the Foundations ‘02 Workshop on Verification & Validation, Columbia, South Carolina, October 2002. [GIR 02] GIRARDOT D., JACQUART R., “A proposed evolution of validation definition”, Foundations ‘02 Workshop on Verification & Validation, Columbia, South Carolina, October 2002. [GUI 05] Guidelines for VV&A Techniques (GUIDE), Technological Arrangement for Laboratories for Defence European Studies Joint Program 11.20, November 2004, www.revva.eu. [HAL 90] HALL J.A., “Seven myths of formal methods”, IEEE Software, vol. 7, no. 5, pp. 11–19, 1990. [HAN 92] HANSON G., WILSON G., ORTIZ M., GRIGGS D., “Development of a phenomena identification and ranking table (PIRT) for a postulated double-ended guillotine break in a production reactor”, Nuclear Engineering Design, vol. 136, pp. 335–346, 1992. [HON 98] HONE G., MOULDING M., “Application domain modeling to support the verification and validation of synthetic environments”, Proceedings of the 1998 Spring Simulation Interoperability Workshop, Orlando, Florida, 1998. [INS 98] INSTITUTE OF ELECTRICAL AND ELECTRONIC ENGINEERS, Modeling and simulation (M&S) terminology, Standard 610.3, 1998. [INT 02] INTERNATIONAL TEST OPERATIONS, Verification and validation of models and simulations, Procedure (ITOP WGE 7.2) ITOP 1-1-002 Draft V7, September 2002. [JAC 96] JACKSON D., WING J., “Lightweight formal methods”, IEEE Computer Journal, Roundtable discussion, vol. 29, no. 4, pp. 21–22, April 1996. [JAC 01] JACKSON D., “Lightweight formal methods”, Proceedings of the International Symposium of Formal Methods, Berlin, Germany, March 12–16, 2001. [JAN 99] JANSSEN W., MATEESCU R., MAUW S., FENNEMA P., STAPPEN P. v.d., “Model checking for managers”, 6th International SPIN Workshop on Practical Aspects of Model Checking, Toulouse, France, September 21–24, 1999. [KAM 04] KAM L., L’HOSTIS B., LUZEAUX D., Application aux simulations Défense, Ingénierie des modèles: logiciels et systèmes, Observatoire Français des Techniques Avancées, ARAGO 30 series, Tec & Doc, Paris, pp. 177–204, May 2004. [KAM 10] KAM L., “Model-driven design and simulation”, in Systems of Systems, ISTE, London, John Wiley & Sons, New York, 2010.
Credibility in Modeling and Simulation
155
[KLE 95] KLEIJNEN J., Statistical validation of simulation models: a case study, Tilburg University, Netherlands, Center for Economic Research, discussion paper 42, 1995. [KNE 93] KNEPELL P., ARANGNO D., Simulation Validation: a Confidence Assessment Methodology, IEEE Computer Society Press, Los Alamitos, California, 1993. [KRO 98] KROEGER P., ROHATGI U., JO J., SLOVIK G., Preliminary Phenomena Identification and Ranking Tables for Simplified Boiling Water Reactor Loss of Coolant Accident Scenarios, NUREG/CR-6472, Brookhaven National Laboratory, Upton, New York, 1998. [KUH 99] KUHL F., DAHMANN J., WEATHERLY R., Creating Computer Simulation Systems: an Introduction to the High Level Architecture, Prentice Hall, Upper Saddle River, New Jersey, 1999. [KUH 02] KUHN D., CHANDRAMOULI R., BUTLER R., “Cost effective use of formal methods in verification and validation”, Foundations’02 Workshop on Verification & Validation, Columbia, Maryland, October 2002. [LEW 92] LEWIS R., Independent Verification & Validation: a Life Cycle Engineering Process for Quality Software, Wiley, New York, 1992. [LUT 93] LUTZ R., “Analyzing software requirements errors in safety-critical, embedded systems”, Proceedings of the IEEE Int’l Symposium Requirements Engineering, San Diego, Southern California, January 1993. [MIL 90] Military Operations Research Society, Report on the First Mini symposium on Simulation Validation (SIMVAL), Albuquerque, New Mexico, October 15–18, 1990. [MIL 02] Military Operations Research Society, Report on SIMVAL 2002 Workshop Test & Evaluation, Modeling and Simulation and VV&A: Quantifying the Relationship Between Testing and Simulation, Albuquerque, New Mexico, October 15–17, 2002. [MUE 97] MUESSIG P., LAACK D., WROBLESKI J., “Optimizing the selection of VV&A activities: a risk/benefit approach”, Proceedings of the 1997 Winter Simulation Conference, (introductory tutorial), Atlanta, Georgia, pp. 60–66, 1997. [NOR 98] NORTH ATLANTIC TREATY ORGANIZATION (NATO), Modeling and simulation master plan, document AC/323 (SGMS)D/2, Version 1.0., August 7, 1998, www.rta. nato.int. [OBE 00] OBERKAMPF W., TRUCANO T., “Validation methodology in computational fluid dynamics”, AIAA Fluids 2000 Conference, Article no.2000-2549, Denver, Colorado, June 2000. [OBE 02] OBERKAMPF W., TRUCANO T., HIRSCH C., “Verification, validation, and predictive capability in computational engineering and physics”, Proceedings of the Foundations ‘02 Workshop on Verification & Validation, Columbia, Maryland, October 2002. [ORE 94] ORESKES N., SHRADER-FRECHETTE K., BELITZ K., “Verification, validation, and confirmation of numerical models in the earth sciences”, Science, vol. 263, no. 5147, pp. 641–646, 1994.
156
Simulation & Modeling of Systems of Systems
[PAC 99] PACE D., “Use of subject matter experts (SMEs) in simulation evaluation”, Paper 99F-SIW-018, Fall 1999 Simulation Interoperability Workshop (SIW), Orlando, Florida, 1999. [PAC 02] PACE D., “Subject matter expert (SME)/peer use in M&S V&V”, Proceedings of the Foundations ‘02 Workshop on Verification & Validation, Columbia, Maryland, October 2002. [PIL 01] PILCH M., TRUCANO T., MOYA J., FROEHLICH G., HODGES A., PEERCY D., Guidelines for Sandia ASCI Verification and Validation Plans – Content and Format: Version 2.0, Sandia National Laboratories, SAND2000-3101, Albuquerque, New Mexico, 2001. [POP 59] POPPER K., The Logic of Scientific Discovery, Basic Books, New York, 1959. [POP 69] POPPER K., Conjectures and Refutations: The Growth of Scientific Knowledge, Routledge and Kegan, London, 1969. [RAB 06a] RABEAU R., TORTEL S., DOUET J.-C., AUGE V., Vérification et validation d’une simulation de défense aérienne élargie, Délégation Générale pour l’Armement, technical report RT1-05 no. 41043/DGA/D4S/CAD/CVL/DR, Arcueil, May 2006. [RAB 06b] RABEAU R., Proposition d’une démarche de validation des simulations technicoopérationnelles utilisées comme aides à la décision, Doctoral thesis, University of Versailles-Saint Quentin in Yvelines, 2006. [RAB 07] RABEAU R., “Credibility of defense analysis simulations: identifying influential factors”, International Conference on Software and Systems Engineering and their Applications (ICSSEA’07), Paris, 2007. [REV 04] REVVA Project Presentation (detailed), Technological Arrangement for Laboratories for Defence European Studies Joint Program 11.20, November 2004, www.revva.eu. [ROH 97] ROHATGI U., CHENG H., KHAN H., WULFF W., Preliminary phenomena identification and ranking tables (PIRT) for SBWR start-up stability, report NUREG/ CR-6474, Brookhaven National Laboratory, Upton, New York, 1997. [SAR 84] SARGENT R.G., “Simulation model validation”, in Simulation and Model-Based Methodologies: an Integrative View, Edited by T. I. OEREN, B. P. ZEIGLER and M. S. ELZAS, Springer-Verlag, Heidelberg, 1984. [SAR 00] SARGENT R., GLASOW P., KLEIJNEN J., LAW A., MCGREGOR I., YOUNGBLOOD S., “Strategic directions in verification, validation and accreditation research”, VVA Panel, Proceedings of the 2000 Winter Simulation Conference, Orlando, Florida, pp. 909–916, 2000. [SAR 04] SARGENT R., “Validation, and verification of simulation models”, Proceedings of the 2004 Winter Simulation Conference, (introductory tutorial), Washington DC, pp. 17–28, 2004. [SHA 88] SHAW R., ROUHANI S., LARSON T., DIMENNA R., Development of a phenomena identification and ranking table (PIRT) for thermal hydraulic phenomena during a PWR large-break LOCA, NUREG/CR-5047, Idaho National Engineering Laboratory, Idaho, 1988.
Credibility in Modeling and Simulation
157
[STE 02] STEVENSON D., “Summary of V&V research needs”, Proceedings of the Foundations ‘02 Workshop on Verification & Validation, Columbia, Maryland, October 2002. [TAR 04] Target of Acceptance Guidelines (TOAGUID), Technological Arrangement for Laboratories for Defence European Studies Joint Program 11.20, November 2004, www.revva.eu. [TOL 03] TOLK A., MUGUIRA J., “The levels of conceptual interoperability model”, Proceedings of the 2003 Fall Simulation Interoperability Workshop, Orlando, Florida, 2003. [TRU 02] TRUCANO T., PILCH M., OBERKAMPF W., “General Concepts for Experimental Validation of ASCI Code Applications”, SAND2002-0341, Sandia National Laboratories, Albuquerque, New Mexico, 2002. [VOA 99] VOAS J., “Certification: reducing the hidden costs of poor quality”, IEEE Software, vol. 16, no. 4, pp. 22–25, 1999. [VVA 04a] VV&A Methodological Guidelines Reference Manual (METGU), Technological Arrangement for Laboratories for Defence European studies Joint Program 11.20, November 2004, www.revva.eu. [VVA 04] VV&A Process Specification (PROSPEC) User’s Manual vol. 3, Technological Arrangement for Laboratories for Defence European Studies Joint Program 11.20, November 2004, www.revva.eu. [VVA 05] VV&A Techniques State of the Art (TECH2), Technological Arrangement for Laboratories for Defence European Studies Joint Program 11.20, November 2004, www.revva.eu. [WIL 97a] WILSON G., BOYACK B., LEONARD M., WILLIAMS K., WOLF L., BWR drywell debris transport phenomena identification and ranking table, internal report, Lockheed Idaho Technologies Co., Idaho Falls, September 1997. [WIL 97b] WILSON J.R., “Conduct, misconduct, and cargo cult science”, Proceedings of the 1997 Winter Simulation Conference (doctoral colloquium keynote address), Atlanta, Georgia, 1997. [WIL 98] WILSON G., BOYACK B., “The role of the PIRT in experiment, code development and code applications associated with reactor safety assessment”, Nuclear Engineering and Design, vol. 186, pp. 23–37, 1998. [ZEI 02] ZEIGLER B., SARJOUGHIAN H., “Implications of M&S foundations for the V&V of large scale complex simulation models”, Proceedings of the Foundations ‘02 Workshop on Verification & Validation, Columbia, Maryland, October 2002.
Chapter 4
Modeling Systems and Their Environment
4.1. Introduction A typical simulation architecture shows different categories of models that are found within a simulation (Figure 1.7): − models of the system itself, − models of the system environment, and − models of human behavior, typically that of system operators. It would not be possible to go into detail on these different models here, as there are a number of possibilities depending on the goal and the technical choices made, and our purpose here is not to give an exhaustive presentation of the modeling of all imaginable types of system because it would require a work of thousands of pages. However, it is essential to have a concrete understanding of modeling activity to be aware of the constraints imposed by specific types of model. Therefore, we provide an overview of some of the main techniques used to model time, physical laws, random processes, human behavior, and the natural environment. If the reader wishes to study any of these subjects in more detail, a selection of published literature on the subject is available, and the contents of the bibliography of this chapter would be a good starting point.
Chapter written by Pascal CANTOT.
160
Simulation & Modeling of Systems of Systems
4.2. Modeling time Time is a fundamental variable in a simulation. It determines the evolution of state variables (e.g. the position of an object depends on its velocity and time) and is an event trigger (e.g. release of a bomb at instant t). Time, at least in the real world, is a continuous variable1. Nevertheless, during the implementation of a computerized model, time cannot remain purely continuous as the computer is unable to deal with an infinite number of possible values for a variable. Therefore, time should be sampled discretely for processing. This arbitrary and artificial sampling may be a source of uncertainties and of causality problems. It is a strategic choice relating to the implementation of the model which must be made before modeling begins, as it may have a major impact on the type of model chosen; for example, if we choose a simulation based purely on discrete events, the modeling of laws of evolution using differential equations will raise certain issues. Several different times may be found in a simulation. One is “real-world” time (wall clock) and the other is simulated time, that is, the time in which the simulated system should be found. These times may be synchronized (apart from the sampling factor) or completely de-correlated. In the case of a distributed simulation – that is, a group of attached simulations – a third time may be encountered, federation time, which is supposed to be synchronized with the time used in each federated simulation, but is not necessarily identical as each federated simulation may manage time differently (Chapter 7). 4.2.1. Real-time simulation In a real-time simulation, logical (simulated) time passes at the same speed as real time: one simulated second is equal to one “real” second. This is the case in simulations where the time used is generated by the computer’s internal clock or by any other device outside of the simulation and capable of supplying a temporal reference. Time is, then, in this case, an exogenous variable not controlled by the simulation (e.g. piloted flight simulators). It goes without saying that the respect of real time imposes particularly strong constraints on simulations, as on all real-time systems; we must ensure that the processes necessary to the simulation take place in real time (or faster), requiring dedicated high-performance materials (image generators, powerful processing units, and so on).
1 At least as we perceive it.
Modeling Systems and Their Environment
161
This type of simulation is found specifically in cases where a component part of the simulation is not directly controlled by the simulation motor (piloted simulation, hardware-in-the-loop simulation, and so on). 4.2.2. Step-by-step simulation In the case of step-by-step simulation, time is endogenous to the simulation and thus controlled by it. The logical time (of the simulation) is desynchronized from real time. A simulation of a nuclear reaction might simulate a phenomenon which lasts for a few microseconds (of simulated time) in an hour of calculations (so an hour of real time). A simulation of the evolution of the universe, on the other hand, might involve the passage of several billion years of simulated time in an hour of calculations. This reasoning is valid for all non-real-time simulations. In a step-by-step simulation, time increases in a discrete manner by a value ∆t. In concrete terms, the simulation calculates the state of the system at time T0, then advances the model time to T0 + ∆t, and calculates the new state at this point. It will then progress to T0 + 2∆t, and so on, as shown in Figure 4.1 (the dotted vertical arrows represent events, the graduations show regular time intervals, and the curved arrows show successive values of the simulated time).
t
0 Figure 4.1. Management of time by steps
This type of time management is particularly well suited to systems modeled by continuous state variables, of which the evolution is typically described by differential equations (such as simulation of continuums). Although the processing of events taking place during a time step is feasible, it is difficult to implement. Moreover, the system state is calculated from one time step to another whatever the situation may be, a process which may not be optimal in terms of use of processing power, although it is possible to vary the time steps to adapt the precision of calculation needed. In the case of modeling the flight of a ground-to-air missile, for example, with the aim of studying the probability of intercepting a target, we might use more rapid simulation in the flight phase, then reduce the time steps to increase precision in the terminal phase of flight, more important in determining the success or failure of the mission.
162
Simulation & Modeling of Systems of Systems
4.2.3. Discrete event simulation In discrete event simulation, evolution is controlled by events, which may or may not be random. Events are discontinuities in the evolution of the system, for example, the launch of a missile, the collision between two entities, the arrival of a client in a shop, the failure of a component, and so on (Figure 4.2). The simulation calculates the system state at an instant corresponding to an event
Ε1, then increases its time until the next event (E2) in chronological order. As the interval between two consecutive events is variable (systems modeled using this method are generally stochastic), the progress of time will be irregular, which may complicate the observation of the progress of the simulation. The advantage of this approach is highly efficient in terms of performance, as the system state is only calculated when change occurs. This approach is particularly suitable for certain types of systems governed by events such as industrial processes (production chains), queues (traffic light management and operation of a computer network), or operational security (impact of component failures on a system mission).
t
0 Figure 4.2. Management of time by events
4.2.4. Which approach? The choice of a time-management approach is critical and must be made very early in the modeling process, as it is linked to the choice of modeling approach. Thankfully, the choice of a time-management approach is generally fairly easy, as the type of problem often imposes the time-management structure required. Note that it is possible to mix techniques: events can be processed in a step-by-step simulation, for example, but this is less natural and somewhat limiting. 4.2.5. Distributed simulation When considering a monolithic, isolated simulation, whatever the timemanagement approach used, the value of the variable t is shared by all of the models involved in the simulation.
Modeling Systems and Their Environment
163
When, on the other hand, distributed simulation, that is, a group of simulations working together with a common aim, is used (Chapter 7), the issue becomes more complicated. The different simulations involved will not necessarily manage time in the same manner. For example, a piloted (so real time) flight simulator might interact with a model of the flight of a missile aiming to intercept it, a model working on a step-by-step approach to logical time. This can cause a number of technical difficulties, particularly in terms of synchronization: time does not pass in the same manner in both simulations. The piloted airplane simulation may end up several seconds out of synchronization with the missile, rendering the simulation meaningless. This can trigger various dysfunctions, including causality problems (e.g. the aircraft may continue to carry out a maneuver and launch a countermeasure at time t, despite the fact the missile destroyed the aircraft at time t−ε). This kind of dysfunction rapidly leads to the simulation, becoming meaningless. For this reason, federation-level time-management mechanisms are essential when using distributed simulation. The IEEE 1278 distributed interactive simulation (DIS) standard does not really offer a time-management structure as such and imposes real-time operation on federated simulations. For non-real-time constructive simulations, it is sometimes possible to arrange to operate at k times real time, but this is essentially a makeshift measure. The absence of a time-management mechanism is, moreover, one of the factors that led to DIS becoming obsolete and accelerated its replacement by the IEEE 1516 high-level architecture (HLA) standard, which offers services allowing synchronization of federated simulations. HLA mostly solves the problem of time management in a federation, but does not remove responsibility from federated parties, which must be capable of coordinating themselves with the other simulations involved. It also does not prevent the occurrence of certain temporal problems, for example, causality errors linked to transmission delays for interactions over a network. Distributed simulation is a technique for the construction of simulations from component parts which opens new perspectives in the context of simulations of complex systems and systems of systems, but it poses a certain number of difficulties, of which time management is just one (although significant) example. Chapter 7 describes distributed simulation in more detail, with discussion of its principles and ways of tackling the specific difficulties involved. 4.3. Modeling physical laws 4.3.1. Understanding the system Systems are usually subject to a number of physical laws. Thus, the falling billiard ball (Chapter 1) is subject to the force of gravity and, in the event,
164
Simulation & Modeling of Systems of Systems
air resistance. Most often, these physical laws lead to the creation of a mathematical model, which provides simplified vision of our understanding of these laws and of the system studied. Understanding a system is a key element which enables it to be modeled, and for this reason modeling can never be completely separated from a certain knowledge of the professional culture in which the system operates. This is particularly true in modeling physical phenomena, which demands solid theoretical bases in mathematics and in the scientific discipline associated with the phenomenon (e.g. fluid mechanics, thermodynamics, structural mechanics, detonics, nuclear physics, and so on). 4.3.2. Developing a system of equations A physical phenomenon is most often modeled using a group of differential or partially derived equations, obtained from the known physical laws of the phenomenon. In the example from Chapter 1, the falling billiard ball, we had an ordinary differential equation with just one variable which was easy to solve. In most “realworld” cases, however, laws are much more complex and require the use of partially derived equations, with unknown functions, for example, ∂u ( x, y )/∂x = 0, where u is an unknown function of the variables x and y. This type of equation is extremely widespread in the simulation of physical phenomena and in the sciences in general. Let us take a more complicated real-world example than our falling ball. Let us consider the flow of a viscous homogeneous fluid in a volume. From the law G G of conservation of mass, we can define the equation: ∂ρ /∂t + ∇( ρ u ) = 0. Here, ρ is G the density of the fluid and u is the velocity in Eulerian coordinates of a particle of the fluid. G We may continue by writing the momentum balance equation: ∂ ( ρ u )/∂t + G GG G G G G G GG ∇( ρ u ⊗ u ) = −∇p + ∇τ + ρ f , where p is the pressure, τ is the viscous stress G tensor, and f is the resultant of body forces acting on the fluid (e.g. the force of gravity). Of course, these values are not usually constant in relation to space and time coordinates. We can obtain a new set of equations by writing the energy G GG GG G G GG G G balance equation: ∂( ρ e)/∂t + ∇ [ ρ e + p) u ] = ∇ (τ u ) + ρ f u − ∇q + r , where q is
the heat flow lost through thermal conduction, r is the heat lost per unit of volume through radiation, and e is the energy per unit of mass.
Modeling Systems and Their Environment
165
These different equations constitute a differential formulation of the Navier– Stokes equations, which can take different forms depending on the problem (compressible or non-compressible fluid, flow speed, borderline conditions, and so on) or on the systems of coordinates used. We are not going to resolve the above differential equations here, but this (simple!) example of an equation system widely used in simulation, for example, to calculate the flow of air around a reactor turbine blade, gives an idea of the modeling methods used for continuum mechanics and of the mathematical and highly technical character of the work. Those who are interested in going further into the subject may refer to several published works on this subject, for example, [ARN 04] or [TEN 03].
4.3.3. Discrete sampling of space The equations shown above allow us to find a certain number of values (the model state variables) in an elementary volume of space by resolving a system of partially derived equations. To cover the whole of the real system we hope to model, we shall break down the space using a mesh. The mesh defines a “paving” of the space using elementary volumes known as “finite elements”, which allow us to produce a digital resolution of the problem to find an approximate solution, that is, the state vector values for each elementary volume (e.g. velocity, pressure, and air temperature). The discrete sampling of the space in which our model is defined is a demanding task, as a complex model (e.g. an airplane reactor) can include thousands, even millions of elements, each with its own more or less important state vectors. Numerous computer programs (known as “meshers”) exist, both commercial and open source, which can discretize a three-dimensional (3D) model produced by, for example, computer-aided design (CAD). Figure 4.3 shows how the discretization of the billiard ball with a mesh, developed using the Matlab calculating and scientific simulation program, which includes, amongst other things, tools for manipulating mesh structures.
Figure 4.3. Mesh version of the billiard ball
166
Simulation & Modeling of Systems of Systems
4.3.4. Solving the problem Once the mesh has been built, we use a program known as a “solver”, which solves the equations using a particular digital method (finite elements, finite differences, and finite volumes). Solvers are generally adapted to a particular domain; for example, Fluent is used in fluid mechanics, PAM-Crash is used for impact and crashes (and is popular in the automobile industry), SAMCEF is used in thermo-mechanics, NASTRAN in structural dynamics, and so on. We also find some “general” solvers, such as Matlab or Scilab or the more powerful ABAQUS, integrated with Dassault Systems’ CATIA CAD program, widely used in the automobile and aeronautics industries. These are, of course, just a few representative examples within a large range of available tools. We thus obtain a complement to the modeling and simulation process (Chapter 3), which are as follows: study and dimensioning of the physical phenomena involved, modeling of the system using partially derived equations, creation of a mesh from a geometric model (CAD), possible optimization of the mesh, simulation using a solver, and exploitation of results (graphic visualization, supplementary calculations, and so on). Note that the resolution of systems of differential equations often involves their representation in the form of matrices so that calculations on the matrices are a key element of simulation of continuum situations. Supercomputers used for this kind of application are generally optimized for matrix calculations. This was the case of the Cray supercomputers during the 1990s, which contained processors capable of carrying out operations on vectors. Nowadays, these simulation applications may be optimized for use over a large number of processors, that is, in massively parallel computers (with several hundreds or thousands of processors or cores), clusters (local groupings of several central units to make one big unit) or computing grids (use of processing capacities spread across several sites).
4.4. Modeling random phenomena 4.4.1. Stochastic processes We have seen above how to model systems for which the laws of evolution are governed by deterministic equations. Following a model of this kind, if we carry out the same simulation several times, we will always obtain the same results. The billiard ball, for example, will always fall in the same way and at the same velocity, and will hit the ground at the same instant. In reality, however, not all phenomena are deterministic; the disintegration of an unstable atomic nucleus, for example, or
Modeling Systems and Their Environment
167
the failure of a device, or the result of throwing a die, or the lottery results, or the ground clutter of a radar, are not deterministic phenomena. Note that chance may be subjective; thus, if we could measure the gesture of an individual throwing a die precisely, and if we also had precise information on the structure of the die (shape, distribution of mass, and so on) and the surface onto which it is thrown (roughness, elasticity, and so on), we would be able to predict the result of the throw. Very precise knowledge of a terrain (shape, objects, and so on) might also allow the computation of ground clutter. The disintegration of an atomic nucleus or of a particle, however, is, according to the laws of physics, purely random. Chance may also be used to make decisions. If, for example, two routes are equivalent and we do not know which to take, we could toss a coin and use the results to determine which route to choose. A simulation can operate in the same way. Finally, we may choose to use chance in modeling: rather than carrying out complex calculations to determine the impact of the airburst effect of a shell on a target, we might base our reasoning on the probability of the said target being destroyed. Depending on the aims of the simulation, this may or may not be an appropriate modeling choice. Remember that there is no “best” model for all situations: a model is developed with a given objective in mind.
4.4.2. Use of probability In the real world, a wide variety of phenomena depend on chance: accidents, breakdowns, radio noise, natural disasters, and so on. These phenomena appear to be unpredictable, and at first glance not obvious to model by the laws of physics. This is both true and false. On the one hand, the non-repeatable nature of these events does not allow the use of systems of deterministic equations as in fluid mechanics. On the other hand, there are ways and means of abstracting random phenomena mathematically, using probability. In the first case, note that the random character of a phenomenon may be absolute (e.g. the disintegration of an atomic nucleus described by the laws of quantum physics) or subjective (e.g. the throw of a die, where it is theoretically possible to predict which face will be uppermost when the die stops, if we have information on all the relevant conditions). In the second case, the process is considered to be random though this is not correct from the point of view of physics. We may also choose for a process to be considered as random. For example, to know whether a vehicle traveling through a forest will be detected or not by an observer, we might determine precisely what proportion of the vehicle is visible through the trees (e.g. using ray tracing), but this technique is demanding in terms of
168
Simulation & Modeling of Systems of Systems
processing power. Another, less demanding method consists of estimating, with reference to the density of the forest and the distance between the vehicle and the observer, the probability – say 20% – of the observer seeing the target. Depending on the aims of the simulation, the use of this model may be completely valid. To return to an example given in Chapter 1, let us imagine that we are shooting at a target with a weapon. As we live in the real world and not in a spaghetti western, our all shots will not pass through the same exact hole, and the result of a large number of shots will be something like what is represented in Figure 4.4. However, we note certain symmetry in the distribution of hits. Might this be useful for our model?
Figure 4.4. Result of a large number of shots at a target
Let us attempt a statistical study. We shall begin by establishing a histogram of the number of hits in relation to their abscissa (Figure 4.5). In Figure 4.5, a curve obtained is a vaguely symmetric bell shape, but it is irregular. Now, we shall multiply the number of impacts by 10. The curve becomes smoother (Figure 4.6), and the obtained curve is perfect, and represents the trace of the following function: f ( x) = 1
2 2π e − x / 2
This is a statistical law known as a Gaussian (or normal) distribution, in its reduced version centered on the point N(0,1).
Modeling Systems and Their Environment
169
Figure 4.5. Distribution of hits (1,000 samples)
Figure 4.6. Distribution of hits (10,000 samples)
In concrete terms, this model will enable us to construct a stochastic model of the process: a normal law enables us to reproduce the probability of hitting a certain area of the target, or to calculate the area of the target our ball will hit, using a mathematical function. Of course, these results will change with each shot; our model is, therefore, nondeterministic, a fact which has consequences on the exploitation and use of the simulation. We shall come back to these points later.
170
Simulation & Modeling of Systems of Systems
Nevertheless, using the function f ( x), we are now able to calculate, for example, the number of hits between two abscissas x1 and x2, or the interval [x−τ, x+τ] such that the probability of the ball hitting within this area is 95%. The possibility of reproducing this chance factor, or of calculating the probability of occurrence of a certain event, is clear; for example, if we want to simulate terrestrial combat at platform level, we have the means of calculating the probability that a tank will hit another vehicle, and, by determining the precise (but random!) point of impact, of calculating the damage incurred. The laws of probability are manifold, characterized by values known as parameters. The following table gives some examples. Concerning the various distributions described in this table, we recall certain situations where these laws are encountered: − Normal distribution describes the variations on a value due to a large number of independent and non-preponderant causes, with additive effects (e.g. variations of dimensions of pieces manufactured by the same workstation and variations in the height of human beings). − Poisson distribution gives the number of occurrences of an event over a time period (e.g. the number of clients in a queue at a ticket booth, number of faults in a system in one year, and number of four-leafed clovers in a field). − Exponential distribution describes the lifespan of a phenomenon, or time intervals between events (e.g. the lifespan of an atomic nucleus, or the interval between the arrival of two customers at a ticket booth, or the duration of a repair process). − Uniform distribution describes a discrete random process in which all results are equally probable (e.g. the throw of a die: a = 1, b = 6). − Binomial distribution describes a series of experiments or independent draws with a binary result, for example, succeed/fail (e.g. successive throws in a game of heads or tails, draws where the token is replaced afterwards, and draws in a large population). Name
Parameters
Normal
m: mean
(Gaussian)
σ: standard deviation
Poisson
λ: average number of occurrences of the event in the period T considered
Law of probability f ( x) =
1 e σ 2π
−( x − m) 2 2σ 2
(x: real number) f ( x) = λ k e
−λ
k!
(k: positive integer)
Modeling Systems and Their Environment Name
Parameters
λ= 1 Exponential
µ
171
Law of probability
(µ: mean)
Case of radioactivity: λ is the constant
f ( x) = λ e
−λx
of disintegration Uniform
Binomial
{
1 b−a 0
a≤ x≤b if not
a, b: limits
f ( x) =
N: number of experiments
⎛N⎞ f ( k ) = ⎜ ⎟ p k (1 − p ) N − k ⎝k⎠
P: probability of success
4.4.3. Use of statistics Statistics is a branch of mathematics which offers tools and methods for the study of populations which are, in principle, “vast”, that is, a population or sample large enough for the error accompanying the use of a limited number of units to be negligible, considering the precision required; once again, everything is dependent on the defined objective. In simulation, statistics may be used in various activities: − Data collection: statistics can help us to dimension our needs in terms of data, for example, by evaluating the sample size required to achieve the aims of the simulation in terms of precision and validity. − Construction of the model: by analyzing the data measuring a stochastic phenomenon, it is possible to construct a model of the physical phenomenon using a law of probability. − Validation of the model: statistical tests allow the suitability of a model of a stochastic process for a sample to be verified. The χ 2 (chi-squared) test or the Kolmogorov–Smirnov test is widely used for this purpose in simulation. − Analysis and interpretation of simulation results: the data output of a simulation can be voluminous. Statistical analysis can be used to pick out trends and facilitate their interpretation. The modeling of a stochastic process by laws of probability is a result of the law of large numbers: when we randomly pick out individuals from a large population, the larger the sample, the more closely the statistical characteristics of the sample will conform to those of the population as a whole.
172
Simulation & Modeling of Systems of Systems
Let us take a concrete example. A production chain is responsible for the application of a protective film to an electrical component. The problem is that the film is never uniform throughout the production, and is subject to random variations linked to temperature and pressure variations in the factory, differences between batches of chemical products used, and so on. To create a stochastic simulation of the production chain, we must model the random process that creates variation in the thickness of the film. We therefore measure the thickness, δ, of the film left on a sample of N = 620 films at the end of the chain. We class these thicknesses by groups of 1 µm and xi will be used to designate the average thickness of class i. This produces the following table of measurements, in which, for greater readability, we have separated the measurements into classes: Measured thickness of film δ < 2.5 µm 2.5 µm ≤ δ < 3.5 µm 3.5 µm ≤ δ < 4.5 µm 4.5 µm ≤ δ < 5.5 µm 5.5 µm ≤ δ < 6.5 µm 6.5 µm ≤ δ < 7.5 µm δ > 7.5 µm
Number of instances 8 47 137 211 154 51 12
Graphically, this gives the distribution, which looks rather like something we have seen before in this chapter (Figure 4.7).
Number of samples
Thickness (µm)
Figure 4.7. Histogram of film thickness by group
Modeling Systems and Their Environment
173
We can hypothesize, then, that we should be able to model this process using normal law. To return to the first table presented above, this law involves two parameters, the mean m and the standard deviation σ. We therefore calculate the mean and standard deviation of our sample to provide an approximation for the model: we find that m = 5.06 and σ = 1.18. Using these parameters, we can easily construct the cumulative distribution function of the theoretical law and thus determine the theoretical effectiveness for each class. We have thus constructed a model of the stochastic process with which we can, for example, determine how many samples should be found statistically within a certain thickness interval, or calculate, using random draws, the probability that an object will be defective (too thick or too thin). We must not forget to validate the model: for this, we can use the χ2 statistical test which indicates a probability of adjustment of over 99%. We thus have good reason to consider our model to be valid.
4.4.4. Random generators Random number generators are used to model chance. It is, however, difficult to find a perfect definition of “random number”. Here, we shall consider a sequence to be random if each term is not predictable by someone who does not know the algorithm, and if the spread of terms is uniform and resists all traditional statistical tests. A perfect random number generator would produce a sequence of this kind within an interval known as the domain. In practice, generators of this kind are very difficult to produce. It is possible to construct one, based, for example, on random physical phenomena (e.g. the disintegration of an unstable atom), but for cost reasons, we tend to use deterministic mathematical algorithms to generate pseudo-random number sequences instead. The choice of a random (or pseudo-random) generator is not neutral as it may have an influence on the results and the validity of the simulation. This choice should not, then, be left to chance! The following quality criteria should be kept in mind when choosing a pseudo-random generator: – Demonstrability: the generator must have mathematical and algorithmic bases, which demonstrate its operation and allow its properties to be determined. – Uniformity: it must conform as closely as possible to the law it is supposed to follow. Each value it can take in its domain must be as equiprobable as possible. – Independence: the generator must be free from interference from external factors.
174
Simulation & Modeling of Systems of Systems
– Long period: due to the discrete character of numbers processed by computer, we cannot avoid cycles in the generator, but this must have the longest period possible; the ideal length of this period is equal to the cardinality of the domain (so a period of 232 when generating 32-bit numbers). – Facility of implementation: an algorithm generator which is too difficult to implement may not only cause coding problems but also raise performance and portability problems. – Execution speed: a stochastic simulation may need to make intensive use of the random number generator. In this case, the generator must be as high-performance as possible so as not to slow down the application using it. – Repeatability of draws: we need to be able to reproduce the random sequence from one or more seeds to replay a simulation exactly for analysis or debugging purposes (this appears to contradict the concept of chance, but as we shall see, it is possible). Of course, the perfect generator does not exist. We must, therefore, compromise between these qualities and technical contingencies to find the generator which best responds to particular needs. Thus, a classic linear congruential generator (LCG) is rapid, simple to implant, and repeatable, but produces cycles and does not guarantee perfect uniformity. How, then, do we construct a generator of this kind? Before the spread of computers, statisticians used dice, drew cards, or random number tables. Thus, in 1927, L. Tippett published a table of over 40,000 random numbers taken from British census data. In 1955, the firm RAND published a massive work containing one million random numbers. This work was used by a large numbers of statisticians and simulation experts for several decades. Nevertheless, these tables had the disadvantage of being both finite and too large to be held in the limited memories of the computers of the 1950s and 1960s. One means of obtaining random numbers to infinity was to use specialized machines, such as that developed by Kendall and Babington-Smith in 1939, which produced a table of 100,000 numbers. The RAND table was also calculated using a specially designed machine. The disadvantage of these machines was that they are non-deterministic in nature: a given sequence could not be repeated, which posed problems for the adjustment of a program or the analysis of the execution of a simulation, and the machines did not respond in a satisfactory manner to all the criteria given above. In parallel with the development of automatic computers, mathematicians attempted to develop deterministic mathematical algorithms. In 1946, John von Neumann proposed a deterministic algorithm for the generation of a random sequence. This algorithm, which initially did not use any random elements,
Modeling Systems and Their Environment
175
produced a predictable and thus repeatable sequence of numbers, but with a fairly quick cycle: after a certain number of iterations, a number would appear for a second time and the random sequence would enter into a loop or degenerate. Nevertheless, using von Neumann’s algorithm, N. Metropolis obtained a sequence of 750,000 38-bit numbers before degeneration. Most current random generators use an LCG based on a formula of type X i +1 = (aX i + b) mod m , where: – a, b, and m are determined positive integer constants, respectively, the multiplier, the increment, and the modulo. – X0, known as the seed, will determine the rest of the sequence so that it will be possible to replay the same situation several times; this ensures repeatability. This function generates a recurring sequence of numbers from 0 to m−1 inclusive. This list of numbers will appear to be pseudo-random, following a uniform law of probability, but with a maximal cycle of iterations (the theoretical maximum being m) as, at one moment or another, the generator will inevitably fall into a loop. The quality of the LCG thus depends essentially on the choice of the parameters a, b, and m. Moreover, certain choices can make calculations easier; using a power of 2 for m, for example, we note a considerable acceleration in the speed of calculations on a computer. To go from an LCG generator, which follows a uniform law, to generators of other laws (e.g. exponential), we can proceed in one of several ways, for example, using mathematical transformations. There are a number of other methods for generating random sequences, but for our purposes, we shall continue to use LCG. Other methods may be found in specialized literature on the subject, for example, the classic [KNU 98]. The choice of pseudo-random generator can have an influence on the validity of a generator using stochastic models. The generators provided in the standard libraries of compilers (C++, Java, and so on) or simulation environments (Matlab, Scilab, and so on) are mostly sufficient. Nevertheless, it is important to be aware that a random generator may be biased. In case of doubt, statistical tests may be used for evaluation. [KEN 05] is one example of a study on several random generators, which shows that certain widely used generators are by no means above suspicion.
4.4.5. Execution and analysis of results of stochastic simulations The digital methods known as Monte-Carlo techniques are a group of statistical simulation methods, based on experiments carried out with the help of random
176
Simulation & Modeling of Systems of Systems
generators. They are of considerable importance in the simulation of stochastic processes, supplying not only bases for the modeling of chance, as we have already seen, but also tools for the analysis and optimization of simulation results, for example giving an estimation of error depending on the number of executions (known as replicas or runs). During the Second World War, the United States’ Manhattan Project encouraged numerous talented mathematicians to work on the problem of modeling nuclear reactions. John von Neumann and his colleagues Stanislaw Ulam and Nicholas Metropolis formalized the required mathematical techniques (probability density functions, pseudo-random generators, and so on). Metropolis named these techniques after the famous Monte-Carlo casino in Monaco (Ulam was a passionate poker player). Monte-Carlo techniques developed rapidly during the 1950s and 1960s, initially in the field of nuclear physics, and spread quickly into other applications. The current applications of Monte-Carlo techniques are extremely varied: – finance (forecasts of evolution on the stock exchange), – technico-operational studies (simulation of operational behavior of a system and combats), – nuclear physics (study of the evolution of unstable nuclei, behavior of particles, and so on), – astrophysics (evolution of stellar bodies), – crude oil prospection, – thermodynamics (molecular agitation), – chemistry (study of polymer behaviors), – material sciences (grain growth in a metallic alloy), – biology (prediction of protein structures), and – medicine (cancer treatments). Monte-Carlo techniques are very widely used in the analysis of stochastic systems and are fundamental to the study of complex systems and systems of systems. A certain number of elements are common to all Monte-Carlo type applications: – probability distribution functions: the system being modeled is described through a group of probability densities; – a random or pseudo-random number generator; – a sampling rule establishing how probability density function calculations will be sampled using random values;
Modeling Systems and Their Environment
177
– rules of evolution allowing results to be quantified; – an estimation of error giving an idea of statistical error (variance), specifically in relation to the number of tries (or replicas), very useful during preparation for the execution of the simulation; – techniques to reduce the variance of the estimated result to reduce the necessary calculation time; – computerized calculation algorithms, allowing Monte-Carlo type programs to be executed more efficiently, for example, on parallel and/or vector architectures. How, then, can these techniques be applied to the case we are interested in here? Remember that, in our simulation of a stochastic system, the results produced vary with each execution. We cannot, then, base our analysis on the results of a single execution; we need to obtain a statistically representative sample of results to obtain general conclusions on the system. To do this, we shall run the system N times. The value of N can be determined empirically, by observing the variation of results, or, more methodically, using Monte-Carlo techniques, which allow us to calculate an estimator of statistical results, specifically the mean, a standard distribution and a level of precision depending on the number of replicas. The interest of this is evident: we can obtain the required precision while limiting the calculation time to the strict minimum. Unusual cases may occur during the execution of a simulation. Considerably different from the average, these values can indicate the non-validity of a model, or simply reflect a case of which the probability is very low, but not zero: if we simulated the lottery, for example, with wins (or losses!) as the statistical variable, after a certain number of millions of replicas, the improbable may happen: we may win the jackpot. This is an extreme and very rare case, but it happens in the real world and does not bring the validity of the model into question. Nevertheless, this result will have a significant impact on the results of the simulation (accumulated wins/losses). In such cases, there is no universal rule: we must look to see if these extreme cases may or should be considered, or if they should be dealt with separately to avoid artificially skewing the average statistics using an event which does happen, but very, very rarely. Of course, the validity of such a result should be verified: to do this, the simulation is re-executed using the seed which produced the atypical model, and the replica is replayed in simulation adjustment mode. The importance of the ability to repeat a random sequence (i.e. repeatable executions) and of adjustment functionality (graphic interface, pause, examination of objects and variables, and so on) is clear in this case.
178
Simulation & Modeling of Systems of Systems
4.5. Modeling the natural environment 4.5.1. Natural environment Generally speaking, the environment of a system is made up of external objects, variables, and processes, which influence the behavior of a system. The natural environment is the sub-section of a simulation environment which represents the natural domain in which the system operates (land, sea, atmosphere, or space), the relief of the terrain, planimetric entities, weather conditions, day/night, solar wind, and so on. This environment may be static (e.g. in the case of the relief of a terrain) or dynamic (e.g. the weather).
4.5.2. Environment databases Environment data is the data that describes a simulation environment. Examples of this might include altimetric data, the load-bearing capacity of a bridge, the type of vegetation present, air temperature, wave height, salinity of water, or wind speed.
Figure 4.8. Terrain mesh (from Thales Training & Simulation)
The representation of the physical natural environment is critical for the acquisition of high-quality synthetic environments for training and mission rehearsal. This is one of the most important elements in determining levels of accuracy and realism. Note that, in the case of engineering simulations, realism is rarely needed; accuracy is more important, for example, when modeling the influence of weather conditions on the operation of an infrared sensor.
Modeling Systems and Their Environment
179
An important data is present in this representation of the natural environment, which is as follows: – geographical data (planimetry, altimetry, and bitmap images). Figure 4.8 gives an example of the discrete sampling of a terrain. Following altimetric data gives the summits of the mesh triangles; – CAD data (for infrastructures); – physical data (temperature, humidity, physical resistance, radar reflection, infrared emissions, and so on); – semantic or operational data (place-names, mined or contaminated zones, and so on). This data is managed within a simulation environment database (SEDB). These are specific database systems used by simulations to obtain information on their environment: terrain, weather and so on. Some of this data is fixed and universal, such as mountains and roads; other data may be conjectural and specific to a scenario (climate, danger zones, and so on) or evolve in the course of a simulation (circadian cycle, weather, results of engineering work, or the effects of arms systems). In the case of a distributed simulation (Chapter 7), the use of a shared dynamic environment poses considerable technical challenges. For example, imagine that a simulation of a tank is linked to a simulation of a helicopter using the DIS or HLA standards. The helicopter fires a missile, destroying a bridge that the tank may need to cross: this modification of a planimetric entity must be made known to the simulation managing the tank, or validity of the results of the simulation will be lost (the tank will calmly cross over a bridge which, in fact, no longer exists). Several technical solutions can be used to solve this problem: – Development of a proprietary protocol between the simulations to exchange information on environmental modifications: for obvious reasons of interoperability and reusability of simulations, this should be avoided, although it would probably work. – Use of HLA objects to represent certain planimetric elements: these objects can then be published within the federation, and their state may be modified during execution. – Use of a shared environment server: the issue of a shared natural dynamic environment within a HLA federation can be resolved by the definition of a federated element diffusing information on, for example, the weather (temperature, humidity, precipitation, wind, and so on) throughout the simulation.
180
Simulation & Modeling of Systems of Systems
4.5.3. Production of an SEDB The quantity of data required to complete an SEDB is extremely variable and depends on the objective of the simulation. For a study simulation on a high-altitude drone, for example, a very rough terrain model will suffice, but different cloud layers must be modeled, along with wind and temperature changes depending on altitude. For the simulation of an unmanned ground vehicle, however, clouds will not be particularly important except as modifiers of ambient luminosity, but the terrain must be modeled precisely. To be fully aware of what the production of environmental data represents, we should remember, for example, that a terrain of 200 km2 in Digital Terrain Elevation Data (DTED) level 3 format (resolution of 10 m) already represents 600 Mb of data. Moreover, altimetric data is just one part of the necessary information; we must add other planimetric elements (e.g. obstacles, ditches, and crevasses, which have a resolution distinctly lower than that used for the terrain mesh, and so risk being invisible), the nature of the terrain (meadow, forest, sand, and marsh) or planimetric elements (houses, roads, bridges, rivers, railroads, and rocks). More precise details may be needed: the size of trees (what level of visibility is there within a forest?), hardness and trafficability of the ground (will a vehicle of a certain weight sink?), maximum load tolerated by a bridge (can the bridge take the weight of a 50 ton tank?), names (toponyms) of roads or bridges, infrared or radar signature of the terrain, and so on. The creation of a digital model of terrain is not an easy affair. If there is no pre-existing geographical database or detailed terrain map with the required resolution, the operation may prove long and costly. This incompatibility with urgency has led the United States, essentially through the Defense Mapping Agency (DMA), to invest significant sums in the digitization of potential theaters of operations around the globe, to be able to put mission-planning simulations into place rapidly. Operations such as those in Iraq and Afghanistan have driven other countries to improve their data production capacities. Note that today, publicly accessible digital geography applications, such as Google Earth, offer cheap (or free) off-the-shelf data sources with global coverage. Other geographic data sources, on top of pre-established maps, include specialized institutions (such as the Institut Géographique National and the Centre Géographique Interarmées in France), observation satellites (SPOT and HELIOS), reconnaissance aircraft (Mirage F1CR and UAVs), and data collected in the field. The importance of access to reliable geographical information is illustrated, for example, by the German campaigns in France in 1940: the German armed forces had access to detailed and up-to-date maps of France which contributed to their success. In the former USSR, maps were produced with deliberate errors to avoid providing enemies with information. Unfortunately for the USSR, satellites can be used to
Modeling Systems and Their Environment
181
acquire precise maps, and Western forces could procure accurate maps in any good bookshop. One recent method for the production of 3D terrains is photogrammetry. Using two-dimensional (2D) photos from aerial reconnaissance or satellites and the coordinates of a few points, photogrammetry programs can build a relief model of the terrain and accompanying structures (such as buildings). It is thus possible to construct a 3D model of a zone in a matter of days, if not hours, of acceptable quality for mission rehearsal applications or urban planning (environmental impact studies), for example. Note that photogrammetry is also used in CAD to produce 3D models of pieces and installations. The production of an SEDB follows different processes depending on the type of target simulation. Figure 4.9 shows a typical SEDB production process for a piloted, real-time flight simulator.
Figure 4.9. Production chain for a flight simulator SEDB
The source database is produced from digital maps, satellite photos, or off-theshelf digital terrain models. It contains geographical data (altimetric and planimetric data), textures (geo-referenced aerial photos and photos of façades) and 3D models (e.g. buildings, trees, and vehicles). The modified generic database (or integrated database) is produced from the source database, using a terrain creation tool, by linking different elements of the source database and integrating additional objects and attributes. The capitalization and reuse of simulation data is most interesting at this level; however, there are no universal formats (although Standard Simulator Database Interchange Format (SIF) and SEDRIS Transmittal Format (STF), which we shall discuss in detail in section 4.5.5).
182
Simulation & Modeling of Systems of Systems
The target database is often specific to an image generator as this requires considerable optimization of data. This database is produced by a compiler from the generic database. Then, data is exported to the synthetic image generator (SIG), which applies textures to 3D models, modifying them in real time depending on visibility conditions (ambient light levels, weather, infrared sensors, and so on). The production of these different databases often requires considerable investment, although this varies considerably depending on the level of complexity of the environment required so that the capitalization and reuse of SEDBs should be a permanent consideration during a simulation project; this data is expensive to collect, enrich, and validate. The choice of a reuse process, accompanied by suitable tools, allows gains to be made in terms of productivity and, just as important, quality.
4.5.4. Quality of an SEDB A large number of objects exist within an environmental database, the coherence and realism of which may be affected by various errors: – insufficient precision for the simulation using them (e.g. a terrain model with a resolution within 50 m for a terrestrial vehicle simulation); – excessive precision (wastes processing land and memory, reducing system performance): precision to within 1 m for a supersonic aircraft piloting simulation is not particularly helpful; – data errors: routes crossing rivers without bridges, erroneous altitude coordinates (phenomenon causing absurd peaks or dips), overlapping polygons; – imprecise topographical elements: in the example shown in Figure 4.10, two sections of road which should join up are not connected. This kind of error is invisible in a cartographical view, but disturbs the correct operation of the simulation. A vehicle arriving at this point sees that the route is cut and, believing itself to be in an impasse, turns around; – incomplete data: “holes” in the mesh (not all of the surface of the terrain is described), non-attached polygons; – non-respect of the laws of physics, e.g. a river flowing uphill; – conceptual ambiguity: an environmental element may be seen in different ways. A dam, for example, could constitute an obstacle along a river, or act as a road bridge.
Modeling Systems and Their Environment
183
Figure 4.10. Bad connection between two sections of road
These errors can have a direct influence on results and, therefore, on the validity of the simulation. For example, let us consider two different, but connected, tank simulators. One uses data precise to within 20 m and the other to within 50 m. A hillock of 30 m would be considered by the first simulator, but not necessarily by the second: the second simulator could “see” an enemy which would, in reality, be hidden by the hillock, as according to this simulator, the hillock does not exist! The enemy, on the other hand, will think they are sheltered. This inconsistency between the SEDBs of distributed simulations working together is the source of numerous errors. In the first distributed simulation systems used under SIMNET, such remarkable phenomena as flying tanks and planes moving underground were produced due to a lack of coherence in the SEDBs: the terrain used was not the same in each of the simulations.
4.5.5. Coordinate systems Another difficulty we may encounter is that of the use of different coordinate systems by different data sources. There are, in fact, different ways of expressing the coordinates of a point in space or on the Earth’s surface, depending on the origin of the reference point, on the projection (in the case of a 2D map), and on the shape used to represent the Earth. First and foremost, the Earth is not flat. We can probably ignore this fact when working with a digital model of terrain covering a few square kilometers, but if the simulation terrain covers thousands of square meters, on the other hand, we must take the horizon into account so as not to damage the results. For example, the field of detection of a radar will be limited at low altitude by the simple fact that the Earth’s surface is curved. The use of spherical coordinates (e.g. latitude, longitude, and altitude) allows us to take this “roundness” into account. The Earth, however, is not spherical, as we mentioned in Chapter 2. If a high degree of precision is required for the terrain, we must take this fact into account; this is further complicated by the fact that the
184
Simulation & Modeling of Systems of Systems
Earth’s surface is not even regular and cannot be represented by a classic volume. For this reason, we use the term “geoid”. However, in cases where the precision required is not excessive, we assimilate the Earth’s shape to a so-called “reference” ellipsoid (Figure 4.11), for which a flattening factor is defined: f = (a − b) a . The WGS84 system (World Geodetic System), used by the IEEE 1278 standard (alias DIS), uses a value of f = 1/298.257223563.
b
a
Figure 4.11. The reference ellipsoid
Figure 4.12 illustrates, in a deliberately exaggerated manner, the differences between the ellipsoid, the geoid, and the “real” Earth.
Geoid
Reference ellipsoid
Earth’sphysical surface
Figure 4.12. Ellipsoid, geoid, and the Earth’s surface
Modeling Systems and Their Environment
185
The difference between this ellipsoid and a spherical approximation may seem to be minimum, but it is sufficiently large, for example, a land vehicle may be seen to fly if two simulations using different reference systems are used together, or if a database designed for one coordinate system is used in a simulation designed for a different system without taking suitable precautions. Once the form of the Earth has been determined, we need to choose a way to express the position of a point, that is, a point of reference (center of the Earth? A point on the surface?) and choose between Cartesian (x, y, z) or polar (e.g. longitude ϕ, latitude λ, and altitude h) coordinates. We also need to choose the type of projection for the 2D visual (cartographic) representation, if necessary. In general, changes of reference points are basic functions of all simulations dealing with geographic data, and a number of products are available which offer conversion. The Synthetic Environment Data Representation and Interchange Specification (SEDRIS) standard for the representation and exchange of synthetic environment data includes a very wide range of coordinate conversions in its specifications.
4.5.6. Multiplicity of formats Until the arrival of SEDRIS, there was no international consensus (i.e. at the level of an organism such as the ISO or the IEEE) on a universal format for environmental data. After decades of geographic information systems and simulations, the multiplication of formats was inevitable. We shall examine some of the formats used for digitizing terrain, although this overview will not be exhaustive, with the aim of giving an impression of the diverse range of formats used (alongside a vague knowledge of some of the most important formats for simulation purposes). The Digital Land Map System (DLMS) was created in 1976 by Italy, France, Germany, and the UK, for use in the preparation of missions using the Tornado strike fighter aircraft. DLMS data (elevation and topographical elements) is regularly updated by a multi-national group established for the exchange of digital topographical data. The NATO standard (STANAG) 7074, entitled Feature and Attribute Coding Catalog (FACC), gives a typology of environmental elements and attributes to take into account. It was developed by the Digital Geographic Information Working Group (DGIWG), which also produced the Digital Geographic Exchange Standard (DIGEST) norm, allowing the exchange of vector (topographical elements) and meshed (elevation) data. Inspired by the DLMS, the American military developed the DTED standard, used for data produced by the National Imagery and Mapping Agency (NIMA), part
186
Simulation & Modeling of Systems of Systems
of the US Department of Defense. It may be used in association with the Digital Feature Analysis Data ((DFAD), now obsolete) and Interim Terrain Data (ITD) standards for topographical data. The DTED is essentially a grid of altimetric data for a geographical zone. It is divided into several levels depending on precision. The most used are undoubtedly DTED1 (mesh units of around 100 m) and DTED2 (mesh of 1 × 1 arc seconds between latitude of 50° south and latitude 50° north, 1 × 2 arc seconds elsewhere, giving a mesh of around 30 m). The ITD, a format that was initially supposed to be temporary, describes terrain elements divided into six categories (vegetation, ground composition, slope, drainage, obstacles, and transport routes). These elements may be punctual, linear or surfacic, and include numerous attributes. The DFAD is similar to the ITD, but better adapted to the description of urban zones. The DTED is very widespread, with excellent global database coverage: the whole world is available at DTED0, Europe, the USA and certain parts of Africa and the Middle East at DTED1, and certain regions of particular interest (warzones) are modeled at DTED2. The Vector Smart Map Level 1 (VMAP) format is a format using three levels of resolution, giving vector-type global coverage in 234 co-produced CD ROMs. The sources of this coverage are extremely mixed and the quality is variable in terms of exhaustiveness and precision of localization and semantics, but it currently offers the best global coverage on such a large scale. VMAP, using WGS84 coordinates, replaces the now obsolete DFAD. VMAP data is organized into 10 layers: administrative limits, data quality, altitude, hydrology, industry, ground, population, transport, energy and transmissions, and vegetation. The SIF standard was developed by the US Air Force as part of project 2851, launched in 1984, with the aim of solving the data exchange problem. This project engendered two military standards: MIL-STD 1821 (SIF) and MIL-STD 1820 (Generic Transformed Data Base, GTDB). Following this, the US Army extended SIF to fulfill the needs of its Close Combat Tactical Trainer (CCTT), producing the “SIF++” specification. Both, however, are disadvantaged by the accent they place on data format rather than representation, making them less flexible. Moreover, they are strongly oriented towards virtual simulation (piloted platforms) due to their nature. They are currently giving way, little by little, to SEDRIS. A format derived from SIF, SIF-France, was created for the needs of construction of a French database for Mirage 2000D simulators. SIF-France, which belongs to the French Ministry of Defense, is compatible with the original SIF format. It features some improvements, including increased precision of certain visual aspects. Its purely national aspect, however, strongly reduces its general interest. Evolution towards a more universal format seems inevitable, but in its current state SIF-France has the advantage of allowing sharing and reuse of environment databases (EDB) between several simulators.
Modeling Systems and Their Environment
187
A considerable number of other data formats exist which are used to describe maps or environmental data. What follows is a recapitulative summary (each list of formats is non-exhaustive): – Military vector formats: VMAP, MIL-STD-2407, Vector Product Format (VPF), DNC. – Civilian vector formats: USGS DLG, SHP, DXF, WMF, Postscript. – Military raster/bitmap formats: MIL-STD-2411A, Raster Product Format (RPF), ASRP, CADRG, CIB, NITF, CRP. – Civil raster/bitmap formats: TIFF, GEOTIFF (georeferenced TIFF format), JPEG, GIF, BMP. – Military mesh formats: DTED. – Civil mesh formats: GRID. – Generic 3D database formats: OpenFlight, VRML, X3D (the successor to VRML), DXF, 3DS, IRIS 3D, STF, CDB, MTDB, NPSI (US Navy), Terravista, VBS2. – Native database formats associated with image generators: Openflight (Multigen), TARGET (PT2000, PT3000, SE2000), EaSiest (E&S 4000, E&S 4530, and so on), ModSAF, Space Magic. – Other data exchange formats: Data Interchange Format, Atmospheric and Oceanographic Data Interchange, Geospatial Data Interchange, WMO Standard. Figure 4.13, taken from [GRO 08], replaces these different formats in the process of production of an environment database for simulation, as discussed above. This represents a wide variety of formats, multiplied still further by the popularity of 3D applications, a situation aggravated by the flagrant lack of coordination and the general compartmentalization between different domains in which these formats are used (not only simulation but also geographic information systems, video games, architecture, and CAD) so that no format has taken the lead. At best, we have a smaller number of competing formats in each domain of activity. Finally, remarkable efforts at standardization, such as SEDRIS, have not been as successful as was hoped, in spite of a relevant approach well suited to a wide variety of needs. The market is currently dominated by proprietary formats such as Openflight, Terravista, or 3D Studio, which have a vast field of applications outside the domain of simulation. Thankfully, most SEDB preparation and generation tools are able to import various formats.
188
Simulation & Modeling of Systems of Systems
Figure 4.13. Situation of formats, standards, and standardization initiatives in environmental data production chain (based on [GRO 08])
SEDRIS deserves a little more of our attention, as it is currently the norm that covers the largest need. SEDRIS is to environmental databases what HLA is to simulations: a universal high-level interoperability standard, designed to facilitate the exchange of environmental data, without loss of information, between very different applications. SEDRIS is one of the elements which should, according to the US Department of Defense, allow the acquisition of high-quality synthetic environments at reasonable cost. Remember that HLA is a necessary but insufficient condition for the interoperability of simulation systems: we must also ensure (among other things) that the different components of a federation share the same environment. However, as we have seen, a large number of different data formats and environment models exist. Thus far, when two simulations needed to share an environment, a filter had to be written to allow passage from one representation to the other, or modification of one or other of the simulations was required. This remains feasible, but, if we want to create complex synthetic environments, with 3, 4, or n applications, we would need as many conversion filters as there are possible information flows (Figure 4.14), that is, n(n + 1) 2 .
Modeling Systems and Their Environment
189
Figure 4.14. Conversions between environment databases (EDB) (without SEDRIS)
One solution would be to use a unique environment model. The problem with this is that simulation, as we have already stated several times, is a very diversified field, and needs vary from one domain to another, so that the idea of being able to define a fixed, universal data format for environmental data is almost laughable. The use of a pivot format allows the number of conversions to be limited to 2n, as seen in Figure 4.15.
Figure 4.15. Conversions between environment databases (with SEDRIS)
190
Simulation & Modeling of Systems of Systems
SEDRIS aims to fulfill the needs of all types of simulation applications (virtual, constructive, and so on) in terms of terrestrial, atmospheric, marine, and even spatial environmental data, using the three following principles: − complete and non-ambiguous representation of data, − universal exchange of data without losses, and − standard interfacing with data. The initial version of the standard (from the late 1990s) included the following elements: – Data Representation Model (DRM): data meta-model for the representation of data, linked to a data dictionary. – Environmental Data Coding Specification identification of objects making up the environment.
(EDCS):
definition
and
– Spatial Reference Model (SRM): unified and precise description of coordinate systems. – Application Programming Interface specification (API): specification of the data access interface. – STF: physical data format. The DRM data model allows the user to describe its own environment data (geometry, characteristics, topology, and so on). To do this, the DRM supplies a hierarchy of 300 classes, which aim to cover all needs. One original feature of SEDRIS is the fact that it allows data polymorphism, that is, the information taken from the description of an element can vary depending on the context. Thus, the number of dimensions described for a building may be two if the data is used by a Plan View Display (display of a tactical situation in the form of a flat 2D map), or three for a 3D visualization. DRM is presented in the form of an impressive UML diagram. The EDCS specifies the characteristics of environmental objects. It allows identification of the object type (terrain element, building, vehicle, vegetation, weather, and so on), its characteristics (size, function, color, and so on), and the units of size by which it is characterized (meters, feet, degrees Celsius, and so on). The EDCS includes nine dictionaries: classification, attributes, metadata, enumeration of attributes, units, equivalences, scales, organizations, and groups of organization. It may be used independently of SEDRIS. The SRM describes the spatial reference systems used by SEDRIS (e.g. points of reference and coordinate systems). SEDRIS provides effective conversion between
Modeling Systems and Their Environment
191
not less than 151 different reference points (e.g. from Transverse Mercator to Lambert Conformal Conic projection) with millimeter precision. Like EDCS, SRM can be used independently of SEDRIS. The programming interface (API) of SEDRIS allows readable and writeable access to the SEDRIS transmission medium. The interface takes the context of the application into account when responding and is capable, among other things, of converting coordinates. The API allows the application to be detached from the database; in this way, the architecture becomes transparent for the application. Furthermore, it allows the reuse of the database for another application, or use of the application with another database, to be more easily envisaged. SEDRIS also provides a data transmission format (STF), which is independent of the platform and allows the exchange of environment data files between different systems. This may seem surprising for a “high-level” standard like SEDRIS, but is linked to the particular needs of database producers and users who must archive their data using physical supports (CD-ROMs, DVDs, magnetic strips, and so on). Unlike HLA, SEDRIS is not yet used everywhere, but, as the only truly international and universal standard, various projects and organizations have already adopted it. The standardization of SEDRIS was carried out under the aegis of the ISO. The standards were promulgated in 2005–2006 under the following references: – ISO/IEC 18023-1:2006(E) Information technology – SEDRIS – Part 1: Functional specification. – ISO/IEC 18023-2:2006(E) Information technology – SEDRIS – Part 2: Abstract transmittal format. – ISO/IEC 18023-3:2006(E) Information technology – SEDRIS – Part 3: Transmittal format binary encoding. – ISO/IEC 18024-4:2006(E) Information technology – SEDRIS language bindings – Part 4: C. – ISO/IEC 18025:2005(E) Information technology – EDCS. – ISO/IEC 18026:2006(E) Information technology – SRM. – ISO/IEC 18041-4:2005(E) Information technology – EDCS language bindings – Part 4: C. – ISO/IEC 18042-4:2006(E) Information technology – SRM language bindings – Part 4: C. Note that the EDCS and the SRM have their own ISO references, in recognition of the fact that they may be used in isolation and evolve in parallel.
192
Simulation & Modeling of Systems of Systems Nature
Standards
Initiatives
Name
SIF
SEDRIS
NPSI
Origin
US DoD
US DoD
US Navy
TSPGCDS US Air Force
Date of introduction
1993
1998
2004
2006
2004
In dev’t.
2008
Standard
Yes
Yes ISO
User standard
User standard
User and commercial standard
User standard
Project
Open?
Yes
Yes
Limited
Limited
Yes
No
To verify
Normed format
Normed format, abstract data model
De facto formats
De facto formats
De facto formats
tbc
De facto formats
Weak
Yes, for de facto formats
Yes, for de facto formats
Yes, for de facto formats
tbc
Yes, for de facto formats
Approach
Support by tool editors
Obsolete
CDB US Army
SECORE US Army
GEMS US Army
Table 4.1. Situation of standardization initiatives in 2008 (based on [GRO 08])
On paper, SEDRIS appears to respond to all expectations, but unfortunately it has not yet been universally adopted in industry, for both technical (performance, adaptation of applications, development of converters, and so on) and political reasons (reticence to adopt a non-proprietary open format). In these conditions, then, what chance is there of a true universal standard emerging? In fact, this chance is fairly small, at least in the short and medium term, despite considerable efforts in standardization, which demonstrate the scale of the issue. Table 4.1 gives a summary of the current situation of international standardization efforts. Three different approaches to the production of a standard for SEDBs can typically be distinguished: – Complete definition of a standard (e.g. SIF), possibly using a data meta-model guaranteeing extensibility (such as SEDRIS). – The transformation of a proprietary standard into an open standard; some editors are already considering this, having already made the specifications of their formats public (Openflight and CDB). We could compare this to the initiative taken by Microsoft with C# in a different domain. – The definition of a standard based on existing formats, even if these are proprietary. This is currently the most pragmatic approach, although it still involves certain risks: even if an editor publishes their specifications, nothing guarantees that their policy will not change in the future. Only a norm validated by a recognized
Modeling Systems and Their Environment
193
organism (ISO, IEEE, and so on) with the adhesion of actors in the domain would guarantee the permanence and effectiveness of a standard; the only current complete example of this for SEDBs is SEDRIS, normalized by the ISO, but SEDRIS has not been as widely adopted as was hoped within the simulation community. It is hard to say what route the simulation community will take in the medium and long term. However, even without use of a crystal ball, we can predict that pragmatism will prevail, and standards such as CDB have real advantages in both technical and commercial terms. A deciding factor will be the directions taken by the geographic information systems and video games industries, major actors in the field of simulation due to their economic weight, innovative character and strong links with engineering, urbanism, and simulation. Up to now, these industries have been too widely ignored or mistrusted; they should be integrated into the strategic orientations of modeling and simulation, a process which is, thankfully, already underway in a certain number of organizations.
4.6. Modeling human behavior 4.6.1. Issues and limitations In simulations at system level and above, we are often confronted with the presence of human operators and/or decision makers in the real system; their influence on the behavior of the system varies, but often constitutes a decisive element: – In a road traffic simulation, not all drivers behave in the same manner: they drive at different speeds and hence different cars produce different levels of pollution. – In a factory simulation, workers may be the cause of delays, errors, and manufacturing faults. – In a war game, fundamental decisions in the course of operations must be taken at different levels of command depending on the tactical situation, the doctrine, and the mission. As we shall see, this modeling of human behavior (Human Behavior Representation, HBR) is complex, even though in numerous cases the human element is represented in a simplified or simplistic manner. For example, the failure of a worker in an assembly line may be expressed by laws of probability and reduced to a stochastic process. In war games, the modeling of information and command systems (ICS) is generally carried out using “perfect ICS”, that is, information sent by units to commanders and orders sent by the commanders to units is always perfectly transmitted and understood. This approach, which is still predominant, is subject to
194
Simulation & Modeling of Systems of Systems
increasing criticism due to the modeling needs of capacitive systems and systems of systems, which are based on these processes of communication between different simulated human entities. These aspects thus receive particular attention. Another point which is not very satisfactory in military simulations is the way human factors are considered. All too often, simulated combatants are “perfect”; they are never tired, are never afraid, do not need to eat, do not feel the cold and will fight to the death. Thankfully, this does not necessarily invalidate all military simulations, but it does raise questions when, in sophisticated and costly simulations, we see units being decimated without deviating from their mission or giving in to panic. It is true that this domain of modeling is (as we shall see) still young and in need of work. There are, however, solutions on the market in the category of computer-generated forces (CGF), which can automate entities of a simulation using a human behavior model sophisticated enough for entities to take autonomous actions (i.e. without a human operator in the loop). We also find Semi-Automated Forces (SAF), which are capable of carrying out basic actions (e.g. finding a path between their location and a point indicated to them by the operator), but do not have real autonomy in terms of decision making. To give an example, VR Forces, a CGF product made by MäK Technologies, perform well in terms of modeling human behavior when integrated with products from partner companies, such as Kynapse (Autodesk Games Technology Group), or taking network performance into account with OPNET 3D Network Visualizer (OPNET Technologies Inc.), but it remains a technical product which does not really interact with unit behavior models.
4.6.2. What is human behavior? In a behavior model, we find two basic essential components: – The decision process, based on pre-defined rules and knowledge (e.g. army doctrine) or existing expertise (combat techniques) or even auto-acquired expertise (using deduction based on the results of preceding simulations), which lead, in the absence of other influence, to an objective and determinist decision. Other aspects may be added to these general rules, for example, constraints or circumstantial rules, for example, rules of engagement in a combat situation. – Human factors, which may have a direct or indirect (e.g. by altering the perception of the environment) effect on the decision made, based on more subjective criteria, such as fear, stress, fatigue, culture, and so on. As is often the case, in practice, the distinction between the decision process and human factors is not always clearly defined. Thus, accounting for certain human factors may well be a rule of doctrine, for example, in planning breaks for simulated human elements to rest. More or less importance is accorded to human factors depending on the objective of the simulation using these behavior models. Thus, a simulation used to study
Modeling Systems and Their Environment
195
ergonomics will play a major role on human factors, whereas these same factors will play a minor role (if they are involved at all) in a high-level operational planning simulation. This depends, of course, not only on the specific needs of the simulation, but also on the capacity of designers to provide valid (and validated) models. The objective evaluation of psychological and physiological criteria is an activity that may rapidly become subjective, and requires great scientific rigor, along with an extensive study of data produced by human activity, if such data is available and useable. How can we measure stress, fear, or cultural influences? Two other components, which allow the HBR to interact with its environment, must also be added: – Sensors, through which the simulated human entity will perceive and possibly interpret its environment. – Actuators, with which the entity will be able to act on its environment, for example, a weapons system, associating physical models (performance of the weapon) with action models (operation of the weapon). Figure 4.16 illustrates the operation of a human behavior model: using a perception of its environment and its knowledge base (e.g. operational doctrine), it establishes a decision which is observed in the form of action on its environment. This decision may be influenced by human factors.
Environment
Perception
Human Factors
Decision-making process
Rules Doctrine Knowledge Constraints Etc.
Human behavior model Action
Figure 4.16. General principle of a human behavior model
196
Simulation & Modeling of Systems of Systems
Thus, if a soldier is particularly stressed due to a dangerous environment, he might open fire more easily than in a calmer context. Stress may also have a negative effect on the soldier’s actions (less precise shooting). If the model is capable of learning or includes memory, its knowledge base will be enriched as it is used.
4.6.3. The decision process The decision process aims to reach the goal (e.g. a mission) by considering the environment, the current state of the system (e.g. distance from a goal), and the constraints acting on the modeled human entity (e.g. a doctrine which must be respected). This doctrine is usually expressed in the form of rules, such as “if an enemy is detected and the aim of the mission is to avoid contact, then hide” or “if the red light shows, then reduce the speed of the engine”. Human factors can act on the decision process through the intermediary of rules (“if I’m thirsty, then I drink”). We may use fuzzy logic to nuance these influences and define motivation levels. Take, for example, an operator. We can define his sensation of thirst as a value from 0 to 1, where 0 is “not at all thirsty” and 1 “dehydrated”. At 14h00, the operator’s thirst level is at 0.3: he is not thirsty and considers work as the priority. His thirst, or rather the value of the state variable associated with thirst, will increase progressively in relation to work, heat, and so on. At 15h00, the thirst value is at 0.8 and the operator’s priority becomes the acquisition of suitable liquid to drink, and this need may take a higher priority than work (Figure 4.17).
Figure 4.17. Influence of a human factor on motivation
Modeling Systems and Their Environment
197
At the risk of becoming simplistic, the decision process must also take the probable evolution of the context into account (environment, physiological constraints, and so on). Thus, the human behavior model must be capable of planning its actions. Take, for example, a tank unit in a war game (Figure 4.18). This unit has the mission of reaching a geographical objective. Using short-term reasoning, the unit would choose the northern route. However, this route is more dangerous, as enemy units (on the right in the diagram) have been observed further down this road. To avoid being damaged before reaching the objective, the unit should choose the southern route, which is longer but safer. If a threat is detected only once the route has been taken (e.g. after crossing the bridge on the northern route), the unit must be able to re-plan its itinerary using its new knowledge of the environment, by turning back and taking the southern route. Threat river
Objective
route N
route S
Figure 4.18. Planning an itinerary
4.6.4. Perception of the environment A human entity perceives its environment through sensors: sight, hearing, touch, smell, and possibly taste, without forgetting sensitivity to classic external physical factors: temperature, acceleration, vibrations, humidity, atmospheric pressure (including lack of pressure due to altitude), the circadian cycle (day/night), and so on. This perception can be achieved directly or via the sensors of the platform being operated, for example, a pair of binoculars or an infrared camera. It may be altered by human factors or by constraints linked to workload, to the human environment
198
Simulation & Modeling of Systems of Systems
(hierarchical organization, internal or external human relations) or physiological constraints (e.g. carrying special equipment/tools, confinement, and so on).
4.6.5. Human factors As we have seen, it is essential to consider human factors if we do not wish to reduce the simulation of a human being to a purely cognitive behavioral model. As “human factors” is a very generic term, we shall attempt to provide some clarification. NATO provides the following definition: “the group of scientific facts concerning human diversity and variability, covering biomedical, cognitive, environmental, physiological and psychosocial aspects. Human factors include principles and applications in the domains of ergonomics, personnel selection, job performance and the evaluation of human performance” [NAT 03]. This definition is very broad. For our purposes, human factors will be the group of influences of human origin acting on a system which modify its performance, for example, tiredness and stress, which can alter the judgment of a commander or the precision of a gunman. These human factors may be linked to the external environment, or to the internal state of the entity being considered. [BOU 02] provides an interesting typology of these factors, which we shall reuse here. 4.6.5.1. External moderators These are human factors that have their origins outside the simulated entity, that is, in its environment. Several categories of external modifiers can be identified, depending on their source: – Physical environment: these factors come from the physical perception of the environment by the simulated entity. Among other things, this may include luminosity, noise, temperature, humidity, acceleration, vibrations, altitude, the circadian cycle, and irradiation. – Organization of tasks: temporal constraints (rate of appearance of information, imposed delays), work hours and duration, organization of breaks, hygiene, and dynamism of the workstation. – Human environment: number of people in the team, roles and functions (responsibility, hierarchical organization), qualifications and competences, interactions (e.g. collaborative working or information exchanges). – Physiological constraints: ergonomics of the workstation and its location (confinement, length, and difficulty of movements), carrying of special equipment/ tools (e.g. anti-neurotoxic clothing), security constraints, access to food and water, lack of sleep.
Modeling Systems and Their Environment
199
– Psychological constraints: inherent danger of the situation, social, cultural, political or moral constraints, lack of control, previous damage/wounds, command support. 4.6.5.2. Internal states These moderators are entity-specific. They are often linked to certain external factors listed above. Wearing anti-neurotoxic protective clothing, for example, will increase stress and accelerate the evolution of fatigue, while reducing anxiety linked to the possibility of contamination by neurotoxins. Once again, different categories can be identified: – Variable individual characteristics, which may change over time and in different ways for each individual: fatigue, hunger, thirst, anxiety, fear, stress, moral, combat aptitude, workload, satisfaction, and so on. – Intrinsic (fixed) individual characteristics: age, sex, ethnicity, religion, ethical considerations, IQ, personality (open-mindedness, honesty, extraversion, optimism, and so on), hierarchical responsibilities, qualifications, competences, training, and time on the job.
4.6.6. Modeling techniques 4.6.6.1. Artificial neural networks Neural networks are computerized and sometimes material constructions which operate by mimicking the operation of neurons in the nervous system. This type of model is, therefore, connectionist: the state of each of the N neurons which make up the system evolves depending on input signals (synapses). The output from each neuron is itself linked to one or more other neurons and will influence them. The system state space, therefore, evolves continuously over time, until it reaches a stable state that corresponds to the answer sought. One major domain in which artificial neural networks are found is signal processing (images, words), where their way of operating is particularly useful. This is specifically the case with major parallel processing, due to the fact that each neuron works concurrently with the others and to the distributed and collective character of operations. Artificial neural networks also allow the inclusion of learning abilities in a simulated system. These qualities make artificial neural networks particularly useful in cognitive modeling projects. Nevertheless, artificial neural networks are not a cure-all and suffer from certain technical limitations. It is difficult, given the current state of knowledge, to construct
200
Simulation & Modeling of Systems of Systems
large and efficient neural networks. Either neurons are simulated by a computer, requiring considerable resources in terms of processing power and memory, or they take a material form, for example, using simple processors, which raises problems of connecting neurons (N2 connections are generally required). In the real world, the “neurocomputers” promised in the 1980s barely made it past the laboratory development stage, and the bulk of current artificial neural networks are computerized. Finally, another limitation of artificial neural networks is the difficulty of finding the path that was followed to reach a given result, as they do not reason sequentially, unlike an expert system based on a series of rules. Although this is not important for, say, a video game, it raises considerable problems in applications such as decisionmaking tools, since the decision process is as important as the answer itself. 4.6.6.2. Multi-agent systems Multi-agent systems are a software technology widely used in artificial intelligence. Systems of this kind are based on collaborative work by a group of agents. An agent is an autonomous software entity (typically implemented in the form of a process) with the ability to adapt to its environment, collaborating with other agents to accomplish a task. These agents interact with each other by means of messages. In principle, none of these agents has global perception or control of the system; only through collaboration can the group of agents obtain the behaviors of the complete system to attain a given objective. These agents may simply be reactive, and apply a pre-defined behavior to respond to an external stimulus. They may also be “intelligent” and have the capacity to adapt, or even to learn. Multi-agent architectures are currently very popular as they are extremely effective in implementing intelligence in a system. They are, moreover, particularly well suited to certain uses, such as the modeling of companies, groups of people (crowds and soldiers) or animals, process surveillance systems, and so on. The video games industry also makes considerable use of this type of technology; the “intelligence” of a monster encountered in a virtual dungeon is often the result of the work of a group of agents. 4.6.6.3. Rule-based systems Historically, this was one of the first techniques used in artificial intelligence, specifically in expert systems: expertise and/or behavior is modeled in the form of a logical rule, of the type “if then ”. For example:
Modeling Systems and Their Environment
201
− If the temperature drops below 18°C, then turn on the heating (decisionmaking rule). − If X is the son of Y and Y is the brother of Z, then X is Z’s nephew (cognitive rule). Rule-based systems may be found in combination with other technologies, such as the multi-agent systems seen above, which generally include a rule-base defining the reaction of each agent to its environment and to stimuli. 4.6.6.4. Fuzzy logic Fuzzy logic has also been around for some time, with origins going back to the 1960s. Unlike Boolean logic, which only recognizes two states, TRUE or FALSE, or 0 or 1, fuzzy logic allows us to reproduce uncertainty or graduation in the evaluation of a logical expression by using all values in the interval [0;1]. Thus, to return to the example of air conditioning cited above, we would consider it to be cold below 18°C and hot (or at least acceptable) above. Using fuzzy logic, between cold (0) and hot (1), we can define a whole range of situations corresponding to different temperature ranges: “cold”, “somewhat cold”, “comfortable”, “somewhat hot”, and “hot”, and thus manage the air conditioning in a more optimal manner: if the temperature is somewhat hot, we might just increase fan speed, but if it is hot, we would start the cooling system. There are a number of works available on fuzzy logic, which uses its own system of algebra. It is widely used today in industry, for example, in onboard computer systems (control of the ABS system in a car, of the cooking time on certain microwave ovens) or in robotics (image analysis, obstacle avoidance). In simulation, fuzzy logic is used to reproduce probability situations or graduated effects, such as the influence of human factors on a motivation or a decision. 4.6.6.5. Finite state automatons Finite state automatons (or machines) are computing tools which allow us to model a process or behavior in the form of a graph made up of states and transitions. The automaton passes from a state A to a state B when the transition from A to B is validated, that is, the condition attached to the transition is verified. An air conditioning system, for example, might pass from the state “off” to the state “heating” if the temperature drops below 18°C. Finite state automatons have innumerable uses, including in formal languages, compilers, control for industrial machinery, robots, and so on. In simulation, they may be used to model deterministic behaviors.
202
Simulation & Modeling of Systems of Systems
4.6.6.6. Bayesian networks Bayesian networks are probability graphs that represent variables and their probability relationships. They are used particularly in the design of decisionmaking systems to calculate the probability of relevance of a decision or diagnosis. Their relevance to the modeling of human behavior lies in their capacity to process subjective data, with the ability to include factors of uncertainty, and to propose and evaluate possible decisions. 4.6.6.7. Evolutionary algorithms Evolutionary algorithms are a group of methods inspired by natural evolution and genetics. They are used specifically to implement research and optimization methods. The basic principle consists of finding the solution to a problem evolve in a stochastic manner to find a better response. Evolutionary algorithms can be found not only in robotics (the Pathfinder Martian probe used genetic algorithms in planning its movements), but also in structural mechanics, image recognition, and so on.
4.6.7. Perspectives The previous section gave a rapid overview of some of the principle techniques used in modeling human behavior but, as elsewhere in this chapter, our aim was not to provide a detailed explanation of how to build models, but to give the reader a general flavor of how this modeling might be carried out. For more detail, the reader has only to consult the abundant specialized literature on the subject. The key point to remember is that human behavior is a domain in simulation which is currently the subject of major research projects, and the stakes are high, particularly in the domain of video games, a market worth tens of billions of euros with an ever-increasing demand for “intelligent” entities; the demands placed on these entities are also increasing, with increasingly complex behavior patterns, collective actions, realistic crowds, real-time decision making, and so on. Of course, CGFs profit from the fallout from these technological advances. Take the example of SCIPIO, the training tool used by the general staff of the French Army: this project, ambitious in terms of the atomization of entities, would probably never have succeeded without using technology developed by MASA for use in video games (see [CAN 04]). Nevertheless, we must not forget that, although video games have become the principle source of innovation in terms of HBRs, their demands are not the same as those of “serious” simulations. Games are designed principally to entertain, whereas simulations are designed to respond to a problem (decision assistance, training, and so on). Approximations in games can be tolerated, but this
Modeling Systems and Their Environment
203
is not the case for professional applications which must, among other things, ensure the behaviors reproduced are valid. The MASA technology did provide a solid architectural base for the SCIPIO simulation, but a few years of additional work were needed to obtain an operationally satisfactory product.
4.7. Bibliography [ARN 04] ARNOLD V.I., Lectures on Partial Differential Equations, Springer-Verlag, Heidelberg, 2003. [BAN 95] BANKS J., NELSON B., CARSON J., Discrete-Event System Simulation, Prentice Hall, Old Tappan, 1995. [BAN 98] BANKS J., Handbook of Simulation, Wiley-Interscience, Malden, 1998. [BOI 04] BOISSIER O., GUESSOUM Z. et al., “Systèmes multi-agents: Défis scientifiques et nouveaux usages”, Actes des JFSMA 2004, Hermès, Paris, 2004. [BOR 08] BORGHI R., Les milieux continus multiphysiques hors d’équilibre et leur modélisation, Cépaduès, Toulouse, 2008. [BOU 02] BOUCHE J.-P., Rapport sur la typologie des modèles de comportements humains, technical report, Direction générale de l’armement, Paris, 2002. [CAM 06] CAMPBELL S.L., CHANCELIER J.-P., NIKOUKHAH R., Modeling and Simulation in Scilab/Scicos, Springer-Verlag, New York, 2006. [CAN 01] CANTOT P., Cours sur les techniques et technologies de modélisation et simulation, ENSIETA, Brest, 2002. [CAN 04] CANTOT P., CHIVA E., Transitioning Technologies From Videogames To Wargames: the DirectIA Case, Proc. NATO Modelling & Simulation Group Workshop on Exploitation of Commercial Games for Military Use, RTA, The Hague, 2004. [CAN 08] CANTOT P., Introduction aux techniques et technologies de modélisation et simulation, ENSIETA, Brest, 2008. [CAR 98] CARPENTIER J., “La recherche aéronautique et les progrès de l’aviation”, Revue scientifique et technique de la défense, n 40, Paris, 1998. [COM 05] COMMISSARIAT CEA, December 2005. [COM 06] COMMISSARIAT release, January 2006.
A L’ENERGIE
ATOMIQUE, Le tsunami de Sumatra, press release,
A L’ENERGIE ATOMIQUE,
Le supercalculateur TERA-10, press
[CHE 02] CHEDMAIL P., CAO et simulation mécanique, Hermès, Paris, 2002. [CHI 05] CHIVA E., “Le gaz la fourmi et le terroriste: Simuler les (f)acteurs humains”, in “Faire Campagne en Ville”, Proceedings of the Retour d’Expérience et Prospective Seminar, Revue Doctrines, CDEF, Paris, 2005.
204
Simulation & Modeling of Systems of Systems
[CRA 08] CRAVEUR J.-C., Modélisation par éléments finis: Cours et exercices corrigés, Dunod, Paris, 2008. [CRE 99] CREVIER D., A la recherche de l’intelligence artificielle, Flammarion, Paris, 1999. [DIR 94] DIRECTION GENERALE DE L’ARMEMENT, L’armement numéro 45, special edition on simulation. ADDIM Publication, Paris, December 1994. [DIR 97] DIRECTION GENERALE DE L’ARMEMENT, Guide dictionnaire de la simulation de défense, DGA/STTC, Paris, 1997. [DOC 93] DOCKERY J.T., WOODCOCK A.E.R., The Military Landscape: Mathematical Models of Combat, Woodhead Publishing Ltd, Cambridge, 1993. [FAU 87] FAURE R., KAUFMANN A., Invitation à la recherche opérationnelle, Dunod, Paris, 1987. [FIS 96] FISHMAN G.S., Monte Carlo: Concepts, Algorithms, and Applications, SpringerVerlag, New York,1996. [FIS 73] FISHMAN G.S., Concepts and Methods in Discrete Event Digital Simulation, John Wiley & Sons, Hoboken, 1973. [FOU 87] FOURASTIE J., LASLIER J.-F., Probabilités et statistiques, Dunod, Paris, 1987. [GIB 74] GIBBS G.I., Handbook of Games and Simulation Exercises, Routledge, 1974, Florence, USA. [GRI 86] GRIFFITHS M., Intelligence Artificielle: Techniques Algorithmiques, Hermès, Paris, 1986. [GRO 08] GROUPE ARMEE, DGA, INDUSTRIE SUR LA SIMULATION (ADIS), Rapport final du sous-groupe environnement, Technical Report, draft of 12th December 2008, Direction générale de l’armement, Arcueil, 2008. [KEL 99] KELTON D.W., LAW A.M., Simulation Modeling and Analysis, McGraw-Hill, New York, 2000. [KEN 05] KENNY C., Random Numbers Generators: An Evaluation and Comparison of Random.org and Some Commonly Used Generators, Project Report, The Distributed Systems Group, Trinity College, Dublin, 2005. [KNU 97] KNUTH D.E., The Art of Computer Programming, Vol. 1: Fundamental Algorithms, Addison-Wesley, Old Tappan, 1998. [KNU 98] KNUTH D.E., The Art of Computer Programming, Vol. 2: Seminumerical Algorithms, Addison-Wesley, Old Tappan, 1998. [LIA 97] LIARDET J.-P., Les wargames commerciaux américains, des années soixante à nos jours, entre histoire militaire et simulation, une contribution à l’étude de la décision, Doctoral thesis, University of Paul Valéry, Montpellier, 1997.
Modeling Systems and Their Environment
205
[MON 00] MONTIGNY-RANNOU F., Simulation numérique dans la conception industrielle: exemple de l’aéronautique, course delivered at the École nationale supérieure des ingénieurs des etudes et techniques d’armement (ENSIETA), Brest, 2000. [MON 01] MONTIGNY-RANNOU F., Sur la mécanique des fluides numérique: solveurs et mailleurs, continued professional training course, Office national d’études et de recherches aérospatiales (ONERA), Châtillon, 2001. [NAT 03] NATO RESEARCH AND TECHNOLOGY ORGANISATION, RTO SAS Lecture Series on “Simulation of and for Military Decision Making” (RTO-EN-017), The Hague, 2003. [PER 90] PERLA P., The Art of Wargaming, Naval Institute Press, Annapolis, 1990. [POO 92] POOCH U.W., WALL J.A., Discrete Event Simulation, CRC Press, Oxon, 1992. [RUB 81] RUBINSTEIN R.Y., Simulation and the Monte-Carlo Method, John Wiley & Sons, New York, 1981. [RUS 03] RUSSEL S.J., NORVIG P., Artificial Intelligence: A Modern Approach, Prentice Hall, Upper Saddle River, 2003. [TEN 03] TEMAM R., MIRANVILLE A., Modélisation mathématique et mécanique des milieux continus, SCOPOS collection, Vol. 18, Springer-Verlag, Heidelberg, 2003. [USD 94] US DEPARTMENT Washington D.C., 1994.
OF
DEFENSE, Directive 5000.59-M: DoD M&S Glossary, DoD,
[USD 97] US DEPARTMENT OF DEFENSE, DoD Modeling and Simulation Glossary, Under Secretary of Defense for Acquisition and Technology, Washington D.C., 1997. [WIN 88] WINSTON P.H., Intelligence Artificielle, InterEditions, Paris, 2003. [ZEI 00] ZEIGLER B., PRAEHOFER H., KIM T.G., Theory of Modeling and Simulation, Academic Press, Oxford, 2000. [ZEI 98] ZEIGLER B., BALL G., CHO H., LEE J.S., SARJOUGHIAN H., The DEVS/HLA Distributed Simulation Environment and its Support for Predictive Filtering, University of Arizona, Tucson, 1998.
Chapter 5
Modeling and Simulation of Complex Systems: Pitfalls and Limitations of Interpretation
5.1. Introduction This chapter does not aim or claim to go into mathematical detail concerning the wide variety of models currently used in simulation. Several works have been published on this subject, covering general formalisms and particular instances and providing illustration of a multitude of applications. In this chapter, we shall discuss the way in which certain formal aspects inherent in modeling techniques can, and must, be considered when interpreting the results of a simulation, particularly in relation to their generality and generalization, that is, their validity outside of the domain initially considered. As any experienced user of simulations of fairly complex systems knows intuitively, or as a result of (often painful) failures, great care is required when using simulation results, especially when considering global (or globalizing) properties. It is thus interesting to return to certain fundamental points, such as nonlinearity, computability, and continuous/discontinuous dialectics, and to look at them from the angle of their impact on activities in the field of complex systems engineering. This allows us, among other things, to understand why certain decisions, for example, in the financial sector or in the field of information and communications systems, have a chance (or no chance) of being effective. It also helps us see why it is difficult to understand and explain to the “man in the street” Chapter written by Dominique LUZEAUX.
208
Simulation & Modeling of Systems of Systems
why certain things happen and why certain responses may be used – although these responses are not always the right ones, not because of the incompetence of decision makers but because of the harsh non-linear reality of the situations considered. This difficulty does not stem from the concepts handled, but from their “unnatural” nature: we are not a priori particularly familiar with them. This is certainly due to the nature of the information diffused, but it can also be explained on a deeper level. In physiological terms, to promote the survival of the human species, our sensorycognitive-motor heritage is based on a priori linear representations of situations which are sufficient for reactive adaptation to the environment, although the physiological substrate is fully able to handle non-linearity. The consideration of non-linear aspects may be a luxury left to the cognitive occupations of animals with no fears for their immediate survival; however, we shall go no further down this route, as it would lead us away from the main theme of this study. Moreover, in this case, we are interested in applications resulting from artificial complex systems, that is, developed and created by humans and not by Mother Nature. Curiously enough, those interested in formal properties (stability, sensitivity to certain initial conditions or limits, genericity, and so on) often look at these problems with certain natural ecosystems in mind, from termite mounds – architectural masterpieces, with a thermo-regulated internal environment, capable of rapid reconstruction in case of disaster, a necessary factor in ensuring the survival of the colony – to the way in which shoals of fish instantly avoid predators by collective movements. Other examples include hives, the flight of migratory birds, “fairy rings” (strange circular patterns created by fungi growing in certain fields), and living organisms, magnificent multi-functional edifices which maintain an unvarying form in highly changeable environments and when under attack from all manner of sources. Surprisingly, when considering road networks, intelligent transport systems, or healthcare systems in which patient services are integrated with a card for accessing a digital file and accompanying financial services, we no longer find the same formal rhetoric, and the mathematical artifacts handled are not the same. We may wonder if there is some hidden specificity which attributes complexity to nature and relative simplicity to human constructions. Might this be a planned, or desired, attribute, insofar as the engineer developing a complex system would be able to limit or master, if not identify, degrees of complexity? Or – more crudely – is this just due to ignorance of the complex nature of models?1
1 In all honesty, the very fact of posing the question gives certain elements of a response. However, to exonerate those who may be targeted by our criticisms, note that their approach is a priori defensible – why not aim for simplicity when we can develop a system from its foundations? – but unfortunately vain when faced with the pitiless formal statement.
Modeling and Simulation of Complex Systems
209
5.2. Complex systems, models, simulations, and their link with reality In this section, we shall discuss the need for a common vocabulary and demonstrate the value of introducing the issue from an original angle. 5.2.1. Systems The main norms used in systems engineering (from MIL-STD-499B to the more recent ISO/IEC 15288, the latest norm, from 2008, via the EIA/IS-632, ISO12207, and SE-CMM norms) define a system as “an integrated group of elements – personnel, products, processes – which are interlinked and interconnected with the aim of achieving one or more defined goals”. This definition will be important later when discussing the modeling and interpretation phases. As of now, note that a system is an identified form acting in space and time – hence, the importance of the notions of topology (form of the subjacent space) and dynamics (evolution in time) – made up of a group of interacting elements which change state and interactions – hence the crucial nature of the internal layout of the system, but also of the evolution of this layout over time. Note that the internal structure of a system is eminently recursive, which naturally leads to stratifications and relationships between component parts. These stratifications may be either vertical (in a social system, we find levels made up of individuals, groups, or companies) or horizontal (in the same social system, this would correspond to the relationships between businesses or political parties). We must therefore find formalisms that can either integrate these two notions of structure and dynamics or be able to simultaneously handle different formalisms which are better adapted to one notion or the other2. A system is interesting to observe, study, and handle because of the properties it presents, which give meaning to the planned use in a particular context. From an epistemological perspective, we find two contradictory (or rather complementary) points of view: reductionism and holism. For the reductionist, all properties – including emergent properties, which we will discuss later – can be explained by properties of the level below, down to the lowest level. On the other hand, the holistic approach considers that these properties give the system its “ontological” totality and individuality. These two points of view are certainly extreme, but reflect two different approaches to design, one by extension (based on the enumeration of
2 Note that although these different affirmations appear more or less obvious, they are not always integrated into existing models. Theoretical developments do not always follow the basic principles of common sense.
210
Simulation & Modeling of Systems of Systems
attributes that characterize the object studied) and the other by intension (or by comprehension, i.e. the object is circumscribed by a collectivizing formula along all potential objects). An engineer working on a system is thus confronted by two conflicting points of view: one, analytical, deconstructs the system into simpler elements, which, when brought back together, recreate the system. The other approach, which is teleological and functional, looks at the stability of the object and its reactions to external stimuli. These two ways of seeing the system guide different approaches to modeling. They can be associated directly with the two following principles: – Control principle: developed in the field of synergetics and in its mathematical form by Hermann Haken [HAK 00, HAK 04], this principle affirms that in the majority of complex systems, certain parameters, known as order parameters, govern the dynamics of the system in the sense that other variables are subject to them, that is, express themselves as a function of the order parameters. These order parameters present “slow” dynamic variables, as opposed to “fast” variables which rapidly attain their equilibrium value if the system is disturbed. This means that we can, on the one hand, consider the variation of slow variables to be negligible when looking at the transitory dynamics of fast variables and, on the other hand, only looking at the equilibrium values of fast variables when studying the dynamics of slow variables. When considering the equations, we may express “fast” variables as a function of the dynamic variations of the “slow” variables (i.e. they are subject to them, as mentioned above); the dynamics of the system may also be resolved using only functions affecting “slow” variables, that is, the order parameters (this boils down to a principle of simplification of the same kind as Occam’s Razor). – Reflexivity and coherence principle: if each individual element contributes to the appearance of a collective entity and if, reciprocally, each individual element is subject to the influence of this collective entity, then this cyclic relationship between cause and effect must be stable and self-organizing. The self-organization phenomenon is found in physics (e.g. when neighboring particles create a collective field which depends on their respective quantum states, and the quantum states depend on the collective field), ecological systems (e.g. the dynamic balance between predators and prey, where a break in the cycle can potentially lead to the extinction of a species), or social systems (democracy allows each person to develop political activities, and in return, the presence of this activity allows democracy to exist; the breakdown of this reflexive process of contribution is a classic sign of a dictatorship). The first principle shows the plurality and the hierarchy of scales of dynamic evolution within global couplings, directly affirming the stability of this hierarchy. The second principle is based on the stability of the cyclical structure of the form
Modeling and Simulation of Complex Systems
211
(in the Kantian sense of the term) of the system. We note, indirectly, the opposition between a dynamic hierarchy, which naturally leads to a reductionist approach, and a fundamental circularity, which promotes a holistic vision of the system. This is hardly surprising, as all these principles and points of view are connected to the centuries-old conflict between nominalism and universalism. 5.2.2. Complexity We shall now look at the notion of complexity, which gives meaning to the whole issue: without complexity, systems behave in a manner which is a priori controllable and does not require the use of specific identification and control tools such as simulation. In etymological terms, “complex” is clearly differentiated from “complicated”: “complex” contains the root “ple”, meaning “to braid”, whereas “complicated” contains “pli”, meaning “to fold”. The key idea is thus one of entanglement and interweaving of relationships within a complex object. However, René Thom defines it [THO 90a, THO 90b]: the notion of complexity is terribly ambiguous. Too often, it is the subject of foggy rhetoric; too often, when faced with a system which we are unable to understand or master, “complexity” serves as an excuse for intellectual laziness. Outside of strictly formalized situations (…), the complexity of an object cannot be evaluated numerically; we can only use qualitative evolutions, which are fundamentally subjective and depend essentially on the perspective from which we view them. (…) when a Boeing in flight crashes to the ground accidentally, throwing out large quantities of formless debris, we would conclude that the complexity of the system has increased dramatically because of this catastrophe. According to the usual finalist point of view, however, the intact airplane in flight is considered more “complex” than all of the debris. (…) Even in pure mathematics, the notion of complexity raises problems (…). The null function f = 0 is the simplest of all functions; however, from a structural stability standpoint, the null function has “infinite codimensions” in the function type space, and so is therefore of infinite complexity. The notion of complexity in a form is not, therefore, intrinsic and is always relative to the nature of the formalism used to describe it: thus, topologically simple forms may be very complex in algebraic terms (e.g. any form obtained by continuous transformation of a sphere, without tearing or gluing, which is therefore topologically equivalent to the sphere but for which the equation may, arbitrarily,
212
Simulation & Modeling of Systems of Systems
be difficult to describe3). It is therefore practically useless to discuss simplicity or complexity without defining the nature of the formalism. This linguistic aspect, however, is not the only way of relativizing the notion. Certain seemingly complicated things are extremely simple. In contrast, apparently simple things may prove to be very complex [KLU 08]: a factory or refinery, with multiple buildings, reservoirs, and tubes snaking across hectares of land through clouds of malodorous smoke, for example, may be considerably less complicated than a pot plant with its micro-hydraulic networks and finely regulated metabolism. A colony of ants may seem more elaborate than certain crowds, and a toy shop may seem harder to manage than certain much larger companies. In short, complexity is a hard concept to define precisely. Take another example: a star in the night sky may fascinate us: I personally remember spending several minutes admiring the Milky Way one night in May in a national park in South Africa, far from all civilization, the somewhat frightening silence of nature interrupted only by the wild music of unknown creatures. On the other hand, a goldfish in an aquarium is quickly forgotten by a child who, only moments before, had begged his parents to buy one. The first example – the star – is a cosmic machine based on fairly rudimentary fusion mechanisms4, whereas the second is a subtle arrangement of cells working in harmony within multiple interlinked systems: circulation, skeleton, optics, neurology, hematology, audition, respiration, enzymes, biomechanics, behavioral, social, and so on. What, then, is simplicity? What is complexity? We clearly cannot find them in ordered, robust, and stable structures, such as a crystalline network; nor do we find them in unordered, random, unstable structures, such as the gas in a room. They must therefore exist between these two extremes [GEL 94]. This would suggest a characterization based on a combination of structural (grammatical or algorithmic complexity) and entropic measurements, but a characterization of this kind includes the implicit hypothesis that complexity is more or less stationary (i.e. does not change over time). However, certain systems clearly do appear more or less complex depending on the length of the observation period: this can be measured by the Deborah number5, used in rheology to characterize the “fluidity” of a material 3 Planet Earth is an example of this: the algebraic complexity of its form is such that it is sometimes described as “geoid”, a clear admission of powerlessness when faced with the tyranny of geometric nominalism. 4 At least for those with a clear understanding of relativist hydrodynamics. 5 This number was introduced by Professor Markus Reiner in an article published in Physics Today in 1964. Its name refers to the prophetess Deborah, who, alongside Barak, led the Israelites to victory against the Canaanites (Judges 4 and 5 in the Old Testament). This passage includes the Song of Deborah, with the verse “the mountains melted before Yahweh”, a possible allusion to the fact that, with infinite time for observation, God might see changes in objects which man sees as changeless due to the brevity of his lifespan. Leaving
Modeling and Simulation of Complex Systems
213
and defined as the ratio of a relaxation time, characterizing the intrinsic “fluidity” of the material and the period of observation. This illustrates the difference between apparently unchanging solids, and fluids, which flow. If the period of observation is long or if the relaxation time of the observed object is short, the observer sees the object evolve and, in a manner of speaking, “flow” like a fluid. Another characteristic of complex systems is the potential for ambiguity in defining their boundaries. A boundary, as the distinction between interior and exterior environments and the principal location of exchanges between these areas, may appear to be relatively well defined. However, it may fluctuate or be somewhat indistinct, in the sense that although some aspects are clearly inside or outside the system, there is a zone which could belong to either category. Thus, for an ant colony, the “interior” part of the system is not limited to the ant hill – which does not even exist for certain non-sedentary species, such as African army ants – but may, in theory, extend up to the edges of the area visited by the insects in search of food. However, a definition of this kind, based on the geometric envelope of trajectories of members of the colony, is either completely unstable, if we consider the specific envelope of trajectories at a given moment, or irrelevant when considering exchanges between the colony and the external environment if we include all trajectories of all ants throughout the lifespan of the colony (which is itself difficult to define, even using the lifespan of the queen as a reference). We might think that the question of boundaries would be easier to consider in the case of artificial complex systems, due to the use of rigorous engineering processes, but this is not the case: how, for example, would we define the boundaries of the gas supply system of a European country? Clearly, these boundaries cannot be fixed at the political borders of the country in question, as shown by the 2009 natural gas crisis between Russia and the Ukraine, which had an impact on other European countries. Should we, then, fix the boundary at the limits of responsibility of the gas provider to the final user or of the entire supply chain, including sub-contractors or even those with a potential influence on component parts (e.g. physical components or financial management)? The systems considered differ greatly depending on our response to this question, with different problems in terms of design and maintenance. The notion of a border is interesting because, as a location for exchange, it is the place where the capacities for adaptation of a complex system can be best observed, via the dynamic response to environmental pressures and other more or less expected disturbances, or via changes to the exchange flow which may have a knock-on effect on modifications of internal fluxes. aside this religious interpretation, a scientist might interpret these events as a geological or meteorological event provoking major landslides, carrying away the enemy forces.
214
Simulation & Modeling of Systems of Systems
One last feature, which is somewhat mysterious in nature and tends to attract media attention, is the emergence of structures and behaviors from the actions of the system and its interactions with its environment. We do not claim to present an explanation for emergence and direct the interested reader to the abundant literature published on the subject. Starting from the principle that each reader has their own understanding of the subject, we shall simply provide a few illustrative examples of emergence, to suggest the potential challenges it poses in terms of simulating this kind of system. Our first examples are taken from the natural world: the way a shoal of fish avoids a predator and the appearance of paths near to watering holes in the African savanna. A shoal of fish is particularly fascinating and may suddenly change direction to avoid a predator, behaving as a single body: this raises interesting questions on methods of collective control. Various models can produce this type of behavior, and it is mathematically possible to pass from some models to others in spite of their apparent differences. The appearance of definite paths near watering holes is due to the fact that large number of animals follow the same itinerary; each new animal following this itinerary reinforces the path. The same phenomenon can be observed on certain lawns, particularly in cases where it is clearly quicker to reach a given point by walking across the grass than by following the path (seen as an imposed detour). Other examples found in artificial complex systems show that it is difficult to model these phenomena a priori: we shall consider the examples of commercial hubs in the air transport system and interest groups in social networks. The concentration of air traffic around hubs is essentially due to economic logic (regrouping of a number of aircraft to facilitate maintenance) and security considerations, particularly for entry into a territory. But the real emergent phenomenon has been the appearance of major commercial zones around these air transport hubs, as much in the United States as in Europe6 (e.g. London Heathrow) or in Asia. The presence of large number of shops allows passengers forced to pass through these hubs to kill time in a pleasant manner, and increased sales encourage the installation of further shops, and so development continues by a process of mutual influence (known, in technical terms, as positive retroaction). In a similar way, interest groups emerge on the Internet within social networks, without really understanding the conditions which led to their appearance and development (much to the displeasure of publicists!). The emergence of behaviors or structures, as illustrated in the two cases given above, shows an important characteristic of complex systems which some consider crucial in defining complexity: self-organization. Whether this is manifested by a modification in the system architecture or by observed behaviors, it is a source of innovation and of new capabilities. The interest of mastering these details, as well 6 Note that this phenomenon can now also be observed in railway stations experiencing a high level of traffic: in addition to the ever-present newspaper stands and refreshment areas, we now find chain stores selling clothing, media, and so on.
Modeling and Simulation of Complex Systems
215
as the conditions in which it appears, is evident: a clear understanding of these details leads to possible new system properties, while an understanding of conditions opens up the possibility of controlling these properties, by finding the means either to create them or modify them depending on external constraints, or to avoid them in the aim of guaranteeing operational security. This is the point where modeling comes in, providing the means of inverting the model to try and control emergence, and simulation can provide heuristics with this aim. Note, however, that these ideas are currently only a potential research subject, except in certain specific cases. To conclude this overview of complex systems, we shall give a few more examples of artificial complex systems, “systems of systems”, for which we use the following definition7: a system of systems is an assemblage of systems which could potentially be acquired and/or used independently, for which the designer, the buyer, and/or the user wish to maximize performance of the global value chain, at a given moment and for an imaginable group of assemblies. In this definition, we use the general definition of “value chain” popularized by Michael Porter: the set of interdependent activities, the pursuit of which allows the creation of identifiable and, where possible, measurable value (i.e. what the client is prepared to pay for the product or the service). The examples developed in greater detail in [LUZ 08a] include intelligent transport systems, which not only consider the availability of transport vehicles but also provide information services (cartography, meteorology, real-time information on traffic conditions) to passengers, or even the possibility of passing from one transport service to another in a transparent manner in terms of payment, such as the Navigo pass used in and around Paris, or other “single ticket” initiatives taken by regional centers operating several methods of transport. Another example is Global Earth Observation System of System (GEOSS), the international system of systems for observing the Earth, which brings together sensors, processing resources, databases, and access portals; one last example is the global financial system, the complexity of which became apparent in 2008 with the emergence of an economic crisis. 5.2.3. The difficulty of concepts: models, modeling, and simulation The meaning of the word “model” when used in reference to a phenomenon is usually common: the representation – that is, an organized and formalized set of opinions, beliefs, and information – in a language specific to the phenomenon. “Modeling” is used to refer to the process of finding and establishing this representation. However, we have a tendency to forget the limitations and constraints 7 See [LUZ 08a, LUZ 08b] for a discussion of this definition and its links to numerous bibliographical references, along with a certain number of examples, and methods and tools for mastering complexity in these systems of systems.
216
Simulation & Modeling of Systems of Systems
of this definition, which are clearly set out in three specific points: we are dealing with a “representation”, in a “particular language”, and the object to which these apply is a “phenomenon”. We shall cover these three points individually. 5.2.3.1. Representation As we can see from the composition of the word itself, a representation is a “re-presentation”, that is, a new presentation of the object to the subject (the observer). The term also contains an aspect of “replacement”: the initial object is replaced with a new object of interest, with the observer’s accord and/or active participation: the observer becomes modeler. The model, by metonymy, itself becomes a sort of creation. In the same way, the link between the model and the reality is not immediate8. As Alain Badiou highlights in his work Le concept de modèle [BAD 07], “The model is not a practical transformation of the reality: it belongs to the register of pure invention and possesses a formal irreality”. To support his affirmation and in a departure from his usual domain of formal logic, Alain Badiou makes reference to Lévi-Strauss and his work Anthropologie Structurale, where the model is said to be “constructed” or even “knocked together”: for Lévi-Strauss, the formal, the constructed, the artifact, is a model relative to a given empirical domain. In positivist semantics, the model is an interpretation of a formal system. It is therefore the empirical, the “given”, which are models of the syntactic artifice. Thus, the word “model” takes on a sort of reversibility. (A. Badiou). To continue our reflection on the relationship between the model and the real system at this level of abstraction9, we should note that a model is only a partial representation, at a given moment. Models should be approached with a critical spirit; not to do so demonstrates ignorance on two levels: first linked to the a priori incompleteness of the representation of reality and second to a posteriori incompleteness of knowledge. “The model, technical moment or ideal figure takes its place, at best, in the domain of scientific practice. Note that, as a transitory adjuvant, it is only destined for deconstruction, and the application of the scientific process does exactly that” (A. Badiou). We may also cite the example of Bohr’s planetary model of an atom, used as a useful image in teaching, which has now been replaced by a probabilistic quantum model allowing precise calculations based on the behavior of the atom. The permanent re-evaluation of a model is necessary as
8 In the etymological sense of the term, that is, direct, without intermediary. 9 Again, in the etymological sense of the term, that is, ab-trahere, pulled from. It is interesting to note that the idea of separation is subjacent to most of the terms usually used in this context.
Modeling and Simulation of Complex Systems
217
knowledge advances; worse, “stopping at a given model turns the model into an epistemological obstacle” (A. Badiou). We should keep these precautions in mind over the following sections, where we shall discuss different types of phenomenological models, not the analytical models used for the most part by engineers. The criteria used to suggest a scale for classifying representations include exhaustiveness and simplicity: the fact of considering all observed phenomena and the “elegance” of the model. We could consider that this attention to esthetics stems from the Kantian or Platonic logic subjacent to our own culture, which considers that the laws of nature must be simple as they are universal. We might also see these considerations in a practical light, as savings in terms of resources are almost always interesting if the model has no particular ontological status in connection with that which it represents. In addition to these criteria, a third property is linked to model usage and is directly relevant to the end-user of a model: credibility. A model is, in fact, a “fabrication of a plausible image” (A. Badiou). Chapter 3 details this essential characteristic. Moreover, as an artificial object, the model is controllable: we may anticipate the way in which it will react if one of its elements is modified. “This precision, in which we find the theoretical transparency of the model, is clearly linked to the fact that it is mounted as a whole, so that the opacity linked to the real system is absent” (A. Badiou). 5.2.3.2. Particular language Here lies the source of the “richness” of models: their variability. A model may be analytical (a set of equations, following a reductionist approach), synthetic (guided, e.g. by the principle of optimality, symmetry, or conservation, in which case the approach followed is more holistic), quantitative (as in the previous examples, with the fundamental objective of calculating values), qualitative (focused on a given interesting characteristic, without necessarily aiming for digital adequacy), behavioral, economic, and so on. For example, depending on the specific case considered, a cloud might be seen as a convection phenomenon or as a homogeneous group of water droplets suspended in air saturated with humidity. This shows the level of diversity in terms of modeling, which depends on the predictive quality of the desired model, in connection with other modeling idealizations made for connected systems and systems interacting with the main object of the model, in this case, the cloud. In the same way, a model of a system of systems such as the European global satellite navigation system (the Egnos and Galileo programs) could concentrate on contractual aspects linking clients, suppliers, partners, and users, or on international flows between different physical components, or on the physical trajectories of
218
Simulation & Modeling of Systems of Systems
satellites, or on economic aspects (the business model of the associated services), and so on. A large variety of formalisms may therefore be used, each with its specificities, strong points, limitations, and interpretational difficulties. However, over and beyond this range of possible formalisms, we must – as pointed out by Alain Badiou – move beyond the position based on the Hegelian perspective of “the word as the murder of the thing”, where the model would “kill” any aspect of intuition in the approach to the phenomenon it mathematizes. This point demonstrates all the ambiguity of modeling, between an arbitrary characteristic of the model type and an intuitive leaning toward the use of a particular model. 5.2.3.3. Phenomenon The usage of the term “phenomenon” places the user in a privileged position. It is not the “abstract” system which interests us, but a set of observed or observable behaviors. This leads us to distinguish between emulation and simulation of a system: simulation is a dynamic use of models, whereas emulation is the quest for the most perfect analogy possible between calculated and observed behaviors. Simulation, then, is mostly based on the activity of the model builder, and its specificity is linked to the use of mainly computerized (but also physical) resources for model usage. In extreme cases, simulation may be seen as a “simple” complicated calculation effectuated over successive states of the model. Beyond the limits intrinsic to the models’ subjacent formalisms, we find limits imposed by the computer usage, limits which are not just linked to processing time or memory size issues, but which are based on the actual impossibility of certain mathematical tasks. We shall return to these intrinsic limits of computability later. 5.3. Main characteristics of complex systems simulation 5.3.1. Non-linearity, the key to complexity The interest of the previous, relatively general considerations lies in the existence of various modeling theories based on different epistemological principles and providing tools for mathematical formalization, explanation, or prediction. These often provide quantitative information on the phenomenon under consideration. Whatever formalization is used, it highlights non-linear aspects, a cause of richness and difficulties of interpretation of phenomena. This will be the subject
Modeling and Simulation of Complex Systems
219
of discussion in the following sections, although we do not aim to cover the topic exhaustively10; for pedagogic reasons, we shall avoid mathematical development, and the reader may consult various works listed in the bibliography for greater detail. 5.3.1.1. Decomposition and reconstruction versus irreversibility A linear system is defined as the behavior produced by the sum of two systems is the sum of the behaviors of their sub-systems, also known as the principle of superposition. Decomposition is therefore direct, and reconstruction is simple, with no difficulties except for that of recombining components. A non-linear system, on the other hand, does not have this property11. Thus, the cooking time of a big cake is not twice the cooking time of a cake half the size – unless we want our cake to be burnt. In general, the dynamic interconnection of elements tends to introduce aspects of non-linearity. Another factor that indicates non-linearity is irreversibility, that is, the fact that we may determine the arrow of time by observing a phenomenon. Although the propagation of a wave without dampening is a reversible phenomenon, the same cannot be said of the propagation of heat through a metallic plate. Irreversibility is an interesting property as it is often linked to diffusion or energy loss effects (through friction or relaxation), which may be a sign of long-term smoothing effects where certain disturbances may be “forgotten”. A number of physical systems show pattern emergence phenomena as a result of this irreversibility. 5.3.1.2. Local to global transition versus chaos One characteristic that greatly simplifies dealings with linear systems is that by understanding the system at local level (more precisely: in an open neighborhood of any regular point); we gain an understanding of the system as a whole. If something is seen to happen under particular conditions, we can deduce what happens under other conditions by analogy. This is not the case in a non-linear system, which may present singularities around which system behavior may be extremely strange. 10 The domain has grown so much in the last decades that it would be futile to try and cover it in its entirety. Moreover, the mathematical theories used are manifold – and themselves complex – as shown by the following partial list of key terms: chaos, turbulence, attractors, fractals, cellular automata, multi-agents, emergence, “small world” graphs, critical phenomena, borderline effects, bifurcations, catastrophes, pattern formation, morphogenesis, diffusion, percolation, phase changes, synchronization, shock waves, solitons, and so on. 11 It may seem strange to define the concept by negation, especially as, mathematically, almost all systems are non-linear. Furthermore, note that all non-linear finite dimension models of the type x′ = f (x) can be transformed into linear models, but with infinite dimensions using the Perron–Frobenius equation: p′(x) = ∫p(y)δ(x − f (y))dy. We shall not, however, go down this path, as it is largely based on formalism and its word-games.
220
Simulation & Modeling of Systems of Systems
Singularities themselves may be arranged in an incredibly bizarre fashion, with the potential for creation of “strange”, or “chaotic”, attractors. As dangerous as identified non-linearities can be, they may also present certain advantages. In aeronautics, for example, the existence of unstable non-linear modes allows the maneuverability of an aircraft to be greatly increased under certain conditions. However, this “richness” of possible behaviors, where regions of instability may appear suddenly and can be difficult to control, can cause serious problems. It is currently fashionable to claim (after the event …) that the financial crisis of 2008 was predictable. The same individuals who make such claims propose very different solutions to the problem, none of which appear a priori to enable us to leave the domain of instability. These are all clear indicators of non-linearity. In a similar manner, the phenomena of hysteresis and bi-stability (existence of several local optima with alternate “hops” from one to the other depending on disturbances) only occur in non-linear situations. These phenomena may be found in various social or financial models (see the passage on catastrophe theory). Sudden hops, sudden divergences, and regions of inaccessibility are all part of the panoply of situations only found in non-linear systems. 5.3.1.3. Predictability versus unpredictability Unpredictability, a consequence of the properties discussed above, is frequently encountered in non-linear systems. On the other hand, in a linear system, knowledge of behaviors over a certain finite time period enables full understanding of the global system, this is not the case in non-linear systems (which is a shame, as otherwise it would be very easy to make a fortune on the stock market). One of the properties on which most literatures have been written, and which is particularly present in literature aimed at a non-specialist public, is sensitivity to starting conditions. Let us illustrate this using a very simple non-linear system: the “left shift” application. This application connects a positive real number x with the fractional part of 2x, that is, the difference between 2x and the nearest natural integer using values less than 2x (or more prosaically, the number made up of the digits after the decimal point when the number in question is developed in base 2). The dynamic system known as “left shift” is defined simply as follows: the initial state value at instant 0 is any number from within the unit interval (so from 0 to 1 inclusive: if we develop it in base 2, we have 0 to the left of the decimal point and any sequence of 0 and 1s after the decimal point). The system state value at instant k + 1 is given by the “left shift” application, applied to the system state value at time k. With a number developed in base 2 where there are no digits other than 0 to the left of the decimal point, this reduces to “forgetting” the first digit after the point and moving all the following digits one place to the left, hence the name of the
Modeling and Simulation of Complex Systems
221
application. Let us take two arbitrarily close numbers, differentiable only after the first n digits after the decimal point, where n may be arbitrarily large. The system trajectories are impossible to tell apart until we reach iteration n; as from this moment, each follows its own path as set out by the following digits. Clearly, this is a simple textbook illustration, but it illustrates sensitivity to initial conditions in that even if we “know” these conditions to an arbitrary level of precision, it is possible that trajectories diverge at some point. In a linear system, on the other hand, the error between trajectories would have been kept arbitrarily small via an increasingly precise knowledge of the initial conditions. To digress a little from our central theme, we shall now consider the conditions of appearance of non-linearities in a system which is a priori linear: these may be the result of interaction or regulation loops. As a textbook example, let us take the discrete system xk + 1 = 2xk + uk, which represents a system known via its state xk throughout sample time periods, with an entry uk acting as a regulation signal. The system is a priori linear if uk is not dependent on xk (if it is an exogenous entry independent of the system history) or if uk is dependent in a linear fashion (e.g. via a homeostatic state return corresponding to a reaction loop uk = −axk, with a “spring” constant a). If, however, the regulation shows non-linearity, the looping system becomes non-linear: consider, for example, uk = [−2xk], where the operator [.] is the integer part, that is, the closest integer using inferior values. For a state from 0 to 1 inclusive, and for the preceding linear system, uk may take the values 0 and −1, which corresponds to a switch or diode type behavior, and the looped system is none other than left shift, the non-linearity of which has already been demonstrated. This situation is far from exceptional in that all phenomena of sudden commutation, saturation, and mechanical wear and tear produce consequences of this kind. Another consequence of non-linearity with an effect on predictability is the non-transitive propagation of knowledge. This is expressed by a theorem developed in 1972 by Kenneth Arrow, an economist at Stanford University, which illustrates why it is difficult to make a preferential choice when faced with three or more options. Consider the example of three users, each preferring a particular solution to a problem. User 1 classes three solutions following their order of preference: 1, 2, and 3. User 2 classes the solutions in the order 3, 1, and 2. User 3 chooses 2, 3, and 1. Given the choice between two solutions, the majority of users prefer solution 1 to solution 2, solution 3 to solution 1, but also solution 2 to solution 3. This goes against intuitive reasoning, which is based on transitivity, that is, the linear propagation of a relationship. In a similarly counter-intuitive manner, if we must choose between all solutions simultaneously in the example provided, no majority preference can be detected. This shows that we cannot easily extend the field of application of a sub-problem to a more general problem, another effect of non-linearity.
222
Simulation & Modeling of Systems of Systems
5.3.1.4. Sensitivity to disturbances Sensitivity to disturbances is an extension of the property described above. In a linear system, the size of a disturbance is directly linked to its effect; this is not necessarily the case in a non-linear system. This is unfortunate when the disturbance is a sign of incomplete knowledge of the signal, as in a linear system a small disturbance only produces a small effect, making the system naturally robust. It is even more problematic when faced with major disturbances, which in a linear system would produce a strong reaction potentially able to deal with the disturbance. In a non-linear system, two other reactions are possible: small disturbances may produce major effects, or large disturbances may only produce small effects. The second situation is not of great interest for our purposes, but the first is more relevant and is known – even outside the domain of popular literature on the subject – as the “butterfly effect”12: in this illustration, the simple fact of a butterfly beating its wings could provoke a storm at a distant point on the globe. The principle of this idea is to illustrate, using a memorable image, the fundamentally non-linear character of weather systems (and more precisely of equations used to model weather systems, something which we shall discuss in greater detail later on). Another example used to illustrate the same issue might be the gunshot that killed the Archduke Franz Ferdinand, triggering the First World War and contributing, albeit indirectly, to the major transformations of the 20th Century. Note that the famous butterfly effect is not necessarily negative or a symbol of powerlessness; it also shows the possibility of gaining considerable benefits from minor investments, if we can define the right utility function! Different economic stimulus actions attempted to activate this phenomenon to provide a way out of the 2008–2009 financial crisis. The figures involved seem enormous – billions of euros or dollars – but are, in fact, negligible when compared with the financial exchange flows across global markets (the difference is of several orders of magnitude, and furthermore, stimulus programs operate over several months, whereas the flows discussed apply on a daily basis13). This is, then, a clear example of a non-linear system in which we attempt to provoke significant effects by actions which are small relative to the dynamics of the system in question.
12 The image was popularized in the 1960s by the meteorologist Edward Lorenz, also famous for the strange attractor notion. 13 Some statistics: in 2007, a little more than 100 billion dollars circulated on the financial markets every day, a little more than 1,500 billion on the exchange markets, and approximately 4,000 billion dollars in terms of derived products. This gives a daily flow of approximately 5,500 billion dollars per day for the global financial economy. The total of effective and promised government assistance in early 2009 was around 3,000 billion dollars – a ratio of less than 0.2%.
Modeling and Simulation of Complex Systems
223
At the risk of appearing contradictory (although this only serves to highlight the difficult nature of any overly general prediction created by excessively early application of certain principles), we wish to raise an additional point concerning the butterfly effect: over the last decades, meteorologists have noted that after a period of 1–2 days, the growth of disturbances ceases to be exponential14, but becomes proportional to time. This is not particularly surprising when considered from the viewpoint of theoretical physics. In fact, statistical mechanics, the theory which allows long-term forecasting of certain statistical quantities which often correspond to average values of physical variables, undermines the dogma of the butterfly effect: even if the system is not microscopically predictable, the visible macroscopic aspects will be predictable. For example, gas molecules within a room take paths which cannot be followed beyond the first few collisions, but we can define a macroscopic quantity characterizing their average paths, that is, temperature, and we can calculate results in terms of this new variable, for example, if we open a door to another room or outside. It would be absolutely impossible to know what would happen if we remained at molecular level. To summarize, non-linearity produces unpredictability at a small scale, but it may give way to linearity at a higher scale level. These last two considerations should be considered in complex systems with a large number of degrees of freedom and which may be described using macroscopic quantities, that is, statistical methods. An example of this would be network-centric systems, where we might define the quantities linked to information and its diffusion within the system. 5.3.2. Limits of computing: countability and computability The use of computing resources requires prior programming of the code needed to run these calculations. All programming of computer code is done via a symbolic language, no matter how complex the instruction sets available. Anyone with the slightest experience of programming languages knows that there are a finite number of available instructions; moreover, a program is itself finite, even if it may be arbitrarily long (sometimes reaching tens of millions of lines), even without adding useless lines! To summarize, if we count all the programs that may be written in all imaginable programming languages over a finite time period, the resulting number of programs will itself be finite. This set of programs is at most countable, that is, at most, it has the cardinality of the set of natural integers15. 14 The divergence of trajectories, whether in sensitivity to starting conditions or the butterfly effect, is often due to a relative error on the state of the demarcated system, mathematically expressed by an equation of the type x′(t)/x(t) = a, where the state derivative x(t) is a time derivative. After integration, we obtain x(t) = x(t0)eat, hence the a priori exceptional behavior. 15 Mathematically, each program may be associated with an integer so that two different integers correspond to two different programs. This is done following Gödel’s numbering technique, for example. We thus obtain an injection of the set of programs into the set of natural integers.
224
Simulation & Modeling of Systems of Systems
This is important as it sets a fundamental limit on what is accessible by informatics, as we shall see in the following sections. We could raise the objection that the reasoning above is conditioned by a model of computation based on the use of computing languages. This is true, but in spite of human ingenuity, all other formalisms16 invented to model the intuitive idea of computation using a mechanical procedure (which therefore operate following a discrete chronology – we can separate, number, and order different instants in the calculation procedure – and give a result in finite time, if a result is given) have reached the same result. More precisely, all these formalisms have been shown to be equivalent in that they define the same class of mathematical objects (up to isomorphism)17. We could also say that a computation depends on the initial conditions provided. This, again, is true, but we should not forget that this initial data input must follow the same constraints of mechanical use, that is, the quantity of data must be finite, although it may be arbitrarily large, and coded as a finite collection of symbols which may also be arbitrarily large. At most, we have a countable quantity of initial conditions which may be coded using computers. The set of calculations and the set of imaginable conditions are the product of two countable sets and are therefore always countable. It is, in fact, possible to escape from this “tyranny of countability”. We must “simply” allow transfinite inferences (i.e. the time taken to obtain a result – in this case, a clear numerical result, and not an absence of result as in a looping program – may be infinite, which, we must admit, is not particularly realistic for a practical application!) or pair the computational resource with an object generating an initial condition non-mechanically, which could then be used and transformed before passing the deciding step of computer encoding (a real challenge, admittedly). This last suggestion only works if the physical world really leads to such objects, which is no longer obvious with the recent development of several theories of discrete space and time which question their continuous aspect, that is, fundamentally question their varying in a space with the cardinal of the continuum (like the set of real numbers or any differentiable real variety), thus immeasurably more than countability. In short, this excludes any reasonable computational model.
16 For example, turing machines, post systems, recursive functions, Chomsky’s formal grammars, cellular automata, lambda calculus, and combinatory logic. 17 This is what Church’s thesis affirms: that any one of these formalisms is the right one to model the intuitive idea of mechanical calculus. An affirmation of this kind clearly cannot be demonstrated as it identifies an intuitive notion with a formal notion. The affirmation is connected to the epistemological paradigm.
Modeling and Simulation of Complex Systems
225
Using a computational model, we can define a variety of notions, such as recursivity, decidability, recursive enumerability, and semi-decidability. Without going into mathematical detail, note that these notions are applicable either to sub-sets of natural integers, or to problems, or to formulae expressed in a certain formal language. Although these domains of application seem different, they are brought together mathematically via a recursive isomorphism (i.e. for each of these applications, we can find a coding algorithm usable by a computer so that the final codes are all the same). The above notions can therefore be defined, informally, as follows: “recursively enumerable” or “semi-decidable” means that we can respond affirmatively, by a computation (if, of course, the response is affirmative18) to questions such as whether a given integer belongs to a specific set, whether a given class of problems has a solution, or whether a set of formulae can be demonstrated in a finite time period based on axioms and interference rules. The notions of recursivity and decidability, on the other hand, imply that the calculation allows a positive or negative response in finite time, something which is much more difficult. This avoids waiting eternally for a response which never appears. These notions are interesting insofar as we can prove that computability, as described above, is equivalent to the notion of “recursive enumerability”. Moreover, we can prove that the notion of computability is well defined, in that, for example, sub-sets of natural integers exist which are recursively enumerable, but others exist which do not have this property. We shall provide an example to illustrate what has been said above, both in terms of cardinality and computability. Let us return to the “left shift” application defined earlier. The application is clearly computable, as, intuitively, it is easy to write a program which will do this. The sequence of output values is the sequence of figures showing the development of the initial condition in base 2. An initial value is therefore computable if a program can unpick the figures which constitute its development in base 2. Note that there is a clear one-to-one map between all the numbers of the unit interval (i.e. all possible initial conditions) and all possible subsets of the set of natural integers: I is in the sub-set considered if, and only if, the figure in position i is a 1. However, the computable initial values only correspond by this one-to-one map to recursively enumerable sub-sets, which only constitute accountable part of the unit interval, that is, a null set. In other words, this simple example19 shows that the trajectories accessible to any computability model only form a negligible set within all the trajectories possible within the system.
18 If the response to the question was actually negative, the computation is authorized to loop, potentially infinitely. 19 Despite its simplicity, this example is not ad hoc: its genericity and its use in commanding robotic systems evolving in a limited environment with obstacles are illustrated in [LUZ 93, MAR 94, LUZ 95, LUZ 97, LUZ 98c], demonstrating its limitations and characterizations of classes of programs which may be used to control systems of this kind.
226
Simulation & Modeling of Systems of Systems
That said, we should remember that any sufficiently regular solution to a problem expressed by differential equations may be approximated in a computable manner with an arbitrarily small precision20. The hypothesis of regularity is obviously important, and this is where we find ambiguity and the necessity of caution when using computer tools: in numerous situations, the limits discussed above have no particular influence, but in specific cases (chaos, non-regularity of applications, and so on – in short, a number of situations which may be encountered among complex systems), computability constitutes a fundamental obstacle to the illustration of certain properties. Thus, using our earlier example of left shift (for generalizations of this example showing a degree of genericity in relation to dynamic systems, see [LUZ 93]), any computerized simulation will give, at most, periodic behaviors due to the finite nature of the encoding of any initial condition, while these behaviors are, in fact, negligible among chaotic behaviors (the trajectory “fills” all the unit interval), which are by their very nature inaccessible to computerized simulations. This section, covering rather technical aspects of the subject, can be summarized as follows: from the point of view of mathematics, all modeling of computer processing resources a priori introduces limitations of complexity in comparison to what might happen in the real environment21. 5.3.3. Discrete or continuous models This question is ever-present in epistemology, and the opposition of discrete and continuous model has been a subject of reflection since the Ancient Greeks, whether in terms of acceptance relating to cardinal multiplicity (discreteness being linked to counting in whole numbers, continuity to large cardinal numbers) or to geometric divisibility (e.g. discreteness being linked to the notion of points and continuity to that of a line). The apparently paradoxical reflections of Zeno of Alea demonstrate this. Which of the two is “superior” is an unsolvable philosophical question, one not even helped by language. In English, “continuous” is opposed by “discontinuous”, whereas in Russian, we find “неперерыв” and “перерыв”; negation, expressed by “dis” in the English example and “не” in Russian, is applied to roots with opposite meanings.
20 If the solution is not too pathological – continuous, for example – we need to simply apply the theorem of approximation of continuous functions by simple functions and then make use of the fact that the set of real numbers have the rational numbers as a dense subset, allowing an approximation of simple functions by simple functions using rational numbers, which are, obviously, computable. 21 Except for fans of The Matrix, for whom the real world is simply an algorithmic simulation … but we shall leave this subject in case a certain Mr. Smith decides to show up!
Modeling and Simulation of Complex Systems
227
To show that the question itself is badly posed, we return to the example of digital computers, used earlier to show the necessity of studying discrete entities. If we refer to the laws commonly used in physics to analyze this type of system (the Maxwell equations), we obtain a continuous model based on differential equations, from which we could conclude that the state of a computer of this kind must, in fact, vary continuously (e.g. no sudden changes in power). However, if we analyze the components of these computers more closely – for example, capacities – we see that the same models used in physics are based on the movement of a finite number of charges, suggesting, this time, using a reductionist approach, that a discrete model would be appropriate. We might then consider the electron from the point of view of its wave function and use Schrödinger’s equations, which send us back to a continuous model. The solutions to this group, from the perspective of quantum mechanics, are quantized, and different particles may be broken down into discrete components, taking us back to a discrete model. These components may themselves be modeled, using quantum field theory, by continuous fields, producing a continuous model, which itself would eventually be discretized, depending on the theory of quantum gravity used. It is as easy to pass from a discrete representation (in cosmology, galaxies are seen as elementary components) to a continuous modeling (the set of galaxies is assimilated to a fluid to apply Einstein’s equations and discover the evolutionary dynamics of the universe) as it is to pass from a continuous representation (in hydrodynamics, a “tube” of fluid) to a discrete model (the “tube” is split into slices, each with a simplified behavior, and the coupling of partially derived equations is brought down to a discrete total of interactions in the form of borderline conditions between the supposedly “perfect” elements, the evolution of which may be explicitly calculated). To show that the hermeneutics of the relationship between discrete and continuous is not limited to the physical sciences, but also find a fundamental way in mathematics, we only need to look to the fierce debates between proponents of the holistic and constructivist visions, opposing the collectivization used in the Zermelo–Fraenkel set theory, for example, with the requirements of constructability and temporal modularity expressed by Brouwer [SAL 91, SAL 92]22. Note that the concept of continuity is fundamentally relative and depends on the logic used in the formalization of the structure considered (something may be continuous from an
22 The relative relationship between discrete and continuous is studied from the angle of non-standard models in [SAL 99] where, for example, the Harthong–Reeb model represents the set of real numbers in a non-standard model of the set of relative integers, in a way compatible with order, showing how discrete calculus and geometry can be carried out through representations of this kind.
228
Simulation & Modeling of Systems of Systems
internal perspective but not from an external point of view23). In the same way, depending on the topological viewpoint, continuous models may be able to deal with an idea of slow and not locally observable deformation, which discrete models cannot do; however, this is to forget that composition, the basic operation in a discrete approach, is itself a continuity, simply defined in an adequate topology (this is, in fact, the formalization of what slow and non-observable deformation actually is!). In an attempt to conclude this debate, it seems to us that the basic question is not whether the model should be continuous or discrete, but which model is the most suitable to the scale at which we are working and for the conclusions we wish to obtain. 5.4. Review of families of models The two main opposing approaches in mathematical modeling techniques are equational and computational approaches. This is linked to the traditional opposition between descending and ascending approaches. Equational approaches start from general principles – conservation of certain values such as energy or the quantity of information, symmetry (i.e. invariance when subjected to sets of transformations, such as translations or rotations, or more complex applications, such as those found in gauge theory), or maximization of certain quantities, for example, a defined action based on an energy – from which we deduce synthetic formulae, the resolution of which shows the desired trajectories of the variables studied. However, these equations often require complex theoretical tools to solve, and an explicit and global solution (not just for the neighborhood of a given point, or over a limited time period) may be difficult, or even impossible, to find. Computational models, on the other hand, do not have this synthetic character, and they are based on local future state construction mechanisms, which start from a knowledge of the current state and, sometimes, the immediate environment. The whole of the trajectory can thus be obtained by successive step-by-step execution of the computational principle. These two approaches are very different. The first is based on a body of mathematical work on resolution techniques developed over two centuries; the second became popular and well known with the appearance of computers and intensive calculation. However, the two are not necessarily independent: it can be demonstrated that certain multi-agent systems or cellular automata, which will be discussed later in this chapter in the context of computational approaches, in fact produce solutions to equations – which can be partially derived by applying the principle of conservation of energy. Or from 23 This refers to the Löwenheim–Skolem theorem, for example, which states that any finitely axiomatizable theory with an infinite model can be represented by a countable model.
Modeling and Simulation of Complex Systems
229
local information exchanges from spatially neighboring states used to calculate the following future state, it is often easy to produce – by carrying out probability density calculations on these exchanges – a “master” equation, which we may then attempt to solve and which, under certain hypotheses, boils down to some of the examples given later. Each of these approaches presents certain advantages and disadvantages. Equational methods profit from their formal character and the theoretical background attached, allowing us to study the characteristics of stability, sensitivity, and robustness. At the modeled system level, this pays dividends in terms of reliability and operational safety. The major drawback of this approach, leaving aside potential mathematical difficulties, is that we are only dealing with a model, and so the property obtained by solving equations – often after considerable effort – is only a property of the model and not necessarily of the modeled system. Computational methods have the advantage of a priori simplicity, and their illustrative character is attractive, but they often require considerable processing resources and are subject to the intrinsic limits of algorithmic complexity, or even computability, of the phenomena being modeled. Moreover, as with the equational approach, the results obtained cannot be considered valid except for a partial representation of a system. The real choice of an approach to consider in a given context, therefore, depends more on the availability of modeling and resolution tools rather than on an a priori analysis of different approaches. The “art” of the modeler and interpreter of a simulation lies in the ability to profit, at the same time and in a complementary manner, from different models, to exploit this multi-scale (in terms of both time and space) and multi-objective approach, and to assign an index of truth and validity to each interpretation, possibly proposing extrapolations but with a clear understanding of the risks attached to this practice. 5.4.1. Equational approaches Equational approaches can be split into two classes based on the systems considered: conservative and dissipative systems. The fundamental difference between the two is that the first may be considered to be isolated as far as the total energy of the system is concerned, whereas the second may not. Typical examples are the propagation of waves without dampening (the vibration of an ideal violin string) for conservative systems and diffusion (propagation of heat through a metal plate heated at any specific point) for dissipative systems. Historically, these two classes of systems form part of two different scientific
230
Simulation & Modeling of Systems of Systems
views of physics: the Newtonian and the Aristotelian viewpoints. Aristotle considered movement as a struggle against environmental inertia (if two objects are placed in a viscous liquid, the heavier object falls more quickly), whereas Newton suggested that movement was invariant in the absence of external forces (e.g. objects, unaffected by wind, falling through a non-viscous environment, such as air: two objects fall at the same speed independently of their weight). In fact, from a strictly mathematical point of view, we pass from conservative system equations to dissipative system equations by the simple addition of nonlinear terms, following the example used earlier to illustrate non-linearities that may be created during the regulation of a system. However, among this continuum of systems, we shall direct our interest to certain particular representative cases. We shall, therefore, pay specific attention to waves and solitons, followed by reaction–diffusion and convection mechanisms. Waves and wave propagation equations are well known, and their occurrence in systems has been mastered: studies have been conducted on potential resonance (which creates destructive “pumping” effects, as when a car is subjected to a pitching movement by turning the steering wheel to the right and to the left alternately at the right frequency) that must be mastered at interface level because, for example, mechanical vibrations might be transmitted to an electronic sub-system where they would cause damage. Solitons, on the other hand, are less well known, although they were first identified as early as 1834 by John Scott Russell, naval engineer and mathematician, who observed and followed a wave created by the sudden halt of a barge in the Union Canal (which links Edinburgh with Fort Clyde) over a considerable distance. Russell’s curiosity was awakened by the fact that the shape and speed of the wave did not change throughout its propagation. In the second half of the 19th Century, H. Bazin and H. Darcy, hydrodynamic engineers, reproduced this phenomenon on the Canal de Bourgogne to analyze it. The equational formulation and explicit resolution were carried out in 1895 by D.G. Korteweg and G. de Vries. The interest of these solutions lies in the fact that they apply to single waves of high amplitude and reduced speed and shape (when compared with “traditional” waves which might have formed in the same circumstances). When two solitons meet, no interference occurs, unlike standard waves which would be modified: we have all, at some point or another, jumped into a pool with the aim of creating a magnificent trail of waves, only for these waves to become formless ripples as soon as they hit the side or another wave created by someone else jumping in. This property of non-interference in solitons makes them interesting as a means of information propagation, a fact validated in the late 1980s by the use of fiber optic communications over distances of several thousand kilometers, and again in 1998 using combinations of solitons of different wavelengths. The first commercial equipment using solitons to transport real traffic over a commercial network appeared in 2001. In the natural world, phenomena that can be modeled by solitons are encountered in tidal bores: under certain
Modeling and Simulation of Complex Systems
231
conditions, tides create waves of this kind which travel up rivers, an interesting phenomenon to observe but one which is potentially dangerous to shipping. In France, tidal bores of this kind can be found on the Dordogne and Garonne rivers, and up until the 1960s along the Seine at Caudebec-en-Caux, but this natural curiosity – which, on occasion, proved fatal to overly adventurous tourists – disappeared following terracing works and dredging of the river. The most powerful tidal bore in the world, in China, produces a wave of 9 m tall which travels up the Hangzhou Bay at almost 40 km/h. Certain cloud formations, “rolls” of several hundred kilometers in length which move at high speeds, can be modeled by solitons. The Tacoma Bridge accident on 11 July 11, 1940, can also be modeled in this way. The same applies to the legendary rogue waves of over 20 m in height which tankers sometimes encounter on certain shipping routes. The ability to model all these phenomena is, therefore, useful in infrastructure or naval engineering, for example. Let us now look at reaction–diffusion equations, which form a particular type of transport equations. Just for information purposes, and to show the variability of behaviors produced by different non-linearities, we shall give a family of equations of this kind: ∂ t u = D∂ 2x u + R (u ) . Depending on the form of the non-linear term R(u), different phenomena are produced, enabling the modeling of a variety of situations. If R(u) = 0, we obtain pure diffusion (the “heat equation”), as in the classic example of a metallic plate which heats up as a whole if it is heated locally. If R(u) = u(1 − u), we obtain a simple model of population evolution with migratory phenomena. If R(u) = u(1 − u2), we have Rayleigh–Bénard convection. This convection phenomenon is that observed in a pan of boiling water heated under certain conditions: rolls of water circulation appear beneath the surface. The same model is applied to magma circulation when modeling plate tectonics; it also explains certain atmospheric circulation phenomena. If R(u) = u(1 − u)(u − a) when 0 < a < 1, we obtain the Zeldovich combustion model. These different equations show three mechanisms: diffusion, reaction, and convection. The diffusion mechanism tends to homogenize concentrations of the elements concerned and is characterized by an irreversible transport phenomenon caused by the migration of elements (hence it is used when modeling this kind of situation). Note that borderline conditions, that is, the form of the edge of the system, are propagated “instantly” in that, at any given moment, each point is influenced by all other points. The reaction mechanism, on the other hand, is the manifestation of a local influence, a function of the current state. The convection mechanism shows a functional influence of the flow of speeds, considerably more important than local speed, creating large-scale circulation phenomena. It is the combination of these mechanisms that creates complex behaviors using large-scale spatio-temporal correlations, hence, for example, the modeling of Turing
232
Simulation & Modeling of Systems of Systems
patterns, such as stripes or spots (on a zebra, leopard or similar) which are also found in the Belousov–Zhabotinsky chemical reactions with oscillation dynamics, and in nature with the appearance of fairy rings (truffle-hunters will understand!), and the propagation of epidemics (to measure the speed of diffusion which is used to identify the peak of the epidemic, something seen every year with the seasonal attacks of Influenza, or even gastroenteritis). Models of this kind may be used (in competition with the computational models described later) for modeling crowd behavior, useful in designing infrastructures to allow rapid evacuation in crisis management situations (e.g. fire, accident, or terrorist attacks; see [ZOL 08]). They are also used to model the propagation of information or even beliefs within a population: we can then attempt to stop or limit this propagation, for example, as a counter-insurgency measure (see end of this chapter). At equation level, this plays out as the introduction of additional terms with the effect of creating other solutions without diffusion. The interesting part of the exercise is to then find the real-world mechanisms, which correspond to these terms in the model. We see, then, why it is useful to look closely at equations and their resolution, in that this process may provide clues for “inversion of the model”, as it is described in command theory. Modeling thus leaves the sphere of pure description and becomes a genuine tool for the architecture of possibilities and assistance with design choices. A detailed knowledge of the equations and the mathematical nature of their solutions allow an appreciation of their behavior depending on the situation, giving the user all possible means of making the right choice. 5.4.2. Computational approaches The first, and simplest, computational models that come to mind are finite state automata: these are made up of a finite number of objects, known as states, and transitions between these states which are carried out by reading the associated tag. Certain specific types of states can be identified: initial and final states. The execution of the automaton consists of starting from an initial state and arriving at a final state, and the list of tags obtained during this series of transitions determines the word retained for this particular journey. For a given finite automation, the set of potentially recognizable words makes up a language; if we consider all possible finite automata, we obtain the class of regular languages, widely studied in computing and used in practice, often unknowingly, for example, when using command-line languages (either under Unix or under Linux, or in the “invited command” box in Windows). It is interesting to complexify the initial finite automaton model: if, for example, we add the possibility of adding a symbol to the memory (known as a stack) with each transition, or of removing the last symbol added to the stack, the class of recognized languages is that of computer
Modeling and Simulation of Complex Systems
233
languages – known as context-free languages in theoretical informatics. If, instead of removing the last symbol of the stack, we add the possibility of taking something from the memory at random in a linear manner (i.e. proportionally to the length of the recognized word up to the transition in question at which point we allow ourselves to look in the memory), the class of recognized languages contains the natural languages – also known as context-sensitive languages in theoretical computer science. If we relax the constraints on access to the memory (giving ourselves the possibility of picking from a finite, but arbitrarily large, memory), we obtain the class of recursively enumerable languages mentioned above. The computability model obtained in this way is, in fact, the most general model possible, and we see that it may be obtained through a number of progressive extensions24. Models using finite state automata, and some of the generalizations presented above, are often exploited due to their ease of use. Other extensions of finite state automata, such as Petri automata or networks, are also widely used in simulation: in this last case, transitions happen when particular events occur, adding a temporal notion to the interpretation of the process – but from a strictly theoretical point of view, this is a tag like any other – and a token mechanism is added, by which certain transitions are only authorized if the correct token is present to “validate” the transition. In this case, the token is also transmitted to the next state. This mechanism is what gives Petri automata powers of expression higher than those found in finite state automata and means that such models are particularly widespread in the analysis of usage scenarios for complex systems. By following the same basic idea of state transitions, but using multi-dimensional states, or non-deterministic transitions (several possible source states or spontaneous transitions), we can model a wide variety of situations with (among other things) structured handled data. This is the case for cellular automata, models used widely for the simulation of urban or agricultural developments, or of the propagation of pollution, fire, or epidemics. The starting point is a very simple idea and can be explained using Conway’s Game of Life: we start with cells which may take the value 0 or 1, which we juxtapose to form a line. The line created by the initial values of all the cells determines the initial state of the cellular automaton. We then apply eight rules that allow us to calculate the value of each cell at the next moment, depending both on its current value and on the value of each immediately neighboring cell.
24 We have simply shown the calculus models associated with the particular language classes which make up Chomsky’s hierarchy.
234
Simulation & Modeling of Systems of Systems
The line made up of these calculated values determines the next instant of the cellular automaton. If we represent the temporal evolution of the cellular automaton graphically, as an image where the top line is the initial state, the second line the following state, and so on, we obtain a black-and-white image showing the trajectory obtained starting from the initial state. By varying the eight rules characterizing a particular cellular automaton, we can obtain very different images using the same starting conditions; we can also observe stable patterns or patterns which are mobile in two dimensions (obviously from top to bottom if we place lines successively in this way). To go from this example to a model for epidemiology, we simply need to interpret the state 1 as the fact that a given person is healthy and 0 as the fact that they are infected. The rule then indicates how the infection is propagated locally depending on the state of a person and of their immediate neighbors. The image produces a “film” or propagation based on a starting population. Clearly, to produce an interesting model, we must simply complicate the algorithm, for example, by considering a neighborhood in two dimensions rather than one, meaning that an image corresponds to an entire population, and the film is a stack of images. The rules may also be modeled to follow certain constraints on the spatial aspect of neighborhoods (an epidemic does not spread in the same manner under different geographic conditions), and they may consider temporal constraints (someone who has already been infected may become immune, or someone not yet infected in a certain environment may have natural immunity, and so on), meaning that the calculation of an image may depend on the calculation of previous images. The immediate interest of these models is that first they are easy to implement on a computer and that they produce graphical representations, a precious aid in simulation, and second their power of expression; in fact, the entire group of cellular automata, after simple generalization of the preceding mechanisms (increase the size of neighborhood to consider when determining the next value of each cell), supplies yet another universal computing model, which in theory allows the implementation of any computable function. If we increase the computational capacity of each individual cell by making it compute functions instead of simply considering information coded as a bit (e.g. as in black/white, infected/not infected) and if we do the same thing by using information from neighboring cells, while preserving the key paradigm of locality, we naturally end up with multi-agent models. In theory, this model class is no more powerful than the previous class, but from an implementation point of view, it is considerably more practical in terms of algorithmic level translation of the local behaviors we wish to model. Used to represent the collective movement of shoals of fish or of flocks of birds [SCH 03], multi-agent models can also be used to model crowds, and this method is used, for example, in designing infrastructures where rapid evacuation needs to be possible, as mentioned above. The interest of these
Modeling and Simulation of Complex Systems
235
models is the ease with which a complex situation can be modeled: we must simply be able to specify how each individual agent evolves from one instant to the next based on interactions with a certain number of neighbors. Following this, step-bystep execution, that is, simulation, shows the descriptive capacities of the model by displaying possible global behaviors. Let us now consider the last family of models of interest to us: networks, also known as graphs in mathematics. Their definition is simple, and reuses the aspects used previously for automata: a finite group of nodes (known as vertices on a graph) and a finite group of connections between nodes (the edges of the graph). The nodes correspond to the entities considered and the connections to the existence (or non-existence) of interactions between two entities. Models of this type can be found in both natural and artificial systems: food chains, river networks (in this particular case, loops are rare, except in the case of occasional more or less stagnant “dead” water courses), road and rail networks, telephone networks, electric networks, Internet networks, social networks, and so on. The existence of interactions, and more generally of looped chains of interactions, is what makes these structures interesting: in cancerology, graphs are used where the vertices are genes or proteins and the edges are the interactions of regulation between vertices. The resulting general behavior is a combination of all interactions, either direct or induced by cycles in the graph. Moreover, graphs may be oriented, in the sense that an edge joining two nodes may have a “departure” and an “arrival” node. This notion is useful in showing positive or negative dependences between these vertices, as the orientation has an influence: in the first case, the quantities associated with the vertices vary in the same direction, and in the second case, one value decreases as the other increases. This type of model is widely used in qualitative physics [KUI 86, PUC 85], where, using a schematization of dependences between variables, we may determine the type of behavior shown (convergence to states, cycles, divergence, and so on) by the system thus modeled. Viewed from the perspective of graph theory, we are interested in dynamics of assembly – preferential attachment to poorly connected nodes, for example – and disassembly – the targeted removal of certain edges – of graphs, and in the influence of certain operations applied to graphs on different measurements of their performance. We may distinguish three broad families of graphs, which have different properties and toward which we may wish to tend: random graphs, scalefree graphs, and small-world graphs. In random graphs, the distribution of degrees of connection (the degree of connection of a node is the number of nodes to which it is connected) follows a random law. Graphs of this kind are, therefore, robust when faced with the random deletion of edges; on the other hand, they are not particularly effective in terms of transmitting information between two given nodes. In scale-free
236
Simulation & Modeling of Systems of Systems
graphs, the distribution of degrees of connection follows a power law (x↦xλ). Certain nodes, therefore, have a large number of connections, but the majority will have few connections. This can be observed in air transport networks, with hubs on the one side and regional airports on the other, where several changes are required to travel from one regional destination to another, but direct flights are available between major airports. Social and biological networks are also often of this type. Scale-free graphs are robust when faced with random deletion (the cancellation of a flight from a regional airport does not disturb traffic at hub level), but they are vulnerable to targeted attacks on nodes with a large number of connections (following terrorist threats to Heathrow Airport, all traffic passing through this hub was seriously affected). In terms of system design, this means that particular protection should be accorded to this type of node: this demonstrates the kind of immediate results obtained from topological analysis of a network. Finally, in small-world networks, each pair of nodes may be connected by a relatively short path: in some ways, this group is somewhere between a random graph and a regular graph. Food chains and certain social networks work in this way. These graphs are often characterized by their Rényi number, which is the average length of the shortest routes between any two vertices, something found in “popular” science under the name “degrees of separation”: for a given person, we might try to see if, in his acquaintance, we can find someone who knows a given actor or politician or someone who knows someone who knows that person, and so on. Experiments of this kind have been conducted, and the degree of separation within the human population is around 6, a number that intuitively seems remarkably small [BAR 03]. On the Internet, the degree of separation is closer to 19. Let us take the example of system of systems introduced at the beginning of this chapter, within which the socio-technical component (i.e. the human, and especially the organizational, dimension) is a key factor for success or failure. Certain views of this type of system can be modeled using networks, and the analysis of these networks becomes an effective means of identifying points for action in terms of maximizing the value chain within complex organizations: how might we improve the circulation of information through different levels of the company hierarchy? How can we optimize competences within the network and identify critical people or functions? How can we obtain scale effects in the proposed products or services? In particular, depending on chains of causality between the final user, their direct suppliers, and the various third parties that enable the supplier to provide that particular product or service, an organization will have different levels of performance, flexibility, and adaptability to, for example, the disappearance of one of these third parties or suppliers. By studying different networks from the point of view of their topology (degree of connectivity, study of transformation of topology when edges are removed, and so on), it is possible to identify both strong and weak points of organization and to suggest targeted means of improvement for certain properties.
Modeling and Simulation of Complex Systems
237
[CRO 09] shows how the analysis of social networks within a company, particularly in innovation, production, and sales teams, and of their environment reveals keys to improved performance. This systemic analysis of network topology aims to provide answers to certain questions. Are teams being influenced by the right people? Are they connected in an appropriate manner for the task in hand? Have they established adequate relations with the exosystem? Are value-added collaborations taking place within the network of teams? Does the quality of relationships within teams promote efficient collaboration (in terms of optimization of competences and access to the necessary expertise at the right moment)? Does the organizational context encourage real collaboration (complementarity of expertise and abilities, collaborative technology at company level, flexible processes?) The pitfalls of certain modes of managing staff and abilities are also analyzed in terms of the topology of their subjacent structure. An example of this kind of analysis is given in [MCK 06] where, in the framework of complex contracts, qualitative influences are studied within contractual relationships between suppliers and sub-contractors, between multiple suppliers and even between competing suppliers in acquisition plans. Note that the topological study of graphs does not provide quantitative information on these modeled systems, contrary to other models presented earlier. It does, however, provide a structural complement which gives additional information concerning certain measures of theoretical complexity. 5.4.3. Qualitative phenomenological approaches The models presented in the following sections are different from those previously described in that they do not have the same power of prediction, but they present an original approach to the explanation, comparison, and classification of systems which is useful in delimiting, if not mastering, the complexity of these systems. 5.4.3.1. Scaling laws Scale is a property of both space and time. It relates either to spatiotemporal resolution (a field of ripe wheat is a yellow polygon seen from an airplane or a more or less ordered group of stalks seen from the edge of the field) or to the extent of coverage of an analysis (we might look a few meters in front of us or look at several places and consider all of these points of view). It is obvious that depending on the scale, we will not be interested in the same aspects of distribution of sources of interest, nor in the same level of interactions between elements. Depending on the scale, then, we have a hierarchy of elements and interactions and look for invariants depending on the level considered and for interactions between different scale levels, which may have a negative effect (homeostatic loop leading to local
238
Simulation & Modeling of Systems of Systems
equilibrium, something often looked for when regulating a system) or a positive effect (auto-amplification processes, such as the greenhouse effect in climatology). Isometric and allometric analysis can be used to identify certain scaling laws, based on similar relationships between parameters at all levels considered, or on the contrary on instances where this symmetry is broken (i.e. discontinuities25), as they may be the result of a particular physical phenomenon which could be beneficially exploited. An example of a scale law of this kind was provided in the 1930s by Max Kleiber, a Swiss chemist studying the relationship between the mass of an animal and its metabolism, who proposed a universal formula: the quantity of energy burnt by unit of weight is proportional to the mass of the animal raised to the power of three quarters. In 1932, George Kingsley Zipf, a linguist at Harvard University, formulated the law that carries his name, another power law: for example, the number of towns with a certain number of inhabitants is expressed approximately as the inverse of the square of the number of inhabitants (this is why there are few very large cities, but a certain number of large towns and a very large quantity of small towns, as this distribution is linear on a logarithmic scale). The same applies to the presence of words of a certain length in the lexicon of a language and so on. The scaling laws cited above characterize the appearance of a phenomenon based on the scale of observation or appearance; there are other situations where the phenomenon is repeated identically from one scale to another in a way which could be said to be recursive. This is known as a fractal phenomenon, a concept first introduced under that name by Benoît Mandelbrot in 1976. Like the fern or the cauliflower, which present an analog structure when we zoom in on one of their details, we find these phenomena in both artificial and natural systems. The internet, for example, may be modeled as a fractal if we look at connection topology; the same applies to certain development models used in urban areas, but also to frontline evolution models in wartime [MOF 02]. 5.4.3.2. Application of catastrophe theory A well-known result of the work of the Italian geometrists of the 19th century is the knowledge that if we project a surface immersed in 3D space and defined by a polynomial equation onto a plan, we obtain a curve known as the visible surface (this is the principle at work in the shadow of an object projected onto the ground by the sun). This curve possesses singularities which almost always belong to a finite catalog: double points, cusps, and so on. This result, demonstrated by Whitney in the 25 One example of discontinuities can be found in groups of sociable insects, including the division of ants into groups of workers and soldiers, where no indications of this division are given initially (i.e. at the egg stage); the future attribution of an ant, while it is still an egg, is determined by the global state of the group at the time.
Modeling and Simulation of Complex Systems
239
20th Century and then generalized, led to the emergence of catastrophe theory. René Thom stated in his inaugural conference of the Colloque de Cerisy on “Logos and Catastrophe Theory” in 1982: we have an environment which is the location of a process, of any nature, and we distinguish two types of points, regular points and catastrophic points. Regular points are points where local phenomenological analysis reveals no particular accidents; variations may be observed, but these variations are continuous. On the other hand, we have points where phenomenological analysis reveals brutal and discontinuous accidents, and particularly observable discontinuities, which I call catastrophic points. (…) Under the hypotheses of genericity for dynamics, catastrophic points are not necessarily a bad thing; their structure is passably regular, for example locally polyhedral …. Mathematically, classification theorems can be obtained by particular dynamics and low dimension spaces: this is what René Thom called elementary catastrophe theory. The generalization to other dynamics and other dimensions may be interesting from an epistemological viewpoint, but is not yet based on exhaustive classification theorems, calling on the excessively abundant and complex theory of bifurcations of dynamic systems, limiting its functional applications. Moreover, modeling via catastrophe theory is qualitative and does not provide functional tools for quantitative prediction: using catastrophe phenomenology, we in fact reconstitute the dynamic of minimum complexity which may produce the observed set of catastrophes, and this is done without the application of isomorphisms, hence the absence of quantitative thresholds. However, looking more closely, this is what happens in other supposedly qualitative models, for example, the modeling of turbulence or the aerodynamics of rotary or beating wing engines. That said, the use of catastrophe theory fits into an invariant-seeking approach under so-called structural stability hypotheses – the idea that the global aspect of a system does not change as a result of small disturbances – and has a strong relationship with modeling approaches for complex systems (illustrations of this may be found in sociology, linguistics, plate tectonics, robotics, and so on; see [BER 77, GIL 81, PET 95, PET 98, POS 96, LUZ 00]). This paradigm can be characterized as “emergential” [BAR 96] in that it recommends the use of theoretical and practical results demonstrating the existence of emergent morphological and qualitative structures, by an organizing dynamic process, in physical substrates. This identifies an intermediary morphological level between the physical and symbolic levels. From the perspective of modeling, one consequence is that the same mathematical tools are used to model the organizational process of physical substrates and the process of organization at the highest level.
240
Simulation & Modeling of Systems of Systems
Morpho-dynamic models are therefore used as models of the categorization process in a way described in the following section. Consider a complex system, with attractors and their basins of attraction, that is, regions where, when a trajectory begins, it is attracted by the attractor. A result of the theory of dynamic systems affirms that almost all trajectories in the system will converge toward one of these basins. Let us discuss each basin as a particular category. Category changes are therefore expressed, from the viewpoint of system dynamics, as a bifurcation phenomenon controlled by physical parameters, intrinsically linked to the structural instability properties of the system. In addition to categorization of a space at state level then in its globality, this approach also allows local categorization, that is, (more precisely) a classification of the systems which depend on the local behavior of these categories. Starting with an equational model which fits data in a certain domain of validity, we observe what happens around a point of operation which is of particular interest from the point of view of the application. At local level, we look at the local form of the equations which define the associated reference catastrophe. This gives an idea of the behaviors that are most likely to occur locally depending on the evolution of different variables. To give an analogy, this is what is often done around a regular point in sufficiently smooth systems, where each system may be assimilated locally, at the point considered, to its linear approximation. The analogy to a catastrophe is, in this case, the constant function, and we can effectively say that all the systems are locally the same at such a point, meaning that such regular points cannot be used to classify different systems (it is for this precise reason that catastrophe theory was developed and that it applies particularly to critical points). The morpho-dynamic approach is attractive, but subject to certain limitations. First, the underlying mathematical tools are highly non-constructive, and second, morphology can be formalized in topological and geometric terms, but not, a priori, in logical terms, a fact which raises certain issues when using it in conjunction with other models. An interesting subject for research [BAR 96] would be to try and develop a scientific theory which could link formal structures at symbolic level – which, in one way or another, are logical forms – to the dynamics which govern the physical level. 5.4.4. Qualitative structuralist approach: application of category theory First, remember that category theory aims, among other things, to provide a common language and a set of unifying concepts for different branches of mathematics: thus, an application between sets and a homomorphism of algebraic structures, or a continuous transformation between topological spaces, are united
Modeling and Simulation of Complex Systems
241
by the common concept of morphism between the corresponding categories (in this case, the category of sets, that of the algebraic structures in question, that of topological spaces, and so on). Due to these unifying concepts, analog results are, in fact, represented by a single result, allowing deeper understanding of the problems in that the subjacent mechanisms are revealed; these were initially hidden by technical details linked to particular structures for which results had been set out. One of the cornerstones of category theory is the systematic adoption of a relational viewpoint: everything may be defined as an arrow (a relationship or a transformation or, more precisely, a morphism) between objects. These objects themselves may be described by exclusive use of arrows (the preconceived conceptual difference between an object and an arrow is simply that arrows can be combined with other arrows and therefore have a “from” and a “to”, but an object may also be defined as an arrow for which the “from” and “to” coincide). Here, the contribution to systems theory is clear: the systemic paradigm and category theory meet in the pre-eminence they accord to relational ontology, on both a dynamic and a static level, with the establishment of the evolution of relational structures. This is, moreover, where we find the main mathematical difference between category theory and set theory. Although the latter characterizes mathematical objects by describing their internal structure, that is, by separating them into different parts and elements, the former takes the opposite point of view and characterizes an object by its connections to other objects. It is interesting to look at the development of systemics in connection with the emergence of category theory. Both reached maturity in the mid-20th Century, as a logical progression of currents of structuralism, and underwent a “golden age” in the 1960s and 1970s. Both then took a back seat during a period where totally analytical approaches were favored, accompanying the generalized reductionist movement in the sciences (promoted by the hope of a take-off in computer processing resources), only to return to favor in recent years as a methodological complement providing global (diachronic and synchronic) vision. Category theory is, moreover, used as much in theoretical computing (modeling information systems, databases, multi-processor languages, and so on) today as in theoretical physics, where it is used in attempts to define a unified framework of physics26. A relational vision of this kind has two consequences. First, an object is only described up to isomorphism, that is, independently of a particular implementation. Second, the strictly representational aspects of an object are removed in favor of the structure of inter-object relationships, a process which reminds us of the key notions of architecture in systems engineering. Moreover, one aim of category theory is
26 On this last point, see the various publications of I. Raptis, A. Döring, C.J. Isham, J. Butterfield, A.K. Guts, and so on available online.
242
Simulation & Modeling of Systems of Systems
to provide a precise definition of “naturality” or so-called universal relationships between properties (remember the basic results, such as the uniqueness of the base of a vector space up to isomorphism, or the use of representatives of equivalence classes to define operations independent of the particular representative, used to define addition on rational numbers and on real numbers as an extension of addition on natural integers). This naturality is used, among other things, to define natural transformations, a demonstration of the intuitive idea that complicated things may be transformed into other complicated things if we modify the corresponding sub-structures correctly and if the transformation is not too ad hoc in the sense that it may be defined by a general mechanism applying to every object being considered. Furthermore, category theory is particularly suited to consider certain representations frequently found in computing, such as automata, tree diagrams and graphs, and object-oriented structures. In this last case, it provides a practical means of dialog between specifications and implementations [FIA 05]. In fact, category theory has another role to play, albeit one of lesser interest to us in our current discussion: it is used in the search for a coherent language which would allow us to establish the foundations of mathematics, effectively the quest for the Holy Grail of meta-mathematics. We shall now provide a rapid overview of certain basic concepts involved in this theory and provide illustrations of its contribution to the modeling of complex systems. A category is specified by a class of objects and a class of arrows, also known as morphisms. An identity arrow exists for each object, and for each arrow at the end of which a second arrow begins, we can define the way in which these two arrows fit together. Moreover, three arrows which may be put together do this in an associative manner, that is, the first two may be put together with the third or the first may be combined to the last two. A category may therefore be seen as an oriented graph, where the vertices are objects and the edges are the arrows. By applying definitions recursively, we can consider a particular category of which the objects are themselves categories: in this case, the morphisms between these composite objects are known as functors. If we clarify the action of a functor, we notice that a functor between categories may be defined as mapping objects of the first category to objects of the second category, so that a morphism between two objects of this first category is mapped onto a morphism between object images. Thus, a functor is a transformation which conserves, in a certain way, the base structure between categories considered. If the objects correspond to system components, and morphisms to links between components (the interpretation of which depends, for example, on the type of architectural view being considered), a functor is a translation between two different architectural views. Taking a step back, consider the category of which the objects are functors: in this case, the morphisms are the natural transformations discussed above. Once
Modeling and Simulation of Complex Systems
243
again, we must define a property between “individual” objects of the categories concerned. The general idea is not to transform a category into another category by observing how the subjacent relational categories interact (this was the role of functors) but to see how this transformation may be parametered at the level of the objects which make up the category. In this way, we model the situation where a global transformation between complex architectures is carried out through local transformations at different levels of zoom. We shall now introduce one last notion, a particular universal notion, that of limit. Let us take a diagram (i.e. objects and arrows between them): an object is “limit” if there are morphisms from this object toward each object in the diagram and if, in addition, each object satisfying this same property is such that there is a single morphism between it and the limit object. Let us consider how this principle can be applied to the modeling of complex systems: consider the basic elements of a complex system as objects and the relationships between them (energy flow, channels of transmission, proximity relationships, transition functions, and so on) as arrows, producing a diagram within a category on the condition that the minimum requirements in terms of mathematical properties are verified. A limit object, then, in some way subsumes the primitive elements, in that it represents a sort of invariant associated with the diagram, which may then be seen as its internal organization. If we apply this kind of approach recursively, we can define a hierarchy where, at each level, the objects are limits of objects at the lower level. A representation of this kind is coherent with the idea that complexity is a relative notion and depends on the level of observation. Applied to different views of the systems which make up a system of systems, for example, it also provides a means of describing the links and hierarchies between these different visions. The above description gives only a broad overview of the use of category theory in modeling complex systems. To go further, we could enrich objects in terms of structure, particularly to consider aspects of dynamic evolution; see [LUZ 98a, LUZ 98b]. An approach of this kind seems to be able to provide some answers to the questions raised at the end of the previous section. The above sections have dealt with the use of a category-based approach to the modeling and simulation of large complex systems (“large” due to the spatial, structural, and/or dynamic extension of the components involved). Although this approach may seem strange at first, this is essentially due to the way in which mathematics is taught. For this approach to be useful in terms of the results obtained, a certain “philosophical” vision is required – involving an understanding of the relational environment of the object studied, and therefore a holistic approach with more significant investment – along with a level of technicity guided by abstractional appetence, or a representational mindset, as the quest for and demonstration of results involves much diagram chasing. This is not, however, too
244
Simulation & Modeling of Systems of Systems
different from the approach used by architects who, far from a specific implementation, must be able to take on board and connect different points of view of a system, or by simulation experts who must be able to quantify these points of view to give life to all useful representations of the system. 5.5. An example: effect-based and counter-insurgency military operations Effect-based military operations have been “in fashion” in different doctrines over the last decade, although the concept was already in use during the Vietnam War and even during certain phases of WWII. An effect is defined in document JP5-0 (Joint Operation Planning, from the US Department of Defense) as “the physical and/or behavioral state of a system which results in an action, a set of actions, or another effect”. The “traditional” vision involves the conversion of desired outcomes into objectives then into missions which can be broken down into tasks and intentions; the Effect-Based Operations (EBO) vision, on the other hand, starts from the final state desired and translates it into missions with objectives and effects. Leaving aside semantic quarrels and doctrinal debates, it is interesting to note that the document Air Force Doctrine Document 2 (AFDD2 – Operations and Organization, published in 2008 by the US Air Force) highlights the fact that conflicts result from the interaction of “adaptive complex systems” and that war is “complex and non-linear”. Conflict resolution may be represented in a twodimensional space, with means on one axis (classified from destructive methods to means of influence) and ends on the other (from physical to psychological). We can then classify different conflicts by interactions and couplings, particularly in terms of the relationship between cause and effect. [JOB 07] distinguishes four different classes in this way (simple, complicated, complex, and chaotic) and represents them using the two-dimensional plan mentioned above, from the resolution of a simple conflict by a pairing of means of destruction and physical results, up to a chaotic conflict where resolution is attained through means of influence with psychological results. This grid places an emphasis on the non-linear nature of military operations and attempts to reach conclusions on how they should be conducted, using analogies with the theory of complexity. This is far from the old world of schemas with linear relationships between cause and effect, the concrete form of which was confrontations between armies, with increasing lethality as the relationship between the forces present grew. This context provides a potential field of application for complex system modeling and simulation techniques. It is particularly relevant for military strategists at the present time as current conflicts appear to have been planned without the necessary consideration of their complex character, particularly the interweaving of different dimensions (physical, psychological, political, and cultural) and the principle of non-linearity (a local action does not necessarily produce only local results, results
Modeling and Simulation of Complex Systems
245
which are not necessarily proportional to the action, and may work with a certain time delay and in the opposite direction to that expected). We do not claim to offer answers here, but we wish to highlight the importance of using the correct paradigms when reasoning to increase the chance of finding a possible solution. .
The domain of counter-insurgency operations (sometimes encountered under the acronym COIN) has also been the subject of renewed interest in recent years in the context of conflicts in the Middle East. It is interesting to tackle this problem from a systemic viewpoint in attempting to provide elements of a response27. We are, in fact, dealing with various groups (factions, networks, and so on) that sometimes cooperate, but may potentially be in competition; these groups have various aims but are all opposed to an authority constituted for the control of a territory and a population. Note the heterogeneity, pairings between elements and the environment, the dynamic nature of pairings and the different dimensions of the problem and of motivations, and the impact of external pairings with the environment on internal structure (counter-insurgency operations attempt to play on these external pairings to destabilize the internal structure of the system as far as possible). Counter-insurgency consists of organizing and leading all military, paramilitary, political, economic, psychological, and civic actions at government level to destroy an insurrection. This is a clear illustration of an attempt to produce systemic effects. Simple Lancaster-style models of attrition do not allow us to consider the dynamics of these linked systems: the insurgents do not work using a closed system – which would mean they had no external support – and the elimination of insurgents is not enough to stamp out the insurgency completely. Moreover, conflicts of this kind are not a simple matter of two identified opposing forces, but involve at least three groups (insurgents, the authorities, the population) with more than one highly permeable frontier between participant groups (the population is more or less neutral and may join either camp depending on circumstances, and there may be defections from insurgent ranks and from the official troops). For this reason, exchanges between these participants cannot be ignored. The situation therefore begins to resemble an ecosystem, where several species are in competition for a set of resources, considering the phenomena of migration and reproduction [DRA 08, SOL 06]. [DRA 08] considers decision trees as in game theory, with values which may be variable; [SOL 06] considers coupled evolution equations
27 Note that the strategies of this combat method, used by T.E. Lawrence against the Turks, Mao Tse-Tung (China), Vo Nguyen Giap and Ho Chi Minh (Vietnam), the Sandinists (Nicaragua), then during the Intifada and al-Aqsa Intifada (Israel/Palestine), and most recently al-Qaeda, themselves use a systemic approach in their texts inciting insurrection. Further reflection on this can be found in [HAM 06], which provides a posteriori justification of the use of systemics as an analytical tool for fighting insurgency.
246
Simulation & Modeling of Systems of Systems
with occasional use of macro-variables for a posteriori analysis of the behavior of certain equations around equilibrium values (see the principle of subjection). In both cases, the results obtained directly depend on the intrinsic limits of the model being used. The combination of these methods may be useful in understanding and then trying to control systems with the aim of obtaining a particular trajectory (usually one which results in the neutralization of the insurrection with the least possible negative impact on the population). The idea is therefore to make use of various models, to find analogies between modeled systems (in particular concerning certain dynamic regimes, in the hope of obtaining the regime seen as presenting the greatest performance in one of the models at system level) and critical analysis of certain strategies so that if, once the analogy is established, the effect observed is not that shown by one of the models, doubts can be raised as to its use for that system. Evidently, these considerations are all based on the fundamental possibility of access to data to support the models; otherwise, all extrapolation is pointless. The above example also shows the richness of trajectory behaviors (the ecological models in question have the various characteristics of complexity discussed throughout this chapter), and it is therefore sensible to apply all the precautions mentioned above so as to avoid planning useless actions. 5.6. Conclusion Throughout this chapter, we have attempted to underline certain fundamental characteristics which must be considered in the modeling and simulation of complex systems, whether natural or artificial. These characteristics may appear to be based on theoretical considerations, even though we took great care not to go into excessive mathematical detail, and the reader would be right to wonder whether what we presented is really motivated by practical considerations and relevant to the field of application of the majority of engineering work. Our response to this question is, of course, yes, and this approach is not unique to us: other recent articles in technical journals use this same focus [SHE 06, SHE 07], although we must acknowledge that there is currently little interest for the subject in the engineering community as a whole. This is mainly due to the fact that theoretical developments require sophisticated mathematical tools, the use of which is reserved to particular system classes and which are not particularly easy to adapt to the kinds of systems handled by engineers. This difficulty of use in conjunction with practical applications is clearly a major obstacle. We should also suggest that in the majority of applications developed up until now, conditions are such that the use of these tools and the management of these limitations could reasonably be left aside (although this is not to say that they would have been useless). In an engineering
Modeling and Simulation of Complex Systems
247
logic which was principally oriented toward industrial products28, the key words were control, stability, and predictability. To achieve these aims, engineers applied the principle of systematic decompositions to the object being studied, advocating the independence of components as far as possible. For this, fairly long temporal cycles were sufficient, and necessary for an approach based on the sequence of expression of need, design, and production, use of the product and, later, periodic updates. For as long as the conditions for success of this kind of approach were met, there was, a priori, no need – except in particular cases – to worry about the potential consequences of uncontrolled complexity. However, in recent times, conditions have been changing. The field of engineering has turned toward information services29 seeking flexibility, adaptability, and a logic different from that of total control (illustrated by the use of “total quality” principles): interest is now directed toward providing capacities and seeking value (following the successive introduction of principles of “quality assurance” and “maturity” of supply of products or services, we are now in a phase of “value assurance”). The breakdown into independent components is replaced by a linking and combination of existing blocks, tangible or otherwise – that is, off-the-shelf products or services – for economic reasons: first, it would be unthinkable to redo all the preliminary work for each new need, and second, the time taken to launch a product responding to the expressed need must be as short as possible. This tendency is reinforced by evolutions in society, with increased focus on the user and on immediate and personalized satisfaction. The reuse and combination of elements, including (or especially) outside of the domain in which they were initially guaranteed to function, is becoming increasingly common, a situation which often necessitates forays into the science of complex systems. We find ourselves in a phase of paradigmatic change, and use of this new logic of combination of services on demand is experiencing exponential growth in extremely varied domains of application. This is even the case in those domains, such as defense and transport, which seemed the least susceptible to use this kind of logic, as it places local user satisfaction above control of risk and operation. There have already been a number of real-world demonstrations of the effective risks and dangers of a lack of management of complexity, including breakdowns in the energy supply systems in the United States and Italy and the recent financial crisis. Let us hope that problems of this kind will not emerge in the defense sector (the kind of problem often seen in disaster films, where information, communications, and control systems develop emergent capacities for autonomous decision-making and suchlike!) nor in the field of air transport (which has proved surprisingly resilient for the moment, with 28 To re-cite the definitions provided by the EIA632 or the IEEE1220 norm, the product is the result of an activity or a process, usually considering a material form. 29 According to the AFNOR-ITIL/ISO20000 norm, a service is a composable immaterial provision, manifested in a perceptible manner and which, in predefined conditions of use, is a source of value for the consumer and the supplier.
248
Simulation & Modeling of Systems of Systems
the capacity to tackle crises without uncontrolled propagation, e.g. the cancellation of all flights to and across the United States on and after September 11, 2001, or the sudden changes in security procedures following threats to London Heathrow in August 2006, which caused enormous queues, were dealt with without causing major repercussions elsewhere). Faced with these changes, traditional engineering problems, based principally on hypotheses involving closed, linear, decomposable, predictable systems with a clear understanding of initial conditions and disturbances, are transformed: the systems under consideration are mostly open, non-linear, and interdependent, with a finite horizon of predictability and little understanding of initial conditions and disturbances. The engineer is therefore confronted with complexity and cannot ignore fundamental characteristics, as they leave the field of ad hoc examples to join that of concrete applications. What, then, can we learn from these pitfalls and limitations which we have touched upon in the preceding sections? First, great care is necessary in not reaching conclusions too quickly, either by intuition or by analogy with other situations. We have already seen the pitfalls created by non-linearity. Second, great importance should be accorded to interactions and thus to the place of man in the global system. From questions of computability of human reason notwithstanding, it is clear that it follows mechanisms which we cannot reliably reproduce: capacities of induction, reliability when faced with errors, but also unpredictable generation of errors (to err is human, whether this be due to fatigue, stress, or a sudden access of incompetence) make human reason a key element of the global architecture, with corresponding strengths and weaknesses. Human behavior, which is fundamentally complex, should be considered as far as possible in the engineering progress. We should remember that the human factor has resulted in the loss of battles which were considered to be won in advance and, conversely, is capable of producing success against all odds. Finally, we should make flexibility and agility an aim from an early point in the engineering cycle to favor adaptability and resilience, and we should be able to question the system architecture if the planned use requires it. We should focus on the traceability of options and decisions made during the decomposition and integration of existing blocks and components developed for the specific project, in order to be able to go back if necessary. We should aim for a reasonable mastery of unplanned occurrences rather than permanent control, with the ability to identify and evaluate in permanence and to react rapidly. How, then, can we deal with these challenges? The solution involves modeldriven engineering, accompanied by judicious interpretation of these models,
Modeling and Simulation of Complex Systems
249
following the example of the living world, where this representation and imagination-based approach (construction and use of models to choose the action to carry out a posteriori) have been the key to the survival and evolution of mammals and humans in particular. By mastering the tools of complexity, we avoid a certain number of crises that would inevitably be provoked by the complexity we aim to apply at all costs. 5.7. Bibliography [BAD 97] BADII R.M., POLITI A., Complexity: Hierarchical Structure and Scaling in Physics, Cambridge Non-linear Science Series 6, Cambridge University Press, Cambridge, 1997. [BAD 07] BADIOU A., Le concept de modèle (new edition), Fayard, Paris, 2007. [BAR 96] BARTHELEMY J.-P., DE GLAS M., DESCLES J.-P., PETITOT J., “Logique et dynamique de la cognition”, Intellectica, vol. 23, pp. 219–301, 1996. [BAR 03] BARABÁSI A.-L., Linked, Penguin Group, New York, 2003. [BER 77] BERRY M.V., MACKLEY M.R., “The six-roll mill: unfolding an unstable persistently extensional flow”, Philosophical Transactions of the Royal Society of London, vol. 287, no. 1337, pp. 1–16, 1977. [CRO 09] CROSS R., THOMAS R.-J., Driving Results Through Social Networks, John Wiley & Sons, San Francisco, 2009. [DRA 08] DRAPEAU M.D., HURLEY P.C., ARMSTRONG R.E., “So many zebras, so little time: ecological models and counterinsurgency operations”, Defense Horizons, a publication of the Center for Technology and National Security Policy, National Defense University, no. 62, February 2008. [FIA 05] FIADEORO J.L., Categories for Software Engineering, Springer-Verlag, Berlin, 2005. [FOU 92] FOULLOY L., “Commande qualitative et commande floue: vers une méthode d’écriture”, 12th International Conference on Artificial Intelligence, Expert Systems, Natural Language, Avignon, 1992. [GEL 94] GELL-MANN M., The Quark and the Jaguar, W.H. Freeman & Co., New York, 1994. [GIL 81] GILMORE R., Catastrophe Theory for Scientists and Engineers, John Wiley and Sons, New York, 1981. [GOL 02] GOLUBITSKY M., STEWART I., The Symmetry Perspective, Birkhäuser Verlag, Basel, 2002. [HAK 00] HAKEN H., Information and Self-Organisation: a Macroscopic Approach to Complex Systems, Second enlarged edition, Springer-Verlag, Berlin, 2000. [HAK 04] HAKEN H., Synergetics – Introduction and Advanced Topics, Springer-Verlag, Berlin, 2004.
250
Simulation & Modeling of Systems of Systems
[HAM 06] HAMMES T.X., The Sling and the Stone: on War in the 21st Century, Manas Publications, New Delhi, 2006. [HOP 97] HOPPENSTEADT F.C., IZHIKEVICH E.M., Weakly Connected Neural Networks, Applied Mathematical Sciences, Springer-Verlag, New York, vol. 126, 1997. [ISI 89] ISIDORI A., Non-linear Control Systems, Springer-Verlag, New York, 1989. [JOB 07] JOBBAGY Z., “Effect-based operations and the problem of causality: simple and complex”, Joint Force Quarterly, National Defense University Press, vol. 3, no. 46, pp. 90–95, 2007. [KLU 08] KLUGER J., Simplexity: Why Simple Things Become Complex (and How Complex Things Can Be Made Simple), Hyperion, New York, 2008. [KUI 86] KUIPERS B., “Qualitative simulation”, Artificial Intelligence, vol. 29, pp. 289–338, 1986. [LUZ 93] LUZEAUX D., “Intelligent control, ergodicity and chaos”, International Simulation Technology Multiconference, Simtech-Wnn-Fnn 93, San Francisco, November 1993. [LUZ 95] LUZEAUX D., MARTIN E., “Intelligent control: a theoretical framework”, 3rd SIAM Conference on Control and Its Applications, St. Louis, Missouri, April 1995. [LUZ 97] LUZEAUX D., ANTONIOTTI J.-F., “Mathematical system theory and digital control”, Southeastern Symposium on System Theory, Cookeville, Tennessee, March 1997. [LUZ 98a] LUZEAUX D., “Towards the engineering of complex systems”, Journées Nîmes 98 sur les Systèmes complexes, systèmes intelligents et interfaces, Nîmes, France, 1998. [LUZ 98b] LUZEAUX D., “Category theory applied to digital systems theory”, 4th World Multiconference on Systemics, Cybernetics, and Informatics, Orlando, Florida, July 1998. [LUZ 98c] LUZEAUX D., “Stabilizing stationary linear discrete systems: minimal and balanced expansions in any real base”, Journal of Mathematical Analysis and Applications, vol. 224, pp. 117–130, 1998. [LUZ 00] LUZEAUX D., “Catastrophes as a way to build up knowledge for learning robots”, 16th IMACS World Conference, Lausanne, Switzerland, 2000. [LUZ 08a] LUZEAUX D., RUAULT J.-R., Systèmes de systèmes: concepts et illustrations pratiques, Hermès, Paris, 2008. [LUZ 08b] LUZEAUX D., RUAULT J.-R., Ingénierie des systèmes de systèmes: méthodes et outils, Hermes, Paris, 2008. [MAR 94] MARTIN E., LUZEAUX D., “Computability in control”, 3rd International Symposium on Artificial Intelligence and Mathematics, Fort Lauderdale, Florida, January 1994. [MCK 06] MC KENNA N.A., The micro-foundations of alignment among sponsors and contractors on large engineering projects, PhD Dissertation, MIT Engineering Systems Division, Cambridge, USA, 2006.
Modeling and Simulation of Complex Systems
251
[MOF 02] MOFFAT J., Command and Control in the Information Age, TSO, Norwich, 2002. [MUR 03] MURDOCK J., Normal Forms and Unfoldings for Local Dynamical Systems, Springer-Verlag, New York, 2003. [NIJ 90] NIJMEIJER H., VAN DER SCHAFT A.J., Non-linear Dynamical Control Systems, Springer-Verlag, New York, 1990. [PET 95] PETITOT-COCORDA J., “Morphodynamics and attractor syntax: dynamical and morphological models for constituency in visual perception and cognitive grammar”, in PORT R., VAN GELDER T.J. (editors), Mind as Motion: Explorations in the Dynamics of Cognition, A Bradford Book, The MIT Press, Cambridge, Massachusetts, pp. 227–281, 1995. [PET 98] PETITOT-COCORDA J., Morphological Eidities for Phenomenology of Perception, Stanford University Press, Stanford, 1998. [POS 96] POSTON T., STEWART I., Catastrophe Theory and Its Applications, Dover, Mineola, 1996. [PUC 85] PUCCIA C.-J., LEVINS R., Qualitative Modeling of Complex Systems, Harvard University Press, Cambridge, 1985. [SAL 91] SALANSKIS J.-M., L’herméneutique formelle: l’infini, le continu, l’espace, Editions du CNRS, Paris, 1991. [SAL 92] SALANSKIS J.-M., SINACEUR H., Le labyrinthe du continu, Springer Verlag, Paris, 1992. [SAL 99] SALANSKIS J.-M., Le constructivisme non standard, Presses universitaires du Septentrion, Lille, 1999. [SAS 99] SASTRY S., Non-linear Systems: Analysis, Stability and Control, Interdisciplinary Applied Mathematics, Springer Verlag, New York, vol. 10, 1999. [SCH 03] SCHWEITZER F., Brownian Agents and Active Particles: Collective Dynamics in the Natural and Social Sciences, Springer Verlag, Berlin, 2003. [SCO 07] SCOTT A.C., The Non-linear Universe: Chaos, Emergence, Life, Springer-Verlag, Berlin, 2007. [SHE 06] SHEARD S., “Definition of the science of complex systems”, INSIGHT Review published by the International Council on System Engineering (INCOSE), vol. 9, no. 1, pp. 25–26, 2006. [SHE 07] SHEARD S., Principles of complex systems for systems engineering, online communication from the Third Millenium Systems company LCC, 2007, http://www. 3MilSys.com. [SOL 06] SOLÉ R.V., BASCOMPTE J., Self-Organization in Complex Ecosystems, Princeton University Press, Princeton, 2006. [SOR 04] SORNETTE D., Critical Phenomena in Natural Sciences: Chaos, Fractals, SelfOrganization and Disorder: Concepts and Tools, Springer-Series in Synergetics, SpringerVerlag, Berlin, 2004.
252
Simulation & Modeling of Systems of Systems
[THO 72] THOM R., Stabilité structurelle et morphogénèse, Ediscience, Paris, 1972. [THO 90a] THOM R., “La boîte de Pandore des concepts flous”, in Apologie du Logos, Hachette, Paris, pp. 587–588, 1990. [THO 90b] THOM R., “Ambiguïté de la complexité en biologie”, in Apologie du Logos, Hachette, Paris, pp. 219–231, 1990. [WEI 00] WEIDLICH W., Sociodynamics: a Systematic Approach to Mathematical Modelling in the Social Sciences, Dover Publications, Mineola, 2000. [ZOL 08] ZOLESIO J.-R., “Protection des infrastructures critiques”, in LUZEAUX D., RUAULT J.-R., (editors), Systèmes de systèmes: concepts et illustrations pratiques, Hermès, Paris, 2008
Chapter 6
Simulation Engines and Simulation Frameworks
6.1. Introduction A typical simulation architecture is illustrated in Figure 1.7: the elements included in this figure can be found in more or less any simulation. Simulations, therefore, share numerous common points, a fact that is reinforced by their homogeneous architecture: given two discrete event simulations, a considerable part of the code is, or could be, identical. In such cases, we typically find that between 40% and 80% of services and objects are similar if not identical. This raises an important point: what changes from one simulation application to another is essentially the code of the simulated system. We have, then, an important source for the optimization of developments, by factorizing common services within a given simulation domain. This is the purpose of simulation engines and frameworks, which provide the basic materials to construct simulations more quickly and at reduced cost. We could even envisage a situation where a library of “off-the-shelf” models would be available, enabling simulations to be built without writing a single line of code. This is perhaps just a dream for now, but it is possible, and certain cases already exist: in a simulation environment such as Matlab/ Simulink, for example, with a rich repository of generic models, a large number of systems can be modeled graphically using pre-defined components, then animated without resorting to programming.
Chapter written by Pascal CANTOT.
254
Simulation & Modeling of Systems of Systems
The aim of this chapter is to see, in concrete terms, how models are animated and how a simulation is built. Understanding these basic mechanisms allows us to identify potential problems (e.g. non-causality of events) and to reconsider the validity of the simulation. This will also help us to specify and use the simulation more effectively by rationalizing developments. To do this, this chapter will be divided into three sections. First, we shall provide an overview of the basic core mechanisms of a simulation to understand the possible implications of the choice of given types of simulation. Second, we shall explore the question of simulation engineering and discover what strategy should be adopted for the development of high-performance complex simulation applications. Finally, we shall discuss the implementation of strategies for reuse of simulation components. These three aspects will be illustrated using real-world examples. 6.2. Simulation engines The simulation engine is the basic component of a simulation. It animates the simulation, specifically by the management of objects, the computation of state variables, the management of events, and the advancement of (discretized) simulation time. It may take various forms depending on the nature of the simulation (discrete event, step by step, real time, hybrid, and so on). The simulation engine is not the only required component. Every simulation needs a certain number of additional services to function: graphical display, the ability to read and interpret data files, debugging, network communications, statistical gathering, mathematical functions, a modeling workshop, and so on. These services can be found in the simulation framework or in a simulation support environment (SSE). 6.2.1. Evolution of state variables In Chapter 2, we defined the state of a system as the set of state variables. However, this definition was somewhat simplified, and as we shall see, the set of state variables is not sufficient to predict the evolution of the system. A state variable x may evolve in several ways: – by explicit evolution depending on time; – by implicit evolution, determined by a differential equation, a logical condition on the system state (state event), an event destined to trigger a given action at a given time (dated event), or an external event (e.g. an action by the operator).
Simulation Engines and Simulation Frameworks
255
We defined the state of a system above as the set of state variables. To this, we should now add the set of state events and dated events. It is then possible, from this new extended system state, to predict its evolution in the absence of external events. If our system X is broken down into n sub-systems, X1, …, Xn (which is usually the case), its state X(t) at the instant t will be described using the values Xi(t), the state of each sub-system of X, and W(t) representing possible state variables which only concern the global system (e.g. time) and not the sub-systems. W(t), therefore, represents global variables. We say that a sub-system Xi is autonomous if the laws of evolution of the state Xi do not depend on the other states Xj(t) when j ≠ i (for the determination of both state variables and events). It may then be independently simulated of the other sub-systems, a property that is useful when designing the structure of simulation components: this sub-system may easily become an independent model. If, on the other hand, the laws of evolution of Xi depend on another sub-system Xj, we say that Xi is coupled to Xj. Two different cases are then possible: – if Xj is autonomous, we can determine the state of Xj, then that of Xi (asynchronous or retarded simulation); – if Xi and Xj are coupled in both directions (interdependent), then they must be simulated simultaneously (synchronous simulation). Once again, this has an impact on the models used. 6.2.2. Management of events and causality Events are sudden changes in system behavior, for example, the failure of a component, the starting of a missile engine, collision with an obstacle, or the entry of a command by a user. Events may be triggered by various means: – by the scenario (e.g. a production chain may be programmed to stop after 6 h of cumulated operation); – by scrutinizing the system state (e.g. safety stops on a production line if the temperature goes above 50°C) or statistical variables (halt for maintenance on the production line once 1,000 objects have been produced); – by the user, if the simulation is interactive or open (emergency halt on a production line if the simulation user presses a pre-defined key).
256
Simulation & Modeling of Systems of Systems
If the event is triggered by a condition on the system state, it is known as a state event. If the event is triggered at a given moment (in the logical time of the simulation), it is known as a dated event. Events must be precisely dated. Typically, an event is a complex data type, including a description of the event (type and parameters), the possible origin, and a time stamp. Events should generally be processed in chronological order to respect the principle of causality. Take the example of a tank firing on a vehicle in a war game using discrete time, where the time step used is one minute of simulated time. The time taken to fire the shell and to reach the target is less than a minute, and this event happens entirely within one time step. If the simulation engine does not respect the chronological order of events, the event “target destroyed” may be processed before the event “departure of shell”, which would clearly be absurd and could falsify simulation results. This is a notion already discussed in Chapter 4, but we shall recollect here, as it involves a fundamental service of numerous simulation engines: management of a scheduler by the simulation engine, which guarantees that events are processed in the correct order. In practice, this usually boils down to managing an event queue, usually the first-in, first-out (FIFO) variety. Nevertheless, in distributed simulation, this process is much more difficult: the principle of causality must be respected not only within a simulation engine but also between several engines, as several simulation applications are made to cooperate in distributed simulation. The issue of event causality in distributed simulations is a complex subject, and although mechanisms exist to guarantee causality, they are not always easy to use and a risk of causality problems is often present in distributed simulation. The possible effects of this on the validity of a simulation, therefore, need to be evaluated. 6.2.3. Simulation modes Simulation modes characterize the way in which state variables are integrated (in a discrete or continuous manner) within a simulation loop performing iterative calculations on the system state. 6.2.3.1. Continuous simulation In a continuous simulation, variables are continuous and the events are state events. The iteration of the loop in the simulation engine thus includes the following two actions: – integration of differential equations describing the system state;
Simulation Engines and Simulation Frameworks
257
– state tests to check for state events; if state events occur, then corresponding action is taken. Continuous simulations (detonics, aerodynamics, nuclear physics, and so on) are typical examples of this kind of simulation. 6.2.3.2. Discrete simulation In a discrete simulation, variables are discrete and events are dated. The iteration of the loop of a discrete simulation engine typically includes the following sequence: – seek the next event, that is, the event with the closest date in the list of waiting events; – advance the simulation date to this point, as nothing can happen between the current date and the date of the next event; – apply the actions corresponding to this event. Discrete event simulations can be found in the modeling of processes with stochastic elements, for example, production lines, queues, road traffic, and so on. 6.2.3.3. Mixed simulation It is not always possible to model a problem using continuous simulation or discrete simulation alone, sometimes we need to combine both. Thus, when simulating combat between two airplanes, we would use continuous models (flight models) with discrete events, and both dated (scenario markers and maximum flight duration for a missile) and state events (detection, missile launch, and explosion of the shell near the target). In a mixed simulation, then, variables and events can take different forms. The simulation engine proceeds as follows: – seek the next event (En which occurs at date tn); – integrate the differential system until tn, test conditions (for state events) and effectuate corresponding actions; – apply the actions corresponding to the dated event En. We may encounter difficulties if one of the actions carried out in the time interval [tn−1, tn] modifies the event scheduler, that is, by generating an event before date tn. This is an example of the causality problems that can arise.
258
Simulation & Modeling of Systems of Systems
6.2.4. Example We shall illustrate these different approaches using a concrete example, based on an original idea by Christian Sautereau, the main creator of the ESCADRE framework (which we will discuss later). We shall return to the example of the falling ball (section 1.4.1.1), ignoring the influence of factors other than gravity g. To make the example more illustrative, we shall suppose this time that when the ball hits the ground, it will bounce with a damping factor of k (Figure 6.1); it is not a billiard ball, then, in this case, but perhaps a tennis ball.
Figure 6.1. Trajectories of the bouncing ball
We shall model this phenomenon using several approaches to provide an illustrative example, and also to show that these approaches differ widely in their execution and that the choice of an approach has a deciding impact on the performance of the simulation. 6.2.4.1. Continuous approach Let us consider the two continuous state variables z (altitude) and v (velocity) of the system constituted by the ball in the Earth’s gravitational field, with the statistical variable n (number of times the ball has bounced since the point t = 0). The evolution of the system is governed by the following two differential equations: dz / dt = v and dv / dt = g . There are two state events: – BOUNCE: on the condition that z = 0 and v < 0. The corresponding action consists of increasing the bounce counter (n ← n + 1) and modeling the rebound by inverting the trajectory (v ← −kv).
Simulation Engines and Simulation Frameworks
259
– STOP: on condition that that z = 0 and 0 ≤v < ε, where ε is the value from which we consider that the ball bounces very weakly. The action corresponding to this state variable consists of fixing the movement (v ← 0, end integration of movement of the ball, and inhibition of BOUNCE and STOP events). The simulation is initialized using the following actions: – initialization of variables: z ← 0, v ← 0, and n ← 0; – activation of the integration of the variables z and v; – activation of BOUNCE and STOP state events. This approach can be used here as our problem has a simple solution. Nevertheless, a model of this kind has little capacity for evolution; a simple change in the modeling hypothesis could require us to start again from scratch. 6.2.4.2. Discrete approach In the discrete approach, we will use a chronological model, with successive bounces represented as events. We will consider the discrete state variables V and Tn (velocity and the date of the first bounce). We might also use a statistical variable n to better follow successive rebounds. These successive rebounds will be posted as dated BOUNCE events, treated in the following manner: – increase n to indicate the existence of a new bounce: n ← n + 1; – model the physical phenomenon of the bounce: Vn ← kVn−1; – memorize the date of the current bounce: Tn ← t; – if Vn < ε then Vn ← 0 and the ball stops bouncing. If not, a new BOUNCE event is generated in the scheduler at date t = Tn + 2 Vn/g. The variables z(t) and v(t) (current position and velocity) have a merely illustrative value here: v(t ) = − g (t − Tn ) + V n and z (t ) = g (t − Tn ) 2 2 + Vn (t − Tn ) . The initialization of the simulation will involve the following actions: – calculate the date of the first bounce (which is “virtual”) when n = 0: T0 = − 2 z 0 g ; – generate the first BOUNCE event in the scheduler at this date: V0 = − gT0 .
260
Simulation & Modeling of Systems of Systems
This solution is fairly well adapted to possible modifications to modeling hypotheses. If the expression of k becomes more complex, there will be no problem in little-by-little calculation: the treatment of the BOUNCE event must simply be modified. In general, discrete event simulations are fairly flexible, but the system must be suitable for chronological modeling. 6.2.4.3. Analytical approach Analytical solutions allow rapid and exact calculations, but this solution does not always exist. Our example covers a fairly simple case, and the state can be directly calculated after bounce n, that is, the residual velocity Vn and the date of the bounce Tn: Vn = k nV0 and Tn = Tn −1 + 2Vn −1 g = (2k n − k − 1)V0 ( k − 1) g . The final state can be calculated using the fact that for the final bounce Vn = ε , giving the index of the final bounce: N = log(ε / V0 ) / log(k ). Of course, we have only been able to find an analytical solution in this case because the system being modeled was trivial. In engineering, the systems encountered are never this simple, and analytical solutions are, at best, complicated, and at worst, impossible to determine; in both cases, simulation can be used to find a solution to the problem.
6.3. Simulation frameworks 6.3.1. Some basic points on software engineering A simulation application is a piece of computer software. As such, it must follow the rules of this domain at a level that conforms to our aims and resources; poorquality software is expensive (to use). High-quality software, such as that used onboard Airbus aircraft, is also very expensive (to design). We must, therefore, find a best alternative that responds to our requirements. Globally speaking, a program will be considered to be “of quality” if it fulfills the needs of the user with respect to cost and time limitations. To obtain this level of quality, the software must fulfill a certain number of criteria, defined in the specifications. Various criteria are used to evaluate the quality of a program and associated metrics. Based on the work of James McCall and Bertrand Meyer [MCC 77, MEY 97], we have chosen to use the following criteria: − validity, − reliability, − scalability,
Simulation Engines and Simulation Frameworks
261
− reusability, − compatibility, − efficiency, − portability, − verifiability, − integrity, − ease of use. Here, we do not intend to go into depth in the field of software engineering, but it is worth discussing these criteria in a little more detail, as they are important elements in the specification of any simulation (and computer applications in general), to observe their impact and level of importance in simulation. 6.3.1.1. Validity Validity is the ability of a program to carry out the tasks defined by its specification with precision. It is, therefore, the basic factor affecting quality. The needs of the client must be fulfilled, but it is important not to spend too much time in developing the program, as this leads to mounting costs: an excess of quality could be considered to be a lack of quality. This factor is truly fundamental in simulation, where it plays a particularly crucial role (and where, of course, its importance is often underestimated). Remember that in simulation, the reuse of a non-validated valid application has no meaning, and may be worse than doing nothing at all: basing a decision on erroneous results can have serious financial and other consequences. 6.3.1.2. Reliability Reliability is the ability of a program to operate even in abnormal conditions. The program should be able to diagnose an abnormal situation and, if possible, re-establish normality or continue to operate in a damaged state, while raising awareness of the problem with the operator through clear error messages that enable the source of the problem to be discovered with ease, whether it came from input data, the execution environment, or the program itself. If the program is unable to continue, it must terminate promptly while providing a diagnostic of the situation. It should do this “cleanly”, that is, without loss of data and without disturbing the operation of the rest of the system.
262
Simulation & Modeling of Systems of Systems
In simulation, a program crashing results in loss of time. A simulation is rarely critical; in certain circumstances, such as collective training or a simulation stimulating an experiment (Chapter 8), problems with the software may be irritating for users, but they do not put lives or goods at risk. Reliability is not, then, an essential criteria in simulation, although a minimum level of reliability is still necessary for the application to be useable. An illustration of what “minimum service” might be is the capacity of an application to diagnose errors, making adjustments considerably easier and rendering dysfunctions traceable. Take, for example, the Windows operating system: the way Windows deals with application or system errors is not a model of how this should be done (it is, however, far from being the worst!). A crash may produce something like Figure 6.2.
Figure 6.2. A rather vague error report
The data given in Figure 6.2 provides us with little assistance in finding out what provoked the fatal error: the name of the program is not even given and the display of the contents of the registers is not particularly helpful. Note that the case given in Figure 6.2 is somewhat extreme: normally, with Windows, it is possible to switch to debug mode in Visual Studio, which allows us to see where the problem occurred in the object code, that is, in machine-language. As the user does not have access to the source code, it will be extremely difficult to link the breakdown to a specific operation in an identified module. The listing given in Figure 6.3, on the other hand, is an example of a clear diagnosis. It shows an error that occurred in a module written in Ada in the ESCADRE framework. We see that an “exception” (abnormal operating condition) occurred when opening the file ar_missile_tno.don, for which the full file address is given.
Simulation Engines and Simulation Frameworks
263
The problem was linked to the name of the file (Name_Error), line 118 of the low_level_io_.verify model, called up at line 83 of the io-open_text.adb module. Io.Open_Text.Open: File managment error File_Name => INPUT_DIR:ar_missile_tno.don Translated => D:\apps50c\src\air\data\ar_missile_tno.don Exception => Ada.IO_Exceptions.Name_Error ---- Exception_Information ---Ada.IO_Exceptions.Name_Error () Exception traceback: 118 system.rts.low_level_io_.verify low_le$1.bdy 190 system.rts.low_level_io_.open_create low_le$1.bdy 278 system.rts.low_level_io.open low_le$1.bdy 79 system.rts.tgt.char_io.open tgt_ch$1.bdy 283 ada.text_io.open text_io.bdy 83 io.open_text.open io-open_text.adb Exception handled at: 85 io.open_text.open io-open_text.adb
Figure 6.3. A more explicit error report
We, therefore, have immediate access to the error type, the data, and the function involved and even the line number for the source of the problem, as well as the whole list of sub-program calls. We are thus able to put our finger on the precise cause of the problem. Even if the user does not have access to the source code, this diagnostic can be used to dialogue with the support of the simulation provider. This level of luxury, however, comes at a price: ESCADRE is full of code used to deal with errors. This can prove extremely heavy in the long term and have a negative effect on the code’s readability, but these disadvantages are compensated by the quality of diagnosis and the consequent ease of adjustment. 6.3.1.3. Scalability Scalability is the ease with which a program can adapt to a modification or extension of its specifications. A program that is not designed to evolve easily can prove costly in the long run, as around 30% of the global cost of a program is estimated to be due to changes in specifications (and this is only an average figure). In simulation, the ability to evolve is extremely useful. When reusing a simulation, it should be possible to take additional entities into account with ease, or to modify models. Specifically, during the construction of a federation using the high-level architecture (HLA) standard, the negotiation of the federation object model (FOM) may require modification of some or all federated parts. We can, however, use “FOM-agile” parts, which can adapt to different FOMs without modifying code by adjusting the parameters.
264
Simulation & Modeling of Systems of Systems
6.3.1.4. Reusability Reusability is the ability of a program to be reused, in part or in its entirety, for new applications. The principle boils down to common sense: we do not need to reinvent what we already have. As Eric S. Raymond, one of the masterminds of open-source software, reminds us (see [RAY 99]): “Good programmers know what to write. Great ones know what to rewrite (and reuse)”. In a certain sense, then, a good programmer is a lazy programmer! In fact, this “laziness” is smart, allowing us to minimize costs and delays while improving the quality of the product, as we are free to concentrate on the quality of the new code (the old part having already been tested and validated). Reusability is a very important quality and is fundamental in object-oriented programming. It is also important in the field of simulation, as we shall see, and is the main purpose of simulation frameworks and of the HLA standard. Unfortunately, reusability is often far from being a top priority of developers, who find themselves under pressure to deliver their applications rapidly and as cheaply as possible. Jean-Pierre Rosen sums up the situation in [ROS 95]: “Reuse is like the Loch Ness Monster of software engineering: everyone talks about it, but very 1 few have used it in practice….” . It is therefore essential to systematically plan for potential reuse in projects, starting with the implementation of a capitalization and reuse policy at company level and setting out these requirements clearly in the specifications of the simulation application. 6.3.1.5. Compatibility Compatibility is the suitability of a program for combination with other programs. Compatibility is a quality which allows different programs to cohabit or collaborate, for example via a programming interface (application programming interface, API) or a shared communication protocol. Compatibility is one of the major objectives of standards such as CORBA, distributed interactive simulation (DIS), HLA, and so on. This quality is similar to interoperability, which is the capacity of a program to provide and accept services from other simulations, and to use these exchanged services for effective cooperation. The importance of standards such as DIS or HLA in simulation shows that there are considerable potential benefits behind this quality criterion, in particular the pooling of capacities of several simulations to create a meta-simulation (i.e. a federation of simulations). This is absolutely essential for a certain number of 1 Original citation: La réutilisation est un peu le monstre du Loch Ness du génie logiciel: tout le monde en parle, mais bien peu la mettent en pratique.
Simulation Engines and Simulation Frameworks
265
applications, such as collective training via distributed simulation, and also in the creation of complex simulations or the use of external software, such as a generic three-dimensional (3D) visualization program. 6.3.1.6. Efficiency Efficiency is the ability of a program to make good use of resources. A program must be sufficiently efficient to fulfill its specifications. This criterion should not, however, be overestimated: programmers often waste time in optimizing their program when the said program is perfectly operational as it is, and the gains from optimization barely noticeable. Nevertheless, efficiency may be crucial in certain programs. Examples of programs where this is a major consideration are large constructive simulation programs (computer usage time can be very costly) or real-time systems (image generators, hybrid simulators, flight simulators, and so on). Specialized programs, known as profilers, can be used to calculate the time taken for execution of each instruction in a program and allow us to pinpoint exactly where time is being lost; this is particularly useful as if we believe a common saying among computer scientists, “time is never lost where we expect it”. To add some real-world experience to the mix, the author of this chapter has, in the course of his career, seen different implantations of the same algorithm or model produce performances differing by a factor of 1 to 100. Certain choices of algorithms or architectures may have a deciding effect; sometimes, the improvement is due to something very small. One example is a federated HLA component which, while waiting for a data update from the federation, executed a waiting loop instead of using the tick() service which hands over control to the HLA run-time infrastructure (RTI). This resulted in excessive use of the processing power of the host computer (almost 100%); use of the HLA ad hoc service would have reduced this processor use to almost 0%, allowing other programs on the host computer to operate better. Another example is that of a distributed simulation in the 1980s (before DIS and HLA), which was spread across a dozen workstations in an attempt to boost its performance. The problem was not only that at a given moment, t, just one machine was occupied in executing models (which followed on from each other more or less sequentially, so distribution contributed very little except to the time lost in synchronization operations) but also that all services were provided by a framework centralized in one machine, which was, of course, overloaded, whereas others spent their time waiting for a response from the framework while dealing with very little workload. In practice, the parallel and distributed architecture was sequential and centralized, hence the mediocre performances of the simulation despite access to considerable technical resources. A better coupling of models and
266
Simulation & Modeling of Systems of Systems
a local distribution of certain framework services across workstations would probably have produced considerable improvements in terms of efficiency. 6.3.1.7. Portability Portability is the capacity of a program for adaptation to different environments. The importance of this criterion depends, a good deal, on the specification and on the intended lifespan of the program. In informatics, environments (material platforms, operating systems, interfaces, and so on) change very rapidly. If a program is designed to last for more than a few years, the question of portability must be considered. Portability can impose certain constraints on the developer (use of normalized languages and standard interfaces, organization of the program in independent layers, coding discipline, and so on), but if considered from the beginning, its cost can be very reasonable. The addition of portability later on in the process, however, can prove costly. Once again, we can provide a good real-life example of the problems caused by negligence in this domain. A very complex program was developed by a group of major industrial contractors for a government department. The investment was of the order of tens of millions of euros and 10 years of study and development, producing almost one million lines of code. This behemoth operated only on Sun workstations, with a Unix operating system, SunOS. Shortly before the launch of the program in question, the Sun Company decided to replace SunOS with a new, more modern and higher performance system, Solaris. Sun rapidly terminated maintenance operations on SunOS and ceased to ensure its new machines would be compatible with the old operating system. Unluckily, the program in question was not designed to be portable, and its conversion for Solaris involved rewriting part of the code, at a cost of several million euros. Add to this a plethora of bugs and mediocre conformity with the initial specifications, and the program was, quite simply, abandoned. The program did have certain qualities, but the failure to plan for the future proved fatal. Twenty years ago, SunOS was a popular operating system, and, apparently, its sudden disappearance was not considered probable at all. Unfortunately, nothing is certain when dealing with evolutions in the world of informatics. Another, more global evolution provides further examples of problems linked to the creation of non-portable programs: the passage from 16 bit (Intel 8086 architecture) to 32 bit (Intel 80386) processors in personal computers (PCs). In the repositories of C, a very popular language at the time, integers (type int) passed from 16 to 32 bits, leading to numerous breakdowns, for example in pseudo-random number generators. The code needed either to be executed in 16-bit compatibility mode, which did not allow optimal exploitation of the performances of the new platforms,
Simulation Engines and Simulation Frameworks
267
or to be modified, a potentially long and costly project as the bugs introduced by changes in the size of words being handled by a system could be quite subtle. The ESCADRE framework, on the other hand, was developed in 1987 in Ada83 for a VMS platform, and its first interface was a proprietary graphical console (Tektronix). Designed with scalability and portability in mind from the outset, it was still working in 2003 on several Unix operating systems and Windows NT, using the Ada95 language. Portability is not, then, impossible: in the world of open source programming, we can find good examples of programs that operate on a significant number of platforms. One example is the GIMP image-editing program that is available for a wide variety of systems, specifically Unix, and also Windows, in spite of possessing a sophisticated graphical interface. The importance of reusability for a simulation varies depending on policy and reuse needs and the planned lifespan of the code, but in general basing the simulation on a perennial and portable framework constitutes good insurance for the future. 6.3.1.8. Verifiability Verifiability is the ease of preparation of receipt and certification procedures. In this case, the greatest possible effort should be made to facilitate tests and debugging of the program, by the integration of adjustment methods, the creation of sets of tests, and the capacity to use these tests (an application which can only be executed using commands entered via a graphical interface complicates this process), possibly reinforced by calculation of the level of coverage of these tests. The instrumentation of the code is often neglected, but is useful in testing and in the adjustment of the program. Instructions can be introduced into the code to print actions carried out by the code to a file or terminal (this output is known as a trace). These instructions are only activated in debugging, or trace, mode. Once again, the verifiability requirements of a simulation program depend on specific needs, but we must remember that verification and validation are linked; and in cases of abnormal execution of the simulation, it is highly desirable to be able to reply the execution in a more “wordy” mode to diagnose, for example, whether the faulty results come from a validity problem in the model or from a software bug. 6.3.1.9. Integrity Integrity is the ability of software to protect its different components (programs, data, documents, and so on) against external access or modification. The basic principle involves protecting the software not only from viruses and malicious actions but also against programming errors. This criterion is partially
268
Simulation & Modeling of Systems of Systems
dealt with the operating system (control of access rights to data, management of access to a resource, and so on) and/or middleware, such as HLA’s RTI (Chapter 7). The importance of this factor is generally reduced in simulation, except in distributed simulation where integrity and security issues may arise. 6.3.1.10. Ease of use Ease of use is a criterion that refers to the ease with which users may learn to use software, operate it, prepare data, interpret results, and diagnose and repair the effects of possible errors. In short, designers and developers must think of the user. Graphical interfaces have improved the situation as a whole, but they are not a cure-all: confusing menus, the absence of clear documentation, and the sending of error messages in digital code are all factors which make the user’s task more difficult. In simulation, ease of use for the final user is very important, and often essential. Non-user friendly software runs the risk of rejection by the final user and a natural reticence to reuse it later; it may also be a source of errors in and of itself, as due to the complexity of the interface the user may struggle to gauge the consequences of their actions. In 2007–2008, a previously little-known program enjoyed enormous success in the world of simulation. This was a simulation environment developed in the video games industry, but adapted to “serious” applications (such as military). This “serious game” is not itself a video game, but a professional application. Nevertheless, it contains certain characteristics of video games, including a productivity and ease of use which are considerably superior to most of the “traditional” competition, winning the approval of users. This demonstrates the importance of making an effort in the ease of use aspect of software, particularly if the program in question is complex, as is usually the case with simulation applications.
6.3.2. Frameworks A simulation or model framework is an assembly of methods, software and, sometimes, hardware used to carry out a simulation by integrating existing models or simulations. We also encounter the term “simulation support environment”, which presents subtle differences in that it usually refers to a system with integrated development tools. A framework typically provides the simulation developer with basic services for simulation: a simulation engine, time management, management of simulation objects and their interactions, access to the operation system, a graphical interface, mathematical and statistical functions, data logging, and so on.
Simulation Engines and Simulation Frameworks
269
A framework generally includes a design and use methodology, as the use of a framework must be integrated into a global process covering the whole life cycle of the simulation. This methodology facilitates the development of applications and renders them more uniform, promoting the reuse of validated models and the portability of applications. The use of frameworks can considerably reduce the costs and delays involved in development and improve the quality of simulation software. In general, between 30% and 60% of the code in a simulation is for what may be regarded as basic services (see the definition given above), which are common to all simulations. By centralizing these services in a framework, we not only avoid having to develop the corresponding code each time but also improve the reliability of this code, as the framework is used and tested again and again. We also increase functionality, as we have immediate access to a sophisticated collection of services. The simulation developer is then free to concentrate on his “own” job of modeling systems (radars, missiles, and so on) and not on strictly software-based problems. This centralization also gives models a certain independence from the software and hardware platforms used, giving them a higher level of portability. Figure 6.4 shows a typical architecture in which models are separated from the user interface and from the operating system, and instead interface with the framework through a programming interface. A change of operating system or user interface will therefore have no effect on the model, although it is desirable to use the fewest different user interfaces possible to avoid having to re-train users with each change. Figure 6.5 shows the typical services provided by a simulation development and use environment.
Figure 6.4. Typical simulation architecture using a framework
270
Simulation & Modeling of Systems of Systems
Figure 6.5. Typical services of a simulation development and use environment
In the medium term, the use of a framework encourages the reuse of models. By being careful to capitalize models as and when they are developed, the user may develop a repository of models for future use which can be used when needed. For example, when modeling the action of a new anti-radar missile, the developer will concentrate on modeling the missile itself, and could take a model of a radar from the repository to use for the target and a model of an airplane to use as the carrier, as long as such models already exist. This process is a source of significant savings in terms of both time and money and also leads to an increase in quality, as the model code had already been validated in a given domain of validity. However, this capitalization process does require a certain amount of effort in terms of documentation and organization.
6.3.3. Obstacles to framework use 6.3.3.1. Organization Frameworks are particularly efficient when they form part of a global highlevel policy, at company or organizational level. The benefits of this approach vary depending on the number of developers using it. To make best use of frameworks, it is necessary to have a product of which the characteristics have been agreed, and for this product to be adopted and shared by all participants with explicit support from managers.
Simulation Engines and Simulation Frameworks
271
The establishment of an infrastructure linked to the framework to train and instruct those concerned, the inclusion of contractual clauses integrating the framework and the development of a repository system to facilitate model sharing are also essential to make the best use of frameworks. 6.3.3.2. Human factors We are not currently concerned with modeling all human factors (Chapter 4), but with those human factors which impact upon the development and use of a simulation. These factors, unfortunately, constitute the principal obstacle to the adoption of a framework by a group of simulation development teams. The reticence of developers can have a number of more or less rational causes: – Feeling of inadequacy for requirements: as the functionalities of the proposed new product are not like those the developer is used to, this leads to suspicions of incompatibilities and omissions. – Resistance to change: the developer must re-analyze his way of working and undergo re-training. – “Not Invented Here” syndrome: a clear tendency in computer science to mistrust anything coming from outside. – Resistance to authoritarianism: computer scientists have a tendency to see themselves as artists or scientists, not as engineers producing industrial products. – Fear of the unknown: or “if it ain’t broke, don’t fix it”. It is hard to abandon methods known to work for a new product, even if this is in the common interest; this phenomenon was observed during the passage from DIS to HLA. – Narrow and/or short-term vision: the adoption of a framework requires considerable efforts, but its benefits are not fully evident until several years have passed, particularly during periods of evolution. 6.3.3.3. Diversity of needs The field of simulation includes a number of domains with needs that may differ widely: for example, the services and models used for technico-operational simulations are not necessarily the same as those needed for a piloted flight simulator. Any framework responding to all these needs runs the risk of being extremely complex and difficult to master, both undesirable attributes. On the other hand, we could define a limited number of homogeneous domains for the level of granularity of models and performance demands, and find an adequate solution for each of these domains.
272
Simulation & Modeling of Systems of Systems
6.3.3.4. Legacy models In financial terms, it is usually unthinkable to ignore models and applications developed before the establishment of a framework (legacy models). A good framework should therefore be able to integrate models not designed for that framework. 6.3.3.5. Resistance to change During the design of a framework, it is necessary to make a certain number of technical choices: platform, language, tools, and so on. We must keep in mind that, in computer science, technical solutions do not last forever. Thus, the DEC VMS operating system, which was widely used in the American simulation community until the early 1990s, has practically disappeared, rendering a number of products obsolete. A framework must, therefore, be easily portable from one system to another or from one language to another, and not depend on proprietary products that may not last. It is also necessary to thoroughly detach the design of models from their implementation. The framework, if it is well designed, ensures this separation and thus makes investments durable.
6.3.4. Detailed example: ESCADRE To provide a clear understanding of frameworks, we shall study one framework in more detail: Ada Simulation development for analysis studies development and reuse (Environnement de Simulation et de Conception orientée objet en Ada95 pour le Développement et la Réutilisation des Etudes, ESCADRE) – an object-oriented simulation and design environment in Ada95 for the development and reuse of studies. ESCADRE is a framework designed to provide support throughout the whole lifecycle of a simulation (design, coding, and execution). It is available for free, on request, with its source code, under a GPL license (GNU Public License). It operates on a number of platforms, including Windows NT/2000/XP, Solaris, Linux, AIX, IRIX, DEC/VMS (up to version 4.6), and so on. It is currently used for dozens of simulations, within the French Ministry of Defense and elsewhere, in universities and for industrial applications. ESCADRE has been used in Germany, Singapore, the UK, the USA, and France. An ESCADRE users’ club exists to facilitate interaction between the developers and users of the product. This section does not aim to provide an exhaustive description of ESCADRE (see [SAU 02]), nor does it claim that ESCADRE is an ideal framework; our intention is to provide an overview of the modeling concepts and processes it recommends, as it is a good example of its kind.
Simulation Engines and Simulation Frameworks
273
6.3.4.1. Historical overview ESCADRE appeared in 1987 in the French DGA’s Centre d’Analyse de Défense and was designed for the purpose of developing and using simulations for technical, technico-operational, and operational studies, and for non-real-time constructive computerized simulations more generally. The awareness of a need for ESCADRE stemmed essentially from the observation, made when studying the CAD’s simulation codes, that 30–60% of the code in a study simulation was given over to provide basic services (graphical interface, statistics, and so on) and that more than half of the remaining code was used to model the natural and tactical environment of the system being studied. These observations led to the idea of centralizing these services, used by all simulations, in a framework structure, while establishing a methodology and an organizational procedure to enable the capitalization of models developed and their reuse from one simulation to another. The particularities of the CAD’s technico-operational simulations were as follows: – the possibility of imprecise and unfixed specifications; – the need for rapid development (results sometimes required in a matter of months). The following constraints were also taken into consideration: – Application developers should be able to concentrate on their area of expertise (modeling weapons systems or sub-systems) and not on software engineering tasks (e.g. the development of a graphical interface, choosing the position and color of a button, and so on). – Tools should be separated from models not only to allow better reuse of the code developed but also to reduce security problems (system models are classified, but the tools used are generic). – Tools and interfaces should be standardized to minimize efforts involved in learning to operate simulations. – Developments should be capitalized and made to last (possibilities of reuse over periods of 10 years or more). After examining different options (those available in 1987), the decision was taken to construct a framework in Ada83 with an object-oriented methodology derived from Grady Booch’s method [BOO 92] (implemented, nevertheless, using a non-object-oriented language, until version 5.0 which made use of the full object capacities of Ada95).
274
Simulation & Modeling of Systems of Systems
ESCADRE has undergone considerable evolutions since its first incarnation to keep up with the changing needs of users: – V1 (1987): evaluation of anti-radar missiles (DEC/VMS); first application (simulation of an aerial attack mission against enemy defenses); – V2 (1988): new time management, data processing, and statistical analysis capacities. 5 ESCADRE applications now in operation; – V3 (1991): new platforms (Unix), inline graphic display (XWindow), and capacity to replay an execution; – V4 (1994): various improvements, passage to multiple Unix platforms and DEC/VMS, graphical user interface based on OSF/Motif; several dozen ESCADREbased applications now in operation; – V4.3 (1996): multi-player capacities (with the possibility of creating war games), virtual joystick for piloting mobile actors; – V4.5 (1997): switch to Ada95 without modification of the application interface; – V4.6 (1999): switch to Windows NT platform, reorganization of low-level code, industrialization of product; – V5.0 (1999–2000): switch to object-oriented programming; new development interface and method revision to support new specialization and sub-component mechanisms; – V5.1 (2000): HLA compatibility; – V5.2 (2004): redevelopment of the user interface with the Gtk graphics repository (freeware), more portable and more modern than Motif. 6.3.4.2. Architecture ESCADRE is based on a multi-layer architecture (Figure 6.6). A basic services layer (e.g. dealing with access to functions of the operating system) and a highlevel graphics layer make applications highly portable: an ESCADRE simulation developed under Unix may be used in Windows or Linux by simply recompiling the applicative code. If, in a few years, the need is felt for ESCADRE to support the MacOS platform, for example, all ESCADRE applications will become compatible with this operating system (by recompiling). ESCADRE includes a set of services, accessible via an Ada95 programming interface. These services are grouped into four categories: – Basic services: system access, high-level input–output, assistance with modification, text and data handling, mathematical functions, and so on.
Simulation Engines and Simulation Frameworks
275
Figure 6.6. ESCADRE architecture
– Graphics services: dialogue boxes, map display, animation, curve tracing, data visualization, virtual joystick, and so on. These services are, in principle, contained within the upper level of the simulation and are called up automatically by ESCADRE’s execution infrastructure, although expert users may use them directly. 2
– Interoperability services: based on HLA/RTI distributed by the DMSO and usable through an Ada95 binding. Note that the Ada95 interface of the HLA standard was designed by the ESCADRE team. – Simulation services: simulation engine (time management, actors, interactions, and so on), visualization of actors, data repository, HLA interoperability, and so on. ESCADRE automatically provides each application with a graphical and textual interface allowing control of simulation execution. The simulation developer does not, therefore, need to think about the color or placement of a button on the user interface, for example, as everything is generated automatically without writing a single line of code: – supervision of replica execution (step by step, until a given date, automatically); – introspection of the application (hierarchy of actor composition and specialization, blind interactions, HLA interoperability, and so on); – interactive command of simulation objects (creation, destruction, alteration of behavior, and so on). 2 The Defense Modeling and Simulation Office (DMSO) was, until recently, the office in charge of simulation within the US Department of Defense.
276
Simulation & Modeling of Systems of Systems
ESCADRE is also supplied with tools for the preparation and use of executions of a simulation: – map and technical operational data editing, – statistical analysis of results (Figure 6.7), – trace analysis, – data format conversions, – graphical replay of an execution (or replica), and so on.
Figure 6.7. Statistical analysis tool
6.3.4.3. Basic concepts An actor, in ESCADRE, is a class of simulation objects. It may be defined as follows: – By specialization of a pre-defined abstract actor (the only option offered by the old method of ESCADRE 4.x and before). The actors Battery, Radar, and Patrol in the Air example given below (attack by bomber aircraft, with missiles and enemy land-to-air defenses) fall into this category. – By specialization of an abstract actor (not saved in ESCADRE) defined previously at application family level. This is the case for the actors SA_Missile,
Simulation Engines and Simulation Frameworks
277
Aircraft, and Bomb, specializations of the abstract actor Movable (with abstract functions covering position and speed). – By specialization of a pre-defined concrete actor (saved in ESCADRE). The actor Bomber, a specialization of the concrete actor Aircraft from which it inherits its roles in relation to properties, is an example of this. The ESCADRE repository supplies the following: – An abstract root actor with its accessors (name_of, team_of, status_of, signal_of, parent_of, and children_of) and primitives (write_data, create, start, suspend, resume, stop, delete, detach, and reparent). – A property pattern (blind interaction), which can be created for a pair of type question/response. This pattern defines the abstract property with its service primitive, receiving the question and returning the response. Sub-components are aggregations (by value) of elements for the benefit of a containing actor. Their technical and state data are generated by the container actor. They may define events and dynamics by identifying themselves by the container and collaborate in the operation of the container. They may be specific to the actor using them or generic (reusable). Technologies can define variants (of reliability or behavior) for a given specification of an actor, which can be distinguished by – the technical data (cruise speed and maximum acceleration); – non-extendible codes, for example, a missile which has three or six degrees of liberty (depending on reliability, a real missile having, on principle, six degrees of liberty), a mechanical or electric sweeping radar (depending on behavior); – extensible codes (private specialization), such as a generic or specific weapon, a ballistic missile, whirling, maneuvering (behavior). In concrete terms, technologies are treated as static variants of an actor, which may be assimilated to a second level of internal specialization. Thus, we may find an actor of the type Bomber that may take the form of several technologies (Tornado, F16, Mirage 2000), which indicate the characteristics of each model of bomber (load capacity, cruise speed, acceleration, resistance to shrapnel, and so on). The parameters of different technologies are contained in text files, which are shown in Figure 6.8.
278
Simulation & Modeling of Systems of Systems
Figure 6.8. Extract from the technology file for the Bomber actor class
The principle of filiation between actors allows a parent actor to make a child actor collaborate in its mission. These interactions operate in both directions following a protocol established by the child actor (cognitive interactions). This guarantees the reuse of all sub-hierarchies of filiation. The child actor retains a certain level of autonomy and may change parents or become autonomous. Its lifespan is not necessarily limited by that of its parent. A “property” defines an exchange protocol (input and output data) between a client actor initiating the interaction and a serving actor which agrees to be subjected to the interaction. Properties allow interaction between actors without imposing visibility of the actor on the server (hence the term “blind interaction”). In this way, property ensures separation (and therefore substitution) of actors, which interact in this way (often used to model physical interactions in the real world). 6.3.4.4. Practical example: the Air application ESCADRE is supplied with Air (simulation of air–ground and ground–air attacks), Land (terrestrial tank and helicopter combats), and Sea (sub-marine patrols) examples. The Air example simulates a Suppression of Enemy Air Defenses (SEAD) mission: an aircraft patrol attacks radars, which can defend themselves using batteries of land-to-air missiles (Figure 6.9). The scenario may be described as follows: “each attack patrol coordinates a group of airplanes following a route. Each airplane carries anti-radar missiles and, on arriving in proximity to the objective, listens to radar emissions to localize its target before releasing an anti-radar missile. Each missile travels toward its target, guided by its passive homing head, in order to destroy the target. Each land-to-air defense system is made up of a radar which detects enemy targets, tracks them, and may cease emissions, and a control center which collects and classifies these tracks, then launches ground-to-air missiles in order to destroy targets”.
Simulation Engines and Simulation Frameworks
279
Figure 6.9. Air application scenario
The analysis of this textual description may be refined by picking out interactions and actors: “each attack patrol coordinates a group of airplanes following a route. Each airplane carries anti-radar missiles and, on arriving in the vicinity of the objective, listens to radar emissions to localize its target before releasing an anti-radar missile. Each missile travels toward its target, guided by its passive homing head, in order to destroy the target. Each land-to-air defense system is made up of a radar which detects enemy targets, tracks them, and may cease emissions, and a control center which collects and classifies these tracks, then launches ground-to-air missiles in order to destroy targets”. We can now easily identify the actors of the simulation and their relationships (Figure 6.10), and the interactions between these actors (Figure 6.11). These diagrams respect the UML formalism, chosen by the ESCADRE methodology from version 5.0 onwards.
Figure 6.10. Diagram of actors
280
Simulation & Modeling of Systems of Systems
Figure 6.11. Actor interaction diagram
In addition to actors and interactions, we must define the properties being modeled. In our example, we only need three properties: – Radar reflection: each “airplane” actor is a server for this property, allowing us to know what proportion of the incoming radar wave is reflected by the aircraft. Typically, the client is the emitting radar. Note that missiles are not servers of this property, as in the Air example, the radars do not take missiles in flight into account, and so this property does not need to be applied to missiles. – Radar emission: this property is used by the anti-radar missile to see whether, depending on the channel it listens to and its position, it can detect the radar which is a server of the property. – Vulnerability: this property is used to evaluate the damage caused to an actor by the explosion of a charge (i.e. the warhead of a land-to-air missile). The radar, airplane, land-to-air missile, and air-to-land missile actors are all servers of this property, and therefore susceptible to damage or destruction. Following our analysis, we obtain a complete topology of the Air application (Figure 6.12). However, the analysis of the system does not stop at this point, as each component in the diagram can itself be analyzed and broken down. Thus, we see that a missile contains the components homing head, engine, explosive charge, and so on. Figure 6.13 gives an example of a breakdown of the Missile actor.
Simulation Engines and Simulation Frameworks
281
Figure 6.12. Diagram of actors, interactions, and properties
Figure 6.13. Sub-components of the Missile actor
Each actor, sub-component, property, and so on corresponds to an Ada95 package, and so to a specification and a body. Once this model design phase is finished, the conceptual models must be converted to Ada95 specifications and implemented. As an example, the specification of the package for the Guidance sub-component of the Missile actor is given in Figure 6.14. An extract from the implementation of a missile launch is given in Figure 6.15. Figure 6.16 shows the results obtained during execution of the Air application, and Figure 6.17 shows the control window for simulation execution, with the events journal (ESCADRE uses a mixed event-integration engine with variable integration steps). We immediately notice the richness of the graphical interface and menus, in spite of the fact that we only worked on modeling the system: the entirety of the necessary
282
Simulation & Modeling of Systems of Systems
execution environment is dealt with the ESCADRE framework and its tools. This example clearly demonstrates the usefulness of this kind of product. Keep in mind, however, that the factorization of shared functions and the use of a design methodology are not sufficient on their own; policies and organization are needed to enable effective capitalization of models, a point to which we shall return later. private package Sa_Missile.Guidance is type Rec_Tno is record ... end record; procedure Read_Tno (Title: in String; Tno: out Rec_Tno); procedure Write_Tno (Title: in String; Tno: in Rec_Tno); type Rec_Data is private; type Ref_Data is access all Rec_Data; procedure Write_Data (Title: in String; Data: in Rec_Data); end Sa_Missile.Guidance; with Root.Movable.Def; generic type Owner_Data is new Root.Movable.Def.Rec_Data with private; with function Tno_Of(Owner: Owner_Data'Class) return Rec_Tno; with function Data_Of(Owner: Owner_Data'Class) return Ref_Data; with function Target_Pos_Of(Owner: Owner_Data'Class) return Position_3d; with function Target_Vel_Of(Owner: Owner_Data'Class) return Velocity_3d; package Sa_Missile.Guidance.Api is procedure Initialize (Owner : in out Owner_Data'Class; Target_Pos: in Position_3d; Pos : in Position_3d; Vel : out Velocity_3d); function Acc_Commanded(Owner: Owner_Data'Class) return Acceleration_3d; end Sa_Missile.Guidance.Api;
Figure 6.14. Package for the Guidance sub-component of the Missile actor
procedure Process_Launch (Data: in out Rec_Data'Class; Target: in Object; Cmpt: in Component_Id) is Tno : constant Aa.Ref_Tno := Aa.Tno_Of(Data); Dirvel: Velocity_3d; begin -- sub-components for Cmpt in 1..Tno.Nbrock loop Rocket.Initialize (Data, Cmpt); end loop; Seeker.Initialize (Data, Target); Guidance.Initialize(Data, Seeker.Target_Pos_Of(Data), Data.Illum_Pos, Dirvel); Airframe.Initialize(Data, Data.Illum_Pos, Dirvel); ... end Process_Launch; ... end Sa_Missile.Def;
Figure 6.15. Implementation of a missile launch (extract)
Simulation Engines and Simulation Frameworks
Figure 6.16. Air combat application based on ESCADRE v4.6
Figure 6.17. Air combat application based on ESCADRE v4.6
283
284
Simulation & Modeling of Systems of Systems
These examples and illustrations are based on the ESCADRE Designers’ Manual; see [SAU 02]. 6.3.4.5. After ESCADRE: DirectSim ESCADRE fulfilled its role for 20 years, evolving to remain up-to-date. However, the technology involved was reaching certain limits when faced with new strategies in software engineering, particularly virtual machine languages and portable pseudocode (Java and C#), frameworks (.NET and J2EE), model-driven engineering (MDE) and domain-specific languages (DSL). From the point of view of the developers and sponsors of ESCADRE, these new technologies showed considerable potential in terms of productivity, services, and ease of development with the idea of reducing the amount of code development required to obtain a simulation to the bare minimum. The ultimate aim was the automatic generation of all required code based on a conceptual model. This goal has been attained by certain environments (Matlab/Simulink being the best-known example), but only in targeted domains and with specific types of models. Furthermore, ESCADRE had not only many positive qualities but also a number of flaws that caused serious problems for certain users. The use of the Ada language was a particular reason for rejecting the program as, despite the reliability and readability of code, the number of individuals with the necessary skills in Ada is falling rapidly (although the level of programming in Ada required to use ESCADRE is relatively low). Another problem was that a certain amount of manual development work is required to develop an ESCADRE simulation once the models have been built; the program lacked a proper integrated development suite. Several different solutions were considered, using criteria of technical quality, performance, availability of support tools, portability, durability, and so on. The two principal options were pairings of Java/J2EE and C#/.NET, the operational richness of which considerably reduced the need to develop specific services in DirectSim. The latter solution (C#/.NET) was chosen and prototypes were built to evaluate the solution using real cases and to finalize the concepts required by the new standard environment for technico-operational simulation required by the French Ministry of Defense: DirectSim. DirectSim is made up of four main components: − a framework; − a development environment;
Simulation Engines and Simulation Frameworks
285
− a repository of models and components; − monitors. As in ESCADRE, the framework comprises a set of services including the simulation engine. This is accompanied by a methodology which, in conjunction with the simulation architecture imposed by the framework, allows the rapid construction of reusable simulations. This framework is also known as a simulation execution environment. The simulation development environment is based on Microsoft Visual Studio, with the addition of specific tools such as DSL SimA which allows definition of the static architecture of models and the generation of a code skeleton, and DSL SimB for designing the behavior of simulation objects. This development environment also enables retro-engineering of models, making it easier to reuse and migrate existing models (Figure 6.18).
Figure 6.18. DirectSimDSL SimA (simulation architecture)
The component repository gives access to a set of generic models which may be used as they are or modified to fit requirements, making development easier. Monitors allow the user to pilot the execution of a simulation. Two monitors are supplied with DirectSim: one is textual and suitable for the batch mode of operation (non-interactive and automatized, typically used when executing several replicas
286
Simulation & Modeling of Systems of Systems
as part of a Monte-Carlo analysis), the other is graphical and facilitates scenario editing and adjustments, and can be used to develop interactive simulations (such as training tools). Figure 6.19 shows this graphical interface. At the top, we see the commands for controlling the execution; in the left, the object class navigator. In the top right of the screen is a graphical visualization of the progress of the simulation, and centerright is the evolution of statistical variables. At the bottom of the screen, we find messages produced by the simulation engine or by models.
Figure 6.19. DirectSim’s graphical monitor
DirectSim is very open and scalable. Users may, of course, develop not only their own components but also add DSLs to the simulation development environment or add a new execution monitor. DirectSim V1 has been in operation since January 2008. It is available for free and in open source within the defense community, and has already been used in a number of projects within the French Ministry of Defense and in industry. Various extensions have already been developed, including an interface with Google Earth for cartographical visualization, interoperability with other simulations via the information technology central services (ITCS, see following section), or the use of a third-party graphical visualization tool (in this case, Agenium’s STK; work is underway on an interface with Bohemia Interactive’s VBS2).
Simulation Engines and Simulation Frameworks
287
6.3.4.6. Beyond the framework The ESCADRE and DirectSim approaches have allowed the creation of a rationalization method for developments in a particular domain of simulation, than that of technico-operational simulations, the bulk of which are carried out within the same center. The DGA’s aim, however, was to rationalize their technical and technico-operational simulations within a number of technical centers spread across France. Each center has its own simulation tools, including frameworks, each dedicated to a particular domain (such as radar modeling) or architecture (e.g. hybrid real-time simulation). A greater degree of coherence was required in simulation operations from both a technical and an organizational point of view. The creation of “systems and simulation engineering” positions within the DGA allowed simulation policies and competences to be mastered at national level. Internal and external simulation coordination authorities were created to promote the sharing of information, needs, and orientations. The HLA standard was adopted for simulation interoperability and network structures set up to allow data sharing and the execution of distributed simulation federations. Finally, the ITCS project was launched to ensure technical coherence within the group. The ITCS (infrastructure technique commune de la simulation) is the DGA’s common technical infrastructure for simulation. It is, in broad outline, an information system for the engineering and execution of distributed simulations (Figure 6.20).
Figure 6.20. Functional architecture of ITCS
288
Simulation & Modeling of Systems of Systems
The ITCS supplies an integrated set of tools, which cover the whole process of creating a distributed simulation (from specification to use and capitalization of results) and speed up the process considerably: simulation architecture design tools, visualization tools, interface code generators, and so on. This environment takes the form of a collaborative simulation portal, accessible over the organization’s intranet, which includes a catalog of the DGA’s simulation resources and a description of processes in different simulation domains. The main contributions of the ITCS are as follows: − an integrated, open, and evolving infrastructure including various services and tools for the development of distributed simulations; − provision of a unified, shared methodology, with a MDE approach for simulations (Figure 6.21); − capitalization of models and simulations integrated into the simulation design process; − interoperability infrastructure for simulations (SI2), allowing freedom from different versions of the HLA standard: a simulation using services provided by SI2 will be interoperable with not only an ITCS simulation but also HLA. As and when new standards appear, the SI2 is modified to take any changes into account, reducing or even eliminating the impact of standard changes on existing simulations. This removes one of the major obstacles to the use of distributed simulation. The ITCS includes various tools allowing a simulation engineering process to be drawn up based on the applicable standards (FEDEP, SEDEP, and so on), with an important capitalization (knowledge management) dimension. These tools can be accessed by different actors in the process (Figure 6.22) using a rich or webbased client. On top of solving the problem of interoperability standards (using SI2), the services provided by the ITCS also allow the integration of simulations based on different frameworks. It is still too early for us to have access to feedback on the ITCS, as version 1.0 was only launched at the end of 2008, but several centers have already oriented their developments around ITCS. Contrary to ESCADRE, therefore, DirectSim will not develop an HLA interoperability service but will be based on ITCS’s SI2 in this respect. This is a good illustration of the benefits of a shared infrastructure in terms of access to extended services at reduced cost. The French Ministry of Defense’s
Simulation Engines and Simulation Frameworks
289
technico-operational laboratory has adopted the ITCS as a technical foundation. Several experiments have already been planned using the services of the ITCS. Finally, in addition to the seven sites originally affected by the ITCS, planning is currently underway for a roll-out across other DGA centers and outside the organization.
Figure 6.21. ITCS studio: requirement importation and editing
Figure 6.22. ITCS: processes and various actors involved
290
Simulation & Modeling of Systems of Systems
An interesting side effect of the ITCS project is that it also promoted emulation and synergy between teams from different centers, effectively playing a federating role within the organization, coordination and development of simulation abilities within the DGA. In the 1990s, the appearance of the DIS and the HLA distributed simulation standards had already caused a major convergence of interests which led to the creation, among other things, of the ADIS group (Armées, DGA, Industrie sur la Simulation) which, 15 years later, is still active and acts as an exchange forum for a representative group of actors from the government, industrial, and academic spheres.
6.4. Capitalization of models The use of a framework allows users to focus on the development of models, but the approach can (and should) go even further in the rationalization of developments. The development of a model is still often a complex, and sometimes costly, procedure and requires rigorous validation. When using simulation in concept studies, we often find the same models used from one simulation to another, with occasional variations (e.g. a change in radar type). The reuse of models between simulations, subject to reconsideration of validity each time to ensure that the model remains within its domain of validity and that modifications have no effect on validity is, therefore, highly desirable. However, component reuse comes at a cost. First, the component must be found (hence the interest of a code repository) and understood (requiring adequate documentation). Its suitability for a particular need should be checked (hence the importance of verification-validation-accreditation, VV&A) and modifications may be required (so the source code must be available). It must be integrated into the application (a common framework or an interoperability standard such as HLA is required) and integration tests must be carried out. Model reuse is therefore only profitable, effective, and correct if a certain number of precautions are taken: – Each model must be documented appropriately, with information on the architecture, entities and objects used (this may be the object of a SOM HLA – see Chapter 7), the hypotheses used, the domain of validity, and so on. – Access to the repository or library of models must be possible through a single access point (e.g. a search engine) even if they are physically stored in different places. In cases where the model cannot be put online (i.e. where there are security concerns), contact details for someone able to respond to a model request must at least be available. – Models must be available in the form of source code to allow modification or portage by a third person.
Simulation Engines and Simulation Frameworks
291
– If models use different frameworks, it is useful if these frameworks are interoperable (e.g. using HLA). – A strong model capitalization and reuse policy should be in place: high-level management must be involved. A capitalization process, although costly at the outset, allows the creation of a library of components which, in the long term, provide considerable savings in terms of time, and therefore of money: globally speaking, the reuse of a component costs a mere 20% of the initial development cost. With HLA, the US military has placed particular emphasis on this concept, known as component-based simulation. Moreover, a secondary effect of capitalization is that it places greater responsibility on developers; as their code may be read and re-used by their peers, developers tend to improve the quality of their code and the associated documentation. This effect is the reason for the remarkable quality of a number of open-source programs, which provide examples of large-scale component reuse.
6.5. Conclusion and perspectives Clearly, it is almost unthinkable today to design a simulation system from scratch. We live in a period of massive reuse of components and services. The success of frameworks such as .NET or J2EE, and the growth of service-oriented architectures (SOAs) are clear proof of this. This tendency has become more and more important in simulation with evolutions in requirements: these days, the aim is to develop simulations of complex systems and systems of systems rapidly, cheaply, and often with limited technical competences. Without rigorous methodology, a strong reuse policy and a suitable simulation environment, these criteria would be difficult or even impossible to fulfill without resorting to a commercial off-the-shelf (COTS) simulation. A selection of simulation environments is now available which provide pre-defined models, including behavior models, such as STAGE, by Presagis, MäK Technologies’ VR-Forces, VBS2 from Bohemia Interactive, Matlab/Simulink by The Mathworks, Scilab (freeware), and so on. Some of these tools, once mastered, allow extraordinary productivity. Unfortunately, we do not always possess the requisite mastery of the contents and validity of models, a significant hindrance to their use as a reference simulation environment. Nevertheless, their capacities make them precious tools for enriching a simulation (e.g. with 3D visualizations) or illustrating concepts, particularly as the use of techniques from other domains such as video gaming has led to improvements in usability and virtual realism. One possible approach would be to integrate these capacities into a well-understood simulation environment (a direction envisaged for DirectSim), or to take the opposite path and integrate “home-made” models into COTS products, which are generally scalable, but often,
292
Simulation & Modeling of Systems of Systems
alas, use a proprietary development interface, leading to dependence on a product and its editor. The future of COTS is certainly brilliant due to the productivity they allow, but their limitations, their (understandable) inability to fulfill all needs of all users, and their proprietary aspects mean that we are not yet able to use them to create universal simulation environments; products such as DirectSim will still be needed for a number of years.
6.6. Bibliography [ADE 00] ADELANTADO M., BEL G., SIRON P., Cours de modélisation et simulation de systèmes complexes, Société des Amis de Sup’Aero (SAE), Toulouse, 2000. [BAN 98] BANKS J., Handbook of Simulation, Wiley-Interscience, Malden, 1998. [BNC 95] BANKS J., NELSON B., CARSON J., Discret-Event System Simulation, Prentice Hall, Old Tappan, 1995. [BOE 06] BOEHM B., Some Future Trends and Implications for Systems and Software Engineering Processes, Systems Engineering, INCOSE, vol. 9, no. 1, Spring 2006. [BOO 92] BOOCH G., Conception orientée objets et applications, Addison-Wesley France, Paris, 1992. [CAM 06] CAMPBELL S.L., CHANCELIER J.-P., NIKOUKHAH R., Modeling and Simulation in Scilab/Scicos, Springer-Verlag, New York, 2006. [CAN 08] CANTOT P., Introduction aux techniques et technologies de modélisation et simulation, ENSIETA, Brest, 2008. [DAH 99] DAHMANN J., KUHL F., WEATHERLY R., Creating Computer Simulation Systems, Prentice Hall, Old Tappan, 1999. [DEL 97] DELEGATION GENERALE POUR L’ARMEMENT, Guide dictionnaire de la simulation de défense, Paris, April 1997. [DOD 97] DEPARTMENT OF DEFENSE, DoD Modeling and Simulation Glossary, Under Secretary of Defense for Acquisition and Technology, Washington D.C., 1997. [EZR 99] EZRAN M., MORISIO M., TULLY C., Réutilisation logicielle: guide pratique et retours d’expérience, Eyrolles, Paris, 1999. [GIR 98] GIRARDOT D., Guide de développement pour les simulations, Technical Report, Centre d’analyse de défense, Arcueil, 1998. [GIR 05] GIRARDOT D., JACQUART R., ZANON G., BOUC P., Méthode d’évaluation, de validation et d’accréditation des simulations (MEVAS), Délégation générale pour l’armement, Arcueil, 2005.
Simulation Engines and Simulation Frameworks
293
[HIL 93] HILL D., Analyse orientée objets et modélisation par simulation, Addison-Wesley France, Paris, 1993. [INS 98] INSTITUTE FOR SIMULATION AND TRAINING, Standard 610.3: Modeling and Simulation (M&S) Terminology, IEEE, Orlando, 1998. [KAM 02] KAM L., LECINQ X., LUZEAUX D., CANTOT P., “ITCS: The technical M&S infrastructure for supporting the simulation-based acquisition process”, NATO Modeling and Simulation group Conference, Paris, 2002. [KAM 08] KAM L., “Conception dirigée par les modèles et simulations”, in Ingénierie des systèmes de systèmes: méthodes et outils, Hermès, Paris, 2008. [LEC 06] LECINQ X., Le Laboratoire technico-opérationnel, Service d’architecture intersystèmes, DGA, Arcueil, 2006. [LES 08] LESTRADE E., DirectSim: vue d’ensemble, version of 8th 2008, Brochure, Centre d’analyse de défense, Arcueil, 2008. [LUZ 03] LUZEAUX D., “Cost-efficiency of simulation-based acquisition”, Proc. SPIE Aerosense’03, Conference on Simulation, Orlando, USA, April 2003. [MCC 77] MCCALL, J., RICHARDS, WALTERS, G., Factors in Software Quality, TIS AD-A049014, Rome Air Development Center, Rome, 1977. [MEY 97] MEYER B., Object-Oriented Software Construction, Prentice-Hall, Old Tappan, 1997. [MYE 79] MYERS, G., The Art Of Software Testing, John Wiley & Sons, New York, 1979. [RAY 99] RAYMOND E.S., The Cathedral and the Bazaar, O’Reilly, Sebastopol, USA, 1999. [ROS 95] ROSEN J.-P., Méthodes de génie logiciel avec Ada95, InterEditions, Paris, 1995. [SAU 02] SAUTEREAU C., Documentation ESCADRE 5.1, Centre d’Analyse de Défense, Arcueil, 2002. [STA 03] STANDISH GROUP INTERNATIONAL., Chaos, Investigation Report, West Yarmouth, March 2003. [DOD 94] DEPARTMENT OF DEFENSE, Directive 5000.59-M: DoD M&S Glossary, DoD, Washington D.C., 1994. [ZEI 98] ZEIGLER B., BALL G., CHO H., LEE J.S., SARJOUGHIAN H., The DEVS/HLA Distributed Simulation Environment and its Support for Predictive Filtering, University of Arizona, Tucson, 1998. [ZEI 00] ZEIGLER B., PRAEHOFER H., KIM T.G., Theory of Modeling and Simulation, Academic Press, Oxford, 2000.
Chapter 7
Distributed Simulation
7.1. Introduction 7.1.1. The principle The basic idea behind distributed simulation is to construct simulation systems by reusing existing simulation systems, often developed with different objectives. The components of distributed systems can be very different in nature and may include the following: – digital simulation applications; – piloted simulators; – potentially, real systems such as C3I systems or real platforms (e.g. combat aircraft or tanks on real terrain); – various software utilities. To construct such simulation systems requires expertise in network technology and the use of suitable engineering processes (see section 7.3.6) by designers and developers. This chapter aims to cover this concept in greater depth, to describe the technologies and methods used, and to present the norms and standards without which this particular form of simulation could not have been developed.
Chapter written by Jean-Louis IGARZA.
296
Simulation & Modeling of Systems of Systems
Figure 7.1. Distributed simulation for anti-radar combat
Figure 7.1 illustrates the principle of distributed simulation. In this example, the aim was to develop a training system for anti-radar combat techniques, combining the following elements: – a piloted fighter aircraft simulator; – an opposing simulation of an aerial defense network; – an aircraft weapons simulator providing, in particular, detailed models of different missiles launched by the aircraft. Note that we are not “always” interested in reusing the simulators and applications totally, but only some of the models implemented by them, which make up the final system. We could, moreover, develop the system by replacing the piloted simulator by a real airplane with the aim of providing in-flight training. This principle of interconnection between real, constructive, and virtual systems has been widely used in training; in the United States, the famous Red Flag exercises, using dozens of aircraft, now use a mixture of real and simulated resources and are known as Virtual Flag exercises. Beyond the simulation reuse aspect and the savings it engenders in terms of time and financial input (the principle objective of distributed simulation), this type of simulation is in the spirit of our times, as the use of distributed systems in computer science, is nothing new. The main applications distributed simulation aims to assist are initial and continued training. These applications of simulation have provided considerable support in the development of distributed simulation technology.
Distributed Simulation
297
7.1.2. History of distributed simulations Distributed simulation has its origins in projects taking place at the end of the 1980s. The first projects appeared following the spread of local and wider networks, raising possibilities of interconnection of previously independent resources, based on different material and software technologies. The technical arsenal used at the time was that of low-level network standards. The first significant creations were installed other local networks with little effort to generalize the developed technologies. Prototypes of this kind seem to have emerged in various different countries, and at the beginning, there seems to have been no particular desire for exchange between different simulation communities. The event that triggered the development of distributed simulation and its generalization was the development of Simulation Network (SIMNET) in the United States. This project, financed by the Defense Advanced Research Projects Agency (DARPA)1 and the US Army, was a technological demonstration that aimed to show the interest of connecting piloted simulators and simulation programs (virtual and constructive simulations) for the collective training of pilots and weapons operatives. The basic scenario focused on ground and air–land combat. These interconnected simulators were not pre-existing simulators, but low-cost simulators (“trainers”) developed specifically for the project. They were connected over a local network, but the possibility of extending the system to distant sites was raised. SIMNET was widely promoted outside the United States from 1983 and was operational from 1987, then launched across 10 sites including one in Germany (see Figure 7.2). The historical interest of SIMNET is that it was the trigger for the development of interoperability standards for simulations (see section 7.3). SIMNET broke the ground for research on distributed simulations: from 1989, the inter-simulation communication system developed for SIMNET was the subject of standardization attempts with the aim of generalization and therefore of applying it to other projects. The first interoperability standard with these efforts, launched at the end of the 1980s, was also called SIMNET, followed by Distributed Interactive Simulation (DIS) in the early 1990s. These efforts were supported by the establishment of DIS workshops, which, from Fall 1989, brought together hundreds of participants. This organization subsequently evolved to become an international organization of primary importance known as Simulation Interoperability Standards Organization (SISO). The foundation of the SISO was consolidated in 1996. 1 An agency of the US Department of Defense, responsible for research and development of new technologies for military use.
298
Simulation & Modeling of Systems of Systems
Typical SIMNET: – Real-time trainers interconnected over a local network – MCC, Management Command & Control – SAFOR, Semi-Automated Forces generator
Figure 7.2. The SIMNET project
The first “official” interoperability standard is therefore DIS, and its publication by the Institute for Electrical and Electronics Engineers (IEEE) dates from 1992 for the “1.0 draft” version and 1995 for IEEE 1278.1 1995. SIMNET, then DIS, marked the beginning of the industrial era for distributed simulation. The first widely diffused document describing the basic concepts of distributed simulation is The DIS Vision [DIS 94], which is now partially obsolete, but easy to read and understand. 7.1.3. Some definitions 7.1.3.1. Distributed simulation system The terms “distributed simulation system” and the simpler “distributed simulation” designate a group of components interacting over a network (local or distant) which constitute a simulation system. These components can take several forms: – computerized simulation applications (known as “constructive simulations” in the current jargon);
Distributed Simulation
299
– piloted simulations, with a human operator in the loop, using entirely simulated material, generally in real time (known as “virtual simulations”); – “live” simulations (exercises or experiments with human operators in the loop, using real or partially simulated material); – various tools, usually programs, to manage, visualize, and supervise the system and its data. Note that we often use the expression “distributed interactive” simulation, signaling the importance of operators in the use of these systems and underlining the fact that they form an integral part of the systems. The subjacent network and the various utility programs that allow the components to work together also form part of the simulation system: service operators and programs are system components in their own right and contribute significantly to the complexity of the system. Other more or less synonymous terms are used instead of “distributed simulation system” by all or part of the simulation community, including “synthetic environments” (SEs) and “federations”. 7.1.3.2. Synthetic environment The term “synthetic environment” is often used in the place of “distributed simulation system” in British and Canadian English2. The two terms are not exactly synonymous: “distributed simulation system” or “distributed simulation” are technical terms used mainly by developers. SE shows a client/user vision of, for example, a training or study system. To give a more precise definition, “a synthetic environment is a computer-based representation of the real world, usually a current or future battle-space, within which any combination of “players” may interact. The players may be computer models, simulations, people or instrumented real equipments”3. The usage of the term “synthetic environment” and its acronym, SE, is unfortunately a source of confusion: American users often use this term to designate a representation of the natural (terrain, vegetation, weather conditions, and so on) or human (homes, roads, bridges, railways, and so on) environment in simulations.
2 In Britain and Canada, the expressions “distributed simulation” or “interactive distributed simulation” are more often used to refer to a group of technologies rather than to a simulation system. 3 Definition taken from the British document “Policy, information and guidance on the Modelling, Simulation and Synthetic Environments aspects of UK MOD Defence Acquisition”, version 1.0.0, March 2008.
300
Simulation & Modeling of Systems of Systems
The Synthetic Environment Data Representation and Interchange Specification (SEDRIS, see section 7.3.6) uses this meaning of SE. The usage of the term “actor” in this definition is also ambiguous, which is often used in simulation, with different meanings. 7.1.3.3. “Federation” and “federates” The High level Architecture (HLA, see section 7.3.2) norm uses the term “federation” to designate a distributed simulation system. The components of an HLA simulation (simulations, simulators, real systems, and various tools) may be known as “federates”. 7.1.3.4. Entities, objects, attributes, interactions, and parameter The very general term “entity” is often used in simulation to refer to real-world systems: in the military domain, it can refer to tanks, radars or missiles, for example, but also to their human users (infantry, snipers, pilots, and so on). Since the 1990s, the object-oriented paradigm in software engineering has become prevalent in the simulation community, a development demonstrated by the emergence of the HLA. In some cases, the general term “entity” has to be replaced by the term “object”, which has a very specific meaning in computing. An object is described by its “attributes”, which not only are qualitative or quantitative variables but may also, in the HLA paradigm, be complex components, sub-systems, or component parts of an entity. Objects interact, and the corresponding event is known as an interaction. For example, an action may be the explosion of a military warhead (the trigger object), which can affect one or more targets. This uncertainty concerning effects means that in HLA, for example, interactions are managed at federate level and not directly by the object classes of a federate (see section 7.2.1). An interaction is characterized by “parameters” which are characteristic variables (like the attributes of an object). The direction of a shot, for example, is a parameter of the corresponding interaction. 7.1.4. Interoperability in simulation According to the US Department of Defense (DoD) [DOD 95], “interoperability in modeling and simulation (M&S) is the ability of a simulation (…) to supply services to other simulations, to accept services from other simulations, (…), and to use the services thus exchanged to allow them to operate together effectively”.
Distributed Simulation
301
This definition is very general and requires commentary. Various publications attempt to provide greater precision concerning this notion. A first point is to distinguish between technical interoperability, that is, the ability to communicate and exchange data, and “substantive” interoperability, that is, the ability to exchange the meaning and context of the communicated information. This distinction over two levels (which seems to be covered by the previous definition from 1995) has been refined several times. In particular, Andreas Tolk [TOL 03] of Old Dominion University, Virginia, defines five different levels of interoperability within the levels of conceptual interoperability model (LCIM). These levels are as follows: – technical level: - physical interconnection established: ability to exchange bits and bytes, - physical connection, network layers; – syntactic level: - ability to exchange data in standardized formats. This is the case in most current interoperability standards (see DIS and HLA, section 7.3) but also in Common Object Request Broker Architecture Interface Description Language (CORBA IDL), Extended Markup Language (XML), Web Services Description Language (WSDL), and so on; – semantic level: - ability to exchange data and contexts, - definition of data models (meta-data): NATO’s Joint Consultation Command and Control Information Exchange Data Model (JC3IEDM), XML “Tag Sets”, and so on; – pragmatic/dynamic level: - ability to exchange and use information (knowledge), - process reference models, Unified Modeling Language (UML), Systems Modeling Language (SysML)™ documentation, and so on; – conceptual level: - shared world view (conceptual model at system of systems level) and the place of the shared model in this global vision, - ontologies and use of systems engineering standards.
302
Simulation & Modeling of Systems of Systems
The LCIM does not attract universal support from simulation professionals, who often find it too detailed4, but it does provide a good description of the complexity of simulation interoperability. Many of the standards presented in section 7.3 below are partial solutions to this problem (as they only cover the first three levels of the LCIM). Solutions or concepts exist to satisfy the last two levels, but these are either only partial or still work in progress. 7.1.5. Standardization The emergence of distributed simulation was accompanied by the creation of numerous interoperability standards. This is something of a virtuous circle: on the one hand, this technology could not have developed without interconnection standards and on the other hand, the discussion of these standards created favorable conditions for the appearance of new technical solutions and engineering processes specific to simulation. The first enthusiasm of the 1990s was followed by a realization of the amount of work required to develop true “plug and play” capabilities for simulation. Above and beyond the ability to interconnect simulations, which was the main aim of earlier efforts, we now recognize the importance of common data models and a shared vision of the simulated world, for example, coherent representations of the natural and artificial environment. At the start of the new millennium, the SISO estimated that between 10% and 15% of the standards needed for industrial production of simulations were available. The SISO is not the only organization involved in the development of standards for simulation. Other industrial consortiums play a role, including SEDRIS5 for the representation of the natural environment or the Object Management Group (OMG), which has produced a considerable number of standards directly applicable to simulation (such as UML, Model Driven Architecture (MDA), or SysML), although simulation is not the only target of these standards. We should not forget the IEEE and the International Standardization Organization (ISO), which publish standards 4 For example, interoperability, as seen by NATO, is split over three levels: physical interoperability (existence of a communication link, wired or otherwise, not necessarily provided by information and communications technology; typically, the human voice may be a communication medium), procedural interoperability (a protocol and a syntax must be known and used), and operational interoperability (which refers to the operation of the system in the context of use of other systems, via usage procedures, conventions linked to the interpretation of information and so to the construction of meaning; this aspect belongs to the field of semantics, contrary to what was discussed regarding procedural interoperability) [LUZ 10]. 5 SEDRIS is made up of eight ISO standards. Note that certain parts of SEDRIS were developed within the framework of the SISO.
Distributed Simulation
303
developed by other organizations (including public powers, academics, and private international groups) and thus contribute to their generalization. 7.1.6. Advantages and limitations of distributed simulation The considerable interest, even enthusiasm, provoked by the emergence of distributed simulation in some ways “hid” its limitations. Among these limitations, poor performance has been the subject of much discussion between advocates of different standards. This is not a characteristic of a particular standard, but a clear weakness in the basic principle: the performance aspects of distributed simulation are of great importance, and no standard, no matter how well designed, has been able to deal with the lack of network performance in terms of bandwidth and delay. All distributed systems produce large volumes of communications and require considerable bandwidth. Real-time systems cannot function if delay is too high. Since the mid-1990s, techniques have been developed to reduce the volume of exchanges, such as publication/subscription and management of data distribution, but all these optimization techniques have their limits. A second weakness is the inherent complexity of the techniques used. It rapidly became apparent that it is difficult to interconnect legacy simulations, which were not designed for this purpose. The interconnection of simulations with other simulations and with real systems is a general service which a simulation system should provide along with other services, such as visualization or data management; when a system was not initially designed to operate in connected mode, the risk of reduced function in distributed mode is clearly high. This poses major limitations on the capacities for reuse of legacy simulations. Remember that SIMNET was based on simulators developed for the project and not on pre-existing simulators designed to operate on a stand-alone basis. Although it is important to be aware of these limitations, we should not forget the considerable benefits of distributed simulations in the field of distributed training, where they open up the possibility of decentralizing activities to make them closer to operators. Economic studies have shown that the benefits of distribution are also considerable in the field of testing when a lack of resources precludes shared use. 7.1.7. Other considerations Distributed simulation has promoted or facilitated the development of new tools, including visualization or simulation tools, tools for the production of computergenerated forces (CGFs), and software tools used to assist in the exploitation of simulations.
304
Simulation & Modeling of Systems of Systems
7.1.7.1. Visualization tools The complexity of simulated worlds in a distributed simulation necessitates access to agile 2D or 3D visualization resources, enabling visualization of particular situations involving limited numbers of entities or, alternatively, of a global situation in a clear and intelligible manner. Two main types of tools exist: 3D “stealth” visualizers and 2D Plan View Displays (PVDs). They process only the data which is exchanged by federates between them, and cannot provide information on modeled entities within a simulation unless these entities publish the attributes needed by the visualizer: positions, speed, orientations, and so on. Another possibility is to visualize interactions using appropriate symbols: for example, an explosion or detection can be shown by changing the color of a symbol associated with the target. Stealth visualizers can only be used in situations with a low level of complexity (limited number of entities). The user has the ability to change the viewpoint using a joystick or other equivalent means. These visualizers are supplied with a library of 3D models for the most common entities (vehicles, buildings, and so on) and visual models of 3D terrains. They use the most common standards, DIS and HLA. PVDs, unlike stealth visualizers, allow visualization of a more complex situation on a map and allow appreciation of a general situation, a useful property in the domain of military simulation. 7.1.7.2. Computer-generated forces The concept of (CGFs) is described by the IEEE as “constructive simulations in which entities have a fairly elaborate behavioral model, allowing them to evolve without intervention from the operator”. At the outset, this type of product was developed to enrich the tactical environment of piloted simulators not integrated in a distributed simulation system. The emergence of interoperability standards has made CGFs into products which may be reused for various applications. Several of these products exist, produced both for commercial and for government use. Certain CGFs have become genuine constructive simulation development systems and are known the world over. 7.1.7.3. Various tools The complexity and the originality of distributed simulation have made the creation of a number of specific support tools essential. These are generally attached to each standard. Many are preparation, management, and exploitation tools for distributed simulation systems. They are sometimes integrated into programs providing support for the implementation of the standards used. Two particularly useful sorts of tools fall into this category:
Distributed Simulation
305
– data loggers, which keep track of the data exchanged in the course of a simulation and allow debugging or replay of an execution; – object model (OM) editors, which allow the coherent development and maintenance of data exchange meta-models between components of distributed simulation systems: the HLA’s Object Model Development Tools (OMDTs) are a typical example of this. Note that most of these tools use the XML format for the description of data. 7.2. Basic mechanisms of distributed simulation The concept of distribution works as long as we realize that certain practices used in monolithic simulation should be avoided when models are shared across different network nodes. These practices relate to two main aspects: – access to variable values and modes of transmission of exchanged data; – the synchronization of exchanges between federates in general. 7.2.1. Some key principles These main principles were first brought to light through SIMNET, then formalized for the first time in DIS. They have since reappeared and gained in precision in the HLA and other standards. New problems have been revealed through practical application, and solutions have been found. This section, then, represents the state of the art at the time of writing. In a monolithic simulation system, the transmission of information between objects and the activation of interactions is carried out by calling up functions or computerized procedures directly. These operations are managed by the simulation core, and this management has almost no impact on the global performance of the simulation. This is not the case in distributed simulation: – any direct call to an operation not available in the same network node blocks the system; this practice must therefore be used sparingly and in some cases prohibited; – not all network nodes necessarily have the same temporal reference, and values transmitted between federates must be accompanied by a time-stamp which constitutes their date of validity. The use of a transmitted value, then, may involve certain corrections before use if the user of the value intends to use it at a date before or after the time stamp.
306
Simulation & Modeling of Systems of Systems
Certain interactions may be “blind” (e.g. an explosion), and the objects affected are not necessarily known to the object responsible. For this reason, interactions are transmitted between federates, and their possible impact is processed locally. The lack of a common time reference is specific to distributed simulation. In a monolithic simulation, there is only one clock, and the current date is the same for all simulation objects. In distributed mode, even in the case of real-time simulations, local clocks may differ, and causality problems may arise. These problems can only be resolved by the presence of temporal references and the use of a certain number of precautions. In certain standards, synchronization mechanisms exist for the avoidance of incoherencies. The use of these methods is always tricky and has an impact in terms of complexity and global performance. Finally, in general, if we have the choice: – objects or models with strong ties to another specific object or model should, where possible, be placed in the same network node; – objects or models between which little interaction takes place may be placed in different nodes. 7.2.2. Updating attributes In line with the principles given above, simulation objects publish the values of their attributes (those of interest to other objects in the simulation) at regular intervals to avoid too frequent requests from other objects. The frequency of value publication depends on the subjacent models and their level of refinement: no general rule is imposed. For example, a tank may publish its position every 10 s, whereas an aircraft in similar conditions would publish its position once per second. These updates are made available to all network nodes using the DIS standard (transmission in broadcast mode). In HLA, there is a subscription mechanism that allows a reduction in the volume of data exchanged: only those federates that have subscribed to updates concerning specific attributes receive these values (multi-cast). The frequency of attribute updates is chosen according to the needs of models using these values and the dynamics of the objects concerned. The choice of these time steps is important for the global performance of the simulation: if the update frequency is too low, it may prevent refined models from functioning correctly, but if the frequency is too high, the system may become saturated.
Distributed Simulation
307
This mode of information transmission can only work if we specify extrapolation mechanisms for values of attributes which are rarely available at the exact date when we wish to use them. This point is discussed in section 7.2.5. These attribute update publication and subscription mechanisms are characteristic of distributed simulation and have a beneficial effect on the global performance of the simulation system. 7.2.3. Interactions An interaction is an event: it is dated and generally triggers a state change in one or more simulation entities. An interaction is activated by a simulation entity either voluntarily or because certain state conditions of the entity become fulfilled. Some examples of interactions: – Firing a missile: this is an interaction activated by a “firing” entity, which makes the missile pass from a state of immobility to a state of movement. It is therefore an “explicit event” (programmed for a set date) decided upon by an entity, following the classic concept of event-based simulation. – Detection of an airplane by a radar: this is a “state event” (also referred to as “implicit”) as the radar only detects its target if the signal-to-noise ratio reaches a certain level6: in this example, we see that there is no direct relationship (of cooperation) between the activating entity and the entity which is affected, contrary to what was observed in the previous example. – Explosion of a warhead: this is an interaction which may occur in a programmed manner (timing of firing) or on detection of a possible target (state event). These three examples show that an interaction will always be characterized by, at the least, its date, the activating entity, and the type of entity which may be affected. In distributed simulation, interactions are triggered by fixed types of entities and are managed by the federates which manage these entities and activate the interactions; they are characterized by their date and parameters that allow their effects to be calculated. They are not directed toward fixed objects, but toward other federates that may be interested in receiving the information.
6 The signal-to-noise ratio is calculated based on the position and posture of the airplane, the performance of the radar, the signal processing methods used, and conditions of electromagnetic propagation.
308
Simulation & Modeling of Systems of Systems
As with attribute updates, this method can only operate in the presence of a publication and subscription mechanism. This processing of interactions is particularly well described in the HLA. The difference between the notions of attribute updates and interactions is that attribute updates are relatively frequent and periodic, whereas an interaction is an asynchronous and relatively infrequent event. These two mechanisms share a declarative aspect: they must be declared, created, and initialized at the beginning of a simulation. Later, each federate must subscribe to the updates and interactions relevant for its own purposes. The different published standards define meta-models7 which can be used to describe these exchange mechanisms. The models thus produced for a particular federation are then used to initialize programs which manage these exchanges during execution of the simulation. 7.2.4. Time management When considering time, we must distinguish between two broad types of distributed simulation: – real-time federations; – non-real-time federations. A federation is “real-time” if it only includes real-time federates. In this case, we might think that each federate would function independently, while ensuring that each network node is synchronized using the same temporal reference at the beginning of the simulation. However, this is not always sufficient, as local clocks may diverge. It is better to use the mechanism described above (dating of information using time-stamps) and that described in the following section (the principle of extrapolation or dead reckoning) to ensure the temporal coherence of the information exchanged. As a general rule, it is advisable to use synchronization mechanisms and a shared reference clock alongside frequent recalibrations if we require strict causality and reproducibility in our simulations. This mode of realtime simulation is typical of the DIS standard, which allows users to establish these methods and understand their limitations. The Test and Training Enabling Architecture (TENA) and HLA standards also allow this type of operation. A mixture of real-time and non-real-time simulations (or a federation of exclusively non-real-time simulations) is harder to use. To our knowledge, the only 7 For example, the FOM in HLA: see section 7.3.2.
Distributed Simulation
309
standard that enables this mode of operation successfully is HLA. If even one federate operates in real-time, then the federation will be real-time. This presumes that all the federates involved will be able to operate faster than real-time and may be synchronized to another clock. HLA introduced the notion of “constrained” federates (where the speed of execution is determined by external synchronizations) and of “regulating” federates, which impose their rhythm on other federates. The federate synchronization mechanism is complex and difficult to implement, and its description lies outside the field covered by this book. For a more detailed explanation, see the HLA norm [IEE 00b] or [IGA 07]. Time management is provided in HLA by a specialized execution program and is the hardest part of the program to implement (see section 7.3.2.3). One characteristic of distributed simulation is the fact that each federate has its own local time8 that may differ from that of other federates. All the implemented mechanisms aim to reduce these differences. A common error is to believe that all federations have a reference time: this notion is only applicable to real-time systems and is not mentioned in the HLA norm, for example. 7.2.5. Dead reckoning This mechanism is required for the attribute update mode used in distributed simulation (periodic publication). Imagine that an object A receives the position and speed of another object B at the date T1. This information is dated, and the attached time stamp carries a date T 0, which is different from T 1 (usually before T 1). If these two dates are not too different, simple linear extrapolation generally gives an acceptable estimation of the position of B at time T 1. This mechanism is valid on the condition that attribute values are published relatively frequently. If publication occurs too often, however, the risk of saturation in the network is high, and this level of precision has no obvious interest for the global operation of the simulation. This risk is reduced by avoiding over-frequent refreshment of information where possible: to return to the example above, object A avoids resending information on its position if its speed does not vary (or is subject to minor variations), as a simple linear interpolation provides other entities with a good estimation of its position (see Figure 7.3). We must, then, add supplementary devices to this mechanism:
8 Or “logical time”.
310
Simulation & Modeling of Systems of Systems
– The same extrapolation algorithm should be used by the publishing object (B) and the client (A). – In this way, B calculates its extrapolated position on a regular basis and compares this to its real position. If the extrapolated position is too different from the real position, B must update its true position and speed to allow subscribing objects to estimate its attributes correctly.
Figure 7.3. Principle of dead reckoning
For this to work, the thresholds that trigger updates must be chosen by senders and clients working together. Agreement is also required on the frequency of examination of attributes which are checked before updating. The extrapolation formula chosen does not have to be linear; it may be adapted to fit the level of models used in various federates. These mechanisms are used independently of the standard chosen. However, dead reckoning is part of the DIS norm but does not feature in HLA. HLA considers that this type of algorithm belongs to the modeling process and that this aspect should be managed by users without the need for normative rules. 7.2.6. Multi-level modeling The HLA standard popularized a mode of transferring the responsibility of modeling a simulation object from one federate to another in the course of a simulation. This transfer can take place dynamically when necessary (during run-time).
Distributed Simulation
311
These transfers allow rigorous processing of modeling changes in cases where, for example, there is not a constant need for highly detailed models. For example, a very simple cinematic model can be used to simulate airplane flight during the cruise phase, but a mode-detailed model is required for landing. This kind of transfer is made possible by two characteristics of the HLA: – the rule declaring that each instance (object) of a simulation should not be managed by only one federate at any given moment; – a process for the transfer of responsibility for object management between federates. These processes are described in the OMs of different standards (e.g. the Federation Object Model (FOM) in HLA) so that they may be used during execution. This mechanism should also be included in the new version of the DIS standard. Obviously, these mechanisms are not particularly easy to use (whichever standard we choose) but present obvious advantages for users. 7.2.7. Section conclusion Most of the modes of operation described up to this point are typical of distributed simulation. They have been discovered by experience when using the earliest federated simulations, as a result of errors which on occasion created blockages (typically in time management). We should remember that a large number of simple mechanisms used in monolithic simulation cannot be used in distributed simulation and that the operation of a distributed simulation is a more complex affair. These basic principles have been established with the aim of obtaining acceptable performances in distributed simulation and of respecting causality in simulation. The various existing standards do not offer the same services, and some aspects are left to the developers. We would make two remarks: – it is sometimes good to leave a certain amount of freedom to developers in cases where there is no general solution to a particular problem; – not all standards are the same. When we have the ability to choose a standard (something which is relatively rare), we must analyze our immediate requirements and analyze the risks associated with different options. Some of the principles described are not necessary for all projects, for example, the transfer of object management or optimized data management in HLA (not described here). It is, however, important to know which norms offer these
312
Simulation & Modeling of Systems of Systems
optimization services for the most complex projects. Furthermore, no norm imposes the use of all of its proposed functions. 7.3. Main interoperability standards 7.3.1. History As mentioned above, SIMNET demonstrated the following: – the usefulness of the concept of distributed simulation (for training support in particular) and its feasibility; – the importance of defining norms for simulation and simulator interoperability to build distributed simulation systems, reusing existing systems to reduce the need for constant development of new systems. Following SIMNET, various different interoperability norms were devised. The first two initiatives appeared almost simultaneously: – Aggregated Level Simulation Protocol (ALSP)9: devoted to the interconnection of event-based constructive simulations (early 1990s), used essentially in the United States (with some experiments by NATO and one in France within a NATO framework); – DIS: this norm, for the interconnection of real-time piloted simulators (IEEE 1278 norm, from 1995), experienced considerable success in Europe as a whole and in the United Kingdom in particular. In parallel with these efforts by the simulation community, industrial input led to the creation of CORBA, an object-oriented standard for distributed systems. CORBA is not specifically aimed at simulation but is still used today within (but mostly outside) the field of simulation. CORBA appears here due to its influence on the design of the most important standard: in the mid-1990s, the US DoD decided to develop its own norm in an attempt to rationalize and accompany the various existing standards (CORBA, DIS, and ALSP). The result of this process was the HLA standard. A preliminary version of the HLA was published in 1996, and the first stabilized and truly operational version appeared in 1998 (the US DoD’s version 1.3, February 1998). The development of the HLA was inspired by previous efforts (DIS and CORBA to some extent, but especially ALSP). The emergence of HLA interrupted 9 Contrary to what its name suggests, this is not a protocol in the network sense of the term, but a genuine architecture which provided considerable inspiration for HLA.
Distributed Simulation
313
previous normalization efforts, DIS and ALSP10, but not CORBA, which had a wider field of application and was supported by a large community of users. A “civilian” version of the HLA standard appeared in the year 2000 and is still in use today as IEEE1516. A new version of this standard is due for publication in 2009. The principles of this standard have not changed from US DoD version 1.3 in 1998 to the new normalized 2009 version of IEEE-1516; the differences lie in added improvements and repairs to teething problems encountered in early versions of the HLA. From 1996, the US DoD pursued a strong policy of imposing HLA as the unique interoperability standard for simulations. This policy (relaxed from 2001 onwards) produced significant effects, making HLA the most thorough and evolved standard available. HLA is the only standard to be accompanied by a methodological guide, a dedicated verification, validation, and accreditation (VV&A) methodology and a certification of conformity to a standard. For this reason, we shall describe HLA in considerable detail in this chapter. DIS will also be described as it is widely used and based on different concepts than those used by HLA. ALSP will not receive particular attention as its use is now marginal, and its main concepts and qualities are reproduced in HLA. We have also included a description of TENA, a more recent development by the US Army (2000 onwards) in section 7.3.4. This description is much less detailed as TENA has only recently been opened to non-American users11, has not been formally standardized, and is not widely known or used outside the United States. Finally, following the incredible consecration of efforts on interoperability standards, the US DoD has launched a study to analyze the positive and negative qualities of existing standards and to attempt to channel future efforts. 7.3.2. HLA HLA Evolved (IEEE Std 1516-2010) was published in 2010. Major improvements are Dynamic Link Compatibility (DLC) between RTIs, Modular FOM (to improve reuse of Federation Objects Models), update rate reduction for data exchanges, better fault tolerance and WEB Services API with a WSDL (Web Service Description Language) binding. 7.3.2.1. Principles of HLA The two main aims of HLA were made clear from its inception in 1996: – interoperability of simulations and simulators; 10 A momentary interruption for DIS, which is still used and continues to evolve, but one which proved definitive for ALSP (although it is still used a little in the United States). 11 Apart from certain exceptions, such as the United Kingdom and Sweden.
314
Simulation & Modeling of Systems of Systems
– reuse of simulations and simulators for different applications, not necessarily those for which they were designed. This second objective refers to problems of validation and revalidation which are not covered in depth in this chapter. The importance of validation processes in simulation has not, however, been underestimated, and a good deal of work is in progress on this subject. A supplementary guide to VV&A for HLA federations was produced in 2007 [IEE 07]. Clearly, reuse should be understood in the context of distributed simulation systems: the aim is to reuse simulations and simulators in their entirety. Other reuse concepts exist, such as simulation frameworks or simulation environments. These environments enable the reuse of components (models, object classes, services, tools, and so on) in a particular development and use environment which do not present the characteristics of universality presumed by HLA. HLA offers the minimum conditions, considering the state of the art, for ensuring technical interoperability of simulations: as we stated in section 7.1.4, interoperability may be envisaged at certain levels. On the preceding scale (LICM) which goes from 1 to 5, HLA corresponds to the base levels 1 and 2 and may contribute to level 312. This shows that more work is required to improve the HLA norm in terms of interoperability. Nevertheless, a broad consensus has been reached that although HLA cannot work miracles, it strongly contributes to the fulfillment of its initial aims, and as a norm, it is the best solution currently available. 7.3.2.2. “Architecture” and “high level” The term “architecture”, in this case, designates “major functional elements, interfaces, and design rules, pertaining as feasible to all simulation applications, and providing a common framework within which specific system architectures can be defined”13. This definition is based on the definition of architecture used in systems engineering norms, such as the ANSI/IEEE Std 1471-200014. Note that this 12 DIS is cited at level 1, as a “manual patch” would be. The main problem identified in DIS is its insufficiency as a standard which obliges each new application to reinvent its own messages and thus move away from the standard! 13 In the words of Dr. Judith Dahmann, recognized as the main inventor of HLA [DAH 97, DAH 98]. 14 This norm describes architecture as “the fundamental organization of a system, embodied by its components, the relationships between these components and with the environment, and the principles governing its design and evolution”.
Distributed Simulation
315
definition does not mention the way concepts are implemented: the HLA standard provides neither indications nor constraints as to the way the architecture should be implemented. The standard gives developers the freedom to choose the hardware and software solutions (including the communications and network aspects) best suited to their problem and usual working environment. In the HLA standard documentation, no reference is made to the use of particular development platforms: operating systems, development environments, network protocols, and so on are not mentioned. This is what gives HLA its “highlevel” character. This also implies that the HLA itself provides no technical infrastructure for the development of models and simulations: once HLA has been chosen, developers must choose their own infrastructure based on their own aims and constraints, while respecting the standard. 7.3.2.3. Functional description and associated vocabulary An HLA federation is a distributed simulation system involving a group of elementary simulations, known as federates, which exchange information. An HLA federate is not necessarily a simulation; the term is used to designate any participant in a federation, whether hardware or software. The HLA standard describes a federation as a group of three components: the federates, the run-time infrastructure (RTI), and the normalized interface. These three components are traditionally illustrated in Figure 7.4 in the order (1), (3), (2) from top to bottom.
Figure 7.4. Diagram of an HLA federation
316
Simulation & Modeling of Systems of Systems
7.3.2.3.1. Federates As stated above, federates can be very different in nature, and to demonstrate this possibility, early in HLA, prototype federations were developed in very different fields of application: – interconnection of piloted simulators to enrich training of pilot/user teams; – federation of computerized land–air–sea simulations for general staff training at inter-army level; – hardware-in-the-loop evaluation platforms; – study simulations. The main role of federates in a distributed simulation system is to represent the systems being simulated. Federates may also be real systems15. 7.3.2.3.2. RTI A RTI is a computerized implementation of the HLA’s interface specifications. It is made up of a set of computer processes written in a given programming language, which plays the role of a reduced distributed operating system. This program allows several federates, making up a federation, to exchange data during the simulation via a local or long-distance network. The basic mechanisms that support communications between federates and the services offered by the RTI are described in section 7.2. An RTI allows federates to communicate information “cleanly” and synchronizes changes within the federation. This distributed program “hides” low-level aspects in particular (network and communications) from users. The RTI also fulfills the difficult role of temporal synchronization of federates, allowing, for example, correct operation between real-time simulators and non-real-time constructive simulations. Contrary to popular belief, an RTI is not a “standard” program. Several different RTIs exist, produced by different groups at state, university, or commercial level. They have different qualities, performance levels, and costs, but must always respect the specifications of the norm. The different RTIs available can be implemented on different platforms16 and sometimes use different network protocols. Their implementation may be based on different technologies (e.g. Java or C++), meaning that federates using different 15 This may involve real operational systems, in which case the federate interacts with the federation via a software interface. 16 PC/Windows and Unix derivatives: PC/Linux, SGI/Irix, SUN/Solaris.
Distributed Simulation
317
RTIs cannot be used together (although portals to allow this should be available soon). All RTIs do, however, offer the same services (in concordance with the norm) and can operate in a distributed simulation system. In any case, all services are offered or accessed through a normalized interface. 7.3.2.3.3. The normalized interface RTIs are not, then, a single standard middleware program, but their interface with federates is normalized. It is this normalization that ensures interoperability between simulations and their capacity to evolve from one HLA federation to another. Normalization also gives us the choice of different RTIs for different applications (for reasons of availability, performance, cost, greater suitability for particular application types, and so on) without modifying the federate–RTI interface. 7.3.2.4. The HLA standard The HLA standard is made up of three normative documents [IEE 00a, IEE 00b, IEE 00c], a methodological guide [IEE 03] that supports design, development, and use of HLA federations, and a second, additional guide to VV&A for HLA federations [IEE 07]. The latter two documents are guides and not norms; only the first three documents have this status. The documents making up IEEE standard 1516 HLA are available from the IEEE Website (www.ieee.org) at a cost of a few hundred dollars. The three normative documents are as follows: – description of architecture and HLA rules [IEE 00a]; – “interface specifications”, that is, the description of interfaces between federates and the RTI [IEE 00b]; – HLA object model templates (OMTs)17 [IEE 00c]. The first of these documents (architecture and rules) is the shortest. In a few pages, it describes the concepts discussed in the previous paragraph and summarizes the correct operation of federates and federations in 10 rules (five for each). For a detailed understanding of these rules, beyond their immediate interpretation, reference to the documents concerning interface specifications and OMTs is required. Two main principles become particularly clear: – all modeling (representation of real systems) takes place in federates and not in the RTI; 17 Note, in passing, that HLA is considered to be “object-oriented”. The acronym OMT in HLA should not be confused with OMT, Object Modeling Technique, in software engineering.
318
Simulation & Modeling of Systems of Systems
– all exchanges of information and synchronization of federates can only take place via the RTI. The second document making up the HLA standard is made up of “interface specifications”. This includes the description of services provided by the RTI and of the services that different RTIs expect to be provided by federates. This document is fairly long (but not all the services described are used in each federation); each particular service is described with: – preconditions for use; – input variables; – output variables; – “post-conditions” at the end of the service; – exceptions to nominal operation. The logical sequence of processes is clarified where necessary. Finally, application programming interfaces (APIs) in the usual languages are provided as an appendix to the norm, although they are not explicitly part of norm 1516 in its current version. The services provided and used by RTIs (of which there are over 100) are divided into seven categories and implement most of the principles described in section 7.2: – federation management; – declaration management; – management of objects and interactions (during run-time); – time management; – management of object properties; – optimized management of data distribution; – specific and additional services. The third document describes the OMT. It is made up of a set of normalized services and a format for the description of federation objects and interactions between federates. In general, the OMT is the standard format attached to these tables. It specifically allows the description of a set of tables known as the FOM, which precisely describes the exchanges between federates during run-time. The tables that make up the FOM are the principle input data in RTIs.
Distributed Simulation
319
Detailed study of these three documents and of their practical application is necessary to understand the HLA standard. Although the general principles of the norm are relatively easy to comprehend, it is widely recognized that its practical use requires considerable technical ability. For this reason, a methodological guide was added to the norm in 2003 [IEE 03]. This is the Federation Development and Execution Process (FEDEP), which is, in fact, an engineering process adapted to the development of simulation systems. In the context of the HLA, the “object” concepts differ from the usual concepts used more generally in computing: there are a number of significant differences that are clearly set out in the first of the normative documents [IEE 00a]. This difference stems from the fact that, in HLA, the “object” concept only concerns communications and not the representation of the real world. The standard is very clear on this point and, in principle, prevents misunderstanding. 7.3.2.5. Advantages and drawbacks The HLA is the most advanced standard for distributed simulation and evolves continually, making use of user feedback. It comes with a specific system engineering process and a VV&A methodology [IEE 07]. The US DoD, moreover, financed the development of an RTI verification process to guarantee, as far as possible, conformity to the norm. The DoD also financed and provided NATO with access to a certification of conformity of federates to the HLA, which facilitates the interoperation of models and simulations not initially designed to work together. Finally, HLA benefits from a wide base of tools (both commercial and free), which allow efficient and effective implementation. The HLA is widely used world over, and the size of the community of users acts to guarantee it endures and evolves. However, the HLA contains flaws alongside its positive qualities. HLA is the standard that offers the most capacities (such as coordinated time management): consequently, it is complex, and its use requires high levels of expertise and user training. It may be costly as it is difficult to use without support tools: the few freeware programs available for RTIs only support old versions of the standard and are often not complete enough for truly professional use. Finally, although HLA is the only standard offering coordinated time management and optimized data distribution, these services are costly in terms of performance. 7.3.3. DIS 7.3.3.1. Principles From the outset, DIS was strongly oriented toward real-time training applications (interconnection of piloted and virtual simulators) such as SIMNET. The basic principles of DIS (directly inherited from SIMNET) are as follows:
320
Simulation & Modeling of Systems of Systems
– no central node for the management of distributed simulation systems, giving autonomy to interconnected simulations; – precision in exchanged information (ground truth): information is subject to noising, if necessary, at model level; – DIS is a protocol (in the network sense of the term), not an architecture: - information is transmitted as coded (binary) messages known as Protocol Data Unit (PDU)18, with time stamps, for periodic diffusion of information (see sections 7.2.2 and 7.2.5), - unreliable communication in broadcast mode (UDP/IP): useful information is filtered on reception (no global relevance controls on exchanged data); – no coordinated time management, but reference to a shared clock representing real time; – vocabulary: usage of the terms “exercise” (now replaced by “federation”, as in HLA), entities, and attributes; – all entities in the simulated world must use the same system of geographic coordinates (WGS84). The main characteristic of the DIS standard is the use of PDUs, binary messages exchanged between federates across the network. The first version of the DIS standard (1995) only offered 27 different PDUs: for starting an exercise, entity creation, entity state (position and posture of an object), detonation, refueling, signals, and so on. Later, new families of PDUs appeared: entities and interactions, weapon actions, logistics, simulation management, and so on. The DIS standard experienced disproportional growth due to extremely variable requirements and the very active imaginations of the developers. The principle advantages of PDUs are the efficiency and compact character of messages (an advantage more or less wiped out by the diffusion mode used). Their main drawback is the lack of flexibility, hence the development of new, specific, non-standard PDUs that are more suitable for new applications. DIS was standardized by the IEEE (IEEE 1278.1-1995 then IEEE 1278.1A1998). For political reasons, the norm evolved very little between 1998 and 2002, but the standards have since been reaffirmed. Standardization efforts were also relaunched within the SISO and new versions should be published in 2009 or 2010.
18 Vocabulary taken from the field of computer networks.
Distributed Simulation
321
In conclusion, DIS is not just important because of its historical role in the field of distributed simulation: it is also a set of technologies which have proved their value in practice. It is important to be aware of the limits of DIS, which have been the cause of misunderstandings in the past: DIS is only applicable to real-time simulations with entities which interact in a limited manner in space and time. At conceptual level, there is an obvious lack of subjacent architecture, although partisans of DIS deny this. The most unfortunate flaw in DIS is the fact that PDUs mix simulation with subjacent modeling services: this makes standardization difficult, as each new application has an irritating tendency to develop its own specific PDUs due to the lack of generality of existing PDUs. This flaw may well be repaired in new versions of the standard. 7.3.4. TENA TENA was developed by the US army, begun in 2001. The term “training” was added later than “test”, the initial objective of TENA being to develop a specific architecture for the interconnection of test centers based on the generic HLA (a goal which was never really attained). Unlike DIS or HLA, TENA is not an open standard, and discussion by a standardization organization such as the SISO, the ISO, or the IEEE was not included in the plan. Nevertheless, there are several good reasons to include TENA in this work: – Initially, TENA is the most recent interoperability architecture, was developed for inter-army use, and is now known outside of the United States, being no longer reserved to the US Army. – TENA is widely used in the United States (somewhat less, however, than the DIS protocol or the HLA). – It was made accessible to allied forces under certain conditions in mid-2008. – Finally, and especially, as TENA is relatively recent, it was able to benefit from previous developments (DIS, HLA, and CORBA) and therefore presents numerous technical advantages. 7.3.4.1. Main principles As TENA is not an open standard and has yet to be tested in France, the information used here comes essentially from articles, various presentations, and tutorials offered by the TENA organization during various conferences. After a period where the data available was mainly promotional in nature, more precise details became available in 2008 and allow us to construct a clear idea of the concepts involved. For reasons of brevity, TENA is presented here by comparison
322
Simulation & Modeling of Systems of Systems
with HLA, although the two architectures are based on different philosophies and development models. The TENA project is supposedly a simulation architecture based broadly on HLA, but in actual fact, it takes more of its inspiration from CORBA. TENA adopted object-oriented concepts more easily than HLA: remember that the first document of the HLA standard provides a clear description of the limitations of the object concepts used in relation to the object-oriented approach in software engineering. Like HLA, TENA includes a data model (for data exchange) and middleware ensuring the interoperability of components in a distributed simulation system. There is also a TENA-compatible engineering progress known as Concept of Operation (CONOPS), similar to but less detailed than FEDEP. The resemblance stops at this point. 7.3.4.2. Objectives The main objective of TENA was to “define a common architecture for the test (and training) community”. More precisely, the TENA project aimed to create one common “OM”, which would be used by all test centers, and to define and develop a common middleware program, based on this model, to allow interoperation and reuse between test centers. Note the first difference between TENA and HLA: TENA concerns a single common middleware program, whereas HLA has always allowed the development of different instances of the RTI. In all its objectives and sub-objectives, TENA clearly identifies the development of standard support programs, whereas HLA recognizes the use of RTIs and provides (partial) assistance in specifying them, but states clearly that these are not part of the standard. TENA defines itself as an “open” architecture, but with a specific meaning, as it is based on a single program developed by the TENA organization. It is “open” in that: – the development of the architecture and associated programs is discussed by a group of users, the Architecture Management Team (AMT), essentially made up of organizations within the US DoD; – the programs developed are freely available (after registration www.tenasda.org), but remain the property of the US Government. 7.3.4.3. Main characteristics TENA is described by its meta-model, which defines “classes” and “objects” in the architectural sense of the term. Unlike previous standards (DIS and HLA), this meta-model is highly developed and is based on the concept of logical range (LR).
Distributed Simulation
323
Communication is maintained by the definition of a common OM and TENA Middleware, which may be compared to the HLA’s FOM and RTI, respectively. The comparison, however, stops here: TENA also identifies a need for object databases, either predefined in the architecture (within the TENA repository) or archived within the project (in the Logical Range Data Archive, LRDA). Figure 7.5 shows the typical structure of a TENA application.
Figure 7.5. TENA (taken from a tutorial offered by the SISO)
Unlike HLA, TENA completely adopted the object-oriented concept, and this gives it a certain number of advantages in terms of interaction: for example, the ability to invoke methods between applications, something which is not possible in HLA. TENA supplies users with standard object libraries and tools for automatic generation of code when developing new objects, and it is clearly a mature standard with a good selection of tools. HLA, on the other hand, does not supply standard object libraries as it is essentially concerned with interoperability aspects, and under this paradigm, all models are considered to be the responsibility of application developers and not of the architecture (application reuse is left to organizations using HLA, and although HLA promotes reuse, it does not provide specific tools for this purpose). On the other hand, HLA offers coordinated time management mechanisms, whereas TENA only applies to real-time simulation, providing a definition of time but no time management mechanisms.
324
Simulation & Modeling of Systems of Systems
7.3.4.4. Conclusion on TENA As with DIS and HLA, the impact of TENA has been felt most strongly within the US DoD, but it does have wider applications19. In terms of concepts, TENA is based on a traditional and rigorous object-oriented approach. This allows the interoperability concepts of previous standards to be extended, but at a price: TENA is less general than its predecessors. The lighter HLA meta-model can be used with any application (virtual, constructive, or living) whether or not it is object oriented. TENA is essentially designed for real-time use, as was DIS: only HLA proposes coordinated time management, which allows real-time and non-real-time applications to be mixed. From what we know of TENA (mainly from secondary sources, for the moment), it is hard to deny that this architecture is very attractive as far as concepts are concerned, despite of certain obvious limitations: it is essentially designed for real-time applications, either with a human operator or hardware in the loop. The real question to ask before using TENA concerns its business model: TENA has never been opened for international standardization and remains the property of the US DoD. Moreover, TENA is supported by software and libraries that also belong to the US Government and have no commercial or freely accessible equivalents. In these conditions, it is clear that the survival of TENA (and projects based on TENA) is sensitive to policy changes and dependent on the availability of a sufficiently large budget within the US DoD. 7.3.5. The future of distributed simulation: the LVC AR study Having supported the DIS standard up until the mid-1990s, the US DoD imposed a single architecture, the HLA, from 1995 to 2000 with the aim of replacing alternative developments; HLA was the only standard with sufficient financial backing in this period. From 2000 onwards, the DoD’s policy was relaxed, allowing the return of DIS (which had not disappeared, but was not being developed) to the frontline of operations, followed by the development of a new architecture, TENA. In 2007, the DoD recognized that the development of various different architectures was limiting interoperability and launched a study to see if existing architectures could be harmonized and to attempt to reconcile the business models 19 A recent survey, published as part of the LVC AR project (see section 7.3.5), highlights the fact that 20% of projects (and not just US projects) use TENA. DIS and HLA are both used in approximately 30% of cases. The 20% or so remaining use different protocols or architectures little known outside of the United States.
Distributed Simulation
325
of the DIS and HLA standards and the TENA. This study was carried out with (limited) participation from US allies and goes by the name of Live, Virtual, Constructive Architecture Roadmap (LVC AR). It was led by the Joint Force Command (JFCOM) and was completed in late 2008. Some of the conclusions of this study are known, but the final report has still not been published at the time of writing (despite several announcements made since October 2008), raising doubts as to the unanimity of its conclusions. The study does not appear to have proposed the definition of a new common architecture, or if it does, this is clearly not a short-term objective. The DIS, HLA, and TENA will therefore be maintained, and it is unlikely that a common solution will be derived from the three in the near future – something which is not particularly surprising given the considerable differences between the three approaches and their success with users. The only action currently underway is a study which aims to create a common data exchange model which would federate the three approaches. It is possible that the DoD will finance (or is financing) other connected projects, but there is little serious information available on this subject, and rumors are all we have to go on. 7.3.6. Other standards used in distributed simulation Other standards have appeared, or are emerging, from within the SISO and other organizations such as the OMG or the World-Wide Web Consortium (W3C), which should facilitate the development of distributed simulation systems. Not all these apply specifically to distributed simulation, but they contribute to facilitating the implementation of this paradigm, for example: – SEDRIS: a series of eight ISO standards that aim to facilitate the creation, exchange, and reuse of natural and artificial environment databases for simulation. These standards were approved in 2007, but their use is progressing only slowly. SEDRIS is not really an interoperability standard, but it provides a number of concepts and tools that are useful in distributed simulation. For example, one of SEDRIS’ components, the Spatial Reference Model (SRM), provides a clear description of geographic coordinate systems and provides validated programs to facilitate the passage from one coordinate system to another. The SRM has been adopted by TENA. – Coalition Battle Management Language (C-BML): among currently unsolved problems in training simulations, one priority is developing interoperability between operational information systems (OIS) and simulations. A number of projects were launched in the United States in the 1990s, without much success, with the aim of defining a standardized interface between OIS and simulations. This aim presents significant challenges as communication is often done manually and is therefore
326
Simulation & Modeling of Systems of Systems
slow and resource intensive. Work is underway at the SISO, with strong NATO support, on the specification of a formal language which would allow simulations to be piloted from OIS and allow the simulation to send messages to real information systems. This must be done in concordance with the standards used in OIS, such as the STANAG JC3IEDM cited elsewhere in this chapter. The first developments and experiments carried out have shown considerable promise and demonstrate the importance of this effort. – Military Scenario Development Language (MSDL): this is a scenario description format which facilitates archiving, reuse, and exchange of simulation scenarios. Experience has shown that the development of new scenarios from scratch is slow and costly. A first version of this standard exists, which has generated considerable interest in spite of its limitations. The standard is expected to develop in concordance with C-BML. – Reference FOM HLAs: with the arrival of HLA, the need to define “reference” OMs for particular domains of activity emerged with the aim of saving developers from having to build up this data from scratch for every new application. A certain number of reference FOM HLAs exist: many are extremely specific and hardly accessible to the public. Only one has been published and standardized by the SISO: the Real-Time Platform Reference FOM (RPR FOM) which was designed to facilitate the passage of DIS distributed applications to HLA. The RPR FOM is widely used in almost all real-time distributed training systems. A large number of other standards currently in development could also be cited here. Following the period of enthusiasm which followed the release of DIS and HLA, developers of distributed simulation systems have been faced with new, often unexpected problems. HLA and the other available standards do not respond to all the procedural and technical questions raised by developers of distributed applications. Progress is made by sharing specific solutions and often by the development of new interoperability standards (not just for the interconnection of simulations). 7.4. Methodological aspects: engineering processes for distributed simulation Surprisingly, no standards describing the engineering process involved in simulation existed before 1999. These standards were clearly not considered to be essential: software engineering methods and the usual standards for systems engineering were judged sufficient for simulation development and use. With the appearance of distributed simulation, the character of the problem changed: it was clear that faced with the complexity and difficulty of application of these concepts, a specific engineering process was required. First, the DIS
Distributed Simulation
327
community published a document, Exercise Management and Feedback [IEE 96], which is barely relevant nowadays20, as it only covers phases 4 and 5 of the HLA’s FEDEP (see section 7.4.1); its contents will, moreover, be covered by the upcoming Distributed Simulation Engineering and Execution Process (DSEEP) standard (see section 7.4.3). With the development of HLA, a real simulation engineering manual was published for the first time: the HLA FEDEP. This first document (final version 1.5, published December 1999) is based on version US DoD 1.3 of the standard and is therefore somewhat out of date. Nevertheless, it provided the inspiration for the two or more complete documents which followed: – the HLA IEEE 1516 version of the FEDEP, published in April 2003 [IEE 03]; – the Synthetic Environment Development and Exploitation Process (SEDEP), based on a EUCLID21 sponsored by the Western Europe Armament Group (WEAG)22. 7.4.1. FEDEP The FEDEP is a seven-step process defined in the diagram below: 1. define federation objectives; 2. perform conceptual analysis; 3. design federation; 4. develop federation; 5. plan, integrate, and test federation; 6. execute federation and prepare outputs; 7. analyze data and evaluate results. Contrary to what the traditional FEDEP diagram (Figure 7.6) seems to suggest, this is not a cascade process, and it is often necessary to return to and update previous steps during the process. Each of the steps involves three or four sub-steps. For each step and sub-step, the FEDEP provides: – a list of reference documents (at the beginning);
20 A fact acknowledged by the DIS developers themselves. 21 European Cooperation for the Long-term In Defence-Research and Technology Programme. 22 The WEAG was dissolved in 2005; its work in the fields of research and technological development now come under the aegis of the European Defense Agency (EDA).
328
Simulation & Modeling of Systems of Systems
– a list of activities to perform; – a list of documents to produce and/or modify. This list of documents is set out in the process description guide. The FEDEP has been widely used in real projects with considerable (and deserved) success. However, a few criticisms have been made, concerning: – unique reference to the HLA standard (as a document in the IEEE 1516 series, the FEDEP is explicitly linked to this standard), hence a relative lack of generality; – relative weakness in characterizing the documentation produced; – the lack of a preliminary phase for analysis of the needs of final users of the simulation system. Existing Authoritative Info on Over all D omain Available Existing Domain Plans Descr iptions R esources Scenarios Information D efine F ederatio n Objectives 1 Existing Conceptual Models
Federate D ocumentation
Supp orting Resources
Initial Planning Documents
Object Model Data Dictionary Elements
Federation Scenario(s)
Perfo rm Conceptual Analysis 2
Existing FOMs and BOMs
Multiple Items: Modified/New Federates
SOMs
Design Federation
Implemented Federation Infrastructure Federation Development and Execution Plan
Fe deration Federate & Conceptual Federation Designs Model
RT I Initialization Data (modified)
List of Selected (existing) Federate s
3
FOM FED/F DD Develop Federation
Scenar io Instances
4
Plan , Inte grate, and Test Federation 5
Federation Agreements
Federation Test Criteria F ederatio n R equir ements Federatio n Objectives Statement
Execution Environment Descriptio n
Supporting Databases
Tested Federation
Execu te Federation an d Pre pare Outpu ts 6
Derived Outputs
A nalyze Data and Evalu ate Results 7 Final Lessons Report Learned
Reusable Products
Figure 7.6. The seven stages of the FEDEP (diagram taken from the guide IEEE 1516.3, with authorization of the IEEE)
These considerations led to the development of the SEDEP, which attempts to repair these flaws. The two developments (IEEE 1516.3 and SEDEP) were carried
Distributed Simulation
329
out at more or less the same time23 and show obvious similarities, but the SEDEP has not been widely diffused. An additional guide was added to the FEDEP in 2007 and describes specific validation activities for use in distributed simulation systems (IEEE Recommended Practice for VV&A of a Federation – An Overlay to the HLA FEDEP, [IEE 07]). 7.4.2. SEDEP As stated above, the SEDEP was created in the course of a EUCLID project by the name of RTP 11.13, Realizing the Potential of Synthetic Environments in Europe. The final version of the SEDEP (V2.0) was published in 2004, approximately 1 year after the FEDEP (IEEE 1516 version). It is made up of eight steps: 1. analyze user needs; 2. define user requirements; 3. define the federation system; 4. design the federation; 5. develop the federation; 6. integrate and test the federation; 7. implement the federation; 8. evaluate the results. If we compare these steps to those of the FEDEP, we see a breakdown of the first phases with the aim of understanding user requirements. Moreover, the SEDEP is explicitly based on a repository of data, models, simulations, and federations, built up using the results of different steps. Compared with the FEDEP, the SEDEP has not benefited from the support of a major standardization organization or from significant promotional resources. It does, however, have several positive attributes: – clear positioning of requirement analysis as the first step of any development; – SEDEP covers both DIS and HLA; – SEDEP clearly highlights the importance of a collaborative repository and working environment in association with a major simulation project.
23 The FEDEP IEEE 1516.3 was published in April 2003 and the SEDEP in March 2004.
330
Simulation & Modeling of Systems of Systems
On the other hand, it seems to be particularly marked by the use of SEs for training purposes, although it was not developed exclusively for this use of simulation. Like the FEDEP, the SEDEP was used in practice in a few projects (probably fewer than FEDEP). In 2006, a new FEDEP update phase was reinitialized (see above): the authors of the SEDEP24 have admitted that this development of the FEDEP would cover the main characteristics of the SEDEP and have officially declared that there is no further reason to update or develop the SEDEP. 7.4.3. DSEEP This new guide is the planned successor to the FEDEP and other similar documents: the IEEE, along with its support organism, the SISO, requires the review of its standards at least once every 5 years. FEDEP (released in 2003) was therefore due for update in 2007. From the start, those involved in this process wanted this guide to be applicable to any interoperability standard used, and not just to HLA, as in previous versions of the FEDEP (1999 and 2003). This proposal was accepted, hence the change of name for the new standard. The FEDEP based on HLA was part of the series IEEE 1516 (HLA); the DSEEP becomes the IEEE 1730 standard. The new edition of the document reflects this desire to be generic and neutral. For example, the terms “federation” and “federates” have been replaced by “simulation environment” and “members” (of a simulation environment), respectively. The DSEEP corrects certain imprecisions in the FEDEP (due to accumulated feedback from numerous experiments) and provides greater precision as to the contents and relationships of the various recommended documents. It also includes three appendices that cover the adaptation of the DSEEP to the IEEE’s HLA and DIS standards and to the current TENA standard. The emergence of distributed simulation has reinforced perception of the importance of systems engineering in general simulation activity. In the near future, the DSEEP constitutes a unique reference document for simulations engineering. The final version has been published in January 2011 [IEE 11]. As its name suggests, the DSEEP only seems to refer to distributed simulation. However, with a few modifications, it can profitably be used for monolithic simulations. Moreover, in the first quarter of 2009, the SISO launched a process of reflection for the development of engineering processes specific to monolithic 24 THALES.
Distributed Simulation
331
simulations. These two initiatives, added to current work on VV&A guides, contribute to the recognition of simulation as an industrial activity in its own right. 7.5. Conclusion: the state of the art: toward “substantive” interoperability Contrary to what we might believe from the emergence of new technologies or demonstrations of industrial competencies during conferences and professional exhibitions, progress in the field of distributed simulation is rather slow. Distributed simulation has benefited from advances in computer science and connectors in general, but it has more of a tendency to “endure” developments in distributed computing than to adapt to them. The constant progress in passing from DIS to HLA and from HLA to TENA is not essentially conceptual, but it applies more to “technical” interoperability than to interoperability of the “substantive” variety. Although the work of Andreas Tolk on interoperability [TOL 03] (briefly covered above) has allowed better definition of growing levels of interoperability and clarification of vocabulary, current publications show little or no significant progress toward the lofty heights of “semantic” interoperability and beyond. Ontology seems to be in fashion at the moment of writing, but significant applications have yet to appear. It is significant that there is still no recommended practice guide for the development of conceptual models of simulation applications, distributed or otherwise. A good deal of intellectual (research) work has certainly been done on these subjects, but we will only be able to speak of truly “mature” distributed simulation activities when higher levels of interoperability have been attained and when the relevant support tools and methodologies for their exploitation are present. 7.6. Bibliography [DAH 97] DAHMANN J.S., “High level architecture for simulation”, Proceedings of the 1st International Workshop on Distributed Interactive Simulation and Real-Time Applications, Eilat, Israel, pp. 9–14, January 1997. [DAH 98] DAHMANN J.S., MORSE K.L., “High level architecture for simulation: an update”, Proceedings of the 2nd International Workshop on Distributed Interactive Simulation and Real-Time Applications, London, pp. 32–40, July 1998. [DIS 94] The DIS vision, a map to the future of distributed simulation (version 1), written by DIS Steering Committee, IST-SP-94-0 I, May 1994. [IEE 93] IEEE 1278, Standard for Distributed Interactive Simulation - Application Protocols, 1993.
332
Simulation & Modeling of Systems of Systems
[IEE 96] IEEE Std 1278.3, IEEE Recommended Practice for Distributed Interactive Simulation – Exercise Management and Feedback, December 1996. [IEE 00a] IEEE Std 1516-2000™, IEEE Standard for Modeling and Simulation (M&S) High Level Architecture (HLA) - Framework and Rules, September 2000. [IEE 00b] IEEE Std 1516.1-2000™, IEEE Standard for Modeling and Simulation (M&S) High Level Architecture (HLA) - Federate Interface Specification, September 2000. [IEE 00c] IEEE Std 1516.2-2000™, IEEE Standard for Modeling and Simulation (M&S) High Level Architecture (HLA) - Object Model Template (OMT) Specification, September 2000. [IEE 03] IEEE Std 1516.3-2003™, IEEE Recommended Practice for High Level Architecture (HLA) Federation Development and Execution Process (FEDEP), April 2003. [IEE 07] IEEE Std 1516.4-2007™, IEEE Recommended Practice for Verification, Validation, and Accreditation of a Federation – An Overlay to the High Level Architecture Federation Development and Execution Process, 2007. [IEE 10a] IEEE Std 1516-2010TM, IEEE Standard for Modeling and Simulation (M&S) High Level Architecture (HLA) - Framework and Rules, August 2010. [IEE 10b] IEEE Std 1516.1-2010TM, IEEE Standard for Modeling and Simulation (M&S) High Level Architecture (HLA) - Federate Interface Specification, August 2010. [IEE 10c] IEEE Std 1516.2-2010TM, IEEE Standard for Modeling and Simulation (M&S) High Level Architecture (HLA) - Object Model Template (OMT) Specification, August 2010. [IEE 11] IEEE Std 1730TM-2010, IEEE Recommended Practice for Distributed Simulation Engineering and Execution Process (DSEEP), January 2011. [IGA 07] IGARZA J.-L., ADELANTADO M., Guide HLA, Délégation Générale pour l’Armement-Office National des Études et Recherches en Aéronautique, DGA reference S-CAT N° 1000X: Guide HLA (High Level Architecture), version 4.0, 2007. [LUZ 10] LUZEAUX D., “Methods and tools for systems of systems engineering”, Systems of Systems, ISTE, London, John Wiley & Sons, New York, 2010. [DOD 95] US DEPARTMENT OF DEFENSE, Modeling and Simulation Master Plan, DoD5000.59-P, Washington DC, October 1, 1995, https://handle.dtic.mil/100.2/ ADA301787. [NOR 98] NORTH ATLANTIC TREATY ORGANIZATION (NATO), Modeling and Simulation Master Plan, document AC/323 (SGMS)D/2, version 1.0., August 7, 1998, www.rta.nato.int. [TOL 03] TOLK A., MUGUIRA J., “The levels of conceptual interoperability models”, Proceedings of the 2003 Fall Simulation Interoperability Workshop, Orlando, Florida, 2003.
Chapter 8
The Battle Lab Concept
8.1. Introduction The fall of the Berlin Wall in 1989 and the disintegration of the Warsaw Pact marked the end of a relatively clear, essentially bipolar geostrategic situation for the armed forces. Since 1989, the identification of future potential enemies has been much less certain, as is the form these enemies and threats may take. At the same time, military operations have become more complex (with asymmetric conflicts, rapid and extensive media coverage of combats, use of multi-national forces, and the war on terror). Combat systems have become more difficult to specify, design, and implement, particularly in the context of network-centric warfare (NCW). These factors oblige decision makers in Western defense organizations to reconsider, in depth, the format and the form of their armed forces and their manner of conducting operations. This evolution is expressed specifically through a “transformation” approach taken by the NATO and a number of individual countries, including France, with the aim of optimizing and improving the effectiveness of military operations. Note that various civilian sectors are under the same brutal pressure to evolve in terms of operations and sizing as in military. Having faced with growing global criminal networks and the emergence of cybercrime, police forces also need to evolve, making use of information and communication technologies, working with an ever-increasing number of telecommunication service providers and legal specialists in the field of computing, multi-national cooperation, increased legal constraints, rapid media pressure, and so on. To give another example, the introduction of personal medical files (dossier médical personnel, DMP) in Chapter written by Pascal CANTOT.
334
Simulation & Modeling of Systems of Systems
France, instituted by a law passed on August 13, 2004, necessitates the involvement of a very large number of individuals from different fields (doctors, pharmacists, social security, hospitals, computer scientists, politicians, and, of course, patients). The Airbus A380 program, carried out by a multi-national group, is another good example; its success depends on considerable joint efforts of people of different interests (constructors, equipment/tool manufacturers, aviation companies, airports, civil aviation authorities, and so on). It is easy to think, after all, that if we managed to send men to the Moon or to the plan operation Overlord (dispatching 156,000 men and 20,000 vehicles to the beaches of Normandy on June 6, 1944, with aerial support, logistics, and reinforcements), then it should be easy to organize the rollout of a project such as the DMP or develop a means of tracking the Taliban. Theoretically, this is true, but in practice the available human resources, demands, and constraints involved (and consequently the complexity of the operation) are not the same today. Let us consider, for example, the human cost of these activities. The Second Battle of the Aisne, in World War I, resulted in 200,000 allied casualties and produced inconclusive results. The Operation Overlord was success, but it costs around 150,000 allied lives, something which was relatively well accepted at the time. The Vietnam War caused, in 12 years, the deaths of less than 60,000 US soldiers, which provoked considerable emotion in the country. The recent war in Iraq has triggered strong reactions among the populations of the US and its allies; the allied casualty level for this war was less than one-tenth of that in Vietnam. It is clear that considerations and scales have changed: operations must be carried out at much lower human cost, with mitigation of collateral damage, attention to the media, and respect of a number of international rules and conventions. Nowadays, then, we need means and methods to deal with complex capacitivetype problems, with high levels of constraints and multiple actors from different fields. This complexity is clearly visible in the increase in the cost of systems and the considerable increase in acquisition delays: for example, 15 years went between the beginning of construction on the first demonstration model of a Rafale aircraft and the delivery of the first mass-made airplane. The French Leclerc tank was developed in the period. In the civilian sphere, the first Airbus jumbo jet was launched in 2007, 12 years after the decision was made to build one (in 1995). Thankfully, means and methods have evolved to overcome these difficulties. Information and communications technologies in particular provide a range of tools to store, visualize and analyze information, share information over long distances, and to facilitate collaborative working. Simulation has also been enriched in terms of capacity for distribution, human behavior models, 3D visualization modules, analytical tools, and so on. These technological developments provide the bases for a new approach to capacitive problems.
The Battle Lab Concept
335
The battle lab concept first appeared in the United States in the early 1990s, rapidly spreading to allied powers. The first battle labs aimed to evaluate concepts and construct capacities, paying particular attention to technological advances very early in the program and doctrines of the US Army, thus maintaining the technological superiority of the American armed forces [WIL 96], notably in the domain of NCW, that is, warfare using the capacities of modern networks and information systems. Although most tools used in battle labs were not (at least at first) new, this fresh approach constituted a revolution in acquisition methods, with more global reflections, massive use of simulation for virtual evaluation of systems and systems of systems, and rationalization of the acquisition process. Battle labs rapidly became important elements in the process of transformation of the armed forces in a post-Cold War context. The NATO’s Concept Development & Experimentation (CD&E) approach is the archetype of this evolution. The CD&E consists of applying methods and structures taken from the experimental sciences to the development of military capacities (i.e. force systems). The CD&E process allows the development and evaluation of new concepts to refine and validate them before heavy investment in their on-the-ground implementation. Concepts are thus evaluated not only on a technical level (what equipment/tool should be used and/or developed?) but also on an operational level: what will the doctrine be? What organizational framework will be required? What formation will the troops involved take, and what training will they require? One particularly interesting characteristic of the CD&E approach is that it encourages the use of different sources of ideas: not only the usual think tanks of the general staff and various ministries (which have a broad, sometimes too broad, vision of the issue), but also industrial contractors (who are generally well placed to know what can be done, but are also there to make money), the armed forces in the field (who know how materials and doctrines are really used in practice, but often have a rather narrow vision of the issue), and academic researchers (who have significant capacities for imagining the future and high levels of creativity, but are often somewhat detached from present realities). The meeting of these different sensitivities and approaches, when it is well led and well structured, facilitates the production of viable, suitable, and innovative concepts, which may be evaluated at reduced cost within an experimental framework (using real on-the-ground experiments and/or virtual experiments through simulation). Battle labs, then, do not constitute a major leap forward in terms of technology, but represent a conceptual and organizational revolution, a new way of using current means in the best possible way to serve a capacitive approach. From their beginning as assemblies of means in an integrated process, battle labs have rapidly evolved towards a pooling of resources between different organizations, a board-based approach to working and partnerships between industry and defense organizations. The French and British battle labs (LTO and Niteworks, respectively) provide
336
Simulation & Modeling of Systems of Systems
a particularly good illustration of this approach, and we shall present them in detail in the following sections. 8.2. France: Laboratoire Technico-Opérationnel (LTO) 8.2.1. Historical overview The French Ministry of Defense began considering collective working in simulation for the specification and development of force systems (then systems of systems) very early on. The first draft of a project for a distributed simulation system for study purposes was written in 1994. This project became a reality in 1998 with the RICOS preliminary studies program. The basic principle of RICOS was to use a network and its services to create capacities for distributed simulation between the Air Force, Navy, Army and DGA simulation and study centers, thus creating the possibility of shared and collaborative work between engineers and operators within pluridisciplinary teams. In the event, the project did not go further than the prototype stage, but it may be considered as a primitive battle lab, which laid down certain bases and technical limitations of the concept. At the beginning of the new millennium, several new elements were added to the reflection within the French Ministry of Defense: − the growing capacities of information and communications technology, enabling systems for collaborative working between geographically widespread teams; − a certain “maturing” of the bases of distributed simulation (the high-level architecture (HLA) standard); − architectural convergence actions within entities and even nations (as seen in the ARCOSIM-ITCS project, which led to the definition of a common technical infrastructure for simulation for acquisition within the French Ministry of Defense); − increasing complexity of systems and the switch to capacitive logic; − the generalization of NCW, which created (de facto) numerous systems of systems with associated engineering problems (e.g. in the SCORPION terrestrial combat system); − the development of the battle lab concept, essentially in the US; − NATO’s development of the CD&E process; and − the desire within the Ministry of Defense to integrate different actors in the acquisition process (operators, state engineers, industrialists, and so on) within a board framework.
The Battle Lab Concept
337
To create a coherent battle lab approach in France and to federate efforts, the laboratoire technico-opérationnel (LTO) project was launched. Its first client and promoter was the Bulle Opérationnelle Aéroterrestre (BOA, Aeroterrestrial Operational Bubble), an ambitious project to create a system-of-systems demonstrator for NCW, preparing the ground for the SCORPION acquisition program, similar to the American Future Combat System (FCS) program and the British Future Rapid Effect System (FRES). Co-managed by the armed forces general staff and the Direction Générale de l’Armement (DGA), the LTO was inaugurated in October 2006 at the Fort of Montrouge. 8.2.2. Aims of the LTO The LTO aims to fulfill three main objectives in improving current approaches: − promote cooperation between operators and engineers (multi-disciplinary group work); − promote the practice of systems engineering; − allow the development of scenarios for use by actors via simulations and experiments. Cooperation between operators and engineers is first established, at strategic level, by co-management of the LTO. This is a fundamental expectation of the LTO, as not only is it unthinkable today to run complex programs without good communications between those involved, but there is also a strong expectation from the armed forces of increased transparency in the way projects are run (i.e. earlier involvement of the client in making choices, both architectural and contractual) and an increased need for trust (which implies the creation of a structure where all involved can express themselves freely and without ambiguity in spite of inevitable differences in background). The current version of the LTO is run by the DGA (which orders the work) and the army general staff (the client). Moreover, there is a strong degree of willingness to share this tool with the industrial sector, although the potential for involvement of project owners in running the LTO is a sensitive point, particularly due to legal constraints on the public market. This particular point is the subject of current reflection, with attempts to find a legal structure to allow greater industrial involvement in orientation. This cooperation is also stimulated by the systematic creation of pluridisciplinary and multi-cultural teams for the LTO projects, bringing together industrial engineers, the DGA and operators. Cohesion is ensured by a shared working environment, with tools and methods which promote and channel creativity during scientific brainstorming sessions. Role-playing games and simulations are used to develop scenarios.
338
Simulation & Modeling of Systems of Systems
The practice of systems engineering is reinforced by the use of specialized methods and tools for modeling systems, organizations, and processes (such as MEGA, AGATE1, and KIMONO2). These tools also allow demands to be traced both through and outside of the LTO cycle; models may be reused during the development phase, if the decision to construct the capacity is taken. The urbanization of systems is also considered with the use of shared technical bases (shared technical simulation infrastructure, [KAM 02], common information and communications system base, and so on). Finally, the placement of participants in the scenario is the subject of particular attention, with use of terrain simulations and experiments to illustrate new concepts and compare architectures. This placement of actors in the scenario is important, as it allows those involved in the LTO process (who, we must remember, generally do not have the same professional background) to develop a clear and common view of objectives and the means of implementing the capacity. All participants in this process may also be involved earlier in the process, crossing the usual cultural barriers between decision makers, users, backers, and project managers. The use of the Virtual Battle Space (VBS2) simulation in PHOENIX 2008 is typical of this approach: during the development of the experimental scenario, the fact of playing out the course of the scenario, as in a film, in front of the different participants in the process gave a level of comprehension and interactivity incommensurable with that which would have been provided using a textual tool. During the experiment, VBS2 was able to provide a video stream, based on virtual 3D images, to the regulatory information system. During debriefing, the simulation tool was used once again to produce animations, illustrating the implementation of concepts during the experiment in a clear and immersive fashion. 8.2.3. Principles of the LTO The main aim of the LTO is to support system of systems, capacitive (force systems), and inter-weapons and Joint (approach promoted by the last White Paper [COM 08]) studies within the Ministry of Defense by providing methods for the resolution of complex problems and services (as opposed to methods) for concept evaluation, based on the use of local or distributed resources. The LTO is not, then, a means to an end, but rather a methodology with associated services. It constitutes an approach to defense system engineering, using modern collaborative working environments, simulation, and experiments. 1 AGATE (atelier de gestion de l’architecture technique) is a software workshop dedicated to the development of different views of an information system. 2 KIMONO (kit de modélisation et de nouveaux outils) is a project run by the French Ministry of Defense with the aim of providing system architects and engineers with a tool to enable them to master systems-of-systems engineering.
The Battle Lab Concept
339
Figure 8.1. Capacity development process using the LTO
The general principle of the LTO approach when constructing a capacity or a system of systems is illustrated in Figure 8.1, and the detailed sequence of events involved in a study is shown in Figure 8.2: − A new capacity or system-of-systems concept is initiated by the general staff in very general terms (notably in terms of capacitive aims). At this level, LTO brainstorming sessions may be used to develop the concept. − Meetings within the group work laboratory allow the concept to be refined and analyzed. Participants also evaluate the potential and limits of existing systems, construct scenarios for use, and make architectural proposals. Emphasis is placed on cooperation between industrial contractors, operators, and DGA engineers (in other words, all stakeholders: project managers, clients, and final users) to guarantee that the constraints, background, aims, and abilities of each are considered. Creativity is stimulated using tools and methods, which lead to production of a certain number of possible architectures (often between 20 and 30), sometimes very varied, some “traditional”, others “innovative”, if the first analysis establishes that the concept is feasible and viable. These architectures will be analyzed during later sessions and (for example) 4 or 5 of the most relevant possibilities will be retained. Thus, we have an architectural analysis and design process for systems of systems based essentially on channeled and stimulated reflection within a multi-disciplinary team.
340
Simulation & Modeling of Systems of Systems
− The architectures that pass the concept analysis stage will then be subjected to experimental verdicts. However, as these experimental processes are generally complicated, long and expensive to implement, the first experiments will be carried out virtually using simulations. At the end of this phase, only one or two architectures (or none, if none is deemed suitable) will be retained. − Once attention has been focused onto one or two architectures and the necessary scenarios and metrics have been defined (Figure 8.2), these aspects may be subjected to field testing using experiments. The design process for these experiments may itself follow part of the LTO process, using brainstorming sessions within the work group to decide on objectives, experimental scenarios, and the organization of the experiments. − Once the concept has been evaluated experimentally, a decision is taken. If the concept is validated and will be taken further, we enter the construction and qualification phase, in which the LTO may again be involved in the specification of component systems of the capacity, the design of capacitive qualification tests, and so on, but the global process is no longer directly supported by the LTO: the “traditional” cycle of armament programs and operations is then used, as set out in instruction ([MIN 04]). − Throughout the concept treatment process carried out by the LTO, effective knowledge management methods are applied to allow reuse of the reflections and results produced during later stages or in different projects.
Figure 8.2. Typical structure of an LTO study
The Battle Lab Concept
341
8.2.4. Services of the LTO The LTO constitutes a group of services, rather than tools: it acts as a host structure for studies and experiments rather than as a simple technical resource. The work group laboratory structure permits directed brainstorming. For this, a leader is chosen, along with (typically) eight participants. The idea generation phase usually produces around 120 relevant ideas in an hour. Interactive grouping is carried out by thematic groups (producing around 120 ideas in 2 hours). Themes are classified by interest following several more detailed and reformulated lines of reflection to reach a consensus in less than 4 hours, allowing the process to be completed within one working day if the methodology is followed correctly. Table-based or role-playing games allow a team to gain a clear understanding of a scenario; by critical analysis of the way this process unfolds, it is possible to evaluate and compare scenarios, methodologies, processes, and doctrines of use following the defined metrics. Role-playing and table-based games constitute an intermediate step between the workgroup and the simulation stages. Architectural modeling services are based essentially on a set of methodology and tool, mostly commercial-off-the-shelf (COTS), which offers the following functionalities: − analysis of need and formalization of requirements; − modeling of functional and organic architectures; − modeling of usage scenarios; − traceability of needs, requirements, architectures, and scenarios; − simulation of usage scenarios; − analysis of systems; − configuration management; − analysis of impact of modifications; and − production of documentation. Moreover, the teams involved in the LTO are trained in complex systems and systems-of-systems engineering; they can, therefore, provide lots of advice and assist in the implementation of the corresponding processes in other entities. Simulations and “serious games” (where the principles of video games are used for professional applications – see Chapter 1) also have an important role to play in the LTO. They allow us to illustrate concepts and carry out “virtual experiments” to
342
Simulation & Modeling of Systems of Systems
compare architectures at low cost. The LTO uses essentially technico-operational type simulations, both specialist, made-to-measure applications, and generic offthe-shelf products (e.g. Bohemia Interactive’s VBS2 or VR Forces, by MäK Technologies). The LTO also has a technical base for simulations and experiments. In addition to the simulation and systems engineering tools already mentioned, the LTO uses the infrastructure technique commune de la simulation (ICTS), the DGA’s shared technical simulation infrastructure (Chapter 6), the DirectSim simulation environment developed by the Centre d’analyse de la defense (CAD), the DGA’s Defense Analysis Center and the French Navy, and the EXAC C3R experimental network, allowing links between national and international participants while respecting security and performance conditions compatible with the requirements of experiments. EXAC provides telecommunications services for the interoperation of constructive or piloted simulations, real platforms, and operational information systems during experiments. The LTO also possesses an information system, which enables collaborative working practices and the capitalization of knowledge. Finally, the LTO has various technical resources for experimental processes, based at their Arcueil site, such as a videoconferencing system and image walls. Using its teams, the ITCS and the EXAC network, the LTO is also able to make use of the resources of the DGA’s technical centers, such as SISPEO (a piloted armored vehicle simulator used in ergonomic studies), IBEOs (illustrations of operational requirements, developed by the Technical Center for Naval Systems in Toulon to facilitate the specification of naval platforms), or operational tools used in experiments, for example, the live simulation resources of the French Army’s CENTAC (combat training center) or the CENZUB (training center for actions in urban environments). Industrial tools may also be used (such as emulators of tactical data links, integration platforms, battle labs, and so on). 8.2.5. Experimental process The organization of experiments follows a process similar to the one shown in Figure 8.4. This process, taken from [GIL 08], represents a fairly traditional approach and bears some similarity to others seen in previous chapters of this book. The first phase consists of defining the objectives of the experiment, that is, the questions being posed. A typical question would follow the format “if I do X, will I obtain Y?” where “X” is an organizational hypothesis, new equipment/tool, and so on and “Y” is a capacitive result. For example, we might find the following question: “If I use the new KY27 radar instead of the KY26, will I be able to detect
The Battle Lab Concept
343
a ballistic missile of SCUD-B type sufficiently early?” Experimental constraints must also be identified very early on to know what may reasonably be done. The second phase is the experimental specification phase. This specification contains several specific aspects, including the following: − An operational scenario, containing the context and the cases of use, is to be considered. In the case of the LTO, this takes the form of a usage framework (operational mission and context) and “scenario miniatures”, a sort of operational sketch showing the framework of use, generally too large to be tested in its entirety without exceeding time and cost limits. − Metrics are essential in enabling evaluation of the results of the experiment, for example, average delay between confirmation of detection of a missile and its impact on a target, proportion of missiles not detected more than 2 minutes before impact, or the workload of the radar operator. − A model of the real system or system of systems to ensure clear knowledge and understanding of the system. − A specification of the experimental equipment/tool: hardware, simulation platforms, tools, operators, and operational actors. This second phase is particularly crucial and also highly sensitive, as it requires dialogue between individuals from different professional backgrounds (engineers, operators, and so on). A certain number of tools are used by the LTO to assist in this phase: creativity workshops, the use of role-play scenarios, and 3D visualization of the “scenario miniatures” mentioned above. The next phase involves designing the experiment. During this phase, the experimental dispositive and the detailed experimental scenario are finalized, including details on materials and operators, interfaces between actors and flows, and the timing of events. This design must, of course, consider the results of previous experiments, results capitalized at the end of experimental processes for this reason. If the experiment is not based entirely on pre-existing materials and software, a development phase may be required for the creation of new systems or interfaces, or for the adaptation of existing resources. This activity is external to the LTO, which is involved in the process only in specifying systems and not in development activity, which is left mostly to industrial actors. The fifth phase involves the installation of the experiment: set-up and configuration of the experimental dispositive (operational materials, simulations, and so on) with assignment of operators, verification of the dispositive (as, during
344
Simulation & Modeling of Systems of Systems
the operational phase, it would be out of the question to spend time “debugging” technical and organizational aspects to the detriment of operational objectives) and possible training of “players” in the use of new equipment/tool or a new interface. The sixth phase involves the execution of the experiment itself, with the collection of data to establish metrics and a first evaluation of results. The course of the experiment is also analyzed to enrich the RETEX (retour d’experimentation, or feedback), which will be capitalized within the LTO’s information system. The final phase is the synthesis phase, involving the creation of an experimental report and the presentation of results to the client who ordered the experiments. Results are also shown to participants, who need to be motivated and thanked for their work by the presentation of the fruits of their labors. The experimental report is, of course, published (if the information it contains is not classified) and capitalized within the LTO’s information systems. Like any engineering process, the LTO’s experimental process includes frequent reviews to ensure that each step is finished and validated before moving on to the next phase, allowing each new phase of the process to start from solid foundations.
Figure 8.3. MEGA modeling of the time sensitive targeting experimental process
The following list shows the main experiments carried out by the LTO in the period 2006–2009 and gives a general idea of the diversity of subjects studied.
The Battle Lab Concept
345
− Time Sensitive Targeting (2006): treatment of high-value targets revealed in short periods of time, which must therefore be dealt with very rapidly (Figure 8.3); − XP-RIENCE 06 (2006): aero-naval tactical data links; − XP-RIENCE 07 (2007): tactical inter-army situation monitoring; − PHOENIX 2007 (2007): integration of robots and image analysis to assist in infantry combat (the 2008 version of PHOENIX will be presented in more detail); − Blue Force Tracking (2008): situation monitoring for allied units (specifically to avoid damage from friendly fire); − Time Sensitive Targeting 2 (2009): extension of TST at inter-army level; − DAMB (2009): initial capacity for ballistic anti-missile defenses; − SA2R (2009): surveillance, goal attainment, reconnaissance and information transmission, sensor maneuvers in Joint situations; − Short loop tank/helicopter (2009): cooperative land–air engagements using armored vehicles and helicopters; − XP-RIENCE 09 (2009): collective tactical situation monitoring for land, air, and sea forces; and − ARTIST (2009): cooperation between armored vehicles and unmanned aerial vehicles in NCW (a collaborative experiment by France and Germany). 8.2.6. Presentation of an experiment: PHOENIX 2008 The PHOENIX 2008 experiment centered on the management of fire power in the course of terrestrial maneuvers. It aimed, in particular, to evaluate two new tools that would be made available to unit commanders: − An execution support cell (known as the Manoeuvre Management Cell (cellule de conduite de la manœuvre, CMM)), to provide services for the unit commander (captain) in assessing the terrain, preparing movements (friendly and enemy itineraries), non-line-of-sight firing (missiles and guns), and preparation of indirect fire. − An information support cell (Specialized Surveillance Cell (cellule de surveillance spécialisée), CSS), the need for which became apparent at the end of PHOENIX 2007, to facilitate information processing and to assist the unit commander by the centralization of images collected by both onboard and independent sensors (the Integrated Equipment and Communication Infantryman (Fantassin à Équipement et Liaisons Intégrés), FELIN) system, UAVs, and so on), and the selection of relevant information in these images. The CSS would then assist in decision making by providing the commander with the necessary elements.
346
Simulation & Modeling of Systems of Systems
P1. Experimental objectives - Definition of objectives and experimental constraints - Identification of experimental phases
Review of experimental objectives P2. Experimental specification - Definition of experimental scenarios - Definition of metrics - Modeling of real system - Specification of experimental dispositive Review of experimental specification P3. Experiment design - Definition of architecture of experimental dispositive - Definition of experimental scenario
Review of experiment design P4. Adaptations and development Activity external to the LTO (industrial)
P5. Installation of the experiment - Configuration of the experimental dispositive - Verification of the experimental dispositive - Player training Review of experimental definition P6. Experiment - Execution of the experiment - Initial debriefing
P7. Synthesis of experiment - Synthesis of experiment - Presentation of results - Capitalization Review of synthesis of experiment
Figure 8.4. Experimental process of the LTO charter based on [GIL 08]
The Battle Lab Concept
347
PHOENIX 2008 also allowed the illustration of new capacities and optimizations of existing capacities: − non-line-of-sight firing and indirect fire support, with a new concept of missiles fired by artillery units; − sorting sensor images to assist the captain in decision-making; and − coordination of collective actions (firing and mobility). Finally, demonstration models and prototypes of hardware and software (sensors, tactical communications networks, simulations, and so on) were tested by industrial partners within the experimental framework. The different actors in the experimental process (industrial partners, the DGA, and the operators) were motivated to provide support by the potential benefits of the results of this process. Thus, the French Army made its camp at Mourmelon available for the project, with a staff of 120 and 10 vehicles; in return, the Army obtained promising leads and information on potential improvements to operational capacities and doctrine. The DGA provided players and leaders alongside technical and methodological support, particularly during the preparation phase and in evaluation of the experimental process. These preliminary and post-experimental steps should not be neglected: the experiment itself only lasted 12 days, but required 8 months of preparation and 4 months of analysis after the event, in which the LTO was heavily involved. The group work laboratory was used in identifying the technical and doctrinal aims of the experiment, alongside scenarios bringing together these objectives and the associated metrics (saved/correlated parameters, personnel and “observed” networks). This process was greatly assisted by the use of highly realistic 3D terrain models of the Mourmelon camp and simulations developed using an off-theshelf simulation suite, VBS2, a professional product derived from video games technology (and therefore known as a serious game (Figure 8.5). These models and simulations were used to give those involved a clear idea of the environment and ambiance in which the experiment would take place. The exchange flows between actors were modeled using the MEGA engineering tool (Figure 8.3). On the ground, the experiment was based on Army regiments, with technical support from the DGA and several industrial participants. These industrial participants used the opportunity to test equipment/tool prototypes: each participant, therefore, made significant gains from the process, in correspondence with the philosophy of the LTO. A number of ergonomists also participated in evaluating the work of operators. A specialized tool, MEEFISTO, can be used to capture and analyze of network exchanges: it is therefore possible to measure the effectiveness
348
Simulation & Modeling of Systems of Systems
of every action of every operator very precisely, supplying precious information on the impact of new methods of organization, workload, and so on. Real-time video streams were sent to the regimental information system from an aerial vehicle (UAV) then from a simulation (VBS2).
Figure 8.5. 3D rendering of a scenario miniature using VBS2 (image by J. Martinet)
Results are used “off-the-bat”, during daily debriefings, then several weeks later after deeper analysis of the data collected. This feedback process also makes use of the technical resources of the LTO. Simulations and videos are used to provide graphic replay or illustration of actions which took place during the experiment, furthering common understanding of the events of the experiment, without ambiguity, for all participants. The information extracted is particularly useful for all those who contributed to the process: − several programs will benefit from evolutions of existing material, the need for which is identified during the experiment; − on-the-ground evaluation of prototypes by industrial partners; − identification of specific needs for evolution in land army doctrines (such as new distribution of functions). − study of the way the experiment was carried out, including management of experimental logistics, methodology, and the integration of simulations, can be capitalized and considered during future LTO activities.
The Battle Lab Concept
349
As for the main subject of the Phoenix 2008 project, the two new command assistance cells, the experiment showed that it would be difficult to manage two cells (the CSS and CMM) as demands in terms of resources would be too high, for coordination in particular. The functions themselves were validated, but require optimization by fusing the two cells to create a single entity. 8.2.7. Evaluation and future perspectives of the LTO The LTO has been very active since its inauguration and has clearly demonstrated its capacities. The results produced by the work group laboratory (the laboratoire de travail en groupe, LTG) have proven particularly convincing, providing rapid responses to a wide range of questions with reduced financial outlay, thanks to the methodical use of pluridisciplinary teams of experts. Numerous groups, both at industry and government level, have integrated the LTO approach into their working methods and a certain maturity has been reached with the installation of the CD&E approach. Nonetheless, there is still room for improvement on certain points, specifically connected to the fact that the LTO is state-run: although the Administration guarantees neutrality and the availability of methods and resources, this involvement creates certain hindrances. One example of this is the considerable preparation required (accompanied by significant delays) due to procuration within the framework of the Public Acquisition Rules (code des marchés publics), which regulates the attribution of contracts by public bodies. Another problem is the lack of flexibility in the domain of human resources, where the rapid adaptability required by different experiments in terms of dimensions and abilities poses certain difficulties. The LTO, however, is still in its infancy and will most probably undergo major evolutions in the near future, based on the results of previous studies and experiments. A greater level of industrial and civilian involvement is very likely when dealing with non-military issues. Note that certain industrial actors have begun to develop their own battle labs, each with a technical infrastructure (often derived from existing internal tools). Examples of this include the following: − Thales’ BTC, with the Thales Integration Centre; − EADS’ System Design Centre (SDC) with NetCOS; − NFCC/Solaris at DCNS; − CS’ Joint Battlelab; − Dassault Aviation’s Atelier d’Emploi;
350
Simulation & Modeling of Systems of Systems
− in the UK, MBDA Bristol’s System Integration Facility, a complement to the Simulation Center outside Paris. These tools are distinct and lack synergy. An important direction for efforts would be to make these resources converge, or at least interoperate, to produce a true partnership logic between clients, owners, and project managers. 8.3. United Kingdom: the Niteworks project Niteworks (Network Integration Test and Experimentation Works) is a battle lab belonging to the British Ministry of Defence (the MoD). At the outset, in 2003, the aim was to create a structure to evaluate the capacities provided to the armed forces by information and communication technologies (Network Enabled Capability, NEC). Niteworks rapidly evolved to become a decision-assistance tool with far wider coverage, used in planning and acquisition of systems, systems of systems, and capacities. Niteworks operates as a partnership between the MoD and the industry, and, like the LTO, involves collaboration between operators from the armed forces, experts, and industrialists. However, although the global aims and basic principles of Niteworks and the LTO are the same, there are significant differences between the two. First, Niteworks is somewhat older, with the benefits of age and considerable financial backing: the current contract, signed at the end of 2007 for a duration of 5 years, is worth £43 million in addition to the £47 million received for the previous contract, signed in 2003. Niteworks was designed and is managed as a partnership between the MoD and the industrialists, whereas, for the time being at least, the LTO is managed exclusively by the French Ministry of Defense. This partnership was set up with the aim of sharing information and in a spirit of openness, something of a cultural revolution, even for the UK; this necessitated the resolution of a number of intellectual property issues. Intellectual property is now shared, at a level dependent on the status of the company within the Niteworks structure. Several classes of actor can be identified: − the MoD holds the copyright on everything produced by Niteworks; − “partners” (a group of 10 large industrialists) who have the right to use the information produced by Niteworks (foreground information) freely for the benefit of the British government, and also have access to the information used by Niteworks in the course of its projects (background information), with certain exceptions; − “associates” (several dozen companies, including a number of smaller businesses) with restricted access rights to information produced by activities in which they participated.
The Battle Lab Concept
351
In the first five years of its existence, that is, the period covered by the first contract and including the creation and “breaking in” of the system, the financial benefit linked to the use of Niteworks has been estimated at £450 million, giving a return on investment of nine to one. To improve the yield still further in the second contract period, this contract only aims to partially finance the battle lab; the industrial partners involved will therefore need to find other clients (essentially from within the MoD) to generate profit. Niteworks is therefore an apparent success, a success which is linked to a certain number of innovative aspects in its approach, including the partnership between government and industry. This means of operation is used from the very beginning of the systems acquisition cycle, allowing clear agreement between owners and project managers on choices, simplification of contracting issues, improved visibility, and reduction of risks. This may have a detrimental effect on competition, but the opening of the Niteworks partnership allows third party businesses to slot into the structure with ease. The simplification in intellectual property rights is also a factor, which promotes efficiency, avoiding long and delicate negotiations at the beginning of each new project. 8.4. Conclusion and perspectives In this chapter, we could have provided a detailed overview of a number of different battle lab structures, in France and elsewhere, but this would not have been particularly valuable in furthering our understanding of the subject. Although there are differences in the tools used, in management and in the processes involved (the selection of themes, the way the theme is dealt with, capitalization of results, and so on), a certain number of aspects are invariably found in all structures of this type: − integrated structure for collaborative working involving different actors (clients, users, experts, and industrial contractors) in all or part of the system acquisition cycle; − contextualization of actors to facilitate reflection; − development of capacitive concepts, then evaluation of these concepts through experiments (on the ground and/or through simulation). The battle lab concept is still in its infancy and has not yet reached its full potential. Nevertheless, the development of battle labs, both by governments and industrial project managers, and their durability shows that they are particularly a useful tool. Their capacity to deal with complex capacitive issues efficiently, effectively, and (often) at reduced cost makes battle labs an ideal tool for systemof-systems engineering. The availability of increasingly sophisticated generic
352
Simulation & Modeling of Systems of Systems
off-the-shelf simulations, with ever-increasing interoperability capacities and openness, provides especially encouraging perspectives for the instrumentation of battle labs by simulation. Methods for collaborative working and the capitalization of knowledge have been available for some time. Everything leads us to believe, then, that the battle lab concept will continue to spread, particularly in the civilian sector, where capacitive and system-of-systems issues also exist (as we would do well to remember). Nevertheless, we should not lose sight of the fact that the creation and management of a battle lab requires a rigorous and methodical approach, with the capacity to establish strong dialogue between those involved, who must “play along” for the concept to work without risk of failure. In this respect, Niteworks’ partnership logic would be a good example to follow. 8.5. Bibliography [ALB 02] ALBERTS D.S., HAYES R.E., Code of Best Practice for Experimentation, CCRP, Washington D.C., 2002. [ALB 05] ALBERTS D.S., HAYES R.E., Campaigns of Experimentation: Pathways to Innovation and Transformation, CCRP, Washington D.C., 2002. [CAN 09] CANTOT P., Simulation-Based Acquisition: Systems of Systems Engineering, Cours, Master of Systems Engineering, ISAE, Toulouse, 2009. [COM 08] COMMISSION SUR LE LIVRE BLANC (presided by J.-C. MALLET), Livre blanc sur la défense et la sécurité nationale, Odile Jacob-Documentation Française, Paris, 2008. [CON 06] CONAN E., Guide pour la mise en pratique du concept CD&E sur les projets pilotés par le CICDE, Centre Interarmées de Concepts, de Doctrines et d’Expérimentations, Paris, 2006. [DOD 97] DEPARTMENT OF DEFENSE, DoD Modeling and Simulation Glossary, Under Secretary of Defense for Acquisition and Technology, Washington D.C., 1997. [GIL 08] GILBERT E., Etude Complémentaire 2 du LTO V0.1, Thales Communications, Massy, 2008. [JON 08] JONES L., Niteworks: Establishing a UK Strategic Asset, RUSI Defence Systems, London, 2008. [KAM 02] KAM L., LECINQ X., LUZEAUX D., CANTOT P., “ITCS: The technical M&S infrastructure for supporting the simulation-based acquisition process”, NATO Modeling and Simulation group Conference, Paris, 2002. [LEC 06] LECINQ X., Le Laboratoire technico-opérationnel, DGA/SAIS, Arcueil, 2006. [LUZ 10] LUZEAUX D., RUAULT J.-R., Systems of Systems, ISTE, London, John Wiley & Sons, New York, 2010.
The Battle Lab Concept
353
[MIN 04] MINISTERE DE LA DEFENSE, Instruction générale n1514 du 7 mars 1988 sur le déroulement des opérations d’armement, 4th edition, 17th September 2004, Paris, 2004. [MIN 99] MINISTERE DE LA DEFENSE, Instruction n 800/DEF/EMA/PEE – 60800/DGA/DPM du 9 février 1994 sur la conduite des programmes d’armement, 2nd edition, 1st September 1999, Paris, 1999. [MOK 05] MOKHTARI, M., BERNIER, F., COUTURE, M., DUSSAULT, G., LALANCETTE, C., LAM, S., LIZOTTE, M., “Modelling and simulation and capability engineering process”, RTOMP-MSG-0035 meeting proceedings, NATO/RTA, Neuilly, 2005. [NOR 00] NORTH ATLANTIC TREATY ORGANISATION, Report to the Military Committee on Concept Development and Experimentation as an Alliance Tool, NATO/SACLANT, Norfolk, 2000. [NOR 98] NORTH ATLANTIC TREATY ORGANISATION, NATO Modeling and Simulation Master Plan version 1.0., NATO AC/323 (SGMS) D/2, NATO/RTA, Neuilly, 1998. [TEC 06] THE TECHNICAL COOPERATION PROGRAM, Guide for Understanding and Implementing Defense Experimentation (GUIDEx), Canadian Forces Experimentation Centre, Ottawa, 2006. [WIL 96] WILSON J.R., “Battle Labs: what are they, where are they going, Acquisition Quarterly Review, Winter 1996, pp. 63-74, Washington D.C., 1996.
Chapter 9
Conclusion: What Return on Investment Can We Expect from Simulation?
9.1. Returns on simulation for acquisition Throughout the previous chapters, we have seen how simulation is used as much in the production of models as in their use for a given objective. Simulation allows us to reproduce characteristics of the environment, systems, and certain behaviors. We have gone into detail on techniques, specificities in terms of formats and standards, and the limitations and pitfalls of interpretation. In addition to this descriptive aspect, simulation allows us to control conditions and situations and thus to test solutions to given problems. This allows a certain level of flexibility, security, and cost reductions not possible using real experiments (e.g. 30 years ago, approximately a hundred real launches were required to test a missile; today, the number of real launches required is considerably smaller). Simulation thus provides precious help in planning equipment, doctrines of use and operation, as well as in training. Beyond the capacity to model and to give life to these models, which sometimes requires considerable processing power (i.e. non-negligible levels of investment), simulation has become the key to success in the engineering process, a bridging tool for working in integrated multi-disciplinary teams on problems of integration of complex systems. Different actors are involved in defining, evaluating, producing, and supporting the project, and they must share information and data produced during tests and the simulations necessary for their own evaluation, validation, Chapter written by Dominique LUZEAUX.
356
Simulation & Modeling of Systems of Systems
qualification, and preparation activities. To do this, they must also identify the information necessary in terms of tests and simulation as early in the process as possible. This is, moreover, the way simulation-based acquisition (SBA) was initially defined by the US Department of Defense: a complete acquisition process placed at the service of project owners and managers, with robust and collaborative use of simulation technologies in a coherent and integrated manner throughout the acquisition phases of programs. The basic idea is to show that simulation is the essential tool for mastering the complexity of systems throughout their life cycle. This logic consists of using and reusing simulation tools and the available technologies on the entire operational breakdown of the system, during project phases and between various projects (particularly for systems of systems). This is paired with global management of modeling and simulation resources throughout the acquisition process: from simple occasional (in terms of time and space) support to program engineering, we may obtain a coherent and integrated process. We can therefore put simulation into perspective: simulation is used in the acquisition of systems as parts of systems of systems, with a capacitive vision, not simply restricted to replacing components one by one. The virtual aspect is widely exploited, so certain architectural or technological choices can be left as late as possible in the project. To fully reap the benefits of using simulation in this way, a few principles should be considered: – apply a descending approach to global design; – verify conformity to specifications via an ascending approach focused on tests and integration issues; – identify key milestones and plan in advance for the whole life cycle; – use integrated multi-disciplinary teams with a user-oriented approach; – manage the evolution of system configuration and roll out technico-economic analysis on all compromises; – use modeling and simulation as a computer-assisted tool for all choices, accompanied by parametric analyses. The approach as a whole is characterized by iterative adjustment of virtual prototypes immersed in realistic synthetic environments, which on the one hand helps to develop a shared vision of the emerging system and on the other hand provides appropriate means for a better understanding of complex interactions between elements of the system configuration. By having design, construction, and test engineers work together, the prototypes created will be more easily constructed and evaluated, leading to reductions in the global acquisition cost.
Conclusion: ROI from Simulation
357
These gains may be quantified, and we shall discuss this fact in the following sections. Through the use of simulation in system acquisition, we can reduce a priori promises of investment based on decisions made (the Pareto curve is often used in this respect: it shows that after 20% of effective spending, decisions will be made which commit 80% of total investment). This results from the fact that simulation allows simultaneous management of a large number of technical alternatives, for which portions of the life cycle of varying lengths can be played out virtually, and the impact of an early decision can be measured later on in terms of performance and/or cost. From this point of view, simulation essentially contributes to the management of the project’s risk portfolio. The other factor for economic gain lies in the reuse of simulations between systems within a meta-system; this reuse should not be limited to simple software “bricks”, but it should cover requirements, architectures, design patterns, interface models, test plans, data, documentation, and so on. Various cost studies over the last few years have allowed us to appreciate the true value of this factor. We are thus able to show [LUZ 03] that well-managed simulation for acquisition is financially worthwhile from the moment the first incident occurs in the course of a project. Reuse between projects (when well managed) allows savings of a similar order of magnitude to the cost corresponding to the design of the system including individual systems. These first technico-economic analyses show strong reasons to invest, and the first figures produced by processes currently underway confirm the results of this theoretical analysis. 9.2. Economic analysis of gains from intelligent use of simulations The first way in which simulation creates savings during system acquisition stems from the greater freedom we have in terms of outlay which is “frozen” by decisions made during the cycle. Simulation allows a wider range of technical alternatives to be considered: for example, successive portions of the cycle may be analyzed for several options simultaneously, and the impact of an early decision can be measured later on in terms of performance and/or cost. This increased freedom should be compared with the added cost of developing and engineering the simulations themselves. To quantify this, we shall compare full project costs (from exploration of a concept to production) for a “standard” acquisition and an acquisition using an SBA process. We will then quantify the use of the increased degrees of freedom obtained through the use of an SBA process while considering the added acquisition costs linked to unpredicted events or planning errors.
358
Simulation & Modeling of Systems of Systems
Note that we have not included usage costs (launch and support), as these costs – particularly for defense systems, which are used over several decades – would skew the analysis; simulation creates savings in a clear and recognized manner, and usage costs form the largest part of global possession costs (the US Department of Defense estimates that the costs of acquisition, use, and support follow a ratio of 28-12-60). Savings from simulation use will therefore increase the longer the system is used. As the sections of the budget devoted to acquisition and use are often independent, we have chosen to maintain the separation between both these phases and concentrate on acquisition to demonstrate the interest of simulation use in this particular phase. 9.2.1. Additional costs of the SBA We shall use the usual curve1, which is familiar to all project managers. This represents cumulative acquisition costs over different phases of a cycle.
Figure 9.1. Cumulative acquisition cost (as a relative percentage of the global acquisition cost) over four phases (feasibility, definition, development, and production)
The values shown in Figure 9.1 are merely indicative and may vary somewhat from one source to another, but these differences are a matter of units (this is the case, e.g. in the program director guides published by the French and US defense ministries; these values also depend on the milestone used to delimit different phases). The part played by simulation in the acquisition process can vary greatly, as much in the feasibility phase as in development, depending on the use of different virtual prototypes and more generally on the degree to which a true SBA process is used. When a process of this type is applied, simulations involved in different 1 A Pareto type curve, with the 80-20 rule: 80% of financial outlay takes place in the last 20% of the project development.
Conclusion: ROI from Simulation
359
acquisitions may be reused as much as possible and possibly included in a knowledge management infrastructure, with validation information for reuse in appropriate contexts without need for redevelopment. In an ideal example of SBA, the cycle can be even run virtually following a spiral logic, leaving the creation of solid prototypes as late as possible. Figures 9.2 and 9.3 show the proportional contribution of simulation over different phases of the acquisition cycle: preparation, feasibility, definition, development, and construction. Figure 9.2 shows the data for an SBA process and is taken from the final report of the NMSG003 group, operating within the NATO/Research and Technology Organization (RTO) [NAT 03]. For Figure 9.3, we considered a standard process, and the data used is taken from normal practice.
Figure 9.2. Part played by simulation throughout the acquisition cycle (SBA process)
Figure 9.3. Part played by simulation in acquisition (standard process)
In both cases, simulation is used almost exclusively during the preparation phase (analysis of needs and concept exploration) and in design (feasibility and definition). This is due to the use of virtual prototypes (which should include construction constraints from the beginning, as in the automobile industry), on the
360
Simulation & Modeling of Systems of Systems
one hand, and to reuse on the other (models used during the preliminary and definition phases should be reused during development). Before continuing with our demonstration, we shall make three remarks and clarify some aspects of notation. REMARK 1.– The SBA requires added effort in the design phase (to deal with questions of reuse and to respect different standards and requirements imposed by capitalization repositories), in validation (to reduce costs of later reuse), and in integration (vertically: from components to sub-systems and systems and horizontally: throughout the cycle) of each phase. Table 9.1 [PRI 01] shows the ratios of different activities (design; implementation; test, verification, and validation (TV&V); project management; documentation; and configuration) during each phase of the traditional software life cycle (concept exploration and analysis of needs, general design, creation, including detailed design, coding and unit tests, integration and corresponding tests) as shown in software engineering literature. The value on the left corresponds to small independent projects, whereas the value on the right refers to complex projects involving interactions with operators and/or other programs, and it gives a good indication of what SBA requires in comparison with a “standard” simulation (used sporadically and independently). Activity/phase
Concept exploration
General design
Effort distribution by phase (as a percentage) Identification of needs General design Implementation Tests, verification, and validation Project management, documentation, configuration
6/7
Integration and tests
15/18
Creation (detailed design, coding, unit tests) 55/44
46/42 20/16 3/10 9/16
15/10 40/42 14/14 11/18
5/3 10/6 58/55 10/20
3/2 6/4 34/48 36/25
22/16
20/16
17/16
21/21
24/31
Table 9.1. Relative effort involved in activities throughout the cycle, taken from [PRI 01]
In what follows, to use this data in the context defined in previous figures, we shall assimilate: – concept exploration and identification of needs with feasibility;
Conclusion: ROI from Simulation
361
– general design with definition; – creation and implementation with development; – integration and TV&V with construction. To check the coherence of this data with the estimations given for simulation, we shall calculate the TV&V activity throughout the cycle dedicated to simulation in a SBA process. By referring to Figure 9.2 and considering an average value for each phase, we calculate the relative part (expressed as a percentage) played by simulation in the SBA process during the four phases of feasibility, definition, development, and construction is 100, 95, 75, and 60, respectively. To calculate the global value of TV&V, we must add, for each phase, the product of the proportional contribution of simulation, the relative effort for the phase, and the proportional contribution of this activity during each phase: 1 × 0.07 × 0.16 + 0.95 × 0.18 × 0.18 + 0.75 × 0.44 × 0.2 + 0.6 × 0.3 1 × 0.25, giving 15.2%. If we refer to [NAT 00], the cost of verification and validation is estimated at 15% of the global cost of the simulation, thus demonstrating the coherence of our data. REMARK 2.– The figures above show the effort required for simulation use during the acquisition process. We shall suppose this effort to be directly linked to cost: this is true in terms of human resources, but ignores the cost of raw materials. Consequently, our estimation for SBA will be relatively pessimistic, as we shall not explicitly consider the fact that SBA reduces the amount of hardware equipment prototyping in a process in favor of “virtual” prototypes. It could, however, be argued that the human cost of material prototyping is lower than that involved in the creation of virtual prototypes, due to the different levels of qualification required of personnel and current market conditions. A detailed study should analyze the exact balance between these various factors: materials including buying and initial treatment of raw materials, software including initial investment for high-performance calculation, qualification and market price of personnel concerned. From our own experience, SBA seems to be the best option for large-scale projects. However, as we do not have the necessary figures to prove this, we shall suppose that these factors are balanced in the following paragraphs; in other words, we ignore this question (if data was available, it could be included either in the cost coefficient, which we will discuss later, or in the γ coefficient which will also be introduced in later calculations). In the same way, the translation of cost in terms of human resources can be a detailed process, as different abilities are needed for different activities and the level of expertise of the team working on the problem. Detailed cost models, such as
362
Simulation & Modeling of Systems of Systems
those provided by Cocomo2.0 [BOE 00], cover these aspects. However, we shall ignore these aspects, and in what follows, the coefficient for the conversion of effort to cost shall be equal to 1. REMARK 3.– To keep our calculations simple, we shall suppose that all constant costs and percentages remain constant throughout a given phase (so the functions illustrated in the figures above can be approximated by simple functions using the mean value for each phase). We could consider more complex curves, but this would change very little apart from higher level corrections. We shall also assume that the cost of the simulation is independent of the phase in the cycle and depends exclusively on the percentage of simulation activity in this phase. This hypothesis ignores horizontal software integration costs (which may arise during different phases of the cycle) and all costs connected with validating implementations of the simulation. These two factors only contribute at a higher order of magnitude, as seen in the previous table. To take them into account would require more detailed analysis of the SBA process, including all efforts linked to the creation and management of a data repository, models and simulations, added verification and validation costs, and all savings generated by reuse. Only this last point will be considered in the next section, devoted to multi-project acquisition. Notation: – RelPartSimi,j is the proportional contribution of simulation to activity i during phase j; – RelPartSimj is the proportional contribution of simulation to phase j; – RelPartActi,j is the proportional contribution of activity i to phase j; – RelPartPhasej is the proportional contribution of phase j to the global acquisition of the system; – GlobAcqCost is the global acquisition cost of the system; – For all variables, the exponent std (or SBA) represents the context: “standard” acquisition (or SBA); – Coeffj and Coeffi,j are coefficients that convert relative efforts such as RelPartSimi,j and RelPartSimj to costs (see Remark 2). Let us now return to the main calculation: for a given phase, simulation is part of the process, and its cost is directly proportional to the part played by simulation during this phase. This gives us a first formula for obtaining the global cost of simulation in each acquisition process, by adding this contribution over each phase. To refine this model, we can give greater detail on what happens within each phase and calculate how simulation contributes to each activity within a phase. This gives
Conclusion: ROI from Simulation
363
us a second formula for global acquisition costs, obtained by adding all activities in a phase, then all the phases. Thus, global simulation costs are calculated in one of two ways, depending on whether we use the first or the second, more refined, model: GlobSimCost = ∑ phase j RelPartSim j × Coeff j × RelPartPhase j × GlobAcqCost [9.1] GlobSimCost = ∑ phase j
(∑
activity i
RelPartSim i , j × Coeff i , j × RelPartAct i , j
× RelPartPhase j × GlobAcqCost
)
[9.2]
Clearly, the added cost due to the SBA process is the difference GlobSimCostSBA – GlobSimCoststd. To calculate this, we must identify the different terms in the previous equations. The values of RelPartSimj are given in Figures 9.2 and 9.3 (remember that we are using the average value for each phase). The values of RelPartPhasej can be deduced from Figure 9.1 (note that the figure gives cumulative costs, integrating the costs of the previous phase(s) for each phase; it is easy, however, to obtain individual costs, and therefore, their average value, to simplify calculations as stated above). The values of RelPartActi,j are given in Table 9.1 for both acquisition processes. The values of RelPartSimi,j are deduced from the RelPartSimk values as follows: we carry out identification as explained in Remark 1, which tells us which k and l correspond to which i and j, and we take RelPartSimi,j = RelPartSimk × RelPartSiml. All Coeffj and CostCoeffi,j are presumed to be equal to 1, as stated in Remark 3. Finally, we use GlobAcqCostSBA = γ × GlobAcqCoststd, where γ is a coefficient taking various investments into account (see Remark 2), alongside global savings due to improvements in systems engineering following a successful SBA process2. As this factor could be seen as giving an unfair advantage to the SBA process and as we lack access to widely validated data, we shall assume that γ = 1. Equation [9.2] gives lower values than equation [9.1] as various non-simulation activities (such as project management, documentation, and configuration) are identified explicitly. If we look more closely at the values presented in Table 9.2, 2 [ELM 08] states that the adequate application of systems engineering allows cost reductions of 20–30% and productivity increases of almost 40%. Parametric studies [HON 01] also state that depending on the percentage accorded to systems engineering (typically closer to 15% than to 1%), risks of overrun are much better managed (typical overrun by a factor of 1 ± 0.1 instead of 1.4 ± 0.3). All these considerations seem to suggest that values of γ = 0.8 or 0.7 would be reasonable.
364
Simulation & Modeling of Systems of Systems
the greatest differences appear in the values corresponding to the construction phase. This is not surprising as this phase contributes a good deal to global acquisition costs, and the associated value is therefore a significant part of the global result. Equation [9.1] [9.2]
Feasibility Definition Development Construction GlobSimCost Added cost std 2.2 1.6 2.1 1.9 7.8 SBA 2.5 2.4 5.6 11.2 21.7 14 std 1.2 0.7 0.6 0.3 2.8 SBA 2 1.6 3.1 5.6 12.3 10
Table 9.2. Relative cost contributions to simulation during different phases for the two acquisition processes (values expressed as percentages of GlobAcqCost)
Using equation [9.1], the ratio between simulation cost for the “standard” process and the full acquisition cost is 8; for an SBA process, it is 22. Using equation [9.2], the ratio for the “standard” process is 3 or 12 for SBA. Depending on the model used, then the additional cost incurred by using SBA is estimated between 9% and 14% of the full acquisition cost.
9.2.2. Additional costs from unexpected events or bad planning Our reference curve (Figure 9.4) is the classic Pareto curve3, which gives the proportion of expenditure which is tied up (or “frozen”, i.e. their use is pre-defined) by decisions made throughout the acquisition process.
Figure 9.4. Evolution of decision-based spending over the acquisition process 3 Once again, this follows the 80-20 rule, but it is applied in a different way: after 20% of effective expenditure, decisions have been made which a priori tie up 80% of the total budget.
Conclusion: ROI from Simulation
365
Figure 9.5 should also be familiar to any project manager; it shows the cost derivative, in terms of cumulated additional expenses due to unplanned modifications. This is, in fact, a curve of the type shown in Figure 9.1 stacked on top of a curve of the type shown in Figure 9.6 from the moment of the unplanned event.
Figure 9.5. Cost derivative (higher gray area) added to decision-based spending
Figure 9.6. Cost of program reuse (y axis) depending on the proportion of the program requiring modification (x axis)
One consequence of the SBA is the reduction of risks by simultaneous study of several alternatives (and by indexing all intermediate results with adequate validation and traceability processes). Thus, final decisions may be taken later in the cycle, avoiding freezing expenses in the early phases. Consequently, the curve of expenses fixed a priori undergoes a vertical shift downwards. A fortiori, the cost derivative will also be reduced; an unplanned modification could be an alternative already studied – canceling the cost derivative – or may involve lower additional costs as it is not necessary to travel as far back in the cycle, meaning fewer elements need to be reworked.
366
Simulation & Modeling of Systems of Systems
If we compare both trajectories, one for a system acquired following the “standard” process and the other using SBA, three means of cost reduction become apparent: – “frozen” costs are a priori lower; – lower cost derivative; – reduced expenses following the cost derivative. The sum of these gains rapidly balances out all additional costs engendered by the SBA, as calculated above; with the qualitative tendencies already mentioned, this happens as soon as the development phase begins. Actually, this calculation is too pessimistic regarding SBA: the added costs calculated earlier applied to the whole acquisition process, while the figure we need should only apply to the period preceding the cost derivative. If we look at these calculations in more detail, we see that development and construction are the main sources of added costs (as they involve higher acquisition costs, and all our calculations are proportional to the global cost of acquisition, although we pay particular attention to proportionality coefficients). In other terms, the major contribution is that SBA already reduces costs by controlling the cost derivative! A rapid analysis then shows that the gains in terms of the cost derivative and lower immediate outlay always balance out the added cost of SBA when compared with a “standard” acquisition. To summarize, we have shown that the added costs of SBA are immediately balanced out and transformed into positive savings as soon as problems are encountered. This situation is analogous to what happens with total quality in other contexts. This is not surprising as SBA increases the amount of systems engineering and effort in general in the initial phases of a project (analysis of requirements, compromises between requirements and technical solutions, and so on), and it is a well-known fact that most errors and avoidable added costs in projects can be traced back to these early phases (e.g. data from software engineering tells us that errors are distributed between requirement definition, design, coding, and integration activities with a ratio of 56-27-7-10, and the distribution of rework costs follows a ratio of 82-13-1-4). If we refer to Table 9.2, the costs added by SBA in the first three phases (i.e. ignoring the construction phase and concentrating on systems engineering) may be estimated approximately 5% of the global acquisition cost, a figure which should convince the reader of the value of SBA in risk reduction!
Conclusion: ROI from Simulation
367
9.3. Multi-project acquisition Let us now consider the acquisition process for a system of systems. We could carry out the analysis described above for each system and cumulate the gains. Savings of this kind are bound to appear because the probability of an uncontrolled risk does not increase in a linear fashion, but more rapidly with several different acquisition teams working on systems which are not temporally synchronized and must be integrated eventually. We shall, in fact, propose a different economic analysis, with special focus on reuse. Reuse has been recognized over the last few years as a major key to cost reduction [EZR 99]. It is motivated by the search for standard solutions to recurring problems found in different contexts. Reuse is based on the availability of modules in standard source code, but it is not restricted to component level. It is also based on a pooling of needs and analogy of solution architectures. Consequently, requirements, architectures, design patterns, user interface models, test plans, data, and documentation must be shared. The immediate results of a correctly implemented reuse process may be measured in terms of: – productivity: less aspects of a simulation system need to be produced and so less effort is required, reducing development costs and improving management of time to delivery; – quality, which may be transferred from one project to another; – performance, with reduced costs and time delays and better anticipation of the clients’ needs. As simulations mainly consist of programs (including programs embedded in hybrid hardware-in-the-loop simulators), we shall use cost models for software reuse, like that of COCOMO2.0 (constructive cost model), initially introduced by Barry Boehm in 1981 and extended in 1987 and 1995; the latest versions, [BOE 95] and [BOE 00], include detailed analyses of commercial-off-the-shelf (COTS) use and software reuse. The following curve, based on an analysis carried out by Richard Selby in 1988 on around 3,000 software modules developed by NASA, gives the cost of reuse as a function of the proportion of the program requiring modification [BOE 95]. This curve shows the following: – savings of more than 50% when less than 10% of the program requires modification; – savings of 25–50% for modifications up to 75%.
368
Simulation & Modeling of Systems of Systems
During the acquisition of a complex system, simulations developed during various phases become more and more specific. Thus, simulations may be reused easily during the feasibility phase, but much less during the development phase, and in all probability not at all during production. Based on Figure 9.6, we can estimate that the cost savings generated by reuse are more than 50% during the feasibility phase and up to 25% during the definition and development phases. If we return to Table 9.2 and correct the costs associated with each phase using these figures, a global cost reduction for SBA can be estimated approximately 3–4% of the global acquisition cost (depending on the equation used). Referring to Figures 9.2 and 9.3, we note that simulation plays a major role in the feasibility phase and a non-negligible role in definition and the first parts of development, whatever acquisition mode is used (standard or SBA). When acquiring a system of systems, where several cycles overlap, reuse produces significant savings during the feasibility phase of recent systems and non-negligible savings during the definition phase. Savings in this area may be redirected to help with the integration of various systems (the feasibility and definition phases for the encompassing meta-system come for free), and furthermore, risk management is improved. To conclude, this qualitative analysis4 shows that reuse, as promulgated in the SBA processes of several systems (on condition that they share a certain coherence and may potentially be combined within a system of systems), may be reinvested in the design of the meta-system, which brings together and organizes all these systems. By modifying the acquisition processes in this way, and by working at system of systems level rather than considering a disconnected group of systems, it is possible to reason at capacitive level (how to acquire a capacity which allows the production of a particular effect via clever use of available systems), within more or less consistent cost limits.
9.4. An (almost) definitive conclusion: conditions for success From what we have seen above, the key information to share involves requirements and specifications, on the one hand, and integration tests and 4 We could also carry out an analytical analysis, giving, for example, the equational formulae for the various curves: thus, the Pareto type curve in Figure 9.1 may be approximated to the function x7 and that in Figure 9.6 by the function x1/7 (it is interesting to note that we thus obtain power laws; see Chapter 5). The interest of these formulae is that they are easier to handle when considering several systems in parallel, and we can even have fun increasing the number of systems considered to look for results at the limit.
Conclusion: ROI from Simulation
369
validation methods and data, on the other hand. This information must be exhaustive and thoroughly traced concerning its origin and configuration. It is through these two aspects of traceability and configuration management that we may obtain a vision of the whole system throughout its life. To share this view, we must have access to various architectural and technological choices, hence mastery of systems engineering. If a more mathematical reading of the global process was required, we might say that, in this case, we arrive at a canonical representation of which the execution corresponds to possible modifications of the system, with the capacity at any given moment to reconstruct the current state of the said system. In addition to this information, which provides a set of discrete elements allowing us to reconstruct the system, it is useful to have a more continuous vision of the construction of each architectural element and level, hence the interest of a process involving tools, which gives immediate access to a global and behavioral vision of the system on condition that we have a capacity for vertical (from the meta-system to the component and vice versa, to provide an extreme example) and horizontal integration (ideally, over the whole life cycle, from first consideration of the system concept to its retirement from service). In this case, we are in fact defining the information system corresponding to the complex system in question, which would serve as a reference framework and memory and could be connected with other tools such as those linked to the way the project is run, its financial management, or even generic logistical assistance systems which might play a support role. This was shown in Chapter 8 with the battle lab approach, which is based on the following: – an engineering infrastructure, based on the coherent use of: - a tool for formalizing requirements, with traceability and configuration management functions for these requirements, along with connection to simulation capacities to explore and justify specification decisions, - a tool to assist in integration and validation tests, making an immediate link to requirements as handled by the previous tool, and once again, simulation capacities to actively contribute to validation, - a collaborative working environment, which may be distributed geographically, giving the various parties involved before and after each level access to information relating to this level, to facilitate returns and potentially accelerate progress between levels; in addition to this secure exchange function and management of information based on levels of confidentiality and user access rights, an environment of this kind becomes the key structure for dialog throughout the life of the system;
370
Simulation & Modeling of Systems of Systems
– a simulation infrastructure: - based on international standards, - offering the required level of interoperability between future or existing models, - providing the necessary service for designing global and often geographically distributed simulations (shared technical and methodological frameworks, libraries of models and tools, management of model configuration, support in the validation process). We thus gain access to methods and tools for model engineering, the aim of which is to provide conceptual frameworks independent of specific implementations and to design models and simulations in general based on expressed needs, the final generation of code relating to a particular framework being carried out automatically. Following the iterative process described above, the battle lab (which is itself a system of systems!) alternatively plays the role of catalyzer and source for technical coherence of simulation resources acquired in the course of different projects and for the harmonization of processes in the new context of complex systems engineering. Note, too, that the whole of the approach previously outlined should be accompanied by a normative approach, with the creation of adequate methodological frames of reference and determination, then appropriation, of the necessary standards, covering the entire chain of information being handled, including system and model description data, validation data for these various elements, data relating to the systems engineering system itself, and data (or metadata) concerning infrastructures. This vision is certainly ambitious, but facilitated by very recent developments in software and systems engineering and by normative efforts and the desires of a vast international community, bringing together industry, academic researchers, public and private establishments, for a coherent and integrated vision of the various current norms (UML2 for model description data, for system modifications, ISO10303-AP233 for the exchange of description data, MDA for the transformation of models and meta-models, ISO 12207 for the software engineering process, and ISO 15288 for the systems engineering process, see [LUZ 10]). It is at this cost, with the aid of these tools, methods, norms, and principles which guide and assist decision making, that the subtle art of systems architecture becomes a science which, if not accessible to others, will at least involve taking risks and considerable reductions in delays and costs.
Conclusion: ROI from Simulation
371
9.5. Bibliography [BER 50] VON BERTALANFFY L., Théorie générale des systèmes, Dunod, Paris (1993), 1950. [BLA 98] BLANCHARD B.S., System Engineering Management, Wiley, New York, 1998. [BOE 95] BOEHM B., CLARK B., HOROWITZ E., WESTLAND C., “Cost models for future software life cycle processes: COCOMO 2.0”, Annals of Software Engineering, Special Volume on Software Process and Product Measurement, J.D. ARTHUR and S.M. HENRY (Eds.), J.C. Baltzer AG, Science Publishers, Amsterdam, 1995. [BOE 00] BOEHM B., ABTS C., WINSOR BROWN A., CHULANI S., CLARK B., HOROWITZ E., MADACHY R., REIFER D., STEECE B., Software Cost Estimation with COCOMO 2.0, Prentice Hall, Upper Saddle River, 2000. [BRI 00] BRILL J.H., CHEVALLIER J., MERCHADOU J.-L., Manuel des meilleures pratiques pour le développement de systèmes spatiaux, Cépaduès-éditions, Toulouse, 2000. [DON 02] DONNADIEU G., KARSKY M., La systémique: penser et agir dans la complexité, éditions Liaison, Rueil-Malmaison, 2002. [ELM 08] ELM J.P., GOLDENSON D.R., EL EMAM K., DONATELL N., NELSA A., A survey of system engineering effectiveness – initial results (with detailed survey response data), Special Report CMU/SEI-2008-SR-034, Carnegie Mellon University, Pittsburgh, December 2008, http://www.sei.cmu.edu. [EZR 99] EZRAN M., MORISIO M., TULLY C., Réutilisation logicielle: guide pratique et retours d’expérience, Eyrolles, Paris, 1999. [HON 01] HONOUR E., MAR B., Value of systems engineering: SECOE Research Project Progress Report, 2001, http://www.secoe.org/0103. [KAM 02] KAM L., LECINQ X., LUZEAUX D., CANTOT P., “ITCS: the technical M&S infrastructure for supporting the simulation-based acquisition process”, NATO Modeling and Simulation Group Conference, Paris, 2002. [LEM 77] LE MOIGNE J.-L., La théorie du système général, PUF, Paris 1977. [LUZ 03] LUZEAUX D., “Cost efficiency of simulation for complex system acquisition”, SPIE Aerosense’03, Conference on Simulation, Orlando, USA, 2003. [LUZ 10] LUZEAUX D., RUAULT J.-R., Systems of Systems, ISTE, London, John Wiley & Sons, New York, 2010. [MCG 07] MCGIBBON T., FERENS D., VIENNEAU R.-L., A business case for software improvement (2007 update): measuring return on investment from software engineering and management, A Data Analysis Center for Software (DACS) State-of-the-art report, DACS Report number 347616, Contract Number SP0700-98-D-4000, September 2007. [MEI 98] MEINADIER J.-P., Ingénierie et intégration des systèmes, Hermès, Paris, 1998. [MEI 02] MEINADIER J.-P., Le métier d’intégration des systèmes, Hermès, Paris, 2002. [MOR 77] MORIN E., La méthode: la nature de la nature, Seuil, Paris, 1977.
372
Simulation & Modeling of Systems of Systems
[NAT 00] NATO Industrial Advisory Group, “Prefeasibility study on simulation based design and virtual prototyping”, NIAG-D(2000)9 AC/141(NG-6)D/25, September 2000. [NAT 03] NATO Modeling and Simulation Group, “Feasibility study on M&S technology in support of SBA”, RTO-TR-064 AC/323(NMSG-003)TP/06, February 2003. [NRC 02] NATIONAL RESEARCH COUNCIL, Modeling and simulation in manufacturing and defense systems acquisition, Washington DC, 2002. [PRI 01] PRINTZ J., DEH C., MESDON B., TREVES N., Coûts et durée des projets informatiques, Hermès, Paris, 2001. [YAT 99] YATCHINOVSKY A., L’approche systémique pour gérer l’incertitude et la complexité, ESF, Issy-les-Moulineaux, 1999.
Author Biographies
Pascal Cantot graduated from the Former Academy for Armament Systems Engineering, now a National Graduate Engineering Institute renamed ENSTA Bretagne (Ecole nationale supérieure des ingénieurs des études et techniques d’armement, ENSIETA) in 1992. He also holds a Master’s degree in computer systems from the National Graduate Engineering Institute for aeronautical systems engineering, now part of ISAE (Ecole nationale supérieure des ingénieurs des constructions aéronautiques, ENSICA). As an officer in the Direction Générale pour l’Armement (DGA), he has occupied various posts in the French Ministry of Defense; from 1999 to 2002 he was head of simulation in the complex systems engineering department of the DGA, from 2003 to 2005 head of the simulation coordination section of the French Airforce’s General Staff, and from 2005 to 2007 head of the information systems projects division of the DGA’s Technical Centre for Information Systems (Centre technique des systèmes d’information, CTSI). Since 2007, he has been responsible for systems and simulation engineering within the systems of systems center at the DGA and works in the Centre for Defence Analysis (Centre d’analyse de défense, CAD), alongside teams from the Ministry of Defense’s technico-operational laboratory (the French Ministry of Defence Battlelab (LTO)). He has participated in numerous projects on simulation, both in systems acquisition and engineering and in simulations for forces training and operational support. He is also a visiting professor at the ENSIETA, where he has taught modeling and simulation of systems since 1999, and at the National Graduate Engineering Institute for Aeronautical and Spacial Systems (Institut supérieur de l’aéronautique et de l’espace, ISAE). Jean-Louis Igarza joined the Antycip Simulation company as Chief Scientist in 2010. Until 2009, he was employed by the DGA and occupied the position of Chief Scientist at the CAD from 2003, when he returned from a 3-year term as technical director of the NATO modeling and simulation coordination office (MSCO, part
374
Simulation & Modeling of Systems of Systems
of NATO/RTA). Before this project, he was head of the simulation methods department at the CAD. He taught simulation at the Université de Versailles as part of the Master 2 “Strategic Systems Analysis (Analyse des Systèmes Stratégiques)”, from 1998 to 2008. He also taught statistics and probability at the National Graduate Engineering Institute for Advanced Technologies (ENSTA) and the School of Statistics National Graduate Engineering Institute for Aerospace Systems, now part of ISAE (ENSAE) for many years. He is a founding member of the direction committee of the European Training and Simulation Association (ESTA), an elected member of the executive committee (EXCOM) of the Simulation Interoperability Standards Organization (SISO) since May 2008, and a member of the NATO Modeling and Simulation Group (NMSG) of which he was president from 2006 to 2008.
Dominique Luzeaux graduated from the Ecole Polytechnique in 1987 and the Ecole Nationale Supérieure des Techniques Avancées in 1989 before completing a doctorate at the University of Paris XI (1991). He is employed by the DGA where he holds the rank of brigadier general. He has occupied various executive posts, including head of research and technology programs for the simulation and engineering of complex systems at the DGA and director of the technical center for information systems. He is currently the director of the division in charge of land systems. He obtained habilitation in 2001, and has overseen around 10 doctoral theses and published over 60 articles in international conferences and journals. He teaches robotics at the ENSTA and systems of systems engineering at ENSIETA and at the ISAE at Master’s level. He received the “Ingénieur général Chanson” prize in 2006 for his work in the domains of autonomous terrestrial military robotics, and he is the author, with Thierry Puig, of A la conquête du nanomonde: nanotechnologies et microsystèmes, published by Félin in March 2007. With Jean-René Ruault, he co-edited the books Systèmes de systèmes; concepts et illustrations pratiques and Ingénierie des systèmes de systèmes; méthodes normes et outils, published by Hermès-Lavoisier in 2008, and Systems of Systems, published by ISTE-Wiley in 2010. He has been the chairman of the French chapter of the International Council on Systems Engineering (INCOSE) since 2009. Roland Rabeau qualified as an engineer from the French Air Force Academy (Ecole de l’Air), (Salon de Provence) in 1980. After fulfilling several roles in the flying division of the French Air Force, he received an engineering diploma from the Ecole Nationale Supérieure de Techniques Avancées (Paris) in 1990, specializing in systems analysis and operational research, before working for the CAD at the DGA from 1998 to 2002, where he was head of division. He submitted a doctoral thesis on the validation of simulations in 2006 (University of Versailles-St Quentin). His current activities include provision of expertise and teaching in the domain of simulation and applied mathematics.
List of Authors
Pascal CANTOT DGA Arcueil France Jean-Louis IGARZA Anticyp Simulation France Dominique LUZEAUX DGA Bagneux France Roland RABEAU Self-employed France
Index
A, B, C accreditation, 102, 123, 292 actor, 276 architecture, 28, 29, 39, 83, 187, 293, 314, 317, 338 battlelab, 5 capitalization, 288 computability, 223 conceptual model, 114 cost, 7, 20, 92, 143, 361 credibility, 103, 125
D, E, F, G, H, I DIS, 53, 163, 184, 264, 297, 298, 301, 306, 312, 314, 319, 321, 324, 326, 331, 332 distributed simulation, 85, 100, 162, 303, 305, 325, 326 DMSO (Defense Modeling and Simulation Office), 275 engineering, 5, 54, 67, 76, 85, 287, 332, 341, 357 FEDEP (Federation Development and Execution Process), 112, 319, 327 federation, 85, 93, 94, 264, 318, 329
FOM (Federation Object Model), 308, 318 HLA (High Level Architecture), 163, 312, 332 human behavior, 193, 195, 202
M, N, O, P, R model, 30, 33, 63, 66, 67, 85, 87, 88, 112, 114, 115, 121, 129, 136, 143, 145, 183, 249 NATO, 185, 301 nonlinearity, 245 reuse, 86, 122, 273, 368
S, T, V SEDEP (Synthetic Environment Development and Exploitation Process), 94, 327 SEDRIS (Synthetic Environment Data Representation and Interchange Specification), 188, 325 services, 318 simulator, 9, 41, 43, 89 standard, 85, 105, 185, 192, 251, 287, 306, 310, 312 stochastic, 88, 173, 175
378
Simulation & Modeling of Systems of Systems
system of systems, 1, 4, 50, 368, 370 TENA (Test and Training Enabling Architecture), 321 time management, 51, 161, 308 validation, 28, 53, 81, 87, 95, 97, 102, 104, 112, 114, 118, 123, 132,
135, 136, 139, 143, 144, 145, 147, 152, 153, 154, 155, 156, 171, 292, 361 verification, 102, 104, 111, 114, 115, 121, 135, 136 visualization, 88, 304