Quantum Decision Theory and Complexity Modelling in Economics and Public Policy (New Economic Windows) [1st ed. 2023] 3031388321, 9783031388323

Table of contents :
Preface
Contents
Part I Quantum Decision Theory
1 A Brief Overview of the Quantum-Like Formalism in Social Science
Introduction
Quantum Versus Quantum-Like
Quantum Probabilistic Modelling of Decision Making: Is This Exotic?
What is the Main Advantage of Quantum Information Processing?
Classical Versus Quantum Probability
Classical (Bayesian) Versus Quantum (Generally non-Bayesian) Rationality and Social Lasing
Agreeing to Disagree
Classical Physics Formalism in Economics and Finance
Quantum-Like Formalism in Economics and Finance
Conclusion
References
2 Cooperative Functioning of Unconscious and Consciousness from Theory of Open Quantum Systems
Introduction
A Few Words About Quantum Formalism
Indirect Measurement Scheme: Apparatus with Meter Interacting with a System
More Technical Details
Indirect Measurements of Mental Observables: Unconscious as a System and Consciousness as a Measurement Apparatus
Contextuality
Concluding Remarks
References
3 Hilbert Space Modelling with Applications in Classical Optics, Human Cognition, and Game Theory
Introduction
A Brief Mathematical Detour
Complex Euclidian Space
Inner Products and Norms of Vectors
Direct Sums and Direct Products
Linear Operator Space
Examples of Hilbert Spaces
Operations on Hilbert Spaces
Bounded and Un-Bounded Operators in Hilbert Space
Hilbert Space in QM
Born’s Rule: A Small Note
Application of Hilbert Space in Probability Theory
Applications of Hilbert Space Representation Outside QM
Hilbert Space Representation of Classical Optics
Classical and Quantum Entanglements
Human Cognition and Decision Modelling
COM Approach (Patra and Ghose)
Discussion: Application of COM in Game Theory
References
4 Remodeling Leadership: Quantum Modeling of Wise Leadership
Introduction
Leadership
Quantum Basics
Classical and Quantum Ontology
Quantum Modeling: Probability and Subjectivity
Using Quantum-Like Modeling in Social Science
Non-optimal But Normal Behavior
Order Effects in Human Cognition
Conjunction and Disjunction Effects
Heisenberg-Robertson Inequalities
Contextuality and Randomness
Emergence of Concept Combinations Through Entanglement
Emergent Cognitive State
Modeling Wise Leader Interaction with Context
Social Interaction Dynamics
Implications for Leadership and Wisdom Research
Wise Leaders as Entangled Actors
Final Comments
Appendix 1
Appendix 2
References
5 Quantum Financial Entanglement: The Case of Strategic Default
Introduction
Cognitive Entanglement
Quantum Decision Theory
Strategic Default
Social Entanglement
Financial Entanglement
Discussion
Conclusions
References
6 Quantum-Like Contextual Utility Framework Application in Economic Theory and Wider Implications
Introduction
Epstien’s Framework
Modelling Background
Overview of Quantum-Like Modelling Set Up
Model Set-Up
Model
Partial Ambiguity Resolution: Dynamics and Hamiltonian Formulation
Ambiguity Aversion- Attraction and Diversity of Investors’ Opinion and Asset Pricing
Entropic Measures
POVM in Decision Models
Conclusion and Further Discussion
Objective or Subjective Probabilities?
Application of Quantum-Like Modelling in Wider Context of Complex Economy
Interpreting the Model- Real World Implications and Decision Making
References
Part II Complexity Modeling in Economics and Public Policy
7 Complexity Economics: Why Does Economics Need This Different Approach?
What Difference Does Complexity Economics Make?
Closing Thoughts
Questions and Answers
8 Policy and Program Design and Evaluation in Complex Situations
Introduction
Sense-Making in Today’s World
The Cynefin Framework
The Stacey Matrix
Integral (Meta) Theory (Ken Wilber)
Complex Adaptive Systems (CAS) and How They Add Value to Public Policy
Why Complex Adaptive Systems Thinking is Important to Public Policy
Value Added of CAS to Public Policy
Applications of CAS Public Policy
Economics
Power and Politics
Law
Health
Education
Sustainability and Complexity
Program Design in Complex Systems
Dealing with Complexity in Policy Design
Monitoring and Evaluation in Complex Situations
From New Public Management to Human Learning Systems
Conclusion
References
9 Market State Dynamics in Correlation Matrix Space
Introduction
Methodology
Data Description
Evolution of Cross-Correlation Structures
Wishart Orthogonal Ensembles
Power Map Technique
Pairwise (dis)similarity Measures and Multidimensional Scaling
Identifying States of a Financial Market
Sectorial Analysis
Trajectories in the Correlation Matrix Space
Comparison of COVID-19 Case with Other Crash and Normal Periods
Conclusions and Future Outlook
References
10 Interstate Migration and Spread of Covid-19 in Indian States
Introduction
Impact of Pandemic on Migrants
Data and Methodology
Data
Methodology of Network Construction
Visualization of Complex Network of Migration
Network Plots Using the Census-2011 Data
Network Plots Using Covid-19 Special Train Data
Possible Relationship Between Covid-19 Cases and Migrants
Relationship Between Migration and Spread to Covid-19
Conclusion and Policy Recommendations
References
11 Trade Intervention Under the Belt and Road Initiative with Asian Economies
Introduction
Data and Methodology
Gravity Model
Neural Network Model
Dataset
Results and Analysis
Conclusions
Appendix
References
12 Innovation Diffusion with Intergroup Suppression: A Complexity Perspective
Introduction
The Model
Model for 2 Groups
Two Group Example with Stable Positive Equilibrium
Empirical Data on Tablet Use
Conclusion
References
Epilogue: Nobel Prize in Physics for Complexity Studies and Weather Behavior—Implications for Social Sciences and Public Policy
References

Citation preview

New Economic Windows

Anirban Chakraborti Emmanuel Haven Sudip Patra Naresh Singh   Editors

Quantum Decision Theory and Complexity Modelling in Economics and Public Policy

Quantum Decision Theory and Complexity Modelling in Economics and Public Policy

New Economic Windows Series Editors Marisa Faggini, Department of Economics and Statistics/DISES, University of Salerno, Fisciano (SA), Italy Mauro Gallegati, DISES, Politecnica delle Marche University, Ancona, Italy Alan P. Kirman, EHESS, Aix-Marseille University, Marseille, France Thomas Lux, University of Kiel, Kiel, Germany Editorial Board Fortunato Tito Arecchi, Scientific Associate of Istituto Nazionale di Ottica (INO) del CNR, Emeritus of Physics, University of Firenze, Firenze, Italy Sergio Barile, Dipartimento di Management, University of Rome “La Sapienza”, Rome, Italy Bikas K. Chakrabarti, Saha Institute of Nuclear Physics, S. N. Bose National Centre for Basic Sciences, Indian Statistical Institute, Kolkata, India Arnab Chatterjee, TCS Innovation Labs, The Research and Innovation unit of Tata Consultancy Services, Gurgaon, India David Colander, Department of Economics, Middlebury College, Middlebury, USA Richard H. Day, Department of Economics, University of Southern California, Los Angeles, USA Steve Keen, School of Economics, History and Politics, Kingston University, London, UK Marji Lines, Università Luiss Guido Carli, Rome, Italy Alfredo Medio, Groupe de Recherche en Droit, Économie, Gestion (GREDEG), Institut Supérieur d’Économie et Management (ISEM), Université de Nice-Sophia Antipolis, Nice, France Paul Ormerod, Volterra Consulting, London, UK J. Barkley Rosser, James Madison University, Harrisonburg, USA Sorin Solomon, Racah Institute of Physics, The Hebrew University of Jerusalem, Jerusalem, Israel Kumaraswamy Velupillai, Department of Economics, The New School for Social Research, New York, USA Nicolas Vriend, School of Economics and Finance, Queen Mary University of London, London, UK

Anirban Chakraborti · Emmanuel Haven · Sudip Patra · Naresh Singh Editors

Quantum Decision Theory and Complexity Modelling in Economics and Public Policy

Editors Anirban Chakraborti BML Munjal University Gurugram, India Sudip Patra Jindal School of Government and Public Policy O.P. Jindal Global University Sonipat, India

Emmanuel Haven Faculty of Business Administration Memorial University St. John’s, NL, Canada Naresh Singh Jindal School of Government and Public Policy O.P. Jindal Global University Sonipat, India

ISSN 2039-411X ISSN 2039-4128 (electronic) New Economic Windows ISBN 978-3-031-38832-3 ISBN 978-3-031-38833-0 (eBook) https://doi.org/10.1007/978-3-031-38833-0 © The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 This work is subject to copyright. All rights are solely and exclusively licensed by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors, and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. This Springer imprint is published by the registered company Springer Nature Switzerland AG The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland

Preface

This volume contains essays that were mostly presented in the International Conference on Applications of Quantum Modeling and Complexity Theory to Economics and Public Policy held during February 19–20, 2020 at the O.P. Jindal Global University. The conference served as a platform for sharing interdisciplinary knowledge between the experts and young minds including Ph.D. scholars, post-doctoral fellows and graduate students. We must mention that a Centre for Complexity Economics, Applied Spirituality and Public Policy (CEASP) was envisaged during the conference, which finally came into existence in September 2020 amidst the global lockdown due to the Covid-19 pandemic. The CEASP, Jindal School of Government and Public Policy (JSGP), then went ahead and organized several panel discussions, as well as its first International Conference on ‘Artificial Intelligence in Complex Socio-Economic Systems and Public Policy’ during 20–22 January, 2021 in online mode. The editors are pleased that discussions at the February 2020 conference eventually led to so much activity in these interdisciplinary areas. The current volume builds upon the emerging fields of Econophysics, Complexity theory and Quantum like modelling in cognition and social sciences, and their plausible applications in economics and public policy. There can be deep linkages between the micro, meso and macro scales at which these paradigms operate. With the availability of big data sets, researchers are now able to address questions of an empirical nature and frame evidence-based and evidence-informed policies. The essays appearing in this volume include the contributions of distinguished experts and researchers and their co-authors from varied communities—economists, sociologists, mathematicians, physicists, statisticians and others. We have received critical thoughts from noted experts in the social and natural sciences to explore possible interconnections. The editors of this volume earnestly hope that the critical reviews presented in this volume would create further scholarly interests, but also interest among policy practitioners to explore possibilities for creating a new paradigm for comprehending pressing issues of deep uncertainty and emergence in social dynamics.

v

vi

Preface

A few papers have been included that were not presented at the meeting since the contributors could not attend due to unavoidable reasons. The contributions are organized into two parts. The first part comprises papers on ‘Quantum Modelling’. The papers appearing in the second part include studies in ‘Complexity Theory’. We are extremely grateful to all the local organizers and volunteers for their invaluable roles in organizing the meeting, and all the participants for making the conference a success. We acknowledge all the experts for their contributions to this volume. The editors are also grateful to Mauro Gallegati and the Editorial Board of the New Economic Windows series of the Springer-Verlag (Italy) for their continuing support in publishing the Proceedings in their esteemed series. The editors and organizers also acknowledge the administrative and financial support from O.P. Jindal Global University, Sonepat, India. Gurugram, India St. John’s, Canada Sonipat, India Sonipat, India June 2023

Anirban Chakraborti Emmanuel Haven Sudip Patra Naresh Singh

Contents

Part I 1

2

3

4

5

6

Quantum Decision Theory

A Brief Overview of the Quantum-Like Formalism in Social Science . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Andrei Khrennikov and Emmanuel Haven

3

Cooperative Functioning of Unconscious and Consciousness from Theory of Open Quantum Systems . . . . . . . . . . . . . . . . . . . . . . . . . Andrei Khrennikov

11

Hilbert Space Modelling with Applications in Classical Optics, Human Cognition, and Game Theory . . . . . . . . . . . . . . . . . . . . Partha Ghose and Sudip Patra

25

Remodeling Leadership: Quantum Modeling of Wise Leadership . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . David Rooney and Sudip Patra

43

Quantum Financial Entanglement: The Case of Strategic Default . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . David Orrell

85

Quantum-Like Contextual Utility Framework Application in Economic Theory and Wider Implications . . . . . . . . . . . . . . . . . . . . 103 Sudip Patra and Sivani Yeddanapudi

Part II

Complexity Modeling in Economics and Public Policy

7

Complexity Economics: Why Does Economics Need This Different Approach? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129 W. Brian Arthur

8

Policy and Program Design and Evaluation in Complex Situations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145 Naresh Singh vii

viii

9

Contents

Market State Dynamics in Correlation Matrix Space . . . . . . . . . . . . . 173 Hirdesh K. Pharasi, Suchetana Sadhukhan, Parisa Majari, Anirban Chakraborti, and Thomas H. Seligman

10 Interstate Migration and Spread of Covid-19 in Indian States . . . . . 195 Debajit Jha, Suhaas Neel, Hrishidev, and Anirban Chakraborti 11 Trade Intervention Under the Belt and Road Initiative with Asian Economies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217 Sunetra Ghatak and Sayantan Roy 12 Innovation Diffusion with Intergroup Suppression: A Complexity Perspective . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231 Syed Shariq Husain, Joseph Whitmeyer, and Anirban Chakraborti Epilogue: Nobel Prize in Physics for Complexity Studies and Weather Behavior—Implications for Social Sciences and Public Policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251

Part I

Quantum Decision Theory

Chapter 1

A Brief Overview of the Quantum-Like Formalism in Social Science Andrei Khrennikov and Emmanuel Haven

Abstract This paper provides for a general overview of some of the main areas of applications where we believe the formalism from quantum mechanics has been (and can be) fruitfully applied. Over the past decade, progress has been made on using elements of this formalism in decision making models but also in data retrieval approaches and even in areas such as finance.

Introduction Over the past century, interactions between the social sciences and exact sciences have increased in a manyfold way. Louis Bachelier (Bachelier 1900) in 1900 introduced the beginnings of stochastic calculus in finance! During the twentieth century, over the course of many years, economics and finance used ideas from physics. As an example, the eminent physicist and mathematician, John von Neumann (von Neumann and Morgenstern 2007), did not only make enormous contributions to mathematics and physics but also to economics. We can give many other examples. We have seen over the last decade, the development of a new movement which this time, very specifically, focusses on using ideas from the formalism of quantum mechanics to the social sciences. This area of research considers applications in an array of fields. For example, within the formalization of decision making in both psychology and economics, new strong results have appeared. For a most recent and strong result, we refer the interested reader to the use of so called non-atomic quantum instruments (Ozawa and Khrennikov 2021).

A. Khrennikov Department of Mathematics and International Center for Mathematical Modelling, Linnaeus University, Växjö, Sweden e-mail: [email protected] E. Haven (B) Faculty of Business Administration, Memorial University, St. John’s, Canada e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 A. Chakraborti et al. (eds.), Quantum Decision Theory and Complexity Modelling in Economics and Public Policy, New Economic Windows, https://doi.org/10.1007/978-3-031-38833-0_1

3

4

A. Khrennikov and E. Haven

Quantum Versus Quantum-Like It is important to stress that in the sequel below, we do not speak about genuine quantum systems. In particular, we are not interested in quantum physical processes in the human brain. The idea of ‘quantum-like’ refers first and foremost to using the formalism of quantum mechanics to areas outside of physics. The quantum-like approach does not claim that social science (or any other non-physics area of application) would be intrinsically quantum mechanical. Can we be more precise on what is meant with ‘quantum-like’? In the introduction of an edited book on quantum-like applications (Haven and Khrennikov 2017, p. v), we indicated that by using the quantum-like paradigm one “need not search for quantum physical processes which might lead to the appearance of quantum-like features in behavior. The quantum formalism is treated as an operational formalism describing outputs of possible measurements, including the self-measurements.” The number of fields where such definition is usable is vast. There is a plethora of essentially non-classical data from domains ranging from political science and finance to psychology and biology.

Quantum Probabilistic Modelling of Decision Making: Is This Exotic? The sceptic observer may well ask the natural question: besides the intellectual satisfaction of using a more advanced mathematical formalism to difficult problems in any of the cognate areas cited above, is there evidence that this ‘exotic’ approach delivers new powerful results? We believe we can be affirmative in our response to this question, when considering the recent results obtained in the formalization of decision making (see Ozawa and Khrennikov 2021 cited above). A well-known paradox in normative decision making, such as the Ellsberg paradox can be tackled via the use of a generalized probability rule, i.e. the quantum probability rule. The existence of this very paradox was already uncovered more than 40 years ago by Kahneman and Tversky (1979) The quantum-like approach, in essence, provides for a formal response to this paradox.

1 A Brief Overview of the Quantum-Like Formalism in Social Science

5

What is the Main Advantage of Quantum Information Processing? As we remarked in Haven and Khrennikov (2017, p. vi), collective social systems, are characterized by a deeper uncertainty. This uncertainty is not representable via classical probability. We do write Haven and Khrennikov (2017, p. vi) “such a deep uncertainty is represented by using two basic structures of the mathematical formalism of quantum theory, superposition and entanglement of states. A core principle underlying this deeper uncertainty refers to incompatible observables which can be found back in many real-world instances where decision making occurs.

Classical Versus Quantum Probability Classical probability is rooted in the axiomatics developed by Kolmogorov (1933). The logical underpinning underlying that probability structure is classical—or also Boolean. The probability of two disjoint events is simply the sum of the probabilities of each event. This is fundamentally different from the quantum probabilistic structure. We note the following salient points: . quantum probability works complex probability amplitudes . the probability measure emerges via the use of the Born rule (one needs to square the absolute value of the complex amplitude) . there is violation of additivity: there is an additional interference term. The presence of the latter term can either increase or decrease probability in comparison to classical probability . the law of total probability is violated.

Classical (Bayesian) Versus Quantum (Generally non-Bayesian) Rationality and Social Lasing In classical decision making, the rational behavior of agents is formalized with the aid of the Savage sure-thing principle. As an example, one imagines a decision maker who is unsure whether to acquire a property before a major election. As per Savage (1954, p. 21) “seeing that he would buy in either event, he decides that he should buy, even though he does not know which event will obtain.” As we have remarked in earlier work (see Khrennikov 2010; Haven and Khrennikov 2013a), the sure thing principle is a consequence of the law of total probability. Hence, violating that law will imply violating the sure thing principle. There are

6

A. Khrennikov and E. Haven

multiple instances of real-world decision-making which clearly steer away from satisfying that sure-thing principle. Another well-known example of the sure-thing principle violation is the so called sequential gambling experiment, where, as remarked by Busemeyer and Wang (2007, p. 92) under a Markovian regime, “the probability of gambling in the unknown case must be the average of the probabilities of gambling in the two known cases.” We note the unknown case indicates that a gambler is not informed of the outcome of the first gamble; whilst in the two known cases, one receives the information one has won or lost in the first gamble. On the face of multiple real world experiments, this required average simply does not materialize. It is precisely such observation which then calls in for the construction of unknown states being the superposition of lost and won states (see Busemeyer and Wang 2007). The question we can now ask is this: if a decision-maker exhibits violating the law of total probability but does conform to the quantum probability rule, what ‘type’ of rationality does he or she exhibit? As we remarked above, the incompatibility of observables is a source for the deeper uncertainty we are attempting to formalize here in a non-physics context. The Kolmogorovian axiomatic framework requires a common event algebra. However, within a quantum framework, such common event algebra need not at all exist. The information overload brough about by the existence of a plethora of internet information sources leads quite naturally to the inability of a decision maker to construct a joint algebra of all possible events. Within quantum rationality, there is no need to edify a single Boolean algebra to capture—all the information. We can extend the above thought. Let us denote the so called ‘information field’ as generated by the various internet information sources. As one of us remarked in Khrennikov (2020, p. 48): “For a social analogue of the physical laser the basic social energy source is the information field…” That information field needs to be modelled as a quantum field and the two key ingredients for such a field are the so called annihilation and creation operators, which respectively absorb information excitation and emit information excitation (Khrennikov 2020, p. 13]. For a more detailed discussion of the quantum field, within the setting of social lasing and stimulated amplification of social action (SASA) (see Khrennikova 2017 and Chap. 2 (pp. 23–) in Khrennikov 2020).

Agreeing to Disagree A key theorem in economic theory is the so called Aumann theorem. One of us published a paper in the Journal of Mathematical Economics (Khrennikov 2015, p. 90) where a central result was that “Aumann’s theorem for quantum(-like) agents is violated, since states and observables are represented by operators which are in general noncommutative.” The quantum like version negates the Aumann theorem—i.e. quantum agents can agree to disagree (even with common knowledge) and a central idea here is that in the quantum-like approach we can make the argument that, within decision

1 A Brief Overview of the Quantum-Like Formalism in Social Science

7

making, the processing of information happens via quantum logical rules. Rationality, here relates, as we already mentioned above, to quantum probability (see further in Khrennikov and Basieva 2014).

Classical Physics Formalism in Economics and Finance The input of classical mechanics (but not quantum mechanics) to economics and finance is not at all new. In fact, option pricing theory was already being formulated (albeit in an incorrect way) back in the beginning of the twentieth century by one of the doctoral students of the illustrious mathematician Henri Poincaré. His name was Louis Bachelier and he was the first person in history to use Brownian motion in social science. Later on, when Black and Scholes came to write their seminal paper on option pricing theory, they devised a PDE which was a backward Kolmogorov PDE. The forward version of that PDE physicists know well—it is the Fokker Planck PDE which describes a time dependent evolution of a probability density function. There are other examples where classical mechanics has made important inroads to finance and economics. See for instance Onella et al. (2003). The area of econophysics has provided for tools from physics to uncover hidden patterns in social science data.

Quantum-Like Formalism in Economics and Finance As already mentioned above, the basic mathematical set up in classical (statistical) mechanics differs quite substantially from that used in quantum physics. In quantum mechanics we use a vector space, often known under the name of Hilbert space. States and measurements are identically the same in classical physics. They are not the same in quantum mechanics. There are interesting avenues in both economics and finance, which can be researched further using elements of the quantum formalism. Some ideas have been shown in the monographs by Khrennikov (2010) and by Haven and Khrennikov (2013a, 2017), Haven et al. (2017), and in a recent special issue of the Journal of Mathematical Economics (Haven et al. 2018) and also in Khrennikova and Patra (2019). Here are some brief examples. There is a connection between the absence of arbitrage and the use of so-called open systems (i.e. the system is not isolated). Open systems have also been used to explain voting behavior in US elections (see Khrennikova 2014; Khrennikova et al. 2014). Another example consists in explicitly using potential functions as ‘devices’ which capture types of information in finance. Next to the real potential function, which can capture an aspect of public information there exists also another type of potential—the quantum potential. That potential is narrowly connected to so-called Fisher information. Groundwork on using this type of potential in finance was laid in the paper by Khrennikov (1999) where from the quantum potential, a pricing rule could be formed (see also Haven and

8

A. Khrennikov and E. Haven

Khrennikov 2013b). Finally, this approach has an ultimate goal: if time dependent price paths can be formalized which are sensitive to changes in specific aspects of public information, we will have found a major contributor to using successfully the quantum formalism in finance.

Conclusion The quantum mechanical formalism has shown to be of practical value in explaining deviations from decision making which does not follow rationality axiomatics like the sure-thing principle in the Savage expected utility model. The approach does not at all imply there are quantum mechanical processes occurring at a macroscopic level. Rather, the use of the formalism to social science problems in general, in effect, does bring in the use of a different mathematical structure, i.e. the Hilbert space.

References Bachelier, L.: Théorie de la spéculation. Annales de l’Ecole normale supérieure, 3rd series, 17, pp. 21–86 (1900). Trans. by A.J. Boness in The Random Character of Stock Market Prices, ed. P.H. Cootner. MIT Press, Cambridge, MA (1967) Busemeyer, J., Wang, Z.: Quantum information processing: explanation for interactions between inferences and decisions. In: Bruza, P.D., Lawless, W., van Rijsbergen, K., Sofge, D. (eds.) Quantum Interaction (Stanford University). Papers from the AAAI Spring Symposium. Technical Report SS-07-08, pp. 91–97 (2007) Haven, E., Khrennikov, A.: Quantum Social Science. Cambridge University Press (2013a) Haven, E., Khrennikov, A.: Quantum-like tunnelling and levels of arbitrage. Int. J. Theor. Phys. 53, 4083–4099 (2013b) Haven, E., Khrennikov, A.: The Palgrave Handbook of Quantum Models in Social Science. PalgraveMcMillan Publishers (2017) Haven, E., Khrennikov, A., Robinson, T.: Quantum Methods in Social Science: A First Course. World Scientific Publishers (2017) Haven, E., Khrennikov, A., Ma, C., Sozzo, S.: Quantum probability theory and its economic applications. Special Issue. J. Math. Econ. 78, 127 (2018) Kahneman, D., Tversky, A.: Prospect theory: an analysis of decision under risk. Econometrica 47(2), 263–291 (1979) Khrennikov, A.: Classical and quantum mechanics on information spaces with applications to cognitive, psychological, social and anomalous phenomena. Found. Phys. 29, 1065–1098 (1999) Khrennikov, A.: Ubiquitous Quantum Structure: from Psychology to Finance. Springer (2010) Khrennikov, A.: Quantum version of Aumann’s approach to common knowledge: sufficient conditions of impossibility to agree on disagree. J. Math. Econ. 69, 89–104 (2015) Khrennikova, P.: Order effect in a study on US voter’s preferences: quantum framework representation of the observables. Physica Scripta T163 (2014) Khrennikov, A.: Social Laser. J. Stanford Publishing (2020) Khrennikov, A., Basieva, I.: Possibility to agree on disagree from quantum information and decision making. J. Math. Psychol. 62–63, 1–15 (2014) Khrennikova, P.: Modeling behavior of decision makers with the aid of algebra of qubit creationannihilation operators. J. Math. Psychol. 78, 76–85 (2017)

1 A Brief Overview of the Quantum-Like Formalism in Social Science

9

Khrennikova, P., Patra, S.: Asset trading under non-classical ambiguity and heterogeneous beliefs. Physica A 521, 562–577 (2019) Khrennikova, P., Haven, E., Khrennikov, A.: An application of the theory of open quantum systems to model the dynamics of party governance in the US political system. Int. J. Theor. Phys. 53(4), 1346–1360 (2014) Kolmogorov, A.: Grundbegriffe der Wahrscheinlichkeitsrechnung. Springer (1933) Onnela, J.P., Chakraborti, A., Kaski, K., Kertész, J., Kanto, A.: Dynamics of market correlations: taxonomy and portfolio analysis. Phys. Rev. E 68, 056110 (2003) Ozawa, M., Khrennikov, A.: Modelling combination of question order effect, response replicability effect, and QQ-equality with quantum instruments. J. Math. Psychol. 100, 102491 (2021) Savage, L.: The Foundation of Statistics. Wiley (1954) von Neumann, J., Morgenstern, O.: Theory of Games and Economic Behavior. Princeton University Press (2007)

Chapter 2

Cooperative Functioning of Unconscious and Consciousness from Theory of Open Quantum Systems Andrei Khrennikov

Abstract We present the mathematical model of cooperative functioning of unconscious and consciousness. The model is based on the theory of open quantum systems. Unconscious and consciousness are treated as bio-information systems. The latter plays the role a measurement apparatus for the former. States of both systems are represented in Hilbert spaces. Consciousness performs measurements on the states which are generated in unconscious. This process of unconscious-conscious interaction is described by the scheme of indirect measurements. This scheme is widely used in quantum information theory and it leads to the theory of quantum instruments (Davis-Lewis-Ozawa). Our approach is known as quantum-like modeling. It should be sharply distinguished from modeling of genuine quantum physical processes in biosystems, in particular, in the brain. In the quantum-like framework, the brain is a black box processing information in the accordance with the laws of quantum theory. During the last 10–15 years this framework has been actively used in cognition, psychology, decision making, social and political sciences. The quantumlike scheme of unconscious-consciousness functioning has already been explored for sensation-perception modeling. Keywords Unconscious · Consciousness · Quantum-like models · Decision making · Indirect measurement scheme · Open quantum systems · Sensation · Perception

Introduction The quantum information revolution (also known as the second quantum revolution) has big impact not only to technology, but also to quantum foundations. This revolution led to elaboration of few information interpretations of quantum mechanics A. Khrennikov (B) International Center for Mathematical Modeling in Physics and Cognitive Sciences, Linnaeus University, Växjö 351 95, Sweden e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 A. Chakraborti et al. (eds.), Quantum Decision Theory and Complexity Modelling in Economics and Public Policy, New Economic Windows, https://doi.org/10.1007/978-3-031-38833-0_2

11

12

A. Khrennikov

(see, e.g., Fuchs (2002, 2011, 2012), Khrennikov (2002, 2005, 2009), Jaeger (2007), Plotnitsky (2009), Chiribella et al. (2012a, b), D’Ariano (2019) and some papers in proceedings’ volumes Khrennikov (2002a, 2004a, 2005), Adenier et al. (2006), D’Ariano et al. (2012)). By these interpretations, the physical properties of the carriers and processors of quantum information are not crucial. In particular, the role of the physical space is diminished essentially; the quantum information scenarios is written for the information space. Tremendous development of quantum information theory endowed with experimental and technological applications stimulated its use outside of physics, especially in cognition, psychology, decision making, social and political science, economics and finance see pioneer works (Khrennikov 1999, 2003, 2004b), monographs (Khrennikov 2004c, 2010; Busemeyer 2012; Haven and Khrennikov 2013; Haven et al. 2017; Asano 2015a) and some papers (Haven 2005; Khrennikov 2006, 2015, 2016; Busemeyer 2006; Choustova 2007, 2008, 2009; Pothos and Busemeyer 2009; Yukalov and Sornette 2009, 2017, 2014; Asano et al. 2011, 2012a, b, 2014, 2015b, 2017a, b; Dzhafarov and Kujala 2012; Bagarello and Oliveri 2013; Wang and Busemeyer 2013; Wang et al. 2014; Khrennikov and Basieva 2014a, b; Busemeyer et al. 2014; Khrennikov et al. 2014, 2018; Boyer-Kassem et al. 2015; Dzhafarov et al. 2015; Basieva and Khrennikov 2015; Basieva et al. 2017; Lawless 2019; Bagarello 2020; Ozawa and Khrennikov 2020; Yukalov 2020; Khrennikova 2014, 2016, 2017). Such modeling is known as quantum-like. Quantum-like systems (proteins, cells, body’s organs, including the brain, human beings, social, political, economic, and financial systems) represent and process information by operating with quantum information states. Such quantum-like, i.e., respecting the quantum laws, information processing has no direct connection with the genuine quantum physical processes in a biological organism. A biosystem is a black box performing quantum information processing. The ability to represent information in the quantum-like way was developed in the process of evolution of biosystems. It has no direct relation with system’s spatial and temporal scales, temperature and other parameters of this sort (cf. Penrose 1989; Umezawa 1993; Hameroff 1994; Vitiello 1995, 2001). In article Asano et al. (2015b), it was pointed out that the quantum representation of information is the distinguishing feature of all biosystems, from proteins, genomes, and cells to brains and ecological systems. One can speak about quantum information biology. What is the main feature of the quantum-like representation and processing? It is the possibility of processing unresolved uncertainties. Mathematically, they are encoded in states’ superpositions. What is the main advantage of such information processing (operating with superpositions)? It saves computational resources: biosystems working in the quantum-like regime need not resolve all uncertainties and determine outcomes of variables and their probability distributions at each step of state’s processing. The outcomes are determined in the process of measurement. Measurements are performed by measurement apparatuses. The quantum formalism is general calculus describing processing of superpositions and extraction of the variables’ values–measurements. The most powerful quantum measurement formalism is based on the theory of open quantum systems (Ingarden et al. 1997; Benatti

2 Cooperative Functioning of Unconscious and Consciousness from Theory …

13

and Floreanini 2010): a system . S interacting with the surrounding environment . E. In the particular application of this theory to the description of measurements, the role of environment is played by a measurement apparatus . M used to measure some observable . A on . S. One of the basic mathematical frameworks for modeling of this situation is the scheme of indirect measurements (Ozawa 1984). The outcomes of . A are represented as outcomes of apparatus’ pointer . M A . The process of measurement is described as interaction between the system . S and the apparatus. M. This interaction generates the dynamics of the state of the compound system . S + M. The state’s evolution is unitary; for pure states, it is described by the Schrödinger equation (for mixed states given by density operators, by the von Neumann equation). Finally, the probability distribution for pointer’s outcomes is extracted from the compound state with the trace-operation. For our model, it is important that in this measurement scheme the features of the system . S are not approachable directly, but only through apparatus’ pointer. We shall explore this in modeling of brain’s functioning. In this paper we consider the quantum-like model in that unconscious.UC and consciousness.C are represented as the system. S and the apparatus. M in the above indirect measurement scheme. Features of unconscious .UC are inapproachable directly. Consciousness .C performs measurements on the bio-information system .UC. The main difference from physics is that to make a decision .C should “read its pointer” by itself. So, .UC performs self-measurements. However, the latter does not change the formal mathematical scheme. This is the good place to remark that even in quantum physics the pointer-reading is the nontrivial step of the measurement process. However, typically it is ignored, even in the foundational discussions. We just remark that Wigner claimed that consciousness is really involved in quantum measurement’s finalization (Wigner’s viewpoint was not accepted by the majority of physicists). This paper is a concept paper. Our aim is to describe the very general scheme of cooperative functioning of unconscious .UC and consciousness .C. The scheme is simple and easily understandable. Its concrete application to quantum-like modeling of the sensation-perception process (von Helmholtz 1866) was presented in Khrennikov (2015). However, paper Khrennikov (2015) is mathematically complicated, it is based on the theory of quantum instruments (Davies 1976; Ozawa 1984, 1997). The indirect measurements and quantum instruments are closely related, but in this paper we shall not discuss this issue (see Basieva et al. (2021), Ozawa and Khrennikov (2021) where these quantum tools are jointly presented as simple as possible—to be used by psychologists and biologists).

A Few Words About Quantum Formalism In quantum theory, it is postulated that every quantum system . S corresponds to a complex Hilbert space .H; denote the scalar product of two vectors by the symbol .. Throughout the present paper, we assume .H is finite dimensional. States

14

A. Khrennikov

of the quantum system . S are represented by density operators acting in .H (positive semi-definite operators with unit trace). Denote this state space by the symbol .S(H). In quantum physics (especially quantum information theory), there are widely used notations invented by Dirac: a vector belonging to .H is symbolically denoted as .|ψ>; orthogonal projector on this vector is denoted as .|ψ> as .|ψ>. Any density operator .ρ of rank one is of the form .ρ = |ψ>. In this case,.|ψ> is called a pure state, so.|ψ> ∈ H, |||ψ>|| = = 1. Observables are represented by Hermitian operators in .H. These are just symbolic expressions of physical observables, say the position, momentum, or energy. Each Hermitian operator . A can be represented as .

A=

∑

x E A (x),

(2.1)

x

where .x labels the eigenvalues and . E A (x) is the spectral projection of the observable . A corresponding to the eigenvalue . x. The operator . A can be considered as the compact mathematical representation for probabilities of outcomes of the physical observable. These probabilities are given by the Born rule that states if an observable . A is measured in a state .ρ, then the probability distribution .Pr{A = x||ρ} of the outcome of the measurement is given by .

Pr{A = x||ρ} = Tr[E A (x)ρ] = Tr[E A (x)ρ E A (x)].

(2.2)

For a pure state .|ψ>, this leads to the relation .

Pr{A = x|||ψ>} = ||E A (x)|ψ>||2 ,

(2.3)

as .Tr[E A (x)ρ] = ||E A (x)|ψ>||2 .

Indirect Measurement Scheme: Apparatus with Meter Interacting with a System The scheme of indirect measurements represents the framework which was emphasized by Bohr, by him the outcomes of quantum measurements are created in the complex process of the interaction of a system . S with a measurement apparatus . M. The latter is combined of a complex physical device interacting with . S and a pointer showing the outcomes of measurements; for example, it can be the “spin up or spin down” arrow. The system . S by itself is not approachable by the observer who can see only the pointer of . M. Then the observer associates pointer’s outputs with the values of measured observable . A for the system . S.

2 Cooperative Functioning of Unconscious and Consciousness from Theory …

15

Can the outputs of the pointer be associated with the “intrinsic properties” of . S or not? This is one of the main questions of disturbing the quantum foundations during the last 100 years. The indirect measurement scheme can be represented as the block of following interrelated components: • the states of the systems . S and the apparatus . M; they are represented in complex Hilbert spaces .H and .K, respectively; • the unitary operator .U representing the interaction-dynamics for the compound system . S + M; • the meter observable . M A giving outputs of the pointer of the apparatus . M. In quantum physics, the operator .U representing the dynamics of interaction . S and . M is a linear unitary operator. All operations in quantum theory are linear and dynamics of an isolated system is unitary, i.e., preserving the scalar product. In cognitive applications, we proceed with linear and unitary operators just for simplicity, just to borrow the well-defined formalism and interpretation from physics. In principle, there are no reason to assume neither linearity nor unitarity. We remark that even in physics there were attempts to develop nonlinear quantum theory. In the indirect measurement scheme, it is assumed that the compound system . S + M is isolated. The dynamics of pure states of the compound system is described by the Schrödinger equation: i

.

d |Ψ>(t) = H |Ψ>(t), |Ψ>(0) = |Ψ>0 , dt

(2.4)

where . H is it Hamiltonian (generator of evolution) of . S + M. The state .|Ψ>(t) evolves as .|Ψ>(t) = U (t)|Ψ>0 , where .U (t) is the unitary operator represented as U (t) = e−it H .

.

Hamiltonian (evolution-generator) describing information interactions has the form .

H = HS ⊗ I + I ⊗ HM + HS,M ,

where . HS : H → H, HM : K → K are Hamiltonians of . S and . M, respectively, and HS,M ∈ H ⊗ K → H ⊗ K is Hamiltonian of interaction between systems . S and . M. The Schrödinger equation implies that evolution of the density operator . R(t) of the system . S + M is described by the von Neumann equation:

.

.

dR (t) = −i[H, R(t)], R(0) = R0 . dt

(2.5)

16

A. Khrennikov

However, the state . R(t) is too complex to be handled consistently: the apparatus includes many degrees of freedom. Suppose that we want to measure an observable on the system . S which is represented by Hemitian operator . A, acting in system’s state space .H. The indirect measurement model for measurement of the . A-observable was introduced by Ozawa in (1984) as a “(general) measuring process”; this is a quadruple (K, σ, U, M A )

.

consisting of a Hilbert space .K, a density operator .σ ∈ S(K), a unitary operator .U on the tensor product of the state spaces of . S and . M, .U : H ⊗ K → H ⊗ K, and a Hermitian operator . M A on .K. Here .K represents the states of the apparatus . M, .U describes the time-evolution of system . S + M, .σ describes the initial state of the apparatus . M before the start of measurement, and the Hermitian operator . M A is the meter observable of the apparatus . M (say the pointer of . M). This operator represents indirectly outcomes of an observable . A for the system . S. The probability distribution .Pr{A = x||ρ} in the system state .ρ ∈ S(H) is given by MA . Pr{A = x||ρ} = Tr[(I ⊗ E (x))U (ρ ⊗ σ )U ★ ], (2.6) where . E M A (x) is the spectral projection of . M A for the eigenvalue .x. We recall that operator . M A is Hermitian. In the finite dimensional case, it can be represented in the form: ∑ .MA = xk E M A (xk ), (2.7) k

where .(xk ) is the set of its eigenvalues and . E M A (xk ) is the projector on the subspace of eigenvectors corresponding to eigenvalue .xk . The change of the state .ρ of the system . S caused by the measurement for the outcome . A = x is represented with the aid of the map .I A (x) in the space of density operators defined as I A (x)ρ = Tr K [(I ⊗ E M A (x))U (ρ ⊗ σ )U ★ ],

.

(2.8)

where .Tr K is the partial trace over .K. The map .x I→ I A (x) is a quantum instrument. We remark that conversely any quantum instrument can be represented via the indirect measurement model (see Ozawa 1984).

More Technical Details Now, we consider some technicalities. For simplicity, they were not present in the previous section.

2 Cooperative Functioning of Unconscious and Consciousness from Theory …

17

The interaction between the system . S and the apparatus . M starts at time .t0 . The interaction turns off at time .t = t0 + Δt. The indirect measurement scheme is based on the following natural assumptions: • The system . S and the measurement apparatus . M do not interact each other before the instant of time .t0 nor after .t = t0 + Δt. • The compound system . S + M is isolated in the time interval .(t0 , t). The latter assumption is too strong. Of course, a macroscopic apparatus . M cannot be completely isolated, it interacts with surrounding material bodies and fields.1 Therefore by speaking about isolation it is more natural to speak not about the whole apparatus . M, but just its part interacting with . S. This situation is formalized as follows. The probe system . P is defined to be the minimal part of apparatus . M such that the compound system . S + P is isolated in the time interval .(t0 , t). The indirect measurement scheme presented in Sect. 2.3 is applied to the probe system . P, instead of the whole apparatus . M. The rest of the apparatus . M performs the pointer measurement on the probe . P. In particular, the unitary evolution operator −iΔt H .U describing the state-evolution of the system . S + P has the form .U (t) = e , where . H = HS + H P + HS P is Hamiltonian of . S + P with the terms . HS and . H P representing the internal dynamics in the subsystems . S and . P of the compound system and . HS P describing the interaction between the subsystems. Consideration of probe systems is especially useful in the following situation: the total apparatus . M is a macroscopic system that interacts (typically in parallel) with a family of systems . S j , j = 1, 2, . . . , m. Different probes of . M interact with the concrete systems, the probe . P j with the system . S j . And the compound system . S j + P j can be considered as an isolated system, even from interactions with . Si and . Pi , i / = j. The indirect measurement scheme is a part of the theory of open quantum systems (Ingarden et al. 1997). Instead of a measurement apparatus . M, we can consider the surrounding environment .E of the system . S (see Asano 2015a; Asano et al. 2011, 2012a, b, 2015b, 2017a, b; Khrennikov et al. 2018 for applications to psychology).

Indirect Measurements of Mental Observables: Unconscious as a System and Consciousness as a Measurement Apparatus The scheme of indirect measurements presented in the previous section was created in quantum physics and applied successfully to a plenty of important problems. Our aim is to adapt it to cognition. The main question is about cognitive analogues of the system . S and the measurement apparatus . M. We suggest to use the framework 1

In advanced experimenting in quantum foundations, experimenters put tremendous efforts to isolate their labs. For example, the crucial experiment on Bell’s inequality violation (Zeilinger’s group (Giustina et al. 2015)) was done in the basement of one of Vienna’s castles.

18

A. Khrennikov

developed in paper Khrennikov (2015) for quantum-like modeling of the Helmholtz sensation-perception theory (von Helmholtz 1866). This scheme can be extended to a general scheme of unconscious-conscious interaction in the process of decision making The measured system . S is a sensation (or generally any state of the unconscious mind). Consciousness as the measurement apparatus to the unconscious interacts with sensations to make the decisions (to generate outcomes of measurements). Consciousness, of course, is not concerned just with a single probe. It is a large environment with many probes interacting with the unconscious. There should be a rule of transformation from a sensation to a conscious decision. This is unitary transformation and measurement of the meter in the probe. The operational description neglects neuro-physiological and electrochemical structures of interaction. The unconscious is a black box that is mathematically described by the state of sensation space (= the unconscious state), so that the unconscious state probabilistically determines the decision (by interaction with consciousness), then the unconscious state is changed according to the previous unconscious state and the decision made. Thus, each probe is described by a quantum instrument. Instruments are probe dependent. For the question-measurements, the question . A is transferred into the unconscious, where it plays the role of a sensation (cf. von Helmholtz 1866), so to say a high mental level sensation. Then, interaction described by the unitary operator .U generates a new state of the compound system—the unconscious-conscious. And consciousness performs the final “pointer reading”, the measurement of the meter observable. Pointer reading can be treated as generation of a perception, a high mental level perception.

Contextuality The scheme of indirect measurements formalizes contextuality of consciousobservations. For the fixed state .ρ, obseravtion’s context is determined by triple .C A = (σ, U, M A ), where, as in the above consideration, .σ ∈ S(K) is the state of .C and .U is the unitary operator representing .ρ − σ interaction and . M A represents apparatus’ pointer. The same observable . A (say question or task) can be realized in various contexts corresponding to variety of states of .C and .UC − C interactions. The only constraint determining the class of contexts generated by .C is the probability distribution of . A, given by equality (2.6), i.e., two contexts for . A-observable, .C A and .C 'A are coupled by the equality: . Pr{ A

'

= x||ρ} = Tr[(I ⊗ E M A (x))U (ρ ⊗ σ )U ★ ] = Tr[(I ⊗ E M A (x))U ' (ρ ⊗ σ )U '★ ]

(2.9) We remark that the state space .K can also vary and it can be considered as a variable determining context. However, we shall not separately emphasize the role of

2 Cooperative Functioning of Unconscious and Consciousness from Theory …

19

variability of state space .K and assume that this variable is associated with the interaction-operator .U and state .σ. Context determines the state transformation (2.8) that has to be denoted not simply as .I A (x), but as .IC A (x). Depending on context .C A , .C can generate different state transformations in .UC caused by the measurement. State transformation can be detected with successive measurement of another observable . B. As was shown in Ozawa and Khrennikov (2020), Basieva et al. (2021), some psychological effects can provide at least partial information on the form of the quantum instrument .IC A (x), in particular, to exclude von Neumann-Lüders instruments given by state trnasformations of the projection form.

Concluding Remarks We understand that appealing to the unconscious-conscious description of cognitive processes is not so common in the modern psychology. However, this description well matches the indirect measurement scheme. We can appeal to the authority of James James (1890) (appealing to Freud (1957) might generate a negative reaction), see also Jung and Pauli (2014). We also can mention the series of works on the use of the unconscious-conscious scheme in quantum-like modeling of cognition (Khrennikov 2002b, 1999, 2008).2 In any event, the scheme of indirect measurements in quantum theory matches well with unconscious-conscious structuring of human mind and decision making as indirect observations on inapproachable bio-information system, unconscious. Acknowledgements The author would like to thank Irina Basieva and Masanao Ozawa for discussions.

References Adenier, G., Khrennikov, A., Nieuwenhuizen, Th.M. (eds.): Quantum Theory: Reconsideration of Foundations-3. AIP, Melville, NY (2006) Asano, M., Basieva, I., Khrennikov, A„ Ohya, M., Tanaka, Y., Yamato, I.: Quantum-like model for the adaptive dynamics of the genetic regulation of E. coli’s metabolism of glucose-lactose. Syst. Synth. Biol. 6, 1–7 (2012a) Asano, M., Basieva, I., Khrennikov, A., Ohya, M., Tanaka, Y., Yamato, I.: Towards modeling of epigenetic evolution with the aid of theory of open quantum systems, vol. 1508, pp. 75–85. AIP, Melville, NY (2012b). https://doi.org/10.1063/1.4773118 2

See also the series of papers (Khrennikov 1997, 2004d, 2007; Iurato and Yu 2015) on modeling of cooperative unconscious-conscious functioning by using the threelike geometry of. p-adic numbers. Although this model is not quantum, the use of . p-adic numbers gives the possibility to model discreteness.

20

A. Khrennikov

Asano, M., Khrennikov, A., Ohya, M., Tanaka, Y., & Yamato, I.: Violation of contextual generalization of the Leggett-Garg inequality for recognition of ambiguous figures. Phys. Scr. T163, 014006 (2014) Asano, M., Ohya, M., Tanaka, Y., Basieva, I., Khrennikov, A.: Quantum-like model of brain’s functioning: decision making from decoherence. J. Theor. Biol. 281(1), 56–64 (2011) Asano, M., Khrennikov, A., Ohya, M., Tanaka, Y., Yamato, I.: Quantum Adaptivity in Biology: From Genetics to Cognition. Springer, Heidelberg-Berlin-New York (2015a) Asano, M., Basieva, I., Khrennikov, A., Ohya, M., Tanaka, Y., Yamato, I.: Quantum information biology: from information interpretation of quantum mechanics to applications in molecular biology and cognitive psychology. Found. Phys. 45, 1362–1378 (2015b) Asano, M., Basieva, I., Khrennikov, A., Yamato, I.: A model of differentiation in quantum bioinformatics. Prog. Biophys. Mol. Biol. Part A 130, 88–98 (2017a) Asano, M., Basieva, I., Khrennikov, A., Ohya, M., Tanaka, Y.: A quantum-like model of selection behavior. J. Math. Psychol. 78, 2–12 (2017b) Bagarello, F., Oliveri, F.: A phenomenological operator description of interactions between populations with applications to migration. Math. Models Methods Appl. Sci. 23(03), 471–492 (2013) Bagarello, F., Gargano, F., Oliveri, F.: Spreading of competing information in a network. Entropy 22(10), 1169 (2020) Basieva, I., Khrennikov, A.: On the possibility to combine the order effect with sequential reproducibility for quantum measurements. Found. Phys. 45(10), 1379–1393 (2015) Basieva, I., Pothos, E., Trueblood, J., Khrennikov, A., Busemeyer, J.: Quantum probability updating from zero prior (by-passing Cromwell’s rule). J. Math. Psychol. 77, 58–69 (2017) Basieva, I., Khrennikov, A., Ozawa, M.: Quantum-like modeling in biology with open quantum systems and instruments. Biosystems 201, 104328 (2021) Benatti, F., Floreanini, R. (eds.): Irreversible Quantum Dynamics. Springer, Berlin-Heidelberg (2010) Boyer-Kassem, T., Duchene, S., Guerci, E.: Quantum-like models cannot account for the conjunction fallacy. Theor. Decis. 10, 1–32 (2015) Busemeyer, J., Bruza, P.: Quantum Models of Cognition and Decision. Cambridge University Press, Cambridge (2012) Busemeyer, J.R., Wang, Z., Townsend, J.T.: Quantum dynamics of human decision making. J. Math. Psychol. 50, 220–241 (2006) Busemeyer, J.R., Wang, Z., Khrennikov, A., Basieva, I.: Applying quantum principles to psychology. Phys. Scr. T163, 014007 (2014) Chiribella, G., D’Ariano, G.M., Perinotti, P.: Informational axioms for quantum theory. In: Foundations of Probability and Physics-6, vol. 1424, pp. 270–279. AIP, Melville, NY (2012a) Chiribella, G., D’Ariano, G.M., Perinotti, P.: Probabilistic theories with purification. Phys. Rev. A 81, 062348 (2012b) Choustova, O.: Quantum Bohmian model for financial market. Phys. A Stat. Mech. Appl. 374(1), 304–314 (2007) Choustova, O.: Application of Bohmian mechanics to dynamics of prices of shares: stochastic model of Bohm-Vigier from properties of price trajectories. Int. J. Theor. Phys. 47(1), 252–260 (2008) Choustova, O.: Quantum probability and financial market. Inf. Sci. 179(5), 478–484 (2009) D’Ariano, G.M.: Quantum Theory from First Principles (An Informational Approach). Cambridge University Press, Cambridge (2019) D’Ariano, M., Shao-Ming, F., Haven, E., Hiesmayr, B.C., Khrennikov, A.Yu., Larsson, J.-A.: Foundations of Probability and Physics-6. AIP, Melville, NY (2012) Davies, E.B.: Quantum Theory of Open Systems. Academic Press, London (1976) Dzhafarov, E.N., Kujala, J.V.: Selectivity in probabilistic causality: where psychology runs into quantum physics. J. Math. Psychol. 56, 54–63 (2012) Dzhafarov, E.N., Zhang, R., Kujala, J.V.: Is there contextuality in behavioral and social systems? Philos. Trans. R. Soc. A 374, 20150099 (2015)

2 Cooperative Functioning of Unconscious and Consciousness from Theory …

21

Freud, S.: The Standard Edition of Complete Psychological Works of Sigmund Freud, vol. I-XXIV. In: Strachey, J. (Ed. and Trans.) The Hogarth Press, London (1957) Fuchs, C.A.: QBism, The Perimeter of Quantum Bayesianism (2012). arXiv:1003.5209 [quant-ph] Fuchs, C.A.: Quantum mechanics as quantum information (and only a little more). In: Khrennikov, A. (ed.) Quantum Theory: Reconsideration of Foundations, pp. 463–543. Växjö University Press, Växjö (2002) Fuchs, C.A., Schack, R.: A quantum-Bayesian route to quantum-state space. Found. Phys. 41, 345 (2011) Giustina, M., Versteegh, M.A., Wengerowsky, S.H., Steiner, J., Hochrainer, A., Phelan, K., Steinlechner, F., Kofler, J., Larsson, J.Å, Abellan, C. et al.: A significant-loophole-free test of Bell’s theorem with entangled photons. Phys. Rev. Lett. 115, 250401 (2015) Hameroff, S.: Quantum coherence in microtubules. A neural basis for emergent consciousness? J. Conscious. Stud. 1, 91–118 (1994) Haven, E.: Pilot-wave theory and financial option pricing. Int. J. Theor. Phys. 44(11), 1957–1962 (2005) Haven, E., Khrennikov, A.: Quantum Social Science. Cambridge University Press, Cambridge (2013) Haven, E., Khrennikov, A., Robinson, T.R.: Quantum Methods in Social Science: A First Course. WSP, Singapore (2017) Ingarden, R.S., Kossakowski, A., Ohya, M.: Information Dynamics and Open Systems: Classical and Quantum Approach. Kluwer, Dordrecht (1997) Iurato, G., Khrennikov, A. Yu.: Hysteresis model of unconscious-conscious interconnection: exploring dynamics on .m-adic trees. . p-Adic Numbers Ultrametric Anal. Appl. 7(4), 312–321 (2015) Jaeger, G.: Quantum Information. An Overview. Springer, Heidelberg-Berlin-New York (2007) James, W.: The Principles of Psychology. Henry Holt and Co., New York, Reprinted 1983. Harvard University Press, Boston (1890) Jung, C.G., Pauli, W.: Atom and Archetype: The Pauli/Jung Letters 1932–1958. Princeton University Press, Princeton (2014) Khrennikov, A. (ed.): Foundations of Probability and Physics-3, Ser. Conference Proceedings, vol. 750. American Institute of Physics, Melville, NY (2005) Khrennikov, A., Basieva, I., Dzhafarov, E.N., Busemeyer, J.R.: Quantum models for psychological measurements: an unsolved problem. PLOS One 9, Art. e110909 (2014) Khrennikov, A.: Quantum-like model of unconscious-conscious dynamics. Front. Psychol. 6, Art. N 997 (2015) Khrennikov, A.: Reconstruction of quantum theory on the basis of the formula of total probability. In: Foundations of Probability and Physics-3, AIP Conference Proceedings, vol. 750, pp. 187–218. American Institute of Physics, Melville, NY (2005) Khrennikov, A.: Växjö interpretation of quantum mechanics. In: Quantum Theory: Reconsideration of Foundations, Ser. Math. Modelling, vol. 2, pp. 163–170. Växjö University Press (2002); Preprint: arXiv:quant-ph/0202107 Khrennikov, A.: Human subconscious as a p-adic dynamical system. J. Theor. Biol. 193(2), 179–96 (1998) Khrennikov, A.: Classical and quantum mechanics on information spaces with applications to cognitive, psychological, social and anomalous phenomena. Found. Phys. 29, 1065–1098 (1999) Khrennikov, A. (ed.): Quantum Theory: Reconsideration of Foundations. Växjö University Press, Växjö (2002a) Khrennikov, A.Yu.: Classical and Quantum Mental Models and Freud’s Theory of Unconscious Mind. Växjö University Press, Växjö, Sweden (2002b) Khrennikov, A.: Quantum-like formalism for cognitive measurements. Biosystems 70, 211–233 (2003) Khrennikov, A.Yu. (ed.): Quantum Theory: Reconsideration of Foundations-2. Växjö University Press, Växjö (2004a)

22

A. Khrennikov

Khrennikov, A.: On quantum-like probabilistic structure of mental information. Open Syst. Inf. Dyn. 11(3), 267–275 (2004b) Khrennikov, A.: Information Dynamics in Cognitive, Psychological, Social, and Anomalous Phenomena, Ser. Fundamental Theories of Physics. Kluwer, Dordrecht (2004c) Khrennikov, A.: Probabilistic pathway representation of cognitive information. J. Theor. Biol. 231(4), 597–613 (2004d) Khrennikov, A.: Quantum-like brain: interference of minds. BioSystems 84, 225–241 (2006) Khrennikov, A.Yu.: Toward an adequate mathematical model of mental space: conscious/unconscious dynamics on .m-adic trees. Biosystems 90(3), 656–675 (2007) Khrennikov, A.: The quantum-like brain operating on subcognitive and cognitive time scales. J. Conscious. Stud. 15(7), 39–77 (2008) Khrennikov, A.: Contextual Approach to Quantum Formalism. Springer, Berlin-Heidelberg-New York (2009) Khrennikov, A.: Ubiquitous Quantum Structure: From Psychology to Finances. Springer, BerlinHeidelberg-New York (2010) Khrennikov, A.: Quantum Bayesianism as the basis of general theory of decision-making. Philos. Trans. R. Soc. A 374, 20150245 (2016) Khrennikov, A., Basieva, I.: Possibility to agree on disagree from quantum information and decision making. J. Math. Psychol. 62(3), 1–5 (2014a) Khrennikov, A., Basieva, I.: Quantum model for psychological measurements: from the projection postulate to interference of mental observables represented as positive operator valued measures. NeuroQuantology 12, 324–336 (2014b) Khrennikov, A., Basieva, I., Pothos, E.M., Yamato, I.: Quantum probability in decision making from quantum information representation of neuronal states. Sci. Rep. 8, 16225 (2018) Khrennikova, P.: A quantum framework for ‘Sour Grapes’ in cognitive dissonance. In: Atmanspacher, H., Haven, E., Kitto, K., Raine, D. (eds.) Quantum Interaction. QI 2013. Lecture Notes in Computer Science, vol. 8369. Springer, Berlin-Heidelberg (2014) Khrennikova, P.: Quantum dynamical modeling of competition and cooperation between political parties: the coalition and non-coalition equilibrium model. J. Math. Psychol. 71, 39–50 (2016) Khrennikova, P.: Modeling behavior of decision makers with the aid of algebra of qubit creationannihilation operators. J. Math. Psychol. 78, 76–85 (2017) Lawless, W.F.: The interdependence of autonomous human-machine teams: the entropy of teams, but not individuals, advances science. Entropy 21(12), 1195 (2019) Ozawa, M., Khrennikov, A.: Modeling combination of question order effect, response replicability effect, and QQ-equality with quantum instruments. J. Math. Psychol. (2021) Ozawa, M.: Quantum measuring processes for continuous observables. J. Math. Phys. 25, 79–87 (1984) Ozawa, M.: An operational approach to quantum state reduction. Ann. Phys. (N.Y.) 259, 121–137 (1997) Ozawa, M., Khrennikov, A.: Application of theory of quantum instruments to psychology: combination of question order effect with response replicability effect. Entropy 22(1), 37 (2020) Penrose, R.: The Emperor’s New Mind. Oxford University Press, New York (1989) Plotnitsky, A.: Epistemology and Probability: Bohr, Heisenberg, Schrödinger and the Nature of Quantum-Theoretical Thinking. Springer, Berlin-New York (2009) Pothos, E., Busemeyer, J.R.: A quantum probability explanation for violations of ‘rational’ decision theory. Proc. R. Soc. B 276, 2171–2178 (2009) Umezawa, H.: Advanced Field Theory: Micro, Macro and Thermal Concepts. AIP, New York (1993) Vitiello, G.: My Double Unveiled: The Dissipative Quantum Model of Brain, Advances in Consciousness Research. John Benjamins Publ. Com (2001) Vitiello, G.: Dissipation and memory capacity in the quantum brain model. Int. J. Mod. Phys. B 9, 973 (1995) von Helmholtz, H.: Treatise on Physiological Optics. Transl. Optical Society of America in English. Optical Society of America, New York, NY (1866)

2 Cooperative Functioning of Unconscious and Consciousness from Theory …

23

Wang, Z., Busemeyer, J.R.: A quantum question order model supported by empirical tests of an a priori and precise prediction. Topics Cognit. Sci. 5, 689–710 (2013) Wang, Z., Solloway, T., Shiffrin, R.M., Busemeyer, J.R.: Context effects produced by question orders reveal quantum nature of human judgments. PNAS 111, 9431–9436 (2014) Yukalov, V.I., Sornette, D.: Conditions for quantum interference in cognitive sciences. Topics Cognit. Sci. 6, 79–90 (2014) Yukalov, V.I., Sornette, D.: Quantum probabilities as behavioral probabilities. Entropy 19, 112 (2017) Yukalov, V.I.: Evolutionary processes in quantum decision theory. Entropy 22(6), 68 (2020) Yukalov, V.I., Sornette, D.: Physics of risk and uncertainty in quantum decision making. Eur. Phys. J. B 71, 533–548 (2009)

Chapter 3

Hilbert Space Modelling with Applications in Classical Optics, Human Cognition, and Game Theory Partha Ghose and Sudip Patra

Abstract In this chapter we provide first a basic introduction to Hilbert space modelling, and its applications outside typical quantum mechanics, for example in classical optics, and human cognition. We then present briefly our framework of human cognition model, which we have called, COM: classical optical modelling. Though our chapter is based on the background of and in the wake of quantum-like modelling in cognition, game theory, and different social science areas, our approach differs from the extant models since we follow the Hilbert space modelling in classical optics, with novel features like classical entanglement which can also be exploited in game theory. Hence, we aim to contribute in cognition as well as quantum-like modelling in game theory. Keywords Hilbert space · Classical entanglement · Nash equilibrium · Quantum mechanics · Human cognition

Introduction In a way Wigner (1990) echoed Galileo, or the Greek paradigm, that nature is written in the language of Mathematics, when he famously observed there was an unknown efficacy of Mathematics in the universe. We can observe though, that in recent decades Physics has also informed Mathematics [Edward Witten being a Physicist to win the Fields Medal in Mathematics]. It might be claimed that there appears to be a P. Ghose Tagore Centre for Natural Sciences and Philosophy, New Town, Rabindra TirthaKolkata 700156, India The National Academy of Sciences (NASI), Allahabad, India S. Patra (B) CEASP, O.P. Jindal Global University, Sonipat, India e-mail: [email protected] The Laszlo Institute of New Paradigm Research, Trieste, Italy © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 A. Chakraborti et al. (eds.), Quantum Decision Theory and Complexity Modelling in Economics and Public Policy, New Economic Windows, https://doi.org/10.1007/978-3-031-38833-0_3

25

26

P. Ghose and S. Patra

wonderful marriage between these great domains of knowledge, one getting reduced to the other, which seems to us an important criterion to do Physics or Science in general. Reductionism of any kind is a futile thought. Given this background, when we turn to QM or QFT a specific language of Mathematics appears to be ‘natural’, which is the Hilbert space construction,1 introduced by Birkhoff and Neumann (1936) while the duo was working on the mathematical foundations of QM. Though, as we will note later in this chapter, Neumann had a different formulation in mind for QM, eventually Hilbert space became the ‘natural’ way of expressing a quantum mechanical description of quantum states with operators acting on them with a complete algebra and the crucial Eigen value-Eigen vector link, the spectral decomposition theorem, and the crucial Entanglement description and its measures. The list goes on. Currently, Hilbert space formulations have been central to fast evolving areas like quantum computation (Egger et al. 2020), quantum information theory (Timpson 2013) and quantum optics. Interestingly, Hilbert space formulation is also being used regularly outside the field of quantum physics, whether in classical optics and electromagnetism (Patra and Ghose 2022a, b), where such a formulation is effective in describing an important phenomenon called ‘classical entanglement’ (which will become an important tool for us in later chapters when we build on our framework), or even in the emerging field of ‘quantum cognition’ modelling which fundamentally has no relation to QM (or so is the standard perception, if we do not include Penrose’s view of a quantum like brain). Based on the rise of the so-called ‘quantum cognition modelling’, Hilbert space formulation has also been used very recently in human choice modelling (Haven et al. 2013). Mathematics is universal indeed. In the following section we present a brief introduction to Hilbert space in general, and then applications of it in QM and QFT, and more importantly for quantum cognition modeling. By Hilbert space we mean a complex normed vector space with an inner product defined on it which is linear on the second slot, and also separable, with countably finite or infinite dimensions (which is crucial since even for QFT we would like to have countable infinite no of degrees of freedom (Patra 2019)). Generally, such a space describe operators bounded from below, for example Hamiltonians. All self-adjoint operators, or Hermitian operators representing observables in QM are defined on a Hilbert space with C* algebra or commutation relations between them. Compound systems can be described by tensor products of individual separable Hilbert spaces of each system, and on such compound systems one can describe operators and partial traces to extract information from a subsystem of the compound system. Compound system states can either be factorized as tensor products of states of the subsystems, the so called product states, or cannot be factorized, the so called ‘entangled states’.2 1

One anecdotal story goes that David Hilbert, the great German Mathematician, on whose name Neumann named this formulation, was rather oblivious of the fact. In one seminar where Neumann lectured on the properties and applications of Hilbert space, in the end, up went a hand asking but what is a ‘Hilbert’ space? The man was Hilbert himself. 2 In modern representations, in case of a product state, the amplitude of the total system is written as a product of amplitudes (outer product to be specific) of its subsystems’ wave functions, but for

3 Hilbert Space Modelling with Applications in Classical Optics, Human …

27

There are certain problematic features about Hilbert space modelling too. For example, if Hilbert space modelling is used in ‘many body’ problems, then one encounters exponentially growing dimensions. There is a growing body of literature in this area known as tensor networks (Adhikari et al. 2021 for example). Another critical point, which we would deal with later, is that authors have demonstrated that for a full exposition of the ‘bra-ket’ formulation of Dirac (1981) one requires a rigged Hilbert space.

A Brief Mathematical Detour Complex Euclidian Space Since Hilbert space is a complex normed linear vector space with an inner product defined on it, we first start with some basic properties of complex linear vector spaces, which in a way is a general extension of standard Euclidian space. An alphabet is a finite non empty set whose elements can be taken as symbols. E Generally, an alphabet is denoted by a Greek letter, for example, and its elements by small case letters like a, b, c etc. Examples of alphabets are a binary {0,1} or its n n-th Cartesian product E{0,1} . E E For any alphabet we can define C( ) to be a set of all functions from E to complex numbers C such that the set forms a linear vector space of dimension | | with addition and scalar multiplication defined by: 1. if u, v are elements of the vector space, then (u + v) also belongs toE the vector space, with an equation (u + v) (a) = u(a) + v(a) for a belonging to . E 2. If u belongs to C( ) and a scalar α belongs to C, then the vector αu also E E belongs to C( ) satisfying the equation (αu) (a) = αu(a) for a belonging to . A vector space thus defined is a complex Euclidian space. The value u(a) is denoted as the entry of u indexed by a. A vector whose entries are all zero is denoted simply as 0. The term Hilbert space is closely related the concept of complex Euclidian set, with more generalization, for example possibility of infinite index sets.

Inner Products and Norms of Vectors E E For vectors u, v e C( ) the inner E product is defined as u(a)* v(a) where the sum is defined over a’s where a e . The inner product has the following properties: entangled states, one needs to sum over the proper number of indices to have the amplitude for the total system. A low entanglement state is one where the number of indices to sum over is minimum. For example, if the number is one, then we are dealing with a product state. The number of indices which one has to sum over is often called as bond or auxiliary dimension.

28

P. Ghose and S. Patra

1. Linearity inE the second argument: = α + β , where u, v, w e C( ), and α,β e C. 2. Conjugate symmetry: = * 3. Positive definitiveness: ≥ 0 with equality holding only for u = 0. E norm of a vector u e C( ) is defined as ||u|| = 1/2 = E EThe Euclidian 2 1/2 ( |u(a)| ) , where the sum is over a s e . The norm so defined has three main properties: positive definitiveness, positive scalability and triangle inequality where triangle inequality means ||u + v|| ≤ ||u|| + ||v||. The Cauchy–Schwarz inequality is given by | < u, v > | ≤ ||u||||v|| where the equality holds when u and v are linearly dependent. E In general, the p form Euclidian normEis given by ||u||p = ( |u(a)|p )1/p , given that p < ∞, ||u||∞ = max {|u(a)|, where a e }. Two Evectors u and v are said to be orthogonal if = 0 for u, vEboth elements of C ( ). Again, a collection {u a s.t a e U} which is a subset of C( ), where U is an alphabet, is said to be an orthogonal set if = 0 for all u a , u b e U where a and b are distinct. An orthogonal set of unit vectors is called an orthonormal set, and when such a set forms a basis it is termed an orthonormal Ebasis. An orthonormalEset as described above can formE an orthonormal basis inEC( ) if and only if |U|= | |. The standard basis on EC ( ) is given by {ea : a e }, s.t. ea (b) = 1 iff a = b or 0 otherwise for all a, b e .

Direct Sums and Direct Products E E Generally the direct sum (⊕) of complex Euclidian E Espaces C( E 1 ) ⊕ C( 2 ) ⊕ …. E U …. En ) whereE U stands for C( n ) is also a complex Euclidian space C(E 1 U 2E disjointEunions of the individual alphabets/ where 1 U 2 U …. n = U{(k, a): a e k } where k e{1, 2,….n}. E Now for vectors i e C( i ) the Edirect sum can be considered as a column vector E uE of dimension | 1 |+| 2 |+· · · + | n |. Such vectors follow the following rules: E (i) If ui and vi are e C( i ) then u1 ⊕ u2 ⊕ · · · ⊕ un ⊕ v1 ⊕ · · · ⊕ vn = (u1 + v1 ) ⊕ · · · ⊕ (un + vn ) (ii) Scalar multiplication rule: α(u1 ⊕ u2 ⊕ · · · ⊕ un ) = (αu1 ) ⊕ (αu2 ) ⊕ · · · ⊕ (αun ). (iii) Inner product rule: = + + · · · + . E E E The Etensor product between C( )s is also a complex Euclidian space C( 1 x 2 x….E n ) where x denotes the Cartesian product. If there are individual vectors ui s them is given by u1 ⊗ u2 ⊗ …. ⊗ un , which is e C( i ) s, a tensor E between E E product an element of C( 1 x 2 …. n ). For such a vector u1 ⊗ u2 ⊗ …. ⊗ un (a1 , a2 …., an ) = u1 (a1 )u2 (a2 )…..un (an ).

3 Hilbert Space Modelling with Applications in Classical Optics, Human …

29

E vectors are also termed as elementary tensors. They span the space C ( 1 x E SuchE 2 …. n ), but not every element in such a space is an elementary tensor. An importantEidentity which elementary tensors satisfy is: If ui , vi e C( i ) then: = …. Tensor products of complex Euclidian spaces or of the vectors defined on them are not associative in so far as the Cartesian products are not associative.

Linear Operator Space For two complex Euclidian spaces X and Y, L (X, Y) denotes a collection of linear maps from X to Y. Such linear maps are called linear operators. Au can be denoted as the application of a linear operator A e L (X, Y) on a vector u e X. The L (X, Y) forms a complex vector space with the following properties: 1. Addition: with A and B e L (X, Y) and u e X, (A + B) e L (X, Y) s.t. (A + B)u = A u + B u. 2. Scalar multiplication: with A e L (X, Y) and α e X, the operator α A also e L, s.t. The operator α A (u) = α (Au) where u e X.E E For two complex Euclidian spaces and E X = C( ), and Y = C( ) where are two alphabets and a e and b e the operator Ea,b e L (X, Y) is defined as Ea,b u = u(b)ea where u e X. All such collections of E s form a standard basis in the space L (X, Y) with Dim (L) = Dim X Dim Y. In later chapters we will come across operators defined on such spaces which are entry wise conjugate, transpose or adjoint with their own linearity properties. The kernel of an operator A e L(X, Y) is the sub space of X defined as kernel (A) = {ue X; Au = 0}. The image of A is the subspace of Y such that Image (A) = {Au; u e X }. An important relation is V

V

V

Dim (kernel A) + Dim (image A) = Dim (X) .

Examples of Hilbert Spaces a. C [a, b], the space of complex valued continuous functions in the interval [a, b] is { endowed with an inner product ( f, g) = f (x) g(x) dx where f, g are continuous functions, but this space is not complete, and hence not a Hilbert space. To make {b it complete one needs to define the norm ||f||= ( a | f (x)|2 d x)1/2 . Such a space is

30

P. Ghose and S. Patra

then denoted by L2 ([a, b]), or more generally for p form norms as Lp (.), but they are Banach spaces and not Hilbert spaces when p /= 2. b. The Hilbert space of bi-finite complex sequences, i.e. l2 (Z) is defined as l2 (Z) = E+∞ { −∞ |z|2 < ∞}. The space of l2 (Z) is a complex linear space endowed with standard rules of addition and multiplication by scalars. The scalar product is similarly defined. c. Ck ([a, b]) is a space of functions with kth continuous derivatives defined over the E {b range [a, b] where the inner product on the space is ( f, g) = k0 a f (x)g(x)d x, but again to complete this space one needs to define the norm E {b || f || = ( k0 a | f |2 d x)1/2 . Such a space endowed with the inner product is complete and termed a Sobolov space, denoted by Hk ([a, b]).

Operations on Hilbert Spaces Similar to complex Euclidian spaces, for Hilbert spaces too, given two or more Hilbert spaces one can generate bigger spaces by taking direct sums or tensor products between them. Orthonormal bases can be introduced for finite dimensional Hilbert spaces, but for infinite dimensional cases one generally refers to the Hamel basis. Using Zorn’s lemma, one can show that every Hilbert space has an orthonormal basis. Two orthonormal bases of the same space have the same cardinality. Hilbert spaces are called separable if and only they have countable orthonormal bases. In physics we mostly use separable Hilbert spaces, and all infinite dimensional separable Hilbert spaces are isomorphic to each other. If ϕ is an element of H* then there exists a unique u in H for which ϕ(x) = , for all x in H. This is known as the reflexivity of a Hilbert space.

Bounded and Un-Bounded Operators in Hilbert Space Continuous linear operators on Hilbert spaces which map bounded sets to bounded sets are called Bounded operators. The norm of such an operator A is defined as ||A|| = Sup {||Ax||: ||x|| X(ω) Y(ω) d P(ω). Hence the Hilbert space of a set of second order random variables will consists of random variables of finite means and finite variances, and two random variables X, Y are uncorrelated if E[X Y] = EX EY. Specifically, orthogonal random variables will have zero means and zero correlations between them.

3 Hilbert Space Modelling with Applications in Classical Optics, Human …

33

Applications of Hilbert Space Representation Outside QM Hilbert Space Representation of Classical Optics There has been a good amount of attention devoted to the Hilbert space structure of classical electrodynamics (Rajagopal and Ghose 2016). The main observation in such important studies have been the ‘classical entanglement’ or ‘inseparability’ as observed in classical optical beams, which are however of a different category than pure quantum entanglement, which can be further demonstrated in the case of spin–orbit entanglement. There have been demonstrations of Bell inequality violations or violations of CHSH types, implying interesting analogies with genuine quantum systems. However, as authors point out that since there is no presence of non-commuting operators in classical formulation, where as the central tenet of quantum formulation is based on non-commuting operators, there need to be serious study on the differences between KvN type and QM Hilbert space formulations. Sudarshan (Misra and Sudarshan 1977 for example) developed a classic work of Hilbert space representation of classical mechanics entirely based on commuting operators. Hence, we will use the abbreviation KvNS from now on. The basic feature of KvNS formulation is that position and momentum operators do commute, where as there are extra ‘unobservables’ or extra canonically conjugate operators, say λq and λp such that the following commutator bracket relations hold: [q, λq ] = i, = [p, λp ], and all other operators commute. Its has been shown that KvNS formulation has a deeper correspondence with QM, since if the original commuting set of operators are transformed appropriately then QM wave function can be conceived as a probability density at a point in the phase space of classical mechanics. However, is this a mathematical correspondence? Or can KvNS indicate how classical mechanics emerges out of QM, is still an open area of inquiry. Sudarshan developed a ‘hidden variable’ quantum theory for classical mechanics based on the only relevant variable ψ = ρ1/2 . According this formulation a super selection rule make various transitions of QM wavefunction with all its phases are unobserved, with classical mechanics emerging out of it. In this regard Sudarshan’s approach was different from KvN. For example if we have ψ = ρ1/2 exp (iS), where S is the phase factor, and plug this expression√into and its √ conjugate expression, we get the following independent equations: i ∂ ρ /∂t = L ρ, and: i ∂ S /∂t = L S. Such independent evolutions are possible due to a super selection rule. In KvN formalism, we have a complete orthonormal set {p, q} of commuting operators spanning the Hilbert space. However, we can also show λq = i ∂/∂q, and λp = i ∂/∂ p, two Hermitian operators, representing two more canonical variables, which commutes with each other but not with q and p respectively. Hence the Hilbert space can be represented by any one of the following four pair of commuting operators: (q, p) or (q, λp ), or (p, λq ), or (λq , λp ). The most interesting fact is that when the Hilbert space is represented by say, (q, λp ) instead of (q, p) the KvN wave function also do

34

P. Ghose and S. Patra

have phase, and the phase and amplitude evolutions are no longer independent as earlier. We can refer to Rajagopal and Ghose (2016) for a fuller treatment. Here however, we mention that if we like to make the phase of the wavefunction unobservable, then we need a super selection rule which would demand that only ρ1/2 is physically relevant observable. This is related to the fact that in classical mechanics both absolute phase and relative phase can not be measured, hence superposition and interference measures are absent. Hence in QM and classical mechanics all states which have same absolute phase are identical to each other. However, in case of classical optics like in QM relative phases are observable. Hence, in both cases all states which have relative phases are identical to each other. Hence Bloch sphere in QM and Poincare sphere in classical optics are used to describe such relative phases. As mentioned above Rajagopal and Ghose (2016), have constructed the KvNS version of classical electromagnetism, which preserves the phase features. Again, since Hilbert space representation also assures the entanglement description (in the sense of non-separability) via Schmidt theorem4 [], so called classical entanglement in optics can also be described. Bondar et al. [] further demonstrated that KvNS wave function can be thought as to be probability amplitude of a quantum particle at a particular point in classical phase space, however to show this we need to make a transform from the {q, p} variables to {q, λp }. Bell violations, one needs KvNS formulation, which is a phase space theory on Hilbert space. Wave functions in KvNS theory is square integrable functions as in QM, the main difference being need for a super selection rule which make the phase features unobservable in case of classical mechanics, but no such need for classical electrodynamics.

Any vector ψ e H1 ⊗ H2 such that |U > and |λ > are orthonormal basis in H1 and H2 respectively, can be expressed as ψ = Ecj |Uj > |λj > , where c s are positive non negative. Again if we have two observables A and B elements of H1 and H2 respectively, then < ψ|(A ⊗ I)|ψ > = Trace (Aρ1 ) and < ψ|(I ⊗ B)|ψ > = Trace (Bρ2 ), where ρ1 and ρ2 operates on respective Hilbert spaces, and |U > and |λ > s can be eigen √ vectors for respective density operators, and eigen values of ρ1 , say {pj } will be given by cj = pj in the current example we have two subspaces of same dimensions, but the decomposition formulation holds for any partitions. For example if we have a 5 Qbit space over all, and we partition the space into sub space A of 2 Qbit, such that A is spanned by 4 basis vectors, and sub space B of 3 Qbit space, such that B is spanned by 23 or 8 basis vectors, we will still have the above decomposition formula for ψ, such that the summation index will be over 4, or the no of basis states for the lower dimensional subspace. Another important concept is of Schimdt number. For example we start with a pure state such as ψ as in the example here, now if we perform a partial trace over it, such that we get back the density matrix of one of the subsystems, E then the no of non-zero coefficients in the density matrix representation of that subsystem: ρ = pj |Uj > < Uj |, will be a measure of degree of entanglement in ψ, this is the Schimdt number. In general we know that from a higher dimensional Hilbert space pure state we can arrive at a lower dimensional mixed state via partial tracing, the converse of this is termed as purification. 4

3 Hilbert Space Modelling with Applications in Classical Optics, Human …

35

Classical and Quantum Entanglements It is generally claimed in the relevant literature (Khrennikov 2020) that classical entanglement is non separability between multiple degrees of freedom of the same system, i.e. intra system entanglement, where as quantum entanglement can be both intra system and inter system, i.e. entanglement between multiple non interacting subsystems. Hence true ‘non-local’ correlations can be observed in genuine quantum scenario only. In a widely cited review of entanglement literature Horodecki et al. (2009) refers to the Hilbert space structure of a compound system, i.e. which can be described as tensor product of individual separable and complete Hilbert spaces of subsystems, where the general state of the compound system can be described as a superposition state. Now it is a hard problem to always show whether such a superposition state can be factorized, or shown as a product state. Non-separability problem is still the most difficult in entanglement literature. However, we should emphasize here that entanglement between two noninteracting subsystems does never mean superluminal communication between them, for example as in the EPR scenario. Hence no signaling condition is always maintained. Authors (Kauffman and Patra 2022 for example) have shown that in case of QM there is an upper limit for violation of CHSH inequality, the Tsirelson limit. However, there are speculations whether there can be other types of theories in which such an upper limit is crossed keeping the non-signaling condition intact. Authors (Rjagopal and Ghose, op cit) have shown that a complete description of a general sate in classical electrodynamics involves a direct product of Hilbert space Hpath of square integrable functions with another 2D Hilbert√space for polarization states Hpole . Here the state of unit intensity can be written as 1/ ||A2 || |A > |λ >, where A (r, t) are solutions of scalar wave equation, and λ e Hpol is the vector described as: Hence polarization-path entanglement in classical optics is a direct consequence of the Hilbert space structure of it. Where entanglement here means non-factorizability or non-separability between two spaces of special and polarization degrees of freedom.

Human Cognition and Decision Modelling Perhaps the most recent application of Hilbert space formulation has been in the area of human cognition modelling. For quite a few decades at least, it was taken for granted that human cognition or decision making can be faithfully represented by using Boolean logic structure, and Kolmogorovian measure theory. Such a theoretical framework (later modified by Bayesian probabilistic framework) was thought as to be the faithful description of rational human decision making. Here rationality means in accordance with certain basic axioms of set theory set up, for example

36

P. Ghose and S. Patra

commutative and distributive axioms, which again was adapted later in economics (whether neoclassical economics modelling or standard game theory set up, as first envisioned by Neumann and Morgenstern (revised edition 2007), and later largely developed by Nash (1950 for example)) via the constrained utility maximization framework. However, from the early 1960s at least, cognitive scientists (Segal 1987 for example), have observed that there are many special features of decision making (casually called as ‘anomalies’, but we would like to study such features deeply) for example, contextuality, order effects, failure of sure thing principle, conjunction and disjunction fallacies, guppy effect, entanglement like features in cognition and visual perceptions, to name a few, which can not be well described by standard set up. Certainly, there has been attempts by the so-called behavioral camp (Thaler 2016) to describe such features in a wide spectrum of models, for example prospect theory to inequity aversion theory to behavioral game theory, however no comprehensive or coherent framework do exist to accommodate most of such features along with standard results. Since late 1990s various authors (Haven et al. 2013 as a seminal work) have developed some Hilbert space formulation for cognition. Roughly we can accommodate such models in the following categories: a. Cognitive scientists have modelled mental states or belief states as rays in a Hilbert space, mostly a finite Dimensional Hilbert space. Hence a mental state can be though as a superposition of basis states spanning the very Hilbert space, such a description can also be thought as the basic uncertainty state of the agent or the decision maker in the question. Then mental state updating can follow Luder rule of updating (where pure states are updated as pure states or superposed states), and a measurement corresponding to answering a question posed to the agent can be described by projecting the mental state to one of its basis states, where probability of such a result can be described by Born’s rule. However, simple one-dimensional projection measures may not capture complex decision-making process always, since there might be noise in the cognitive process, hence more general class of measures, or POVMs have been used in describing cognitive models. One recurrent feature in Hilbert space modelling of cognition is certainly the presence of ‘interference’ terms, while measuring the total probability of any relevant event, such constructive or destructive interference terms (which naturally arises in the set up due to square of amplitude representation of probabilities) produces deviations from classical measure theory-based probabilities. One of the central features of Hilbert space modelling in cognition is the formula or law for total probability (LTP), which provides additional ‘interference’ terms. Such interference terms have been empirically measured, and might be thought as to be an important describer of deviations from classical measure theory-based predictions. Effects like conjunction and disjunctions are often described using the modified formula for total probability.

3 Hilbert Space Modelling with Applications in Classical Optics, Human …

37

Hilbert space representation of states has been successfully utilized in classical optics, where we work with optical beams obeying Maxwell’s equations of electromagnetism. Specifically, Hilbert space modelling is effective in describing ‘classical entanglement’, which is entanglement between multiple degrees of freedom of the same optical system, for example path-polarization entanglement. Applications in decision theory, human cognition and other social science areas: The outburst of ‘quantum like modelling’ in social science (Khrennikova and Patra 2019 for example) or ‘quantum cognition’ modelling (Busemeyer et al. 2006) or use of quantum like modelling in economics and finance (Patra and Ghose 2020) can all be perceived as the extension of Hilbert space representation. For example in human cognition, to resolve age old ‘anomalies’, which are nontrivial if not impossible, to resolve via use of Boolean logic or Kolmogorovian measure theory based description of choice making, Hilbert space(H) representation has proved to be effective. In such a representation a cognitive state can be thought as to be a unit vector, properly normalized, on a finite or countably infinite dimension Hilbert space. Again observables, which are here cognitive observables (can be thought of as responses or answers to specific questions like ‘are you happy’? ‘are you a feminist’? referring to the famous ‘Linda the bank teller’ scenario) can be represented as Hermitian operators on the Hilbert space defined. Then with standard rules on H, for example superposition of vectors description, or at times entanglement description, one can describe various really observed phenomenon, e.g., Failure of Sure thing principle (which is rather a central tenet of Neoclassical decision theory/ economics), Order effect, Disjunction and Conjunction effects etc. Most importantly, use of Hilbert space modelling, allows for a modified formula for total probability [FTP] to emerge, which is very common in standard QM. Such a modified FTP holds the key to describe deviations from the predictions of standard decision models based on classical probability theory [CPT]. There are different versions of modified FTP though, for example a standard version [going back to standard textbook versions of Double slit experiment, or what is termed as Vaxjo (Khrennikov 2002) version] has the CPT based FTP plus additive perturbative terms with a significant phase factor. One advantage of such a FTP is that it can reduce to classical FTP whenever the observables commute with each other.5 However, there are other type of modelling too, specifically, QBism, as mentioned earlier, which has entirely a different formulation of FTP, QBism is a very promising candidate for decision theory in general, but the jury is still out. Hilbert space modelling is also very widely used now in financial market modelling. Use of Hilbert space set up and operator algebra defined on it can make the basic trade models simple and more elegant, however one difficult issue to tackle in such models is the presence of Non-Hermitian operators, specifically Non-Hermitian 5

Here we mention that the commutation relations between cognitive variables are not as ‘naturally’ perceived as in standard QM, there is no general theory of commutative relations between cognitive variables, and contextuality may play a deeper role here.

38

P. Ghose and S. Patra

Hamiltonians. It is interesting to mention here that authors have found ‘no arbitrage’ or ‘Martingale’ properties of an efficient financial market leads to the formulation of Non-Hermitian Hamiltonian.

COM Approach (Patra and Ghose) Finally, we briefly discuss here Hilbert Space modelling-based cognition framework developed by the current authors. For a fuller reading of the COM framework and its applications in game theory for example, please refer to the above referred papers. Heisenberg held the view that there is always an uncontrollable and inherently probabilistic disturbance caused by interaction between a system to be observed and a probe, which is a central feature of QM. However, Bohr’s interpretation was subtle and different, in the sense that the state of the measuring device and the state of the object cannot be separated from each other during a measurement but they form a dynamical whole or an unanalyzable whole. Hence QM becomes in this picture is inherently contextual. Hence QM, where as described earlier in this chapter, a state (more specifically a pure state) is described as a normalized vector in a finite or infinite dimensional Hilbert space, exhibits three most important phenomenon: contextuality, interference and entanglement. Later contextuality has been shown to be a central feature of human cognition too (Dzhafarov and Kujala 2016 for example), there have been studies claiming interference patterns and entanglement present in human cognition or decision-making processes. However, for us the question was do we really need to do pure quantum mathematical modelling for human cognition? Here another central classical physics theory comes to help, classical optics or classical electromagnetism. All the three central features mentioned above is also observed in this field, certainly with observations on the differences between genuine quantum superposition and entanglement with classical superposition and entanglement. In classical optics too states are fully described by vector representation in Hilbert space, and such Hilbert space modelling of classical electromagnetism is rather a strong growing study. Hence in our approach of modelling cognition we rather adopt the suitable ‘classical Hilbert space’ mathematical formulation. We have shown in our works (Patra and Ghose, op cit) that such a mathematical description is suitable enough to describe features of human cognition which we have mentioned in the earlier sections. Since we have already described Hilbert space modelling in brief, in the final section we discuss how our approach; COM (classical optical modelling); can be applied to another emerging field of quantum-like modelling in game theory, or quantum games.

3 Hilbert Space Modelling with Applications in Classical Optics, Human …

39

Discussion: Application of COM in Game Theory Recently good amount of attention has been directed toward studying classical entanglement phenomenon, exhibited by classical optics or even by classical mechanical systems. The critical difference lies in the fact that true quantum entanglement or non-local correlations (without violating relativistic causality for sure) can be exhibited either by space-like separated subsystems or entanglement or coupling between degrees of freedom of the same system (for example path-spin entanglement for electrons, etc.), whereas classical entanglement is only for multiple degrees of freedom for a single classical system. Hence based on the above discussed literature of designing superior strategies or games relative to standard classical games, the question arises whether the shared physical system among the players might have ‘classical entanglement’ and still generate superior strategies which results into superior NE (Nash Equilibrium). Earlier literature have suggested that quantum entanglement guarantees better NE generation, but now we have demonstrated even much simpler shared physical systems might generate similar NE, for example for PD game. For the fuller mathematical treatment, we refer to our recent work. Here we briefly mention the salient features of our framework. To re-design the PD game such that superior strategy profiles emerge, we closely follow the refs of Patra and Ghose as above with (a) sources of two polarized classical light beams each for one player (b) a set of optical instruments which help each player to manipulate each light beam in a strategic manner (c) a measurement device (in this case polarization analyzer) which generates payoffs for every player (generally, classicality of the shared physical system between the players has some degree of correspondence to factorizability of joint probabilities, however non-factorizability of those joint probabilities may not always mean violations of Bell inequalities, hence non-factorizability is a sufficient but not necessary condition for ‘quantumness’ of the shared physical system) based on the polarization sates of each optical beam. Again the game set up is a common knowledge to all players, say to Alice and Bob in case of a 2 × 2 game. Here then, we assign two vectors |C > and |D > to two possible classical strategies C and D (co-operate and defect namely) which spans the 2 D Hilbert space H of a polarized beam. Hence state of the game at every instant is provided by a vector in the tensor product space HA ⊗ HB spanned by the basis |CA > ⊗ |CB >, |CA > ⊗ |DB >, |DA > ⊗ |DB >, |DA > ⊗ |CB >, now if |C > is assigned as |0 > and |D > is assigned as |1 >, two standard basis vectors on the Poincare sphere,6 representing horizontal and vertical polarization respectively, then, the ‘optical’ states with Alice and Bob will be:

6

Poincare sphere is a widely used tool for describing states, like polarization states in Optics as well as in information theory in general. https://encyclopediaofmath.org/wiki/Poincar%C3%A9_ sphere.

40

P. Ghose and S. Patra

|ψ > I = cos 0i |C > I + e iϕ i sin0i|D > I where I denotes Alice or Bob. With 20 in the range [−π/2, + π/2], and 2ϕ in the range [0,2π]. Based on this representation we can then define, ‘strategy space’ of the players using Unit Matrices7 U (2), where U is the unit matrix symbol, with the property that U*U = UU* = I, where * here means the operation of complex conjugate transposition, and I is the identity matrix. Hence, we can define U(0,0) which is the identity matrix or operator in operator representation, and the other matrices in form of Puli Matrices. Again based on the algebra followed by the Pauli matrices, which forms a closed set of non-trivial operators, one can associate at most three independent strategies. Another important advantage of this modelling is since the results are based on general Non-Abelian algebra, there is no presence of Plank’s constant in our equations, which has always been a worry in quantum-like modelling, due to lack of proper interpretation. Hence, we find three independent strategies emerging out, C which corresponds to σ3 , D which corresponds to iσ2 and L (say, which means abstain) which corresponds to σ1 . Hence, we already see extended strategy profiles for players, it may also be noted that a linear combination of C and L would produce H, the well-known Hadamard Gate. We would request the readers to refer to our detailed framework (also in a forthcoming book, interdisciplinary approach to cognitive modelling by Ghose and Patra) for the full mathematical model. Here we just mention that based on the extended strategy profile, if classical entanglement feature is introduced in the system of optical beams, then D,D no longer remains the dominant NE, rather we get a Pareto improved NE. over all there are certain advantages of this framework as we list below. 1. We have distinguished our frame work of designing strategic decision making in form of a prisoners dilemma game (PD), which is based on novel features of classical optics; mainly classical entanglement which is robust; from pure ‘quantum like’ description of similar games, which are often called as ‘quantum prisoners dilemma’ (QPD). 2. We have been able to demonstrate the main point of departure from neoclassical game theory, that is, providing a mechanism to bypass the stubbornly dominant ‘defection’ Nash equilibrium in a PD. 3. We have demonstrated that based on classical optical modelling (COM, a frame work of cognition model which we have developed recently) or designing of PD we can achieve, or demonstrate Pareto superior equilibrium outcomes even in non-repetitive PD. 4. Our design of PD is thus an alternative to extant models which try utilize pure quantum entanglement (which is very fragile due to noise in the environment, in this case information environment), and hence our design is practicable while using simple optical tools, namely polarization. We have provided a detailed description of our design in the paper with simple geometric representations of decision moves by players via rotations of Poincare sphere. Such 7

https://en.wikipedia.org/wiki/Unitary_matrix.

3 Hilbert Space Modelling with Applications in Classical Optics, Human …

41

correspondence between decision making and rotation operations on Poincare sphere is an elegant way to capture new strategy profiles of players, and Pareto improved equilibrium points. We also refer to here our forthcoming book which elaborates our mathematical and philosophical framework (Ghose and Patra 2023-forthcoming).

References Adhikary, S., Srinivasan, S., Miller, J., Rabusseau, G., Boots, B.: Quantum tensor networks, stochastic processes, and weighted automata. In: International Conference on Artificial Intelligence and Statistics, pp. 2080–2088. PMLR (2021) Birkhoff, G., Von Neumann, J.: The logic of quantum mechanics. Ann. Math. 823–843 (1936) Busch, P.: Quantum states and generalized observables: a simple proof of Gleason’s theorem. Phys. Rev. Lett. 91(12), 120403 (2003) Busemeyer, J.R., Wang, Z., Townsend, J.T.: Quantum dynamics of human decision-making. J. Math. Psychol. 50(3), 220–241 (2006) Dirac, P.A.M.: The Principles of Quantum Mechanics (No. 27). Oxford University Press (1981) Dzhafarov, E.N., Kujala, J.V.: Context–content systems of random variables: The contextuality-bydefault theory. J. Math. Psychol. 74, 11–33 (2016) Egger, D.J., Gambella, C., Marecek, J., McFaddin, S., Mevissen, M., Raymond, R., Simonetto, A., Woerner, S., Yndurain, E.: Quantum computing for finance: State-of-the-art and future prospects. IEEE Trans. Quant. Eng. 1, 1–24 (2020) Fuchs, C.A., Mermin, N.D., Schack, R.: An introduction to QBism with an application to the locality of quantum mechanics. Am. J. Phys. 82(8), 749–754 (2014) Ghose, P., Patra, S.: An Interdisciplinary Approach to Cognitive Modelling: A Framework Based on Philosophy and Modern Science, 1st edn. Routledge (2023-forthcoming). https://doi.org/10. 4324/9781003429913 Haven, E., Khrennikov, A., Khrennikov, A.I.: Quantum Social Science. Cambridge University Press (2013) Horodecki, R., Horodecki, P., Horodecki, M., Horodecki, K.: Quantum entanglement. Rev. Mod. Phys. 81(2), 865 (2009) Kauffman, S., Patra, S.: Human cognition surpasses the nonlocality tsirelson bound: Is mind outside of spacetime? (2022). https://osf.io/zacsh/ Khrennikov, A.: Quantum versus classical entanglement: eliminating the issue of quantum nonlocality. Found. Phys. 50(12), 1762–1780 (2020) Khrennikov, A.: Växjö Interpretation of Quantum Mechanics. Växjö University Publication (2002) Khrennikova, P., Patra, S.: Asset trading under non-classical ambiguity and heterogeneous beliefs. Physica A 521, 562–577 (2019) Misra, B., Sudarshan, E.G.: The Zeno’s paradox in quantum theory. J. Math. Phys. 18(4), 756–763 (1977) Nash, J.F.: The bargaining problem. Econometrica: J. Econ. Soc. 155–162 (1950) Patra, S., Ghose, P.: Classical optical modelling of the ‘prisoner’s dilemma’ game. In: Sriboonchitta, S., Kreinovich, V., Yamaka, W. (eds.) Credible Asset Allocation, Optimal Transport Methods, and Related Topics. TES 2022a. Studies in Systems, Decision and Control, vol. 429. Springer, Cham (2022a) Patra, S., Ghose, P.: Classical optical modelling of social sciences in a Bohr-Kantian framework. In: Credible Asset Allocation, Optimal Transport Methods, and Related Topics, pp. 221–244. Springer International Publishing, Cham (2022b)

42

P. Ghose and S. Patra

Patra, S., Ghose, P.: Quantum-like modelling in game theory: Quo Vadis? A brief review. Asian J. Econ. Bank. 4(3), 49–66 (2020) Patra, S.: A Quantum Framework for Economic Science: New Directions (No. 2019–20). Economics Discussion Papers (2019) Penrose, R.: The road to reality. Random House (2005) Rajagopal, A.K., Ghose, P.: Hilbert space theory of classical electrodynamics. Pramana 86, 1161– 1172 (2016) Segal, U.: The Ellsberg paradox and risk aversion: An anticipated utility approach. Int. Econ. Rev. 175–202 (1987) Thaler, R.H.: Behavioral economics: Past, present, and future. Am. Econ. Rev. 106(7), 1577–1600 (2016) Timpson, C.G.: Quantum Information Theory and the Foundations of Quantum Mechanics. OUP Oxford (2013) Von Neumann, J., Morgenstern, O.: Theory of games and economic behavior. In: Theory of Games and Economic Behavior. Princeton University Press (2007) Wigner, E.P.: The unreasonable effectiveness of mathematics in the natural sciences. Math. Sci. 291–306 (1990)

Chapter 4

Remodeling Leadership: Quantum Modeling of Wise Leadership David Rooney and Sudip Patra

Abstract Since the late 90s a paradigm shift began in decision research that has implications for leadership research. Due to the limitations of standard decision theory (based on Kolmogorovian/Bayesian decision theory) scholars began to build a new theory based on the ontological and epistemological foundations of quantum mechanics. The last decade has witnessed a surge in quantum-like modeling in the social sciences beyond decisionmaking with notable success. Many anomalies in human behavior, viz, order effects, failure of the sure thing principle, and conjunction and disjunction effects are now more thoroughly explained through quantum modeling. The focus of this paper is, therefore, to link leadership with quantum modeling and theory. We believe a new paradigm can emerge through this wedding of ideas which would facilitate better understandings of leadership. This article introduces readers to the mathematical analytical processes that quantum research has developed that can create new insights in the social scientific study of leadership.

Introduction We begin with three observations. First, quantum theory is argued to lend itself to developing new empirical insight into complex, indeed, indeterminate human behavior. Second, there is reason for the research community to take stock of how we approach leadership research so that we might develop wiser leaders who make more positive impacts on the world. Third, at its most fundamental level, wise leadership is an indeterminate social practice that is amenable to quantum social scientific investigation. By indeterminate, we mean that it is uncertain exactly what the best and wisest thing to do is as a leader as each leadership challenge unfolds. This is partly D. Rooney Macquarie University, Macquarie Park, Australia S. Patra (B) OP Jindal Global, Jindal school of Government and public policy, OP Jindal Global University, Sonipat, India e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 A. Chakraborti et al. (eds.), Quantum Decision Theory and Complexity Modelling in Economics and Public Policy, New Economic Windows, https://doi.org/10.1007/978-3-031-38833-0_4

43

44

D. Rooney and S. Patra

to do with the complexity of the human mind and behavior, and also the complexity and ambiguity of the context in which leaders act. Dealing excellently with this leadership indeterminacy, we argue, requires wisdom. Leadership and wisdom are both deeply situated practices that are necessarily shaped by context. This article shows that researching indeterminate social practices can benefit from the ontological and epistemological tools developed to understand quantum mechanics. Specifically, we show how quantum-like modeling can address researching wise leadership and introduce the mathematical procedures that leadership research can employ. The ontological assumptions in quantum theory are relevant to the indeterminacy of social behavior and, in particular, the impacts of social, cultural, economic, political and other contextual factors on behavior that create ambiguity, uncertainty, bounded rationality, imperfect knowledge, and so on. For reasons of economy, we will henceforth refer to the totality of these contextual factors as context. We suggest that better understandings of the way leaders practice leadership require researchers to examine any inadequate ontological assumptions in research designs and to consider the methodological implications that flow from a revised ontology, particularly in relation to indeterminacy and context. Thus, we will show that the ontology and epistemology inherent in quantum mechanics and quantum physics research (including quantum mathematical analysis) are useful for the indeterminate ontology of leadership.1 Indeed, McDaniel and Walls (1997), Lord et al. (2015), Dyck and Greidanus (2017), Hahn and Knight (in press) argue that quantum theory is a useful new lens with which to understand ambiguity, paradox, diversity, relationships, social interaction, time and change in organizations. Importantly, quantum mechanics begins with “an undefined state, and offers an innovative approach for understanding the unfolding of complex organizational phenomena” (Lord et al. 2015: 264). Arguably, the most important message about the value of quantum theory to management and leadership research is, according to Dyck and Greidanus (2017), that it offers us an alternative to entrenched ontological assumptions that no longer serve us as well as we would like. This article extends the early work of McDaniel and Walls (1997), Lord et al. (2015), Dyck and Greidanus (2017), and Hahn and Knight (in press) by, firstly, linking it specifically to leadership and wisdom research through the mathematics and logic of quantum-like modeling (QM) as well as discussing examples of its use in behavioral economics, decisionmaking and cognitive science research where quantum modeling is already used. Secondly, by taking this body of research beyond making ontological and epistemological arguments we show how empirical research can operationalize quantum theory and quantum mathematical formalism. In this second respect, we introduce

1

However, we should caution readers that we would like to adapt the mathematical and logical framework of quantum theory only, rather than the physics of it. The emerging quantum like modelling in social sciences aims at that, for example, see Haven and Khrennikov (2013).

4 Remodeling Leadership: Quantum Modeling of Wise Leadership

45

quantum logic and mathematics to assist quantitative modeling of leadership generally and wise leadership in particular.2 If research is to contribute to developing wise leadership practitioners, then, a QM approach may be an important way to do this? It may surprise some that the mathematical and logical tools for describing and predicting the microphysical or subatomic world (for example, Haven and Khrennikov 2017; Khrennikova 2017; Khrennikova and Patra 2019) are useful in researching human behavior. However, just as calculus was invented for describing physical systems and was found to have significant utility in social science, so too, we argue, does quantum mathematics. However, we go further and argue, specifically, that QM is able to overcome significant limitations faced by those using standard quantitative social science tools that do not adequately link behavior and context, and lean too unrealistically on assumptions about linear and deterministic dynamics in what is a non-deterministic, uncertain, ambiguous, and even paradoxical world. “Quantum theory can represent multiple interacting paths through time and, thus, can represent the complexity of change in ways that more conventional models cannot” (Lord et al 2015: 265). In other words, we see that the underlying ontological assumptions in quantum theory are more realistic and have more empirical utility than those that underpin standard positivist (Hahn and Knight, in press) social science statistical analyses. Quantum-like modeling, therefore, can lift the impact of leadership research in an ever more intractable world. We do not, however, argue that the laws of quantum physics govern social behavior. In mathematical (epistemological) terms, another reason for taking a QM approach is to extend its quantum probability theory framework to understanding the impacts of leadership and wisdom’s Contextuality. “[Q]uantum mechanics is inherently probabilistic, rooted in the idea that all we can know about reality is the probability of experiencing a specific instantiation of it” (Hahn and Knight, in press: np.). This understanding of probability is based on the central role of context in quantum mechanics because the context, that is, a quantum system contains the full range of possibilities that the system can achieve. This system, therefore, is indeterminate and does not become something in particular until it is observed (that is, measured or experienced by someone). It is salient for leadership research that the limitations of standard decision theory have been noted many times since the seminal works of Tversky and Kahneman (1992). Important findings have also been provided by QM in behavioral economics and cognitive science research (Haven and Khrennikov 2013, 2017; Thaler 1994). If leadership is done in an indeterminate context and wisdom is the human quality for delivering excellence under conditions of indeterminacy it is well worth considering QM. We begin with a review of critiques of leadership research including of its ontological positions that problematic for good research design. We then introduce some foundational concepts in quantum mechanics followed by more detailed explanations of quantum Beyesian (QBism) theory (Caves et al. 2002), quantum field theory, and 2

A good coverage of extant emerging literature can be found in the research hand book edited by Haven and Khrennikov (2018).

46

D. Rooney and S. Patra

quantum decomposition theory. Having established a conceptual foundation, we then survey how QM is currently being used in cognition and decisionmaking research. We do this to give readers concrete examples of how to conceptualize research designs based on QM theory. Finally, we set out a range of research questions that QM can help solve. We have added a brief but relevant mathematical appendix setting out the basic mathematical tools of quantum theory and also a brief technical note on measures of entanglement.

Leadership It is hard to avoid criticisms of poor leadership around the world (Clegg et al. 2016; Dhiman 2017; Tourish 2013). Going a step further, researchers (Grint 2007; King and Nesbit 2015) argue that leadership training is ineffective at developing graduates who embody excellent leadership qualities that leadership theories call for. Leadership researchers are also interested in problematic kinds of leaders such as toxic (Pelletier 2010), destructive (Schyns and Schilling 2013), narcissistic (Rosenthal and Pittinsky 2006), and psychopathic (Boddy 2015) leaders, because the experience of being led in contemporary workplaces consistently does not meet minimum standards for ethical and professional practice. The weight of concern about poor leadership lead us to suggest that quantum modeling is an important new option for researchers to consider. Leadership judgment, thinking, decisionmaking, and behavior are done in and are products of uncertain environments (contexts). Furthermore, leaders, like all people, are boundedly rational and often appear to make what look like ‘irrational’ or rash decisions. But why is it the case that such problematic leadership continues and how can we rethink leadership research and theory from a QM perspective? Before exploring these questions, we discuss transformational, authentic and servant leadership theories to identify gaps that QM could fill. Transformational, servant and authentic leadership are commonly used frameworks for leadership research. Since Burns (1978), transformational leadership theory has dominated research. Transformational leadership theory focuses on four dimensions: (1) idealized influence, or a leader’s ability to inspire followers to identify with them; (2) inspirational motivation, excellence in communicating the leaders vision to followers; (3) intellectual stimulation, the ability of a leader to inspire followers to be innovative, take risks and to challenge assumptions; and (4) individual consideration, the ability of a leader to foster individuals to meaningfully meet their own needs. Transformational leadership focuses sharply on the individual qualities and capabilities of the leader (Zacher et al. 2014). There is a strong emphasis on individual behavior within group settings in the theory. Nonetheless, some research findings say that transformational leadership is a poor predictor of leader job performance (Judge and Piccolo 2004). Authentic leaders are self-aware and act in harmony with, their values, knowledge, and emotions (Harvey et al. 2006), they are future-oriented (Luthans and Avolio 2003), use balanced information processing (Avolio and Gardner 2005), and (as a

4 Remodeling Leadership: Quantum Modeling of Wise Leadership

47

consequence of these dispositions) are concerned to make a positive contribution to the external world (Ilies et al. 2005). Authentic leaders, therefore, embody these characteristics of excellence (Reh et al. 2017). An authentic leader, like a transformational leader, has a clear set of laudable values, including courage, as well as the skill to negotiate the complexities of the workplace and their leadership role to enact such excellence. Little, however, is said about the background or context in which these leaders must be so wise. Servant leadership theory focuses on the benevolence and selflessness of the excellent leader (Neubert et al. 2016). Servant leaders put others’ (followers) needs and wellbeing ahead of their own (Van Dierendonck 2011). Interestingly servant leadership blurs the boundaries between leader and follower. Servant leaders, then, are necessarily humble, compassionate and wise, and are not ‘power-junkies’. Arguably, servant leadership theory presents the most idealized version of leadership but is the theory in which leader, follower and context are most integrated, making leader, follower and context difficult to separate analytically. While few will doubt the attractiveness of this kind of leader, little effort has been made to deal with developing research designs that adequately account for this interfolded/entangled leader–follower-context ontology. Shamir et al. (2005) argue that separating cause and effect is very difficult given the way leadership research is approached. Judge and Piccolo (2004) go so far as to say that meta-analysis shows authentic leadership and transformational leadership are largely overlapping and that they amount to much the same thing. Van Knippenberg and Sitkin (2013) argue that we should, in fact, abandon transformational leadership theory (Van Knippenberg and Sitkin 2013), and by extension other similar theories. Going further Batistiˇc et al. (2017) say that conceptual progress is mostly being made by researchers on the fringes and that mainstream leadership research is overly focused on individual characteristics of leaders and insufficiently deals with the multilevel materiality in which leaders work. Examples of this fringe research that would be sympathetic to quantum ontology use phenomenology (Küpers 2007, 2013) and eastern (Case 2013; Yang 2016) philosophical frameworks. As Neubert et al. (2016: 905) found: [T]he relationships of servant leadership with creativity and with patient satisfaction mediated through job satisfaction were moderated by organizational structure such that the associations were enhanced under conditions of high levels of organizational structure ... High levels of structure combined with high levels of servant leadership yield the highest level of satisfaction, while the lowest levels of satisfaction result from combining high levels of structure with low levels of servant leadership or low levels of structure with high levels of servant leadership. Alternatively, high levels of structure uniformly relate to lower levels of creative behavior, an overall effect that is buffered slightly with high levels of servant leadership. Together, the findings support the hypothesized effect of structure enhancing the associations of servant leadership with nurse job satisfaction and creativity, while also indicating that high levels of organizational structure suppress both outcomes in the absence of servant leadership.

In other words, leaders’ impacts are clearly not only the sum of personal traits because context matters. Wisdom in leadership research is relevant to this discussion because it takes context seriously and de-centers the individual. Wisdom is

48

D. Rooney and S. Patra

also an indeterminate phenomenon. Integration, or harmonious interactions (Küpers and Statler 2008) are important in wisdom as a social practice but these interactions occur in a messy political world of resource constraints in specific times and situations (McKenna et al. 2009). A wise leader is wise because s/he understands the quantum-like ontology of the life-world and is able to adroitly work with it (McKenna and Rooney 2008). The complex dynamics that enable wisdom to be displayed in leadership practice through relational accomplishments and by overcoming the hindering pragmatics of life (Yang 2011), and the impact of culture, history and political economy on present day leadership practices (Oktaviani et al. 2015) are discussed in the wise leadership literature. Despite the multilevel/contextual complexity that leadership is practiced in, leadership research continues to take an individualistic focus. One result of this focus is that leadership theory continues to prosecute the idea of the leader who is something of a (moral) hero rather than a context constrained and boundedly ethical social agent, whose performance is determined largely by broadly sociological variables, and so, we argue, a significant theory–practice gap continues to thwart the impact of research on practice (Alvesson and Sveningsson 2003; Learmonth and Ford 2005). A different way of thinking about leadership research questions is exemplified in questions like what is the probability that “leadership emergence differs for males and females when they demonstrate the same pattern of behavior” (Lord et al. 2015: 280), which a QM approach can answer. The dominant methodological paradigm in contemporary leadership research is a quantitative one that has relied mostly on classical probabilistic statistics and classical objectivist ontologies, and is based heavily on limiting but largely unwritten assumptions (latent variables and latent constructs) implying that context is static and unambiguous, and that self-report data adequately accounts for context. Indeed, explicit discussions of ontological assumptions are rare in quantitative leadership research articles. However, recent critiques of leadership research designs (Anderson et al. 2017; Batistiˇc et al. 2017) include the important observation that too much emphasis is given to single level designs and, relatedly, that context is not well handled by those designs. The purpose of this article, then, is to propose an alternative form of quantitative leadership research in the form of quantum (like) modeling (QM). We argue for this because of the different ontological assumptions that quantum theory makes and because of the ability of quantum modeling to operationalize those assumptions quantitatively in meaningful and powerful ways. A clear advantage of QM theory is that its predictions can be empirically tested and are presentable in numerical simulations (Haven and Khrennikov 2013). Asano et al. (2017) show that applying quantum theory and empirical social science observations in context (e.g., in uncertainty) produces context driven frameworks that are more flexible in explaining non-trivial paradoxes like people making seemingly irrational choices that go against their or their organization’s best interests.3 The extant organizational 3

Such contextual utility models can show various effects like preference reversals, ambiguity aversion or attraction, all embedded in a single coherent framework. Our point is that a single coherent framework is critically needed in leadership decision theory also.

4 Remodeling Leadership: Quantum Modeling of Wise Leadership

49

research literature that advocates for the use of quantum theory has not yet explained how to use it to create new approaches to quantitative research. Another important set of questions relate to how leadership teams function in, for example, making strategic choices. Recently, quantum decision theory has been applied to probability-based problems and the role of shared knowledge (Aumann 1976). Such research considers how social agents in a group can disagree on posterior probabilities of events even though knowledge of prior events is shared by all group members and each person knows what prior beliefs that all group members hold. This literature uses common knowledge theory, where every agent knows a specific event, or knows the probability of it happening, and also knows that everyone knows that everyone knows the probability of that happenstance. In brief, the event is, as it were, a public knowledge. Clearly, though, the assumptions stated here are unlikely to hold in most social situations. Aumann (1976) provides a set-theoretic structure of the theorem, where, if two rational agents start with a common prior belief about the event, and update their beliefs according to a Bayesian updating model, then reach a posterior degree of belief (represented by a probability measure, strictly speaking a Kolmogorov set theory measure) about the same event where the posteriors are common knowledge to every agent, then there is no way that agents can disagree on the probability measures for the event. However, in workplaces this almost never happens casting doubt on Beyesian theory approaches to deal with social complexity. Coming back to reality, we often see examples in decisionmaking where even if common knowledge holds good, agents still disagree about their degree of beliefs, which, in turn, may lead to failure to achieve agreement. Thus, Quantum Decision Theory has been extended to understand ‘disagreements’ among agents, and demonstrates (Khrennikov 2015) that when probability computation and updates are based on quantum theory, rather than set theory, different solutions emerge, where rational agents have freedom to disagree (or simply come to different decisions) while keeping common knowledge intact. This approach allows for more complex cognitive and decisionmaking processes than does standard decision theory. We provide some basic quantum ontology outlined above but we must now look deeper to unpack some epistemic insights that bring new possibilities of quantitative leadership research.

Quantum Basics Quantum theory is useful to explain human action “by adopting a process-oriented approach that attempts to understand how different presents are actively created” from the range of potential outcomes that a context allows (Lord et al. 2015: 269). Quantum reality is set in a context of relationships and interactions between many variables. A very important ontological feature of quantum theory is that it deals with what is called ‘deep uncertainty’. In quantum theory, the fundamental or pure state of any system is represented by a ‘superposition’ (the sum of all interacting variables in a system prior to taking any measurements/observations, i.e., probability) of basis

50

D. Rooney and S. Patra

states. This ‘pure’ superposition is the context out of which emerge the events we experience as our social reality. This emergent subjective experience is called a mixed state. The potential for all basis states is contained in the Hilbert space superposition. The contents of a Hilbert space superposition interact with and influence each other creating a large number of potential outcomes from those interactions. This interaction process is called entanglement. Entanglement is a state in which “two or more quanta interact to form a composite superposition that results in a new, single quantum entity” (Hahn and Knight, in press: np.). In an organization, Hilbert space is very much a superposition of intersubjectivities, interacting in shifting patterns of relationships where the impact of interactions is fundamentally uncertain or indeterminate (McDaniel and Walls 1997: 369) because: In the quantum world, the problem is not that we do not have enough information about the present state of affairs or even the past state of affairs to predict the future. No matter how accurate or complete our information is, the world is fundamentally unknowable … When we try to know the world, particularly through measurement of it states, we come face to face with the Heisenberg uncertainty principle that says that if you measure position accurately, you must sacrifice and accurate knowledge of momentum (Herbert 1985, p. 68). Every attempt to know one attribute of a system reduces our ability to understand other attributes; this leaves us with a world that we can never completely know.

Most importantly, quantum uncertainty does not vanish with the addition of more information. It is important for this article that Aerts et al. (2013) have summarized a two decades of research on the correlation between quantum theory and human decisionmaking and cognition. These researchers say that just as quantum theory’s measurement entities (for example, observables like position, momentum, or the energy of particles) are influenced by the context of the measurements (measurement apparatus or the measurement environment as a whole) and deep uncertainty, so too is human cognition and decisionmaking. Aerts et al. (2013) have also demonstrated that quantum like correlation (known as entanglement) exists in human decision states. For example, one person’s belief state interacts (is entangled with) other peoples’ belief states. We can also explore much more challenging aspects of human life using QM. In one, study (Dalla Chiara et al. 2018: 78) the semantics of poetic and musical metaphor expressed in songs was conducted to understand how extra-musical meanings are created by interlacing musical ideas in the musical score and poetics devices in the lyrics as an example of quantum emergence from an indeterminate context. They argue that; [A]n important feature of music is the capacity of evoking extra-musical meanings: subjective feelings, situations that are vaguely imagined by the composer or by the interpreter or by the listener, real or virtual theatrical scenes (Dalla Chiara et al. 2018: 79).

A formal analysis using a quantum approach is possible because: As happens in the case of composite quantum systems, musical ideas (which represent possible meanings of musical phrases written in a score) have an essential holistic behaviour: the meaning of a global musical phrase determines the contextual meanings of all its parts (and not the other way around) (Dalla Chiara et al. 2018: 78).

4 Remodeling Leadership: Quantum Modeling of Wise Leadership

51

Given the complex and nuanced intersubjective dynamics that are necessary for excellence in leadership (Küpers and Pauleen 2013), there is clearly a place for a quantum approach in leadership research. Indeed, the logic of QM indicates that quantum theory’s mathematical and logical framework is very adaptable for social science. We believe leadership is a fertile ground to which to extend QM because of the indeterminacy of it as a practice and because we need to understand how to foster excellent or wise leadership by working with rather than against its deep uncertainty and unknowability.

Classical and Quantum Ontology To reiterate, we are not proposing a physical theory of quantum leadership. However, there is a growing awareness (Haven et al. 2017) that the mathematical, logical and ontological structures of quantum theory are compatible with the realities faced in social action, and this article extends this view to the deeply complex phenomenon of leadership. The basic conflict of world views between quantum physics and classical physics lies in conceptions of probability and locality. In classical, deterministic physics (from Galileo to Einstein) probability is understood to be, at best, a secondary concern and arises in classical thinking because the experimenter has an incomplete set of information about the underlying variables that create the external world. Relatedly, randomness and uncertainty are not central aspects of classical scientific ontology. Underpinning this assumption is the additional assumption that if one has full knowledge of reality everything is predictable. Quantum theory, on the other hand, interprets nature as fundamentally random; that is, there is an irreducible randomness to the universe, which is described by what is called uncertainty relations, where uncertainty remains no matter how much information we have (Birkhoff and Von Neumann 1936). Uncertainty in quantum theory is deep, and refers fundamentally to the superposition principle, where before a system is measured/observed it is in a superposed state of possibilities: only measurements/observations can alter the superposition and ‘collapse’ or crystallize it into a state that we observe as reality.4 Importantly, randomness is, therefore, what defines uncertainty in quantum theory. A related debate is the measurement problem debate. Classical physics assumes that a deterministic model of uncertainty underlies everything in the universe but that complete knowledge of the model is lacking and is yet to be discovered. In this paradigm, experimenters, therefore, have to hypothesize a hidden or latent and unmeasurable variable (as an assumption), which they place in their underlying 4

In quantum physics there are stylized uncertainty relations, for example, the product of momentum and position uncertainty measures are greater than or equal to h/2π, where h is Plank’s constant. In social science we can refer to the superposition description readily, for example, as in quantum decision theory, where deep uncertainty is described by the superposition of beliefs, which is defined in terms of density matrix operators (we present more detail on this formalism later).

52

D. Rooney and S. Patra

model of reality. Each successive study can (hopefully) yield information on everincreasing numbers of hidden variables underlying the deterministic reality. That is, a more complete description evolves as each hidden variable is discovered to gradually complete our knowledge of reality. Even chaos theory uses such deterministic philosophy, to explain what happens prior to the point of emergence. Quantum ontology, therefore, is, in many ways, the opposite to the classical, deterministic view. Social scientists also use this deterministic assumption to build their research designs that state assumptions and set out testable hypotheses. But what if these assumptions are flawed? Bell (1966) proposed that if hidden variables do exist, then certain inequality correlation relations among random variables in a quantum experiment should work according to standard physics theory. Bell’s experiments showed that quantum level behavior does violate the inequality relations in classical physics. Empirical validations of Bells inequality results have, therefore, ruled out hidden variable theories in quantum mechanics and raise interesting challenges for how we understand the role of information and knowledge in research design. Bell’s inequalities says that no hidden or latent underlying structure of reality exists and, therefore, that the predictive ability of quantum theory is not hindered because it does not use latent variables. In a social science context, where latent variables are commonly used, this is a potentially ground-breaking change for research design. Bell type inequality relations are readily observed in human decisionmaking experiments (Dzhafarov and Kujala 2016), and are clearly linked to the influence of contextuality in cognition (Aerts et al. 2013). Another implication of Bell type inequalities is that ‘entangled’ states exist between random variables. Very simply put, entangled states are composite states of at least two entities that cannot be conceived of separately (Aerts et al. 2013). Entangled entities are, for examples, two or more particles that need to be related to create some aspect of or entity in the observed world, and which interact with (or react to) each other to create this entity (Hahn and Knight, in press). If one were to alter one of these entangled entities, then all the other entities that are entangled with it will react instantly to the change and “take complementary states depending on the measurement of the first entangled element, as if they “knew” what type of measurement was performed … [and this] implies that single elements of an entangled system cannot be fully described individually, but bear properties that depend on their interaction with other elements and the properties of the overall system” (Hahn and Knight, in press: np.). Two or more objects that are correlated such that they interact with each other’s behaviors are entangled. Moreover, entangled entities do not need to be close to each other in time and space to influence each other. Thus, a person’s memories of being in a serious mid-flight emergency ten years ago on the way to Greenland is entangled with their decision today to not fly to a South Pacific Island for a holiday next year. This decision (or measurement/observation) is a result of ‘objects’ in the mind that are separated by many years and long distances but nevertheless are entangled (correlated). It is easy to imagine that complex patterns of relationships will emerge through entanglement. An outside observer can have full information about the system as a

4 Remodeling Leadership: Quantum Modeling of Wise Leadership

53

whole (i.e., the probabilities of mid-flight emergencies on long haul flights), but the sub-systems (the information that the decider used and their interpretation of it in relation to next year’s holiday) is at a random state. In purely rational, statistical terms, the decision not to go to the South Pacific is irrational and some other people with the same experience might also make the same choice but others will make a range of other very different choices. We might more accurately say that it is an emotional or anxious decision rather than irrational and that more information will not necessarily change that decision. Physicists describe entanglement as beginning with the larger system that exists in a ‘pure’ state of infinite probabilities and subsystems that exist in ‘mixed’, finite states. Physicists use the term pure state because this state is a superposition of the basis states in the given Hilbert space that contains all possibilities. We describe entanglement in greater detail later, but it is important to note that entanglement is often understood as the most striking difference between classical and quantum ontology but should be less controversial in social science. Entanglement allows deeper correlations between sub-systems than is allowed in classical probability theory. Leaders are deeply entangled in quantum-like systems and are parts of many patterns of relationships. Quantum modeling ontology assumes that reality is constantly changing and ambiguous, and that this indeterminate background or context is the platform on which the unique events of reality are enabled (or not). For example, if human behavior is based on each person’s ‘belief state’ and the ways in which an individual person’s belief state interacts (or is entangled) with the collective belief state of society will collectively produce specific behaviors. Many social scientists are comfortable with this ontology of becoming and intersubjective sociology (of knowledge). People do not often make decisions in everyday life based solely on fixed preferences. However, revealed preference theory holds that deviations from time invariant or context invariant preference patterns are irrational. Quantum decision theory focuses on context specific utility maximization (Aerts et al. 2018), and hence more variance, uncertainty and randomness in preference patterns. To pull these core quantum ideas together, we can say that quantum probability, entanglement, inequality, interdependence, randomness, and uncertainty are aspects of reality that decisionmakers and leaders face. In this regard, quantum theory and otology can contribute new ways to do leadership research by meaningfully accounting for this messy reality. For example, how was President Trump’s belief state able to interact (entangle) with the aggregate of American society’s collective belief state to make him President is a question quantum modeling can seek answers for by using quantum probability, entanglement, inequality relations, interdependence, randomness, and uncertainty. As we stated above, in quantum ontology, randomness is held to be an intrinsic part of reality, which means that randomness or uncertainty is not produced by incomplete knowledge, it is a state that is independent of knowledge. Thus, even with complete knowledge (if that were possible) randomness remains. The questions arising from this situation for research design is what do we do in place of standard quantitative methods and that relies on randomness and entangled variable in a context.

54

D. Rooney and S. Patra

A very important point to make at this juncture, is that a person’s reality is emergent and personal. By this we mean than a person is an observer taking his or her ‘measures’ or observations to create the information, meaning and knowledge that they use to navigate life. Social reality is, therefore, emergent. The reality we experience emerges from Hilbert space upon our observation. In Fig. 4.1 we show the observer with a pure Hilbert state space (above) of potentially infinite probabilities of quantum reality, which, in quantum parlance, ‘collapses’ (below) into an event (or interaction) within the mixed and finite state space of social reality. The observer is in this sense an interface between the two state spaces bringing an emergent social reality to life (in the bottom triangle) as he or she observes an aspect of the entangled pure Hilbert state space. Readers must keep in mind that is diagram is an over simplification because we make observations regularly across time and in the presence of other observers.

Observers, social networks, information, power relations, natural environment, built environment, etc.

Pure state system of randomly distributed probabilities

Observer (a leaders subjective belief state)

Wave collapse observer reality Observer belief state update and decision Observer behavior

Time Fig. 4.1 The position of an observer in state spaces

Mixed state system – Phenomenological experience (in a messy social reality)

4 Remodeling Leadership: Quantum Modeling of Wise Leadership

55

However, for the sake of clarity, we present Fig. 4.1 as a single observer at a single point in time. We now shift the focus to specifically social scientific applications of QM.

Quantum Modeling: Probability and Subjectivity We now discuss how probability and subjectivity work in quantum epistemology. Both probability and subjectivity work in different ways in quantum research compared to classical scientific epistemology. These important epistemic differences are, however, very useful for understanding social phenomena, and leadership in particular. Recently, as summarized in Khrennikov (2015), researchers have used quantum probability theory to model human decisionmaking. This research shows that human information processing and, indeed, the mind are non-deterministic because they are contextual and adaptive. Consequently, probabilistic information processing cannot be well described by standard models of probability. However, the Vaxjo approach was developed to mode this indeterminate process (Haven and Khrennikov 2013). Vaxjo theory5 uses additive perturbative terms. Additive peterbative terms are an extension of classical total probability expression. An example is as follows (a more detailed mathematical approach is presented in Appendix 2).6 Let’s assume we would like to predict the total probability of event A happening given event B, where B is dichotomous to A and therefore has two values B1 and B2. Based on standard Bayesian probability theory we would express it as: P(A|B) = P(A|B = B1) + P(A|B = B2), however the quantum probability the expression would be: P(A|B) = P(A|B = B1) + P(A|B = B2) + 2ρ(P(A|B = B1)P(A|B = B2))1/2 where the additive terms are perturbative. Specifically, ρ is a phase angle (as elaborated in the appendix). Again, since quantum theory is used to derive such a formula, the result resembles the quantum formula for total probability, which is well established in quantum mechanics (please see appendix for a detailed working). Quantum Bayesian modeling (QBism) (Caves et al. 2002) is another approach. It attempts to interpret the basic quantum state of any system as subjective and contextual, hence its probability measures are also subjective. There is a subtle difference between Vaxjo type interpretations of QM and QBism’s interpretations. While the former uses statistical interpretations based on average results of an ensemble of identical states, the latter is concerned with how information processing and decisionmaking happens rather than the mean results. In other words, a personalist version of information processing and measuring probabilities is used in QBism. 5

Vaxjo interpretation of the modified formula for total probability has emerged out of efforts by scientists at Vaxjo conferences on quantum foundations since last twenty years (Khrennikov 2023). 6 The formula presented here is originally motivated by the superposition principle in quantum mechanics, as discussed in the paper, and this formula famously appears in the probability computation of ‘double slit experiment’in quantum physics, which is so well emphasised in Feynman lectures on physics Volume 3.

56

D. Rooney and S. Patra

Thus, Khrennikov (2015) proposes that QBism is a general decisionmaking model. However, there are complexities in computing the formula for total probability in that model. QBism agrees with the personalist Bayesian probability theory as pioneered by Ramsey (1997), De Finetti (1974) and others. De Finetti (1974) suggested that there is no such thing as probability because there is only a personalist degree of belief. In this approach 0 and 1 probabilities are degrees of beliefs. However, QBism holds that the measurement outcome is not pre-existent, rather it is created in the act of measurement. In social behavior there are numerous instances (Yearsley 2017) where measurement effects are observed; for example, order and conjunction effects, and disjunction fallacies, which we discuss later in the paper. Thus, Chinese whispers-like errors can arise in decisionmaking processes as interactions between agent’s (belief states) within the information context unfold. Inaccuracies in information transmission will likely occur and this process can be described effectively by Quantum formalism.7 But what does this formalism look like in social science research practice?

Using Quantum-Like Modeling in Social Science All the QM examples that follow are based on research on cognition, decisionmaking behavior under uncertainty, or decisionmaking in specific contexts. Each of the examples has implications and applications for leadership research. The basic idea we explore in this section is that in quantum theory the context of measurement of any property influences the outcome. This is important for this article because social, political, economic, cultural and other contextual factors deeply influence social behavior and the ability to realistically model such complexity’s’ impacts on behavior are important. In much of the social sciences, when contextual behaviors deviate greatly from an ideal, for example, when leaders practice narcissistic leadership rather than, for example, servant leadership, those behaviors are considered deviant, irrational or foolish outliers. However, QM provides explanations for such behaviors based on a set of ontological assumptions that render these ‘outliers’ as natural parts of reality, even if they are less than desirable. Clearly, some leaders are eccentric to the point of deviant but there are so many examples in history of undesirable leader behavior that research should not ignore because they are, in fact, part of what is within the 7

In quantum theory measurements are described by projection operators, or projection postulate, act of measurement is equivalent to projections of the initial superposed state into a definite Eigenvalue, probability of such a projection is provided by the Born’s rule. Such projection operators live in the Hilbert space of the system and are orthonormal to each other. There are other projection operators which are named as positive operators, which describes ‘unsharp’ measurements. In decision theory terms, orthogonal projection operators will project the intial belief state to a specific final state immediately after the measurement (for example immediately after a question is asked, where the act of asking question is measurement), where as a positive operator will project the initial superposed belief state into an unsharp state, for example ‘may be’ type of response.

4 Remodeling Leadership: Quantum Modeling of Wise Leadership

57

range probable (if not good) leader behavior. Moreover, unwelcome leader behavior is something that we need to know more about. Although we certainly see enough unwelcome or unsavory leader behavior in the world, the questions remain, what responses should we make to it and why does it keep happening? We still do not have good and actionable answers to such questions. We, therefore, now turn to discuss empirical QM research approaches to some specific context-shaped behaviors that are relevant to leadership beginning with the sure thing principle and uncertainty avoidance.

Non-optimal But Normal Behavior The sure thing principle is a central assumption in standard decision theory. Simply put, it means that rational, utility maximizing agents do not include irrelevant information while making decisions. This, of course, is an unrealistic assumption because people use irrelevant information quite often. For example, take Bob, who is deciding to buy a house. In one scenario, his information set contains the information that Alice will win the next presidential election, and because of this, Bob decides to buy. In the second scenario, the information set contains the information that Alice will lose the election, and Bob still decides to buy the house. In this example we might conclude that Alice’s state does not affect Bob’s decision at all. However, using QM, Haven and Khrennikov (2013) show that under uncertainty, for example, when there is no information at all about the election, Bob may behave differently. Thus, in the face of uncertainty about the outcome of the election, Bob decides not to buy. In standard decision theory, such a behavior is considered irrational, but without any deeper explanation. Such behavior might be irrational, but it is quite normal in uncertain conditions. Going further, and similar to Bob’s response to uncertainty, in standard decision and game theory, the ‘irrelevance of irrelevant alternatives’ principle is well known. In the prisoner’s dilemma (Rasmusen 2007), for example, strategy equilibrium theory suggests that irrespective of what the other player chooses, the first player should always choose to defect; that is, not to cooperate. However, real people behave differently when the same game is played in an uncertain context; for example, when players have no idea about the move of other players or the players feel a sense of loyalty to the other players. Haven and Khrennikov’s (2013) QM research lists many ‘deviant’ or non-optimal behaviors of players that cannot be described by standard decision theory mathematics. QM’s Quantum Decision Theory (QDT) looks at these examples from a perspective that enlarges our typical utility framework to accommodate normal but non-optimal behaviors? More important, however, is the difference between standard decision theory models and QDT is that the total probability formula (FTP) is different in QDT. In QDT the additional perturbation term modifies the FTP to account for the impact of contextuality (Khrennikov and Haven 2009). Hence, contextual factors are quantifiable in a probabilistic sense and the ‘non-optimal’ behavior of agents under novel

58

D. Rooney and S. Patra

contexts such as uncertainty are meaningfully measurable. Leaders constantly face uncertainty and dilemma and so including the impacts of uncertainty in leadership research is important. Even though prisoners’ dilemma-like scenarios abound in a leadership contexts, the failure of the sure thing principle suggests that people behave non-optimally, and frequently do not choose the cooperation strategy. Most importantly, Haven and Khrennikov (2013) show how decision states of agents can be reconstructed based on a QM theory framework that more realistically models complex social behaviors for large ensembles of people. Using such methods in leadership contexts, we could explain or even predict, for example, co-operation among leaders, or leader and followers in scenarios where standard game theory would fail to capture the social and emotional complexities of reality. Hence, leadership is a fertile ground in which to test QDT predictions, more specifically we can deliberately include so-called irrational behaviors by leaders in analysis. Quantum modeling also allows research to do more complex studies of leader social cognition.

Order Effects in Human Cognition Order effects research aims to understand how the order in which things happen to someone influences their choices. Essentially, researchers observe how peoples’ responses to questions differ when the questions are asked in different orders (Haven and Khrennikov 2013). However, the effects of different orderings are not simple. Bruza et al. (2015) who have pioneered the use of QM modeling to explain order effects in cognition show that if questions are asked in random orders and the questions have positive operator representations (each question is unrelated to each of the other questions), then it is not automatic that such operators will commute. In other words, if a change in the order of operators (questions) produces different output states (answers), we would conclude there is an order effect.8 Further, if the questions are represented by non-commuting operators (that is, they have mutually complementary semantics), responses by agents will differ if the order of questioning changes because each question is semantically conditioned by (entangled with) each of the other questions. The implications of these kinds of order effects are potentially profound, yet we know little about them in leader behavior. It is useful to explore the mathematical logic that QM research uses to understand order effects. For social science research, it is important that the measurement process in quantum theory occurs in two stages. First is a preparation state, which corresponds to the initial belief-state of a person, such as a leader. The preparation state (of beliefs) is represented by density matrices. In mathematical language, a density matrix is a description of information. In our case, information captured about a person’s belief-state, or an ensemble of peoples’ belief-states, which are represented as direct 8

Readers can be referred to a formal mathematical literature on the commuting and non-commuting observables or questions in decision theory.

4 Remodeling Leadership: Quantum Modeling of Wise Leadership

59

products (Tensor products) of initial matrices representing the pure states: ρ = |X> = a|0> + b|1> , where 0 and 1 are the basis states of the underlying two dimensional Hilbert space in this example, which may mean, for example, a down and up state for any future event happening, and |a|2 and |b|2 are the probabilities/ degree of beliefs (according to a quantum probability frame work) of such occurrences. Here, if the modulus square of a and b is ½ each, then both 0 and 1 states are equally possible, hence, this superposition reflects ignorance about the system. Importantly though, 0 and 1 states are symbolic because there can be as many states as the number of basis states in state space (that is, Hilbert space), and this superposition description is actually based on knowledge about the possible states before we measure in the next stage. For decision theory purposes the superposition state of beliefs is the state the belief system is in before any measurement is performed (i.e. before dealing with any questions related to a person’s beliefs). There is a fundamental ‘uncertainty’ when in this unmeasured/uninterrogated superposition state. Belief states can change, that is, they are updated over time, and so a model of belief states needs to measure updating. Researchers can ask participants to answer a question regarding 0 or 1 states, where any dichotomous choice variable in an experiment is represented by 0 or 1. For example, 0 and 1 can be belief states of agents in a market, where 0 is the belief that an asset’s price is decreasing and 1 is the belief state that the asset’s price is increasing (Khrennikova and Patra 2019). Based on participants answers, we update their belief states. This is done by making the superposition state collapse to 0 or 1 by recording participants answers to questions. In more technical quantum theory terms, then, there are two phases; first, is the state prepared for experiment, say the belief state of the agents before they face questions, and, second, a random ‘collapse’ state that results from the act of measuring (observing/answering) the initial ‘superposed’ state causing it to change immediately to its final state (0 or 1). Probabilities are ascertained from observed frequencies. In leadership decisionmaking contexts, this could correspond to a cognitive experiment (Dzhafarov and Kujala 2016) where the leader provides questionnaires to respondents, and probabilities of respondents answers are found by calculating frequencies of choices by the respondents. The mathematical psychology literature (see Yearsley 2017) has numerous relevant experimental design examples. The collapse postulate is geometrically described as an action of a projection operator (in our case, a question) on the ρ. However, these projection operators are orthogonal and form a complete orthonormal (scalar products of ith and jth projection operators = 0) set in the given Hilbert state space. Going further, in real decision scenarios, there is always noise in the system, which means decisionmakers may erroneously choose, say, the option 0 when they actually believe 1. Such errors in decisionmaking cannot be captured by simple projectors, hence a positive-operator valued measure (POVM), or positive semidefinite projector is used to capture any ‘error-prone’ or imperfect decisionmaking (Yearsley 2017). Hence, if questions are represented by non-commuting operators, it is not difficult to see how the final output states or responses by agents will differ when the order of

60

D. Rooney and S. Patra

questions changes. Non-commuting operators are operators that represent the observables that cannot be observed or measured simultaneously with indefinite precision; for example, where [A, B] is non 0, or where [A, B] = AB − BA. Commutation relations are building blocks of any operator theory. The above bracket is known as the commutation relation between two operators, A and B. A and B in quantum theory represent observables; for example, A can be a position operator and B can be a momentum operator. In classical logic, AB-BA can only be 0 or, in other words, both observables can be observed simultaneously. However, in quantum theory this assumption can be relaxed. In quantum physics the non-commutation relation is famously expressed through Heisenberg’s uncertainty relations between conjugate variables like position and momentum of sub-atomic systems. We can think of any pair of random variables representing different tenets of leadership, which may not be compatible with each other, for example, unethical and ethical leadership styles. In the case of decisionmaking models, operator representations of observables to be measured must be built from scratch. In QDT these operators represent the questions which when asked to change the belief states of agents from their initial belief state and make them collapse to a new belief state. In mathematics, this kind of operation is called an Eigen value or Eigen state link. However, there are still challenges such as, for example, when questions are repeated? Will there still be an order effect (Aerts et al. 2018)? The implications of order effects are potentially profound, yet we know little about them in leader behavior. Beyond leader decisionmaking, order effects have implications in, for example, leader communication and how it influences other decisionmakers in an organization. Standard order effects theory predicts that agents will choose different options if the order of questions is changed. However, the findings from QDT shows that because changing the order of questions also changes the context in which behaviors happen, it is not just question order that drives change but also the changed context. Mathematically, operator representations of observables (which do not commute with each other) are an elegant way to analyze such change in behaviors. However, we can tease more mathematical insight out of the quantum perspective by discussing conjunction and disjunction effects.

Conjunction and Disjunction Effects Based on probabilistic behavioral models, we find regular violations of standard probability axioms, for example, P(A & B) > P(A) + P(B), the conjunction fallacy, or the opposite of it (the disjunction fallacy), i.e. P(A U B) < P(A and B) A and B being two events. The Kolgomorovian (2017) measure theory also does not accommodate such violations. However, if belief states are described by the superposition of basis states in Hilbert space, and measurements are represented by projections onto specific Eigen sub-spaces, and the probabilities of actualizing one final state (a behavioral outcome) is given by Born’s rule, then such probability inequalities can be justified. Sequential choices can then be described by sequential measures/ projections. The

4 Remodeling Leadership: Quantum Modeling of Wise Leadership

61

implications of conjunction and disjunction effects in leadership are significant. For example, what is the relationship between doing management and doing leadership? And what is the difference between understanding leadership as a role, a position on an organizational chart, and leadership as a way of being? We are still unclear on such effects. Hence, we can think of variables as A’s and B’s, as in the above expressions, and then give scores to the variables based on a Likert scale. Having done this, we can study the correlations between variables. Any conjunction or disjunction effects as expressed via the joint probabilities might throw light on how various aspects of leadership are perceived by agents in an organizational context. In the leadership context, we need to have variables that reflect and measure A/B (doing management = managerial observables, or doing leadership = leadership variables or attributes), which can then be provided as a questionnaire with a Likert scale for responses. Response frequencies can be used as probabilities, which can then be used to detect conjunction and disjunction fallacies. Such experimental designs might provide new insights into how leaders’ influence their organizations and vice versa.

Heisenberg-Robertson Inequalities Importantly, Heisenberg’s uncertainty relation in the form of Robertson inequality is used to quantify uncertainty in decision-making (Pothos and Busemeyer 2013). As mentioned above, uncertainty is a challenge; for example, risky situations are used as proxies for uncertainty, but this is inadequate. Risky situations are situations with known or subjective probability distributions, whereas ambiguous or uncertain situations are where such probability computations are non-trivial. One example is the Ellsberg two urns paradox (al-Nowaihi and Dhami 2017; Ellsberg 1961). There are two urns of red and blue balls, in one of them the proportions of red to blue are known and in another the proportions are unknown. Participants have to place bets on what color ball they will take out of an urn. When agents are asked to choose one of these urns they tend to choose the urn with known proportions over the urn with unknown proportions. Such behavior is known as ambiguity aversion and is covered by the Bob and Alice example (above). To explore ambiguity aversion, mathematical psychologists (Pothos and Busemeyer 2013) have used Hermitian operator representations of incompatible questions asked to respondents. In such models, mental states are dependent on the mutual uncertainties of incompatible questions. In these models, questions are represented as operators: as self-adjoint projector operators, and the actions of such operators are used to predict the mental states of participants as Eigen sub-spaces of the initially superposed belief states. To recapitulate, cognitive quantum-like modeling provides a Hilbert space representation of belief states, where the belief state is considered to be a normalized vector in state space, or a general density matrix representation that shows a mixture of different pure states. Hence, it is possible to use Heisenberg’s uncertainty principle to describe state space distributions. Recently some inequality relationships have been studied (Bagarello et al. 2018) that reflect the behavior of agents under

62

D. Rooney and S. Patra

uncertainty. Such inequalities can be used in leadership decisionmaking, since leadership is always exposed to uncertainty as they act within their context: a context that can both enable and constrain behavior.

Contextuality and Randomness We need to look more closely at context to properly understand QM. Dzafarov and Kujala (2016) have modelled contextuality in human behavior based on analogical mathematics within quantum theory. Contextuality models are particularly effective when outcomes are binary and some factors in the measurement context interact with the measurement process to influence the outcome. For example, in the cognitive experiments we have already discussed, if the order of the questions is changed, or new questions are asked along with the target question, responses vary widely. A famous example is Linda, the bank teller. When questions like, is Linda a feminist? are added in different orders with other questions, research participants answers are different because the context has changed. That is, the research participant can be directed to draw on different contextual elements in Hilbert space that influence their answer. Other questions that draw the respondent’s attention to the history of women in traditionally male work domains (like banking) may elicit answers of yes, Linda is a feminist. Recently, researchers (Basieva and Khrennikov 2017) have measured contextuality in human decision data. In behavioral experiments, responses to questions in different contexts can be treated as random variables. Researchers have found two types of influence on the distribution of such random variables (i.e. the probabilities for the random variables attracting yes/no responses). Direct influence is observed when change in the distribution of one response changes as the context is varied. However, when direct influences are eliminated from the experiment, residual ‘true’ or deep contextual effects may be brought to light. Dzafarov and Kujala (2016) have demonstrated such contextuality in decision making. This is a fast evolving area of research and leadership is a fertile ground for future empirical tests of the contextuality hypothesis. For example, do leaders make better decisions when in their office or when they are traveling in foreign countries? Also, how does leadership decisionmaking change when the information environment changes? For example, when a leader formally studies business or leadership and, therefore, is exposed to business school information environments, do they change significantly? Later we provide a simple quantum field theoretic framework to study such questions.

4 Remodeling Leadership: Quantum Modeling of Wise Leadership

63

Emergence of Concept Combinations Through Entanglement At this point, we need to enlarge our explanation of entanglement so we can usefully apply it to leadership. In physics terms, we can think of a system that contains two particles and that these two particles collide with each other. After collision each particle is separated from the other. However, the pure state of the system remains a Hilbert space superposition of wavefunctions of individual particles, that is, subsystems. In this scenario, if observers, Alice and Bob, measure any property of individual subsystems, for example, the direction of spin of the particles, even when the particles sit vast distances apart, then as soon as one measurement is taken, say by Alice on her particle, the result of the measurement on Bob’s particle is already determined. It is important to note that measurement of each subsystem is random. Thus, for both for Alice and Bob the probability of observing upwards spin or downwards spin (if we assume that there can be only two orientations of the spin of particles) is 50%, and, further, it is not possible to know the spin direction before measuring. Given these probability conditions, subsystems (that is, the individual particles) are in a random or mixed state. Entanglement is certainly ‘non-classical’ since classical correlations (say between Alice’s and Bob’s systems) can never account for the instantaneous and unobservable ‘communication’ between the two particles. We emphasize here that entanglement actually does not mean any instantaneous travelling of signals for communication, which would certainly mean moving faster than the speed of light, but neither is it the same as classical correlation. As strange as this communication seems, it is, nevertheless, easily demonstrable in experiments and, indeed, is used for quantum computing, atomic clocks, MRI scanners, and GPS navigation systems. Even though the communication in entanglement is unobservable, it is highly accurate, and it can be understood by using information theory, which makes the link between quantum theory and social science research possible. Von Neuman (2018), one of the founding fathers of quantum theory, proposed the Neuman entropy concept: which holds that if the state of a system is denoted by the density matrix ρ, that can be a pure state or a mixed one, then the entropy measure is ρLNρ, where LN is the natural logarithm. Aerts et al. (2018) argue that we need to consider the system (Alice + Bob’s in this case) as one entity, and that any measurement of the systems is a measurement of the whole system. In the appendix we have provided a simple mathematical description of entangled states. To better understand this, we draw on quantum cognition research. The Brussels group (Aerts et al. 2013) are pioneers in quantum cognition entanglement of combinations of concepts/ideas. They argue that potentiality and contextuality in cognition is analogous to a quantum system. In quantum cognition, experimental context interacts with the system to influence the result. Social actor entanglement at its simplest is a system composed of, say, Alice and Bob, who are each sub-systems and as sub-systems act as agents who take sub-measurements; that is, they each make decisions or evaluations. At this simple level of explanation, decisions made by Alice or Bob are random. This randomness is because before Alice

64

D. Rooney and S. Patra

does her measurement the outcome is uncertain and the same goes for Bob, but as soon as one of them makes a decision (a measurement/observation) and, thus, obtains a result, the result of the other (Alice’s) sub system is fully determined because they are entangled systems, including sub-systems and therefore they mimic each other’s changes instantly even if they do not know what each other has decided. For example, if we have a superposition of possibilities, say that an asset price can be up or down, this belief state is a superposition of up and down beliefs and is a pure state because no outcome is yet known. However, once measured (that is, someone decides if it is up or down) the superposition collapses to a final state (an actual belief). Furthermore, pure states become entangled with the environment of different people with different beliefs and different information, they become mixed states, and are entangled. Fake news on Facebook is a good example of an information environment, which may influence the pure belief states of agents who read it and use it for final decisionmaking, creating large scale shifts in behavior. The stock market is another example. When large numbers of people decide to sell shares based on similar but new beliefs, share prices might plummet quickly.9 The standard description in physics is, as we mention above, that measurements must be treated as joint measurements over a subsystem, and added to that, the whole system is always at a pure state, whereas the subsystems are at a mixed state (Susskind and Friedman 2014). An important implication of this is that we have full information about the system as a whole. In quantum theory, full information means that all the information is available or, more accurately, is accessible for observation even if it has not been observed yet. Going even further, we can think of an ensemble of many pure states. For example, in decisionmaking experiments, when many pairs of decisionmakers are performing the same set of choice-makings. Thus, pure state systems are pure in the sense that they are uncollapsed, or unobserved (yet), and have not been converted into a semantic entity or idea or data point. They are, as it were, ‘unsullied’ by observation. Sub- or mixed systems, however, are random and unknown because observation has created semantic entities that are unstable. Subjective interpretations and interpersonal communication introduce interpretive errors and idiosyncratic meanings that are the foundation of instability. The Chinese whispers game is an example of this randomness and instability. It is important, though, that Aerts et al. (2018) have demonstrated that there can be two levels of entanglements in decisionmaking; (1) between-states entanglement and (2) between-measurements entanglement. Aerts et al. (2018) suggested that either the belief states of the decisionmakers can become entangled or the belief states of the agents can become entangled with the measurement process, which is analogical to the workings of quantum physics, where contextuality of experiments directly influences the outcomes of measurements.

9

In finance for example, there is a wide literature on soft and hard information: soft being Facebook like environment which is less verifiable and hard being Balance sheet like which is more readily verifiable.

4 Remodeling Leadership: Quantum Modeling of Wise Leadership

65

Interestingly, there are three main mathematical conditions that have to be violated to demonstrate contextuality and entanglement. Most importantly, each of these conditions are readily violated in the social world. Those three laws are the: 1. Law of separability: explains whether the probabilities of different events happening can be expressed as products of individual probabilities. This is the standard mutually independent event test. The violations in this test point to a correlation between events or in this case choices made by respondents. 2. Law of marginal probability: is a mathematical extension of the first law. In this law, we can say that A has two values A1, A2, then for a context B P(A|B) = P(A1|B) + P(A2|B). However, the total probability formula in quantum theory is fundamentally different from this expression, since it contains interference terms. We have a detailed note in the appendix about the emergence of extra additive interference terms in the formula. 3. Clauser, Horne, Shimony, Holt (CHSH) inequalities: is perhaps the most used and important inequality type for demonstrating deeper correlations between events than cannot be predicted by classical probability theory. Following Aerts et al. (2018) we can have two dichotomous variables, A and B, such that A can have values (A1, A2) or (A1' , A2' ) when A is changed to A’, B can have values over (B1, B2) or (B1' , B2' ). Based on the classical probability theory it can be shown ():−2 < CHSH < + 2, where CHSH = E(A, B) − E(A, B' ) + E(A' , B) + E(A' , B' ), where E(.) are the expectations or joint probability values. Fundamentally, if the CHSH measure in an experiment violates the range then the events are correlated with each other in a more profound way than in classical probability theory, and it violates the predictions of Kolmogorovian measure theory. If all such inequalities are violated in cognition experiments (as shown by Aerts et al. 2018), then, there is high degree of entanglement in decisionmaking. Which means that there is entanglement not only among states/ events but also among measures. Aerts et al. (2018) have used the same approach for decisionmaking experiments, where A' s and B' s are concepts; for example, placing animals and acts in two different sets. In one set we have animal names and in the other we have acts which may or may not be associated with the animals. The A' s and B' s will, therefore, have different pairs of values. Frequencies or E(.)s are computed based on the frequencies of joint choices made for each A, B pair by respondents, and then CHSH is tested. Violation is always detected.

Emergent Cognitive State The above empirical and theoretical considerations however also indicate that human mind is more complex than either quantum logical or classical logical. There is no clear ground to assume that cognition would always be represented by a full quantum formalism. Hence it would be better (Geneva-Brussels approach) to conceive of a

66

D. Rooney and S. Patra

complex description of mind, a combination of quantum logical and classical logical states. Mathematically, such description can be provided by ‘Fock’ space representation, which is a more general state-space with direct sums and tensorial products of individual Hilbert spaces. More specifically; )1/2 iϕ ' ( |AB>m eiϕ (|A> + |B>)/21/2 + 1 − m2 e |C> where |a> is in H |b>is in H, and |c> is in H ⊗ H, M[0, 1]. The second term is a tensor product representation which can be thought as a product state which might be used for satisfying classical logic inequalities, where as the first term is a superposition representation which might be used for representing deviations from classical logical inequalities, inequalities refer to the basic set theoretic probability rules. Hence the general state |AB> is rather a superposition of two: equivalently a Fock space representation. Human mind is more closely like this: emergent.

Modeling Wise Leader Interaction with Context We need leaders to act wisely and leadership research needs to account for wisdom. Wisdom, however, is one the most challenging social science constructs to research but it is ideally suited to a QM research design. Based on the above exploration we further summarize some specific directions in which such modelling might prove productive for leadership research where the goal is to develop excellence, that is, wisdom, in our leaders for the benefit of the planet. To do this, we draw on the multilevel social practice wisdom (SPW) framework (McKenna et al. 2009; Oktaviani et al. 2015), which has been used in the context of leadership research and translates well to QM research. Social Practice Wisdom understands wisdom as excellence in social practice the depends on integration of (1) Qualities of mind, (2) agile, transcendent and reflexive reasoning, and (3) ethical purpose and virtuosity in one’s everyday life, including leadership. This complex and indeterminate integration produces (4) praxis (wise practice) when successful and it (5) creates short and long-term positive change for the conditions of life on our planet. We now briefly consider each of these five theoretical elements in turn. 1. Qualities of mind and consciousness: An aware, equanimous, compassionate, humble, and actively open mind with an integrated habitus of dispositions that drive insightful and virtuous action. This involves mindfulness, empathy, nonattachment (distancing), acceptance, and self-awareness to understand uncertainty and the relativities of life, including conflicting values, identities, cultures

4 Remodeling Leadership: Quantum Modeling of Wise Leadership

67

and politics, as well as imperfect knowledge. This is a complex ontological constellation and there is no formula for predicting how to integrate these factors to produce wisdom in any given situation. The cognitive, affective and cultural context is clearly vastly complex and it seems misguided to treat wisdom as a radically parsimonious version of these factors. Quantum modeling is the best opportunity we have to embrace this complexity quantitatively. 2. Agile, transcendent and reflexive reasoning: Reflexively integrating knowledge, including aesthetic knowledge (direct, embodied, sensory, non-rational knowing and conceptual knowing), transcendence (e.g. creativity, foresight, intuition, trans-conceptuality [non-linguistic knowing]), different perspectives, and clear insight to adroitly deliberate and judge to assist transformative understandings of a situation despite uncertainty and ambiguity. The creative, meaning-making, learning, deciding, and judgmental aspects of wisdom are clearly non-trivial. Given the breadth of mental qualities that wisdom needs to be able to draw on, QM, by understanding them as existing in multidimensional Hilbert space and becoming entangled in the act of reasoning can begin to unpick the hitherto very difficult to access empirically mental dynamics of wisdom in leaders. 3. Ethical purpose and virtuosity: This includes virtues, ethical competence, and the ability to understand and act positively in response to people’s emotional, social and material needs. Furthermore, it entails ego transcendence and virtuous alignment of values with social behavior; and insight into the human condition and shifting social relations to find the right and virtuous thing to do at the right time. Self-transcendence and working to a higher purpose and critical in the ethicality of wisdom, this is by definition about phenomenological entanglement, through shared consciousness, communication of ideals, and culture. The complexity of this kind of correlation of entangled beliefs is challenging empirically and analytically but QM presents as a good candidate for moving the wisdom and leadership research effort forward by meeting the complexity without a reductive epistemology and excludes empathy, for example. These first three qualities and abilities recursively interact with each other as a habitus (or system of dispositions) to create the conative impulse for an embodied wise praxis that leads to excellent outcomes that improve the conditions of life. The degrees of freedom that necessarily apply to this three-part integration in the attempt to be a wise social practitioner, a leader, is vitally important to understand, yet we have not sufficiently developed the methods to do this in leadership research. 4. Embodiment and Praxis (or mastering wise action): Drawing from one’s habitus of dispositions to creatively, responsively, and decisively embody and enact wise performative skills in a situation. Wise performance draws on experience and understanding and is based on judgements that are executed and communicated in a timely and aesthetic way. This involves sensing and knowing why, how, and when to adapt to the surroundings and why, how, and when to change them, and how to astutely make necessary trade-offs. The very idea of habitus makes it clear how important context is to social practices like leadership. Habitus speaks

68

D. Rooney and S. Patra

directly to the quantum axiom that the events experienced as reality are emergent properties of contexts. 5. Outcomes that improve the conditions of life: This involves galvanizing, purposeful leadership and artful communication to effect virtuous change with exceptional outcomes. Creating positive cultures and sustainable communities are central to this. Ultimately, we need leadership to be a significant driving force to creating improved conditions of life. But as researchers, we might be humble enough to say that we still need to do better in assisting this process. Indeed, research, and, therefore, researchers, can understand themselves as leaders. We argue that part of that leadership role we can play is the relentless pursuit of new and more suitable approaches to research that will enable us to be those leaders. Quantum field modeling of decisionmaking enables analysis of leaders’ interactions with the information environment (Bagarello 2015; Khrennikova and Patra 2019). Quantum field theory is useful for describing instantaneous interactions of a decisionmaker with their information environment. In physics, quantum field theory (QFT) integrates special relativity theory and non-relativistic quantum mechanics. For this article, QFT is of interest as a mathematical toolset that focuses on creation and destruction operators and their commutation rules. Although Bayesian learning models are used to research adaptive decisionmaking, they have limitations (Haven et al. 2017). An important advantage for quantum field theory is that it can accommodate the large number of degrees of freedom in the information environment (which in our context includes many different categories of information: hard information which is verifiable, soft in formation which is less verifiable, media, noise, etc.), and then describe how individual decisionmaker’s belief states interact with the environment. Technically, we can imagine a decisionmaker’s initial belief state as a pure state that is a simple superposition of a few possibilities. However, this state is irreversibly correlated/ entangled with the information environment as soon as the observer queries it and makes semantic sense of it. Over time, an updated steady state evolves as beliefs are modified through learning. Learning is modelled via a decoherence mechanism that collapses a pure superposed state to a mixed state by building an operational theory based on quantum field theory tools. Any pure decisionmaking state has to interact with the environment. Hence the evolution of the pure/ isolated state ρ0 will, in general, be non-unitary: ρ(t) = U(t)ρ0, where U(t) = exp (−itL) (1) with L being the generator of GKSL10 equation (a Lindblad equation that describes the non-unitary evolution of the system). The very equation describes the adaptation of the isolated system to the surrounding environment called the reservoir or R, with large degrees of freedom (indicated by a parameter K). Hence the direct way to study the dynamics is to set up the L function and use Heisenberg dynamics: d/dt(ρ(t) = ([H, ρ] +L}. This formulation is interesting since 10

These equations are known as Master equations in quantum theory, which describe generally how a systems state evolves over time with interactions and with the information environment embedded in the equation’s parameters.

4 Remodeling Leadership: Quantum Modeling of Wise Leadership

69

L, the so-called super operator, which maps density matrix to density matrix, contains environmental d.of. If we consider the pure state of the decisionmaker as S, then the separable S+R (that is, sub-system or our decisionmaker (S) plus the reservoir (R)) state space has a unitary evolution as a whole, which is provided by the Hamiltonian of the compound state. However, the interaction between S and R induces entanglement which makes the compound state non-separable. Hence, the state of the subsystem S becomes mixed. Furthermore, to obtain information for S we then need to take the partial trace for all degrees of freedom of R. We then study the dynamics of the subsystem R with the non-unitary evolution (1). If Alice is a leader, then Alice’s pure belief state is captured by ρ0 = Iϕ> and is the pure, uncertain state described as superposition of I0> and I1>, where I0> can be a no response to the dichotomous question (or the observable here, say A) and I1> the response, yes. The reservoir, or R also comprise of many agents like Alice, who are faced with the same A question, which introduces a large number of dichotomous degrees of freedom. Hence in the state space of Alice, a 2D complex Hilbert space, I0> and I1> forms the orthonormal basis vectors. The Reservoir, or R, also comprise of many agents like Alice, who are faced with the same A question, hence also comprises of dichotomous degrees of freedom. Hence, in the state space of Alice, a 2D complex Hilbert space, I0> and I1> forms the orthonormal basis vectors. Alice’s decisionmaking process (subsystem S), or the R is described in terms of creation-annihilation operators, a, and a* for Alice and b(K), and b*(K) for the bath/ R. K being the degrees of freedom of the reservoir. The anticommutation algebra for the operators (Fermionic operators as in QFT) is given by {a, b} = 0, a2 = b2 = 0. Where {a, b} = ab + ba. The operations of a, a* on I0> and I1> is standard: a*I0> =I1>, a*I1>=I0>, and so on. Again the initial conditions are: a* (0) = a*, a(0) = a. Hence, we come up with the representation of A, or the question posed to the agents as a number operator: N= aa*, where the eigen values of N are 0 and 1, authors categorize such an operator as decision operator. Where the average of the decision operator N(average) = (tensor product with I). This average is with respect to some initial states of the compound system (RUS). Since the agent’s belief state is entangled with R(K), the density matrix ρ(t) can be obtained through partial trace over environmental degrees of freedom: TRKR(t), hence, = TRρ(t)N. For dichotomous observable A, this average coincides with the probability A = 1, hence we can study the dynamics of this probability.

Social Interaction Dynamics A typical closed system will evolve according to the Shrödinger mechanics ϕ(t) = exp (−iHt) * ϕ(0). However, here we have interaction between S and R(K), where there is a large degree of freedom (K). We assume here that S (Alice) will interact

70

D. Rooney and S. Patra

with immediate next subsystem S’. Hence for S U S’, the Hamiltonian of the system H = a NS+ b NS' + c(a * b + b * a), where the operators NS and NS' are decision operators for S and S' respectively, and a, a*, and b, b* have usual anticommutation properties. In this case the bracketed term describes the interaction which in a very simple case describes if NS increases by one unit and NS’ decreases by one unit. Since N + N' commutes with H, it is an integral of motion, or is conserved. In such cases, the law of unitary evolution based on the Heisenberg model is applicable. However, we should be careful, since unitary evolution where the norm of the state vector at the start may remain conserved, and this may not be the case in decision theory models (Bagarello et al. 2018). Hence, in some models we may also need to consider non unitary evolutions.

Implications for Leadership and Wisdom Research We believe that three elements of quantum theory are useful for explaining complex features of the practice of leadership. These are (1) adequately accounting for context by using a quantum probability framework (Pothos and Busemeyer 2013), (2) Quantum Bayesianism (QBism) as a general framework for decisionmaking and cognition, and (3) quantum field theory and decoherence theory-based (Bagarello et al. 2017) frameworks for decisionmaking and cognition in a complex interactive context with large degrees of freedom. An instructive research example is the application of quantum field theoretic formulations to asset markets (Bagarello and Haven 2014; Patra 2019). These studies focus on modelling interactions between traders based on operator formalism in quantum field theory. For example, in the earlier two-agent model (Alice and Bob), if we introduce the interaction between the agents and the information environment, the so-called reservoir (which can be considered as a vast reservoir comprising of many degrees of freedom, comprised of hard and soft information), then a new model of decisionmaking can be formulated. In this case, agents may start with initial pure states of beliefs, however, the decoherence theory of quantum mechanics can measure when they interact with the information environment as the pure state ‘decoheres’ to become a mixed states; that is, to become an actual belief about something in particular at a particular point of time. Do deal with this analytically, we can formulate the Hamiltonian value of the system because it is comprised of different creation and destruction operators and their commutation relationships. Acknowledging this complexity means also acknowledging that there are different conserved quantities represented by number operators. For example, in a restricted model the total number of shares traded in a market can be conserved. Finally, the time evolution of such operators would also provide us with time evolution equations for the market as a whole, which can now be computed. Researchers (Busemeyer and Wang 2018) have recently developed a procedure based on multidimensional Hilbert space modelling which predicts: (1) the degree

4 Remodeling Leadership: Quantum Modeling of Wise Leadership

71

of contextuality in a data set and (2) given true contextuality in the data, it describes, or predicts, how outcomes were obtained. Future leadership research can use QM to: 1. model contextuality in leadership to understand how agents form beliefs about how to be a wiser leader. 2. model multidimensional Hilbert Space Modelling (HSM) as a predictive model that can predict wise leadership. 3. apply quantum modelling for describing contextual dynamics in wise leadership. Specific QM methodological advances that would assist in advancing leadership theory that will help us understand wise leadership as a practice. Potential methods include: 1. Using Dzafarov and Kujala (2016), and Busemeyer and Wang (2018) framework based on general joint distribution of random variables, expressed in terms of inequalities (Bell inequalities or CHSH inequalities), and have used multidimension Hilbert space modelling (HSM) to predict decisionmaking outcomes (we have provided basic outlines of such frame work in the appendix). This method is directly applicable to leadership research. Most HSM models use four random variables. Such variables are dichotomous values. One variable could be for wise leadership, the other variables could be an important variables that may or may not be compatible with wisdom. 2. Using the HSM model could also compute the conditional belief state of a wise leader given the values of other variables. 3. Using compatibility and order effect analysis in Quantum modelling. If questions related to specific variables are represented by self-adjoint operators, then the final state of an actor will alter if the order/ sequence of questions is altered. Symbolically, [X, Y] /= [Y,X], there is a developed mathematical description of this, non-commutation. Where [X, Y] = XY − YX, where X and Y are random variables or observables which are provided operator representations, such operators may be Hermitian or non-Hermitian (please see appendix). An extension to leadership would be how a leader’s judgement on a specific matter changes once contexts X, Y are placed in different orders. The founders of the Quantum Bayesian school (Caves et al. 2002) interpret their theory as inherently a decisionmaking theory. However, because this school subscribes to a subjective and personalist view of quantum measurement, they imply that decisionmakers are knowledgeable about the underlying (quantum) decision rules. Since quantum Bayesian interpretations of quantum theory are personalist (subjective), rather than objective, and acknowledge the role of knowledge, leadership is a natural field for its extension. We suggest that QBism can offer insights for leadership, generally, and wise leadership in particular. Wisdom is an inherent human quality present in truly excellent leaders when they have stood out in ambiguous and difficult contexts (Mahatma Gandhi, Nelson Mandela, etc.).

72

D. Rooney and S. Patra

Wise Leaders as Entangled Actors Quantum field-inspired modeling is used for describing dynamic cognition between agents entangled in a given information environment. Thus, because organizations, leaders and wisdom are quantum-like systems, research can assume that those systems “predispose the possibility space for different configurations of interwoven tensions, [and] their actual enactment depends on the specific socio-material context” (Hahn and Knight, in press). Leadership contexts are clearly a fertile ground for applying such a model. Leadership judgment and communication are carried out in a complex information environment where a leader’s initial isolated belief state, can be modelled as a pure state superposition. This approach models how leaders are entangled with the information environment, with many, or even infinite, degrees of freedom. In the entanglement process, isolated states lose their ‘purity’; that is, they ‘decohere’ as they become entangled with the overall belief environment and develop particular understandings that lead to decisions to act in particular ways. Again, we are theorizing the belief states of the actors only. It would be intriguing to analyze the role of a leader and follower entanglement in a belief state world. There is a tradition of leader-follower game theory models, which produce different Nash equilibria compared to simultaneous move games. However, in an entangled belief world, standard game theory solutions might not work. We emphasize here that entanglement means losing the purity of one own state and becoming correlated with the surrounding environment. Applications of quantum decision theory to game theory are at a nascent stage (Yukalov and Sornette 2011). However, promising research has been done using the strategy profiles of players acting in adaptive information environments. A strategy profile is the set of strategy choices that players have, as in standard game theory, however, quantum game theory (Piotrowski and Sładkowski 2003) greatly expands the available choices to more closely represent reality and this changes the Nash Equilibrium solutions significantly. Interestingly, researchers (Piotrowski and Sładkowski 2003) have shown that if ‘quantum strategies’ are accessible for the players, games can find equilibrium solutions much earlier than predicted in standard games. Using quantum game theory, we may observe that quantum strategies are responses made by actors (corresponding to best response curve in standard game theory) that are analogous to operations allowed in quantum information theory. That is, intelligent people entangled in an information environment and who are subject to the influences of order effects, learning, etc. Adaptive information environments assumes that games are played in different contexts and that and intelligent players do not behave according to standard Nash Equilibrium models when uncertainty and ambiguity pertain. Under such conditions, players actively respond to contexts and, in particular, to their information environment. In other words, they have to explore, learn, communicate, and adapt to find a course of action. Leadership communication and cognitive process is a most productive ground for the extension of such models.

4 Remodeling Leadership: Quantum Modeling of Wise Leadership

73

Final Comments We return now to the very early point made in this article, that quantum-like modelling is different not identical to standard quantum physics, but provide more information about why? Research (Baaquie 1997, 2018) shows that there are significant differences between Quantum-like modelling in social science and standard quantum physics. Though the current article is not designed to provide a complete discussion of this literature, we nevertheless want to finish our discussion by pointing out the most salient differences. • Quantum like modelling often requires non-Hermitian Hamiltonians, which means the Hamiltonians describe systems that are not equal to their complexconjugate transposes. In standard quantum physics this is not always required, since a Hermitian Hamiltonian guarantees real Eigen values. When dealing with non-Hermitian Hamiltonians, we need to adopt nontrivial techniques to describe the dynamics of systems (Baaquie 2018). It is still not entirely clear whether a general theory can be established here, which warrants further research. • For the most part, it is decisionmaking models that warrant time dependent Hamiltonian operators, which give rise to violations of probability conservations. • Underlying state space in social systems may not be the same as standard finite or infinite dimensional Hilbert space, but a more complex Fock space or even a time dependent state space. • Entanglement observed in decisionmaking models are more complex than in the physical world. • Entanglement does not mean any kind of superluminal communication occurs in signaling between subsystems of the composite system, the same is also true in case of entanglement for human cognitive experiments. Dzhafarov and Kujala (2016) have demonstrated that many cognitive experiments have claimed entanglement might be undermined if some kind of signaling is present between subsystems, since in the presence of signaling CHSH inequalities can be violated but that in such cases it is not true ‘contextuality’, rather it may be the influence of one subsystem’s result by another. Hence, recently scientists () have claimed that such signaling effects have to be controlled for if CHSH inequalities are violated because if the signaling measures are introduced and subtracted from LHS of the CHSH inequalities we can be sure of true contextuality. This caveat is important since in case of organizational or leadership decisionmaking processes there may be ample of opportunities for such signaling, for example, group members in a decisionmaking process influencing each other’s results via hidden signaling. These differences signify that the QM paradigm is unique and, for example, can address non-linearity, non-ergodicity, chaotic dynamical systems, and of analytically challenging dynamics in social systems.11 We see much scope for extending QDT 11

In this regard QDT can also play a fundamental role in complexity theory, which describes society and economy as a complex dynamical system, with deep uncertainty.

74

D. Rooney and S. Patra

models to leadership cognition, communication and judgment to transform leadership theory.12 To conclude, then, we would like to observe that there is a growing interest in extending the QM framework in organizational behavior and management research, however, leadership can also provide a rich and promising ground for empirically testing some of the central techniques of QM. The underlying ontological framework that QM uses, including its focus on context, ambiguity, and entanglement offer the promise of new kinds of research designs that enable researchers to redouble their efforts to fully understand leaders and leadership. We can do this by contributing the foundational knowledge with which to better develop leaders, better predict leader performance, and, ultimately, we hope, to bring much needed wisdom to a global leader cohort; something the world desperately needs given our uncertain future and complex and rapidly changing social, economic and environmental world.

Appendix 1 Basic Concepts in Quantum Mechanics Quantum physics began when Max Plank and Albert Einstein proposed that energy can only be radiated and absorbed in small units, now called quanta, and that light or electromagnetic radiation are streams of massless particles called photons. The word quantum mechanics was coined in 1920s by Heisenberg, Born, Pauli, Jordan and other eminent scientists. The core structure of Quantum theory was built by 1930s, and since then scientists and philosophers have continued to develop it. Prior to quantum physics, classical physics was tied to three principles: (1) locality, which demands that there has to be a speed limit to signaling between events in space-time, which is challenged by entanglement, (2) causality, which demands a strict cause and effect relationship in nature, or a strict one directional arrow of time, and (3) realism which demands a subject-object split in (an objective) nature. However, at the quantum level of reality each of these principles is violatable. Such bold new ontological insights changed the course of modern physics by challenging classical assumptions about the nature of the physical universe and even the idea of an objective reality. To compliment this new interest a new language of mathematics and logic was developed for quantum research by such people as Heisenberg, Shrödinger, Born, and Neuman. Quantum statistics, in the form of Boson and Fermion statistics enables significant research break-throughs by, for example, Satyen Bose and Einstein. Theoretical advances, most notably by Richard Feynman, gradually reformulated quantum mechanics by, for example, introducing the path integral or sum over

12

These modelling challenges are significant, however, recently models have been devised to tackle such uniqueness in decision making models (Bagarello 2015; Khrennikova and Patra 2019).

4 Remodeling Leadership: Quantum Modeling of Wise Leadership

75

histories technique, which opened the door for quantum field theory, quantum electrodynamics, and the ‘standard model’ of particle physics, which remains the most successful model of the universe. Here we just provide a few definitions of the basic objects in quantum mechanics, in Appendix 2 we provide a more detailed account of the mathematical structure of the theory. Wave function: the description of a quantum state or a quantum system, a complex amplitude, whose modulus squared (square of the absolute value) provides the probability of the system to be found in a specific region. Wave function is described in a superposition of possibilities, or eigen values, until it is measured/observed. Wave function evolves over time in a deterministic manner following an equation of motion, namely Schrödinger’s equation of motion. The wave function lives in a complex normed vector space, named as Hilbert space. Measurement: wavefunction evolves deterministically, until the experimenter measures a specific property of the system: for example, position, or velocity, or spin. Orthodox views suggest that measurement makes the wave function collapse to one of the eigen values measured/observed in the superposition. However, this process of measurement and collapse is a truly random process and is not dependent on our state of knowledge of the initial conditions of the system. Hence, randomness in quantum theory is ontological rather than epistemological. More recently, some of the features of quantum reality such as contextuality, entanglement, and observer effects have drawn the attention of social scientists because social systems have important quantum-like features to which the logical and statistical tools of quantum physics can be applied.

Appendix 2 Basic Mathematical Tools or Concepts We begin with a brief comparison between classical probability theory (CPT) and quantum probability theory (QPT). The main features of classical probability theory are: Events are represented by sets, which are subsets of jW. Sample space, sigma algebra, measure (probability)*, are the main features of the related Kolgomorov measure theory. Boolean logic is the type of compatible logic with CPT, which allows for deductive logic, and basic operations like union and intersection of sets, DeMorgan Laws () of set theory are valid. Conditional probability: P(a/b) = p(a and b)/p(b); p(b)>0We see conditional probability is a direct consequence of Boolean operations.

76

D. Rooney and S. Patra

Based on the Boolean logic the set theory of probability also directs to Bell’s inequalities: P(A and B) + P(B− and C)> /= P(A and C). The main features of Quantum Probability Theory are: State space is a complex linear vector space: Hilbert space***; Finite/ infinite D, symbolized as H. H is endowed with a scalar product (positive definite), norm, and an orthonormal basis, non-degenerate. Any state can be visualized as a ray in this space. Superposition principle: which states that a general state can be written as a linear superposition of ‘basis states’, in information theory language the basis states are |0> or |1>. Measurement: most of the times projection postulate**. Measurement implies projection onto a specific Eigen sub-space. Probability, updating can be visualized as sequential projections on Eigen subspaces. Non-Boolean logic is compatible with such state space structure, which means violation of commutation and associative properties. The main features of Non-Boolean Logic are: Algebra of events is prescribed by quantum logic. Events form an event ring R, possessing two binary operations, addition and conjunction. P(A U B) = P(B U A) (this Boolean logic feature is invariant in Quantum logic). P{A U (B U C)} = P{(A U B) U (A U C)} (associative, property also holds good). A U A = A (idempotency). P(A and B) # P(B and A) (non commutatitivity, incompatible variables). A and (B U C) # (A and B) U (A and C) (no distributivity). The fact that distributivity is absent in quantum logic was emphasized by Birkhoff and von Neumann. Suppose there are two events B1 and B2 that, when combined, form unity, B1 ∪ B2 = 1. Moreover, B1 and B2 are such that each of them is orthogonal to a nontrivial event A # 0, hence A ∩ B1 = A ∩ B2 = 0. According to this definition, A ∩ (B1 ∪ B2) = A ∩ 1 = A. But if the property of distributivity were true, then one would get (A ∩ B1) ∪ (A ∩ B2) = 0. This implies that A = 0, which contradicts the assumption that A # 0. The main features of Quantum-like Modeling of Belief States are: Bruza et al. (2015): cognitive modelling based on quantum probabilistic frame work, where the main objective is assigning probabilities to events Space of belief is a finite dimensional Hilbert space H, which is spanned by an appropriate set of basis vectors Observables are represented by operators (positive operators/Hermitian operators) which need not commute

4 Remodeling Leadership: Quantum Modeling of Wise Leadership

77

[A, B] = AB − BA = 0 Generally, any initial belief state is represented by density matrix/ operator, outer product of ψ with itself ρ =|ψ >< ψ|, this is a more effective representation since it captures the ensemble of beliefs Pure states and∑mixed states Mixed states: w|ψ> 0: positivity,

Trace ρ = 1

Measuring the probability of choosing one of the given alternatives, which is represented by the action of an operator on the initial belief state. While making decision superposition state collapses to one single state (can be captured by the Eigen value equation). Observables in QPT represented by Hermitian operators: A = (A∗)T E(A) = Tr(A ρ), every time measurement is done one of the Eigenvalues of the A is realized. ∑ A= aP spectral decomposition rule: a’s are the Eigen values and P’s are the respective projectors which projects the initial state to the Eigen subspace with a specific a Trace formula: p(ai) = Tr(Pi ρ) As soon as the measurement is done the state ρ’: Pi ρPi/Tr(Pi ρ). Simultaneously updating of the agents’ belief state. A Quick Review of Formula FOT Total Probability/Law of Total Probability (LTP), Modified in Quantum Like Set Up

78

D. Rooney and S. Patra

First we see the LTP in classical set theory as below: P(B and (A or C)) = P(B and A) + P(B and C) P (B and (A or C)) = P(B and A) +P(B and C) (measure theoretic additivity) P(B and A) = P(A)P(B|A), and P(B|A) = P(B and A)/P(A) Hence it follows: P(B|(A or C)) = P(B|P(A or C) = {P(B|A) ∗ P(A) + P(B|C) ∗ P(C)}/P(A or C)

Hence in particular if P(A or C) = 1, then P(B) = {P(B|A) * P(A) + P(B|C) * P(C)}, this is the LTP (law of total probability) as we know in familiar CPT(classical probability theory). But in the QPT (quantum probability theory) additivity does not follow, which means LTP is violated since there are interference terms. To get the modified LTP as in non Kolgomorovian QDT set up we have to go through the concept of positive valued operators (POVM) as below. A positive operator valued measure (POVM) is a family of positive operators ∑ {Mj} such that Mj = I, where I is the unit operator. It is convenient to use the following representation of POVMs: Mj = V ∗ j Vj, where Vj: H → H are linear operators. A POVM can be considered as a random observable. Take any set of labels α1,..., αm, e.g., for m = 2, α1 = yes, α2 = no. Then the corresponding observable takes these values (for systems in the state ρ) with the probabilities p(αj) ≡ pρ(αj) = TrρMj = TrVjρV * j. We are also interested in the post-measurement states. Let the state ρ was given, a generalized observable was measured and the value αj was obtained. Then the output state after this measurement has the form: ρj = VjρV * j/(TrVjρV *j). Both order effects and interference terms in LTP can be demonstrated using POVM. Consider two generalized observables a and b corresponding to POVMs Ma = {V * j Vj} and Mb = {W * j Wj}, where Vj ≡ V (αj) and Wj = W(βj) correspond to the values αj and βj. If there is given the state ρ the probabilities of observations of values αj and βj have the form: pa(α) = TrρMa(α) = TrV(α)ρV∗ (α), p(β) = TrρMb(β) = TrW(β)ρW∗ (β).

4 Remodeling Leadership: Quantum Modeling of Wise Leadership

79

Now we consider two consecutive measurements: first the a-measurement and then the b-measurement. If in the first measurement the value a = α was obtained, then the initial state ρ was transformed into the state ( ) ρa(α) = V(α)ρV∗ (α)/ TrV(α)ρV∗ (α) For the consecutive b-measurement, the probability to obtain the value b = β is given by p(β|α) = Trρa(α)Mb(β) = TrW(β)V(α)ρV∗ (α)W∗ (β)/(TrV(α)ρV∗ (α)) This is the conditional probability to obtain the result b = β under the condition of the result a = α. We set p(α, β) = pa(α)p(β|α). Now since operators need not commute p(α, β) = p(β, α). We recall that, for two classical random variables a and b which can be represented in the Kolmogorov measure-theoretic approach, the formula of total probability ∑ (FTP) has the form pb(β) = pa(α)p(β|α). Further we restrict our consideration to the case of dichotomous variables, α = α1, α2 and β = β1, β2. FTP with the interference term for in general non-pure states given by density operators and generalized quantum observables given by two (dichotomous) PVOMs: pb(β) = pa(α1)p(β|α1) + pa(α2)p(β|α2) √ + 2λ {pa(α1)p(β|α1)pa(α2)p(β|α2)}, or by √ using ordered joint probabilities pb(β) = p(α1, β) + p(α2, β) + 2λβ p(α1, β)p(α2, β). Here the coefficient of interference λ has the √ form: λ = Trρ{W*(β)V*(αi)V(αi)W(β) − V*(αi)W*(β)W(β)V(αi)}/ 2 pa(α1)p(β|α1)pa(α2)p(β|α2) Introduce the parameters ( ) γ αβ = TrρW∗ (β)V∗ (α)V(α)W(β)/ TrρV∗ (α)W∗ (β)W(β)V(α) = p(β, α)/p(α, β) This parameter is equal to the ratio of the ordered joint probabilities of the same outcome, but in the different order, namely, “b then a” or “a then b”. Then, √ √ Interference term λ = ½ { (p(α1, β)/p(α2, β) * (γα1β -1) + (p(α2, β)/p(α1, β) * (γα2β − 1). In principle, this coefficient can be larger than one. Hence, it cannot be represented as λ = cosθ for some angle (“phase”) θ, cf. However, if POVMs Ma and Mb are, in fact, spectral decompositions of Hermitian operators, then the coefficients of interference are always less than one, i.e., one can find phases θ.

80

D. Rooney and S. Patra

One important note is that such phase terms cannot always be expressed in trigonometric terms, Hyperbolic phase terms are also possible, which are typical of results obtained from decision making models (Haven and Khrennikov 2013). Entanglement Mathematics As we have seen throughout that quantum theory allows superposition of the basis states to form new states, many of such superpositions, but not all, poses the quality of entangled states. For example, we start with a qubit system (i.e. a system which has only two basis states |0> and |1>, where they may represent up and down states, for example in decision making models they represent belief sets of decision makers as up state or down state related to any future event), now such a system can be written in superpositions of the basis states in a number of ways. √ |x> = 1/ 2 {|00> + |11>}, this state can be called as an entangled state, since say if these qubits are given to Alice and Bob, and even they are separated light years apart, if Alice measures her system there is always a 50–50 chance of finding a |0> or |1>, however as soon as she discovers that it is determined with 100% probability that Bob has to have |0> in the first case and |1> in the second case. Hence there is no superluminal communication happening, only that subsystems are in a random state and the system as a whole is in a pure state. Again, another hallmark of such states is that mathematically they are not separable, in the sense that |x> cannot be written as a sum over tensor products of only |0> or |1>. √ Comparatively, separable states are like |y> = 1/ 2{|00> + |01>}, in such a case Alice will always with probability 1 measure her subsystem to be in |0> but √ Bob still will have a 50% chance of |1> or |0>, again |y> can be separated as 1/ 2{|0>(|0> + |1>)} which means a tensor product between |0> and the superposition of |0> and |1>. Measure of degree of entanglement: concurrence measure is a type of measure of degree of entanglement, say a general entangled state is written as: a |00> + b|01> + c|10> + d|11>. Then the state is maximally entangled if |ad − bc| = 1, and there is no entanglement if |ad − bc| = 0.

References Aerts, D., Gabora, L., Sozzo, S.: Concepts and their dynamics: a quantum-theoretic modeling of human thought. Top. Cogn. Sci. 5(4), 737–772 (2013) Aerts, D., Haven, E., Sozzo, S.: A proposal to extend expected utility in a quantum probabilistic framework. Econ. Theor. 65(4), 1079–1109 (2018) al-Nowaihi, A., Dhami, S.: The Ellsberg paradox: a challenge to quantum decision theory? J. Math. Psychol. 78, 40–50 (2017) Alvesson, M., Sveningsson, S.: The great disappearing act: difficulties in doing “leadership.” Leadersh. Quart. 14(3), 359–381 (2003)

4 Remodeling Leadership: Quantum Modeling of Wise Leadership

81

Anderson, H.J., Baur, J.E., Griffith, J.A., Buckley, M.R.: What works for you may not work for (Gen) Me: Limitations of present leadership theories for the new generation. Leadersh. Quart. 28(1), 245–260 (2017) Asano, M., Basieva, I., Khrennikov, A., Ohya, M., Tanaka, Y.: A quantum-like model of selection behavior. J. Math. Psychol. 78, 2–12 (2017) Aumann, R.J.: Agreeing to disagree. Ann. Stat. 1236–1239 (1976) Avolio, B.J., Gardner, W.L.: Authentic leadership development: getting to the root of positive forms of leadership. Leadersh. Quart. 16(3), 315–338 (2005) Baaquie, B.E.: A path integral approach to option pricing with stochastic volatility: some exact results. J. Phys. I 7(12), 1733–1753 (1997) Baaquie, B.E.: Quantum Field Theory for Economics and Finance. Cambridge University Press, Cambridge (2018) Bagarello, F.: A quantum-like view to a generalized two players game. Int. J. Theor. Phys. 54(10), 3612–3627 (2015) Bagarello, F., Basieva, I., Pothos, E.M., Khrennikov, A.: Quantum like modeling of decision making: quantifying uncertainty with the aid of Heisenberg-Robertson inequality. J. Math. Psychol. 84, 49–56 (2018) Bagarello, F., Haven, E.: Toward a formalization of a two traders market with information exchange. Phys. Scr. 90(1), 015203 (2014) Bagarello, F., Haven, E., Khrennikov, A.: A model of adaptive decision-making from representation of information environment by quantum fields. Philos. Trans. Royal Soc. A: Math. Phys. Eng. Sci. 375(2106), 20170162 (2017) Basieva, I., Khrennikov, A.: Decision-making and cognition modeling from the theory of mental instruments. In: The Palgrave Handbook of Quantum Models in Social Science, pp. 75–93. Springer (2017) ˇ Batistiˇc, S., Cerne, M., Vogel, B.: Just how multi-level is leadership research? A document cocitation analysis 1980–2013 on leadership constructs and outcomes. Leadersh. Quart. 28(1), 86–103 (2017) Bell, J.S.: On the problem of hidden variables in quantum mechanics. Rev. Mod. Phys. 38(3), 447 (1966) Birkhoff, G., Von Neumann, J.: The logic of quantum mechanics. Ann. Math. 823–843 (1936) Boddy, C.R.: Psychopathic leadership a case study of a corporate psychopath CEO. J. Bus. Ethics 1–16 (2015) Bruza, P.D., Wang, Z., Busemeyer, J.R.: Quantum cognition: a new theoretical approach to psychology. Trends Cogn. Sci. 19(7), 383–393 (2015) Burns, J.M.: Leadership. Harper and Row, New York (1978) Busemeyer, J.R., Wang, Z.: Hilbert space multidimensional theory. Psychol. Rev. 125(4), 572 (2018) Case, P.: Cultivation of wisdom in the Theravada Buddhist tradition: implications for contemporary leadership and organization (2013) Caves, C.M., Fuchs, C.A., Schack, R.: Quantum probabilities as Bayesian probabilities. Phys. Rev. A 65(2), 022305 (2002) Clegg, S., e Cunha, M.P., Munro, I., Rego, A., de Sousa, M.O.: Kafkaesque power and bureaucracy. J. Polit. Power 9(2), 157–181 (2016) Dalla Chiara, M., Giuntini, R., Negri, E.: Metaphors in science and in music. A quantum semantic approach. Paper presented at the Probing the Meaning of Quantum Mechanics: Information, Contextuality, Relationalism and EntanglementProceedings of the II International Workshop on Quantum Mechanics and Quantum Information. Physical, Philosophical and Logical Approaches (2018) De Finetti, B.: Theory of Probability, vol. 1. Wiley, New York (1974) Dhiman, S.: Introduction: on becoming a holistic leader. Holistic Leadersh. 1–15. Springer (2017) Dyck, B., Greidanus, N.S.: Quantum sustainable organizing theory: a study of organization theory as if matter mattered. J. Manag. Inq. 26(1), 32–46 (2017)

82

D. Rooney and S. Patra

Dzhafarov, E.N., Kujala, J.V.: Contextuality-by-default 2.0: systems with binary random variables. Paper presented at the International Symposium on Quantum Interaction (2016) Ellsberg, D.: Risk, ambiguity, and the Savage axioms. Quart. J. Econ. 643–669 (1961) Grint, K.: Learning to lead: can aristotle help us find the road to wisdom? Leadership 3(2), 231–246 (2007) Hahn, T., Knight, E.: The ontology of organizational paradox: a quantum approach. Acad. Manag. Rev. (In press) Harvey, P., Martinko, M.J., Gardner, W.: Promoting authenticity in organizations: an attributional perspective. J. Leadersh. Organiz. Stud. 12(3), 1–11 (2006) Haven, E., Khrennikov, A.: Quantum Social Science. Cambridge University Press (2013) Haven, E., Khrennikov, A.: Editorial: applications of quantum mechanical techniques to areas outside of quantum mechanics. Front. Phys. 5(60) (2017) Haven, E., Khrennikov, A., Robinson, T.: Quantum Methods in Social Science: A First Course: World Scientific Publishing Company (2017) Haven, E., Khrennikova, P.: A quantum-probabilistic paradigm: non-consequential reasoning and state dependence in investment choice. J. Math. Econ. 78, 186–197 (2018) Ilies, R., Morgeson, F.P., Nahrgang, J.D.: Authentic leadership and eudaemonic well-being: understanding leader-follower outcomes. Leadersh. Quart. 16, 373–394 (2005) Judge, T.A., Piccolo, R.F.: Transformational and transactional leadership: a meta-analytic test of their relative validity. J. Appl. Psychol. 89(5), 755 (2004) Khrennikov, A.: Quantum version of Aumann’s approach to common knowledge: sufficient conditions of impossibility to agree on disagree. J. Math. Econ. 60, 89–104 (2015) Khrennikov, A.Y.: Open Quantum Systems in Biology, Cognitive and Social Sciences. Springer Nature (2023) Khrennikov, A., Haven, E.: Quantum mechanics and violations of the sure-thing principle: the use of probability interference and other concepts. J. Math. Psychol. 53(5), 378–388 (2009) Khrennikova, P.: Modeling behavior of decision makers with the aid of algebra of qubit creation– annihilation operators. J. Math. Psychol. 78, 76–85 (2017) Khrennikova, P., Patra, S.: Asset trading under non-classical ambiguity and heterogeneous beliefs. Physica A 521, 562–577 (2019) King, E., Nesbit, P.: Collusion with denial: leadership development and its evaluation. J. Manag. Dev. 34(2), 134–152 (2015) Küpers, W.: Phenomenology and integral pheno-practice of wisdom in leadership and organization. Soc. Epistemol. 21(2), 169–193 (2007) Küpers, W.: The art of practical wisdom: Phenomenology of an embodied, wise ‘inter-practice’ in organisation and leadership. In: Küpers, W., Pauleen, D. (eds.) A Handbook of Practical Wisdom: Leadership, Organization and Integral Business Practice. Gower, London (2013) Küpers, W., Pauleen, D.: A Handbook of Practical Wisdom: Leadership, Organization and Integral Business Practice. Gower, London (2013) Küpers, W., Statler, M.: Practically wise leadership: towards an integral understanding. Cult. Organ. 14(4), 379–400 (2008) Learmonth, M., Ford, J.: Examining leadership through critical feminist readings. J. Health Organ. Manag. 19(3), 236–251 (2005) Lord, R.G., Dinh, J.E., Hoffman, E.L.: A quantum approach to time and organizational change. Acad. Manag. Rev. 40(2), 263–290 (2015) Luthans, F., Avolio, B.J.: Authentic leadership: a positive development approach. In: Cameron, K.S., Dutton, J.E., Quinn, R.E. (eds.) Positive Organizational Scholarship: Foundations of a New Discipline, pp. 241–261. Berrett-Koehler, San Francisco (2003) McDaniel, R.R., Walls, M.E.: Diversity as a management strategy for organizations: a view through the lenses of chaos and quantum theories. J. Manag. Inq. 6(4), 363–375 (1997) McKenna, B., Rooney, D.: Wise leadership and the capacity for ontological acuity. Manag. Commun. Quart. 21(4), 537–546 (2008)

4 Remodeling Leadership: Quantum Modeling of Wise Leadership

83

McKenna, B., Rooney, D., Boal, K.: Wisdom principles as a meta-theoretical basis for evaluating leadership. Leadersh. Quart. 20(2), 177–190 (2009) Neubert, M.J., Hunter, E.M., Tolentino, R.C.: A servant leader and their stakeholders: When does organizational structure enhance a leader’s influence? Leadersh. Quart. 27(6), 896–910 (2016) Oktaviani, F., Rooney, D., McKenna, B., Zacher, H.: Family, feudalism and selfishness: looking at Indonesian leadership through a wisdom lens. Leadership 1742715015574319 (2015) Patra, S.: Agents’ behavior in crisis: can quantum decision modeling be a better answer? In: The Globalization Conundrum—Dark Clouds Behind the Silver Lining, pp. 137–156. Springer (2019) Pelletier, K.L.: Leader toxicity: an empirical investigation of toxic behavior and rhetoric. Leadership 6(4), 373–389 (2010) Piotrowski, E.W., Sładkowski, J.: An invitation to quantum game theory. Int. J. Theor. Phys. 42(5), 1089–1099 (2003) Pothos, E.M., Busemeyer, J.R.: Can quantum probability provide a new direction for cognitive modeling? Behav. Brain Sci. 36(3), 255–274 (2013) Ramsey, F.P., Lowe, E.: Notes on philosophy, probability and mathematics (1997) Rasmusen, E.: Games and Information, 4th edn. Blackwell Publishing, Hoboken, New Jersey (2007) Reh, S., Van Quaquebeke, N., Giessner, S.R.: The aura of charisma: a review on the embodiment perspective as signaling. Leadersh. Quart. (2017) Rosenthal, S.A., Pittinsky, T.L.: Narcissistic leadership. Leadersh. Quart. 17(6), 617–633 (2006) Schyns, B., Schilling, J.: How bad are the effects of bad leaders? A meta-analysis of destructive leadership and its outcomes. Leadersh. Quart. 24(1), 138–158 (2013) Shamir, B., Eilam, G.: “What’s your story?” A life-stories approach to authentic leadership development. Leadersh. Quart. 16(3), 395–417 (2005) Susskind, L., Friedman, A.: Quantum Mechanics: The Theoretical Minimum. Basic Books (2014) Thaler, R.H.: Quasi Rational Economics. Russell Sage Foundation (1994) Tourish, D.: The Dark Side of Transformational Leadership: A Critical Perspective. Routledge (2013) Tversky, A., Kahneman, D.: Advances in prospect theory: cumulative representation of uncertainty. J. Risk Uncertain. 5(4), 297–323 (1992) Van Dierendonck, D.: Servant leadership: a review and synthesis. J. Manag. 37(4), 1228–1261 (2011) Van Knippenberg, D., Sitkin, S.B.: A critical assessment of charismatic—transformational leadership research: back to the drawing board? Acad. Manag. Ann. 7(1), 1–60 (2013) Von Neumann, J.: Mathematical Foundations of Quantum Mechanics, New Edition. Princeton University Press (2018) Yang, S.-Y.: Wisdom displayed through leadership: exploring leadership-related wisdom. Leadersh. Quart. 22(4), 616–632 (2011) Yang, S.-Y.: Exploring wisdom in the Confucian tradition: wisdom as manifested by Fan Zhongyan. New Ideas Psychol. 41, 1–7 (2016) Yearsley, J.M.: Advanced tools and concepts for quantum cognition: a tutorial. J. Math. Psychol. 78, 24–39 (2017) Yukalov, V.I., Sornette, D.: Decision theory with prospect interference and entanglement. Theor. Decis. 70(3), 283–328 (2011) Zacher, H., Pearce, L.K., Rooney, D., McKenna, B.: Leaders’ personal wisdom and leader-member exchange quality: the role of individualized consideration. J. Bus. Ethics 121(2), 171–187 (2014)

Chapter 5

Quantum Financial Entanglement: The Case of Strategic Default David Orrell

Abstract According to the field of quantum cognition, a decision to act is best expressed as a quantum process, where entangled ideas and feelings combine and interfere in the mind to produce a complex, context-dependent response. While the quantum approach has proved successful at modeling many aspects of human behavior, it may be less clear how relevant this is to the economy. This paper argues that the financial system is characterized by three kinds of entanglement: at the individual level between concepts, at the social level with other people, and at the financial level through the use of credit. These entanglements combine in such a way that cognitive processes at the individual level scale up to affect the economy as a whole, in a manner which is best modeled using quantum techniques. The approach is illustrated by making a retroactive “postdiction” about the prevalence of strategic mortgage default during the financial crisis. Keywords Quantum social science · Quantum economics · Quantum cognition · Entanglement · Mortgage default JEL Classification D00 · D81 · G01

Introduction A quantum paradigm of finance and the economy is slowly emerging, and its nonlinear, complex nature may help the design of a future global economy and financial architecture … Financial assets and virtual liabilities have quantum characteristics of entanglement with each other that are not yet fully understood. Andrew Sheng, Bretton Woods Committee (2019)

D. Orrell (B) Systems Forecasting, Toronto, ON, Canada e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 A. Chakraborti et al. (eds.), Quantum Decision Theory and Complexity Modelling in Economics and Public Policy, New Economic Windows, https://doi.org/10.1007/978-3-031-38833-0_5

85

86

D. Orrell

Quantum social science is a growing area of research that is based on the idea that techniques developed originally in quantum physics can be usefully applied in the social sphere (Höne 2017). In recent years the quantum approach has proved useful in generating new insights in fields ranging from psychology to finance to international relations (see for example Qadir 1978; Schaden 2002; Haven and Khrennikov 2013; Wendt 2015). At the same time, the complexity or unfamiliarity of the methods have been a barrier to uptake by researchers, and meant that its impact on these fields has been somewhat limited. Most mathematical modelers will (rightly) want to avoid undue mathematical complication when building a model, unless it leads to better predictions. In psychology, for example, quantum methods which model things like human decision-making as quantum processes may provide more general and powerful explanations for the paradoxes of human behavior than behavioral psychology (Busemeyer and Bruza 2012), but for a practitioner who is untrained in quantum approaches the advantage is probably less clear. Researchers working in the academic area of quantum finance can explain option pricing using the Schrodinger wave equation (Haven 2003; Baaquie 2007) or a quantum walk (Orrell 2021a), but at the time of writing this has had little effect on quantitative finance—few quants use quantum methods (though this is beginning to change with growing interest in quantum computers; see Orús et al. 2019). And as in physics, even if quantum approaches work at the individual level, this doesn’t mean that such effects scale up or have any influence at the societal level instead of washing out (Hubbard 2017). This paper argues that quantum social effects do scale up very strongly to the societal level, through the phenomenon of social entanglement, of which we consider three forms. The first form of entanglement can be seen as a kind of self-entanglement, where a person’s ideas and feelings interfere during the decision-making process. The second form is entanglement with society, where discussions in the media or among neighbors influence the person’s decision. Finally, the third form of entanglement is the direct financial entanglement through the use of money and credit, which feeds back to affect the financial system as a whole. All these types of entanglement elude classical models, which assume rational behavior and independent actors, but are amenable to the quantum approach. Entanglement with ideas and norms at the personal or social level is a staple of quantum decision theory and quantum cognition. The most economically-important form of quantum entanglement, though, is that due to the financial system, which creates entanglement by design. Indeed, the financial system can be viewed as a kind of vector for transmitting quantum social effects from the individual to the societal level. We illustrate the argument using the paradigmatic example of the decision to default on a loan. The theory is used to make a “postdiction” about the number of U.S. homeowners who opted to make a strategic default during the financial crisis. The plan of the paper is as follows. “Cognitive Entanglement” section discusses the role of entanglement in quantum cognition, and demonstrates an equivalence between projection sequences and entanglement. “Quantum Decision Theory” section gives an overview of the preference reversal criterion from quantum decision theory. “Strategic Default” section describes how cognitive entanglement applies to the

5 Quantum Financial Entanglement: The Case of Strategic Default

87

case of strategic default, and particularly the U.S. housing crisis. “Social Entanglement” section considers the effects of social entanglement. “Financial Entanglement” section shows how direct financial entanglement occurs through a loan contract. “Discussion” section discusses the results and considers some possible weaknesses in the approach, and “Conclusions” section summarizes the main findings.

Cognitive Entanglement Economics has traditionally been based on classical utility theory, which assumes that people act in a consistent and rational way to optimize a utility function which encodes their preferences (von Neumann and Morgenstern 1944). Behavioral psychologists however have uncovered a long list of cognitive effects which appear paradoxical in the light of classical logic (for a summary see Kahneman 2011). The quantum approach is different in that it assumes that these cognitive effects come about not so much because we are irrational, but because we are using a different kind of logic, that can best be modeled using the quantum formalism. As a simple example, consider the mental state of a person who is faced with a choice between two uncertain prospects, denoted |B1 〉 and |B2 〉 . The person’s attitude towards these prospects will be shaped by a context which we describe through the orthonormal eigenstates |A1 〉 and |A2 〉 . In a lottery, for| example, the |Ai 〉 terms could capture subjective attitudes towards risk, while the | B j 〉 terms could represent different payouts. Alternatively, for an experiment exploring the order effect (Wang et al.| 2014), the |Ai 〉 could represent the response to one question in a survey, while the | B j 〉 represent the response to a subsequent question. In quantum cognition, this type of problem can be handled using a sequence of projections, as illustrated in Fig. 5.1 (where we are assuming that coefficients are real). The gray diagonal line u represents a decision maker’s initial state. This is projected first onto a subjective state depicted by the axes pair A1, A2 and from there onto a decision depicted by the rotated axes pair B1, B2. If the decision state u is at an angle θ to the horizontal | axis, and axis B1 is at an angle ϕ, then the probabilities of the various outcomes | Ai B j 〉 are as given in Table 5.1. The probability of the outcome |B1 〉 when context is not taken into account is simply the projection of the initial state u onto the axis B1, so P(B1 ) = cos2 (θ − ϕ). This can alternatively be written as P(B1 ) = P(B1 |A1 )P(A1 ) + P(B1 |A2 )P(A2 ) + ∂(A, B1 )

88

D. Orrell

Fig. 5.1 A projection sequence. The initial state u is rotated by an angle θ from the horizontal axis, while the axis B1 is rotated by an angle ϕ from the horizontal axis

Table 5.1 The probability of different contexts and decisions in Fig. 5.1

Context A

Decision B

Probability

A1

B1

|α11 |2 = cos2 (θ )cos2 (ϕ)

A1

B2

|α12 |2 = cos2 (θ )sin2 (ϕ)

A2

B1

|α21 |2 = sin (θ )sin2 (ϕ)

A2

B2

|α22 |2 = sin (θ )cos2 (ϕ)

2

2

which is a version of total probability, but with an addition interference term ∂( A, B1 ) = P(B1 ) − P(B1 |A1 )P( A1 ) − P(B1 |A2 )P(A2 ) = P(B1 ) − α11 − α21 = 2cos(θ )cos(ϕ)sin(θ )sin(ϕ). Another way to express this problem is in terms of a quantum circuit, of the sort The input to the circuit is used in a quantum computer, see Fig. 5.2 (Orrell ( 2020c). ) 1 . The upper qubit is acted on by two qubits, each initialized in the state |0〉 = 0 the unitary operator ( R A (θ ) = and the lower qubit is acted on by

cosθ −sinθ sinθ cosθ

)

5 Quantum Financial Entanglement: The Case of Strategic Default

89

Fig. 5.2 Quantum circuit for a projection sequence. The input on the left is two qubits initialised to |0〉 (not shown). Each is acted on by a rotation gate, and then the C-NOT gate, which is typically denoted as shown, with the control being the lower qubit. The output on the right is two entangled qubits

( R B (ϕ) =

) cosϕ −sinϕ . sinϕ cosϕ

Referring to Fig. 5.2, the matrix R A (θ ) rotates A1 to u, and R B (ϕ) rotates A1 to B1. Let X c be the C-NOT gate ⎛

1 ⎜0 Xc = ⎜ ⎝0 0

0 1 0 0

0 0 0 1

⎞ 0 0⎟ ⎟. 1⎠ 0

Then starting from the initial state ψ1 = |0〉 ⊗ |0〉 , the final state is ⎛

⎞ cosθ cosϕ ⎜ cosθ sinϕ ⎟ ⎟ ψ = X c (R A ⊗ R B )ψ1 = ⎜ ⎝ sinθ sinϕ ⎠ sinθ cosϕ which yields the same probabilities as in Table 5.1. The choice of a C-NOT gate for the entanglement matrix reflects the fact that a change in the subjective context acts as a toggle on the starting point for the final decision. This approach can be generalised to any projection sequences where the matrices R A and R B are unitary transformations. It follows that in such cases the resulting interference can be viewed as being caused either by the projection sequence, or by entanglement.

90

D. Orrell

Quantum Decision Theory The use of entanglement is similar to that of quantum decision theory (QDT), with the difference that here we have explicitly computed the entanglement matrix, while in QDT it is simply assumed that objective and subjective factors are entangled (see for example Yukalov and Sornette 2008, 2014, 2015a, b; Favre et al. 2016). In QDT, prospects are written in the general form of superposition states |π1 〉 = γ11 |A1 B1 〉 + γ12 |A1 B2 〉 |π2 〉 = γ21 |A2 B1 〉 + γ22 |A2 B2 〉 where the coefficients γ ji can be complex. The person’s state prior to making a choice is |ψ〉 = α11 |A1 B1 〉 + α12 |A1 B2 〉 + α21 |A2 B1 〉 + α22 |A2 B2 〉 where the coefficients satisfy |〈ψ|ψ〉|2 = |α11 |2 + |α12 |2 + |α21 |2 + |α22 |2 = 1. The probability of the person choosing the prospect π j is therefore |2 )( ) ( ) 1| 1( ∗ ∗ ∗ α j1 γ j1 + α ∗j2 γ j2 α j1 γ j1 + α j2 γ j2 p π j = |〈ψ|π j 〉| = P P where P = |〈ψ|π1 〉|2 +|〈ψ|π2 〉|2 is a normalization term to ensure that the probabilities add to 1. This can be written in the form ( ) ( ) ( ) p πj = f πj + q πj where ( ) 1 (|| ||2 || ||2 || ||2 || ||2 ) α j1 γ j1 + α j2 γ j2 f πj = P is called the utility function, and ) ( ) 1( ∗ ∗ ∗ α j1 γ j1 α j2 γ j2 + α ∗j2 γ j2 α j1 γ j1 q πj = P is called the attraction function. The utility function separates out the two terms corresponding to the outcomes |A1 〉 and |A2 〉 (in a lottery for example it would correspond to the expected payout), while the attraction function represents their

5 Quantum Financial Entanglement: The Case of Strategic Default

91

entanglement through the different contexts |B1 〉 and |B2 〉 . Note that if ) ( subjective there is no entanglement, then q π j = 0, the probabilities are the same as for the classical approach, and there is no need to evoke quantum methods (Yukalov and Sornette 2014). Since a classical utility term is in the form of a probability term, we need to have f (π1 ) + f (π2 ) = 1. But since p(π1 ) + p(π2 ) = 1 it follows that q(π1 ) + q(π2 ) = 0. As an example, for the case of the projection sequence shown in Fig. 5.1, the | |2 probabilities |α ji | are as in Table 5.1, and the other coefficients can be set to γi j = 1. We then have f (π1 ) = |α11 |2 + |α21 |2 = cos2 (θ )cos2 (ϕ) + sin2 (θ )sin2 (ϕ) q(π1 ) = 2α11 α21 = 2cos(θ )cos(ϕ)sin(θ )sin(ϕ) f (π2 ) = |α12 |2 + |α22 |2 = cos2 (θ )sin2 (ϕ) + sin2 (θ )cos2 (ϕ) q(π2 ) = 2α12 α22 = −2cos(θ )cos(ϕ)sin(θ )sin(ϕ) If we assume, in the absence of any information about the structure of the attraction function, that the various probabilities are uniformly distributed, then the attraction function of the more attractive choice has an expected value 1/4, and the less attractive choice −1/4. This result is known as the “quarter law” (Yukalov and Sornette 2015a, b) and has been tested empirically in a variety of situations using controlled experiments. Now, according to classical utility theory, the person is expected to choose prospect π1 if f (π1 ) − f (π2 ) > 0. In QDT however we have to take into account the interference terms, so the relevant test becomes f (π1 ) + q(π1 ) − f (π2 ) − q(π2 ) > 0, or equivalently f (π1 ) − f (π2 ) > 2|q(π1 )|. In other words, the attraction function sets a threshold which needs to be exceeded in order for an option to be seen as preferable (Orrell 2020c, 2021a). Following the quarter law, a starting guess is that the utility (on a scale of 0 to 1) of an option has to exceed that of the other one by 0.5. Yukalov and Sornette (2015a, b: 4) call this the preference reversal criterion, for reasons discussed below. Put another way, suppose that the subjectively less attractive option has an associated cost x1 and the more attractive option has a cost x2 . We can assign the relative utility functions

92

D. Orrell

f (π1 ) = 1 −

x2 x1 = x1 + x2 x1 + x2

f (π2 ) = 1 −

x1 x2 = x1 + x2 x1 + x2

which have a possible range of 0 to 1, and sum to 1 (Yukalov and Sornette 2018). The preference reversal condition is then f (π1 ) − f (π2 ) =

1 x2 − x1 > . x1 + x2 2

Equality in the above expression is attained if x2 = 3x1 , and in general if the condition holds we might expect x2 /x1 > 3. Again, this should only be viewed as a first approximation assuming no prior information, but highlights the significant role that subjective effects play in decision making. The next section shows how this type of cognitive entanglement applies to the case of mortgage default, for which a unique dataset was supplied by the U.S. housing crisis.

Strategic Default Quantum decision theory has so far mostly been applied to experiments where participants are asked to choose between carefully crafted lotteries with different balances of risk and reward. One example is preference reversal, where the choice is between two lotteries, the first offering a high probability of a low payout, the second a low probability of a high payout. If the expected utility of the second lottery is a little higher, then people still tend to choose the first lottery for themselves. But if the question is reframed so they are asked to price a ticket which can be sold to someone else, they value the second lottery more highly (Tversky and Thaler 1990). Yukalov and Sornette (2015a, b) analyzed experimental data sets for such lottery examples to show that the transition point between the two choices follows their preference reversal criterion. A more economically relevant application, though, is provided by the case of default among mortgage holders. Default usually occurs because factors such as unemployment or divorce mean that the homeowner can no longer afford the mortgage payments. However, if house prices have declined so that the home is worth less than the mortgage, then the homeowner may also decide to walk away, which is known as strategic default. If this occurs at a large scale, it can have a significant economic impact. As Gerardi et al. (2018: 1) note, “Understanding the relative importance of these determinants of default is central for designing policies aimed at reducing the probability of a future wave of mortgage defaults and foreclosures, and for designing loss mitigation policies that reduce the negative economic impacts of future possible foreclosure crises on lenders and homeowners.”

5 Quantum Financial Entanglement: The Case of Strategic Default

93

One approach to understanding the phenomenon is to directly ask home owners under what conditions they would consider strategic default. Guiso et al. (2009) used available data from a quarterly survey of a representative sample of U.S. households from December 2008 to September 2010 in order to analyze prevailing homeowner attitudes. One question was: “If the value of your mortgage exceeded the value of your house by $50 K would you walk away from your house (i.e., default on your mortgage) even if you could afford to pay your monthly mortgage?” The answer was affirmative for 9% of respondents. Of those who replied “no” to that question, 23% said they would default if the mortgage exceeded the value of their house by $100 K. And of those who replied “no” again, 41% said they would default if the shortfall was $200 K. So adding these up, roughly 30% of respondents said they would default if the shortfall was more than $100 K, and a 64% majority said they would default if it exceeded $200 K. However the actual statistics for foreclosure paint a very different picture. By mid-2009 over 16% of U.S. homeowners had negative equity exceeding 20% of their home’s value, and over 22% of homeowners had negative equity exceeding 10% of their home’s value (White 2010). Given the high value of homes in the mostaffected markets, many of these homeowners were underwater by well over $100 K. If 30% of these had opted for strategic default, in accordance with the survey results, it would have represented in total around 5% of American homeowners. However, while by the third quarter of 2009 the combined foreclosure and thirty-plus-day delinquency rate for home mortgages did reach a historic high of 14%, only a small fraction of these were strategic. Bradley et al. (2015) estimated the proportion to be in the range of 7.7–14.6%, which would put the overall strategic default rate at only 1–2%. The main cause of default it seemed was not utility optimization, but unemployment leading to an inability to maintain payments (Gerardi et al. 2018). According to one estimate from the Federal Reserve (Bhutta et al. 2010: 21), the “median borrower walks away from his home when he is 62 percent underwater.” On the face of it, this behavior seems irrational, since even given the various costs of foreclosure the best option from a narrow utilitarian point of view would often be default. For example White (2010: 983) describes the case of a hypothetical homeowner who bought an average home in Miami at the peak of the housing market for around $360,000: That home would now be worth only about $159,000, and, assuming a 5% down payment, the homeowner would have approximately $170,000 in negative equity. Assuming he intended to live in the house for five years, he could save approximately $147,000 by walking away and renting a comparable home … The advantage of walking away is even more starkly evident for the large percentage of individuals who bought more-expensive-than-average homes in the Miami area – or in any bubble market for that matter – in the last five years. Millions of U.S. homeowners could save hundreds of thousands of dollars by strategically defaulting on their mortgages. Homeowners should be walking away in droves. But they aren’t.

As it turned out home prices in Miami bottomed in 2012 according to the S&P/ Case-Shiller FL-Miami Home Price Index1 and have since recovered much of their 1

See https://fred.stlouisfed.org/series/MIXRNSA.

94

D. Orrell

losses, but of course few people expected that at the time (defaulters in early 2010 could have invested their savings in the stock market and made a huge profit, but that wasn’t a popular option either). Behavioral economists typically explain such effects by appealing to the idea that homeowners suffer from cognitive biases which lead them to make poor economic decisions (Ubel 2009), and behavioral models exist that fit the data by adjusting for things like present bias and discount rates (see for example Atlas et al. 2017). However this does not explain the fact that even when homeowners can see it makes economic sense to default—and say they would default in a survey—they usually decline to do so in practice. Instead it seems that the primary motivation for staying in the home is the desire to avoid shame and social stigma, and fear of the perceived (and often exaggerated) consequences of default (White 2010). In other words, the response is driven not so much by cognitive deliberations but a powerful mix of emotions. And the fact that this combination of guilt and fear is felt far more keenly when actually making a decision to stay or move, as opposed to when answering a survey question, is why observed default rates are far lower than one might expect from calculations based on either survey results or utility maximization. The situation is therefore similar to the phenomenon of preference reversal described above, where we evaluate an option differently depending on whether we are coming up with a hypothetical price, or making an actual choice. Again, preference reversal is a challenge to both classical and behavioral models because it implies the existence of a subjective component to decision making which presents differently according to context. Instead of adjusting behavioral models in an ad hoc manner, a simpler and more elegant explanation is to apply the methods of quantum decision theory, where the attraction function accounts for the context-dependent subjective factors including shame, guilt, and fear. This also has the advantage of being a consistent model which can be applied for a broad range of phenomena. Suppose that the cost of staying in the home for a certain period is x1 and the cost of defaulting, including renting for the same period, is x2 . According to the preference reversal criterion, in order for strategic default to be selected we would expect the cost ratio x2 /x1 to be around 3. In the above example of a Miami homeowner, White (2010: 983) estimated the cost of renting for 5 years at $60,000, while the cost of staying was x2 = $207,000 which differs by a factor of x2 /x1 = 3.45. What seems like a clear-cut argument for default in terms of utility, is in fact borderline from a QDT perspective. The Federal Reserve estimate for the critical threshold to initiate strategic default meanwhile was a 62 percent fall in value, which corresponds to a fall in utility compared to the purchase price by a factor 2.63. Quantum decision theory, and in particular entanglement between objective calculations and subjective emotions, therefore helps to explain why so few people in similar situations actually chose to default during the crisis, even if their behavior seems to defy both classical utility theory, and the results of surveys: when it came to the crunch, entangled emotions such as guilt and fear interfered with and outweighed abstract considerations of utility. This isn’t really a prediction, since we are applying the theory after we know the results, but it certainly adds credence to the quantum approach to cognition. Perhaps the main message is that for the complex issue of

5 Quantum Financial Entanglement: The Case of Strategic Default

95

strategic default, the discrepancy between utility-maximizing and observed behavior is so large that standard calculations of utility—despite the foundational role they play in mainstream economics—are of rather little relevance.

Social Entanglement The above analysis is based on the idea that subjective and objective attitudes are entangled within the mind of the decision-maker, and so resolve themselves in a manner that is context-dependent. However entanglement also operates at the social level, for example as a result of the exchange of information between the society members. To accommodate such effects, Yukalov and Sornette (2015a) {extend}the decision space to be the tensor product H = H I ⊗ H S . Here H I = Span Ai B j is the decision space for the individual defined in “Cognitive Entanglement” section, and H S represents the decision space for the rest of society, which can similarly be expressed as the tensor product of the individual spaces of the other society members. The state of the society is represented by the statistical operator ρ I S . This is normalized so that Tr IS ρ I S (μ) = 1 where the trace operation is performed over H, and μ represents an amount of information. By examining how the prospect probabilities evolve as a function of the information level μ, Yukalov and Sornette (2015a) then show formally that the net effect of interaction between the individual and society is to attenuate the individual’s attraction function, so that /\

/\

( ) ( ) logμ→∞ p π j , μ = f π j In other words, as information increases, the probability converges to the classical utility factor. A similar approach was used by Khrennikova and Haven (2016) to model how voter preferences are shaped by the informational “bath” generated by mass media. Indeed, experimental evidence shows that cognitive biases tend to reduce when participants exchange information by consulting with others (Charness et al. 2010). For the case of mortgage default, we would expect the strategic default rate to increase with information flow. This was again confirmed by the data, which showed that people who know someone who has strategically defaulted are 82% more likely to declare their intention to default: as Guiso et al. (2013: 1514) write, “This effect does not seem to be due to clustering of people with similar attitudes, but rather to learning about the actual cost of default. We find a similar learning effect from exposure to the media, an effect that is reduced when the media start to cover the topic more extensively.”

96

D. Orrell

Financial Entanglement While most of the attention in quantum social science towards entanglement has focused on its psychological and social forms, in the financial system there is actually a far more direct form of entanglement, which occurs through the use of money and credit (Orrell 2018b, 2020c). Unlike with most social entanglements, financial entanglement is explicitly encoded in contracts. And it is this entanglement which scales up the effects of cognitive and social entanglement to a point where they can affect the global financial system. The status of a loan contract between a creditor C and a debtor D can be expressed as the superposition |L〉 = α1 |C ↓〉|D ↑〉 + α2 |C ↑〉|D ↓〉 where the first term represents default, and the second term represents no default, with the α j being normalised coefficients. The situation is therefore the same as the case for two entangled electrons with opposite spin, with the difference that the arrow represents the direction of payment rather than spin. Now suppose that we perform a measurement on the (quantum) state of the debtor. If they choose to default, then the system collapses to the eigenstate |L〉 = |C ↓〉|D ↑〉 . At any time after that, if the creditor decides to assess (i.e. measure) the state of the loan, the result can only indicate default. The two parties are thus entangled. Note the key point is that we are treating the debtor’s state regarding the loan as being in a superposition of the two states “default” and “no default”. The state of the loan is therefore indeterminate yet still correlated, which is the essence of entanglement. Also, the creditor will not discover a default immediately, and the default will be subject to legal procedures, but these qualifications do not change the fact that the status of the loan, viewed as an independent entity, changes instantly. Figure 5.3 shows a quantum circuit which simulates such a loan arrangement. As in Fig. 5.2, the input is two qubits initialised to |0〉 . The top qubit now represents the creditor, and the bottom qubit represents the debtor. The NOT gate flips the first qubit to |1〉 , and represents the creation of a debt. The rotation gate R sets the second qubit to a superposition state cosθ |0〉 + sinθ |1〉 so the probability of the state |1〉 , corresponding to default, is sin2 θ . Finally the C-NOT gate, which represents the binding contract, flips the first qubit depending on the state of the second, resulting in the entangled state cosθ |10〉 + sinθ |01〉 which indicates the status of the loan. Since not just the mortgage market but the entire financial system is based on credit, we see that this form of financial entanglement is one of the main characteristics of the global economy. Credit products such as mortgages therefore act as a vector of transmission for quantum cognitive effects, and create a feedback loop

5 Quantum Financial Entanglement: The Case of Strategic Default

97

Fig. 5.3 Quantum circuit for a loan contract. The input on the left is two qubits initialised to |0〉 (not shown). The output on the right is two entangled qubits which represent the state of the loan

between the individual and societal levels. The quality of a bank’s loan book is determined by the aggregate quantum state of its debtors. As White (2010) notes, the social and psychological norms which prevent people from defaulting were actively promoted by financial institutions and the government, exactly because mass default would have threatened the security of the financial system. According to an estimate from First American, it would have cost some $745 billion, or slightly more than the size of the 2008 bank bailout, to restore the lost equity of all underwater borrowers (Streitfeld 2010). These entanglements were further extended and amplified through the use of complex derivatives such as collateralized mortgage obligations, which were held internationally. Prior to the crisis, these were seen as having a stabilizing effect on the economy. As the International Monetary Fund (2006: 51) noted: “The dispersion of credit risk by banks to a broader and more diverse group of investors, rather than warehousing such risks on their balance sheets, has helped to make the banking and overall financial system more resilient.” Bernanke (2006) echoed the IMF when he said that “because of the dispersion of financial risks to those more willing and able to bear them, the economy and financial system are more resilient.” These conclusions were apparently not based on modeling, since as discussed further in the next section, models at the time didn’t even include a banking sector. Financial entanglement is therefore a dominant factor in the economy, but also one of the least understood.

Discussion As seen above, while quantum decision theory has usually been applied and tested in controlled settings, it can also help explain observed economic behavior such as strategic default. This section discusses the results and considers some possible weaknesses in the approach. One complication is that, while for a well-defined lottery it is possible to calculate the utility of a particular choice, for something like a mortgage the situation is far more complicated, and depends on factors such as estimated value of the home, discount rates into the future, and so on, which cannot be known with precision. It therefore isn’t clear how exactly to partition the utility and attraction functions, since both contain a subjective component. However it is reasonable to suppose that

98

D. Orrell

homeowners can make a rough estimate of the value of their homes and the costs of moving, and they will certainly remember what they paid for it. And the main point is that emotions such as fear and guilt, which are part of the attraction function, play a huge but context-dependent role in the decision-making process even if they don’t come with a price tag. Another potential criticism is that we don’t need elaborate quantum models to simulate the phenomenon of preference reversal, since in the end it is just an illustration of the old saying that you should watch what people do, not what they say. However, the fact that this simple fact is a severe challenge for both classical and behavioral approaches shows the need for an alternative approach. And the underlying idea behind quantum decision theory is actually very simple, namely that subjective factors can interfere with our decision making in a manner that depends upon context. For the question of social entanglement, the role of the information parameter μ is even less well-defined, and the mathematics necessary to express quantum entanglement in a social setting is even more elaborate (though again the actual result is very simple). Also, while social interactions increase the information flow which should help make better decisions, there is plenty of evidence to show that the opposite can happen too, as in the well-known phenomenon of herd behavior. In the case of strategic default, it seems that the guilt and fear experienced privately by homeowners is lessened when they see other people defaulting, which itself is a kind of herd behavior. These effects can be modeled using a quantum methodology as well, though it may make more sense to adopt a quantum agent-based modeling approach, where financial transactions are modeled as interactions between entangled quantum entities (Orrell 2020a). Finally, while financial entanglement can be expressed mathematically in terms of quantum entanglement, one could also use a classical model (quantum models aren’t of course always used in physics either, even if the system is ultimately quantum). However, the quantum method seems a natural approach for exploring the role of money and credit in the economy, which points to a larger question. A puzzling feature of mainstream economics is that it has long downplayed the importance of money, treating it as little more than an inert medium of exchange. Macroeconomic models such as dynamic stochastic general equilibrium (DSGE) models treat the economy like a kind of barter system, and have traditionally excluded the financial sector (though this started to change after the crisis). In particular, a working assumption in such models has been that the possibility of default can be ignored, in part because it allowed for simplification (Goodhart et al. 2016). And even when default is the subject of study, as White (2010: 990– 991) notes, it is usually framed in terms of an objective calculation: “Emotion is rarely considered in and of itself as a primary factor motivating both people and markets … researchers have shown little interest in the relationship between guilt and mortgage default. Nor have they shown any interest in the relationship between fear and mortgage default.” Quantum decision theory therefore seems to be saying something interesting about the field of economics itself. Money is a highly psychoactive substance, and it arouses

5 Quantum Financial Entanglement: The Case of Strategic Default

99

powerful and often contradictory emotions. By definition it combines objective utility and subjective desire in a single package (Orrell 2016, 2018a, 2020b, 2021b). One reason it has been excluded from traditional models is exactly because it doesn’t fit neatly with the assumption of rational utility-optimizing behavior. According to QDT, our response to money should depend strongly on context. When we are crafting economic theories, we tend to downplay its effects; but when it comes to making actual decisions, we act in a very different way. Our attitudes towards issues such as debt are complex, and tensions around the subject come to a head when contemplating the prospect of default. The quantum approach is a natural framework for addressing such topics where cognitive and financial elements entangle and collide.

Conclusions The quantum approach shows that the phenomenon of financial entanglement is a defining feature of our financial system, which affects decision-making at the individual and societal levels. The fact that money and credit have powerful and sometimes contradictory emotional effects is one reason they have traditionally been downplayed in economic models. The quantum approach to decision making, in contrast, is specifically designed to handle the interplay between objective utility and subjective feelings. The same quantum framework is also ideally suited to modeling the debtor/creditor relationship as entangled entities; and for showing how, far from averaging out at the macro level, cognitive processes at the individual level scale up to affect the economy as a whole. This was graphically illustrated by the U.S. housing crisis, which offered a unique illustration of how psychological and financial entanglement affects the economy. Perhaps the greatest contribution of the quantum approach, other than its use as a mathematical toolbox, will be to help draw attention to these entanglements which have been largely ignored—with very real social consequences—by traditional models.

References Atlas, S.A., Johnson, E.J., Payne, J.W.: Time Preferences and Mortgage Choice. J. Mark. Res. 54(3), 415–429 (2017) Baaquie, B.E.: Quantum Finance: Path Integrals and Hamiltonians for Options and Interest Rates. Cambridge University Press, Cambridge (2007) Bernanke, B.: Basel II: Its Promise and Its Challenges. Retrieved from: http://www.federalreserve. gov/newsevents/speech/bernanke20060518a.htm (2006) Bhutta, N., Dokko, J., Shan, H.: The depth of negative equity and mortgage default decisions. Federal Reserve Board, FEDS Working Paper No. 2010-35 (2010) Bradley, M.G., Cutts, A.C., Liu, W.: Strategic mortgage default: the effect of neighborhood factors. Real Estate Econ. 43, 271–299 (2015)

100

D. Orrell

Busemeyer, J., Bruza, P.: Quantum Models of Cognition and Decision. Cambridge University Press, Cambridge (2012) Charness, G., Karni, E., Levin D.: On the conjunction fallacy in probability judgement: new experimental evidence regarding Linda. Games Econ. Behav. 68, 551–556 (2010) Favre, M., Wittwer, A., Heinimann, H.R., Yukalov, V.I., Sornette, D.: Quantum decision theory in simple risky choices. PLoS ONE 11(12), e0168045 (2016) Gerardi, K., Herkenhoff, K.F., Ohanian, L.E., Willen, P.S.: Can’t pay or won’t pay? Unemployment, negative equity, and strategic default. Rev. Financ. Stud. 31(3), 1098–1131 (2018) Goodhart, C., Romanidis, N., Tsomocos, D., Shubik, M.: Macro-modelling, default and money. LSE, FMG Discussion Paper DP755 (2016) Guiso, L., Sapienza, P., Zingales, L.: The determinants of attitudes toward strategic default on mortgages. J. Financ. 68(4), 1473–1515 (2013) Guiso, L., Sapienza, P., Zingales, L.: Moral and social constraints to strategic default on mortgages 2. European Univ. Inst., Working Paper No. ECO 2009/27. Available at http://www.nber.org/ papers/w15145.pdf (2009) Haven, E.: A Black-Scholes Schrödinger option price: “bit” versus “qubit.” Phys. A 324, 201–206 (2003) Haven, E., Khrennikov, A.: Quantum Social Science. Cambridge University Press, Cambridge (2013) Höne, K.E.: Quantum Social Science. Oxford Bibliographies. Available at: http://www.oxfordbiblio graphies.com/view/document/obo-9780199743292/obo-9780199743292-0203.xml (27 April 2017) Hubbard, W.H.: Quantum economics, Newtonian economics, and law. Michigan State Law Rev. 425 (2017) International Monetary Fund: Global Financial Stability Report: Market Developments and Issues. Washington, DC (2006) Kahneman, D.: Thinking Fast and Slow. Farrar, Straus and Giroux, New York (2011) Khrennikova, P., Haven, E.: Instability of political preferences and the role of mass media: a dynamical representation in a quantum framework. Philos. Trans. R. Soc. A Math. Phys. Eng. Sci. 374(2058), 20150106 (2016) Orrell, D.: A quantum theory of money and value. Econ. Thought 5(2), 19–28 (2016) Orrell, D.: Quantum Economics: The New Science of Money. Icon Books, London (2018a) Orrell, D.: Quantum economics. Econ. Thought 7(2), 63–81 (2018b) Orrell, D.: A quantum model of supply and demand. Phys. A 539, 122928 (2020a) Orrell, D.: The value of value: a quantum approach to economics, security and international relations. Secur. Dialogue 51(5), 482–498 (2020b) Orrell, D.: Quantum Economics and Finance: An Applied Mathematics Introduction. Panda Ohana, UK (2020c) Orrell, D.: A quantum walk model of financial options. Wilmott 2021(112), 62–69 (2021a) Orrell, D.: The color of money: threshold effects in quantum economics. Quantum Rep. 3(2), 325–332 (2021b) Orús, R., Mugel, S., Lizaso, E.: Quantum computing for finance: overview and prospects. Rev. Phys. 4, 100028 (2019) Qadir, A.: Quantum economics. Pak. Econ. Soc. Rev. 16(3/4), 117–126 (1978) Schaden, M.: Quantum finance. Phys. A Stat. Mech. Appl. 316, 511–538 (2002) Sheng, A.: A new Bretton woods vision for a global green new deal. In: Revitalizing the Spirit of Bretton Woods: 50 Perspectives on the Future of the Global Economic System, pp. 360–367. Bretton Woods Committee (2019) Streitfeld, D.: No Help in Sight, More Homeowners Walk Away. New York Times (2010) Tversky, A., Thaler, R.H.: Anomalies: preference reversals. J. Econ. Perspect. 4, 201–211 (1990) Ubel, P.: Human Nature and the Financial Crisis, Forbes.com. Available at http://www.forbes. com/2009/02/20/behavioral-economics-mortgage-opinions-contributors_financial_crisis.html (2009)

5 Quantum Financial Entanglement: The Case of Strategic Default

101

Von Neumann, J., Morgenstern, O.: Theory of Games and Economic Behavior. Princeton University Press, Princeton, NJ (1944) Wang, Z., Solloway, T., Shiffrin, R.S., Busemeyer, J.R.: Context effects produced by question orders reveal quantum nature of human judgments. Proc. Natl. Acad. Sci. 111(26), 9431–9436 (2014) Wendt, A.: Quantum Mind and Social Science: Unifying Physical and Social Ontology. Cambridge University Press, Cambridge (2015) White, B.T.: Underwater and not walking away: shame, fear, and the social management of the housing crisis. Wake For. Law Rev. 45, 971–1023 (2010) Yukalov, V.I., Sornette, D.: Mathematical structure of quantum decision theory. Adv. Complex Syst. 13, 659–698 (2008) Yukalov, V.I., Sornette, D.: Conditions for quantum interference in cognitive sciences. Top. Cognit. Sci. 6, 79–90 (2014) Yukalov, V.I., Sornette, D.: Preference reversal in quantum decision theory. Front. Psychol. 6, 1–7 (2015a) Yukalov, V.I., Sornette, D.: Role of information in decision making of social agents. Int. J. Inf. Technol. Decis. Mak. 14(05), 1129–1166 (2015b) Yukalov, V.I., Sornette, D.: Quantitative predictions in quantum decision theory. IEEE Trans. Syst. Man Cybern. Syst. 48(3), 366–381 (2018)

Chapter 6

Quantum-Like Contextual Utility Framework Application in Economic Theory and Wider Implications Sudip Patra and Sivani Yeddanapudi

Abstract Quantum-like modeling is a new but well received paradigm in social science that draws from various mathematical tools used in quantum science, such as information theory. However, we argue that there are deeper meta-principles, such as Contextuality-complementarity, Uncertainty, and Non-locality, that give meaning to these models. These meta-principles are equally applicable in both the physical and cognitive domains, but with different specific measures. It is important to exercise caution and recognize that economic theory should not be equated with quantum theory per se. In this paper, we demonstrate a simple application of quantumlike modelling in financial economics, specifically in the much-debated portfolio diversification theory given radical uncertainty. Our findings suggest that quantumlike modelling can provide a useful framework for understanding complex financial systems and making informed investment decisions. Keywords Quantum-like framework · Hilbert space modelling · Economic theory · Portfolio diversification · Meta principles · Paradigms

Introduction Paradigms in economic theory can be broadly categorized as beginning with classical political economy, followed by the rise of mathematical modeling, which gave rise to axiomatic rationality modeling, general equilibrium theory, social choice theory, neoclassical theory of utility optimization, and the standard neoliberal economic theory of frictionless markets. However, there have always been heterodox approaches, such as Marxian economic theory, including neo-Marxian S. Patra (B) OP Jindal Global University, CEASP, Sonipat, New Delhi, NCR 131001, India e-mail: [email protected] S. Yeddanapudi CEASP, OP Jindal Global University, Sonipat, India © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 A. Chakraborti et al. (eds.), Quantum Decision Theory and Complexity Modelling in Economics and Public Policy, New Economic Windows, https://doi.org/10.1007/978-3-031-38833-0_6

103

104

S. Patra and S. Yeddanapudi

approaches, Austrian economic theory, and more recently, complexity economics or complex adaptive systems perspectives. We may refer to these approaches as paradigms since their foundational assumptions differ. For example, behavioral economics is a significant modification of neoclassical economics, drawing upon cognitive science. However, complexity economics represents a foundational shift, with its basic assumption of non-equilibrium and emergence, rather than DSGE (dynamic stochastic general equilibrium). In the spectrum of paradigms, quantumlike modeling, or as we claim, the quantum-like framework, is a recent and evolving approach. Quantum-like modelling is based on different widely used mathematical tools in quantum science, and its possible extensions in cognitive modelling and hence economics, finance, political science and alike fields. However, there are deeper meta principles which might lend meaning to such approaches, rather than isolatively technocratic modelling. Over the course of this paper, it becomes increasingly clear that Finance, specifically portfolio diversification theory, is one such application. Kenneth Arrow is widely regarded as one of the founding fathers of modern neoclassical economics. His contributions and legacy have influenced generations of economists, including iconic figures such as Professor Amartya Sen. Arrow’s seminal contributions in areas such as social choice theory, organizations, and diversification theory have had a profound impact on the field. This paper focuses on one such area that Arrow and Debreu (1954) developed, which then led to the generation of Nobel Prize-winning theories, such as the Capital Asset Pricing Model (CAPM). The portfolio diversification literature, which is the theory being referred to here, is a versatile area with a dense history. It encompasses standard utility-optimizing frameworks as well as behavioral models (Dhami 2017). In this paper, we do not deviate entirely from the utility framework, but rather provide a more robust basis by introducing the ‘contextual utility’ framework, which has been recently proposed (Aerts et al. 2018). In recent years, there has been a surge of interest in quantum-like modeling in social science, particularly in decision-making models (Khrennikov and Haven 2013). The main reason for this upsurge is the failure of classical decision theory, based on set theory and Boolean logic, to explain various facets of human behavior, such as the behavior of decision-makers under ambiguity. This failure has given rise to deviations or anomalies that are hard to explain from the basis of classical set theorybased decision theory. Examples of such anomalies include order effects, conjunction and disjunction fallacies, failure of the basic law of total probability, and failure of standard Bayesian probability theory to explain the updating of beliefs under uncertainty. The pioneers in this emerging field (Bagarello et al. 2015) have noted with surprise and caution that a new foundation based on Hilbert space setup and non-Boolean logic is required in decision-making theory to resolve such anomalies. In basic quantum theory, a state is supposed to live in a finite or infinitedimensional complex Hilbert space. Hilbert space is a complex vector space that has a defined norm and is spanned by basis vectors. These basis vectors are orthogonal to each other. Although the linearity property holds on the space, some logical operations, such as commutation and distributive properties, do not hold. These properties

6 Quantum-Like Contextual Utility Framework Application in Economic …

105

can be used to explain anomalies such as order effects observed in human decision behaviors if such a foundation is adapted. Unlike in the whole of classical decision theory, the Hilbert space description of an initial pure state is that of a linear superposition of basis states. Such a superposition cannot be compared with any probability distribution over states but simply a superposition or possibility of all states co-existing until any measurement is done. The coefficients of basis states in a superposition description give the probability amplitudes, the squared moduli of which provide the probability of such states getting actualized when a measurement is performed. Hence in case of decision theory modelling, or cognitive modelling a pure state or a pure belief or mental state is considered to be represented by such Hilbert space model, where the space need not to be infinite dimensional and complex. Density matrix representation can be given to ensemble of such pure mental states. Here we always note that the main purpose of such description is to measure the probability to reduce to one of the Eigen values from the superposed state, which is obtained by the famous Born’s rule of squaring the amplitudes as in the superposition description (Basieva and Khrennikov 2017). It is important to note that quantum theory is an inherently probabilistic theory, which means there is a natural limit to predict or measure observables with certainty. Unlike classical decision theory then quantum decision theory is also based on an irreducible randomness. This point is the fundamental departure from the classical deterministic philosophy where probability can come only due to ignorance. Recently, there has been a surge of studies applying quantum probabilistic formulation outside of physics, particularly in various areas of decision-making (Bagarello 2015). This novel movement has created a new branch of knowledge, the so-called quantum decision theory (QDT). The application of this theory in the areas of cognitive and mathematical psychology is pioneered by Busemeyer and others (Bruza et al. 2015), while its application in the areas of management, finance, and social science in general is pioneered by Khrennikov and Haven (2013), along with many other notable contributions by Yukalov and Sornette (2011). The basic reason for the application of this setup is that, as noted by psychologists since Kahnemann (1992), human decision-making in general cannot be captured by the standard expected utility theory (EUT), which is grounded in classical Kolmogorovian set-theoretic/measure theory of probability (Kolgomorov 1933). There are many violations of the basic axioms of EUT as proposed by its founders, such as Savage (as in Khrennikov and Haven 2013, 2009) and others. For example, the violation of the sure thing principle, the presence of conjunction and disjunction effects in human decision-making, the presence of order effects in human cognition, and the overall inability of standard probabilistic theories to explain human decision-making under uncertainty or ambiguity. Yearseley (2017) has observed in many studies that the violation of classical probabilistic theoretic predictions is mainly due to the contextual behavior of decisionmakers. Different contexts under which the same decision-maker acts make it difficult to represent the behavior based on standard measure theory. Khrennikov goes further

106

S. Patra and S. Yeddanapudi

to formulate a specific version of quantum decision theory, now called the Vaxjo interpretation, which attempts to explain the so-called interference or non-classical results of quantum probabilistic predictions based on contextual probabilities. More on this is elaborated below in the relevant literature sections. Bruza et al. (2015) have further observed that managerial/organizational decisionmaking carries all the traits that warrant the use of quantum probabilistic theory rather than standard EUT for faithful description as well as predictions. Uncertainty, contextuality, ambiguity, and various types of violations of predictions of EUT are the hallmarks of decision-making in an organizational setup. There have been recent efforts to describe organizational decision-making based on QDT (Khrennikov and Haven 2009), and such studies are gaining currency rapidly. A new paradigm is on the rise. For economic decision-making theory, it is critical to consider the limitations of the expected utility theory (EUT), since neoclassical modeling is fundamentally based on the premises of EUT. EUT has been the standard utility modeling since the works of Savage and others. This theory is so successful in general that the whole of neoclassical economic modeling is based on the applicability of EUT. However, deep down, the theory is based on the deep axioms of classical probability theory. One specific axiom is the so-called sure thing principle. Savage originally formalized the principle, and the whole EUT is based on it. The sure thing principle is quite simple to follow, and the crux of it is similar to the irrelevance of irrelevant options while making decisions. For example, if Bob is asked whether he will buy a house if presidential candidate A wins, and the answer is yes, and the answer remains yes even if candidate B wins, then Bob is indifferent to both candidates’ win, and hence, if he does not have any information on the win, he should still choose the buy option. However, there is strong evidence since Kahnemann (1992) that the sure thing principle (SUP) is regularly violated under uncertain contexts. For example, if Bob is under an ambiguity scenario regarding the wins, then he might not behave according to the sure thing principle at all. The same violation of SUP is also observed in the case of experimental data in prisoners’ dilemma scenarios, the famous game theory scenario of cooperation failure (Giloboa et al. 2008). In the case of the prisoner’s dilemma situation, there is a clear dominant strategy for each player, which they should play irrespective of what the other player chooses. However, there is very strong experimental evidence that under conditions of ambiguity or uncertainty, the behavior of players may not coincide with the prescribed Nash equilibrium. It is quite recent when the quantum theoretic setup is being used to describe certain economic or financial decision-making (Haven and Khrennikov 2013, 2009). Financial decision-making, in general, is a good candidate for quantum modeling since the information environment is generally uncertain or ambiguous. Earlier, Haven (2003) described an asset pricing model in terms of a quantum information theory setup where such a qubit setup can actually describe the uncertain information environment. Classical or neoclassical decision theory has failed to capture the uncertain information environment in finance since the typical probability distribution setup can depict risky scenarios but not uncertain scenarios (Sozzo and Haven 2017).

6 Quantum-Like Contextual Utility Framework Application in Economic …

107

Given the backdrop of successful application of quantum-like modeling in economics and allied areas, we can try to apply the same foundation to another central area of modern economics: portfolio theory. Since the works of Markowitz (1991), Sharpe (1977), Arrow and Debrue (1954), portfolio theory has been the central tenet of neoclassical economics and finance. However, here too, the problem is with describing the behavior of agents under ambiguity in a faithful way so that the model can capture the real behavior as closely as possible. There are serious limitations in the theoretical models in the standard frameworks. For example, in the celebrated CAPM or capital asset pricing model, the results are based on the assumption of homogeneous expectations of agents. Homogeneous expectations or beliefs can only hold in a certain information environment, where at least the probability distributions over future events are common knowledge. However, such a utopian world breaks down under ambiguity or uncertainty about world states. In standard neoclassical finance models, there has been a debate on the impact of uncertainty in the information environment on the asset prices, but the debate has always been without any consensus, mainly on the exact nature of the impact of uncertainty on prices (Miller 1977, was the first author to model divergence of investors’ opinions as a measure of uncertainty). Certainly, prospect theory (Kahneman et al. opcit) and other behavioral models by economists like Thaler, Shleifer, Vishney, and Shiller have captured some important deviations in so-called rational behavior as the standard utility model would claim. However, to formulate such models, different behavioral biases, heuristics, complicated utility frameworks, Bayesian learning models, etc., have been used profusely. There seems to be a lack of coherence among all such various modeling approaches. Contextuality in decision-making has been studied in detail recently using a quantum-like framework (Khrennikov and Haven 2020), Copenhagen school (or using Leifer’s (2016) term ‘Copenhagenish’ since there are intriguing differences between many sub-schools, for example, QBism (Quantum Bayesianism) and Relational quantum mechanics though stem from Bohr’s philosophy of quantum mechanics, but the way they theorize observer and agency are fundamentally different) of QM (Quantum Mechanics) have held that contexts in which observations are made directly influence the outcomes. Contexts are also equally significant for decision-making processes. Certainly, there is a huge behavioral finance literature (for example, Dhami 2017) which does base decision-making on contexts like uncertainty and ambiguity, but again such models are either based on heuristics or modifications of linear utility frameworks of neoclassical economics. Here one can observe that a quantum-like framework might provide a true alternative since it can use widely used setups like CHSH inequalities for ‘measuring’ or ‘testing’ contextuality. Here we might also observe that ‘context’ can be defined in different ways. Bohr originally thought of an ‘inseparable’ whole between the quantum system and the classical measurement apparatuses forming a context. Later, for example, in relational QM, context is any general interaction between systems. In QBISM, context is provided by the specific ‘experiences’ which agents have while updating their belief states.

108

S. Patra and S. Yeddanapudi

Another fruitful related framework is the contextuality by default framework, CbD (Contextuality by Default), whose results coincide with the quantum-like framework for human cognition. CbD takes a statistical view of modeling contexts as tuples of random variables to be measured together. Hence, the investigation is on whether they might form a joint probability distribution. QBISM has also been proposed as a general decision-making framework, which is a personal and subjective use of the quantum framework for predicting outcomes and updating beliefs. Here too, contexts play a central role. Again as we can see in the current proposal (also see very recently developed proposals by Sozzo (op cit)) the standard neo-classical utility framework it-self can be given a contextual reformulation. Here is where quantum like modelling can provide a more coherent and comprehensive base to observed behaviours under market uncertainties. The current proposal is not suggesting that we need to reject neoclassical optimization framework altogether, since we would like to maintain the underlying economic logic of ‘revealed preference’ in the spirit of Arrow-Debrue framework, but certainly make the same more robust by introducing contextual decision making, which is at best less developed in the standard framework. The application field for the current proposal is portfolio choice behaviour, since that is the central feature of an Arrow-Debrue economy. Since the model presented will be a prototype model, the tools used would be basic: finite dimensional Hilbert space modelling with standard orthonormal projectors, however in the appendix a more general version is provided or suggested with POVMs (Positive Operator Value Measures). Some specific models which have attempted to describe decision making (hence asset pricing and portfolio diversification) under ambiguity is discussed briefly below, which might help to distinguish them from the standard neo-classical models.

Epstien’s Framework Epstein and Scheneider (2007) proposed a comprehensive framework of learning by Baysian rational agents in the scenario of ambiguity, which is different from risk. Here again the reference study is that of Ellsberg paradox, which is also the reference for the current proposal. However, one important difference between their framework and quantum-like is that ‘fundamental’ or ‘ontological’ uncertainty is described better in the latter. In any Bayesian framework it is the credence of the agents which act as subjective probabilities, which describe the ambiguity atmosphere, however, in quantum-like framework uncertainty is described more deeply by the ‘linear superposition’ of states allowed in the Hilbert space structure. Such linear superposition of states cannot be reduced to classical mixture of probability states, and thus would describe ‘ontological’ uncertainty. Chen and Epstein (2002) however have developed a ‘multiple prior’ based utility framework which allows agents to exhibit ambiguity aversion. Such a framework is different from the continuous-time stochastic utility models which are based on

6 Quantum-Like Contextual Utility Framework Application in Economic …

109

standard probability assignments by rational agents. However, in real asset markets with deep uncertainty, it has been well documented that ambiguity aversion as well as ambiguity attraction both are exhibited by agents. Recent development of quantum decision theory which is dynamical in nature demonstrate both aversion and attraction, hence might be considered advantageous. Based on the above background literature, the paper is organised as follows, modelling background provides the specific approach amongst different quantumlike models which we adopt here, model set-up which provides the basic model with some examples, conclusion and discussion which invites further discussion and thoughts in the same direction.

Modelling Background Since late 1990s there is a voluminous literature developed on quantum-like modelling for human decision making. The qualifier ‘like’ is important here since by no means we propose that there is a quantum physical explanation of human brain (which might be a feasible theory, but here it need not concern us). Overall mathematical description of human decision making is the main aim here, which is generally accomplished by adopting finite or infinite dimensional Hilbert space set up as the state space for describing the decision making process, tools widely used in standard quantum theory framework, viz, self adjoint operators, Neumann- Luder ansatz for state updating or Born’s rule for probability computation, are the main ones. In case of human cognition modelling instead of usage of one-dimensional orthogonal projection operators POVM or simply positive operators have been suggested. Some authors adopt a very experimental view of quantum-like cognition models, we may draw an analogy between quantum state preparation phase and then measurement upon that state to obtain the final result which is ‘inherently’ probabilistic in nature, with the decision makers state preparation (say the experimental context in which the agent finds herself) and then measurement state which might be agents answering questions related to the choice or the mental state. Such ‘measurements’ themselves alter the cognitive states of agents, analogous to measurements on quantum systems which update the state of the system as a whole, which can again be mathematically modelled via quantum instruments. Before describing the basic quantum probability framework more, it’s better to point out some cautionary remarks about the limitations of this basic model presented, as below: First, it is a theoretical model of description of ambiguity aversion (in Ellsberg sense) for the choice over assets or portfolio diversification, with some implications for pricing. Hence the immediate limitation is a lack of thorough simulation type exercise which might be the next step from this proposal. Second, in this discrete finite dimensional Hilbert space treatment continuous-time evolution of behaviour like ambiguity aversion is not exhibited, though in the appendix some suggestions are made. Again ‘interference’ terms with its implications for ambiguity aversion/

110

S. Patra and S. Yeddanapudi

attraction is suggested. Certainly the formula for total probability, or FTP in short, is the main departure of quantum-like modelling from the more familiar measure theoretic probability formulation. One specific scenario of portfolio diversification might be ‘investor’s diversity of opinions’ which is quite explored in finance, but very recently a quantum-like framework has been used (Khrennikova and Patra 2019), the current proposal refers to the same.

Overview of Quantum-Like Modelling Set Up Basics of QP (Quantum Probability) framework We begin with a brief comparison between classical probability theory (CPT) and quantum probability theory (QPT): The main features of classical probability theory are: Events are represented by sets, which are subsets of Ω. Sample space, sigma algebra, measure (probability)*, are the main features of the related Kolmogorov measure theory. Boolean logic is the type of compatible logic with CPT, which allows for deductive logic, and basic operations like union and intersection of sets, DeMorgan Laws of set theory are valid. Conditional probability: pr obabilit y(a|b) = prprob(a&b) ob(b) ; p (b) > 0 We see conditional probability is a direct consequence of Boolean operations.1 The main features of Quantum Probability Theory are: State space is a complex linear normed vector space: Hilbert space; Finite/ infinite D, symbolized as H H is endowed with a scalar product (positive definite), norm, and an orthonormal basis, non-degenerate Any state can be visualized as a ray in this space Superposition principle: which states that a general state can be written as a linear superposition of ‘Basis states’, in information theory language the basis states are |0> or |1>. Measurement: most of the times projection postulate; Measurement implies projection onto a specific Eigen sub-space. Probability, updating can be visualized as sequential projections on Eigen subspaces

1

In QM there are different ways of introducing conditional probabilities, either sequential measures, or based on two state vector formalism (Aharnov et al.), where expectation values of observables are calculated given final and initial boundary conditions.

6 Quantum-Like Contextual Utility Framework Application in Economic …

111

Non –Boolean logic is compatible with such state space structure, which means violation of commutation and associative properties. The main features of Non-Boolean Logic are: Algebra of events is prescribed by quantum logic. Events form an event ring R, possessing two binary operations, addition and conjunction P (A U B) = P (BUA) (this Boolean logic feature is invariant in Quantum logic). P {A U (BUC)} = P{(AUB)U(AUC)} (associative, property also holds good) AUA = A (idempotency) P (A and B) /= P (B and A) (non -commutativity, incompatible variables) A and (B UC) /= (A and B) U (A and C) (no distributivity). The fact that distributivity is absent in quantum logic was emphasized by Birkhoff and von Neumann. Suppose there are two events B1 and B2 that, when combined, form unity, B1 ∪ B2 = 1. Moreover, B1 and B2 are such that each of them is orthogonal to a nontrivial event A /= 0, hence A ∩ B1 = A ∩ B2 = 0. According to this definition, A ∩ (B1 ∪ B2) = A ∩ 1 = A. But if the property of distributivity were true, then one would get (A ∩ B1) ∪ (A ∩ B2) = 0. This implies that A = 0, which contradicts the assumption that A /= 0. The main features of Quantum-like Modeling of Belief States are: Bruza et al. [27]: cognitive modelling based on quantum probabilistic frame work, where the main objective is assigning probabilities to events Space of belief is a finite dimensional Hilbert space H, which is spanned by an appropriate set of basis vectors Observables are represented by operators (positive operators / Hermitian operators) which need not commute [A, B] = AB –BA = 0 Generally, any initial belief state is represented by density matrix/ operator, outer product of ψ with itself ρ =|ψ >< ψ|, this is a more effective representation since it captures the ensemble of beliefs Pure states and mixed states ∑ Mixed states: ρ = i wi , |Ψi >< Ψi | hence mixed state is an ensemble of pure states with w’s as probability Weights. Some properties of ρ: ρ = ρ + , or it is a Hermitian operator, equal to its transposed complex conjugate, for pure states ρ = ρ^2, (ψ, ρ ψ) > 0: positivity, Trace ρ = 1. Measuring the probability of choosing one of the given alternatives, which is represented by the action of an operator on the initial belief state While making decision superposition state collapses to one single state (can be captured by the Eigen value equation). Observables in QPT represented by Hermitian operators: A = its transposed complex conjugate

112

S. Patra and S. Yeddanapudi

E (A) = Trace (A ρ), every time measurement is done one of the Eigenvalues of the A is realized. ∑ A = i ai Pi , Spectral decomposition rule: a’s are the Eigen values and P’s are the respective projectors which projects the initial state to the Eigen subspace with a specific Trace formula: p(ai) = Trace(Pi ρ). As soon as the measurement is done the state ρ’: Pi ρPi/ Tr(Pi ρ) Simultaneously updating of the agents’ belief state

Formula for total probability (FTP) in quantum-like framework. FTP is one of the most important concepts of any probability theory; simply put the unconditional probability computation of any event, via a ‘reference’ event. In standard classical probability theory we always have pr ob(A) = ∑ pr ob(A|B) pr ob(B), where A and B are two random variables representing B two different observables (or events). However in quantum-like framework, since an underlying Hilbert space structure is assumed as explored briefly above, and finally any probability is computed using square of the amplitude rule or Born’s rule, the above FTP does not remain simple. Rather there is a strong literature which shows additive terms to the above expression, such terms are often called as ‘interference’ terms. Recently (Khrennikov 2022) impact of such interference terms on decision making has been empirically demonstrated. We would like to conclude with some observations on implications of FTP in social sciences. Specifically, in financial decision making under contexts like uncertainty or ambiguity (Khrennikova and Patra 2019) the FTP formulation can play an important, and yet, not fully recognized role in resolving some long standing confusions regarding mechanisms through which market observables change. A market can be perceived as a network of agents, such that we can provide a finite dimensional Hilbert space description to the state of the market as a whole. Such a state can then undergo updating based on a similar description as we provided in the earlier sections. For example, in neoclassical finance (Khrennikova and Patra 2019), there is a long-standing debate on the exact mechanism through which agents’ beliefs about asset prices impact asset returns. As we have discussed earlier, there can be both pure state to pure state and mixed state updating of beliefs. Now in the later case we have a classical ensemble of beliefs (probability distribution over possible states) with no interference terms, such a mixed state can be analogous to standard dispersion of belief description in finance (Chatterjee et al. 2012a, b), which in neoclassical finance has been studied statistically. However, if there is a resultant pure state of the market as a whole with interference terms (as described for the bivariate observables in the earlier sections), there is no analogous description in the standard finance literature since the standard decisionmaking models are again based on classical Boolean Logic. In such a case the impact of phase factor in the FTP would be crucial on observables like asset prices. Recently (Haven and Khrennikova 2018) secondary market data has been used to reconstruct phase factors for game theoretic scenarios involving investors. In our particular framework presented here, we can exploit the geometric representation of states (say market states as a whole) on Poincare sphere, to derive phase factors.

6 Quantum-Like Contextual Utility Framework Application in Economic …

113

Model Set-Up In this basic model economy as a whole is thought of being composed of some asset classes, and the ambiguity in the investment scenario is considered as to be Ellsberg type. There is no doubt there can be various ways in which uncertainty can be introduced in a financial market/decision making model. For example based on recent works of Khrennikov et al. (2018) one can introduce generalized HeisenbergRobertson type inequalities in decision making.2 However if there is common knowledge about the proportion of assets as in this model, we might expect such ambiguities to be absent. Say the economy is composed of certain categories of assets, call it S, M, and J.3 Now if the exact proportions of these securities are a common knowledge then there is no ambiguity, and rational investors can always maximize an objective utility function, generating optimal weights and fair prices of these assets.4 However if the exact ratios, or proportions of each type of asset is not known, much like the cases in the Ellsberg pay off matrix, then there will always be some portfolio choices with ambiguity, and investors would like to avoid such acts/ choosing such portfolios, and rather would invest in such portfolios which are ambiguity free.5 Here the investors would choose based on the subjective utility satisfying rule, which can be derived from the operator formalism and Born’s rule formulation as in Aerts, Haven, and Sozzo (2017). However, from the portfolios chosen based on the subjective/ state specific utility satisfying rule would not be equivalent to optimal portfolio as in the unambiguous case here can also be some other world states, or incomplete world states, for example some with 0 M type or 0 J type assets, correspondingly there would be choices which might be completely state dependent and might not be consistent according to standard model, say CAPM.6 Overall, if ambiguity aversion persists in economy then there might be skewed distribution of asset holdings with inflationary or deflationary impact on prices. Ellsberg Paradox: The Ellsberg Paradox is a classic example of decision-making under ambiguity, where individuals are faced with a choice between two urns, each containing a mix of red and black balls. In the first urn, the ratio of red to black balls is known, while in the second urn, the ratio is unknown. The paradox arises when individuals consistently choose the first urn, even though the second urn may offer a higher expected value. This behavior is inconsistent with expected utility theory and has been a subject of much debate in the field of decision theory. 2

Compatible or in compatible questions asked to agents or investors in this case. We can imagine these to be senior, mezzanine and junior claims or securities. 4 UNDER HOMOGENOUS EXPECTATIONS, as in any asset pricing model like CAPM. 5 We can have this, for example if individual agents have different information / expectations regarding the mentioned proportions. 6 Since then these incomplete portfolios will be what held by the investors. 3

114

S. Patra and S. Yeddanapudi

Recently, quantum-like modeling has been used to explain the Ellsberg Paradox. In this framework, the urns are represented as quantum states, and the decision-making process is modeled using quantum probability theory. The unknown ratio of the second urn is represented as a superposition of different states, and the decision-maker must choose between two measurements, each corresponding to a different urn. The choice of measurement is represented by a quantum operator, and the resulting outcome is determined by the collapse of the superposition. This approach has been shown to provide a more accurate description of decision-making under ambiguity and has been applied to a wide range of decision-making scenarios (Busemeyer and Wang 2015; Khrennikov and Haven 2013). An example of the application of the Ellsberg Paradox in portfolio diversification theory is the concept of ambiguity aversion. Ambiguity aversion refers to the tendency of investors to prefer investments with known probabilities over those with unknown probabilities, even if the latter offer a higher expected return. This behavior is consistent with the Ellsberg Paradox and has important implications for portfolio diversification. In traditional portfolio theory, investors are encouraged to diversify their investments across a range of assets to reduce risk. However, in the presence of ambiguity, investors may be reluctant to invest in assets with unknown probabilities, leading to a suboptimal portfolio. To address this issue, researchers have proposed the use of quantum-like modeling to better capture the decision-making process under ambiguity and to develop more effective portfolio diversification strategies (Sozzo and Haven 2017; Haven and Khrennikov 2013).

Model Every agent has a choice, or action of selecting risky assets, s = (E 1 , x1 , E 2 , x2 , . . . ), where E’s are the events of choosing specific assets, and x’s are potential pay offs from those assets. Any standard asset pricing model will propose that the pay offs are still a function of both idiosyncratic and systematic or market wide uncertainties. Hence f is a 2n tuples of events and pay offs. However choosing an asset is itself a composite event, since every choice is related choosing over underlying uncertainties accompanying the specific assets (or idiosyncratic factors). Hence any choice can be modelled as a Tensor product of the event space say A (which is a Hilbert space spanned by basis states say {|n i >} and the uncertainty space say B (which is also a Hilbert space spanned by basis states say, {|αi >}. Hence the projector operators on the composite space corresponding to any act f of choosing any risky asset is given by: Pi = |πi >< πi |where|πi >= |ni > |αi > where the tensor product is implied

(6.1)

6 Quantum-Like Contextual Utility Framework Application in Economic …

115

Hence we can measure both the state specific utility and the probability of choosing portfolios. Say if the initial strategic state of the decision maker DM is represented by the density matrix ρ, then the probability of choosing a specific portfolio is given by the Born’s rule, pi = trace (ρPi ), where ρ is the density matrix representation

(6.2)

Again following Sornette and Yukolov (2011) p(i) can be decomposed into diagonals and off diagonal terms; pi = f(πi ) + q(πi ),7 ,8 the later part q(.) has many interesting properties which can be used to explain attraction or repulsion from a specific portfolio choice. For example, in the current model q(.) may generate aversion or attraction towards a specific ambiguous portfolio. However for our model the main purpose of decomposing the composite choice probability is to show that given the world state, or the information set available to the DMs choice making is inherently probabilistic in nature. Given the probabilistic nature of choice, now the task is to formulate the state specific, or context specific9 utility obtained by the DM. we invoke (Haven and Sozzo 2017) the action operator F =

∑

u(xi )Pi

(6.3)

i

Hence state specific utility say, I I I I∑ ∑ I I Wv =< vI u(xi )Pi Iv >= u(xi )|< πi |v > |2 I I i

(6.4)

i

And certainly all the interesting properties of comparison between state specific utilities from different actions, say f, g, h… will hold.

7

Sornette and Yukolov interpret f(2011) as the objective probability part and q(.) as the subjective probability part. If again we live in a classical world of no information asymmetry and homogenous beliefs then q(.) vanishes, as argued by Sornette and Yukolov. 8 Detailed mathematical properties of f and q parts can be found in Sornette and Yukolov, generally ∑ ∑ f is the sum of the diagonal terms; f(πi ) = b2 , and q(πi ) = b* b the sum of the off diagonal terms. 9 Action operator formulation, for the initial choice of portfolios.

116

S. Patra and S. Yeddanapudi

Specific portfolio formulations and choices under ambiguity The economy is comprised of three types of assets of different risk categories, say, (S, M, J), generally we can assume S refers to senior class, M refers to Mezzanine class and J refers to junior class. Hence the original strategic belief state of the DM comprises S,M and J. for simplicity we assume that risk free rate of return, and returns from the risk categories, and the underlying cash flows are a common knowledge. In such a world the only task which a rational DM is assigned to is maximizing U(x), which then implies asset prices based on benchmark models like CAPM.10 However, whenever we relax the common knowledge assumption, we need to formulate the state specific utility, as provided above, Wv (.). However W(.) is also dependent on the subjective attraction/ repulsion factors as shown above, which is embedded in the pi (.) terms, and these factors are dependent to some extent on the idiosyncratic nature of the underlying assets. However in our model the ambiguity is introduced on a marstate wide or aggregate level. One simple way would be to base the problem on famous Ellsberg type context, where the exact proportions of S,M and J in the economy is unknown.11 ,12 Under this scenario, there can be incomplete strategic states also like a state say, u with 0 M type of assets, or w with 0 J type of assets. Along with this there can be: 1. Ambiguous states say choosing only from asset M or asset J, where W(.) can’t be measured 2. Ambiguity free states like choosing only S where W(.) can be measured Hence in this model, there are two types of uncertainty, idiosyncratic / subjective attraction factors, and systemic uncertainty due to Ellsberg type contexts. I∑ ) Again since, Wv =< vI i u(xi Pi |v >, for any strategic belief state v, the same can also be decomposed of objective and subjective utility parts, namely:

I ) I ) I I∑ ∑ ∑ I I Wv =< vI u(xi Pi Iv >= u(xi ∗ |< πi |v > |2 = u(xi )(f(πi ) + q(πi )) I I i

i

i

(6.5) Hence while selecting between two portfolios of risky assets, the comparison will be between the differences between the f and q values of the respective portfolios. Now q (.) part of the utility or the probability measure for choosing one portfoliois taken as to be the subjective attraction factor of the DM towards that choice/ portfolio. Here q (.) needs to be interpreted as the reflection of choice under an irreducible uncertainty, only if the world state is completely a common knowledge that q(.)=0, and the utility of choosing one portfolio be totally a classical maximization problem. 10

The same problem can also be perceived from known proportions of these classes of assets, say if the ratios are a common knowledge too then always an optimal diversification will be achieved by the rational agents. 11 We can think this as a parallel construct to three urn example as discussed in Haven and Sozzo (2017). 12 Intuitively then suboptimal portfolios will be resulted.

6 Quantum-Like Contextual Utility Framework Application in Economic …

117

Specific example of portfolio selections Here we can construct a simple Ellsberg pay off matrix with three types of asset classes as in the model, S, M and J, and can then consider say 4 actions, f: ∑ Action f1 : choose S type assets only, implying F = i u(xi )Pi ambiguity free choice. Action f2 : choose J types of assets only, implying ambiguous choice Action f3 : choose from S and M only, implying ambiguous choice again Action f4 : choose from M and J only, implying ambiguity free choice

Hence the current model captures two types of uncertainties: 1. The fundamental uncertainty captured in the term q(.), however we can measure the state specific utilities here, Wv 2. We can term this as systemic uncertainty, or systemic ambiguity13 which generates from the incomplete knowledge of the proportions of asset types in the economy, which renders many portfolio choices ambiguous since Wv ’ s can not be measured, and thus agents would exhibit ambiguity aversion, or choosing suboptimal portfolios.

Partial Ambiguity Resolution: Dynamics and Hamiltonian Formulation There are recent main stream studies which have attempted to model partial resolution of ambiguity in asset allocations in a portfolio. However there is still no consensus on the process through which such resolution happens. Again since the standard paradigm is based on Kolgomorovian measure theory, or the learning or belief updating based on Bayesian probability theory there are some serious constraints, as explained in Basieva et al. (2014). For example, if the agents who are Bayesian, starts with non-informative or 0/1 priors on some events then its non-trivial to update such states to significantly different posteriors based on Bayesian updating schema. Here in the model again we have n types of securities, say for simplicity n = 3, S,M and J, these may be senior, mezzanine, and junior securities. Hence as we have already seen there can be Ellsberg type ambiguity, with unknown M and J PROPORTIONS. In such a context, there will be incomplete choices, where agents might try to select incomplete portfolios to avoid ambiguity altogether as above. However, in the current scenario of our model, ambiguity resolution can be based on sudden jumps of belief states from non-informative priors about say the proportions of asset types in the economy.14

13

Two types of uncertainties can be theorized here, one is the fundamental idiosyncratic uncertainty captured in the tensor product, and the other one is the systemic uncertainty. 14 Such sudden jumps in belief states again can not be captured by standard Bayesian modeling due to the problem of ‘zero prior’ trap, which states that if the prior of any event is 1 or 0 then any amount of new information is not capable enough to generate different posteriors.

118

S. Patra and S. Yeddanapudi

Hence to capture such jumps in belief states we invoke the creation and destruction operators, a and a*.15 Let’s consider the agent in an ambiguous state of a superposition of state 0 and state 1, say state A, where state 0 has the meaning that the belief state is that proportion of M securities < proportion of J securities, Which means that if this state is actualized then the price of the portfolio is down, Whereas state 1 means proportion of M> proportion of J securities, meaning that if this state actualizes then price of the portfolio is up. The reason behind this assumption is based on the standard portfolio theory, where greater proportion of riskier assets in the portfolio should be producing greater risk premium which should drive down the asset prices. Hence the current model is not typically a behavioral model with irrational choices; here the source of sub optimal behavior comes from genuine uncertainty in the information atmosphere. With all the mathematical properties of a and a*, we invoke the operator aa* as the price behavior operator here, such that when the operator aa* operates on the initial ambiguous state A the state collapses to state 1, aa* effectively work as the projector operator for the state state 1.

Ambiguity Aversion- Attraction and Diversity of Investors’ Opinion and Asset Pricing In finance a strong body of work is devoted to asset pricing given diversity of investors’ opinion, where the opinion can be about quality of the asset, or future prospects of cash flows. The standard literature is unclear about the impact of such an ambiguous atmosphere on the asset pricing. For example if the diversity of opinion increases due to information asymmetry problems, for example adverse selection, then ‘rational’ investors should demand a premium which should be reflected in the depressing of the prices. However there are contrary empirical evidence (Chatterjee et al. 2012a, b) where price inflation is observed due to diversity of opinions, hence suggesting rather over optimism among investors regarding fundamentals. Hence the impact is unclear. Certainly, there are cognitive bias based explanations from the behavioral camp, but in the above described quantum-like framework such inflation and depression of asset prices might be better described.

15

Such operators are used widely in quantum field theory, have their own commutation relations (anti commutation rather for Fermions) and c* aljebra (Khrennikov and Haven 2013). Such operators are used to generate excitation states from the ground states, for example creation operator will create a particle from the state I0 > and the annihilation operator will destroy a particle from the previous state. Number operator which is a critically important operator in quantum field theory can be constructed from such operators. Number operator is utilized for conservation of physical properties.

6 Quantum-Like Contextual Utility Framework Application in Economic …

119

Entropic Measures Ambiguity states in the above model, can also in general be represented by Neumann entropy measures, since it measures the purity of states. A pure state has a minimum value of 0, and a completely mixed state has a maximum value which is reciprocal of the dimensionality of the Hilbert space considered. To continue the discussion, it is worth noting that the use of entropic measures in quantum-like modeling has been shown to provide a more accurate description of decision-making under ambiguity. For example, researchers have used the concept of entropy to develop a quantum-like model of decision-making under ambiguity, which has been shown to provide a better fit to experimental data than traditional models (Busemeyer and Wang 2015). Moreover, the use of entropic measures has also been applied to other areas of decision-making, such as portfolio diversification. For instance, researchers have proposed the use of entropy-based measures to quantify the level of ambiguity in financial markets and to develop more effective portfolio diversification strategies (Sozzo and Haven 2017). These approaches have the potential to provide a more accurate representation of decision-making under ambiguity and to improve our understanding of how individuals make decisions in uncertain environments.

POVM in Decision Models Recently POVM has been used in cognitive modelling related to describing choice behaviour of agents under uncertainty, this is a very helpful tool in describing agents’ behaviour in case of uncertainty in financial markets since many interesting results like order effects can be explained. Authors (Patra and Ghose 2022) point out that positive operators are increasingly used to model decision making since in real life scenarios there can be noise in the decision-making process. Positive operators are a class of projection operators which have more general properties, for example, if E is one positive operator then it can be conceived of as E = M’M, where M is a self-adjoint operator and M’ is the transpose conjugate of M, such that for all such ∑ observations M’M = I where I is the identity operator. Again, M can be given a square for example, if ∊ is the noise in the system then M = ( √ matrix representation, ) 1− ∈ 0 √ . Noise in the system has an important interpretation in the decision ∈ 0 theory literature; for example, say due to some noise in the final choice action, or due to some error, the agent rather choosing the optimal chooses a wrong option, now such actions can be represented by positive operators, rather than more stringent projection operators as described earlier. There are several interesting properties of positive operators (Yearsley 2017), such as: they are non -repeatable (E2 is not equal to E), they are not unique, they are used when the basic elements in the Hilbert space of the model need not be orthogonal, they are used when there are more responses

120

S. Patra and S. Yeddanapudi

than there are basis states, this last property can be used in the decision making models with noise in the system. Hence, A positive operator valued measure (POVM) is a ∑ family of positive operators {Mj} such that Pm j = 1 Mj = I, where I is the unit operator. It is convenient to use the following representation of POVMs: Mj = V* j Vj, where Vj: H → H are linear operators. A POVM can be considered as a random observable. Take any set of labels α1,…,αm, e.g., for m = 2,α1 = yes,α2 = no. Then the corresponding observable takes these values (for systems in the state ρ) with the probabilities p(αj) ≡ pρ(αj) = TrρMj = TrVjρV* j. We are also interested in the post-measurement states. Let the state ρ was given, a generalized observable was measured and the value αj was obtained. Then the output state after this measurement has the form ρj = VjρV * j /(TrVjρV* j). Hence, we see that the agents may still update the belief states following the same Born’s rule.

Conclusion and Further Discussion Recently various concepts from quantum theory (whether the first quantization or quantum mechanics or the second quantization or quantum field theory) has been applied to economic theories, for example, Orrel (2016) has applied quantum uncertainty principle concepts in a proposed monetary theory. Khrennikov et al. (2018) very recently has extended the concept of Heisenberg- Robertson inequality in the context of human decision making in general. The current paper builds upon the extant literature and extends to a central area of financial economics. Ellsberg type uncertainty has not been studied carefully in the mainstream portfolio theory, and also the main stream theory uses standard probabilistic models to describe uncertainty. As has been explored in the review section these are the shortcomings of standard models. Hence the current paper is an early attempt to formulate a quantum like modeling of portfolio choice behavior under uncertainty. Since the model proposed here is based on Ellsberg type uncertainties there is an empirical side to it, or either via experimental methods or via simulations we can obtain results of choice behavior of agents which can further be investigated from the model’s predictions. Overall, the points of departure from the standard decision-making theory are: introduction of inherent randomness in choice making, introduction of Non-Boolean logic in the form of Hilbert space formulation, scope of further experimental observations based on Ellsberg type uncertainties.

Objective or Subjective Probabilities? There has been a profound debate at the heart of interpretation of QM, and much of it also revolves around the probabilistic picture of the world the frameworks depict. However over time generally two types of approaches have evolved: ontological and

6 Quantum-Like Contextual Utility Framework Application in Economic …

121

epistemological understandings of quantum state. Certainly there are deep issues, for example the difference between psi epistemic and psi ontic representations, where the former would mean there is presence of hidden variables (mostly local) which is incompletely described by the wave function or the quantum state, where as the ontic description would reject such sub quantum description of world. Then there are two specific Copenhagenish (to quote Leifer (2014)) schools, one relational quantum mechanics proposed first by Carlo Rovelli (1996 onwards) and two, QBism (Fuchs et al.). Relational view holds that relation between states or systems in general is the only ontology of the world, where ‘measurement’ is nothing else but general interactions between ‘observers’ which are just passive systems getting correlated, probabilities are objective here. QBism on the other hand is a personalist and subjective ‘user guide’ or ‘normative’ decision theory, where QM is universal and can be used by any active agency (like a human rational decision maker) to update ‘belief’ states and assign coherently right probabilities to future events. Hence here probabilities are subjective. Some authors have suggested that QBism might be more compatible with general quantum-like decision framework. Choice of QBism might be more natural since normativity is one hallmark of decision theory, for example in Arrow-Debrue asset pricing framework, the underlying assumption is that of revealed preference theory which is a normative theory based on optimization of utility functions. However the mathematical complexity would be to introduce SIC POVM (symmetric and informationally complete positive value measures) measures, which is central to the deductions of QBism. It’s an open question to prove existence of such measures for any arbitrary dimension of Hilbert space.

Application of Quantum-Like Modelling in Wider Context of Complex Economy Portfolio diversification theory is not just restricted to financial markets, but the philosophy of risk diversification is a macroeconomic issue now. While discussing macro economy it is now well recognised that (for example see Arthur 2020) it is by definition a complex adaptive system, which is foundationally different from the neo-classical general equilibrium picture of economy. The field of complexity economics (Arthur 2013) is versatile and still developing. The foundational concepts, for example, non-linear relations between different processes, self-organization of economy, agent based modelling which treats agents as heterogeneous rather representative of homogenous constrained utility optimisation, have been very well debated (Arthur et al. 2020). However, even though pioneers accept that deep uncertainty should play an important role in understanding workings of macro economy, there has not been a concrete framework in which such deep uncertainties be represented. Here quantum-like modelling in general and contextual utility modelling in

122

S. Patra and S. Yeddanapudi

particular can provide deep insight. Quantum-like modelling can be thought as individual decision-making model, but then networks of ‘quantum-like’ agents can be built. There is already a rising literature in quantum Bayesian network (Moreiera and Wichert 2016), which retains the basic structure of network theory, but consider that agents update their beliefs according to the updating scheme of quantum-like modelling, as explored in this paper also. Recently (Mutlu et al. 2021) there have been some works in connecting quantum-like modelling to social contagion phenomena. Hence overall, contextuality in decision making might provide newer insights into understanding matters from financial markets to macro economy.

Interpreting the Model- Real World Implications and Decision Making The quantum-like modeling framework proposed in this paper has the potential to provide a more accurate representation of decision-making under ambiguity. This has important implications for financial decision-making, particularly in the area of portfolio diversification. Traditional portfolio theory assumes that investors have complete information about the assets they are investing in and can accurately assess the risks and returns associated with each asset. However, in reality, investors often face ambiguity and uncertainty, which can lead to suboptimal investment decisions. This QLM framework can help address this issue by providing a more accurate representation of decision-making under ambiguity. For example, researchers have proposed the use of quantum-like modeling to develop more effective portfolio diversification strategies. In one study, Sozzo and Haven (2017) used a quantum-like model to analyze the impact of ambiguity on portfolio diversification decisions. They found that investors who are ambiguity averse tend to invest in a smaller number of assets, which can lead to a less diversified portfolio. However, by using a quantumlike model, they were able to develop more effective diversification strategies that take into account the impact of ambiguity on investment decisions. Another example of the application of quantum-like modeling in financial decision-making is the use of entropic measures to quantify the level of ambiguity in financial markets. For instance, researchers have proposed the use of entropy-based measures to develop more effective portfolio diversification strategies (Sozzo and Haven 2017). These approaches have the potential to provide a more accurate representation of decision-making under ambiguity and to improve our understanding of how individuals make decisions in uncertain environments. The implications of QLM for investment decisions vary depending on the scale of the investor. For small-scale investors, the use of quantum-like modeling can help improve their investment decisions by providing a more accurate representation of decision-making under ambiguity. For example, small-scale investors may face ambiguity when deciding which stocks to invest in, particularly if they are not familiar with the industry or the company. By using a quantum-like model, they can better

6 Quantum-Like Contextual Utility Framework Application in Economic …

123

assess the risks and returns associated with each investment and make more informed decisions. For medium-scale investors, the use of quantum-like modeling can help improve their portfolio diversification strategies. Medium-scale investors may have a larger portfolio than small-scale investors, which can make it more difficult to diversify optimally. By using a quantum-like model, they can develop more effective diversification strategies that take into account the impact of ambiguity on investment decisions. This can help them reduce their overall risk and improve their investment performance. For large-scale investors, the use of quantum-like modeling can help improve their risk management strategies. Large-scale investors may have a significant amount of capital invested in the market, which can make them more vulnerable to market fluctuations. By using a quantum-like model, they can better assess the risks associated with each investment and develop more effective risk management strategies. This can help them reduce their overall risk and improve their investment performance. For institutional investors, the use of quantum-like modeling can help improve their investment strategies and decision-making processes. Institutional investors may have a large number of clients and a significant amount of capital invested in the market, which can make it more difficult to make informed investment decisions. By using a quantum-like model, they can better assess the risks and returns associated with each investment and develop more effective investment strategies. This can help them improve their overall investment performance and better serve their clients. In summary, the use of quantum-like modeling has important implications for investment decisions across all scales of investors. By providing a more accurate representation of decision-making under ambiguity, quantum-like modeling can help investors make more informed decisions, develop more effective portfolio diversification strategies, and improve their overall investment performance. Type of Investor

Implications

Small-Scale

Improved investment decisions

Medium-Scale

Improved portfolio-diversification strategies

Large-Scale

Improved Risk management strategies

Institutional

Improved investment strategies and decision making processes

References Aerts, D., Gabora, L., Sozzo, S.: Concepts and their dynamics: a quantum–theoretic modeling of human thought. Top. Cogn. Sci. 5, 737–772 (2013) Aerts, D., Haven, E., Sozzo, S.: A proposal to extend expected utility in a quantum probabilistic framework. Eco. Theory (2017) Aharnov, Y., Vaidman, L.: The two state vector formalism: an updated review. Time in quantum mechanics, Springer (2008)

124

S. Patra and S. Yeddanapudi

Arthur, W.B., 2013, Oxford University Press. Complexity and the Economy, 1st Edition. Arthur, W.B., Beinhocker, E., Stranger, A.: Complexity economics: an introduction. Proc. Santa Fe Institute Fall Dialogue, pp. 13–25 (2020) Arrow, K.J., Debreu, G.: Existence of an equilibrium for a competitive economy. Econom. J. Econom. Soc. 22(3), 265–290 (1954) Bagarello, F., Haven, E., Khrennikov, A.: A model of adaptive decision making from representation of information environment by quantum fields. Phil. Trans. A (2017) Bagarello, F.: A quantum-like view to a generalized two players game. Int. J. Theor. Phys. 54(10), 3612–3627 (2015) Basieva, I., Khrennikov, A.: Decision-making and cognition modeling from the theory of mental instruments (2017) Bhattacharya, S.: Imperfect information, dividend policy and the ‘bird in the hand fallacy.’ Bell J. Econ. 10, 259–270 (1979) Bruza, P.D., Wang, Z., Busemeyer, J.R.: Quantum cognition: a new theoretical approach to psychology. Trends Cogn. Sci. 19, 383–393 (2015) Busemeyer, J.R., Wang, Z.: Quantum models of cognition and decision. Cambridge University Press (2015) Chatterjee, S., John, K., Yan, A.: Takeovers and divergence of investor opinion. Rev. Finan. Stud. 25(1), 227–277 (2012b) Chatterjee, Sris, John, Kose, Yan, An.: Takeovers and divergence of investor opinion. Rev. Finan. Stud. 25, 227-277 (2012a) Ellsberg, D.: Risk, ambiguity, and the Savage axioms. Quart. J. Eco. 75 (1961) Epstein, L.G., Schneider, M.: Learning under ambiguity. Rev. Econ. Stud. 74, 1275–1303 (2007) Epstein, L.G., Zhang, J.: Subjective probabilities on subjectively unambiguous events. Econometrica 69, 265 (2001) Fabio Bagarello, Irina Basieva, Emmanuel M. Pothos, Andrei Khrennikov: Quantum like modeling of decision making: quantifying uncertainty with the aid of Heisenberg–Robertson inequality. J. Mathem. Psychol. 84:49–56 Gilboa, I., Postlewaite, A., Schmeidler, D.: ìProbabilities in economic modelingî. J. Econ. Persp. 22, 173–188 (2008) Haven, Emmanuel, Khrennikov, Andrei: Quantum-like Modelling. Asian J. Econ. Banking (2020) Haven, E.: A Black-Sholes Schr¨odinger option price: ‘bit’versus ‘qubit.’ Physica A 324, 201–206 (2003) Haven, E., Khrennikov, A.: Quantum social science. Cambridge University Press (2013) Haven, E., Khrennikova, P.: A quantum-probabilistic paradigm: non-consequential reasoning and state dependence in investment choice. J. Mathem. Eco. 78(C), 186–197 (2018) Khrennikov, Andrei, Emmanuel, Haven, Quantum Social Science, CUP (2013) Khrennikov, A., Bagarello, F., Basieva, I., Pothos, E.M.: Quantum like modelling of decision making: quantifying uncertainty with the aid of the HeisenbergRobertson inequality. J. Math. Psychol. 84, 49–56 (2018) Khrennikov, A., Haven, E.: Quantum mechanics and violations of the sure-thing principle: the use of probability interference and other concepts. J. Mathem. Psychol. 53, 378-388 (2009) Khrennikov, A.: “Social laser:” action amplification by stimulated emission of social energy. Phil. Trans. Royal Soc. (2017) Khrennikov, Polina, Patra, Sudip: Under review unpublished working paper (2018) Khrennikov, A., Haven, E.: Quantum mechanics of decision making. J. Math. Psychol. 57(6), 301–313 (2013) Khrennikova, P., Patra, S.: Asset trading under non-classical ambiguity and heterogeneous beliefs. Physica A 521, 562–577 (2019) Khrennikov, A.: On applicability of quantum formalism to model decision making: can cognitive Signaling be compatible with quantum theory? Entropy 24(11), 1592 (2022) Knight, F.H.: Risk, uncertainty, and profit, New York (1921) Lancaster, Tom, Blundel, Stephen: Quantum Field Theory for Gifted Amateur, OUP (2014)

6 Quantum-Like Contextual Utility Framework Application in Economic …

125

Leifer, M.S.: Is the quantum state real? An extended review of psi -ontology theorems (2014). arXiv preprint arXiv:1409.1570 Leifer, M.S., Pusey, M.F.: Is a time symmetric interpretation of quantum theory possible without retrocausality? R. Soc. A 473, 20160607 (2016) Markowitz, H.: Foundations of portfolio theory. J. Finance 46, 469–477 (1991) Miller E.M.: Risk, uncertainty, and divergence of opinion. J. Fin. 4 (1977) Moreira, C., Wichert, A.: Quantum-like bayesian networks for modeling decision making. Front. Psychol. 7, 11 (2016) Mutlu, E.C., Ozmen Garibay, O.: Quantum contagion: a quantum-like approach for the analysis of social contagion dynamics with heterogeneous adoption thresholds. Entropy 23(5), 538 (2021) Neumann, J. Von: Mathematical Foundation of Quantum Mechanics, Google Books.com (1932) Orrell, D.: A quantum theory of money and value. Econ. Thought 52(1776), 19–28 (2016) Patra, S., Ghose, P.: Classical optical modelling of social sciences in a Bohr-Kantian framework. In: Credible Asset Allocation, Optimal Transport Methods, and Related Topics, pp. 221–244. Springer International Publishing, Cham (2022) Penrose, R.: The Emperor’s new mind. Oxford University Press (1989) Savage, L.J.: Foundations of statistics. Wiley, New York (1954) Sharpe, W.: The capital asset pricing model: a ‘Multibeta’ interpretation. in H. Levy, M. Sarnat (eds.), Financial Decision Making Under Uncertainty, Academic, New York (1977) Sozzo, S., Haven, E.: Quantum-like modelling of subjective uncertainty and ambiguity in economics. J. Math. Econ. 69, 31–45 (2017) Stiglitz, J.E., Weiss, A.: Credit rationing in markets with imperfect information, Part I. Am. Eco. Rev. 71, 393–410 (1981) Susskind, L., Art, Friedman: Quantum Mechanics: The theoretical minimum, Basic Books (2014) The Palgrave Handbook of Quantum Models in Social Science. Applications and Grand Challenges. Haven E and Khrennikov A (eds). Palgrave Maxmilan UK, pp. 75–93 Tversky, A., Kahneman, D.: Advances in prospect theory: cumulative representation of uncertainty Journal of Risk and uncertainty, – Springer (1992) Yearsley, J.M.: Advanced tools and concepts for quantum cognition: a tutorial. J. Math. Psychol. (2017) Yukalov, V.I., Sornette, D.: Decision theory with prospect interference and entanglement. Theory and Decision (2011)

Part II

Complexity Modeling in Economics and Public Policy

Chapter 7

Complexity Economics: Why Does Economics Need This Different Approach? W. Brian Arthur

Abstract This is the reproduction of the talk given earlier at Santa Fe Institute which has appeared in the volume Complexity Economics: Proceedings of the Santa Fe Institute’s Fall 2019 Symposium.

W. Brian Arthur Good evening. What I want to talk about tonight is how complexity economics came to be, at the Santa Fe Institute here, what it’s about, and above all what difference this new approach makes. People often say to me, you have this complexity economics or nonequilibrium economics, call it what you might, but isn’t it something of an add-on to economics? It gives agent-based models, and emergent phenomena, and self-organization, and all that, but isn’t it really a bolt-on to standard economics? So, what does complexity economics provide that’s different, that would change the way we see the economy or understand the economy, or change policy? Where does it fit in? And how should we think of it? Let me begin from some basics. The economy, whether in Europe or the US, is an enormous collection of arrangements and institutions and technologies and human actions, buying and settling and investing and exploring and strategizing. It’s a huge hive of activity, where the individual behavior of agents—banks, consumers, producers, government departments—leads to aggregate outcomes. A couple of hundred years ago, economists such as Adam Smith noticed in this that there was a loop. Individual behavior leads to aggregate outcomes, and aggregate outcomes— the aggregate patterns in the economy—cause this individual behavior to adapt and change. The individual elements of the economy in other words react to the patterns they cause. It is this loop that tells us the economy is a complex system. In this sense complexity is not new in economics. As in all complexity, individual elements (human W. B. Arthur (B) PARC (Part of SRI International), Palo Alto, USA e-mail: [email protected] Santa Fe Institute, Santa Fe, USA © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 A. Chakraborti et al. (eds.), Quantum Decision Theory and Complexity Modelling in Economics and Public Policy, New Economic Windows, https://doi.org/10.1007/978-3-031-38833-0_7

129

130

W. B. Arthur

agents in this case) react to the pattern they create. Complexity, in a phrase, is about systems responding to the context they create. For the first hundred or so years of the field, many economists thought in these terms, but this view wasn’t easy to formally analyze. So economics came up with a simpler way to look at things. Starting about 1870 it asked what individual behavior would lead to a collective outcome that would validate or be consistent with that behavior—what outcome would give behavior no incentive to change? The situation would then be in stasis or equilibrium and we could much more easily look at it via equations and mathematize economics. We could keep the object still while we were examining it This was a kind of finesse; it was clever because you’re saying if we can simplify the economy into an equilibrium system, we can reduce economics to algebraic logic. If you are willing to assume in the part of the economy you are looking at that individual behavior produces an outcome that doesn’t give that behavior any incentive to change, you have the idea of a solution. This worked. It’s a brilliant strategy: it gave insight, you can teach it, and you can use elegant mathematics. I was trained as a mathematician and this attracted me into economics. I thought I could make a killing because I knew quite sophisticated mathematics. § It’s worth looking more formally at how this equilibrium finesse works in modern economics. You take some situation in economics, whatever it might be; it could be the theory of asset pricing, or of insurance markets, or international trade, and you define this as a logical problem. You assume each category of agents—they might be consumers, or investors, or firms—has identical agents. They are all the same, and they know that everyone else is identical to them, and that they all face the same problem. Further you assume the problem is well-defined, and agents are infinitely rational, so they can optimize given the constraints of the problem and arrive at the best outcome for themselves, knowing others are behaving the same way. What overall outcome validates and gives no incentive to change their individual behavior? This is the basic recipe for doing theoretical economics, and its modern version is largely due to Paul Samuelson and a few economists who preceded him. The recipe is quite sensible in a way. It gives us a logical approach that can be made mathematical, and therefore scientific. I’ve put this in a slide (Fig. 7.1) that’s a bit complicated. The payoff is that if you can understand this slide it’s worth five years of graduate school in a good economics department, because this is the standard method they teach you. And so everything in economics is brought into this framework and put through this form of analysis if it’s to be seen as theory. If agents were identical drivers of cars in crowded traffic, this is like asking what speed would produce a traffic flow where no car has any incentive to change. It has a rather lovely property of everyone achieving what is best reachable for them in an outcome where everyone else is attempting to achieve the best reachable to them. Any time I look at this, I find it beautiful. It’s elegant. It’s pure and it’s perfect. You can prove theorems showing that if people behave this way, it’ll be in some way optimal or efficient and it’s highly mathematical. I want to say that this particular way of looking at the economy—it

7 Complexity Economics: Why Does Economics Need This Different …

131

Fig. 7.1 Standard approach in economics (reproduced with permission)

may be mechanistic; a lot of it was borrowed in the late 1800s from physics—has been useful. It has allowed economists to solve problems of finance, of central banking, or industrial production. We understand markets, we understand international trade, we understand currency regulation, and many other issues. We really understand large parts of the economy and we’re not having major depressions like we had in the 1930s. This way of looking at the economy has achieved a lot. But of course there’s a “but.” A very large “but.” And that is if we assume equilibrium so that we can do mathematics, it puts a very, very strong filter on what we can see. If we can only look at equilibrium situations there’s no scope for adjustment or for exploration or for creating startups or for any phenomenon in the economy like bubbles and crashes that appear and disappear like clouds in the sky. There’s no structural change; there’s no unexpected innovation—they’re not equilibrium things. And, if everything is the same over time, time disappears and there’s no history, so there’s no path dependence. We accepted equilibrium because it is so analytically useful, but it gives us a Platonic universe. It’s beautiful, and ideal, and pristine, and lovely, but it’s not really real. So economists in the last fifty years have begun to question equilibrium. They’re questioning the standard theory’s authenticity. I asked the computer scientist John Holland once in the early days at Santa Fe what he thought of equilibrium. Anything at equilibrium, said John, is dead. So the question is, is the economy more alive, more vital, than equilibrium would show us? This was the question that landed in the late 1980s in Santa Fe. § In 1987, the Santa Fe Institute was in the old convent on Canyon Road and it was very much a startup. My colleague Kenneth Arrow at Stanford brought me there, and he brought several other economists there. He and the physicists David Pines and Phil Anderson had brought some scientists to meet their counterparts in

132

W. B. Arthur

economics. One of the people on the scientific side was John Holland. Another was Stu Kauffman. There were quite a few people: David Ruelle, mathematician, and physicists Richard Palmer and Doyne Farmer. On the economic side we had Larry Summers, Tom Sargent, Buz Brock, and other eminent economists. The two groups met and paraded ideas and this went on for ten days. It was awkward at first but in the end extraordinarily successful. We got a kind of sugar high on the ideas, and the Institute, after this now-famous conference, decided it would start its first program that was going to be funded by Citibank, by John Reed. Arrow and Anderson and Pines would be the godfathers of this, and they asked John Holland and myself to run the program. John couldn’t get away from Ann Arbor, so I was brought in on a sabbatical from Stanford to lead this program. That was the good news. You know, “Brian, you’re running a program on ‘The Economy as a Complex System.’ You’re going to have the equivalent of a couple of million dollars, which allows you to buy out quite a few people on sabbaticals. You can have Arrow and Anderson (both Nobel Prize winners) do the inviting so nobody is going to say no. You can bring them all to a convent in the Rockies. They can all think what thoughts they want and nobody’s going to object. It’ll be isolated and nobody’s going to know what you’re doing.” The bad news was that when we got together in the fall of 1988 we weren’t sure what to do. I remember Ken Arrow saying, “Well, we could do something in chaos theory. That’s interesting.” I was thinking we could do something on network effects or increasing returns, but I’d already spent many years doing that and was tired of it. We thought maybe we could take spin glasses or something from physics and translate that into economic terms. Nothing was quite settled, and this went on for about a month. We’d meet in the kitchen of the old convent. We’d have coffee, and we’d say, “Okay, what’s the theme going to be? What’s the direction going to be?” Finally I called Ken in Stanford to ask him what direction we should move in, and he called Phil Anderson, who called our funder, John Reed, the chairman of Citibank. The word came back, and it was unexpected. “Do whatever you want, providing it is at the foundation of economics and is not conventional.” I was stunned. I couldn’t believe it. So we had permission to do what we wanted but still didn’t know what that might be. Strangely, it was someone who didn’t know anything about economics, Stuart Kauffman, who broke the logjam. One morning Stu said, “You know, I’m listening to all of this,” he said, “and you keep talking about equilibrium. It’s like a spider’s web where everything’s in equilibrium, the economy, but what would it be like to do economics not in equilibrium?” I tried to quiet Stuart. I thought this wasn’t a good idea. This was freaky. We weren’t going to mess with that. It was a kind of third rail in economics and I didn’t see how we could do it. But Stu had planted a seed. It started to grow in my brain and in other people’s brains and we started thinking seriously about it. § Let me show you for a moment what it’s like to think outside equilibrium. Here’s a realistic situation. I have in mind typical problems these days in Silicon Valley

7 Complexity Economics: Why Does Economics Need This Different …

133

Fig. 7.2 The fundamental uncertainty (reproduced with permission)

where I work. Maybe you’re starting up a company or several companies are starting up in the autonomous trucking industry, where there might be fleets or convoys of driverless trucks rolling across the country. Each company is trying to figure out how to strategize, how much to invest, what the technology should be. In a case like that, it’s not at equilibrium. The companies don’t know what they’re going to face. They don’t know the technology or how it will work out. They don’t know who the other players are going to be in this business. It could be Google—likely Google. It could be Amazon even. They don’t know what the reception is going to be for this new technology. They don’t know what the regulations are going to be, the legalities, the insurance arrangements. And yet they’re in a position where they have to ante up maybe two, three, four billion dollars just to sit at this casino table and play. Now, an economist would say that’s not a question of probabilities. It’s a situation where the firms face what economists call fundamental uncertainty. As Keynes put this in 19371 “the prospect of a European war … the price of copper in three years’ time …. About these matters there is no scientific basis on which to form any calculable probability whatever. We simply do not know.” Of course our firms know plenty, but if they want to define a real optimization problem or calculate probabilities, they simply do not know (Fig. 7.2). There’s a syllogism here. If a situation is subject to some high degree of fundamental uncertainty, the problem it poses is not well defined—you can’t express it in clear logical terms. If the problem is not well defined, rationality is not well defined— there can’t be a logical solution to a problem that is not logically defined. There is then no optimal solution, no optimal set of strategies, for agents and players—optimality is not well-defined. So if you simply don’t know what you’re facing, rationality isn’t well defined and optimal behavior isn’t well defined. So where an economic situation 1

Keynes, J.M. The General Theory of Employment. Quarterly Jnl Econ, 51, 209–33 (1937).

134

W. B. Arthur

contains significant fundamental uncertainty, if we are rigorous it can’t be credibly reduced to a rational, deductive economic model. And where rational, deductive behavior is not well defined, any outcome is likely to be temporary and not in equilibrium. I remember sitting in Santa Fe in the early days and saying to Ken Arrow, “We should do something out of equilibrium, Ken. We should do something that has fundamental uncertainty. Don’t you think that’s important?” And Arrow, who was our godfather and arguably the top theorist in the world, looked at me and he said, “I realize these are huge problems,” he said, “but what can you do about them? What can you do?” In the face of real, fundamental uncertainty, the standard approach gets stalled. § This is where we were in the fall of 1988. And we talked a lot about it. But something had resonated from the previous year, and it came as an inspiration from a modest, diminutive man. In 1987, I had a house Santa Fe Institute had rented for me with a housemate who turned out to be John Holland. I had no idea of what he did; I had no idea that he was a large figure in computer science. I’d had a lot of discussions with him late at night over beer, and we talked a lot about John’s passion of teaching computers to play chess or checkers. A month later when we held this big symposium, John gave a talk the first afternoon. I listened and he was on about genetic algorithms and classifier systems and how to teach machines to get smart playing board games—how to write algorithms that could change their code and get smarter as they develop. We now call this evolutionary programming. John had all this down and was describing it and I’m sitting there thinking, there’s something here we need to pay attention to. And the nearest I got to figuring out what it might be was just a hunch. What John was saying seemed to me immensely important. I sat there thinking, if John Holland is the solution, what is the problem? [Laughter] John—and it took me quite some months to realize this—was pointing out two things: that people, or, if you like, automata, algorithms—can and do act in situations that are not well defined. This was mind blowing for us in the social sciences. Of course we do that. We enter contracts like marriage—not well defined. We decide we’re going to get into Buddhism, but we don’t know what that is. We do these things all the time and John was pointing this out, but I’d been overeducated. Second, in not-well-defined situations, John said, in various ways people try to make sense: they experiment, they explore, they adjust, they readjust, but not just in terms of having some wondrous mathematical model of the situation and updating a parameter. They form a hypothesis, maybe they have multiple hypotheses or ideas about the situation they are in, and they put more belief in the ones that work over time and throw out hypotheses that don’t. This resonated with me, at least a bit later as our program progressed. I was working for a large bank in Hong Kong, in their Analytical Division. I had a team and we were trying to crack the foreign-exchange market at three in the morning when all the play was happening. We were using what now would be called machine learning. I noticed that the actual traders had hypotheses: “Oh, such and such has just

7 Complexity Economics: Why Does Economics Need This Different …

135

happened, the central bank of New Zealand has just come into the market. But hang on, I think the Chinese are going to do that. And if the Chinese central bank does this, then that’s going to happen.” But then something else would happen. They’d throw out the hypothesis they were using and take in others. John had been explaining all this to us and the breakthrough was I realized we could do this in economics. We could look at problems that were not well defined, whether there’s fundamental uncertainty, problems where there is no equilibrium, and we could unleash agents that could explore, take up hypotheses, throw them out, and generate new ones and get smart. We would have to use computers of course to track all this, and they would have to be based on the behavior of multiple “agents”—artificially intelligent computer programs that could have multiple hypotheses and use or change these. There was no name for this in 1990. And people were already writing programs that had multiple elements following rules and getting smarter. Not long after that the approach came to be called agent-based modeling. Agent-based modeling emerges from this approach as a natural technique. Let me come back to our idea of an economic approach. We could model agents who had disparate ideas or hypotheses about the situation they were in, they could base their actions on these, and learn which hypotheses worked and which didn’t, over time getting smarter. Notice one thing here: these very actions of agents’ exploring, changing, adapting, and experimenting further change the outcome, and they’d have to then re-adapt and re-adjust. So, they are always re-adapting and re-adjusting to the situation they create. This behavior is at the heart of complexity. I remember being asked in the middle of a talk like this, “Professor Arthur, how would you define ‘complexity’?” I could have talked about elements reacting to the situations those elements create. But I had a better definition, “Have you ever had teenage children?” [Laughter] Here’s what I’m saying. In general in a difficult or novel economic situation, there’s no optimal solution. Agents are coping, exploring, adjusting, experimenting, whatever. But that very behavior changes the outcome and then they have to change again. It’s a bit like surfing, where you don’t know where the wave will go next and you’re adjusting and readjusting to stay in the green water. This lands us in a world where forecasts and strategies and actions, whichever you’re using for that sort of situation, are getting tested within a situation—I would like to call it an ecology—those forecasts and strategies and actions create. And, if you look at it that way, that’s the essence of agent-based models. Complexity economics is basically seeing the economy not as a machine, balanced and perfect and humming along, but as an ecology. New strategies, new things are coming and going, and striving to survive and do well in a situation they mutually create. We can describe this algorithmically, but not easily by equations, not just because the situation is complicated to track but because new behaviors and categories of behavior are not easily captured by equations. This approach began to clarify itself in Santa Fe in the early 1990s, and there were other pods of researchers doing parallel work elsewhere. By the end of the decade the

136

W. B. Arthur

approach needed a name. In 1999 I was publishing a paper in Science on complexity and the economy. The editor called me from London and said, “You need a name for this approach.” I said, “No, I don’t.” “Yes, you do.” “No, I don’t.” He prevailed, and standing in a corridor on a landline I called it “complexity economics.” I thought it should have been called “nonequilibrium economics” afterwards, but it was too late. It’s been locked in. By the way, I’m not saying here that human agents in a particular situation are algorithmic, that they’re walking around like robots. It’s more subtle than that. You can model people learning in new situations where they don’t know. The outcomes likely will not be in equilibrium. So people adjust again. The outcome changes again. More readjustment. The situation may eventually settle or it may show perpetually novel behavior. And one other thing. With algorithms or computation, you can model agents acting as well. This is because if you describe something algorithmically, you can allow verbs. Equations only allow quantitative amounts of things—nouns. With algorithms you can directly use verbs: agents can buy, sell, change their minds, throw things out, create new things. These are actions. Verbs. So this new economics doesn’t have to be about the levels of prices, or interest rates, or the amount of production, or levels of consumer trust. In complexity economics you get real verbs—real red-blooded actions—happening, and the state of affairs they produce could be complicated and real.

What Difference Does Complexity Economics Make? I’ve given a picture of how complexity economics came to be. Now I want to go back to our original question and ask what difference does it make? Complexity economics I believe makes practical differences to economics, and also a difference to how we understand the economy. I’ll point quickly to four specific areas. 1. The persistence of financial crises. Standard economics has become good at dealing with recessions and preventing full-scale depressions, but if you look at financial crises in developed countries, these haven’t abated in the last fifty years. If anything they have got worse. And when they hit, they cause serious hardship and disruption. Contrast this with the number of fatalities to passengers on jet aircraft. Around 1950, this were about forty-five fatalities per million passenger miles. Now it’s down to four or five, close to zero. Passenger flying, seismic codes, cardiac procedure deaths have all improved greatly in the last fifty years. Your overall health has improved, your pet’s health has improved. What hasn’t improved is the persistence of financial crises. In Russia’s big bang around the early 1990s, Russia decided to go capitalist, but the thinking was equilibrium based. Old socialist planning equilibrium. New capitalist equilibrium. What might happen in between was fuzzy, not understood. In reality a small group of private players took control of the state’s newly freed assets for their own benefit and industrial production plummeted. You can see the results of this in the vicious mortality

7 Complexity Economics: Why Does Economics Need This Different …

137

statistics Russia suffered for five or ten years after, because they got it wrong. Similarly in 2002 California freed up its electricity market, and a small number of outside suppliers manipulated the market to their own profit. The state’s finances were put in jeopardy. And we all remember the US subprime mortgage collapse of 2008, where exotic derivative products and negligent oversight caused an unstable structure to spectacularly collapse. Each of these systems was manipulated or “gamed,” and all broke down. Nearly always when you see a financial crash, you’ll find a small group exploiting part of the economy; this causes wider fissures and failures and may eventually lead to a collapse. After the financial crisis of 2008, in England the Queen famously asked, “Why didn’t economists anticipate this disaster?” It’s a good question. Some did of course. A few said, “We have this weird mortgage-backed securities market in New York and I don’t feel good about it.” But overall the profession does not excel at seeing trouble ahead and unlike engineering disciplines it doesn’t have a branch of forwardlooking failure-mode analysis or of post-crash forensic analysis. Why is this? The reason I think is subtle. If you assume equilibrium, which economists do, then by definition cascades of collapse cannot happen. It’s like assuming an engineering structure is in stasis, therefore it can’t collapse. Similarly at equilibrium there’s no incentive for any agents to diverge from their present behavior—that’s the definition of an equilibrium. Therefore, exploitive behavior can’t happen. A subtle, muted, unconscious bias precludes ideas of exploitation or collapse. Complexity economics, by contrast, sees the economy as a web of incentives for novel behavior. It sees the economy as open to new behavior, and that might turn out to be exploitive behavior. And it starts to ask, we are in this situation now, or we have this policy system in place now, what’s the likely response going to be? Can someone exploit the system? Can some group game it? These are healthy questions to ask. The economy is not a closed static equilibrium system; it is a system perpetually open to novel behavior, and complexity economics forces us to keep this in mind. 2. Awareness of the propagation of change. The second thing complexity economics can help a great deal with is an awareness of the propagation of change. If you’re only looking at equilibrium, there isn’t any change, and so there isn’t any propagation of change. But, as we heard in a previous talk from Matthew Jackson, banks—or people, or economic agents in general—can and do affect each other, and they do this through networks of connection: trading networks, information networks, lending and borrowing networks, networks of disease transmission. In these networks, events can trigger events and failures can cascade and cause disaster. Banks in distress can infect other banks. If I’m a bank and I’m in trouble and fail, then I pass on distress to my creditor banks who may pass on distress to their creditor banks. Stress can cascade through a system like this. Equilibrium economics traditionally didn’t cope with this sort of thing very well. It traditionally assumed that firms were independent, and so changes would be independent, and so their sizes and aggregate effects would be distributed normally. One of the great things about complexity is we now understand this sort of thing—interconnections between firms and agents and how these work—much, much better. We understand that individual banks and firms are not independent. Events with one bank

138

W. B. Arthur

can trigger events in other banks in the network, and so systemic risk—overall risk to the system as a whole—is not the summation of independent events, but it is reflected in domino-style avalanches of various sizes and duration. This gives higher probability to large disturbances than normal distributions would predict. And it yields power laws of various kinds. Propagation lengths, sizes of firms, and magnitudes of cumulated events are distributed not normally but according to some law where lengths or sizes or magnitudes fall off logarithmically, much like earthquakes do on the Richter scale. This sort of thinking is now crucial in modern finance and derivatives trading, and understanding gained from complexity in general and network behavior in particular excels in this area. 3. A Strong Link with Political Economy. Complexity economics also connects with the insights of earlier political economists and this gives both a depth and a validation to its insights. And it opens up a venerable literature to modern perusal. This isn’t obvious, so let me explain. I realized quite a while ago that in economics there are two large groups of problems. See Fig. 7.3. You could call one group allocation problems. That’s international trade theory, strategic outcome theory or game theory, general equilibrium theory, etc. Those are all well understood. And they have been mathematized, heavily mathematized. They are about quantities and preferences and prices in balance, and this gives itself to the equating of things—to balance and to equilibrium. The other group of questions are ones of formation. How do economies come into being? How do they develop? How does innovation work? Where do institutions come from? How do institutions change things? What really is structural change, and how does it happen? These latter questions of formation are all ones that complexity economics can look at. Formation precludes equilibrium, and it’s about how structures or patterns form from simpler or earlier elements. That’s what

Fig. 7.3 Two great problems in Economics (reproduced with permission)

7 Complexity Economics: Why Does Economics Need This Different …

139

complexity is about, par excellence, and complexity opens the door to rigorous study of all these questions of formation. Formation is not new just because complexity has discovered it. It was studied in earlier economics, particularly within the older schools of classical economics, political economy, and Austrian economics. These differ somewhat, but together they provide a view of the economy as not necessarily being in equilibrium, always being in process, always subject to historical contingency, always creating and responding to a rich context it creates. None of this can be mathematized easily into balance equations; but it can be studied via the methods of pattern formation, network analysis, and algorithmic models heavily used in complexity economics. And so not only can we can meet up with these older and wiser approaches to economics and learn a lot from them, we can help make rigorous their insights, and thereby see the economy not as a static system, but as one in process, always exploring, and always forming itself from itself. 4. We can model reality more rigorously. Complexity economics sometimes uses mathematical equations, but more often its models are complicated, so to track things properly we have to resort to computation. This gives us if used properly an important advantage. Mathematical models in economics are forced to be kept simple, largely to allow pencil-and-paper analysis. They typically track rates of how aggregate variables influence aggregate variables—how average wages, say, are related to unemployment rates. And they need to be based on simple assumptions: identical agents that are rational and behaving in an equilibrium setting. With algorithmic models we can include as much detail as we want, to an arbitrary depth. And so we can free models from the tightness and inaccuracy of unrealistic assumptions. We can have agents who are realistically diverse; we can include details of how they interact, of the institutions that mediate these interactions, details of networks and interconnections. Of realistic behavior. We have to be careful here, as with all modeling it’s easy to lapse into throwing in useless or inaccurate detail. But done properly such detail adds realism—and thereby rigor. Not surprisingly this new realism is giving us much better understanding of the processes of the economy, how, say, the 2008 crash happened, how diseases actually transmit, how economic development actually takes place. More precise detail allows give sharper resolution to the instrument of economic analysis and we see things that would not be visible otherwise. I don’t think there is an upper limit to what we can learn here.

Closing Thoughts Let me close with some thoughts here. Complexity economics brings a different view of the economy—a different understanding to the economy. In standard economics, problems are well-defined, solutions are perfectly rational, outcomes are pure, possibly artificial in a way, but above all they are elegant. And by assumption outcomes are in equilibrium. Loosely the economy is

140

W. B. Arthur

in equilibrium. Standard economics in a word is orderly. To borrow architect Robert Venturi’s phrase, it has “prim dreams of pure order.” In complexity economics, agents differ and in general lack full knowledge of each other and of the situation they are in. Fundamental uncertainty is therefore the norm; ill-defined problems are the norm; and rationality is not necessarily well defined. Agents explore and learn and adapt and are open to novel behavior. Outcomes may not be in equilibrium. Complexity economics deals with historical contingency, path dependence, and to quite a degree indeterminacy. It is organic, with one thing building on top of another. It is a cascade of events triggering other events, and so it is algorithmic. To borrow another phrase of Venturi, it’s full of messy vitality. In philosophical terms, standard economics I would say is Apollonian, pure and ordered; complexity economics is Dionysian—it is structured and generative, with just a dose of wildness. To go back to our original question, is this economics just a bolt-on to standard economics? Absolutely not. Nonequilibrium includes equilibrium, so it’s a more general theory. It’s relaxing the restrictions of equilibrium economics—welldefinedness, rationality, identical agents—and thus generalizing standard economics. Does this go with the Zeitgeist? I’d say yes. The sciences from roughly 1900 to about now have gone through a shift, I believe, from order, determinacy, deductive logic, and formalism to formation, indeterminacy, inductive reasoning, and organicism. There are many reasons for this. Part of it is the rise of biology and molecular biology as serious sciences and computer science and new types of mathematics. And, by the way, one of the symptoms of this is complexity itself. Complexity is not the cause of this viewpoint; it is more the outcome. Let me finish with a rather beautiful allegory that the economist David Colander has given. Colander says, imagine that around 1850 all economists are gathered together at the foot of two huge mountains and they decide they want to climb the higher mountain. But both mountains are in the clouds and they can’t see which one’s higher and which one is not. So they decide to climb the one that’s more accessible, the one that they can get their equipment through to, and the one where the foothills are better known. This is the mountain of well-definedness and order. They get to the top of that and once they’re above the clouds, they declare, “Oh my God. This other mountain, the one of ill-definedness and organicism, is higher.” That’s the mountain we are starting seriously to climb, and much of the journey started at Santa Fe. [Applause]

Questions and Answers Alex Hess I’m with Third Point. About financial crises, you said there’s always a small group of actors that create the crisis. In the case of the 2008 financial crisis, who do you think that group constitutes? W. B. Arthur I don’t know this history very exactly, but a number of people in the audience do know it well and could answer this better than I could. What really

7 Complexity Economics: Why Does Economics Need This Different …

141

caused this crisis, I think, were people who allowed good credit ratings to be given to what turned out to be worthless products and derivatives. What I would point out is this: I’m trained as an engineer and after the crash I read a lot about failure mode analysis in aircraft. What you learn is that there’s always some tiny crack that appears, some tiny event that triggers another event that might trigger another event. Nearly all the time nothing disastrous happens. But sometimes these events triggering events lead on and eventually disaster happens. Why should complexity economics be any better looking at things like that than static equilibrium economics? Complexity economics is a way of looking at the economy based on events triggering events. Static economics equilibrium, looks at, well, equilibrium. It doesn’t give you any feel for this triggers that, that triggers this. An economics that’s built on stasis isn’t biased toward seeing cascades of collapse. Vitaliy Katsenelson: I’m a value investor. My question to you is, there is an underlying assumption in modern portfolio theory that people are rational. That seems to be the same assumption in traditional economics. Are there similarities between modern portfolio theory and traditional economics? W. B. Arthur: I spent time with Blake LeBaron, who’s in the room, and John Holland and Richard Palmer, looking at what’s called asset pricing under conditions where people aren’t necessarily rational. They’re exploring and trying to find out how the market works as they invest. Sometimes standard economic theory works very well. Sometimes you can think in terms of rational players, think of probabilities of what earnings might be, and you could think of lowering risks by diversifying among stocks that are not that correlated. That’s portfolio theory and it’s fine. When our team in Santa Fe looked at asset pricing we used a computer model with “artificial” investors; these weren’t “rational,” and didn’t know anything at the start (say, as in Alpha_Zero) but could learn from scratch what behavior was appropriate in what market situation. We found that realistic market phenomena emerged from our model: technical trading, bubbles and crashes, autocorrelated prices, random periods of high and low volatility. These were emergent phenomena and they occur in real markets. I was talking about this Santa Fe artificial stock market one time in Singapore and a hand went up in the audience. “You’ve said in this talk that ninety-eight percent of pricing can be established by standard economic models and about two percent of pricing can be explained by these events triggering other events, bubbles, crashes, and so on. So we’re only off by two percent. What do you think of that?” I said, “Well, that two percent is where the money gets made.” [Laughter] You could equally argue that from outer space the oceans and the world are perfectly at equilibrium spherically, within maybe fifty or a hundred feet. But the nonequilibrium part is where the ships are. And that’s what counts. It’s the startups that make it. It’s people who are lucky, the ones who understand how the nonequilibrium patters work. You can’t model that with equilibrium. Audience Member Could one argue that complex systems like emergent phenomena are a form of equilibrium? Maybe they don’t last for a long time, maybe they change slowly, but there is still some kind of equilibrium.

142

W. B. Arthur

W. B. Arthur: My answer to that is no. Not really. You can do that I suppose, but it’s a case of paradigm stretching. You can think, as I said, of cars with an equilibrium flow of cars going down a freeway. And you could say that, in a situation like that, if the cars get quite dense, then an emergent phenomenon occurs, there might be a traffic jam. A car slows up for some reason. Maybe some animal runs on the road, cars behind it slow up, and quite soon there’s a traffic jam, an emergent phenomenon. Now, and this is subtle, you could define a stochastic process that was in equilibrium, a stationary stochastic process of car flow, and you could set that stochastic process up so that every so often it fetches up occurrences like traffic jams. You’ve preserved “equilibrium.” But I’ll give you my reaction to that—Yeecchh! [Laughter] Why not just relax and say, “Equilibrium’s wonderful. I’m not against it. It’s given us a huge amount of economics. It’s given us wonderful insights, but there are situations that are not in equilibrium,” rather than try and say it’s all equilibrium. Nancy Hayden I’m from Sandia National Laboratory. How do you build a common knowledge with the new kinds of analytic frameworks that we have for nonequilibrium so that people easily can understand what they’re seeing when they’re looking at analyzing these kinds of verb type of measures and ways of framing the problem rather than the nouns? We haven’t figured out how to do that easily for just kind of the general common knowledge other than specialists. W. B. Arthur Earlier today Eric Beinhocker gave a brilliant talk. He said we need to look at things—in the economy, maybe in politics, or situations in general around the world—using different vocabulary, different types of instruments. We need to admit different types of phenomena besides just equilibrium ones. We need to allow that the world is messy, always unfolding, always giving us new things. I think the entire Zeitgeist is changing, and we’re in the last stages of Enlightenment thinking. In the 1730s, people thought everything was mechanism and everything was right and everything was properly ordered, and “All nature is but Art, unknown to thee; all Chance, Direction, which thou canst not see; […] all Partial Evil, universal Good.” That’s from Alexander Pope’s 1733 Essay on Man, basically saying everything is ordered, everything is understandable if only we could look into it and understand its mechanisms or “art.” And that’s changed. I don’t know whether it’s world wars, or quantum physics, or biology. We’re looking now much more at the world as swirling and changing and contingent and not fully knowable and organic, with one thing building on top of another. Santa Fe Institute would be in a very good position to say, if there is a new way of looking at the world, what exactly is it? Where is it coming from? What evidence do we have? Why do we think this way now, when we didn’t do this forty or fifty years ago? Our outlook has changed and I believe we will be talking a slightly different language. If you think everything should be perfect, and ordered, life gets brittle. “Oh, my life isn’t perfect. Things are out of order. I haven’t done this properly. I haven’t done that.” We get nervous. If we say, “It’s pretty good but could be better,” then it’s one damn thing after another, which is a definition of life, and we’re okay. We will adapt. Then people relax, the polity relaxes, social ethics relax.

7 Complexity Economics: Why Does Economics Need This Different …

143

I think we’re missing a way of looking at the world where we accept life. I was influenced by Robert Venturi’s 1965 book Complexity in Architecture. He meant complication. But he made a huge distinction between what he called the “prim dreams of pure order “of the Bauhaus, all these geometric perfect things, versus the ambiguous, the contingent, and then a whole list of things. “What I’m for is messy vitality,” he said. I think if there’s a message in complexity and in its Zeitgeist, it is let’s cool it on the order. Let’s just relax into a world of some degree of messy vitality. For my money, complexity shows us what that world looks like. It’s not optimal, but it’s pretty damn good. And it’s alive.

Chapter 8

Policy and Program Design and Evaluation in Complex Situations Naresh Singh

Abstract As the complexity in the world increases due to increased interconnections, new communication technologies, and the widespread internet of things so too is the complexity of the public policy space changing. However, much of public policy analysis, formulation, analysis and monitoring and evaluation is being done like it was for the last 80 years or more since the beginning of the policy sciences as the formal study of public policy as a social science. Over the last 2- or 3-decades complexity science ideas have been increasingly applied in the social sciences to deal with social systems and other arenas of interest to public policy which displayed complex adaptive systems characteristics such as ecosystems. This has resulted in some refreshingly new ways of thinking of wicked policy problems and even in some new social science sub-disciplines like complexity economics. Yet a chasm exists between these developments and the use of complexity sciences in the teaching and practice of public policy. This chapter brings together in a selected way a lot that has been learned in the use of complexity thinking in public policy and summarizes the practical tools that can be used for policy and program design and evaluation in complex situations.

Introduction The public policy space in many countries is being contested in ways that do not reflect traditional policy analysis. As this is being written in early 2021, the world seems crazy with political divisiveness and racial division in the United States of America, ‘Brexit’ or the United Kingdom’s withdrawal from the European Union, Hindu fundamentalism in India, China’s growing global hegemony, Brazilian populism, and Turkish, Hungarian and Saudi Arabian dictatorial tendencies. Casteism and other forms of social exclusion continue to be prevalent in many countries including the N. Singh (B) Centre for Complexity Economics, Applied Spirituality and Public Policy, Centre for Legal Empowerment of the Poor, Jindal School of Government and Public Policy, Sonipat, India e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 A. Chakraborti et al. (eds.), Quantum Decision Theory and Complexity Modelling in Economics and Public Policy, New Economic Windows, https://doi.org/10.1007/978-3-031-38833-0_8

145

146

N. Singh

USA and India.1 These are overlaid by devastating global challenges like climate change and the Covid-19 pandemic, revealing and exacerbating long standing social and economic inequalities, whether hidden or ignored. Global and national wicked policy issues include: poverty, hunger, species extinction and slow progress towards the Sustainable Development Goals, especially in Fragile and Conflict Affected Situations (FCAS) in the Middle East, Africa and Asia. We are living in what has been described as a “VUCA” world—one that is volatile, uncertain, complex and ambiguous (Wikipedia 2021). The paradox of the public policy world is that as the amount of data increases exponentially, the levels of uncertainty increase similarly and our capacity to formulate policy solutions seems weak and insufficient. At the same time, the world is increasingly interconnected by global supply chains, cheap and mostly free communication technologies, the internet of things, big data, and pervasive artificial intelligence algorithms, all of which result in emergent challenges and opportunities for public policy to support human flourishing. The list of vexing public policy problems is long and growing. These problems are worsened when there is a breakdown in trust among people and between people and national governments and public institutions as is the case in many of the world’s democracies from India to the USA. For example, in the USA, 75% of people think trust in government is shrinking and 64% feel that trust in each other is declining (PEW 2019). The issues themselves have no clear answers as there is much social contestation about what is to be done while, at the same time, the technical solutions to the problems are far from certain. These issues have been called “wicked problems” and they occur in a zone of complexity as described in the next section. Rittel and Weber (1973) described wicked problems in public policy as having many causes and sources, imprecise ways of addressing them, and as being tamable but ultimately unsolvable. Not only do conventional processes fail to tackle wicked problems, but they may exacerbate situations by generating undesirable consequences. Every wicked problem is essentially unique. An ordinary problem belongs to a class of similar problems that are all solved in the same way. A wicked problem is substantially without precedent; experience does not help to address it. As Rittel and Weber (1973) say in the abstract of their paper of policy problems in general. Policy problems cannot be definitively described. Moreover, in a pluralistic society there is nothing like the undisputable public good; there is no objective definition of equity; policies that respond to social problems cannot be meaningfully correct or false; and it makes no sense to talk about “optimal solutions” to social problems unless severe qualifications are imposed first. Even worse, there are no “solutions” in the sense of definitive and objective answers.

This kind of situation will require non-traditional thinking including additional tools and frameworks to replace or complement our existing public policy toolkits. In his recent book, We the Possibility, Mitchell Weiss (2020) echoes the call for public entrepreneurship (or Possibility Government), which was earlier made by Osborne and Gaebler (1993), in Reinventing Government: How the Entrepreneurial Spirit Is Transforming the Public Sector. That was a call for a revolution that, while 1

See for example Wilkerson (2020).

8 Policy and Program Design and Evaluation in Complex Situations

147

it is yet to happen, is likely to follow the somewhat dismal motto of the dominant New Public Management (NPM): “Markets, Managers, Metrics”, unless the NPM’s shortcomings are addressed. This is discussed in the Sect. 8.7 of this chapter. The challenges facing the world are not only increasingly complex and interconnected in ways effect dynamic change, but some like climate change, are civilizational in scale. If we are to successfully navigate these turbulent times (this “VUCA world”) we need to marshal all of our ingenuity. We humans learn about the world around us though our direct sense experiences and observations, with instruments that aid our senses (telescopes, microscopes, medical imaging, laboratories etc.); with models of world, and through intuition, reflection and contemplation. While individual striving for betterment is important, more effective public policy is critical for the needed speed and required scale to effect global change. The purpose of this chapter is to assess the potential of complex adaptive systems’ (CAS) theory, approaches and tools to help address the challenges of public policy analysis, design, monitoring and evaluation. It is based on a review of selected relevant literature as well as on the author’s own experience, over the last three decades. In order to explain the value added by the CAS approach, a brief review of the existing approaches to public policy analysis, formulation, implementation and monitoring and design is first introduced. As not all situations require a CAS approach, this chapter will first discuss an approach to sense-making in the rather confusing existing policymaking space, which will help to decipher how to select the right methods and make optimal use of available resources. This sense-making approach draws from the Cynefin framework developed by Dave Snowden and colleagues (Brougham 2015); and the Stacey Matrix (Stacey 2002). The introduction to the sense-making section will segue into an introduction to CAS intended primarily to provide an alternative way to conceptualize cause effect relationships and change management in public policy. This is followed by an illustration of complexity in various public policy settings. Next is a brief review of tools for public policy design in the CAS context. The final section discusses policy monitoring and evaluation in complex situations and provides practical guidance.

Sense-Making in Today’s World The way humans have historically interpreted the world, our weltanschauung, can be framed as classical, modernist or post-modernist. It can now be argued that none of these frames is adequate and we are better off thinking of a non-modern era characterized by VUCA conditions. In the context of social sciences and public policy, modernism refers to the period between at least 1900 and 1940 in which industrial development, grand designs, universal truths, value-free hypothesis testing, centralized decision-making, and concepts of progress as a linear phenomenon were all common. Later in the twentieth century, ideas of modernism were heavily criticized and alternatives put forward. These alternatives were post-modernist in nature, characterized by skepticism and irony, and included the criticism of Enlightenment

148

N. Singh

rationality, advocating that knowledge claims and value systems were socially conditioned. Post-modernism is often associated with cultural studies, feminist theories, literary criticisms, deconstruction, and similar bodies of thought. The modernist and the early part of the post-modernist eras can be characterized as normal science, but later the contingent nature of reality as defined by social values at the time was reflected in an approach called “post-normal” science (Funtowicz and Ravetz 1993) in which social concerns were being reflected. We will now examine some pragmatic frameworks for sense-making to aid the policy- and decision-making processes, including the Cynefin framework and the Stacey Matrix.

The Cynefin Framework The Cynefin framework shown below in Fig. 8.1 was derived from action research into the use of narrative and complexity theory in organizational knowledge exchange, decision-making, strategy, and policy-making. In the ‘simple’ domain, problems and solutions are known. There is a one-to-one relationship between cause and effect. The connection between system components is strong and centralized. In the ‘complicated’ domain, problems and solutions are knowable. Here, knowable simply means that deeper investigations are needed, as the problems and solutions are not obvious. There is a one-to-many relationship between cause and effect. The connection between system components is strong and distributed. In the ‘complex’ domain, problems or solutions are unknown. There is a many-to-many relationship between causes and effects. The relationship between causes and effects can only be identified with hindsight, referred to as “retrospective coherence”. The connection between system components is weak and distributed. ‘Chaos’ is the realm of the unknowable and feedback loops. In essence, nothing makes sense (Lamotte 2013). The Cynefin framework is a sense-making framework, not a categorization framework. Systems or situations can overlap and move between different domains. In fact, visualizing these movements within the Cynefin framework helps to make sense of changes within systems. Furthermore, there is no domain deemed intrinsically to be Fig. 8.1 The Cynefin framework

8 Policy and Program Design and Evaluation in Complex Situations

149

a better place than the others. For instance, being in the simple domain is not necessarily, rather counter-intuitively, the best place to be for an organization because the risk of falling into ‘chaos’ due to complacency is very high. Most policy situations fall in the complicated or complex domains, or both, which is perfectly fine as long as the tools used are adapted to the domain. The most common pitfall in these domains is entrained thinking: re-applying what has worked before, without ensuring that the ontology remains the same. In more recent renditions of the Cynefin framework, the zone in the centre previously labelled ‘disorder’ in now labelled ‘confusion’: not knowing which domain you are in. It is divided into A = Aporia, in which state you are aware that you are confused or not know which zone you are in; and C, in which state you do not know that you are confused. This zone is best avoided but can be approached with the deliberate creation of paradox and puzzlement to get people thinking differently (Snowden et al. 2020). Take for example the insurrection at the United States Capitol on January 6th, 2021, which occurred as Congress was in session to certify the results of the recent U.S. Presidential election. Prior to the attack, the congressional session was in what would be the simple zone of the Cynefin framework, following well-established rules, using well-structured actions, and cause and effect were direct. The sudden attack launched them into a chaotic zone, resulting in complete confusion for a while, requiring police action—sensing and response. As reinforcement of police and national guards were mobilized, a high degree of uncertainty prevailed, but after the initial shock, Members of Congress and security forces could probe, sense and respond leading to the resumption of the congressional session. The security situation was still complicated however, requiring combined security and the Federal Bureau of Investigation’s forces to sense, analyze and respond, which included laying charges. Further details of the dynamics in each of the four quadrants are summarized in Fig. 8.2 which extends this analytical description to the ‘complex’, as unknown but knowable unknowns while chaos is seen as unknowable unknowns.

The Stacey Matrix The preferred way for policy making—as shown by approaches like New Public Management and evidence-based policymaking—is to rely on complexity reduction approach which works in the zone closest to the bottom left in Figs. 8.3 and 8.4. The Stacey Matrix shown below is a guide for navigating complexity concepts and is a method used to select the appropriate management actions in a complex adaptive system based on the degree of certainty and level of agreement on the issue in question (Stacey 2002). As indicated, close to certainty and close to agreement regarding policy development is the commonest assumption for policy design and its implementation. However, such assumptions are not fulfilled for complex policy issues. These can be or are ‘wicked’ issues. Zones 2 and 3 require some changes in work

150

N. Singh

Fig. 8.2 Further details on the Cynefin framework

process not least in terms of understanding the environment, but do not fundamentally change the policy ‘landscape’. However, this changes significantly in Zones 4 and 5. In the chaotic zone the major achievement may be to understand and translate the environment into actionable patterns which is a kind of ‘order’ that moves issues from chaos to the complexity zone, where action still has to be characterized by presencing, co-creation, applying design thinking and similar approaches. Figure 8.5 (Cooke et al. 2015) shows possible policy actions along a spectrum of situations from simple through complicated and complex to chaos. This is clearly just an approximate guide. For example, in chaos, patterns are not easily discernible and stabilization is first required. In the simple zone actions change from direct to modify structure as complexity increases. In the complexity zone convening and pattern recognition are suggested.

Integral (Meta) Theory (Ken Wilber) The frameworks discussed so far deal with the observed world that we seek to change through public policy interventions. They omit the conditions of the observer or the policy maker, the lenses through which they view the world, their world views, their mental models, and their biases and prejudices. These approaches are based

8 Policy and Program Design and Evaluation in Complex Situations

151

Fig. 8.3 Stacey matrix; wicked problems in the zone of complexity

Fig. 8.4 Stacey matrix with strategies in each zone

on a premise that there is a real objective situation independent of the observer that can be acted upon without reference to the observer. Increasingly however, whether from quantum physics, consciousness studies or contemplative spiritual practices, we are learning that this premise is often false. In any case, in the social sciences and in research (though not in the practice of public policy), participatory approaches have become an important aspect of good practice. Participatory approaches use more systemic and inclusive approaches and engage us as individuals but also as the

152

N. Singh

Fig. 8.5 Policy design using the stacey matrix (Stacey 2002)2

communities or collectives with whom we work. The perspectives of each of us (I) as individuals and also as collectives (We) need to be recognized. Ken Wilber developed a very useful approach to being truly inclusive and holistic in which perspectives of observers and observed and the important notion that we are not separate from the systems in which we intervene, whether human or natural. This is briefly described below from the perspective of public policy. Complex adaptive systems (discussed in some details in Sect. 8.3 below) provide a holistic systems approach to dealing with policy issues in the zone of complexity. However, they deal only with the collective exterior where public policy issues commonly reside. Issues in the individual external quadrant are now increasingly being addressed in behavioral economics and behavioral science and public policy (see for example, Sunstein 2020).3 Also, occurrences in collective internal are being addressed in anthropology research related to policy. While much relevant work is ongoing in the individual internal quadrant, such as consciousness studies, world views, mental models and mindfulness practices are not yet being linked to public policy work. This is an area that needs attention. In this chapter, however, our focus is on the collective external while bearing in mind the need to address issues in the other quadrants shown in Fig. 8.6.

2 Cooke (2015). Implementing the Regional Innovation Strategy for Skåne. Researchgate. https://www.researchgate.net/publication/277310046_Implementing_the_Regional_Innova tion_Strategy_for_Skane. 3 Sunstein (2020). Behavioral Science and Public Policy. Cambridge University Press.

8 Policy and Program Design and Evaluation in Complex Situations

153

Fig. 8.6 The 4 quadrants of integral theory (Ken Wilber)

Complex Adaptive Systems (CAS) and How They Add Value to Public Policy Public policy is likely to seek change in systems which could be described as physical, social or ecological. Physical systems cover infrastructure of all kinds including communications, transportation, construction, energy and those areas typically covered by the study of physics. These systems as such would typically be dealt with approaches in the simple or complicated domains. Social systems include all people-based systems like those dealing with education, health, economics, finance, culture, and arts, and their interactions with physical systems or ecological systems. Policy situations in these domains could be in any of the 4 Cynefin domains and indeed shift from one to the other quite often. Policy interventions in the complicated and simple domains are more readily elucidated and much more attention is required when they are in the complex zone. Ecological systems whether they be forests, fisheries, wildlife, mangrove swamps, parks and protected areas are typically complex adaptive systems. In reality, of course, public policy is important when people are involved and engaged with all these systems at the same time. Issues arising might be in any of the Cynefin domains, but our focus in this chapter will be on those in the complex zone. The reasons are that, as explained earlier, this is the zone in which public policy problems are increasingly encountered, and at the same time strategies to address them continue to be poorly understood. Examples of situations or structures of importance to public policy which can benefit by applying CAS thinking are all around us. These include cities, villages or communities; pandemics, health and education systems, or public sector organizational behavior more generally; and complex conflicts like in Syria or Yemen. A major area of relevance is that of sustainability in which we seek some balance among social,

154

N. Singh

economic and ecological systems—each of which is themselves a CAS. Sustainable Development and the transformations required to achieve the United Nations Sustainable Development Goals (SDGs) will require public policies informed by socio-ecological systems thinking and adaptive co-management. A complex adaptive system is comprised of many interdependent but autonomous agents interacting with each other and dynamically adapting, co-evolving and selforganising over time and space (Singh 2021). As such they are characterised by attributes such as non-linearity, adaptability, attractors, emergence, adjacent possible, self-organisation, inherent or ontic uncertainty, feedback loops and non-Gaussian distribution. These characteristics are briefly explained here. Non-Linearity: Changes in one property of the system is not proportional to changes elsewhere in the same or a coupled system. So, a small change could result in huge impact, like the flapping of a butterfly wings in one part of the world which could cause a hurricane somewhere else. Or a large effort might have little or no effect like heavy bombing of a distributed terrorist network might affect one or two cells. Adaptability: Complex systems are formed by independent constituents that interact, changing their behaviors in reaction to those of others, thus adapting to a changing environment. The interactions need not be direct or physical; they can involve sharing of information, or even be indirect; for example, as one agent changes an environment, another responds to the new environmental condition. Attractors: States in which CAS consistently settle after being present in other more turbulent states and sometimes persist. Examples might include the norms and customs of a society or the harmonic swing of the pendulum. Tipping points or phase transition points: Points or situations at which a system makes an abrupt transition. Boiling water, a physical system, is an example of sudden shift at 100C from liquid to vapor. An example of a tipping point in social system could be crowd of protesters suddenly erupting into violence. In the case of an ecosystem, the Nova Scotian fisheries come to mind, which suddenly collapsed and never recovered due to overfishing as explained later in this chapter. Emergence: Novel patterns that arise at a system level that are not predicted by the fundamental properties of the system’s constituents or the system itself are called emergent properties. What emerges is beyond, outside of, and oblivious to any notion of shared intentionality. Each agent or elements of pursues its own path but as paths intersect and the elements interact, patterns or interaction emerge and the whole of the interactions becomes greater than the separate parts. Examples include community resilience in a community adopting certain healthy and other good practices and the reverse vulnerability or building civil society organizations capacity resulting in democratic transitions. Adjacent Possibles: Each emergent element—material change, product, process, idea, or organization—opens up new possibilities, called “adjacent possibles”. For example, widespread distribution of personal computers opened up the adjacent possible of the internet and the internet social media and so on. The actualization of any particular adjacent possible is not entailed by the new conditions even though it is made possible by them. The actualization may itself be an instance of emergent creativity. Self-organization: A system that is formed and operates through many mutually adapting constituents is called self-organizing because no entity designs it or directly controls it. Selforganizing systems will adapt autonomously to changing conditions, including changes

8 Policy and Program Design and Evaluation in Complex Situations

155

imposed by policymakers. For example, a market operates through all the independent decisions of buyers and sellers. Prices evolve through interactions. While markets can be influenced, they cannot be directly controlled. They will make their own—sometimes surprising and undesirable—responses to direct interventions. Ontic or Inherent Uncertainty: This is uncertainty that is an inherent feature of the system and is commonly found in complex adaptive systems due to their nature and structure. This kind of uncertainty cannot be removed or reduced by more information about the system nor by statistical treatment of its individual variables. Feedback loops: This refers to mechanisms in CAS by which a change in a variable results in positive (reinforcing or amplifying) feedback or a negative (dampening or balancing) feedback of that change. In other words, positive feedback loops accelerate change while negative feedback seeks to keep the system in autopoiesis or a balanced state. Non-Gaussian distribution: CAS tend not to follow standard Gaussian distribution. They tend to have “fat tails” which result from the internal interconnections, adaptability and resilience. They tend to follow power laws of the type y=a log x. The fact that extreme catastrophes can occur at higher-than-expected rates is surely of concern to policymakers. Moreover, the presence of fat tailed distribution in the domain of finance is manifested in recurring huge market movements, breakdowns, and crises that occur with a higher probability than conventional economic theory would have us believe (OECD 2009).

It is important to note however, that not all CAS must have all of the above characteristics, but many will commonly be found as they link and overlap with each other.

Why Complex Adaptive Systems Thinking is Important to Public Policy Public policy takes place in the public or political milieu and deals with the allocation of public resources to solve public problems. Harold Laswell defined politics as “who gets what when and how” (Laswell 1936). Public problems can be considered as those which affect people in communities as distinct from those within a family or corporate or other organizational structure, except to the extent that people feel that such problems need public action. Public policy then, is the business end of political science. It is where theory meets practice in the pursuit of the public good. The sources of public policy, at least in democracies, are widely varying and can include political party manifestos, crises and disasters, media, social movements including through social media, and vested interest lobbying groups, among others. When a government decides to take action on an issue of concern, the process tends to follow phases such as data and information collection and analysis, consultations with stakeholders, formulation or design of policy options, political level decisionmaking which selects the option to be implemented, implementation, monitoring and evaluation and sometimes feedback and course correction. Such an approach seems logical and scientific and indeed it is as these characteristics provide advantages of clarity, rigor, and ease of communication to the public as to what is being done, and how, with what resources, and with what expected

156

N. Singh

results. The approach is linear in the sense that it assumes inputs such as human and financial resources coupled with activities that will lead to tangible outputs. These outputs, when used by the would-be beneficiaries would lead to the intended policy outcomes. It is also highly deterministic, implying that the outcomes can be determined upfront with high degrees of certainty. This view of the world as being linear and deterministic provides a sense of comfort to policymakers as they feel they are in control of the process and can demand accountability from those entrusted with implementation. Beneficiaries also get a sense of comfort as they feel they can trust their government which is in control of solving their public policy issues whether these be in services such health, education, security, financial regulation, etc. or in economically productive sectors of energy, industry, agriculture, transport, tourism, forestry, mining, etc. This approach has served us well from the time of Newton and Descartes and continues to do so but with increasing recognition of its limitations. These limitations were always present but could be ignored because most people were comfortable with the established world views and the results were spectacular. However, upon entering the Anthropocene and with rapid growth in the complexity of social, economic systems as discussed earlier, limitations can no longer be ignored. Much of the world’s public aspirations are now captured in the 17 Sustainable Development Goals (SDGs) which cover a range of issues referred to earlier from poverty and hunger, to health and education, through gender equality, economic inequality and social exclusion to climate change, oceans governance, and peace. There are widespread calls for systems transformation without which the goals will not be achieved. And while systems thinking has always been present in policy processes the view of the world as linear and deterministic was also present. Increasingly we are coming to the recognition that social, economic and ecological systems exhibit characteristics which are far from linear and deterministic and more closely resemble complex adaptive systems (CAS) in many ways. Fortunately, over the last three or four decades we have learned a lot about the inherent complex and adaptive behaviours of these systems from complexity theory and CAS research.

Value Added of CAS to Public Policy The most important value added of complex adaptive systems thinking to public policy is the shift in our mental models of the world, or as the Germans would say, our weltanschauung. This shift is perhaps more important than the computational models associated with CAS work, because important as the quantitative models are to public policy analysis, their results will be misapplied by analysts and policymakers alike, if they continue to use traditional linear deterministic world views. This shift can be difficult because of our conditioning from childhood to think in terms of linear cause and effects, the success of this characteristic in our evolution, and finally the influence of our education and the pervasive use and success of Newtonian/Cartesian linear deterministic thinking. To be clear, this is not a call to abandon this way of thinking

8 Policy and Program Design and Evaluation in Complex Situations

157

that has been so successful, but rather is one to complement with a novel approach, the CAS approach. Why? Because so many of our current and emerging problems can no longer be adequately addressed by the traditional methods alone. This has now been recognized by the Organization for Economic Cooperation and Development (2009, 2017, 2020) and addressed in several books including Drift into Failure (Dekker 2011), Embracing Complexity: Strategic Perspectives for an Age of Turbulence (Boulton et al. 2015), Complexity and the Art of Public Policy: Solving Society’s Problems from the Bottom Up (Colander and Kupers 2015), Complexity and Public Policy Handbook (Geyer and Cairns 2015), Complexity and Public Policy: A new approach to twenty-first century politics, policy and society (Geyer and Rihani 2010) and Embracing Complexity in Health Policy (Sturmberg 2019). Journals dedicated to the field include the Journal on Policy and Complex Systems. The importance of this shift can be illustrated by reference to the collapse of the Nova Scotian fisheries in Canada. A dynamic, multi-species, multi-fleet model (Allen and McGlade 1987) predicted the collapse of the fisheries. These models took a complex adaptive systems approach which was quite new and different from the traditional kind which tended to seek equilibrium and investigate single species. These latter models were consistent with past best practice and with linear deterministic thinking with which decision makers were more comfortable and so the former models were ignored. About 5 years later the Nova Scotian cod fisheries collapsed. An important lesson to be learned, beyond the shortcomings of the modeling, is that the results of the CAS model threatened the short-term interests of some of the players, and was unsettling to those with traditional mindsets. So, it is important at the outset that stakeholders understand that models are experiments which can tremendously aid our understanding, that the strengths and weaknesses are shared, and that the stage is set for the results to be used and in the right way. One of the uses of models is to provide insights to policymakers, but there are always higher-level questions related to values which determine what to include and exclude in your models. Currently, most modeling is done by academics and researchers and there appears to be a chasm between the outputs of these models and their use for policy. Some notable exceptions are the current pandemic and climate change where much policy action is being determined both by modeling and political considerations. The understanding that models are experiments, that they are intended for learning and that their limits need to be clearer will help in their judicious use. The most common model used in CAS is Agent Based Models (ABM) in which local agents are assigned local behavioral rules and links to others. As we know, CAS is characterized by ontic uncertainty and so for the model to be helpful, it must be the system, which can require significant human and other resources. Fortunately, much good policy work can de designed without sophisticated computational models, though using them when they are available can be quite helpful. The linear deterministic mindset leads to hierarchical organizational structures, command and control structures and the need for leaders and managers to be in control. Indeed, at the level of the individual we are all conditioned to want to be

158

N. Singh

in control even when we are dealing with a dynamic complex situation. Learning to know when you cannot be in control and giving up control in these circumstances while adopting other approaches more consistent with CAS is very valuable. These other approaches include adaptive and iterative planning, as well as being alert to adjacent possible opportunities and unintended effects, and being aware that sudden tipping points can be reached that may evoke massive change, including collapse. These and other related techniques will be discussed later in this chapter. Examples of the use of CAS thinking for public policy can be illustrated in a range of social sciences as in economics, politics and law, and policy domains such as sustainable development, international development cooperation, health, education, international trade, and urban planning.

Applications of CAS Public Policy4 In this section the use of complexity thinking in several common pubic policy areas such as economics, power and politics, law, health, education and sustainability science are addressed.

Economics In economics the old debate between laissez-faire and government activism is well known. What that debate typically missed was that policy interventions based on either approach or even a mix of the two would change the system, as well as people’s choices and preferences, so that a new system would emerge with new opportunities and unforeseen circumstances. From the CAS point of view the economic system is an evolving system beyond the control of government or anyone. On the other hand, it is not a self-steering system requiring no government action. As a complex adaptive system, it is endogenously creating control mechanisms which make it work. Government is one of the control mechanisms. The role of government is norm influencer, encouraging people to adopt positive social norms like self-reliance and care for others. Complexity economics provides mathematical models that can capture the interconnections between the view points of liberals and market advocates as well as the characteristics of the economic system which behaves more like a living system than a mechanical one. In the complexity frame there is no compass for policy except the highly educated mind. Such education should include at least a basic understanding of complexity, 4

Many of the applications in this section are based on relevant sections in Complexity and Public Policy Handbook (Geyer and Cairney 2015).

8 Policy and Program Design and Evaluation in Complex Situations

159

mathematics and humanities. Scientific models provide half-truths at best (Colander and Kupers 2015). Brian Arthur, one of the main exponents of complexity economics, has well described the economy as a complex adaptive evolving system driven by both endogenous and exogenous factors. When these result in self-reinforcing feedback loops (positive feedback) the economy sees increasing returns from certain technological innovations or investments (Arthur 2014). In terms of policy, Hirschman (1958, 66) illustrates this approach in which he advocates for, instead of balanced development across sectors, policies which create tensions, disproportions and disequilibria among sectors. Investments on one front can then, if carefully chosen, leverage developments on other fronts. Schumpeter’s gales of creative destruction is another example of CAS systems thinking at work in the economy.

Power and Politics While power and politics are at the heart of public policy, complexity theory and one of its most important tools, Agent Based Models (ABMs), are more concerned with patterns and data analysis and as such tend to omit these critical factors. It is timely to remind ourselves that evidence is only one input into policy making and is not always as or more important than power, politics and the resulting contestation among vested interest groups which decide the policy outcome. Complex systems allow bottomup emergent activity by people and groups at the microlevel to unwittingly result in macro patterns that are greater or lesser than the sum of their parts. Looking at power and politics through the lens of complexity theory allows for opportunities for making power a positive sum game instead of the traditional zero sum that linear dynamics will point to. Positive sum games allow significant social change such as self-empowerment of the poor to take place, which would otherwise be impossible from a hierarchical, top-down control by the powerful. An examination of real politics and of what facts and data realism is based on reveals epistemological and ontological gaps. The revelation that we cannot know with any certainty what constitutes realist politics on what people will base their decisions, opens up space in complex adaptive systems that can generate useful insights, the “real” being a highly contingent and transient phenomenon.

Law Law can provide a basis for policymaking by prescribing certain broad limits. However, policy does not necessarily require law. All laws are policies but not all policies are laws. The inter-relationships between law and policy are therefore dynamic and variable but they are both categories of rules which seek to guide the behavior of human society. This is a complex adaptive system, as CAS societies demonstrate emergence in which order can be generated without a centralized controlling agent

160

N. Singh

and in which the system outputs are greater than the sum of its parts. Human societies demonstrate high levels of self-organization although the attractor state in which they sometimes settle is not desirable from a standpoint of social and economic justice, ethics, or inclusion. Reductionist approaches are the converse of emergence. Take for example, the notion that more specific rule making leads to better control, and that the character and capabilities of rule-making and rule-administering bodies can be reduced to the sum of their parts (Geyer and Cairney 2015). Reductionism in environmental law in the United States, for example, seeks certainty through the proliferation of granular rule structures and seeks to avoid risk through more regulation. From a CAS perspective the certainty they seek is likely to be elusive and on the other hand, risk to be embraced and a risk-based management approach would make more sense. The hyper-regulatory climate depends on a top-heavy administrative structure for implementation. The result is that the system cannot manage and deal with novelty and shock (Geyer and Cairney 2015). The answer is clearly not deregulation, which leaves the environment unprotected, but an adaptive risk-based approach consistent with CAS principles.

Health Health policy can be seen in terms of upstream and downstream components. The upstream component or macro level dealing with social determinants of health, and the downstream or micro level with delivery of health care services. Complexity theory can add valuable insights at both levels but more significantly it seems at the micro level. Plsek and Greenhalgh (2001) acknowledged the complex nature of health care in the twenty-first century, and emphasized the limitations of reductionist thinking and the “clockwork universe” metaphor for solving clinical and organizational problems. To cope with escalating complexity in health care, they concluded that “we must abandon linear models, accept unpredictability, respect (and utilise) autonomy and creativity, and respond flexibly to emerging patterns and opportunities” (Plsek and Greenhalgh 2001). In a special issue of Social Science and Medicine, Tenbensel (2013) documents several case studies of the application of complex adaptive systems thinking to health and health care. A few are used here as illustration. They include two instances of ‘scale-up and spread’ of improvement initiatives—the use of mobile phone messaging to improve adherence to anti-retroviral treatment for HIV in Kenya, and measures to reduce the incidence of MRSA infection in hospitals in the United States. The question was how to achieve scale-up and spread efforts across a range of organizations and settings with a wide variety of local contextual features. The researchers found that “understanding self-organization is critical to understanding variation across local contexts”, and understanding the role of interdependencies within and between organizations as well as the sense-making of participants, are crucial features of what they term “productive self-organization” (Lanham et al. 2013). There is an investigation into London TB services to illustrate the ways in

8 Policy and Program Design and Evaluation in Complex Situations

161

which the macro context of health services based on New Public Management principles and procedures serves to enable and/or in most cases inhibit the opportunity for productive self-organisation to occur. In this way, there is an attempt to explain “the London TB control system’s (in)ability to respond” to the epidemic (Trenholm and Ferlie 2013). The complexity of local, ‘micro’ responses to disasters is employed in order to develop a prescriptive model “to identify potential points of intervention to promote population health and resilience” (O’Sullivan et al. 2013). Using a community-based participatory research design, some key principles are extracted to inform local responses to disaster that eschew a ‘one size fits all’ approach, and advocates the development of flexible, adaptive intervention designs which ‘must emerge from the complexity of the situation and be tailored to the community context at any point in time. Others apply a systems dynamic modelling (SDM) framework to model the impact of a range of interventions that aim to address social determinants of health status in Toronto. SDM shares a conceptual ancestry with complexity theory. This contribution is notable for its incorporation of positive feedback loops into predictions regarding the efficacy of policy interventions and combinations of these interventions. A recent comprehensive review of system dynamics applied in health and medicine provide an excellent illustration of the use of this complex adaptive systems tool in health policy and interventions (Darabia and Hosseinichimeh 2020). The SDM approach has been increasingly recognized as a powerful method for understanding and addressing complex health issues. Over the past four decades, SDM has been applied to a wide range of health care problems, including the prevalence of major infectious or non-infectious diseases and the performance of health care delivery systems. The approach has also provided decision support for national and global policy-makers (Homer et al. 2016; Thompson et al. 2015). Consequently, several influential domestic and international organizations have encouraged and supported SDM applications for understanding the causes of illnesses and associated trends as well as for the design of prevention, treatments, and policy interventions (De Savigny and Adam 2009; Mabry et al. 2008). “When you change the way you look at things, things you look at change”.5 It is high time for healthcare professionals to embrace the challenge of complexity. The linear reductionist view of health and disease is failing to deliver the best health care for our societies. The four main components contributing to our health and disease experience are our somatic (or bodily) condition, our social connectedness, our emotional feelings and our semiotic (or sense-making) abilities—these four domains define the somato-psycho-socio-semiotic model of health and disease as discussed in Sturmberg’s recent book Embracing Complexity in Health (2019). From a public policy perspective, the health portfolio is typically one of many disconnected policy segments as the health portfolio itself is segmented into many discrete disease/condition-specific silos. Each has its own budget line, promotes its

5

Variously attributed to Max Planck and Wayne Dyer.

162

N. Singh

own agendas and priorities and competes against others in the continuous competition to gain/maintain funding. The policies for the delivery level are twofold: monetary—applying incentive payments around clinical targets or service delivery modes, and directive—demanding the uniform micromanagement of discrete diseases/ conditions at the community and/or service delivery level. From a complex adaptive systems perspective, the health system will be organized around its attractor state—i.e., the needs arising from the patient’s illness or disease experience. Such a system constantly learns how to best work together to achieve the best possible outcome for the individual patient, the local community and the nation as a whole. From an organizational perspective, it presupposes a collaborative culture between members at every functional unit within the system. Such complex adaptive dynamics at each system level result in the spread of skills and knowledge among and across members of functional units. The diversity in perspectives, interpretations and experiences are respected, which ultimately achieves better outcomes. Greater resilience emerges from the mastery of challenges and reinforces shared learning, adaptation and self-organization (Sturmberg et al. 2012).

Education Complex systems application to health policy seems much more established than that to education research and policy. However there have been some significant contributions in the education field as well which are briefly reviewed in this section. According to Jacobsen et al. (2016) the contexts in which learning occurs are complex systems with elements or agents at different levels including neuronal, cognitive, intrapersonal, interpersonal, and cultural. There are feedback interactions within and across levels of the systems so that collective properties arise (i.e., emerge) from the behaviors of the parts, often with properties that are not individually exhibited by those parts. They analyzed the long-running “cognitive versus situative learning” debate and proposed that a complex systems conceptual framework of learning (CSCFL) provides a principled way to achieve a theoretical convergence. The use of Agent Based Models (ABM) has very well demonstrated the value of complexity approaches in the education sector, for example in a study to provide parents with school choice in the United States (Maroulis et al. 2014) as described in Jacobsen et al. (2016). It worth reviewing this model study in some detail for illustration purposes. The proponents of school choice reform argued that competition introduced by allowing parents to select the schools their children attended will lead to better schooling and incentives for school reform. In contrast, opponents of this type of reform claimed that resources were drained away from schools and that school quality was thus hurt, not helped, by such a reform. Research into this issue, since the 1990s, employed standard quantitative and qualitative methods, but provided inconclusive and even conflicting findings. This policy debate was investigated by creating ABMs of a school district’s transition from a local neighborhood school catchment area system to a school choice

8 Policy and Program Design and Evaluation in Complex Situations

163

system. The agents in the system were schools and students. School agents varied in terms of the quality and building capacity of existing schools, and new schools entered the system by imitating top existing schools. Student agents varied in their ability and background, and the researchers ranked schools according to achievement and geographic proximity. The academic achievement of the student agents combined individual traits and the value-added by the quality of the school they attended. Real data from Chicago Public Schools was used to initialize the model. The use of these ABMs helped identify dynamics—such as CSCFL conceptual perspectives of micro–macro levels, nonlinearity, and emergent properties—that had not been revealed in previous quantitative and qualitative research. Specifically, the model demonstrated that the timing of new schools entering the system was a critical factor. The overall system improved because new schools entering the system imitate the top existing schools. However, a high emphasis on achievement at the schools led to new schools entering the system earlier, which resulted in lower-achieving new schools. Thus, there was a paradoxical mismatch between macro-level and microlevel behaviors of the system in that increasing the emphasis on school achievement at the household level did not generally lead to increasing achievement at the district level. From a policy perspective, results of using ABMs suggest that the critics of school choice reform were correct that school achievement in the overall system would not rise. However, the reason proposed by the critics—draining of resources away from existing schools—was not actually the causal factor; rather, it was the timing of new schools entering the system. This ABM also provided other school choice policy insights, such as the unintended transfer of top students to private schools where vouchers issued by the government were used to pay for the private schooling, which was an emergent property of the changes in the Chicago Public School system. Another unexpected dynamic of the model was that being a top-rated school (based on the mean achievement levels of its students) was an unstable (i.e., nonlinear) state: the top-rated school attracted many new students, some of whom did not achieve as highly, thus bringing down the school’s achievement rating, so that another school became a top rated one. This unexpected insight from their modeling has policy implications for the domain being modeled. Many choice schools avoided this issue by being selective, but if school choice is really implemented in the socalled free market form that advocates sketched out, then this instability will become a reality. This ABM application thus provides detailed insights into the value added of complexity approaches to education policy analysis for decision making.

Sustainability and Complexity A recent comprehensive review of sustainability science towards a synthesis concluded that “compelling evidence has accumulated that those (society-nature) interactions should be viewed as a globally interconnected, complex adaptive system

164

N. Singh

in which heterogeneity, nonlinearity, and innovation play formative roles. The longterm evolution of that system cannot be predicted but can be understood and partially guided through dynamic interventions” (Clark and Harley 2020). The review shows that recent research in sustainability science has demonstrated how thoroughly the elements of nature and society are intertwined in deeply coevolutionary relationships. An immediate consequence of these findings it deduces, is that “talk of environmental-sustainability, or social-sustainability or other forms of “hyphenatedsustainability,” is fundamentally misleading and at odds with the integrating aspirations of sustainability science. A research-informed use of the term sustainable should therefore always—and only—refer to the integrated pathways of development resulting from nature–society interactions in the Anthropocene System” (Clark and Harley 2020). The term socio-economic subsystem is used to describe the social and economic actors and processes and the complex interactions among them. The social aspects include the political, cultural, emotional and spiritual dimensions and the related institutions and rules by which human society is organized and functions. The economic aspects include those actors and processes primarily involved in the production and distribution of goods and services to satisfy some need or demand. In many instances, the actors and processes in the social and economic spheres are the same, there is an intimate relationship between them and hence the term socio-economic (Singh 1996). The ecological system refers to the earth’s natural systems either as single ecosystems such as a coral reef, a mangrove swamp, a stand of Douglas fir, one of the Great Lakes, or planetary systems such as the ocean–atmosphere coupling. An ecological system is comprised of various interactive groups of species, genera, families and communities of organisms. In certain regions ecological features are present which define the region as a bioregion. We use the term “ecozone” to describe these planetary subregions which include coastal zones, arid and semi-arid lands (including the prairies), mountains, forests, large agricultural plantations, and towns and cities.6 The search for sustainable development or sustainable livelihoods is a search for harmony between the activities and inherent evolutionary processes and tendencies of the linked socio-economic and ecological systems. This linked socio-economic and ecological construct is what we refer to as the socio-ecological system. This could equally be called a socio-natural or society-nature system. In this arrangement, the socio-economic subsystem is embedded in and dependent on the natural or ecological system. Because of this dynamic interactive process, we need to consider the community in this environment as a single system which can be described as a socio-ecological system. An understanding of the attributes of a socioecological system then becomes a fundamental pre-requisite to a region’s perception of community adaptation and how such adaptation can provide a basis which can result in sustainable livelihoods as a desirable outcome. Socio-ecological systems display all or most of the characteristics of CAS discussed earlier. In this particular context these features arise as diversity, categories, 6

Ibid.

8 Policy and Program Design and Evaluation in Complex Situations

165

measures of association, non-linearity, resilience, co-evolution, learning capacity and community participation. These features as well as guiding principles to be used in public policy and programing for sustainable livelihoods are described by Singh (1996). The value of understanding these basic issues and principles include: . Help understand the reality of the community with its vast array of beliefs, knowledge, strategies and practices situated in a dynamic and interlocking social and ecological system from which livelihoods are derived. . Establish an epistemological basis for policy making in the face of uncertainty, constant change and complexity. . Develop an approach to identifying entry points and interventions, which when made at these leverage points lead to massive amplification and self-organization within the system to significantly increase the sustainable livelihood options. . Help transform intuitions and anecdotes into a deeper understanding of the complexity of socio-ecological systems.

Program Design in Complex Systems In Sect. 8.2 on sense-making it was shown that in general we face situations which could be simple, complicated, complex or chaotic, and not knowing which of these is occurring can result in some confusion (See Fig. 8.1, Sect. 2.1.). Let us consider how these ideas might be applied in a practical situation like promoting more sustainable livelihoods in a poor community. The approach will be quite different from design using a linear logical framework in which you first carry out a needs assessment and design a project in which inputs and activities lead to outputs, which when used by the beneficiaries, lead to outcomes that over time, translate into impacts. Instead, the design will codeveloped through a facilitated process involving the community. The steps would include the following: . Multidimensional assessment of the assets/activities and coping/adaptive strategies of would-be beneficiaries of development cooperation. Assets include human, social, physical, natural, economic and political capital. . Establishment of the vision of a life worth living, or a more sustainable livelihood. . Define what the communities can/will do on their own to get to their vision. . Then finally define what help they need from outsiders such as development agencies. This simple sequence of steps can, in the hands of a skilled facilitator help build on the dominant log frame to incorporate several dimensions of CAS. For example, the starting point is not needs but assets. This helps set the stage for building on local strengths and endowments and encourage innovation. Human capital assessment, for example moves us away from the common assumption that the only form of human capital that the poor have is labour, and opens up a wider conversation that

166

N. Singh

includes their creativity. Similarly, each of the forms of capital assessment opens up a wide range of avenues through which alternative pathways to the local vision can be pursued. The indicators to be used in the program design emerge from the communities themselves during the visioning exercise in which they define a better life. Yet the need to a have a framework with action plans and goals which the funders would ask for is not lost.

Dealing with Complexity in Policy Design Almost by definition public policy must include statements of future desirable social goals and so must have a degree (preferably a high degree) of predictability. On the other hand, uncertainty in CAS is high and predictability low. So, a contradiction arises, requiring careful management. It will be unwise to suddenly abandon well accepted traditional tools of policy analysis such as cost benefit analysis. As in the case of evaluation it will be better to gradually bridge the gap between policy making assumptions of a deterministic clockwork universe to an evolutionary adaptive and uncertain world. A few practical strategies of making such a transition are described in this section. This followed by some more recent tools which are currently being used. In terms of strategy, the first step is to start with existing policy analysis tools and make change incrementally. For example, start with Cost Benefit Analysis, the work horse of assessing public policy options. Strengths and weaknesses of alternatives can then be used to determine options which provide the best approach to achieving benefits while preserving savings, and can be used to compare completed or potential courses of actions, or to estimate (or evaluate) the value against the cost of a decision, project, or policy especially public policy. CBA helps predict whether the benefits of a policy outweigh its costs (and by how much), relative to other alternatives. It also allows for the ranking of alternative policies in terms of a cost–benefit ratio. Generally, accurate cost–benefit analysis identifies choices which increase welfare from a utilitarian perspective. Although CBA can offer an informed estimate of the best alternative, a perfect appraisal of all present and future costs and benefits is difficult; perfection, in economic efficiency and social welfare, is not guaranteed. CBA has been applied to a wide range of public policy issues in health, education, and environmental issues. It depends on predicting outcomes over a time and so uses time value of money, discount rates and various risk analysis methods. In order to deal with complex situations in which outcomes cannot be predicted CBA might be strengthened or even replaced by tools derived from complex adaptive systems teaching such as agent-based models (ABMs); causal-loop diagrams; social network analysis; data mining; scenario analysis; horizon scanning; group model building; SWOT analyses; cognitive mapping; qualitative case study methodology; leverage points; sensitivity analysis; and non-linear dynamical systems modelling. Newer techniques including anecdote circles, chaos dynamics, participatory systems mapping; behavioral insights; and human-centred design thinking have recently come

8 Policy and Program Design and Evaluation in Complex Situations

167

to the fore. Among this rather long list, ABMs and other non-linear dynamical computational models, data mining, network analysis, design thinking, and scenario and strategic foresight are commonly used.

Monitoring and Evaluation in Complex Situations The monitoring and evaluation of change in complex adaptive systems or complex situations in general require some fundamentals in what we measure, how we measure these, and what indicators and tools we use. Under the New Public Mangement (NMP) approach, still the dominant style of public service management, performance management is linked to outcomes-based performance management which in turn rests on Results Based Mangement (RBM). RBM relies heavily on the linear logical framework design method (log-frame) in which desired outcomes are precisely defined upfront in terms of objectively verifiable indicators (OVIs). The outcomes (ultimate, intermediate, and immediate) are derived from the use of the outputs by the beneficiaries. The outputs are in turn produced by inputs (human and financial resources) and appropriate activities. While this is a significant improvement over inputs and activities monitoring, it works well in simple and even complicated situations as defined earlier, but not in complex situations. In complex situations, system level outcomes are emergent from a wide range of people-to-people interactions, interconnections and interdependencies, information flows, interactions with the inputs, activities and outputs being introduced by the policy or project intervention. Indicators traditionally used under NPM were termed “SMART” indictors, indicating that their desirable characteristics should be: Specific, Measurable, Attainable, Realistic, and Timely (OECD 2014). For complex adaptive systems it has been considered that so-called SPICED indicators might be better suited, having the following characteristics: . . . .

Subjective: using insights from informants (beneficiaries/stakeholders). Participatory: indicators should be developed with stakeholders. Interpreted: easily interpreted to different stakeholders. Cross-checked: comparing indicators and using different stakeholders, methods to ensure validity. . Empowering: the process should allow stakeholders to promote their own agency. . Diverse: indicators should be different, from a range of groups and across genders. We also need to consider a shift from evaluation for single loop learning (formative or summative) to evaluation for double loop learning. In single loop learning changes are made to improve immediate outcomes as the difference between actual and desired outcome is evaluated. Double loop learning involves questioning the assumptions, policies, practices, values, and system dynamics that led to the problem in the first place and making changes to the underlying system either to prevent the problem or embed the solution in a changed system.

168

N. Singh

The evaluation tools or techniques that are more suited to evaluations include (Better Evaluation Organisation 2021): . Developmental evaluation: uses evaluative questions and logic, to support program, product, staff and/or organizational development. The evaluator is part of a team whose members collaborate to conceptualize, design and test new approaches in a long-term, on-going process of continuous improvement, adaptation and intentional change. . Outcomes mapping7 : which assesses a programme’s theory of change and changes in the behaviour of people, groups and organisations with which a programme works directly. The results map traces outcomes from outputs to immediate, intermediate and ultimate levels both intended and unintended. . Outcomes harvesting8 : which collects (“harvests”) evidence of what has changed (all outcomes) and, then, working backwards, determines whether and how an intervention has contributed to these changes (a kind of reverse mapping to outcomes-based mapping) . Contribution Tracing: A rigorous quali-quantitative method that is used in impact evaluations to test the validity of claims. It allows you to not only test a contribution claim (whether is it valid or not), but to also determine a quantifiable level of confidence in contribution claim. The developers of the method call it ‘putting a number on it’ . Most significant change (MSC): a qualitative method that collects and analyses personal accounts/stories of change among stakeholders and qualitative impact assessment protocol (QUIP)9 an impact evaluation method that uses narrative causal statements that are taken directly from intended programme beneficiaries. This method provides an independent verification (or not!) of a programme’s theory of change.

From New Public Management to Human Learning Systems The NPM has been critiqued because of its reliance on public choice theory which has similar limitations as rational choice theory in public economics as well as a market orientation to meeting people’s needs. Because of this a lack of trust between public servants and the public is not unusual under NMP approaches. A fundamental reexamination to measuring change in complex systems situations from policy or program interventions by government or other funders has been recently introduced by Toby Lowe (Lowe and Plimmer 2019) which appears to address some of these shortcomings. In this work the authors show that outcomes are not produced by individual organizations (including those who might be funding or commissioning the work) but rather through complex systems interactions. Meaningful social change must 7

See Better Evaluation, “Outcomes Mapping”. See Better Evaluation, “Outcomes Harvesting”. 9 See Better Evaluation, “Qualitative Impact Assessment Protocol”. 8

8 Policy and Program Design and Evaluation in Complex Situations

169

therefore consider that people are complex: everyone’s life is different, everyone’s strengths and needs are different. The issues we care about are complex issues from homelessness and obesity, to economic inequality or social injustice, and to global issues like climate change and the Covid-19 pandemic. They are tangled and interdependent. The systems that respond to these issues are complex: the range of people and organisations involved in creating ‘outcomes’ in the world are beyond the management control of any person or organisation. They therefore suggest that those involved in public policy and programs work in a way that is human, prioritises learning and takes a systems approach (a so-called Human, Learning Systems (HLS) approach). This means recognising the variety of human needs and experiences, building empathy between people so that they can form effective relationships, understanding the strengths that each person brings, and deliberately working to create trust between people (Variety, Empathy, Strength and Trust). They suggest that for funders and commissioners, being human means creating trust with and between the organisations they fund. Trust is what enables funders and commissioners to let go of the idea that they must be in control of the support that is provided using their resource. People working effectively in complex environments adopt a continuous process of learning and adaptation. This requires ongoing experimentation which reveals ways of working that are more likely to be effective in particular contexts. This in turn gives valuable insight as to where to begin the next set of experiments. For providers and funders this means an iterative, experimental approach to working with people. They now use data to learn more than mere compliance, thus creating a learning culture. Measurement continues to be very important but now for the purpose of enabling learning, rather than control. The HLS approach recognizes that outcomes are produced by a complex system of actors, inter-relationships, interconnections and interdependencies and not a single organization or leader. As such it is to be expected that healthy systems will be more efficient at producing outcomes. Some characteristics that been tentatively identified as desirable include people having a common vision, as they are seen as resources, and they recognize their interconnections. Others include decentralized decision making, shared power, and feedback and collective learning.

Conclusion In a world of rapid change, to address unprecedented and widely varying challenges to public policy, sense making tools such as the Cynefin Framework and the Stacy Matrix are useful to decide how to proceed. Generally, these situations can be ordered (simple or complicated) or unordered (complex or chaotic). Solutions to ordered situations are quite well known. Chaotic systems require special actions for stabilization before interventions. Complex adaptive systems on the other hand are of great interest because they are more common than is recognized, and are situations of emergent patterns, innovation, and self-organization. Many tools have been

170

N. Singh

developed over the last 3 or 4 decades to work with these complex situations but these are not commonly known to public policy analysts and decision makers. This chapter has surveyed public situations which demonstrate these complex adaptive systems characteristics, as well as the tools which are being applied for design or evaluation of policies or programs with these characteristics. Many of these tools such as non-linear dynamic computational techniques, quantum-like social science models, agent-based simulations, and so forth are discussed in detail in other chapters. However, as demonstrated here, and useful as they are, much good policy work can proceed even without these mathematical models.

References Allen, P. M., McGlade, J. M., Modeling complex human systems. A fisheries example. Eur. J. Oper. Res. 30(2) 147–167 June (1987) Arthur, B.: Complexity and the Economy, p. 240. Oxford University Press, London (2014) Better Evaluation Organization.: Various tools (2021). https://www.betterevaluation.org/ Boulton, J. G., Allen, P. M., Bowman, C.: Embracing complexity. Strategic perspectives for an age of turbulence. Oxford University Press (2015) Brougham, G.: The Cynefin Mini-Book: An Introduction to Complexity and the Cynefin Framework, p. 53. C4 Media, InfoQ: Enterprise Software Development Series (2015) Clark, W.C., Harley, A.G.: Sustainability science: toward a synthesis. Annu. Rev. Environ. Resour. 45, 331–386 (2020). https://doi.org/10.1146/annurev-environ-012420-043621 Colander, D., Kupers, R.: Complexity and the Art of Public Policy: Solving Society’s Problems from the Bottom Up, p. 309. Princeton University Press (2015) Cooke, P., Eriksson, A., Wallin, J.: Implementing the Regional Innovation Strategy for Skåne, pp. 1–29 (2015). https://doi.org/10.13140/RG.2.1.1278.7363 Darabia, N., Hosseinichimeh, N.: System dynamics modeling in health and medicine: a systematic literature review. Syst. Dyn. Rev. 36(1), 29–73 (2020) De Savigny, D., Adam, T.: Systems Thinking for Health Systems Strengthening, p. 112. World Health Organization, France (2009) Funtowicz, S.O., Ravetz, J.R.: Science for the post-normal age. Futures 25(7), 739–755 (1993) Geyer, R., Cairney, P.: Handbook on Complexity and Public Policy, p. 482. Edward Elgar Publishers (2015) Geyer, R., Rihani, S.: Complexity and Public Policy: A New Approach to 21st Century Politics, Policy an and Society. Routledge (2010) Hirschman, A.O.: The Strategy of Economic Development. Yale University Press, New Haven (1958) Homer, J., Milstein, B., Hirsch, G.B., Fisher, E.S.: Combined regional investments could substantially enhance health system performance and be financially affordable. Health Aff. 35(8), 1435–1443 (2016). https://doi.org/10.1377/hlthaff.2015.1043 Jacobson, M., Kapur, M., Reimann, P.: Conceptualizing debates in learning and educational research: toward a complex systems conceptual framework of learning, educational psychologist. Educ. Psychol. 51(2), 210–218 (2016). https://doi.org/10.1080/00461520.2016.1166963 Lamotte, A.: The Cynefin framework. In: Red Badger Blog (2013). https://blog.red-badger.com/ 2013/04/25/the-cynefin-framework Lanham, H.J., Leykum, L.K., Taylor, B.S., McCannon, C.J., Lindberg, C., Lester, R.T.: How complexity science can inform scale-up and spread in health care: understanding the role of self-organization in variation across local contexts. Soc. Sci. Med. 93, 194–202 (2013)

8 Policy and Program Design and Evaluation in Complex Situations

171

Laswell, H.D.: Politics; Who Gets What When How. Whittlesey House, McGraw-Hill Book Co., New York (1936) Lowe, T., Plimmer, D.: Exploring the new world: practical insights for funding, commissioning and managing in complexity. In: Collaborate for Social Change [webpage] (2019) Mabry, P.L., Olster, D.H., Morgan, G.D., Abrams, D.B.: Interdisciplinarity and systems science to improve population health: a view from the NIH office of behavioral and social sciences research. Am. J. Prev. Med. 35(2), S211–S224 (2008) Maroulis, S., Bakshy, E., Gomez, L., Wilensky, U.: Modeling the transition to public school choice. J. Artif. Soc. Soc. Simul. 17(2), 3 (2014). https://doi.org/10.18564/jasss.2402 O’Sullivan, T.L., Kuziemsky, C.E., Toal-Sullivan, D., Corneil, W.: Unraveling the complexities of disaster management: a framework for critical social infrastructure to promote population health and resilience. Soc. Sci. Med. 93, 238–246 (2013) Organization for Economic Cooperation and Development.: Applications of Complexity Science for Public Policy: New Tools for Finding Unanticipated Consequences and Unrealized Opportunities, pp. 1–28 (2009) Organization for Economic Cooperation Development.: Measuring and managing results in development cooperation (2014) Organization for Economic Cooperation Development.: Debate the issues. Complexity and public policy (2017) Organization for Economic Cooperation Development.: From transactional to strategic. Systems approaches to public service challenges (2020) Osborne, D., Gaebler, T.: Reinventing Government: How the Entrepreneurial Spirit is Transforming the Public Service, p. 432. Plume (1993) PEW. Trust and distrust in America. Pew Research Centre (2019) Pslek, P.E., Greenhalgh, T.: Complexity science: the challenge of complexity in health care. BMJ 323(7313), 625–628 (2001). https://doi.org/10.1136/bmj.323.7313.625 Rittel, H.W.J., Webber, M.M.: Dilemmas in a general theory of planning. Policy Sci 4, 155–169 (1973). https://doi.org/10.1007/BF01405730 Singh, N.: Development as emergent creativity. In: Redekop, V., Redekop, G. (eds.) Transforming: Applying Spirituality, Emergent Creativity, and Reconciliation, pp. 395–418. Lexington Books, Lanham (2021) Singh, N.: Community Adaptation and Sustainable Livelihoods: Basic Issues and Principles. [Working Paper.] International Institute for Sustainable Development (1996) Snowden, D., Goh, Z., Greenberg, R.: Cynefin—Weaving Sense-Making into the Fabric of Our World, p. 376. The Cynefin Co., Cognitive Edge (2020) Stacey, R.D.: Strategic Management and Organizational Dynamics: The Challenge of Complexity to Ways of Thinking About Organisations, 3rd edn. Prentice Hall, Harlow (2002) Sturmberg, J.P.: Embracing Complexity in Health: The Transformation of Science, Practice, and Policy. Springer (2019) Sturmberg, J.P., O’Halloran, D.M., Martin, C.M.: Understanding health system reform: a complex adaptive systems perspective. J. Eval. Clin. Pract. 18(1), 202–208 (2012) Sunstein, C.R.: Behavioral science and public policy. In: Cambridge Elements: Public Economics, p. 84. Cambridge University Press (2020) Tenbensel, T.: Complexity in health and health care systems. Soc. Sci. Med. 93, 181–184 (2013). https://doi.org/10.1016/j.socscimed.2013.06.017 Thompson, K.M., Duintjer Tebbens, R.J., Pallansch, M.A., Wassilak, S.G., Cochi, S.L.: Polio eradicators use integrated analytical models to make better decisions. Interfaces 45(1), 5–25 (2015) Trenholm, S., Ferlie, E.: Using complexity theory to analyse the organizational response to resurgent tuberculosis across London. Soc. Sci. Med. 93, 229–237 (2013) Weiss, M.: We the Possibility: Harnessing public entrepreneurship to solve our most urgent problems. (p. 256). Harvard Business Review Press (2020)

172

N. Singh

Wikipedia.: Volatility, uncertainty, complexity, and ambiguity. In Wikipedia (2021). https://en.wik ipedia.org/wiki/Volatility,_uncertainty,_complexity_and_ambiguity Wilkerson, I.: Caste: The Origins of Our Discontent, p. 496. Random House, New York (2020)

Chapter 9

Market State Dynamics in Correlation Matrix Space Hirdesh K. Pharasi, Suchetana Sadhukhan, Parisa Majari, Anirban Chakraborti, and Thomas H. Seligman

Abstract The concept of states of financial markets based on correlations has gained increasing attention during the last 10 years. We propose to retrace some important steps up to 2018, and then give a more detailed view of recent developments that attempt to make the use of this more practical. Finally, we try to give a glimpse to the future proposing the analysis of trajectories in correlation matrix space directly or in terms of symbolic dynamics as well as attempts to analyze the clusters that make up the states in a random matrix context.

H. K. Pharasi (B) · A. Chakraborti (B) School of Engineering and Technology, BML Munjal University, Gurugram 122413, Haryana, India e-mail: [email protected] A. Chakraborti e-mail: [email protected]; [email protected] S. Sadhukhan School of Advanced Sciences and Languages, VIT Bhopal University, Kothrikalan Sehore 466114, Madhya Pradesh, India e-mail: [email protected] P. Majari · T. H. Seligman Instituto de Ciencias Físicas, Universidad Nacional Autónoma de México, Cuernavaca 62210, Mexico e-mail: [email protected] T. H. Seligman e-mail: [email protected] A. Chakraborti School of Computational and Integrative Sciences, Jawaharlal Nehru University, New Delhi 110067, India Centre for Complexity Economics, Applied Spirituality and Public Policy, Jindal School of Government and Public Policy, O.P. Jindal Global University, Sonipat 131001, India A. Chakraborti · T. H. Seligman Centro Internacional de Ciencias, Cuernavaca 62210, Mexico © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 A. Chakraborti et al. (eds.), Quantum Decision Theory and Complexity Modelling in Economics and Public Policy, New Economic Windows, https://doi.org/10.1007/978-3-031-38833-0_9

173

174

H. K. Pharasi et al.

Introduction The purpose of this chapter is to give an overview of recent developments in the evolution of the concept of ‘states’ and ‘regions’ of financial markets based on correlation analysis (Münnix et al. 2012; Rinn et al. 2015; Pharasi et al. 2020). Many refinements have been conceived, some of which try to account for a practical applicability of the concept (Münnix et al. 2012; Pharasi et al. 2018, 2019; Chetalova et al. 2015; Stepanov et al. 2015; Rinn et al. 2015; Guhr and Schell 2021; Heckens et al. 2020; Meudt et al. 2015; Guhr 2015). It opens a new perspective, namely the stochastic dynamics of markets in the space of correlation matrices; the dynamics are discrete in time by the nature of markets governed by individual and countable transactions. The space in which the correlation moves, is of high dimensionality and not a discrete grid! This continuous approach may also be relevant to other applications such as social systems, traffic, economy, ecological systems, neurology and many more (Husain et al. 2020; Wang et al. 2020; Scheffer 2009; Scheffer et al. 2012; May et al. 2008; Sornette 2004; Weiss and McMichael 2004; Wang et al. 2017; Kawamura et al. 2012; Müller et al. 2011; Kwapie´n and Dro˙zd˙z 2012). Two points stand out in this context: On the one hand, the correlation matrices of financial markets have been studied over fairly long epochs, mostly with zero or 50% overlapping epoch, which led to a low number of points for the time scale considered. On the other hand, the clusters which define market states are not obviously visible in the reduced spaces, where they have been visualized using multidimensional scaling (MDS) (Torgerson 1952). Smaller epoch shifts (bigger overlapping epochs) in time, to overcome the above mentioned reason as well as for practical purpose, indicate that a trader wants up to date analysis to make his decisions as pointed out in Ref. Pharasi et al. (2020). Using MDS, one can visualize the similar correlation matrices into 2D and 3D clusters but without a cleanly separated boundaries. The transition matrices, consist of transition counts between the clusters, have shown their underlined relevance (Pharasi et al. 2018, 2019, 2020). Therefore, following the trajectories on shorter time scales as well as introducing symbolic dynamics seems very attractive. The former is in an advanced stage and will be described in this chapter in some detail, while the latter will be proposed and potential usefulness will be pointed out. In both cases, practically useful results are still outstanding. In the framework of these ideas, we start with a brief description of the origins of the correlation analysis and its evolution. We show how this naturally leads to an evolution in a space, whose dimension is determined by the length of the time series considered, i.e., by the time horizon as well as the number of free matrix elements in the correlation matrix. The analysis, based on the distances in the correlation matrix space between the points touched in time, defines the trajectory and the geometrical structure of the trajectory. Studying the correlation of market sectors following a technique proposed in Ref. Rinn et al. (2015) yields similar results as we shall see. The high dimensionality of the space in practice makes visualization very difficult. We could try to represent each state by, say, its average correlation matrix and this would reduce the problem to a discrete space as we would identify all correlation

9 Market State Dynamics in Correlation Matrix Space

175

matrices in a cluster with a single label or symbol. At this point it should be noticed that for traffic a three phase theory (Kerner 2009) is used; similarly, a four market states model is possible for the S&P 500 market and a higher number of states is certainly required for the Nikkei 225 market due to higher complexity of the Japanese market (Pharasi et al. 2018). Furthermore in the Japanese market a dominance of the average correlation for the structure of market states, found in the S&P 500 market (Münnix et al. 2012), clearly does not hold. This opens a perspective for the search of a second relevant parameter, which we shall not pursue here. In the framework of published and submitted works, which we will briefly describe, we propose two routes to advance the use of correlation matrices and market states. The first is MDS, say, to two or three dimensions, and the second is a symbolic dynamics after a partitioning of the set of observed correlation matrices according to the cluster structure used for defining the states. It, in turn, partitions the space on which the symbolic dynamics evolve in time. We shall show that in the two relevant financial markets (S&P 500 and Nikkei 225) this structure is only marginally affected by MDS, which thus emerges as a very effective tool for visualization and interpretation. We will barely outline the second option, namely the possible usefulness of the symbolic dynamics resulting from the developed cluster structure. In financial markets, we have previously opted epochs shifted by one day (Pharasi et al. 2020), thus omitting the intra day trading data, both because they are more difficult to access and of different nature than daily trading data. We limit our study to a time interval 2006–2019 excluding the year 2020, though we will show scaled trajectory for 2020 that is interesting yet lack of interpretation at present. We shall develop basics and outline the development that led over the years to the present work. Next we give some technical aspects of data handling and processing. Then we proceed to show results for the correlations of the USA market as reflected by the S&P 500 and the Tokyo Market by the Nikkei 225 using both full correlation matrices and market sector averages. We start by discussing the market states, their transition matrices and then passing to the trajectories in the space of correlation matrices with visualizations including at this point some aspects of the 2020 development of markets during the starting period of COVID-19 pandemic. At this point we shall also mention trajectories in the symbolic dynamics. No conclusions in either case can be presented as it is an ongoing work. Finally we shall try to give an outlook as to where we think the field is heading and what could be done about applications to other fields.

Methodology Data Description We analyze the daily adjusted closure prices . Sk (t) of stock .k for the S&P 500 and Nikkei 225 indices at different time horizons. The daily trading data is freely available

176

H. K. Pharasi et al.

on the Yahoo finance website (Yahoo finance database 2020). We consider only those stocks which are continuously traded over the considered time period. We again filtered out a few stocks time series that have more than two consecutive missing trading days entries. Missing prices entries have been taken to be same as the previous ones. We then calculate the logarithmic price returns .rk (t) = ln Sk (t + Δt) − ln Sk (t) for each stock .k at time .t, where .t = 1, 2, . . . , Ttot with .Ttot as total number of the trading days present in the time horizon considered, and .k = 1, . . . , N for . N number of stocks and .Δt = 1 day for daily price returns.

Evolution of Cross-Correlation Structures To understand the co-movement of the financial market constituents (stocks) and the time evolution of the market conditions, we calculate the Pearson correlation coefficients for .τ th epoch of size .T days as .Ci j (τ ) = ( − )/σri σr j where .i, j = 1, . . . , N , and .σ is the standard deviation. Here . denotes an averaging over an epoch length of .T trading days which is shifted by .ΔT days through the data. The choice of epoch is important because market conditions evolute with time and the correlations between different stocks may not be linear for longer epochs, on the other hand, short epoch size leads to “noise” or “fluctuation” due to the singularity present in the corresponding correlation matrix. Therefore, the distribution of eigenvalue spectrum of the empirical cross-correlation matrix .C(τ ) contains “random” and as well as non-random contribution (Plerou et al. 2000; Pandey et al. 2010). That motivates us to compare the eigenvalue statistics of .C(τ ) with a large random correlation ensemble constructed from mutually uncorrelated time series or white noise known as Wishart matrix. We will introduce Wishart Orthogonal Ensembles (WOE) in detail in the next subsection.

Wishart Orthogonal Ensembles We construct a rectangular random data matrix . A = [ Ai j ] of . N random time series of length .T , i.e., of order . N × T with real independent elements drawn from a standard Gaussian distribution with fixed mean and variance. The Wishart matrix is then constructed as . W = T1 A A' of size . N × N , where . A' denotes the transpose of the matrix . A. We here consider the entries as real, known as Wishart orthogonal ensemble and also, by construction, it is a real symmetric positive semidefinite matrix. To obtain the correlation matrix, we fix the value of mean to zero and variance to 2 .σ = 1. Following the Mar˘ cenko-Pastur distribution, the probability density function .ρ(λ) ¯ of the eigenvalues of such correlation matrices can be analytically shown for the limit . N → ∞ and .T → ∞ as

9 Market State Dynamics in Correlation Matrix Space

{

177

Q √ (λmax 2πσ 2

− λ)(λ − λmin )/λ if Q > 1 (λmax − λ)(λ − λmin )/λ + (1 − Q)δ(λ) if Q ≤ 1 where. Q = T /N fixed. The maximum and minimum eigenvalue can be derived from ( √ )2 max 2 1 ± 1/ Q . For . Q ≤ 1, one has to take into account of at least . N − T .λmin = σ ¯ is normalized to . Q and not to unity. zeros, hence the density .ρ(λ) ρ(λ) ¯ =

.

Q 2πσ 2

√

Power Map Technique To obtain the stationarity in time series, the consideration of shorter time series by breaking a long time series into epochs, each of size .T (such that .Ttot /T = Fr ) is useful. However, if . N (stocks).> T , we get a singular correlation matrix with . N − T + 1 zero eigenvalues, results in poor eigenvalue statistics. We introduce here the power map technique (Guhr and Kälber 2003; Vinayak 2014) to lift the degeneracy of zero eigenvalues by providing a non-linear distortion (. ∊, also called noise suppression parameter) to each element .(Wi j ) of the Wishart matrix . W : 1+ ∊ . Wi j → (sign Wi j )|Wi j | . This technique helps to get an “emerging spectrum” of eigenvalues by removing the degenerated eigenvalues at zero even for very small distortions, e.g., . ∊ = 0.001 (see e.g., Refs. Chakraborti et al. (2020); Pharasi et al. (2018); Vinayak (2013) for recent studies and applications).

Pairwise (dis)similarity Measures and Multidimensional Scaling In pairwise (dis)similarity analysis, the correlation matrix is first calculated for each epoch (which represents a specific time epoch) and we compute the (dis)similarity by measuring the distance between the two correlation matrices. Such analysis is performed for each pair of correlation matrices (epochs) and defined as: .ζ (τ1 , τ2 ) ≡ , where .|...| and . denote the absolute and average value over all the matrix elements, respectively and .τ = 1, . . . , Fr with . Fr is the total number of epochs of size.T constructed from the overlapping shift of.ΔT days. This technique yields a symmetric matrix, with all positive off-diagonal elements . Fr (Fr − 1)/2. Diagonal elements are zero for no such (dis-)similarities present between the same epochs. We use multidimensional scaling Torgerson (1952), which exhibits the structure of (dis-)similarity data by creating a map, it constructs a geometrical representation from the relative distances (.ζ ) of correlation epochs where similar objects are located nearby than the dissimilar objects.

178

H. K. Pharasi et al.

Identifying States of a Financial Market In this section, we present an overview of recent developments in the evolution of the concept of ‘states’ of financial stock markets based on correlation analysis. We start with the very first work on the characterization of “market states” based on the similarity of the correlation of market states by Münnix et al. (2012). Figure 9.1 shows the temporal evolution of the market states of the S&P 500 market for 19year observation period 1992–2010. Each point in the figure represents a correlation matrix measured over the previous two months. The market is clustered into eight states (S1–S8) based on a top-down clustering method using a similarity matrix .ζ . In this method, initially all the correlation matrices are considered as a single cluster and then further divided into sub-clusters using .k-means clustering algorithm. The division process stops as the average radius of all the points to centroids becomes smaller than a certain threshold. In the Fig. 9.1, initial points are clustered in lower states (S1 and S2) with less transitions to other states, hence shows a stable behavior of the market before 1996. On the other hand, one can observe more frequent jumps for higher states with scattered points after 1996 and shows high market volatility. In this method, on one side, the number of clusters is dependent on an arbitrary threshold and on the other side, the correlation matrices constructed by short time series are highly singular and dressed with noise. The noise dressing has large impact on portfolio optimization and risk assessment (Schäfer et al. 2010). It has been found by Laloux et al. (1999) that the correlation matrices are noisy due to the finite size of the time series. Guhr and Kälber (2003) has presented the power map method to suppress the noise of the correlation matrices, which effectively increases the length of time series. This method is more suitable for the analysis of short time series. In Fig. 9.2, we show the effect of the power map method and the length of time series on the spectral density .ρ(λ) of the Wishart orthogonal ensemble (WOE). We use various epoch lengths .T = 1000, 5000, 10000, 40000 and noise suppression parameters . ∊ = 0, 0.265, 0.383, 0.63 and show that the spectral density .ρ(λ) of WOE

Fig. 9.1 The plot shows the dynamical evolution of the market states for S&P 500 market over the 19-year period 1992–2010. The market shows the transitions between eight characterized states (S1–S8) based on the top-down clustering method. The lower states (i.e., S1 and S2), denoted as normal periods of the market, tend to cluster in time revealing that the market remains in those particular states for a longer time. On the other hand, higher states are scattered and irregular in time showing the rapid to and fro transitions between the states with the higher probability of transitions to the nearby states. Figure is adapted from the Ref. Münnix et al. (2012)

9 Market State Dynamics in Correlation Matrix Space

(a)

179

(b)

Fig. 9.2 Effect of the power map method and length of time series on the spectral distribution of the Wishart orthogonal ensemble (WOE). a shows the effect of finite size on the spectral densities .ρ(λ) of a Wishart matrix ensemble (1000) of size . N × N , constructed from N time series (. N = 500, fixed) of real independent Gaussian variables, each of finite epoch length .T (varying) with unit mean and varying variance .σ 2 . The variance of the distribution decreases with the increment of T and the ensemble is becoming an identity for higher .T . The same can be achieved by increasing the value of the noise suppression parameter . ∊ of the power map. b shows the effect of the power map by varying . ∊ and keeping . N and .T fixed

approaches towards identity matrix for higher value of .T (for fixed . N and . ∊) and . ∊ (for fixed . N and .T ). Figure 9.2a shows the spectral density .ρ(λ) of a Wishart matrix ensemble (ensemble .= 1000) of size . N × N , constructed from . N = 500 (fixed) time series of real independent Gaussian variables, for four different epoch lengths 2 . T (.1000, 5000, 10000, 40000) with unit mean and variance .σ . The variance of the distribution decreases with the increment of .T and move towards the identity matrix for higher .T . The same can be achieved by increasing the value of the noise suppression parameter . ∊ of the power map. Figure 9.2b shows the effect of the power map by varying the value of . ∊ but keeping . N and .T fixed. The comparison of spectral densities .ρ(λ) in Fig. 9.2a, b show that the similar variance can be achieved either by increasing the length of the time series or by increasing the value of noise suppression parameter . ∊. To suppress the noise of the correlation matrices and to avoid the arbitrariness of the threshold in the top-bottom clustering technique. Münnix et al. (2012), Pharasi et al. (2018) developed a new approach to obtain the optimum number of market states for a financial market. Similar approach can be applied to other complex systems. The optimization is achieved using two parameters—intra cluster measure and noise suppression parameter. Intra cluster measure .dintra is an averaged euclidean distance of points to cluster centroids of the .k-means clustering performed on the MDS map. It is constructed from the similarity matrix .ζ using 805 noise-suppressed (. ∊ = 0.6) correlation matrices of USA market and 798 noise-suppressed (. ∊ = 0.6) correlation matrices of Japanese market and overlapping epoch .T = 20 days shifted by .Δ = 10 days over the period 1985–2016, shown in Fig. 9.3a, b, respectively. Five hundred different initial conditions are used to calculate the intra cluster distances denoted

180

(a)

(c)

H. K. Pharasi et al.

(b)

(d)

Fig. 9.3 Plots of (top row) inter cluster distance as a function of the number of clusters and (bottom row) transition counts of paired market states for S&P 500 (a) and (c) and Nikkei 225 (b) and (d) markets of noise suppressed (. ∊ = 0.6) correlation matrices constructed from the short epoch of length .T = 20 days with shifts of .Δ = 10 days over the period 1985–2016. Five hundred different initial conditions are used to calculate the inter cluster distances denoted by colored lines in a, b. Based on the optimization conditions of keeping the standard deviation of the ensemble lowest and the number of clusters highest, we have found .k = 4 and .k = 5 are the optimum number of clusters for the USA and JPN, respectively. Color map plots for c S&P 500 and d Nikkei 225 markets show the transition counts of paired market states. In both markets, the highest transition counts in the diagonal show high re-occurrence of the market states followed by first off diagonal transitions to nearest states. Thus the penultimate state to the critical state behaves like a precursor to the market crash and is the best state to hedge against the crash. The figures are recreated based on the Ref. Pharasi et al. (2018)

by colored lines as shown in Fig. 9.3a, b for S&P 500 and Nikkei 225, respectively, as a function of the number of clusters. Figure 9.3c, d show the transition counts of paired market states for S&P 500 and Nikkei 225 markets, respectively. The optimum number of cluster for.k ≥ 4 is chosen such as the standard deviation of the ensemble is lowest while keeping the number of clusters highest. Our analysis depicts that .k = 4

9 Market State Dynamics in Correlation Matrix Space

181

and .5 are the optimal number of clusters for S&P 500 and Nikkei 225, respectively. We cluster similar correlation matrices into these optimized .k number of “market states”. The market evolves through the transitions between these market states. Transitions to nearby states are highly probable. Highest transition counts of market states occurs when market remains in the same state i.e. along the diagonal followed by first off diagonal as transitions to nearby states. The jump from the lowest . S1 and . S2 state to the critical state (. S4 in USA and . S5 in JPN) is less probable which acts as a precursor to the critical state and as a best state to hedge against the critical state. Recently, Pharasi et. al. in Ref. Pharasi et al. (2020) have presented an improved criteria for the classification of “market state”, mainly due to the increased attention to the transition matrix of paired states. The optimization of the number of market states is done by using the following two parameters- number of clusters and noise suppression. However, here the preference is given to the transition matrix which avoids large jumps from lower states to the critical state. It brings a better perspective to hedge against the market’s critical events without a prohibitive cost. By using the same overlapping epoch of length 20 days but the shifts of 1 day, the results show significantly improved statistics as compared to the shift of 10 days in Ref. Pharasi et al. (2018). Figure 9.4a, b show the identification of number of market states for S&P 500 (. N = 350 stocks) and Nikkei 225 (. N = 156 stocks) markets, respectively, over 14-year period from 2006–2019 using overlapping epochs of length .T = 20 days with shift of .Δ = 1 day. In contrast to Fig. 9.3, here we use ensemble of 1000 different initial conditions for different number of clusters .k and noise suppression parameter . ∊. It is found that the minimum of .σdintra for .k ≥ 4 at .k = 5 and . ∊ = 0.9 for S&P 500 and at .k = 7 and . ∊ = 0 for Nikkei 225 markets. This optimum number of cluster is used for the k-means clustering for 3503 correlation epochs with noisesuppression .( ∊ = 0.9) for S&P 500 market and 3439 correlation epochs without noise-suppression (. ∊ = 0) for Nikkei 225 market. The 2D projection of 3D .k-means clustering for S&P 500 market on .x z-plane with .k = 5 clusters and Nikkei 225 market on .x y-plane with .k = 7 clusters, respectively, are shown in Fig. 9.4c, d. The distribution of clusters is different for two market—for S&P 500 market, .k-means clustering divides the distribution of correlation epochs in 3D correlation space in the horizontal direction (. S1, S2, S3, S4, S5) but for the Nikkei 225, it divides in both vertically stacked (S2 & S3 and S4 & S5) as well as horizontal directions. Figure 9.5a, b depict the schematic diagram of average correlation matrices of each market state for S&P 500 and Nikkei 225 markets, respectively. From lower (calm period) to higher (crisis period) market states, mean correlation as the correlation of intra- and inter-sectorial correlation increases. Figure 9.5c, d show the color map plots of the transition counts of paired market states for S&P 500 with .k = 5 clusters and noise-suppression parameter . ∊ = 0.9 and Nikkei 225 market with .k = 7 clusters and without noise-suppressed (. ∊ = 0), respectively over the period of 2006–2019. Transition matrix for S&P 500 market is nearly tridiagonal and shows zero transitions from lower states (S1, S2, and S3) to critical state (S5) with twelve transitions from penultimate state (S4) to critical state (S5). Thus, S4 behaves as a precursor to the market crashes and a good agent to hedge. For Nikkei 225 market, the transition counts of the paired market states between S2 and S3 (S2.→ S3 = 25, vertically

182

H. K. Pharasi et al.

(a)

(b)

(c)

(d)

Fig. 9.4 Identification of number of market states based on minimum standard deviation of intra cluster distances .σdintra (colorbar) for a S&P 500 and b Nikkei 225 markets. For the correlation analysis considered in Ref. Pharasi et al. (2020), . N = 350 stocks of the S & P 500 index and . N = 156 stocks of the Nikkei 225 index traded over 14-year period from 2006–2019 for overlapping epochs of length .T = 20 days shifted of .Δ = 1 day are used. Plots show the measure of .σdintra (colorbar) as a function of noise suppression parameter . ∊ (zoomed in portion of Fig. 9.1 in Ref. Pharasi et al. (2020)). We have found that the minimum of .σdintra calculated over 1000 initial conditions for .k ≥ 4 at .k = 5 and . ∊ = 0.9 for S&P 500 and at .k = 7 and . ∊ = 0 for Nikkei 225 markets. Plots c and d show the .2D projection of .3D .k-means clustering for S&P market on . x z-plane and Nikkei 225 market on . x y-plane, respectively. It is important to note that the cluster distributions (only horizontal in the USA but both horizontal and vertical in JPN) for the two markets are significantly different. The figure is adapted from Ref. Pharasi et al. (2020)

stacked) are smaller than S2 and S4 (S2.→S4 = 45, in horizontal direction). A similar behavior is observed during the transitions between states S4 and S5, and S4 and S6. Therefore, state S6 behaves as a precursor to the critical state S7 with one transition, contrary to S&P 500, from S5 to critical state S7.

9 Market State Dynamics in Correlation Matrix Space

183

(a)

(b)

(c)

(d)

Fig. 9.5 Schematic diagram of average correlation matrices of a five market states of S&P 500 and b and seven market states of Nikkei 225 markets over the period of 2006–2019. The average is taken over all the correlation matrices of each market state. The correlation structure as well as mean correlations varies over different market states. Color map plots denote the transition counts of paired market states for c S&P 500 with .k = 5 clusters and noise-suppression parameter . ∊ = 0.9 and d Nikkei 225 market with .k = 7 clusters and without noise-suppression (. ∊ = 0). The transition matrix for b S&P 500 market is nearly tridiagonal and shows zero transitions from lower states (S1, S2, and S3) to critical state (S5) with twelve transitions from the penultimate state (S4) to critical state (S5). Thus, S4 behaves as a precursor to the market crashes and a good agent to hedge. For d Nikkei 225 market, the transition counts of the paired market states between S2 and S3 are smaller than S2 and S4. Here, state S6 behaves as a precursor to the critical state S7. The figure is adapted from Ref. Pharasi et al. (2020)

184

H. K. Pharasi et al.

Sectorial Analysis We consider here the same data of . N = 350 stocks of the S&P 500 index (. N = 156 stocks of the Nikkei 225) over the period from 2006–2019 with .Ttot = 3523 .(Ttot = 3459) trading days (Pharasi et al. 2020). We then calculate a sectorial averaged matrices . M by taking intra- and inter-sectorial average of the correlation matrices .C constructed from the overlapping epoch of .T = 20 days and shifted by .Δ = 1 day. The symmetric matrices . M are not correlation matrices any more and the diagonal elements are not trivial. We adopt the similar approach used by Rinn et al. (2015), where they performed a cluster analysis of sectorial averaged correlation matrices and stochastic process analysis of financial market dynamics by identifying a few important variables. The market showed dynamical stability and transitions between the stable quasi-stationary states. The method allows to project the dynamics of high-dimensional system to a low-dimensional space. In this manner, we have the matrices . M with dimension . N S × N S , where . N S is the number of the sectors in the market—. N S = 10 for S&P 500 and . N S = 6 for Nikkei 225. The number of independent variables in the . M matrix are . N S (N S + 1)/2. The direct advantage of analyzing these matrices . M, in comparison to full correlation matrices .C, is their low dimensional space, i.e., 55D for S&P 500 and 21D for Nikkei 225 markets which are much smaller than the original . N (N − 1)/2 dimensional space of correlation matrix .C. Working in lower dimensional space has an advantage of reduced computational time and provides a useful tool for studying sectoral behavior. We then apply the .kmeans clustering analysis on these matrices . M after applying the appropriate noise suppression (Guhr and Kälber 2003; Vinayak 2013), based on the optimization using Fig. 9.6. We can achieve the same dimensional reduction by taking the average of log-return time series for each sectors first and then calculate the correlations among these . N S time series. But, we have found that this method is not effective as by taking average of time series, we average out a lot of information and unusually land up on high correlations even in the normal calm periods. In Fig. 9.6, we optimize of number of market states based on the minimum standard deviation of intra cluster distances .σdintra . Figure 9.6a, b show the optimization using intra- and inter-sectorial averaged matrices . M for S&P 500 and Nikkei 225 markets, respectively. We apply .k-means clustering on MDS map, constructed from noise suppressed sectorial averaged matrices . M. We use 1000 different initial conditions for the calculation of.σdintra . According to the minimum of standard deviations of intra cluster distances, the best choice (.k > 4) for S&P 500 is .k = 5 clusters and . ∊ = 0.2 (see, Fig. 9.6a), and for Nikkei 225, .k = 5 clusters and . ∊ = 0.3 (see, Fig. 9.6b). Figure 9.7 shows the classification of the sectorial averaged matrices . M of S&P 500 and Nikkei 225 markets. The US market is clustered into five market states and corresponding .k-means clustering and transition matrix of paired market states are shown in Fig. 9.7a, b, respectively. Here we choose.k = 5 clusters based on the robust measure (minimum of .σdintra for .k > 4) of .k-means clustering using Fig. 9.6a. We have found that the .k-means clustering and transition matrix show similar behavior as shown in the sectorial analysis (see, Fig. 9.4). But for the Japanese market, we

9 Market State Dynamics in Correlation Matrix Space

185

(a)

(b)

Fig. 9.6 Optimization of number of market states based on the minimum standard deviation of intra cluster distances .σdintra for a S&P 500, and b Nikkei 225 markets. We use MDS map, constructed from noise suppressed sectorial averaged matrices. M, for.k-means clustering. We use 1000 different initial conditions for the calculation of .σdintra . According to the minimum of standard deviations of intra cluster distances, the best choice (.k > 4) for a S&P 500 is .k = 5 clusters and . ∊ = 0.2, and for b Nikkei 225, .k = 5 clusters and . ∊ = 0.3

186

H. K. Pharasi et al.

(a)

(b)

(c)

(d)

Fig. 9.7 Classification of the US market into five market states (top row) and the Japanese market into eight market states (bottom row). Plots show the 2D projection of 3D .k-means clustering constructed from noise suppressed matrices . M with a . ∊ = 0.2 for S&P 500 and c . ∊ = 0.7 for Nikkei 225 markets. Color map plots for b S&P 500 and d Nikkei 225 markets show the transition counts of paired market states. For S&P 500 market, we optimize the number of clusters using .σdintra using Fig. 9.6a. For the classification of the Japanese market into market states, we preferred eight clusters over five due to the transition matrix which avoids large jumps from lower states to the critical state

preferred eight clusters over five due to (.i) the transition matrix which avoids large jumps from lower states to the critical state and (.ii) the similar distribution of clusters as obtained from the stocks analysis (see, Fig. 9.4d). The Japanese market is more complex than USA market and the distribution of .k-means clusters (.k = 8) is in both vertical and horizontal directions in JPN market which is only in the horizontal direction for USA market. Transition matrices, consist of transition counts between different states, for S&P 500 and Nikkei 225 represent a valuable tool to analyze the behavior of critical state (S5) are shown in Fig. 9.7b, d, respectively. In this method, we have found that the number of epochs in the crash state of S&P 500 market are higher (false positive) than one obtained from stock analysis in Fig. 9.5. Furthermore, by increasing the number of clusters provide us appropriate way in which risk assessment can be studied with further details. The transition matrix, shown in Fig. 9.7d for Nikkei 225, with eight number of clusters has less number of epochs in critical state which would be helpful for better risk assessment. The displacement between stocks analysis and sector analysis for S&P 500 with five market states and noise suppressed . ∊ = 0.2 is equal to .465 days. We must

9 Market State Dynamics in Correlation Matrix Space

187

note that the maximum of displacement, in this case, is .di = ±1, with respect to the stock analysis of the Ref. Pharasi et al. (2020). There are .3038 epochs which remain in the same states and we have .309 epochs with displacement of .di = −1 and .156 epochs with displacement of .di = +1. On the other hand, the maximum of displacement for Nikkei 225 with eight market states and noise suppressed . ∊ = 0.7 is equal to .±3. We have .14, 25, 813, 2074, 419, 94, 0 epochs with displacement of .di = −3, −2, −1, 0, 1, 2, 3, respectively.

Trajectories in the Correlation Matrix Space In this section, we compare the COVID-19 crash of S&P 500 and Nikkei 225 market with other critical events as well as normal periods of both markets. For this purpose, we considered the daily adjusted closure price data and the observation period is given in Tables 9.2 and 9.3 (. N = 475 for S&P 500 and . N = 204 for Nikkei 225 from November 2019–June 2020). We consider the observation period of .125 trading days keeping the crash day at the center.

Comparison of COVID-19 Case with Other Crash and Normal Periods We use MDS that reproduces the original metric or distances. Classical multidimensional scaling at various dimensions (D = .1, 2, 3, 4, and size of .ζ ) of (dis)similarity matrix produces co-ordinates from a D-dimensional map. We can measure the Euclidean distance between pairs of all co-ordinates. Table 9.1 shows the correlation between the Euclidean distances at various dimensions for various crashes and normal periods. We consider here crashes at .1987, .2008, .2010, .2011, .2015, COVID-.19, two normal periods—.2006 and .2017, and for the .14 years data from .2006–2019. Except for the last case, we have taken the data of .125 trading days. It is evident that the correlation coefficient is maximum for .4D and the size of .ζ for all the analysis periods. Figures 9.8 and 9.9 show the dynamics of the (dis-)similarity measure between consecutive correlation matrices in the correlation matrix space for S&P 500 and Nikkei 225 markets, respectively. In the figures, we use MDS map to show the trajectories in 3D space for different time horizons: (a) .1987 Black Monday crash, (b) 2008 Lehman Brother crash, (c) .2017 normal period, and (d) COVID-19 crash. The crash date is highlighted with hollow black star symbol in the figure. Each point in the figure corresponds to a correlation matrix of 20 days epoch. The distance among correlation matrices is small when market remains in the same state or transition occurs in nearby market states showing nearly equidistant increments in the trajectory whereas when the transitions happens from lower states to higher market states during

188

H. K. Pharasi et al.

Table 9.1 The table shows the correlation, measured for all consecutive pairs of co-ordinates of MDS at various dimensions. Here, . D1 = 1, . D2 = 2, . D3 = 3, . D4 = 4, . Dmax =size of .ζ . We consider here crashes in.2008, .2010, .2011, .2015, COVID-.19, .1987, two normal periods—.2006 and .2017, and for the .14 years’ data from .2006–2019 for the S&P 500 market. Except for the last case, we have taken the data of .125 trading days. The correlation is more between a higher dimension with the maximum dimension (size of .ζ ) Correlation between

1987

2006

2008

2010

2011

2015

2017

COVID19

2006– 2019

D.1 , D.max

0.9455

0.7360

0.7065

0.8817

0.8957

0.8940

0.7916

0.8602

0.8361

D.2 , D.max

0.9490

0.7673

0.7384

0.9035

0.9205

0.9010

0.8797

0.8731

0.8689

D.3 , D.max

0.9569

0.8407

0.7776

0.9253

0.9362

0.9119

0.9146

0.9081

0.8802

D.4 , D.max

0.9633

0.8561

0.8694

0.9374

0.9399

0.9279

0.9274

0.9145

0.8855

critical events then corresponding distance measure shows some intermittent abrupt increments. The Tables 9.2 and 9.3 show the analysis of characterization of critical and normal events based on the variance of MDS map of correlation matrix trajectories of .125 days (.6 months approx.) for the S&P 500 and Nikkei 225 markets, respectively. We have considered data from various critical and normal periods, details are given in the second column of the table with the corresponding starting and ending date in third and fourth columns, respectively. We categorized the normal events into two groups: Normal-1 period, where the trajectory is bounded between states S1 and S2, and Normal-2 period, where the trajectory is bounded among three states S1 to S3 based on the characterization shown in Figs. 9.4 and 9.5. We use the variances of the 3D MDS map to calculate the ratio of variances of y- and x-directions (.σr 2 = σ y 2 /σx 2 ). Note that, the ratio stays below 0.4 for critical events and goes above 0.4 for normal periods. There is a false positive event for the S&P 500 market where the value for the critical event IPO Facebook Debut (Serial No 7) goes above 0.4. Also, for Nikkei 225 market the ratio of variances for the critical events in August 2011 fall (Serial No 2) and IPO Facebook Debut (Serial No 8) show the value higher than 0.4. We are still investigating the reason behind it.

Conclusions and Future Outlook We have presented a brief overview of the evolution of the states of financial markets based on the similarity of correlation matrices obtained over .T = 40 and .20 days epochs with shifts ranging from zero percent (non-overlapping) to 95 percent (one day shift). The adjusted closing data of the S&P 500 and the Nikkei 225 indices are appropriately purified. The line of ideas starts with the work by Münnix et al. (2012) in 2012, where they proposed to group similar correlation structures as “market state” and showed the evolution of the financial market through these market states.

9 Market State Dynamics in Correlation Matrix Space

(a)

(b)

(c)

(d)

189

Fig. 9.8 Evolution of the trajectories of the (dis-)similarity measure between consecutive correlation matrices in the correlation matrix space for S&P 500 market. The trajectories in 3D space is the projection of . Fr (Fr − 1)/2 dimensional space of the similarity matrix .ζ , with dimension . Fr × Fr , using MDS for S&P 500 market for different time horizons: a .1987 Black Monday crash, b 2008 Lehman Brother crash, c .2017 normal period, and d COVID-19 crash. For the analysis, we have considered a period of .125 trading days keeping the crash date at the center (hollow black star). The trajectory consists of equidistant points with some intermittent abrupt increments during critical events. The (dis-)similarity measure is the distance among correlation matrices so when the transition occurs between two consecutive correlation matrices in the same or nearby market states then the corresponding distance measure shows small and nearly equidistant increments. When the transitions happen between different market states then the corresponding distance measure shows big increments. We have also compared here three crashes and one normal period

They used a top-bottom .k-means clustering technique, which was later replaced by minimization of intra cluster size while requiring stability of noise reduction in order to determine the optimal number of clusters. We then describe a more recent development to take advantage of some remaining freedom of choice by searching for desirable properties of the transition matrix between states. It is important at this stage, to mention that noise reduction is carried out at the level of the correlation

190

H. K. Pharasi et al.

(a)

(b)

(c)

(d)

Fig. 9.9 Evolution of the trajectories of the (dis-)similarity measure between consecutive correlation matrices in the correlation matrix space for Nikkei 225 market. The trajectories in 3D space is the projection of . Fr (Fr − 1)/2 dimensional space of the similarity matrix .ζ , with dimension . Fr × Fr , using MDS for S&P 500 market for different time horizons: a .1987 Black Monday crash, b 2008 Lehman Brother crash, c .2017 normal period, and d COVID-19 crash. For the analysis, we have considered time period of .125 trading days keeping the crash date at the center (hollow black star). We have also compared here three crashes and one normal periods

matrix rather than at the level of individual time series. We briefly discuss the power map introduced and used extensively by the group of Guhr and Kälber (2003), Schäfer et al. (2010). The amusing interpretation of this method as an artificial lengthening of the time series (Guhr and Kälber 2003) is revisited by analyzing its effects in synthetic data obtained from correlated Wishart ensembles (Pharasi et al. 2020). The line of argument leads up to recent work detailing the use for risk assessment (Pharasi et al. 2020). It brings a better perspective to hedge against the market’s critical events without a prohibitive cost. By using the same overlapping epoch of length 20 days but the shifts of .1 day, the results showed significantly improved statistics as compared to 10 days shifts. Using this criterion over the period of 2006–

9 Market State Dynamics in Correlation Matrix Space

191

Table 9.2 The table shows characterization of critical and normal events based on the variance of MDS map of correlation matrix trajectories of.125 days (.6 months approx.) for the S&P 500 market. The critical events are kept at the center of the trajectory, constructed from the overlapping epoch of .T = 20 days and shifts of .Δ = 1 day. The second column provides the name of the various events and corresponding time periods are shown in third (Starting date) and fourth (End date) columns. Fifth, sixth, and seventh columns show the variances .σx 2 , σ y 2 , σz 2 , respectively, of the 3D MDS map using the similarity matrices. For the analysis, we take the ratio of variances of yand x-directions (.σr 2 = σ y 2 /σx 2 ) in eighth column. We have found that the ratio stays below 0.4 for critical events and goes above 0.4 for normal periods. There is a false positive event, the ratio for the critical event IPO Facebook Debut (Serial No 7) shows the value higher than 0.4, we are still investigating the reason behind it Serial No.

Period

Starting date

End date

.σx 2

.σ y 2

.σz 2

.σr 2

1

Black Monday crash

Aug 4, 1987

Jan 5, 1988

0.0248

0.0035

0.0029

0.142

2

August 2011 fall

May 23, 2011

Oct 21, 2011

0.0213

0.0029

0.0017

0.1349

3

DJ Flash crash

Feb 19, 2010

Jul 22, 2010

0.034

0.0038

0.0022

0.1109

4

Lehman Brothers crash

Jul 1, 2008

Dec 1, 2008

0.0239

0.0035

0.0026

0.1442

5

Covid-19 crash

Dec 27, 2019

Jun 1, 2020

0.0303

0.0062

0.0032

0.2059

6

Brexit

Apr 8, 2016

Sep 8, 2016

0.0212

0.0054

0.0038

0.2538

7

IPO Facebook Debut

Mar 5, 2012

Aug 3, 2012

0.0089

0.0045

0.0034

0.5134

8

Flash Freeze

Jun 7, 2013

Nov 6, 2013

0.0114

0.0042

0.004

0.3661

9

Treasury Freeze

Jul 31, 2014

Dec 31, 2014

0.0156

0.0051

0.0038

0.3249

10

Chinese Black Monday

Jun 9, 2015

Nov 6, 2015

0.0255

0.0043

0.004

0.1671

11

Normal-1(S1 to S2)

Jul 26, 2006

Dec 26, 2006

0.0071

0.005

0.0046

0.7098

12

Normal-1(S1 to S2)

Sep 12, 2006

Feb 14, 2007

0.0061

0.0049

0.0047

0.8122

13

Normal-1(S1 to S2)

Oct 10, 2016

Mar 14, 2017

0.0081

0.0063

0.0048

0.7823

14

Normal-1(S1 to S2)

Mar 30, 2017

Aug 30, 2017

0.0074

0.0056

0.0041

0.7628

15

Normal-1(S1 to S2)

Sep 1, 2017

Feb 5, 2018

0.0079

0.0065

0.0051

0.8255

16

Normal-2(S1 to S3)

Jun 1, 2006

Oct 31, 2006

0.0124

0.0052

0.0039

0.4161

17

Normal-2(S1 to S3)

Mar 27, 2007

Aug 27, 2007

0.0105

0.0043

0.0036

0.4117

18

Normal-2(S1 to S3)

Apr 21, 2009

Sep 21, 2009

0.0059

0.0049

0.0029

0.8198

19

Normal-2(S1 to S3)

Sep 28, 2010

Mar 1, 2011

0.0124

0.005

0.0042

0.40

20

Normal-2(S1 to S3)

Jul 10, 2012

Dec 11, 2012

0.007

0.0051

0.0039

0.7345

21

Normal-2(S1 to S3)

Jan 20, 2015

Jun 22, 2015

0.0073

0.0051

0.004

0.7066

22

Normal-2(S1 to S3)

Apr 24, 2018

Sep 24, 2018

0.0082

0.0058

0.0047

0.7054

2019, the S&P 500 and Nikkei 225 markets is characterized into five market states with . ∊ = 0.7 and seven market states with . ∊ = 0 (no suppression). Further, we studied the cluster analysis of market states constructed from averaged intra- and inter-sectorial matrices to identify the market dynamics. Using the same data over the period 2016–2019, the results for S&P 500 and Nikkei 225 are statistically similar to the one obtained from the stock analysis. We showed that the better choice of transition matrix over a minimum of .σdintra . In the Japanese case, the preference is given to a transition matrix that avoids large jumps from lower states to the critical state. Using this approach, we have a better transition matrix

192

H. K. Pharasi et al.

Table 9.3 The table shows characterization of critical and normal events based on the variance of MDS map of correlation matrix trajectories of .125 days (.6 months approx.) for the Nikkei 225 market. The details of the method is given in Table 9.2. The second column provides the name of the various events and corresponding time periods are shown in third (Starting date) and fourth (End date) columns. There are a few false positive events, the ratio for the critical event August 2011 fall (Serial No 2) and IPO Facebook Debut (Serial No 8) show the value higher than 0.4 Serial No.

Period

Starting date

End date

.σx 2

.σ y 2

.σz 2

.σr 2

1

Black Monday crash

Aug 10, 1987

Dec 26, 1987

0.0479

0.0055

0.0043

0.115

2

August 2011 fall

May 24, 2011

Oct 25, 2011

0.0083

0.004

0.0025

0.4768

3

DJ Flash crash

Feb 15, 2010

Jul 22, 2010

0.0118

0.0033

0.0024

0.2807

4

Lehman Brothers crash

Jul 1, 2008

Dec 4, 2008

0.0234

0.0038

0.0035

0.1638

5

Covid-19 crash

Dec 3, 2019

May 15, 2020

0.014

0.0038

0.0035

0.2707

6

Tsunami/Fukushima

Dec 21, 2010

Jun 1, 2011

0.0227

0.0042

0.0032

0.1835

7

Brexit

Apr 5, 2016

Sep 8, 2016

0.0168

0.0052

0.002

0.3072

8

IPO Facebook Debut

Feb 29, 2012

Aug 2, 2012

0.0088

0.0041

0.0031

0.4676

9

Flash Freeze

Jun 7, 2013

Nov 11, 2013

0.0071

0.0025

0.002

0.3526

10

Treasury Freeze

Jul 29, 2014

Jan 6, 2015

0.0185

0.0043

0.0032

0.2346

11

Normal-1(S1 to S3)

May 9, 2017

Oct 4, 2017

0.0084

0.006

0.0039

0.7171

12

Normal-1(S1 to S3)

Jun 15, 2017

Nov 10, 2017

0.0075

0.0062

0.0053

0.8251

13

Normal-2(S1 to S4)

Apr 19, 2017

Sep 19, 2017

0.0104

0.006

0.0038

0.582

14

Normal-2(S1 to S4)

Jun 19, 2017

Nov 14, 2017

0.0072

0.0061

0.0053

0.845

15

Normal-2(S1 to S4)

Aug 15, 2017

Jan 10, 2018

0.01

0.0052

0.0045

0.523

and similar distribution of clusters as obtained from the analysis of the stocks. Note that in this approach, we do not perform the clustering of a correlation matrix. A simple-minded approach averaging the series corresponding to each sector yields a different result that does not seem satisfactory. Finally, we try to look a little into the future, by inspecting the time dynamics in the dimensionally scaled space of the correlation matrices in a representation dimensionally scaled to three dimensions, which seems quite interesting, though no conclusions emerge. This leads to an alternative proposition using the market states as the domains on which symbolic dynamics can be built and this also seems like a promising lane to expand the research. A more detailed analysis of the structure of each cluster is an another interesting topic of future research. Acknowledgements The authors are grateful to Francois Leyvraz for their critical inputs and suggestions. The authors also thank CIC AC -UNAM for their hospitality in various events and DGAPA, Mexico for financial support under grant number AG101122 and CONACyT, Mexico for financial support under FRONTERAS grant number 425854. H.K.P., P.M. and S.S. are grateful for financial support provided by UNAM-DGAPA and CONACYT Proyecto Fronteras 952. T.H.S. and H.K.P. acknowledge the support grant by CONACYT Proyecto Fronteras 201, UNAM-DGAPAPAPIIT AG100819 and IN113620. T.H.S. and H.K.P. also acknowledge computing support under project LANCAD-UNAM-DGTIC-016

9 Market State Dynamics in Correlation Matrix Space

193

References Yahoo finance database. https://finance.yahoo.com/ (2020). Accessed on 7th January, 2020 for S&P 500 and 14th January, 2020 for Nikkei 225 Chakraborti, A., Sharma, K., Pharasi, H.K., Bakar, K.S., Das, S., Seligman, T.H.: Emerging spectra characterization of catastrophic instabilities in complex systems. New J. Phys. 22(6), 063,043 (2020) Chetalova, D., Schäfer, R., Guhr, T.: Zooming into market states. J. Stat. Mech. Theory Exp. 2015(1), P01,029 (2015) Guhr, T.: Non-stationarity in financial markets: dynamics of market states versus generic features. Acta Phys. Pol. B 46(9) (2015) Guhr, T., Kälber, B.: A new method to estimate the noise in financial correlation matrices. J. Phys. A: Math. Gen. 36(12), 3009 (2003) Guhr, T., Schell, A.: Exact multivariate amplitude distributions for non-stationary gaussian or algebraic fluctuations of covariances or correlations. J. Phys. A: Math. Theor. 54(12), 125,002 (2021) Heckens, A.J., Krause, S.M., Guhr, T.: Uncovering the dynamics of correlation structures relative to the collective market motion. J. Stat. Mech. Theory Exp. 2020(10), 103,402 (2020) Husain, S.S., Sharma, K., Kukreti, V., Chakraborti, A.: Identifying the global terror hubs and vulnerable motifs using complex network dynamics. Phys. A: Stat. Mech. Appl. 540, 123,113 (2020) Kawamura, H., Hatano, T., Kato, N., Biswas, S., Chakrabarti, B.K.: Statistical physics of fracture, friction, and earthquakes. Rev. Mod. Phys. 84, 839–884 (2012) Kerner, B.S.: Introduction to modern traffic flow theory and control: the long road to three-phase traffic theory. Springer Science & Business Media (2009) Kwapie´n, J., Dro˙zd˙z, S.: Physical approach to complex systems. Phys. Rep. 515(3–4), 115–226 (2012) Laloux, L., Cizeau, P., Bouchaud, J.P., Potters, M.: Noise dressing of financial correlation matrices. Phys. Rev. Lett. 83, 1467–1470 (1999) May, R.M., Levin, S.A., Sugihara, G.: Ecology for bankers. Nature 451, 893–895 (2008) Meudt, F., Theissen, M., Schäfer, R., Guhr, T.: Constructing analytically tractable ensembles of stochastic covariances with an application to financial data. J. Stat. Mech. Theory Exp. 2015(11), P11,025 (2015) Müller, M.F., Baier, G., Jiménez, Y.L., García, A.O.M., Rummel, C., Schindler, K.: Evolution of genuine cross-correlation strength of focal onset seizures. J. Clin. Neurophys. 28(5), 450–462 (2011) Münnix, M.C., Shimada, T., Schäfer, R., Leyvraz, F., Seligman, T.H., Guhr, T., Stanley, H.E.: Identifying states of a financial market. Sci. Rep. 2, 644 (2012) Pandey, A., et al.: Correlated Wishart ensembles and chaotic time series. Phys. Rev. E 81(3), 036,202 (2010) Pharasi, H.K., Seligman, E., Sadhukhan, S., Seligman, T.H.: Dynamics of market states and risk assessment. arXiv preprint arXiv:2011.05984 (2020) Pharasi, H.K., Seligman, E., Seligman, T.H.: Market states: a new understanding. arXiv preprint arXiv:2003.07058 (2020) Pharasi, H.K., Sharma, K., Chakraborti, A., Seligman, T.H.: Complex market dynamics in the light of random matrix theory. In: New Perspectives and Challenges in Econophysics and Sociophysics, pp. 13–34. Springer International Publishing, Cham (2019) Pharasi, H.K., Sharma, K., Chatterjee, R., Chakraborti, A., Leyvraz, F., Seligman, T.H.: Identifying long-term precursors of financial market crashes using correlation patterns. New J. Phys. 20(10), 103,041 (2018) Plerou, V., Gopikrishnan, P., Rosenow, B., Amaral, L.N., Stanley, H.E.: A random matrix theory approach to financial cross-correlations. Physica A: Stat. Mech. Appl. 287(3–4), 374–382 (2000) Rinn, P., Stepanov, Y., Peinke, J., Guhr, T., Schäfer, R.: Dynamics of quasi-stationary systems: finance as an example. EPL (Europhys. Lett.) 110(6), 68,003 (2015)

194

H. K. Pharasi et al.

Schäfer, R., Nilsson, N.F., Guhr, T.: Power mapping with dynamical adjustment for improved portfolio optimization. Quant. Finance 10(1), 107–119 (2010) Scheffer, M.: Critical Transitions in Nature and Society. Princeton Studies in Complexity. Princeton University Press (2009) Scheffer, M., Carpenter, S.R., Lenton, T.M., Bascompte, J., Brock, W., Dakos, V., van de Koppel, J., van de Leemput, I.A., Levin, S.A., van Nes, E.H., Pascual, M., Vandermeer, J.: Anticipating critical transitions. Science 338(6105), 344–348 (2012) Sornette, D.: Why Stock Markets Crash: Critical Events in Complex Financial Systems. Princeton University Press (2004) Stepanov, Y., Rinn, P., Guhr, T., Peinke, J., Schäfer, R.: Stability and hierarchy of quasi-stationary states: financial markets as an example. J. Stat. Mech. Theory Expe. 2015(8), P08,011 (2015) Torgerson, W.: Multidimensional scaling: I. theory and method. Psychometrika 17(4), 401–419 (1952) Vinayak, Schäfer, R., Seligman, T.H.: Emerging spectra of singular correlation matrices under small power-map deformations. Phys. Rev. E 88, 032,115 (2013) Vinayak, Seligman, T.H.: Time series, correlation matrices and random matrix models. In: AIP Conference Proceedings, vol. 1575, pp. 196–217. AIP (2014) Wang, S., Gartzke, S., Schreckenberg, M., Guhr, T.: Quasi-stationary states in temporal correlations for traffic systems: Cologne orbital motorway as an example. J. Stat. Mech. Theory Exp. 2020(10), 103,404 (2020) Wang, W., Tang, M., Stanley, H.E., Braunstein, L.A.: Unification of theoretical approaches for epidemic spreading on complex networks. Rep. Prog. Phys. 80(3), 036,603 (2017) Weiss, R.A., McMichael, A.J.: Social and environmental risk factors in the emergence of infectious diseases. Nat. Med. 10(12), S70–S76 (2004)

Chapter 10

Interstate Migration and Spread of Covid-19 in Indian States Debajit Jha, Suhaas Neel, Hrishidev, and Anirban Chakraborti

Abstract There has been a huge impact of the Covid-19 pandemic on the Indian economy. Almost all the sectors of the economy have been severely impacted. The vulnerability of the economy was highest after the imposition of the nationwide lockdown in March 2020, when thousands of migrant workers were forced to return to their homes from the large or metropolitan cities of the country. After the lockdown was lifted and some months passed, people had started migrating again to their workplaces. However, at the advent of the second wave and state-wise lockdowns imposed, a similar exodus of migrants was seen. While this humanitarian crisis and the associated government policies in India have been heavily discussed in the national and international policy discourses, there has been no systematic study or estimates of the mobility of migrants during the pandemic. Moreover, although reverse migration has been perceived to be one of the primary reasons for the spread of the virus in the Indian states, there is still no study showing the exact relationship between reverse interstate migration and the spread of Covid-19 cases in Indian states, D. Jha Centre for Complexity Economics, Applied Spirituality and Public Policy, Jindal School of Government and Public Policy, O. P. Jindal Global University, Sonipat 131001, Haryana, India e-mail: [email protected] S. Neel School of Engineering, Jawaharlal Nehru University, New Delhi 110067, India e-mail: [email protected] Hrishidev · A. Chakraborti (B) School of Computational & Integrative Sciences, Jawaharlal Nehru University, New Delhi 110067, India e-mail: [email protected]; [email protected] Hrishidev e-mail: [email protected] A. Chakraborti School of Engineering and Technology, BML Munjal University, Gurugram 122413, Haryana, India Centre for Complexity Economics, Applied Spirituality and Public Policy, O. P. Jindal Global University, Sonipat 131001, Haryana, India © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 A. Chakraborti et al. (eds.), Quantum Decision Theory and Complexity Modelling in Economics and Public Policy, New Economic Windows, https://doi.org/10.1007/978-3-031-38833-0_10

195

196

D. Jha et al.

most probably due to unavailability of a consistent dataset. This has further affected policymaking for tackling the crisis. In this paper, first, we have tried to construct a proxy dataset of “reverse migration” using interstate train running information during the second wave of the Covid-19 pandemic. To understand the complex trend and pattern of interstate human mobility during the second wave of the Covid-19 pandemic, we construct the complex networks of mobility of people using the train running data. Then, we have carried out an econometric exercise to identify the relationship between reverse migration and the spread of Covid-19 in different Indian states, using both the train running information data that we have constructed, as well as the Census-2011 data on migration. The results show that both train migration and census migration have positive and significant impact on the spread of the Covid-19 cases in the Indian states. Besides migration, population density per square kilometre, percentage of the urban population, and state per capita income were also found to have a positive impact on the spread of the virus. Moreover, we found that the availability of hospital beds in different states has helped to reduce the spread of the virus. Keywords Covid-19 · Pandemic · Migration · India · Reverse migration

Introduction Migration of people is a complex phenomenon where a large number of factors often interact non-linearly to influence the decision to migrate from one place to another. People normally migrate to the places where earning opportunity is more from the places where opportunity of earning is less. However, people can also migrate for reasons which may not be necessarily economic. The pandemic induced reverse migration in India is one of such types of migration. As Covid-19 pandemic started, with a rising case count, the Government of India restricted mobility of people by declaring a nationwide lockdown to curb the spread of the virus. However, the announcement of the lockdown had created anxiety, fear and uncertainty among the migrants stranded in different Indian states with uncertain income; residence etc. which had resulted in a huge exodus of migrants from different parts of the country to the origin states. After the lockdown was lifted and some months passed, people had started migrating back again to their workplaces. However, at the advent of the second wave and state-wise lockdowns imposed, a similar exodus of migrants was seen. While it has been apparent from various media sources and policy discourses that the movement of the migrants from one state to another has been assumed to be a major source of the spread of the virus (Ray and Subramanian 2020), academic study on the possible relationship between the reverse migration and the spread of the virus in Indian states is still limited, most probably due to the unavailability of an official estimate of the number of reverse migrants from different Indian states during this period. While there are three main sources of internal migration data in the country are—Census of India, NSSO surveys on migration, and the IHDS database,

10 Interstate Migration and Spread of Covid-19 in Indian States

197

none of them provide an estimate of the number of migrants in different Indian states and districts in recent times, let alone the number of reverse migration induced by the recent pandemic. There are two main problems. First, while the principal source of migration data, the Census of India, provides numbers of migrants till 2010, the NSSO survey on migration was last conducted in the year 2007. Similarly, the most recent IHDS-II survey provides information on migrant laborers until 2011–12. Thus all these data sources are outdated for the purpose of estimating the number of reverse migrants in Indian states. Second, often these three data databases provide different estimates of the number of internal migrants due to the differences in the methodology of the surveys. In the absence of reliable data the policymaking for tackling the crisis was based only on the assumption on the number of migrants. In the present study, we have attempted to explore the relationship between pandemic-induced reverse migration of domestic migrants and Covid-19 cases in Indian states for both the first wave and the second wave of the Covid-19 pandemic. There are two objectives of this study. First, we try to construct a dataset that can provide a reliable estimate of pandemic-induced mobility of people from different parts of the country using train running information on the Covid-19 special trains. The Economic Survey of India (2017) had shown that most of the poor interstate migrants in India use the train as the main source of transportation. Second, we conduct an econometric exercise to uncover the relationship between the mobility of people from Maharashtra, one of the largest migrant-receiving states, to different Indian states during the second wave of the pandemic using train passenger data constructed in the first part. We have focused on the reverse migration from Maharashtra because the second wave of Covid-19 in Maharashtra has started at least two months ahead of the second wave in other Indian states. The rest of the paper has been organized as follows. In section “Impact of Pandemic on Migrants” first, we discuss the list of different data used in this study along with their sources. In this section, we also discuss the method to construct the train passenger mobility data among the Indian states using the Covid-19 special train information following a complex network approach. The complex network diagrams constructed from the Census-2011 data and train passenger mobility data have been presented in section “Data and Methodology”. We present the detailed specification of the econometric models and the results of the econometric exercise in section “Visualization of Complex Network of Migration”. Section “Relationship between migration and spread to Covid-19” concludes the study with a discussion on the possible public policies to tackle the complex issue like reverse migration induced health crisis in the country.

Impact of Pandemic on Migrants As already pointed out above an important aspect of interstate human migration is its complexity and dependence on multiple factors often interacting nonlinearly. There are many impacts of internal migration that have been discussed in the literature.

198

D. Jha et al.

While relocation of migrants is deemed to be necessary, possibilities of circular (or reverse) migration can exacerbate the health crisis (Ginsburg et al. 2021). Although the restrictions on mobility may not help impede the progression of the pandemic, such policies may end up imposing a negative externality like unemployment, hunger, and impoverishment at an unimaginable scale (Abi-Habib and Yasir 2021; Carter 2016). In the context of the earlier SARS pandemic in 2009, it has been identified that the impact of lockdown and quarantine-related measures may impose a utility cost on the remaining population, thus acting as an incentive to vacate the quarantined locale (Mesnard and Seabright 2009). Similar arguments with added baggage can be applied to the low-income interstate migrants in India. The lockdown was enforced in different countries soon after the pandemic struck in the hopes of limiting interaction and policies were framed to limit the negative externalities resulting from these interactions (Bethune and Korinek 2020). However, the closure of various sectors and restrictions in others was a big problem for developing countries such as India. Soon after the lockdown, we witnessed an exodus of migrants heading back to their native places both internationally and domestically (World Bank 2020; Mitra et al. 2020). Even the second wave had a repeat of the 2020 exodus but to a lesser extent (Padhee et al. 2021). During the first wave, with a rising case count and death toll, the government started restricting international travel. The government made prior permission mandatory to enter different Indian states. This was done to curb the spread of the virus. However, an obvious negative externality of these policies was a lack of mobility for the many stranded workers in large cities across the country (Rajan et al. 2020). This caused a financial and moral crisis, the result being the introduction of the shramik train, special busses etc. that were made available by the government to escort migrant workers to their native places. As noted above, the reverse migration resulting from Covid-19 and the lockdown caused a financial crisis. The contraction of the economy and the many closed businesses were not a solitary phenomenon limited to industries and the state, but it had a profound impact on the financial health of the families of the workers and their communities. This was especially true for low-income migrants whose families sustained themselves on remittances, often as the sole source of income (Bahera et al. 2021). As most of these low-income migrants came from economically backward states, this leads to further socio-economic troubles in the originating states. Furthermore, low-income migrants and their families in India often struggle for basic amenities; it is quite evident that they would not have been able to sustain themselves for long without governmental assistance or employment opportunities. The onus once again comes to employment which was the primary reason the migrants left in the first place. Given that a substantial portion of the migrants was from rural areas (Census 2011), it could mean strain on the rural economy. The economy of places like tourist destinations, education hubs etc. are also impacted heavily. A unique example of this would be the town of Kota in Rajasthan. The city runs on lakhs of students temporarily moving in for their education. As they leave, the prime source of income is lost resulting in heavy financial strain on the people depending on the education sector.

10 Interstate Migration and Spread of Covid-19 in Indian States

199

Another specific group that has been severely affected by the pandemic and the ensuing lockdown is workers from the unorganized sectors. It is known that the number of jobs in the unorganized sector quantitatively far surpassed that of the organized sector. About 40% of India’s export is from the unorganized sector only. It is quite easy to apprehend that the effect of the pandemic is expected to be akin to the way global financial crises had affected people in the unorganized sector (Estrada 2020). The 2009 global financial crisis had badly affected the unorganized sector in India; about 22 million jobs were lost in this sector since the onset of the crisis. Around that time, 100,000 Gem and jewelry-related jobs were lost in Gujrat, and 500,000 garment-related jobs were lost around Ludhiana in Punjab and Tirupur in Gujarat (UNDP; Kumar et al. 2009). A defining trait of low-earning migrants is a lack of social security, owing to the overflowing supply of their labour. This means that amid the recession these groups were especially vulnerable to a substantial decrease in quality of life. In parallel to the issues discussed above, the same group has sustenance problems due to being able to earn barely more than a sustenance wage. This does not mean that all the migrants are bound to face similar problems; however, a substantial number of such migrants, especially those employed in the unorganized sector are expected to be impacted the most. Furthermore, the migrant workers without written contracts (and/or unskilled workers) are even more likely to fall under such a category. After the mass exodus amid deplorable conditions that had spurred the collective consciousness of the Indian public, the government responded with a draft on national migration labour policy (Mehrotra 2021a). This draft was formulated by NITI Aayog and the working subgroup of officials and members of civic society. The draft builds on the 2017 report’s views that call for comprehensive laws regarding the protection of the rights of workers in the unorganized sector which comprise a sizable portion of the migrant workforce, The draft marks out two viewpoints on policy design for the migrants. The initial approach consists of cash transfers and special quotas. While the second and the preferred method “enhances the agency and capability of the community and thereby removes aspects that come in the way of an individual’s own natural ability to thrive”. Some proposals include the setup of coordination mechanisms between the origin and destination states of migrants (Mehrotra 2021b). The draft also emphasizes on the loss of political representation for the migrants due to being registered in their state of origin. Hence some proposals include mechanisms to allow migrant participation, thus enabling political inclusion. Furthermore, use of Aadhar card for inclusion of migrants in social welfare programs like PDS which should remain accessible and undeterred across state borders (Niti Aayog 2020). A special unit under the ministry of labour and employment has also been proposed to implement the aforementioned proposals and developing maps of the focal points of migration and for the set-up of migrant worker cells. A major difference from the 2017 report was the recommendation of raising the minimum wage in the origin states to “stem” migration.

200

D. Jha et al.

Data and Methodology In this study we have used different datasets and a complex network approach to construct the interstate reverse migration data in India.

Data To identify the existing network of interstate mobility of people before the pandemic we have used Census 2011 data from the census of India. The census uses a two-step process to collect the data. It is in the second step that the data regarding migration and its reasons were collected. The data is available as an adjacency matrix in the form of a interstate dyadic variable. In order to construct the Covid-19 induced reverse migration data, we have used publicly available information on Covid-19 special trains from India Rail Info (https:/ /indiarailinfo.com/). Because the ministry of railways does not provide detailed information on the mobility of passengers, we have scraped the train running information of Covid-19 special trains from the website of India Rail Info (https://indiarailinfo. com/) using a web crawler. Then the scraped data has been used to construct a complex interstate mobility network of train passengers in the country for the period 1st June 2020 to 7th June 2021. The aforementioned website is a crowd sourced database holding information regarding all types of trains, and currently hosts the data regarding trains run since the advent of the Covid-19 spurred lockdown. The database is being updated continuously. The data for the confirmed Covid-19 cases have been collected from the website of Covid-19 India Org (https://www.Covid19india.org/). The website stores data for daily cases and total cases for recovered, deceased, confirmed, and active cases. Finally, the data on population density, the fraction of urban population, gross state domestic product per capita, etc. have been taken from the Handbook of Statistics on Indian States, Reserve Bank of India, while the state-level data on health infrastructure and health personnel have been collected from Health Profile of India, Government of India.

Methodology of Network Construction As discussed above, to construct the estimate of reverse migration in Indian states we have used a complex network approach discussed by Sadekar et al. (2021). The nodes in the network represent states while the outgoing edge represents emigration and the incoming edge represents immigration. The total numbers of nodes are 35 including states and union territories. The number is fewer than the current number of states and union territories due to Telangana and Andhra Pradesh being treated

10 Interstate Migration and Spread of Covid-19 in Indian States

201

like one state. This was done due to a lack of explicit data for Telangana for some of the other variables used in this study. The same can be said about Ladakh and Jammu and Kashmir which were under the same jurisdiction before August 2019. In constructing the train passenger mobility network we used two assumptions— (i) all the trains run at full capacity, and (ii) the number of people who move from state i to state j is proportional to their populations. The exact formulae that has been used are as follows ][ ] [ { k Nb , for, 2 ≤ a < b ≤ k, Fab = C { kNa N N j=a

j

j=b+1

j

] [ Fab = C { kNb N , for, 1 = a < b ≤ k. j=a

j

Here F(a, b) is the movement from the state a to state b. C is the total capacity of the trains. N i is the population in state i. To identify the total number of passengers travelled by a train, we needed the information on the days it had run and the capacity it had per run (i.e., the seating capacity of the bogies). The seating capacity was calculated using the number of bogies which were available as the rake position. However, this data was not available for all trains hence we assigned the average seating capacity as the seating capacity for the trains where the rake position is missing. The individual bogie types and the number of their kinds in a train have been iterated to identify the total seating capacity in a train. This data regarding the bogie type and capacity was taken from the website https://www.trainman.in/. After getting the total capacity using the days the train had run, we used the prior mentioned formulae to calculate the edges of the network. To calculate the days the train had run, we used the date of the first run and the date of the last run.1

Visualization of Complex Network of Migration In order to understand the pattern of interstate migration of individuals between different Indian states before the pandemic, first, we present the network plots constructed using Census-2011 migration data in section “Introduction”. In section “Impact of pandemic on migrants” we present the network plots constructed using the data on Covid-19 special trains for two periods—1st of June 2020–7th June 2021 [full sample] and 14th Feb 2021–14th May 2021 [Second Wave]. Finally, 1

This data was not available for all the trains. So to construct the network, we have only used those trains for which the date of the first and last run was available. This shortened the number of trains used in forming the network; hence we have used only 36 percent of a total of 2832 trains running during the entire period. Also note that when we have constructed the network for shorter periods, the capacity variable is altered according to the amount of overlapping running days of each train with the start and end dates we are considering.

202

D. Jha et al.

in section “Data and Methodology” we discuss the nature of a possible lead-lag relationship between reverse migration from Maharashtra to other Indian states and Covid-19 cases in those states.

Network Plots Using the Census-2011 Data The network for the 2011 census migration is given above. For the sake of visualizing better, we have only included the first 200 edges in this plot as in the next. The most prominent edges are the connections from Uttar Pradesh to Maharashtra and from Uttar Pradesh to Delhi. These are shown in brown and black colours respectively and have around 1 million migrants each. This is not surprising as Uttar Pradesh has the highest quantity of intra-national emigrants (Census of India 2011). Following these, we have connections from Bihar to Delhi, Uttar Pradesh to Gujarat, and Karnataka to Maharashtra. As is apparent from the network, Maharashtra has the most immigrants. There are 8 states from where Maharashtra has more than 100 thousand immigrants. In the network, this has been represented by edges with yellow tints and darker tints. A similar number is seen for Uttar Pradesh but the flow was opposite. Uttar Pradesh has 8 states for which it has more than 100 thousand emigrants, represented in the same manner in the network with arrows reversed. However, unlike Bihar, Uttar Pradesh has substantial in-migration itself. For the 2011 census, Uttar Pradesh, Bihar, Rajasthan, and Madhya Pradesh [in order of quantity of emigrants] had more than 50 percent emigrants. While states like Maharashtra, Delhi, Gujarat, Uttar Pradesh, and Haryana account for 50 percent of the total immigrants. Note that Uttar Pradesh is a part of both lists. This has been further demonstrated in the Sankey diagrams plotted for the same data. Sankey diagrams are used to represent flow and the width of the lines represent the flow rate used, mainly used in engineering. Here the flow rate is synonymous with migrants moving from the node on the left side [out-migrating state] to the node on the right [in-migrating state]. Along with the complete census network, we have included a Maharashtra-specific network, which shows the states from where migration occurs in Maharashtra (Figs. 10.1, 10.2 and 10.3). These trends have been consistent over the decades and several studies have been conducted to understand such behavior with the lens of various competing models. One of the most popular models, Lee’s model has been originally introduced in Lee (1966), Rani (2016), Wang (2010), Massey (1993) are some other works in this pursuit. In India, migration has been on the rise since 1981 (Raja and Bhagat 2021). The list of states with high out-migration and in-migration of the population has not shown much change (Das and Saha 2013).

10 Interstate Migration and Spread of Covid-19 in Indian States

203

Fig. 10.1 Interstate migration network before the pandemic using Census-2011 data. Source Author’s estimation

Fig. 10.2 Sankey diagram using census-2011 data. Source Census Data

204

D. Jha et al.

Fig. 10.3 In-migration in Maharashtra using Census-2011 data. Source Census Data

Network Plots Using Covid-19 Special Train Data The network constructed from the Covid-19 special trains is then plotted for two time periods. The first one is for the entire period and the other one is only for the second wave. Just like for the Census-2011 data, the entire network is very dense, so only the top 200 edges based on the weights are shown. Each edge is between two states and the nodes corresponding to each state are marked in its capital city. The width of the line as well as the color of each edge is drawn according to the weight of that edge. Thicker and darker (Towards the black) edges represent the connections with higher weights and the thinner and lighter (Towards the yellowish-white) edges represent the ones with the lower weights. Both the networks show heavy connections between the states of Maharashtra, Gujarat, Bihar, Uttar Pradesh, and Delhi. In the case of the second wave we have chosen only those trains that have run between the 14th February and 14th May then the same procedure of visualization is followed. Sankey plots have been used as well to demonstrate the total movement of people between the states. Similar to the Census-2011 data, the width for a state represents the total mobility of people from that state and the right side shows the total mobility to that state (Figs. 10.4, 10.5, 10.6 and 10.7). Since we are interested in the role of Maharashtra particularly, the outgoing links from Maharashtra alone are plotted. There were 20 non-zero edges.

10 Interstate Migration and Spread of Covid-19 in Indian States

205

Fig. 10.4 Full sample network using Covid-19 special train data. Source Author’s estimation

Fig. 10.5 Train mobility network for data between 14th Feb and 28th June. Source Author’s estimation

Possible Relationship Between Covid-19 Cases and Migrants In Fig. 10.8 the time series plots have shown a clear lead-lag relationship in the Covid confirmed data for individual states, where many of the states are observed lagging behind Maharashtra. Maharashtra is also observed to be a major node in

206

D. Jha et al.

Fig. 10.6 Network for outmigration from Maharashtra between 14th Feb and 14th May. Source Author’s estimation

Fig. 10.7 Sankey diagram for the train dataset. Source Author’s estimation

10 Interstate Migration and Spread of Covid-19 in Indian States

207

the migration network with very strong links with other important states such as Uttar Pradesh and Delhi. In order to explore this relationship between migration and Covid-19, this figure plots the number of people who have travelled to different states from Maharashtra and the total number of confirmed cases in those states in the two axes. We are using the proxy data constructed from the COVID special express train information as a measure of migration between the states. And the daily confirmed cases data is integrated for the required period to calculate the total number of Covid19 cases reported. Two scatter plots (Figs. 10.10 and 10.11) are made for the two time periods that we are interested in which are (a) 14th Feb–14th May [From beginning to the end of the second wave] and (b) 14th Feb–28th Jun [From the beginning of the second-wave to the last available date]. Along with the data points, a linear fit is also shown to observe any general trend and the correlation coefficient between the two variables is calculated to see the strength of the relationship. Figure 10.8 shows confirmed cases for different states in India. It is a time series plot showing the various phases of the Covid-19 pandemic that unfolded in India for all the different states. Since Maharashtra is the state with the highest number of immigrants and at the same time, the highest contributor state to the number of total Covid-19 cases in India, we will focus on that more staunchly. The start of the first wave in India amid lockdown was visibly slower than the second, and we can see a simultaneous increase in the numbers for every state. The peak of the first wave was reached around 11th September 2020 which was also the peak of the first wave

Fig. 10.8 Time plot of Covid-19 cases in Indian states. Source Covid 19 Crowdsourced data

208

D. Jha et al.

Fig. 10.9 Migration from Maharashtra using Covid-19 special train data. Source Author’s estimation

in Maharashtra. This changed for the second wave when Maharashtra bore witness to its second wave much earlier than the rest of India. While the case count was stable for most of the states and even practically disappeared for some states like Bihar which reported fewer than 100 cases for weeks. Nearly a month after cases started rising in Maharashtra, other states started reporting a rise in confirmed cases count as well; effect seen in the constructed network too (vide Fig. 10.9). Several vouched for caution and depicted the possibility of a second wave. At the same time, these concerns were unwarranted to some even after a surge in cases in Maharashtra (Noronha 2021). It is also important to note that because Delhi already had 2 such upswings, it is understandable where such views come from.

Relationship Between Migration and Spread to Covid-19 In order to find out the relationship between reverse migration and the spread of Covid-19 cases in Indian states during the second wave of Covid-19 spread, we relied on the ordinary least squares (OLS) regression approach. In this study, we have taken only the 20 Indian states which are connected through a train network. The dependent variable in all the models is the total number of confirmed Covid-19

10 Interstate Migration and Spread of Covid-19 in Indian States

209

cases per lakh population in different Indian states between 14th Feb 2021 and 14th May 2021. The independent variables used in the model are— I. The number of train passengers travelled from Maharashtra per lakh population of a state during this period. II. Number of migrants from different Indian states to Maharashtra. III. Population density. IV. Percentage of the urban population. V. Hospital beds per lakh population. VI. Doctors per lakh population. VII. Log gross state domestic product per capita. The first two independent variables try to capture the reverse migration of labourers from Maharashtra to different Indian states. The hypothesis for taking these two variables is that the virus can spread from one state to another only through human transmission. Since migrant workers return to their place of origin whenever a lockdown has been imposed, these returning migrants may be one of the crucial factors behind the spread of the virus from one state to another besides international migrants. Studies have already shown a positive and significant relationship between international migrants and the spread of the virus in the district of India during the first 45-days of the onset of the pandemic. The first variable is the number of train passengers travelling from Maharashtra to different Indian states per lakh population of that state. Since the train is one of the most important mediums of transport for the migrant workers in the country (Economic Survey 2017) and the second wave of the pandemic has started in Maharashtra almost two months ahead of other Indian states (see Fig. 10.8), reverse migration from Maharashtra may be one of the principal sources of the spread of the virus from Maharashtra to other Indian states. Accordingly, we expect a positive impact of this variable on the spread of Covid-19 cases in Indian states. The second variable is the number of 0–9 year migrants in Maharashtra from different Indian states per lakh population of that state. Since the exodus of migrant workers has been perceived to be one of the crucial spreaders of the virus to different states and Maharashtra is one of the largest migrant-receiving states in the country, the reverse migration from Maharashtra is expected to be high. Accordingly, we can expect a positive relationship between the spread of Covid-19 cases in different Indian states and the number of migrants in Maharashtra from other states. For an initial assessment of the possible relationship between these two variables, we present two scatter plots of the number of train passengers travelling to different Indian states from Maharashtra during the second wave of the pandemic and Covid-19 confirmed cases during the same period in Figs. 10.10 and 10.11. Both the figures show a positive relationship between confirmed cases and train passenger mobility data during the second wave of the pandemic. In different studies on the spread of the infectious virus during earlier epidemics, it has been found that population density helps the virus to spread from one person to another. Similarly, the population density is more in urban areas relative to rural areas. Hence, population density per square kilometre and percentage of urbanization population in a state may also be important determinants of the Covid-19 virus

210

D. Jha et al.

Fig. 10.10 Reverse migration from Maharashtra versus Covid-19 confirmed cases from 14th Feb 2021 to the end of the sample. Source Author’s estimation

Fig. 10.11 Reverse migration from Maharashtra versus Covid-19 confirmed cases between 14th Feb 2021 and 14th May 2021. Source Author’s estimation

10 Interstate Migration and Spread of Covid-19 in Indian States

211

spread. Therefore, we can expect a positive impact of these two variables on the total Covid-19 cases per lakh population in the regression models. Moreover, the availability of health infrastructures such as the number of hospital beds and the availability of health personnel such as doctors, nurses can have a negative impact on the spread of the virus. In order to treat the infected persons and quarantine suspected Covid-19 patients, we need more hospital beds. Hence the availability of hospital beds may reduce the spread of the virus. In the states where more hospital beds are available per lakh population the chance of spread of the virus is less. Similarly, the availability of doctors can help in the dissemination of crucial information regarding the safety norms during a pandemic. Hence, these two variables, hospital beds per lakh population and availability of doctors per lakh population, are expected to have a negative relationship with Covid-19 cases in a state. The last independent variable that we have considered in this study is log per capita gross state domestic product. Higher per capita income states can spend a higher amount of state resources to curb the spread of the virus. Accordingly, we can expect a negative association between log per capita income and Covid-19 cases. However, more urbanized and densely populated states are found to have higher per capita income in this country. Hence, higher per capita income and Covid-19 cases can also have a positive relationship with Covid-19 cases in a state. Hence, it is difficult to predict the expected sign of this variable. Based on the above discussion the regression model that we have used in this study is as follows— Y = f (T M I G, C M I G, U R P O P, P O P D E N , I N F B E D, I N F D OC, PC I ) (10.1) where Y is our dependent variable which denotes Covid-19 cases per lakh population in the states. TMIG represents the number of train passengers per lakh population who traveled from Maharashtra to another Indian state during the second wave of the pandemic. CMIG is the number of 0–9 year migrants in Maharashtra from other Indian states per lakh population of the respective state. URPOP represents the percentage of the urban population in a state. Similarly, POPDEN denotes the density of population in a state. INFBED is a health infrastructure variable and represents the number of hospital beds available per lakh population in a state. INFDOC represents the number of doctors available per lakh population. PCI denotes log per capita GSDP in a state. Based on the above equation we have carried out two sets of regression models. In the first set, the main independent variable is TMIG, i.e., the number of train passengers per lakh population traveled from Maharashtra to another Indian state during the second wave of the pandemic. In the second set, the main independent variable is CMIG i.e., the number of 0–9 year migrants in Maharashtra from other Indian states per lakh population of the respective state. In both the first and second set of regression models, we have carried out six different regression models. Therefore, the functional forms of the first set of our regression models are as follows—

212

D. Jha et al.

Y = β0 + β1 T M I G + β2 U R P O P + β3 PC I + ε1 Y = β0 + β1 T M I G + β2 U R P O P + β3 PC I + β4 I N F B E D + ε2

(10.2) (10.3)

Y = β0 + β1 T M I G + β2 U R P O P + β3 PC I + β4 I N F B E D + β5 I N F D OC + ε3 (10.4)

Y = β0 + β1 T M I G + β2 P O P D E N + β3 PC I + ε4 Y = β0 + β1 T M I G + β2 P O P D E N + β3 PC I + β4 I N F B E D + ε5

(10.5) (10.6)

Y = β0 + β1 T M I G + β2 P O P D E N + β3 PC I + β4 I N F B E D + β5 I N F D OC + ε6

(10.7) The results of the above regression models have been presented in columns 2–7 respectively in Table 10.1. Since both urban population and population density have a similar impact and may have a high correlation, we have not taken them in the same regression model. While URPOP has been taken in models 1–3, in models 4–6 we have used POPDEN. In all the models the main independent variable TMIG is found to be positive and significant. This suggests that an increase in one more person traveled from Maharashtra through train per lakh population in a state would increase 0.13 number of Covid-19 cases per lakh population of that state in model 1. While POPDEN, URPOP, and PCI have been found to be positive and significant, INFBED and INFDOC are insignificant. This indicates that densely populated, more urban, and richer states have a higher chance of spread of the virus. Since most of the high per capita income states in India are more urbanized and have relatively more population density, the positive sign of PCI is not surprising. The sign of INFBED is as expected negative. This indicates that the availability of more hospital beds can help to reduce the spread of the virus. While INFDOC is insignificant, the sign is positive. This may be due to the fact that the availability of more doctors per lakh population can enhance the detection of Covid-19 cases. As discussed above, we have used an alternate variable to represent the reverse migration of the labourers from Maharashtra using Census-2011 data in the second set of regression models viz. CMIG. The functional forms of these regression models are as follows— Y = β0 + β1 C M I G + β2 U R P O P + β3 PC I + ε7 Y = β0 + β1 C M I G + β2 U R P O P + β3 PC I + β4 I N F B E D + ε8

(10.8) (10.9)

Y = β0 + β1 C M I G + β2 U R P O P + β3 PC I + β4 I N F B E D + β5 I N F D OC + ε9 (10.10)

10 Interstate Migration and Spread of Covid-19 in Indian States

213

Table 10.1 Regression results using train migration data Var

Model 1

Model 2

Model 3

TMIG

0.13 (0.022)*

0.12 (0.022)* 0.13 (0.023)*

URPOP

34.06 (6.743)* 33.72 (6.950)*

1844.77 (966.57)***

INFBED

Model 6

0.14 (0.029)*

0.14 (0.030)*

0.15 (0.030)*

0.16 (0.054)*

0.15 (0.058)**

0.16 (0.056)**

1916.70 (973.93)***

1876.04 3257.97 3304.58 3123.08 (1078.589)*** (1047.556)* (1074.51)** (966.629)**

−1.25 (1.723)

−1.29 (1.699)

INFDOC Cons

Model 5

33.86 (8.621)*

POPDEN PCI

Model 4

−0.67 (1.69)

0.18 (3.695)

−0.85 (1.721) 0.89 (3.708)

−8847.08 −9095.68 −9466.29 −15,096.79 −15,272.98 −14,424.15 (4694.193)*** (4748.61)*** (1034.730)*** (5117.604)* (5259.95)* (4657.31)*

R-squared 0.7662

0.7692

0.7692

0.7601

0.7609

0.7614

Note: *, **, and *** represents significance at 1, 5 and 10% level respectively

Y = β0 + β1 C M I G + β2 P O P D E N + β3 PC I + ε10

(10.11)

Y = β0 + β1 C M I G + β2 P O P D E N + β3 PC I + β4 I N F B E D + ε11 (10.12) Y = β0 + β1 C M I G + β2 P O P D E N + β3 PC I + β4 I N F B E D + β5 I N F D OC + ε12

(10.13) The results of these regression models have been presented in Table 10.2. In these models as well the independent variables are the same as in the first set of regression models. Similar to the first set, we have used URPOP in the first three regression models and POPDEN in the final three models due to the possibility of a high correlation between these two variables. The results of the second set of regression models are very similar to the first set of the models. In all the regression models, the main variable of concern (CMIG) has been found to be positive and significant. In model 7, this indicates that an increase in one more migrant per lakh population from an Indian state to Maharashtra is expected to increase 1.90 more number of Covid-19 cases in an average Indian state. Similarly, while the coefficient of URPOP, POPDEN, and PCI are found to be positive and significant, INFDOC is also positive but insignificant. The only difference we found in the second set of regression models from the first set of models is the coefficient of INFBED. Here, the coefficient is negative and significant in all the models. This suggests that an increase in the number of hospital beds per lakh population may reduce the spread of the virus.

214

D. Jha et al.

Table 10.2 Regression results using Census-2011 migration data Var

Model 7

Model 8

Model 9

Model 10

Model 11

Model 12

CMIG

1.90 (0.832)**

2.17 (0.603)**

2.40 (0.547)*

2.07 (0.878)*

2.32 (0.673)*

2.62 (0.520)*

URPOP

29.43 (7.961)*

28.99 (7.56)*

29.94 (8.320)** 0.11 (0.052)**

0.11 (0.51)**

0.12 (0.044)**

2308.256 1033.70 (793.747)** (3.92)*

3985.91 (970.63)*

3603.76 (688.249)*

−5.51 (1.636)*

−4.66 (1.449)*

−5.56 (1.738)*

POPDEN PCI

2721.23 2638.626 (1044.891)** (850.038)* −4.78 (1.379)*

INFBED INFDOC Cons

1.68 (3.211) −13,581.79 (5110.27)**

R-squared 0.7097

2.11 (2.963)

−12,841.97 −11,347. −19,569.33 −18,916.43 −17,181.97 (4116.961)* 15 (5074.30)* (4694.912)* (3425.08)* (3795.941)* 0.7548

0.7563

0.6985

0.7410

0.7433

Note: *, **, and *** represents significance at 1, 5 and 10% level respectively

Conclusion and Policy Recommendations One of the major problems that India faced in recent times, which emerged after the imposition of initial lockdown in March 2020 to stop the spread of the Covid19 virus, is the reverse migration of labourers and the associated anxiety of the spread of the virus in the origin states of the migrants from the return migrants. The issue of reverse migration is so complex that most of the policies undertaken by the government have failed to produce any fruitful outcome. While it has been perceived that reverse migration of the labourers is associated with the spread of the virus, academic research to confirm this hypothesis is still limited. We identified that one of the possible reasons may be the unavailability of data on the number of reverse migrants in different states. Accordingly, in this study, we tried to construct a dataset on reverse migration of the interstate migrants in India from the information available on the Covid-19 special trains using a complex network approach. Then we applied an econometric exercise to understand the possible relationship between the confirmed Covid-19 cases in different Indian states and the reverse migration data that we have constructed during the second wave of the pandemic. In order to check the robustness of the results we also carried out a similar econometric exercise using the Census-2011 migration data. The results derived from the econometric models identified a positive and significant relationship between the reverse migration from Maharashtra and the Covid-19 cases in different Indian states during the second wave of the pandemic. Moreover, we found that while population density and the fraction of urban population are also

10 Interstate Migration and Spread of Covid-19 in Indian States

215

positively related to the Covid-19 cases in the states, the availability of hospital beds has a negative relationship with the number of Covid-19 cases in the Indian states. Therefore, to tackle the crisis and the spread of the virus, the government should focus on the problems of reverse migrants and increasing the health infrastructures like hospital beds, isolation centres, availability of ventilators, etc. Acknowledgements The authors are grateful to Naresh Singh, Ramaswamy Sudarshan, Sudip Patra, Thomas Seligman for their critical inputs and discussions. DJ and AC thank JGU for facilitating them to present an earlier version of this work in ICPP5 - BARCELONA 2021 (https://www. ippapublicpolicy.org/conference/icpp5-barcelona-2021/13).

References Abi-Habib, M., Yasir, S.: India’s Coronavirus Lockdown Leaves Vast Numbers Stranded and Hungry (Published 2020) (2021). Retrieved 5 July 2021, from https://www.nytimes.com/2020/03/29/ world/asia/coronavirus-india-migrants.html Ashfan, Y.: Threat of second wave of COVID-19 still exists: experts (2021). Retrieved 5 July 2021, from https://www.thehindu.com/news/national/karnataka/threat-of-second-wave-of-Covid-19still-exists-experts/article33805526.ece Bethune, Z.A., Korinek, A.: Covid-19 infection externalities: trading off lives vs. livelihoods. Technical report, National Bureau of Economic Research (2020) Behera, M., Mishra, S., Behera, A.R.: The COVID-19-led reverse migration on labour supply in rural economy: challenges, opportunities and road ahead in Odisha. Ind. Econ. J. 00194662211013216 (2021) Carter, D.: Hustle and brand: the sociotechnical shaping of influence. Soc. Med. Soc. 2(3), 2056305116666305 (2016) Census of India (2011): https://censusindia.gov.in/census.website/ Covid 19 crowdsourced data. Retrieved 5 July 2021, from https://www.Covid19india.org/ Check Train Coach Position. Retrieved 5 July 2021, from https://www.trainman.in/coach-position COVID-19 lockdown leaves Kota’s coaching centres deserted; shift to online classes offers only hope-India News, Firstpost. (2021). Retrieved 5 July 2021, from https://www.firstpost.com/ india/Covid-19-lockdown-leaves-kotas-coaching-centres-deserted-shift-to-online-classes-off ers-only-hope-8664341.html Das, K.C., Saha, S.: Inter-state migration and regional disparities in India. online http://iussp. org/sites/default/files/event_call_for_papers/Inter-state%20migration_IUSSP13.pdf (2013). Accessed 15 March 2015 Economic Survey (2017): https://www.indiabudget.gov.in/economicsurvey/ Estrada, M.A.R.: The Difference between The Worldwide Pandemic Economic Crisis (COVID-19) And The Global Financial Crisis (Year 2008) (2020) Empowering Migrant Workers through Skill Development and Livelihood Generation (2020). Retrieved 5 July 2021, from http://niti.gov.in/sites/default/files/2020-11/MigrantsWorker_Ins ide_10.pdf Gupta, N., Tomar, A., Kumar, V.: The effect of COVID-19 lockdown on the air environment in India (2021). Retrieved 5 July 2021, from https://doi.org/10.22034/GJESM.2019.06.SI.04 Ginsburg, C., Collinson, M.A., Gómez-Olivé, F.X., Gross, M., Harawa, S., Lurie, M.N., White, M.J.: Internal migration and health in South Africa: determinants of healthcare utilisation in a young adult cohort. BMC Public Health 21(1), 1–15 (2021) Kumar, R., Debroy, B., Ghosh, J., Mahajan, V., Prabhu, K.S.: Global Financial Crisis: Impact on India’s Poor Some Initial Perspectives (2009). Retrieved October, 15, 2018

216

D. Jha et al.

Lee, E. S.: “A theory of migration.” Demogr. 3(1), 47–57 (1966). https://doi.org/10.2307/2060063 Massey, D.S., Arango, J., Hugo, G., Kouaouci, A., Pellegrino, A., Taylor, J.E.: Theories of international migration: a review and appraisal. Popul. Dev. Rev. 431–466 (1993) Mesnard, A., Seabright, P.: Escaping epidemics through migration? quarantine measures under incomplete information about infection risk. J. Public Econ. 93(7–8), 931–938 (2009) Mehrotra, K.: Explained: What is NITI Aayog’s draft national policy on migrant workers? (2021a). Retrieved 5 July 2021a, from https://indianexpress.com/article/explained/niti-aayog-migrantworkers-policy-Covid-lockdown-7201753/ Mehrotra, K.: Enable vote, map skill, inter-state teams: First migrant policy draft (2021b). Retrieved 5 July 2021b, from https://indianexpress.com/article/india/migrant-worker-first-draft-policyniti-aayog-labour-ministry-Covid-19-7197599/ Mitra, R., Rawat, C., Varshney, G.: Return migration and COVID-19: Data suggests Kerala, TN, Punjab, UP, Bihar may be future red zones for contagion risk. FirstPost (2020) Niti Aayog. (2020). https://www.niti.gov.in/ Nornha, G.: India’s Covid-19 resurgence unlikely to result in second wave, no impact on medium term outlook: Nomura. Retrieved 5 July 2021, from India’s Covid-19 resurgence unlikely to result in second wave, no impact on medium term outlook: Nomura—The Economic Times (indiatimes.com) (2021) Padhee, A.K., Kar, B.K., Choudhury, P.R.: The Lockdown Revealed the Extent of Poverty and Misery Faced by Migrant Workers (2021). Retrieved 5 July 2021, from https://thewire.in/lab our/Covid-19-poverty-migrant-workers Ray, D., Subramanian, S.: India’s lockdown: an interim report. Ind. Econ. Rev. 55(1), 31–79 (2020) Rajan, S.I., Sivakumar, P., Srinivasan, A.: The COVID-19 pandemic and internal labour migration in India: a ‘crisis of mobility.’ Ind. J. Labour Econ. 63(4), 1021–1039 (2020) Rajan, S.I., Bhagat, R.B.: Internal Migration in India: integrating migration with development and urbanization policies | KNOMAD (2021). Retrieved 5 July 2021, from https://www.knomad.org/publication/internal-migration-india-integrating-migration-dev elopment-and-urbanization-policies Sadekar, O., Budamagunta, M., Sreejith, G. J., Jain, S., Santhanam, M.S.: An infectious diseases hazard map for India based on mobility and transportation networks (2021). arXiv preprint arXiv:2105.15123 Rani, S.: Analysing lee’s hypotheses of migration in the context of Malabar migration: a case study of Taliparamba block, Kannur District. Themat. J. Geogr. 1(1) (2016) Wang, Z., Song, K., Hu, L.: China’s largest scale ecological migration in the three-river headwater region. Ambio 39(5), 443–446 (2010) World Bank (2020): https://www.worldbank.org/en/publication/wdr2020/brief/world-develo pment-report-2020-data 2011 Census data. (2021). Retrieved 5 July 2021, from https://censusindia.gov.in/2011census/mig ration.html/ 1,500 buses deployed to take migrant workers to their native districts: Ghaziabad DM. Retrieved 5 July 2021, from https://economictimes.indiatimes.com/news/politics-and-nation/1500-busesdeployed-to-take-migrant-workers-to-their-native-districts-ghaziabad-dm/articleshow/748749 85.cms

Chapter 11

Trade Intervention Under the Belt and Road Initiative with Asian Economies Sunetra Ghatak and Sayantan Roy

Abstract International trade and association are the key contributors to economic integration and cohesion. The Belt and Road Initiative (BRI) is an auspicious development initiation taken by the Chinese government to endorse the logistics and infrastructure expansion under the One Belt One Road (OBOR) so that economic cohesion can be achieved between China and its trading partners. After half a decade of the BRI initiative, arguments have been elevated whether this remained profitable for the connected partner countries in the long run. Against this backdrop, this paper tries to estimate the trade performance of participating and connecting country’s using the traditional gravity model along with neural network model. The paper is set to examine whether the BRI is certainly a worthwhile initiative for Asian economics by associating the relative estimation powers of the two mentioned techniques using bilateral trade flow from 1990 to 2019 of 32 Asian partner countries. Keywords Trade · BRI · Gravity model · Neural network analysis JEL codes F43 · O24 · C23 · C63

Introduction International trade and association are the key contributors to economic integration and cohesion. The key to understanding the economic growth and cohesion between China and its neighbouring countries in Asia lies in the endorsement of logistics and infrastructure development. Efficient and improved infrastructure is expected to S. Ghatak (B) Jindal School of Government and Public Policy, O. P. Jindal Global University, Sonipat, Haryana, India e-mail: [email protected] S. Roy School of Engineering, Jawaharlal Nehru University, New Delhi-110067, India © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 A. Chakraborti et al. (eds.), Quantum Decision Theory and Complexity Modelling in Economics and Public Policy, New Economic Windows, https://doi.org/10.1007/978-3-031-38833-0_11

217

218

S. Ghatak and S. Roy

connect economies of Asia which are expected to minimise transportation costs and further increases the overall competitiveness of the system (Ghosh and De 2005; De and Iyengar 2014; and De et al. 2019; Ghatak and De 2021). The Belt and Road Initiative (BRI) is an auspicious development initiative promised to connect China with its Asian trading partners by endorsing logistics and infrastructure development so that economic cohesion can be achieved in long run. This initiative taken by the Chinese government to improve connectivity commenced in 2013 with the idea of constructing the One Belt One Road (OBOR). The initiative is anticipated to boost China’s foreign trade, especially with the countries connected with BRI through ports and infrastructure projects to strengthen policy coordination, financial cooperation, facilities connectivity, trade facilitation, and bond across people. This project started with the vision to connect with 65 countries accounting for 30 percent of the world’s GDP, 62 percent of the world’s population, as well as 75 percent of energy reserves by strengthening relations in terms of trade activity, investment climates, and infrastructure developments with China (Huang 2016; Zhai 2018). The BRI mainly includes the Silk Road Economic Belt which connects China with Asia and further Europe. Apart from these links, six other economic corridors are to be operated to enhance the trade and economic activities of connected countries by making many belts and roads in near future. It can be expected that the development and completion of the BRI project can encourage the economic environment of the entire regions by creating regional cohesion on transport infrastructure reforms. It further significantly might reduce trade cost, improve connectivity, increase investment as well as growth, and of course would contribute to the bilateral trade relationship. In the past decade, bilateral trade flows have shown an increasing tendency across connected countries through the BRI. Various empirical studies have examined the impact of BRI on trade and growth using by the traditional gravity model with their economic scales and distance (Anderson, et al. 2003; Herrero, et al. 2017; Beverelli et al. 2018; Lu et al. 2018). The results from these studies supported the idea that infrastructure development accelerates trade across connected regions and so happened for the BRI member countries over time (Anderson et al. 2003; Baier et al. 2009; Lu et al. 2018). It is widely acknowledged in the existing literature that the panel gravity model has appropriateness of illustrating the individual effect of the independent variable on dependent variable (i.e., bilateral trade here) with their significance level. Alternatively, Artificial Neural Networks (ANNs) has been introduced in the field of economic analysis to understand complex non-linear relations between the host country with the independent partner countries by considering the dichotomous characteristics in the trade scenario. Few recent studies have experimented with countries’ trade interactions using the neural network model along with the traditional gravity model analysis and reported that the former has a high degree of accuracy in forecast as compared to the later (Tillema et al. 2006; Pourebrahim et al. 2018). It is important here to mention that all these literatures looked at the contribution of BRI on either European countries or African countries using only gravity model framework (Anderson et al. 2003; Baier et al. 2009; Herrero et al. 2017; Beverelli et al. 2018; Lu et al. 2018). Few of them compared the impact of BRI for European countries or African countries using the gravity model with the

11 Trade Intervention Under the Belt and Road Initiative with Asian …

219

neural network model in an extreme starter endeavour manner (Tillema et al. 2006; Pourebrahim et al. 2018). Overall literature supported that BRI has a role on influencing bilateral trade interactions as it is committed to improve and connect with many belts and corridors. Over time arguments have been elevated whether this project will be profitable to Asian participating countries as well like European or African countries in the long run as the project has spent half a decade after its initiation. As mentioned above that prior studies have not taken Asian member countries and also there were no studies deals with both the gravity model and neural network model to estimate the contribution of BRI on China with its Asian partners. Therefore, assessing Asian partner country’s economic and trade performance with China under BRI network, and estimating the extent to which it is poised for growing economy is a highly important exercise to do. Motivated by the above, this paper is set to examine the contribution of the BRI on bilateral trade flows across Asian economies using the traditional gravity model estimations along with the neural network model estimations under the frame of the One Belt One Road (OBOR) project. The results of both estimation methods are being compared with actual trade data to understand the relative estimation power on the basis of the prediction value. The paper specifically would try to answer whether BRI is certainly a worthwhile initiative for Asian partner countries by comparing the relative estimation powers using bilateral trade flows from 1990 to 2019 of 32 Asian countries. A set of explanatory variables indicating the role of economic, geographic, and regional trade arrangements such as GDP, infrastructure, the distance between the importer and exporter are included to serve the model of our analysis. The rest of the paper is organised as follows. Section 11.2 introduces the data and the methodology used in the analysis. Findings from the empirical analysis are reported in Sect. 11.3 and conclusions are in Sect. 11.4.

Data and Methodology To estimate the bilateral trade between china with its Asian partners under the light of the Belts and Roads Initiative (BRI) or One Belt One Road (OBOR) the traditional gravity model framework has been employed. Along with the gravity model, the paper also relied on the neural network model to compare the estimates with actual trade data to understand accuracy given by the method more closely able to match the real scenario. To understand the contribution of BRI, the model of this paper is to estimate China’s bilateral exports between its OBOR trading partner countries. This section introduces the gravity model followed by the neural network model used for the analysis along with the description of the dataset.

220

S. Ghatak and S. Roy

Gravity Model The paper follows the gravity model estimation method with their economic scales and distance to estimate the volume of bilateral trade flows between the importer and exporter countries (following Pöyhönen 1963; Leibenstein 1966). The traditional gravity model of trade between country i and j represents as: Ti j = α

G D P iθ 1 × G D P θj 2 Diθj

(11.1)

where Ti j is the volume of bilateral trade country i and j; G D P i and G D P j are the GDP of the trade country i and j; Di j is the distance between the country i and j. α, θ1 , θ2 , θ3 are the parameters to be estimated in the model. With the economic scales and distance between trading countries, now some dummy variable factors to be introduced to determine whether trade accomplices on a contiguity, a common linguistic, a colonial connection, and a regional trade arrangement. Therefore, the gravity model specification appears as Ordinary least Square (OLS) as follows: I n X i j,t = θ0 + θ1 I nG D Pi,t + θ2 I nG D P j,t + θ3 I n Disti, j + θ4 Contigi, j + θ5 Comlangi, j + θ6 Coli, j + θ7 I n I n f rai + θ8 I n I n f ra j + θ9 O B O Ri, j + θ10 AS E ANi, j + θ11 S A A RCi, j + θ12 SC Oi, j + εi jt (11.2) where X i j,t is the bilateral export between country i and country j at the time t, G D P i,t and G D P j,t the GDP of country i and partner country j at the time t, Dist i j stands for the distance between country i and country j at the time t, and dummy variables, capture a contiguity (Contig i j ), a common linguistic (Comlang i j ), a common colonial relationship (Col i j ) between country i and country j. Here infrastructure index has been introduced in Eq. (2) of the trading partner countries (I n f ra i , I n f ra j ), and regional trade agreements (O B O R i j , AS E AN i j , S A A RC i j , SC O i j ). εi jt is an error term at the year t for pair of i-th and j-th country. Now inclusion of country fixed effects, the above equation will be as follows. ln X i j,t = θ0 + θ1lnG D P i,t + θ2 lnG D P j,t +θ 3ln Dist i j + θ4 Contig i j + θ5 Comlang i j + θ6 Col i j + θ7ln I n f ra i + θ8ln I n f ra j + θ9 O B O R i j + θ10 AS E AN i j + θ11 S A A RC i j + θ12 SC O i j + μi + α j + εi jt

(11.3)

where μi + α j is exporter and importer fixed effects respectively. Alternatively, with the country and year fixed effect the above equation will be as follows.

11 Trade Intervention Under the Belt and Road Initiative with Asian …

221

ln X i j,t = θ0 + θ1lnG D P i,t + θ2 lnG D P j,t +θ 3ln Dist i j + θ4 Contig i j + θ5 Comlang i j + θ6 Col i j + θ7ln I n f ra i + θ8ln I n f ra j + θ9 O B O R i j + θ10 AS E AN i j + θ11 S A A RC i j + θ12 SC O i j + μit + α jt + εi jt

(11.4)

where μit +α jt represents exporter-time and importer-time fixed effects respectively. This is the gravity framework for this paper with the outsized assortment of variables. In Eq. (4), including the importer and exporter GDP economic variables with the country-year dummies along with infrastructure and regional trade agreements variables. The trade flows to be measured into the logarithmic frame. Other variables like, GDP, distance, infrastructures are also measured by taking the logarithm of each indicator. To estimate the bilateral trade the OLS estimation procedure has been employed and besides, introduced the use of Poisson Pseudo Maximum Likelihood (PPML).1 This framework is built with the assumption that the degree of exports influences countries’ domestic trade, and provides greater trade volume calculation that would be useful to policymakers in near future.

Neural Network Model Neural Networks (NN) are computational models inspired by the biological neuron of humans, where each neuron is connected to many others to process and infer information. Neural Networks—being able to learn non-linear complex relationships have been used exhaustively in forecasting, natural language processing, medical diagnosis, etc. in different disciplines with successful widespread applications. Evidence suggests that mainly why neural networks perform well is because basic assumptions like linearity are restricted, although it performs well in linear models as well—as has been applied in other fields. Fully connected layers are one of the basic but powerful types of neural networks which learn these complex relationships in large datasets. This consists of a fundamental computational unit called, neuron or node. This node receives input from all input features available and produces an output after making a weighted sum and passing through an activation function. These weights (w) corresponding to each node are the ones calculated from the NN model. the equation to each node after passing through activation f looks like: y = f (w1 X 1 + w2 X 2 + b)

(11.5)

Here, b is the bias term calculated inside the NN model taken as a weight to unit input. Function f is an activation function, used to introduce non-linearity into the 1

It allows heteroscedasticity in the data by providing unbiased and consistent estimates even if there is large proportion of zero trade values.

222

S. Ghatak and S. Roy

output—that will be passed to further neurons forward into layers. This will help us map real-life non-linear relationships. They can be of types: 1) ReLU (x) = max(0, x) 2) Sigmoid(x) =

1 1 + ex p(−x)

3) tanh(x) = 2 × sigmoid(2x) − 1 In this paper, we’ve made our NN model take input as the features as described in Table 11.1: descriptive statistics. In total, there was 1 input layer with 20 input nodes, 6 hidden layers with (32, 40, 60, 70, 40, 25) nodes corresponding to each layer, and a final output layer with 1 node. Every layer had ReLU as an activation function except output which was tanh. Regularizers—a technique to prevent overfitting so that the model can be improved further inaccuracy, was used in the second hidden layer. Both L1&L2 regularizers with values 1e−5 &1e−4 was used so any kind of loss will be penalized more making our model reach convergence faster. As our inputs get into the model while training it, it passes into each node and each layer—ultimately reaching our output node. Since the weights to each node are initiated randomly, output too will be random which then will be used with our original target variable ‘Log of bilateral trade between country i and j’ to calculate error—Mean Squared Error (MSE) in our case. After a particular batch from the dataset passes through the network, the MSE of this total batch will be calculated and an attempt would be made to reduce this error after each iteration or epoch. Optimization here will be done by stochastic gradient descent (sgd) algorithm, Adaptive Moment Estimation (adam) & backpropagation algorithm. Backpropagation calculates the gradient of the total error with respect to each node’s weights - which will be then used to update the randomly initiated weights using gradient descent to reduce error at the output layer. Adam uses the same method as sgd except it converges better using concepts of momentum and direction making it efficient for large numbers of sparse 0 features. All this is performed using Python and, with Keras and Tensorflow as libraries for ease. The network was trained for 600 epochs with a learning rate of 0.01 with batch sizes varying between 16 to 64.

Dataset The paper has used a panel of 32 Asian countries from 1990 to 2019 for the analysis of the bilateral trade flows between China and its Asian partner countries. Thus, the dataset of this paper consists of bilateral export flows of 32 × 31 country pairs. In

11 Trade Intervention Under the Belt and Road Initiative with Asian …

223

Table 11.1 Descriptive statistics Variable

Obs Mean

Std. Dev

Min

Log of China’s (i) bilateral exports to partner country ( j)

930

8.877358

1.303214

4.481199 11.18078

Max

Log of China’s (i) GDP per capita

930

3.714672

0.322017

3.153478 4.207275

Log of partner country’s ( j) GDP per capita

930

3.922958

0.4623914 2.783516 4.990094

Log of bilateral distance between China 930 and partner country

3.587802

0.2241806 2.978637 4.047158

China (i)/ partner country ( j) contiguity 930 indicator

0.3870968 0.4873482 0

1

China (i)/ partner country ( j) language indicator

930

0.4827957 0.4999728 0

1

China (i)/ partner country ( j) colonial relationship indicator

930

0.3870968 0.4873482 0

1

3.661104

Economic indicators

Trade indicators

Development indicators Log of infrastructure of China (i)

930

3.366299

0.1549746 3.11

Log of infrastructure of partner country ( j)

930

2.843331

0.5504047 1.862039 4.190336

Regional trade agreement indicators OBOR member

930

0.6139785 0.4870976 0

1

ASEAN member

930

0.2903226 0.4541554 0

1

SAARC member

930

0.1935484 0.3952916 0

1

SCO member

930

0.4193548 0.493719

1

0

Source: Authors’ calculation

the Asian sub-regions, the paper has included South Asian2 countries; East Asian3 countries; Southeast Asian4 countries; Central Asia countries5 ; and Pacific6 countries (see Appendix 11.5). The bilateral exports trade volume has taken from the UNComtrade database, Gross Domestic Product (GDP) has taken from World Development Indicators (WDI). Data of distance between the capitals of China and its trading partner country are taken from the Centre d’Etudes Prospectives et d’Informations Internationales (CEPII) distance database. Data on contiguity (i.e., common border), common linguistic (i.e., use a common official language), and colony (i.e., a common 2

Bangladesh, Bhutan, India, Nepal, Pakistan, Sri Lanka. China, Japan, Korea, Mongolia. 4 Brunei, Indonesia, Lao PDR, Malaysia, Myanmar, Philippines, Singapore, Thailand, Vietnam. 5 Armenia, Azerbaijan, Georgia, Iran, Kazakhstan, Kyrgyz Republic, Tajikistan, Turkey, Turkmenistan, Uzbekistan. 6 Australia, Fiji, New Zealand. 3

224

S. Ghatak and S. Roy

colonial relationship) are taken as dummies in this paper are obtained from CEPII. We have created the infrastructure index using three variables proxying the infrastructure based on total road networks, total railway networks, and the number of telephones mainlines at per capita availability. These infrastructure-related variables are taken from International Road Federation (IRF), World Road Statistics, and WDI (see Appendix 11.4). Finally, we have four regional trade agreement dummies for Asian economics which are namely, One Belt One Road (OBOR), Association of Southeast Asian Nations (ASEAN), South Asian Association for Regional Cooperation (SAARC), and Shanghai Cooperation Organisation (SCO).

Results and Analysis The analysis is done to understand the contribution of BRI on Asian economics’ trade intervention. The analysis relied on the neural network model against the gravity model with the objective to compare the accuracy in predicting trade intervention under BRI. Before doing the analysis, several data cleaning techniques have been applied to avoid deviations in data and probable obstacles in the prediction procedure. First of all, there is a removal of all data points that have missing/no trade values as they are economically insignificant and can cause outlier problems. Then we have summarised the descriptive statistics of each variable used in the study and have presented in the following Table 11.1. The mean value of China’s bilateral exports to Asian partner countries is 8.87 percent with a standard deviation of 1.30 percent. China is an active trading partner with Asian countries with the mean value of exporter’s GDP 3.71 percent and importer’s GDP 3.92 percent with their respective standard deviation 0.32 percent and 0.46 percent. This shows the resilient growth opportunities for the participating countries of Asia under BRI with China as the GDP figures are close to each other and hence connecting regions expect to reduce trade cost, increase investment, economic growth, and of course, would contribute to the bilateral trade relationship near future. The mean value of geographical distance between China and Asian trading partners included in this study is 3.587802 km with a standard deviation 0.2241806 km. The statistics of the infrastructure index of the host country (i) and the partner country ( j) show the mean value of 3.366299 ton-km and 2.843331 ton-km with a standard deviation of 0.1549746 ton-km and 0.5504047 ton-km respectively. Of the remaining common language and common border variables, the mean value of common language (0.48 percent) is higher than the mean value of common border (0.39 percent) depicts that a common official linguistic play a crucial role in BRI trading partners. The average performance of the colonial relationship included countries in this paper amounted to 0.39 percent with a standard deviation of 0.49 percent. From this one can expect that culture and the same colonial structure could definitely be correlated with an expansion in trade flows under BRI trading partners. Finally, regional trade agreements dummy in the BRI trading partners recorded mean of 0.61 percent, 0.29 percent, 0.19 percent, and 0.42 percent with a standard deviation of 0.49 percent, 0.45 percent, 0.40 percent, and 0.49

11 Trade Intervention Under the Belt and Road Initiative with Asian …

225

percent respectively for OBOR, ASEAN, SAARC, and SCO member countries. The performance of regional trade agreements indicator recorded highest for OBOR partners (0.61 percent) keeping SAARC member countries to the lowest (0.19 percent). This motivates the study to proceed further and confirms that improve infrastructure strengthens trade and BRI boosts regional integration by improving the economic growth of participating countries. This section formally evaluates and presents the results-driven from the gravity model and neural network. To understand the impact of BRI on trade intervention as well as the estimation power of the gravity model and the neural network model, first the data from 1990 to 2019 has been employed using baseline model. The forecasting application has been done using the dataset from 1990 to 2014 through the both estimation procedure and predictions are calculated for next five years. The accuracy has been measured from the out-of-sample predictions using the root mean square error (RMSE). This calculates the square root of the average of squared errors between the predicted values and the actual values: RMSE =

N i=1





y i − yi N

2 (11.6)



where yi is the actual observation, and y i is the predicted observation. N represents the number of observations (Fig. 11.1). Table 11.2 presents the trade estimation results using different estimators using data from 1990 to 2019 of 32 Asian partner countries. Using Eq. (2), the baseline model of the OLS estimator has reported $9.96 billion out-of-sample RMSE, the PPML estimator stands $2.29 billion out-of-sample RMSE, and the neural network estimator has an out-of-sample RMSE of $19.61 billion. Using Eq. (3), the outof-sample RMSE stands $8.87 billion, $2.17 billion, and $19.60 billion for OLS estimator, PPML estimator, and neural network estimator respectively with country fixed effects. The whole dataset gives an out-of-sample RMSE of $7.66 billion, $1.96

Fig. 11.1 Mean squared error, root mean squared error, R2 plotted over 600 epochs. Source Authors’ calculation

226

S. Ghatak and S. Roy

Table 11.2 Trade estimates (in billion US$) Models

Gravity models

Neural networks

OLS

PPML

NNA (Log of Trade)

Baseline Model, Eq. (2)

$9.96

$2.29

$19.61

Country-fixed effects, Eq. (3)

$8.87

$2.17

$19.60

Country-year fixed effect, Eq. (4)

$7.66

$1.96

$19.61

Adjustment and architecture

Dependent variable: T radei j + 1

Dependent variable: T radei j

Trade, distance, infrastructure, and GDP, Standardized (mean = 0, Std. Dev. = 1)

Source: Authors’ calculation

billion for the OLS estimator, PPML respectively, while the neural network prediction of an out-of-sample RMSE is $19.61 billion with country-year fixed effects using Eq. (4). These exercises have been done for several times7 and these outcomes are representative. In the second step, to compare the relative estimation powers of gravity model and neural network model forecasting application has been done for the from 1990 to 2014 with country effect. Now the prediction of bilateral trade values has been computed for all countries for the next five consecutive years. The predicted bilateral trade values between China and selected trading partner countries can be compared with the actual data set to judge and understand the estimation power of the two models. The results of the predictions with the actual trade values in billion US$ are reported in the following Table 11.3. The estimation covers different countries have been randomly reported in the following table from selected regional trading blocks to maintain the heterogeneity of the observations. The gap between the estimated value and the actual value is the indicator of the accuracy of the measurement technique and the power of estimation. For example, in the case of India and Armenia, the gravity model estimations are sensibly closer to the actual values from 2015 to 2019. The gravity model predicted a $55.62 billion trade flow for India in 2015 which is closer to the actual trade happened the value of $58.22 billion. Moreover, it has been observed that the neural network model shows a good prediction for Lao PDR and Tajikistan. For instance, the predicted $1.58 billion trade flow for Tajikistan in 2015 which is closer to the actual trade happened at the value of $1.79 billion and is better than the gravity prediction of $2.62 billion. Overall, one thing is important to notice that bilateral trade flows are showing increasing trend for partner countries.

7

To allow variation in the random training-test division and the development of the neural network.

11 Trade Intervention Under the Belt and Road Initiative with Asian …

227

Table 11.3 Trade estimation, Actual vs Prediction (in billion US$) Countries

Estimations

2015

2016

2017

2018

2019

India

Actual Gravity prediction Neural network Prediction

58.22 55.62 21.89

58.39 57.05 39.81

68.04 58.53 48.51

76.88 60.14 51.76

74.92 61.59 52.11

Armenia

Actual Gravity prediction Neural network Prediction

0.11 0.12 0.60

0.11 0.12 0.60

0.14 0.12 0.60

0.21 0.13 0.60

0.22 0.13 0.75

Lao PDR

Actual Gravity prediction Neural network Prediction

1.22 1.87 1.54

0.98 1.90 1.02

1.41 1.93 1.14

1.45 1.95 1.31

1.78 1.98 1.14

Tajikistan

Actual Gravity prediction Neural network Prediction

1.79 2.62 1.58

1.72 2.77 1.31

1.30 2.94 1.00

1.42 3.11 1.54

1.61 3.29 2.45

Source Authors’ calculation

Conclusions The paper is set with the objective to understand the contribution of the BRI which has crossed half a decade after it initiated. Existing literature has recorded that bilateral trade flows showed a tendency to increase under BRI for European and African countries with China. Now arguments have been in surfaced as to whether this initiative will be economically viable to Asian economies as well in the long run. Earlier literature focused on BRI contribution either looking at European or African countries and also there were no studies deals with the neural network model against the gravity model to estimate the trade contribution of BRI for Asian economies. To fill this gap in the literature, this paper has tried to answer whether BRI would encourage trade intervention across Asian economies with China. The analysis also has helped to understand the relative estimation power of two models namely, the traditional gravity model with the neural network model by using a panel of 32 Asian countries from 1990 to 2019 though the prediction procedure. The analysis is solely based on the relative predictive powers of the gravity models against neural the network analysis. The trade prediction estimations have been tested with OLS and PPML estimation procedures under the gravity model framework. The gravity model framework has been repeatedly tested several times by estimating the baseline model, including country-fixed effects and country-time fixed effects, etc. Secondly, the neural network model has been estimated allowing for variation in the random training-test division and developed neural network and outcomes. Further forecasting exercises have been done for some selected countries and a comparison has been done with the actual trade data to approximation the estimation power of the used techniques.

228

S. Ghatak and S. Roy

Findings from the analysis suggest that bilateral trade flows have increasing trend for connecting Asian countries. Further, both the models have strong estimation power depending upon the trade volume and country size. In the case of India and Armenia, the gravity model has shown better estimation by predicting the closer trade value with the actual trade. Moreover, the neural network model has shown a good prediction for Lao PDR and Tajikistan for the period 2015 to 2019. Therefore, it is recommended to use both the model and compare the results to get robustness in the result. These results can deliver impetus for all partner countries under a common trade agreement to measure the effects of this trade and economic collaboration. The estimation of trade using two different estimation techniques offers one direction of many through which trading countries can measure the impact of any particular newly formed agreement and policy can be made by evaluating the possible sustainability of such partnership. Acknowledgements The authors are grateful to the Editors of the book for their inputs.

Appendix See Tables 11.4, 11.5.

Table 11.4 List and description of the variables Variable

Proxy

Definition

Source

ln X i j,t

Exports from China (i) to the partner country ( j) in US$

Logarithm of china’s (i) bilateral exports to partner country ( j) at year t

UN-Comtrade International Trade Statistics database

lnG D P i,t

GDP per capita, PPP (constant 2017 international $)

Logarithm of china’s (i) GDP per capita at year t

World Development Indicators (WDI)

lnG D P j,t

GDP per capita, PPP (constant 2017 international $)

Logarithm of partner country’s ( j) GDP per capita at year t

WDI

ln Dist i j

Bilateral distance in kilometre

Logarithm of distance between China and partner country

Centre d’Etudes Prospectives et d’Informations Internationales (CEPII)

Contig i j

Contiguity

1 if China shares common border CEPII with the respective trading partner, 0 otherwise (continued)

11 Trade Intervention Under the Belt and Road Initiative with Asian …

229

Table 11.4 (continued) Variable

Proxy

Definition

Source

Comlang i j

Common language

1 if China has common official or primary language with the respective trading partner, 0 otherwise

CEPII

Col i j

Colonial relationship

1 if China has colonial relationship CEPII with the respective trading partner, 0 otherwise

I n f ra i

Infrastructure Index

Logarithm of infrastructure of China (i)

International Road Federation (IRF), World Road Statistics and WDI

I n f ra j

Infrastructure Index

Logarithm of infrastructure of partner country ( j)

IRF, World Road Statistics and WDI

O B O Ri j

One Belt One Road

1 if the country i is OBOR member, 0 otherwise

Authors’ estimation

AS E AN i j

Association of Southeast Asian Nations

1 if the country i is ASEAN member, 0 otherwise

Authors’ estimation

S A A RC i j

South Asian Association for Regional Cooperation

1 if the country i is SAARC member, 0 otherwise

Authors’ estimation

SC O i j

Shanghai Cooperation Organisation

1 if the country i is SCO member, 0 Authors’ estimation otherwise

Source Authors’ Compilation Table 11.5 List of country names in the sample

China

Korea, Rep

Singapore

Iran, Islamic Rep

Bangladesh

Mongolia

Thailand

Kazakhstan

Bhutan

Brunei Darussalam

Vietnam

New Zealand

India

Indonesia

Armenia

Kyrgyz Republic

Nepal

Lao PDR

Australia

Tajikistan

Pakistan

Malaysia

Azerbaijan

Turkey

Sri Lanka

Myanmar

Fiji

Turkmenistan

Japan

Philippines

Georgia

Uzbekistan

Source Authors’ Compilation

230

S. Ghatak and S. Roy

References Anderson, J., van Wincoop, E.: Gravity with gravitas: a solution to the border puzzle. Am. Econ. Rev. 93, 170–192 (2003) Baier, S.L., Bergstrand, J.H.: Bonus vetus OLS: A simple method for approximating international trade-cost effects using the gravity equation. J. Int. Econ. 77, 77–85 (2009) Beverelli, C., Keck, A., Larch, M., Yotov, Y.: Institutions, trade and development: a quantitative analysis; EconStor: Munich, Germany (2018) De, P., Iyengar, K.: Developing economic corridors in South Asia. Asian Development Bank, New Delhi (2014) De, P., Ghatak, S., Durairaj, K.: Assessing economic impacts of connectivity corridors in north east india—an empirical investigation. Econ. Polit. Wkly. 54(11) (2019) Ghatak, S., De, P.: Income convergence across asian economies: an empirical exploration. J. AsiaPac. Bus. 22(3), 182–200 (2021) Ghosh, B., De, P.: Investigating the linkage between infrastructure and regional development: Era of planning to globalisation. J. Asian Econ. 15(1), 1023–1050 (2005) Herrero, A.G., Xu, J.: China’s belt and road initiative: Can Europe expect trade gains? China World Econ. 25, 84–99 (2017) Huang, Y.: Understanding China’s Belt & Road initiative: motivation, framework and assessment. China Econ. Rev. 40, 314–321 (2016) Lu, H., Rohr, C., Hafner, M., Knack, A.: China belt and road initiative: measuring the impact of improving transport connectivity on trade in the Region—A Proof-Of-Concept Study; RAND: Cambridge, UK (2018) Pourebrahim, N., Sultana, S., Thill, J.C., Mohanty, S.: Enhancing trip distribution prediction with twitter data: comparison of neural network and gravity models, pp. 5–8. ACM, New York, NY, USA (2018) Pöyhönen, P.A.: Tentative Model for the Volume of Trade between Countries. Weltwirtsch. Arch. 90, 93–100 (1963) Tillema, F., Van Zuilekom, K.M., Van Maarseveen, M.F.: Comparison of neural networks and gravity models in trip distribution. Comput. Aided Civ. Infrastruct. Eng. 21, 104–119 (2006) Tinbergen, J.: Shaping the world economy: Suggestions for an international economic policy. Econ. J. 76, 92–95 (1966) Wohl, I.; Kennedy, J.: Neural network analysis of international trade; U.S. International Trade Commission: Washington, DC, USA (2018) Zhai, F.: China’s belt and road initiative: A preliminary quantitative assessment. J. Asian Econ. 55, 84–92 (2018)

Chapter 12

Innovation Diffusion with Intergroup Suppression: A Complexity Perspective Syed Shariq Husain, Joseph Whitmeyer, and Anirban Chakraborti

Abstract We present a new model for the diffusion of innovation. Here, the population is segmented into distinct groups. Adoption by a particular group of some cultural product may be inhibited both by large numbers of its own members already having adopted but also, in particular, by members of another group having adopted. Intergroup migration is also permitted. We determine the equilibrium points and carry out stability analysis for the model for a two-group population. We also simulate a discrete time version of the model. Lastly, we present data on tablet use in eight countries from 2012 to 2016 and show that the relationship between use in the “under 25” age group and “55+” age group conforms to the model.

S. S. Husain · A. Chakraborti (B) Centre for Complexity Economics, Applied Spirituality and Public Policy, Jindal School of Government and Public Policy, O. P. Jindal Global University, Sonipat Narela Road, Sonipat 131001, India e-mail: [email protected]; [email protected] S. S. Husain e-mail: [email protected] J. Whitmeyer Department of Sociology, University of North Carolina at Charlotte, Charlotte, NC 28223, USA e-mail: [email protected] A. Chakraborti School of Computational and Integrative Sciences, Jawaharlal Nehru University, New Delhi 110067, India Centro Internacional de Ciencias AC Avenida Universidad 1001, Campus UAEM-UNAM, 62210 Cuernavaca, Mexico School of Engineering and Technology, BML Munjal University, Sidhrawali, Gurugram 122413, Haryana, India © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 A. Chakraborti et al. (eds.), Quantum Decision Theory and Complexity Modelling in Economics and Public Policy, New Economic Windows, https://doi.org/10.1007/978-3-031-38833-0_12

231

232

S. S. Husain et al.

Introduction Complex systems is a multidisciplinary subject emerged from statistical physics. Its application extends through the fields of natural sciences to now even social sciences, such as sociology and linguistics Castellano et al. (2009). It deals with the behaviour of large group or heterogeneous populations having non local interaction between them. It brings modeling aspect in studies of social systems to capture insight and enrich the field of Sociophysics Sen and Chakrabarti (2014), Chakrabarti et al. (2006), Abergel et al. (2017). Modeling the diffusion of innovation, which can be technological or cultural, has an extensive history, over decades and across disciplines Coleman et al. (1964), Rogers (1962), Mahajan and Peterson (1985). One process absent from all innovation diffusion models is suppression by one group of adoption or use of a cultural item in another group. Yet, as we describe below, this process clearly occurs for some items and has been recognized in some theoretical and empirical work. We present, analyze, and preliminarily test here, therefore, such a model, adapted from the Bolker-Pacala model in population biology. People may adopt or abandon cultural items for a variety of reasons, including intrinsic value to them, identity signals to themselves or others, and social pressure. There also may be external coercion applied. It has long been noted in social science that adoption of some cultural item may depend on its use by others. This effect can be positive for many reasons Leibenstein (1950), Lieberson (2000) but also negative, in, for example, what Leibenstein called the snob effect (Leibenstein 1950). More recently, and more specifically relevant to the process we are newly including here, Berger and colleagues (2008, 2009) have pointed out that adoption of a cultural item by one group may induce members of another group to abandon it (Berger and Heath 2008; Berger and Le Mens 2009). Berger and Heath (2008; 2007) make a strong case for this for many cultural items, such as clothing brands and kind of automobile, explaining this effect with an identity motivation (Goswami and Karmeshu 2001, 2008; Berger and Heath 2008). In Freakonomics, Levitt and Dubner allege this phenomenon for first names, claiming that their California names data show that lower classes adopt names that the higher classes are using, but then that the higher classes abandon those names because the lower classes are now using them (Levitt and Dubner 2006). Note that in the above examples the negative effect is an internal phenomenon, in that the suppressing group is not trying to lower use and adoption by the other group. Historically, however, external suppression also has occurred. One clear example is sumptuary laws, for example, in the Middle Ages in Europe, wherein clothes of certain colors and materials were not permitted to people below a certain social status. Some barriers to entry, such as requiring men who join the cavalry to come with a horse or charging high fees at golf courses can be seen also as a higher socioeconomic class keeping certain cultural items at a low level in a lower socioeconomic class. In the case of external suppression, the reason for lower adoption and use will be different from when the phenomenon is internal; it will not be due to identity moti-

12 Innovation Diffusion with Intergroup Suppression . . .

233

vation, for example. For our purposes, however, constructing a model incorporating this negative effect, the mechanism is not important. As yet, models of the diffusion of innovation have not included such negative effects of use by one group on adoption and use by another group. Early models naturally were simplest, assuming a single homogenous population with a single mode of diffusion. These were made more complex, again, in various natural ways. Populations were made heterogeneous, sometimes by positing segmented populations, such that the process works differently in different segments, sometimes modeling a continuous distribution of the population in characteristics or adoption propensity. Different sources of diffusion were considered, for example, other people or media, and in some models, such as the Bass model (1969), different sources were combined. Bartholomew added a loss-of-interest mechanism to those in the Bass model and also presented a stochastic model, in contrast to the common deterministic models (Bartholomew 1976). Alternative assumptions were made concerning the underlying mechanism of diffusion, such as social contagion simply through exposure, social pressure or conformity, and social learning (Young 2009). Different channels of diffusion also have been examined, for example, with new attention to online diffusion (Goel et al. 2012). One impetus behind the proliferation of models was that not all data showed the same pattern. For example, the archetypical innovation diffusion pattern is the logistic curve, with an adoption rate peak somewhere in the middle of the diffusion process. But for some products, adoption was bimodal, with an early rate of adoption peak, then a lull with relatively few new adoptions, then another rate of adoption peak. This required new models (Goswami and Karmeshu 2001, 2008; Sharma and Karmeshu 2004). To state a general principle, different cultural products may differ in their underlying diffusion mechanisms, and so may have fundamentally different diffusion patterns and require different models. Our new model is presented in the spirit of this principle. Similar to previous models, it relies on something like contagion as part of the underlying mechanism. It also is a mean field model, i.e., works with variables aggregated above the individual level, and it divides the population into different segments. The crucial novelty of this model is that adoption of the product by one segment may have a negative effect on adoption by another segment. Adoption by one segment is also allowed to have a positive effect on adoption by another segment and excessive adoption within a segment may have a negative effect on further adoption within the segment. These three effects we call “external suppression,” “external stimulation,” and “internal suppression,” respectively. In the model, they are all mathematically second-order effects (Bessonov et al. 2013; Chakrabarti et al. 2018). We present as an example and test this model on tablet use from 2012 to 2016. In numerous countries, tablet use increases in the oldest age group (55+) and at first increases even faster in the youngest age group (under 25). It appears, however, that when the oldest age group use reaches a certain level, somewhere close to 30%, tablet use in the youngest age group starts to decrease. A plausible mechanism is that when use among older adults gets sufficiently high, the young begin to perceive tablets as an older person’s device, or at least not something that can differentiate them from older

234

S. S. Husain et al.

people. Consequently young people become less likely to adopt and some even stop using the device. This may well be motivated by identity considerations, consistent with Berger and Heath’s (2008, 2007) argument, but, again, it is not necessary to know the exact mechanism to model its population effect. For this process, we borrow an appropriate, existing model that has been adapted from the Bolker-Pacala model of population dynamics (Bolker and Pacala 1999; Bolker et al. 2003). We describe the model in the second section of the paper, summarizing the relevant theoretical finding. In the third section we present the empirical data on tablet use in the form of plots, along with plots of simulations that produce qualitatively similar patterns. The fourth section summarizes and draws conclusions.

The Model We introduce the following model of innovation diffusion, which we call the BP model of innovation diffusion because it is taken from a multi-group mean field approximation of the Bolker-Pacala model of population dynamics in biology (Bessonov et al. 2013, 2018). The population dynamics model posits an initial population of individuals living on a lattice, i.e., a multi-dimensional grid. The lattice can represent geographical space, its typical biological use, but it also can represent other spaces on which a population may be distributed. For example, it could be a one-dimensional space of age or a multidimensional space with dimensions of age, ethnicity, various socioeconomic status measures, and so forth (McPherson 1983). Each individual can give birth to another individual or die or migrate, all at certain rates. In addition to their intrinsic rates, the existence of individuals may be affected negatively (suppressed) by the presence of other individuals. A mean field treatment of this is mathematically tractable, and, in fact, is equivalent to a kind of random walk (Bessonov et al. 2013). In the multi-group version, the population is partitioned into . N different groups. Suppression can occur both within a group and across groups (Bessonov et al. 2018). Before exposition of the BP model, let us discuss why the mathematical models we present are useful, that is, why we might want to develop and analyze them instead of, for example, simply looking at a simulation of individuals making certain probabilistic choices. We present two versions of the BP model, one a stochastic version, equivalent to a random walk, and the other a system of differential equations giving a deterministic trajectory, together with other equations describing the fluctuations around that trajectory. We can begin by noting that the random walk is exactly equivalent to the simulation of individuals making probabilistic choices. Nevertheless, by casting it as a random walk we gain the ability to use the theoretical apparatus that has been developed for random walks, such as the conditions under which it approaches a steady state distribution and other outcomes that we do not develop here. Analyzing it as stochastic fluctuations around a deterministic solutions to a system of differential equations allows us to identify and classify equilibria, to precisely partition the parameter space with regard to equilibria, that is, the likely

12 Innovation Diffusion with Intergroup Suppression . . .

235

fate of the process, and even to note the possibility of interesting rare events such as a large fluctuation pushing the system from one equilibrium to another. To model innovation diffusion, the initial population consists of the initial adopters of the cultural product. Adoption of the product by a new person corresponds to birth and abandonment of the product corresponds to death. Suppression within and across groups can inhibit further adoptions or even reduce use of the product within a group. Migration corresponds to movement by an adopter from one group to another. The continuous time model may be presented as follows in Bessonov et al. (2018). Represent the number in each group . Q i,L , .i = 1, . . . , N , at time .t who have adopted the cultural product by .

n(t) = {n 1 (t), n 2 (t), . . . , n N (t)},

(12.1)

a continuous time random walk on .(Z+ ) N with rates obtained from, for .i, j = 1, 2, . . . , N .

n(t + dt|n(t)) ⎧ ei ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ −ei ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎨ e j − ei . = n(t) + ⎪ ⎪ 0 ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎩ other

(12.2) w. pr. βi n i (t)dt + o(dt ) N ∑ w. pr. μi n i (t)dt + niL(t) ai j n j (t)dt + o(dt 2 ) 2

j=1

w. pr. n i (t)qi j dt + o(dt 2 ), j /= i N ∑ w. pr. 1 − (βi + μi )n i (t)dt i=1 ∑ ∑ − L1 n i (t)n j (t)ai j dt + n i (t)qi j + o(dt 2 ) i, j

w. pr. o(dt 2 )

i, j

where .ei is the vector with .1 in the .ith position and .0 everywhere else. Let us define the variables and parameters. .βi is the adoption rate and .μi is the abandonment rate. The subscript means that they may vary by group. The multiplication of .β j by .n i fits the mechanism being contagion or exposure: it depends on the number who have adopted already. The multiplication of .μ j by .n i is because it is precisely those who have adopted a product who can abandon it subsequently. .qi j is the rate of migration from group .i to group . j. Whether this is possible depends on the nature of the groups. For example, if they are adjacent age groups, then a positive migration rate from the younger to the older group is inevitable, but the reverse rate must be 0. In contrast, if the groups are social classes, then movement between all social classes, which is likely, would be conveyed by all migration rates being positive. The parameter .ai j is the rate of supression of group . j by group .i, where .i and . j can be the same. Finally, . L is adoption capacity, a control for the total number that can adopt, in other words a scale parameter. Table 12.1 lists the model parameters together with their meanings.

236

S. S. Husain et al.

Table 12.1 Model variables and parameters Parameter

Meaning

.n i

Number of adoptees in group .i Adoption rate in group .i Abandonment rate in group .i Competition or self-limiting rate for group .i Rate of suppression of group . j by group .i Migration rate from group .i to group . j Adoption capacity

.βi .μi .aii .ai j .qi j .L

The Eq. (12.2) allows us to construct a system of differential equations, but it is convenient to normalize the number of adoptees by dividing by . L. We set . z i (t) :=

n i (t) , L

i = 1, . . . , N .

and define, for .i = 1, . . . , N ⎛ ⎞ ∑ ∑ ∑ . Fi (z(t)) = ⎝βi − μi − qi j ⎠ z i − aii z i2 − a ji z i z j + q ji z j . j/=i

j/=i

(12.3)

j/=i

Then, the normalized system of differential equations is .

dz(t) = F(z(t)). dt

(12.4)

An equilibrium for the system occurs precisely at the points where 0 = F(z),

.

(12.5)

with one solution being .z ≡ 0. This process has a functional Law of Large Numbers and functional Central Limit Theorem, that is, as . L → ∞ the process converges to a Gaussian diffusion (Bessonov et al. 2018; Kurtz 1971). What this means is that for reasonably large . L the process will be very close to the following. There is a central tendency that is a deterministic trajectory, given by the system of partial differential Eq. (12.4) together with an initial value .z0 (see Eq. (12.8) for the system for our two-group model). Because of the stochasticity of the process, however, the values at any given time .s will be normally distributed about that deterministic trajectory, with . N × N covariance matrix .G(z(t)) (Bessonov et al. 2018; Kurtz 1971),

12 Innovation Diffusion with Intergroup Suppression . . .

.G(z(t))

=

237

⎧ ∑ ∑ ∑ ⎨ G ii (z(t)) = (βi + μi + j/=i q ji )z i + a ji z i z j + q ji z j ⎩ G (z(t)) = −q z − q z ij ij i ji j

j

j/=i

i /= j

(12.6) This means that from time .s to .s + δ, for small time increment .δ, the covariance matrix will be .G(s) multiplied by .δ. For our two-group model, the diffusion is in two dimensions with a .2 × 2 covariance matrix. The normalized system (Eqs. 12.3–12.5) shows that the process possesses equilibria or steady states, that is, points where the deterministic trajectory remains constant (Bessonov et al. 2018). From the normalized system, we can find the equilibria and whether the equilibria are stable or unstable, that is, whether when close to an equilibrium the process will approach the equilibrium or not. In addition, for purposes of simulation, we need transition probabilities for discrete time, which are easily available from Eq. (12.2). Specifically, for . N groups, we can simulate the embedded discrete time random walk on .(Z+ ) N , denoted .{X n }∞ n=0 , associated with the continuous random walk (12.1). For .x = (x1 , . . . , x N ) ∈ Z+ N , set N ( N ∑ ∑ aii ) xi xi + βi + μi + .c(x) = qi j x i . L i, j=1,i/= j i=1 {X n } has transition probabilities, for .x, y ∈ (Z+ ) N , .x /= 0

.

⎧ ⎪ ⎪ βi xi 1 ⎨ μi xi + · . P(x, y) = qi j x i c(x) ⎪ ⎪ ⎩ 0

aii L

if y = x + ei , i = 1, . . . , N xi2 if y = x − ei , i = 1, . . . , N if y = x − ei + e j , i /= j otherwise

(12.7)

Recall that we use .ei ∈ Z N to denote the vector with .1 in the .ith position and .0 everywhere else. Bessonov et al. shows that a random walk with these transition probabilities is geometrically ergodic (Bessonov et al. 2018). That is, it is positive recurrent with exponential convergence to a stable distribution.

Model for 2 Groups We now focus on a model restricted to two groups. Figure 12.1 shows a diagram of the random walk for this process starting at a point where .n 1 members of group 1 have adopted and .n 2 members of group 2 have adopted. To match our empirical case and keep the calculations straightforward, we assume only one-way migration. That is, we assume .q12 = 0 but allow .q21 ≥ 0. We obtain the equilibria using Eqs. (12.3)–(12.5) and multiplying by . L to scale up to population values, or alternatively, by using the following

238

S. S. Husain et al.

Fig. 12.1 Innovation diffusion for two groups as random walk

( dn .

1

dt dn 2 dt

= v1 n(1) − = v2 n(2) −

a11 2 n (t) − L 1 a22 2 n (t) − L 2

a21 n (t)n 2 (t) + q21 n 2 (t) L 1 a12 n (t)n 2 (t) − q21 n 2 (t), L 1

(12.8)

where, to simplify notation, we use a “net rate of adoption” for each group by setting v := βi − μi , for all .i. Mathematically, up to four equilibrium points exist, which we label . E 1 , . E 2 , . E 3 , and . E 4 , but in fact four valid points are never realized. For some configurations of the parameters, only two real singularities exist; the other two are complex and therefore can be disregarded. For the remaining configuration of parameters, one equilibrium point has a negative value in one of its coordinates. As that is impossible when the coordinate represents the number of people who have adopted some cultural product, this equilibrium is not valid. Once the two or three singularities are identified, we carry out a stability analysis by evaluating the Jacobian of the system of differential equations at the different points and using the eigenvalues to classify the kind of singularity in the usual fashion (Logan 2017). We will not present an exhaustive description of the kinds of singularities that can exist; that is available in numerous textbooks. The most important for our purposes are the following. A stable proper node is a point that a trajectory approaches directly, a focus one that it approaches by spiraling around it. Unstable versions of these exist: instead of approaching the equilibrium the trajectory leaves it, in the same fashion. A saddle point is a singularity that a trajectory approaches in

. i

12 Innovation Diffusion with Intergroup Suppression . . .

239

one direction but leaves in a different direction (e.g., approaches from the South but recedes heading West); the result is the trajectory makes a more-or-less near pass by the singularity and leaves. As simple inspection of Eq. (12.8) shows, . E 1 := (0, 0) is an equilibrium point. The eigenvalues at .(0, 0) are .λ1 = v1 and .λ2 = v2 − q21 . Thus, if the net rate of adoption in group 2 is greater than the rate of migration from group 2 to group 1, that is, .v2 > q21 , which is by far the most likely scenario, then .(0, 0) is an unstable proper node; over time, trajectories go away from it. Should the migration rate be greater, .q21 > v2 , then, .(0, 0) will be a saddle point—still not a point of attractive stability. A second singularity always exists at . E 2 := ( va111L , 0). One eigenvalue is .λ1 = −v1 and the second is .λ2 = v2 − aa1211v1 − q21 . Here, if the net adoption rate in group 2, .v2 , is small enough, then, .λ2 < 0 and this singularity will be an asymptotically stable proper node. That is, trajectories will converge to this point; use of the cultural product in group 1 will die out. Two other possible equilibria exist. They are complicated, involving the complementary square roots of a quadratic equation. Setting .

.

R :=

/

2 − q21 v2 ) (a12 q21 − a21 q21 − a22 v1 + a21 v2 )2 − 4(a11 a22 − a12 a21 )(q21

E 3 and . E 4 are, respectively, observing the .∓ and .± () n1 = L .

n2 =

L a22

a21 q21 −a12 q21 +a22 v1 −a21 v2 ∓R 2(a11 a22 −a12 a21 )

)

(

v2 − q21 +

() ,

a12 a12 q21 −a21 q21 −a22 v1 +a21 v2 ±R 2(a11 a22 −a12 a21 )

() .

Simplification Let us simplify the situation by assuming only one-way suppression, namely that.a21 = 0. This corresponds to the empirical application in the next section. In this case, singularities . E 3 and . E 4 never can exist together. Either both are complex or one takes a negative value for one of its coordinates. In fact,. E 3 can never be positive in both of its coordinates, so . E 3 is not a viable equilibrium point. . E 4 can be a viable equilibrium point, however. If so, it can be either a stable spriral, or an asymptotically stable proper node. Either way, . E 2 has to be a saddle point and . E 1 has to be an unstable proper node. If neither . E 3 nor . E 4 are viable singularities, then . E 1 can be either an asymptotically stable proper node or a saddle point and . E 2 is an asymptotically stable proper node. Following Table 12.2 summarizes the three possible configurations of singularities, along with a simple necessary condition. The full conditions distinguishing the first and second singularity are complicated and so omitted from the Table 12.2.

240

S. S. Husain et al.

Table 12.2 Possible equilibrium states for innovation diffusion model Necessary . E1 . E2 . E3 condition .v2

> q21

.v2

> q21

.v2

≤ q21

Unstable proper node Unstable proper node

Saddle point

Not viable

Saddle point

Not viable

Saddle point

Asymptotically stable proper node

Not viable

. E4

Asymptotically stable focus Asymptotically stable proper node Not viable

Two Group Example with Stable Positive Equilibrium As an example of the innovation diffusion process, and to provide a comparison with the empirical data from the next section, we present the model outcomes for parameter settings chosen in the range in which there is a stable positive equiibrium. Specifically, we use as parameters the following values: initial values, .n 1 (0) = n 2 (0) = 10; scale, . L = 1000; adoption rate and drop rate for group 1, .β1 = 0.0003, .μ1 = 0.0001, for group 2, .β2 = 0.0006, .μ2 = 0.0001; internal suppression, .a11 = 0.0002, .a22 = 0.0001; suppression (inhibition) of group 2 by group 1, .a12 = 0.0003; migration from group 2 to group 1, .q21 = 0.00005. This approaches an Ornstein-Uhlenbeck process. In other words, there will be stochastic fluctuations about the mean trajectory given by the system of differential equations in Eq. (12.8), and once the trajectory nears the equilibrium point these fluctuations will be distributed normally. There, the trajectory will have local drift ' .F (E 4 ) (see Eq. 12.3) and local covariance matrix .G(E4) (see Eq. 12.6). We also carried out a discrete time simulation of the innovation diffusion process with the same parameters. This used the embedded random walk with transition probabilities given in Eq. (12.7). Figure 12.2 shows the trajectories of the numbers of adoptees in the first and second groups. Another way of looking at the outcome is the projection of the trajectory in the .n 1 − n 2 plane, showing how the numbers of adoptees in each group relate to each other. Labeling the groups “older” and “younger” to match the empirical examples to follow, one such simulated trajectory (the wobbly curve) is shown in Fig. 12.3, together with the numerical solution of the differential equation system, Eq. (12.8), with the same parameters and initial conditions (the smooth curve). In the simulated trajectory, the number of adoptees in the older group (.n 1 ) increases monotonically in time, so that time may be taken as increasing from left to right. The presentation of the differential equation system solution is parametric, so that time increases as the smooth curve proceeds away from the origin. Clearly, the simulation produces a stochastic path close to the smooth deterministic path given by the differential equation system. The stability analysis for the model with these parameters gives three singularities, an unstable proper node at . E 1 = (0, 0), a saddle point at

12 Innovation Diffusion with Intergroup Suppression . . .

241

Fig. 12.2 Trajectories of simulation of innovation diffusion for two groups

E 2 = (1000, 0), and an in-spiral at . E 4 = (1193, 921); this last corresponds to the limit point of the deterministic curve in Fig. 12.3. The classification of . E 4 follows because the Jacobian has one positive eigenvalue and one negative eigenvalue at . E 4 . The simulated trajectory clearly conforms to the theoretical analysis. At the equilibrium point, the Gaussian diffusion has local diffusion .F' (E 4 ) = (−0.48, −0.54). That the drift is negative means that the farther it deviates from the equilibrium point the more stronger the trajectory will be pulled toward the equlibrium point. The local covariance matrix is [ ] 761.9 −0.046 .G(E 4 ) = −0.046 1059.2

.

Empirical Data on Tablet Use We present data for a situation that we suggest corresponds to the scenario being modeled. The cultural product in question is the tablet (computer), and the groups in question are age groups. We focus on the youngest age group, “under 25,” and the oldest age group, “55+.” We suggest that the youngest group will have a greater net adoption rate than the older age group, due to characteristics such as greater longterm expected payoffs to adopting new technology and greater intensity of social contacts, which facilitates the spread of information and influence. We also assume, however, that if use of tablets in the older group rises too high, the younger group will

242

S. S. Husain et al.

Fig. 12.3 Simulation of trajectory for innovation diffusion for two groups

begin to perceive the device as something for older people, at least not special for younger people; the tablet will lose much of its status value for younger people and this will inhibit or suppress its use. We assume there is no corresponding suppression of use by older people due to use by younger people. Finally, while there clearly is no direct migration from the “under 25” group to the “55+” group, there will be migration from “under 25” to “25–34,” from “25–34” to “35–44,” from “35–44” to “45–54,” and from “45–54” to “55+.” This we may take to be indirect migration from the youngest to the oldest age groups. Figure 12.4 show tablet use from 2012 through the first half of 2016 in 6 countries, with data from the Google Consumer (GoogleCB 2016). These graphs the youngest group use against the oldest group use; the third dimension, time, is omitted. It may be noted, however, that tablet use in the oldest age group increases monotonically with time in the oldest age group. Thus, in each graph, time increases from left to right. All six countries show that the increase in tablet use initially progresses more quickly in the younger age group. In four countries, clearly, tablet use ultimately declines in the younger group along with the last rise in the oldest group. This is not the case in two countries, Japan and France. The plots of the deterministic trajectory and of the simulation shown in Fig. 12.3 resemble the empirical graphs of Fig. 12.4. Recall that for those plots, the parameters specify a higher net adoption rate in the younger group, a very small migration rate from the younger to the older group, and external suppression from the older group to the younger group but not the other direction. Concerning Japan and France, note that in these two countries tablet use in

12 Innovation Diffusion with Intergroup Suppression . . .

243

Fig. 12.4 Tablet Use in UK, USA, Norway, Australia, Japan and France for under 25 and 55+ age groups, 2012–2016

the oldest age group has not reached the levels that it has in the other four countries. Thus, arguably the model may apply to these two countries as well, they are just at an earlier portion of the innovation diffusion process. The results of a goodness-of-fit test support this interpretation as well as the applicability of the model more generally. We tested the goodness-of-fit of the empirical data for tablet use to predictions of the simulation model, using the same parameter settings as given above. In fact, the model predictions used were those from the simulation run depicted in Figs. 12.2 and 12.3. Table 12.3 below shows the results using a chi-squared goodness-of-fit statistic. For each country, the maximum values are equated, which calibrates the data, then the simulation gap corresponding to one year (usually 4,000 iterations) is estimated, and finally a starting time in the simulation is estimated. This leaves seven degrees of freedom. Clearly, for all six countries the model fits somewhat; we cannot reject the null hypothesis that the model fits the data (. p > 0.05). It may be that a simpler model, namely a linear one, fits two of the countries, France and Japan, just as well. Nevertheless, the BP model fits sufficiently well the data from all six countries.

244

S. S. Husain et al.

Table 12.3 Fit of empirical data on tablet use, 2012–2016, to simulation model outcomes. Chisquared statistic with seven degrees of freedom Country

Chi-squared

United Kingdom United States Australia Norway France Japan

12.51.∗ 9.31.∗∗ 13.51.∗ 9.78.∗∗ 12.17.∗ 13.81.∗

∗p

.

> 0.05, .∗∗ p > 0.1

Conclusion We have presented here a new model of the diffusion of innovation. This is a model for a population divided into different groups, where adoption and use of the cultural product by one group may be negatively affected by use by a different group. The mathematics of the model is taken from the multi-layer Bolker-Pacala model of population dynamics. We present empirical evidence from several countries for tablet use that conforms to a pattern generated by the model, as shown by a simulation and supported by goodness-of-fit tests. The empirical pattern of tablet use, with quick adoption in one group but then decline, while the adoption in another group is slower, without decline, is unusual. Many models of the diffusion of innovation have been developed but none have been applied to such a situation, hence, the need for at least one more model. We do not claim that this BP model is generally appropriate, but we suggest that in situations of external suppression and inhibition, that is, from one group vis-a-vis another, this model can work well. Our empirical analysis here was for tablet use but we noted above other examples of this phenomenon in the literature such as first names and automobile makes, as well as historical examples such as sumptuary laws. We might note that the BP model can quickly present analytical difficulties. With age groups, fortunately, migration can occur in only one direction, but with other sorts of segmentation of populations, say along social class or region, migration would be possible in both directions. Even this small complication makes analyzing the steady states much more difficult. Considering more than two groups also would be desirable but, again, this greatly raises the level of analytical difficulty. It is always possible to simulate more complicated models, but a mathematical analysis is valuable for providing understanding. For example, in Sect. 12.3, through finding the singularities and evaluating the Jacobeans at the singularities, we gain a fairly thorough understanding of the dynamic system, what its tendencies are, and how these are affected by the parameters. Acknowledgements The authors would like to thank various colleagues for critical discussions and inputs. SSH acknowledges the Non-NET University Grants Commission (Ministry of Human

12 Innovation Diffusion with Intergroup Suppression . . .

245

Resource Development, Government of India) for research fellowship. JW acknowledges the support in part by NSF grant DMS-1627479. SSH & AC acknowledge the support by University of Potential Excellence-II grant (Project ID-47) of JNU, New Delhi, and the DST-PURSE grant given to JNU by the Department of Science and Technology, Government of India.

References Abergel, F., Aoyama, H., Chakrabarti, B.K., Chakraborti, A., Deo, N., Raina, D., Vodenska, I.: Econophysics and Sociophysics: Recent Progress and Future Directions. Springer (2017) Bartholomew, D.J.: Continuous time diffusion models with random duration of interest. J. Math. Sociol. 4(2), 187–199 (1976) Bass, F.M.: A new product growth for model consumer durables. Manag. Sci. 15(5), 215–227 (1969) Berger, J., Heath, C.: Where consumers diverge from others: identity signaling and product domains. J. Consum. Res. 34(2), 121–134 (2007) Berger, J., Heath, C.: Who drives divergence? identity signaling, outgroup dissimilarity, and the abandonment of cultural tastes. J. Pers. Soc. Psychol. 95(3), 593 (2008) Berger, J., Le Mens, G.: How adoption speed affects the abandonment of cultural tastes. Proc. Natl. Acad. Sci. 106(20), 8146–8150 (2009) Bessonov, M., Molchanov, S., Whitmeyer, J.: A mean field approximation of the bolker-pacala population model (2013). arXiv preprint arXiv:1302.1169 Bessonov, M., Molchanov, S., Whitmeyer, J.: A multi-class extension of the mean field BolkerPacala population model. Rand. Oper. Stochastic Eq. 26(3), 163–174 (2018) Bolker, B.M., Pacala, S.W.: Spatial moment equations for plant competition: understanding spatial strategies and the advantages of short dispersal. Am. Nat. 153(6), 575–602 (1999) Bolker, B.M., Pacala, S.W., Neuhauser, C.: Spatial dynamics in model plant communities: what do we really know? Am. Nat. 162(2), 135–148 (2003) Castellano, C., Fortunato, S., Loreto, V.: Statistical physics of social dynamics. Rev. Mod. Phys. 81(2), 591 (2009) Chakrabarti, B.K., Chakraborti, A., Chatterjee, A.: Econophysics and Sociophysics: Trends and Perspectives. Wiley-VCH Verlag GmbH (2006) Chakraborti, A., Husain, S.S., Whitmeyer, J.: A model for innovation diffusion with intergroup suppression (2018). arXiv preprint arXiv:1802.08943 Coleman, J.S., et al.: Introduction To Mathematical Sociology (1964) Goel, S., Watts, D.J., Goldstein, D.G.: The structure of online diffusion networks. In: Proceedings of the 13th ACM Conference on Electronic Commerce, pp. 623–638. ACM (2012) GoogleCB: Google consumer barometer, trended data: Percentage of people who use a tablet (2016). https://www.consumerbarometer.com/en/trending/. Accessed 1 Nov 2016 Goswami, D., Karmeshu: Stochastic evolution of innovation diffusion in heterogeneous groups: Study of life cycle patterns. IMA J. Manag. Math. 12(2), 107–126 (2001) Goswami, D., Karmeshu: Transient bimodality and catastrophic jumps in innovation diffusion. IEEE Trans. Syst. Man Cybern.—Part A: Syst. Hum. 38(3), 644–654 (2008) Kurtz, T.G.: Limit theorems for sequences of jump Markov processes approximating ordinary differential processes. J. Appl. Probabi. 8(2), 344–356 (1971) Leibenstein, H.: Bandwagon, snob, and Veblen effects in the theory of consumers’ demand. Quart. J. Econ. 64(2), 183–207 (1950) Levitt, S.D., Dubner, S.J.: A Rogue Economist Explores the Hidden. HarperCollins Publishers (2006) Lieberson, S.: A Matter of Taste: How Names, Fashions, and Culture Change. Yale University Press (2000) Logan, D.: A First Course in Differential Equations. Springer (2017)

246

S. S. Husain et al.

Mahajan, V., Peterson, R.A.: Models for Innovation Diffusion, vol. 48. Sage (1985) McPherson, M.: An ecology of affiliation. Am. Sociolo. Rev. 519–532 (1983) Rogers, E.: (1995) Diffusion of Innovations. Free Press, New York (1962) Sen, P., Chakrabarti, B.K.: Sociophysics: An Introduction. Oxford University Press (2014) Sharma, P., Karmeshu: Stochastic evolution of innovation diffusion in heterogeneous population: emergence of multimodel life cycle pattern, pp. 150–158 (2004) Young, H.P.: Innovation diffusion in heterogeneous populations: contagion, social influence, and social learning. Am. Econ. Rev. 99(5), 1899–1924 (2009)

Epilogue: Nobel Prize in Physics for Complexity Studies and Weather Behavior—Implications for Social Sciences and Public Policy

On the Nobel Prize website (www.nobelprize.org), one could read that the year 2021’s prize in Physics was awarded “for groundbreaking contributions to our understanding of complex systems”. “Complex systems” is gaining traction as a fascinating area of research and there is more ongoing work on the applications of complexity science in the social sciences to better model socio-economic phenomena. So, it might be just a matter of time before someone is recognized by the Nobel committee for ground-breaking insights in the economic sciences going beyond linear deterministic equilibrium models to far from equilibria realities. Those ‘realities’ are then better able to deal with a world of increasing complexity and they can aid in reducing the risk of catastrophic market and policy failures. Recently, one of us contributed an opinion piece to The New Indian Express (Chakraborti and Sharma 2021), a well-known newspaper in India, on the impact of this recent Nobel prize on science in India. One of the points we raised in that article argues that a paradigm shift has occurred over the last 20 years or so, where we have transited from conducting research exclusively from a reductionist perspective (i.e., we study the aggregate behavior of constituting parts) to a more holistic approach, where essentially, we focus on the study of collective behavior of the constituting parts. A very interesting (and early) example in economics of this collective behavior approach, is contained in a paper by Alan Kirman (1993) where the author considers a process ants follow when foraging for food. He uncovers endogenously determined regime switches that point towards the essentiality of collective behavior. An important field of research, econophysics, has contributed in many important ways to this paradigm shift. Econophysics essentially applies ideas from statistical mechanics to study many interesting aspects appearing in the social sciences. This field of science showed how financial markets are complex dynamical systems (Mantegna and Stanley 2000; Kauffman and Felin 2021), uncovered power laws in the distribution of city sizes (Gabaix 1999), studied power law correlations in weather phenomena like El Nino (Ausloos and Ivanova 2001), among many others. © The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 A. Chakraborti et al. (eds.), Quantum Decision Theory and Complexity Modelling in Economics and Public Policy, New Economic Windows, https://doi.org/10.1007/978-3-031-38833-0

247

248

Epilogue: Nobel Prize in Physics for Complexity Studies and Weather …

Weather systems, the economy, cities and ant or insect populations can all be seen as examples of complex systems. The study of complex systems has duly entered mainstream research and by doing so, graduate students can now access programmes which teach the essentials of such systems. The Santa Fe Institute, the flag bearer of this research, has spread its ‘complexity’ radiance to institutions across the US; Canada; Europe and the UK. In India, we have seen examples of several top-level educational institutions like Jawaharlal Nehru University and IIT Madras taking on such programmes. Besides the established statistical mechanics concepts and methods used in complex systems, we now can notice that dynamical social systems can also be modelled using tools from the quantum mechanics framework. A good example is the recent work by Khrennikov on ‘social laser’ theory (Khrennikov 2019). Even applications from quantum field theory have found inroads in social science (Baaquie 2018) and (Bagarello 2013). In the current volume, we have included contributions from pioneers in the complexity economics field, mainly drawing from the Santa Fe initiated literature. Here, without repeating those novel ideas, it is useful to point out some more details on quantum-like modelling research. As we already mentioned, Baaquie (2018) and Bagarello (2013) have contributed work using mathematical constructs from quantum field theory to describe complex systems. In the quantum-like approach, which sources formalisms from quantum mechanics (but not from quantum field theory), we can witness important new developments in the modelling of agents. Such agents are non-Boolean, and the quantum-like approach then calls for a quantum version of Aumann’s theorem (Khrennikov 2015; Khrennikova and Patra 2019). Furthermore, within the area of so-called quantum games do we see the construction of a quantum version of the majority rule (Bao and Halpern 2017). The quantum-like approach also shows features, which are quite uncommon in physics. The construction of Hamiltonians in complex social systems needs to follow different rules. It is not uncommon that one needs to work with non-Hermitian Hamiltonians, which is a hallmark of market systems (Baaquie 2005; Haven and Khrennikov 2013). Hence, we begin to see a harmonizing of both complexity modelling and quantumlike modelling. One point of focus where such harmonizing can start is around the idea there is a level of uncertainty which is deeper than what we would define when studying social systems from a purely economics or say sociological perspective. The holistic stance leads us away from mechanistic interpretations. The task is a challenging one, especially when one must attempt to implement policy. The current publication is narrowly associated with the Centre for Complexity Economics, Applied Spirituality and Public Policy (CEASP). The Centre was established at Jindal Global University (India) in September 2020, and it has as one of its objectives to study public policy within the realm of society as a complex system. The Center is transdisciplinary and wants to draw on many fields to shed light on societal issues of great importance (please see https://jgu.edu.in/jsgp/centre-for-com plexity-economics-applied/).

Epilogue: Nobel Prize in Physics for Complexity Studies and Weather …

249

With the current volume, it is felt by the organizers and the contributing authors that complexity needs to be more and more integrated in the social sciences, particularly the works we aspire to at CEASP. The 2021 Physics Nobel prize is of special significance and moral strength for us, since we are now more confident about applying the established methodologies in the complexity domain to social dynamical systems like economics, and financial markets. We believe such applications will have policy implications. We hope this volume will further work and foster research collaboration in complex systems.

References

Ausloos, M., Ivannova, K.: Power-law correlations in the southern-oscillation-index fluctuations characterizing El Nino. Phys. Rev. E 63(4), 047201 (2001) Baaquie, B.: Quantum Finance. Cambridge University Press (2005) Baaquie, B.: Quantum Field Theory For Economics and Finance. Cambridge University Press (2018) Bagarello, F.: Quantum dynamics for classical systems: with applications of the number operator. J. Wiley (2013) Bao, N., Halpern, N.Y.: Quantum voting and violation of arrow’s impossibility theorem. Phys. Rev. A 95, 062306 (2017) Chakraborti, A., Sharma, K.: Why the Nobel Prize in Physics Going Complex Systems’ Way Will Change the Way We Look at Science in India. The New Indian Express— EDEX (2021). https://www.edexlive.com/opinion/2021/nov/01/why-the-nobel-prize-in-phy sics-going-complex-systems-way-will-change-the-way-we-look-at-science-25259.html Gabaix, X.: Zipf’s law for cities: an explanation. Q. J. Econ. 114(3), 739–767 (1999) Haven, E., Khrennikov, A.: Quantum Social Science. Cambridge University Press (2013) Kauffman, S., Felin, T.: The search function and evolutionary novelty. In: Cattani, G., Mastriogiorgio, M. (eds) New Developments in Evolutionary Innovation: Novelty Creation in a Serendipitous Economy. Oxford University Press (2021) Khrennikov, A.: Quantum version of Aumann’s approach to common knowledge: sufficient conditions of impossibility to agree on disagree. J. Math. Econ. 60, 89–104 (2015) Khrennikov, A.: Social Laser: Application of Quantum Information and Field Theories to Modeling of Social Processes. Routledge, Taylor and Francis Group (2019) Khrennikova, P., Patra, S.: Asset trading under non classical ambiguity and heterogeneous beliefs. Physica A 521, 562–577 (2019) Kirman, A.: Ants, rationality, and recruitment. Q. J. Econ. 108(1), 137–156 (1993) Mantegna, R.N., Stanley, H.E.: An Introduction to Econophysics. Cambridge University Press (2000) The Nobel Prize in Physics 2021—nobelprize.org

© The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 A. Chakraborti et al. (eds.), Quantum Decision Theory and Complexity Modelling in Economics and Public Policy, New Economic Windows, https://doi.org/10.1007/978-3-031-38833-0

251