Resonance: From Probability to Epistemology and Back 1783269200, 9781783269204

Table of contents :
res_preface
res_intro
main-philosophies-of-probability-2016
skepticism-2016
the-frequency-philosophy-of-probability-2016
the-subjective-philosophy-of-probability-2016
the-logical-philosophy-of-probability-2016
common-issues-2016
epistemology-2016
religion-2016
science-2016
the-science-of-probability-2016
decision-making-2016
frequency-statistics-2016
bayesian-statistics-2016
on-ideologies-2016
paradoxes-wagers-and-rules-2016
teaching-probability-2016
mathematical-methods-of-probability-and-statistics-2016
9781783269211_bmatter

Citation preview

RESONANCE Resonance Downloaded from www.worldscientific.com by 80.229.203.126 on 06/24/16. For personal use only.

From Probability to Epistemology and Back

Krzysztof Burdzy University of Washington, USA

ICP P1073_9781783269204_tp.indd 2

Imperial College Press

28/4/16 8:15 AM

Published by Imperial College Press 57 Shelton Street Covent Garden London WC2H 9HE Distributed by World Scientific Publishing Co. Pte. Ltd. 5 Toh Tuck Link, Singapore 596224 USA office: 27 Warren Street, Suite 401-402, Hackensack, NJ 07601

Resonance Downloaded from www.worldscientific.com by 80.229.203.126 on 06/24/16. For personal use only.

UK office: 57 Shelton Street, Covent Garden, London WC2H 9HE

Library of Congress Cataloging-in-Publication Data Names: Burdzy, K. (Krzysztof ) Title: Resonance : from probability to epistemology and back / Krzysztof Burdzy (University of Washington, USA). Description: New Jersey : Imperial College Press, 2016. | Includes bibliographical references and index. Identifiers: LCCN 2015049769 | ISBN 9781783269204 (hc : alk. paper) Subjects: LCSH: Probabilities--Philosophy. | Knowledge, Theory of. Classification: LCC QA273.A35 B868 2016 | DDC 121--dc23 LC record available at http://lccn.loc.gov/2015049769 British Library Cataloguing-in-Publication Data A catalogue record for this book is available from the British Library.

Copyright © 2016 by Imperial College Press All rights reserved. This book, or parts thereof, may not be reproduced in any form or by any means, electronic or mechanical, including photocopying, recording or any information storage and retrieval system now known or to be invented, without written permission from the Publisher.

For photocopying of material in this volume, please pay a copying fee through the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, USA. In this case permission to photocopy is not required from the publisher. Desk Editors: Dipasri Sardar/Mary Simpson Typeset by Stallion Press Email: [email protected] Printed in Singapore

Dipa - Resonance.indd 1

28/4/2016 10:18:51 AM

Resonance Downloaded from www.worldscientific.com by 80.229.203.126 on 06/24/16. For personal use only.

April 29, 2016

9:7

Resonance: From Probability to Epistemology and Back

To Agnieszka, my wife

v

9in x 6in

b2422-fm

page v

April 29, 2016

9:7

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-fm

Resonance Downloaded from www.worldscientific.com by 80.229.203.126 on 06/24/16. For personal use only.

Preface

Do you believe that two and two makes four? Yes? Then why? You may reply that “It works” or “Everyone knows that 2 + 2 = 4.” Both answers are perfectly reasonable. We need this kind of attitude in everyday life. Any reasoning more formal than that would paralyze our normal activities. But the two common sense answers completely fail in some situations. This book is devoted to one of these situations — the philosophical controversy surrounding the concept of probability. The philosophy of probability has several major branches, the best known of which are frequency and subjective. These two philosophies are (mistakenly) associated with the two main branches of statistics — frequency and Bayesian. The temperature of the intellectual dispute concerning the meaning and applications of probability has been high in both philosophy and statistics. Why did philosophers and statisticians fail to check what “works?” They did not. They checked and they know very well what works except that representatives of each side of the dispute swear that their own methods “work” and those of their opponents do not. This controversy shows that we have to try to understand the sources of our knowledge at a level deeper than “It works” — this is what this book is about. My previous book, “The search for certainty,” [Burdzy (2009)] codified the scientific laws of probability. All I did was to formalize what was already present in probability and statistics textbooks. In this volume, I will try to supply epistemological foundations for probability. One particular philosophical idea provides a strong motivation for my analysis — the idea that probability is subjective. In the previous book, I showed that the interpretation of subjectivism advocated by Bruno de Finetti made no sense whatsoever. But the subjectivist current in philosophy of probability vii

page vii

April 29, 2016

9:7

Resonance Downloaded from www.worldscientific.com by 80.229.203.126 on 06/24/16. For personal use only.

viii

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-fm

Preface

is not limited to the extravagant theory of de Finetti and it deserves an honest analysis. Contrary to the popular belief, de Finetti was an antisubjectivist because he denied any value to subjective probabilistic opinions. Paradoxically, my epistemological analysis of personal knowledge (I borrowed this phrase from [Polanyi (1958)]) will make me more of a subjectivist than I ever expected or wanted to be. I will base my epistemology on the concept of “resonance,” loosely related to its namesake in physics. I will use resonance in my analysis of the philosophical problem of induction. I will also briefly analyze three other well known philosophical problems of consciousness, intelligence and free will. It is not my intention to present detailed explanations for these phenomena but to propose a small set of ideas that will remove the mystery out of these seemingly incomprehensible aspects of human intellect. Going back to the original motivation for this project, I will use this epistemology to outline my opinions about the apparent subjectivity of probability. I imported much of the discussion of philosophy of probability and its relationship to statistics from [Burdzy (2009)] but I changed the presentation of that material and I put it in a new context — that of the epistemological ideas developed in this volume. I am grateful to people who offered their comments on [Burdzy (2009)] or the draft of this manuscript and thus helped me improve the book: Itai Benjamini, Erik Bj¨ ornemo, Nicolas Bouleau, Arthur Fine, Adrew Gelman, Artur Grabowski, Peter Hoff, Wilfrid Kendall, Vlada Limic, Dan Osborn, Jeffrey Rosenthal, Jaime San Martin, Jim Pitman, Christian Robert, Pedro Ter´an, John Walsh, Larry Wasserman and anonymous referees. Special thanks go to Janina Burdzy, my mother and a probabilist, for teaching me combinatorial probability 40 years ago. The lesson about the fundamental role of symmetry in probability was never forgotten. I am grateful to Agnieszka Burdzy, my wife, for discussing with me a number of philosophical problems related to this book. I acknowledge with gratitude generous support from the National Science Foundation. Seattle, 2015

page viii

April 29, 2016

9:7

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-fm

Resonance Downloaded from www.worldscientific.com by 80.229.203.126 on 06/24/16. For personal use only.

Contents

Preface

vii

About the Author

ix

1. Introduction

1

1.1 1.2 1.3

1.4 1.5

Knowledge . . . . . . . . . . . . . . . . . . . . . . . . . Probability . . . . . . . . . . . . . . . . . . . . . . . . . Summary of the Main Claims . . . . . . . . . . . . . . . 1.3.1 Resonance . . . . . . . . . . . . . . . . . . . . . 1.3.2 Critique of frequency and subjective philosophies of probability . . . . . . . . . . . . . . . . . . . 1.3.2.1 Positive philosophical ideas . . . . . . . 1.3.2.2 Negative philosophical ideas . . . . . . 1.3.2.3 Innovative technical ideas . . . . . . . . 1.3.3 Scientific laws of probability . . . . . . . . . . . 1.3.4 Statistics and philosophy . . . . . . . . . . . . . Historical and Social Context . . . . . . . . . . . . . . . Disclaimers . . . . . . . . . . . . . . . . . . . . . . . . .

Philosophy of Probability

The The The The

Classical Theory . . Logical Theory . . Propensity Theory Subjective Theory .

4 5 6 8 8 9 11 13

17

2. Main Philosophies of Probability 2.1 2.2 2.3 2.4

1 2 3 3

. . . .

19 . . . .

xi

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

20 20 22 22

page xi

April 29, 2016

9:7

xii

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-fm

Contents

Resonance Downloaded from www.worldscientific.com by 80.229.203.126 on 06/24/16. For personal use only.

2.4.1 2.4.2 2.4.3

2.5 2.6 2.7 2.8 2.9

Interpreting subjectivity . . . . . . . . . . Verification of probabilistic statements . . Subjectivity as an escape from the shackles of verification . . . . . . . . . . . . . . . . 2.4.4 The Dutch book argument . . . . . . . . . 2.4.5 The axiomatic system . . . . . . . . . . . . 2.4.6 Identification of probabilities and decisions 2.4.7 The Bayes theorem . . . . . . . . . . . . . The Frequency Theory . . . . . . . . . . . . . . . . Summary of Philosophical Theories of Probability From Ideas to Theories . . . . . . . . . . . . . . . . Popular Philosophy . . . . . . . . . . . . . . . . . . Is There Life Beyond Von Mises and De Finetti? .

. . . . . .

23 24

. . . . . . . . . .

26 27 29 30 30 31 33 34 35 38

. . . . . . . . . .

. . . . . . . . . .

3. Skepticism 3.1 3.2 3.3 3.4

3.5 3.6 3.7 3.8

How Do You Prove that You are Not a Camel? Skepticism . . . . . . . . . . . . . . . . . . . . . 3.2.1 Frustration of philosophers . . . . . . . Anything Goes . . . . . . . . . . . . . . . . . . Von Mises’ Brand of Skepticism . . . . . . . . . 3.4.1 The smoking gun . . . . . . . . . . . . 3.4.2 Inconsistencies in von Mises’ theory . . De Finetti’s Brand of Skepticism . . . . . . . . 3.5.1 How to eat the cake and have it too . . On Approximate Theories . . . . . . . . . . . . Temperature, Beauty and Probability . . . . . Latter Day Subjectivism . . . . . . . . . . . . .

41 . . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

4. The Frequency Philosophy of Probability 4.1 4.2

4.3 4.4 4.5 4.6

Collective as an Elementary Concept . . . . . . . . . . . Applications of Probability Do Not Rely on Collectives . 4.2.1 Stochastic processes . . . . . . . . . . . . . . . . 4.2.2 Unlikely events . . . . . . . . . . . . . . . . . . . 4.2.3 Graphical communication . . . . . . . . . . . . . Collectives in Real Life . . . . . . . . . . . . . . . . . . . Collectives and Symmetry . . . . . . . . . . . . . . . . . Frequency Theory and the Law of Large Numbers . . . Why is Mathematics Useful? . . . . . . . . . . . . . . .

41 42 46 47 48 49 49 51 52 55 56 57 59 59 61 61 62 63 65 67 68 69

page xii

April 29, 2016

9:7

4.7 4.8 4.9 4.10 4.11 4.12 4.13 4.14

Resonance: From Probability to Epistemology and Back

9in x 6in

Benefits of Imagination and Imaginary Benefits Imaginary Collectives . . . . . . . . . . . . . . Computer Simulations . . . . . . . . . . . . . . Frequency Theory and Individual Events . . . . Collectives and Populations . . . . . . . . . . . Are All i.i.d. Sequences Collectives? . . . . . . Are Collectives i.i.d. Sequences? . . . . . . . . . Martin-L¨ of Sequences . . . . . . . . . . . . . .

. . . . . . . .

b2422-fm

Contents

xiii

. . . . . . . .

70 71 72 73 74 75 76 77

. . . . . . . .

. . . . . . . .

. . . . . . . .

Resonance Downloaded from www.worldscientific.com by 80.229.203.126 on 06/24/16. For personal use only.

5. The Subjective Philosophy of Probability 5.1 5.2 5.3 5.4

5.5

5.6 5.7 5.8 5.9 5.10 5.11 5.12 5.13 5.14 5.15 5.16 5.17 5.18 5.19 5.20 5.21

“Subjective” — A Word with a Subjective Meaning . The Subjective Theory of Probability is Objective . . A Science without Empirical Content . . . . . . . . . . If Probability does not Exist, Everything is Permitted 5.4.1 Creating something out of nothing . . . . . . . 5.4.2 The essence of probability . . . . . . . . . . . De Finetti’s Ultimate Failure . . . . . . . . . . . . . . 5.5.1 Lazy decision maker . . . . . . . . . . . . . . . 5.5.2 Interpreting Dutch book . . . . . . . . . . . . 5.5.3 Dutch book with a lapse of time . . . . . . . . 5.5.3.1 The butterfly effect . . . . . . . . . . 5.5.4 Rule of conditionalization . . . . . . . . . . . . All Sequential Decisions are Consistent . . . . . . . . . Honest Mistakes . . . . . . . . . . . . . . . . . . . . . Cohabitation with an Evil Demiurge . . . . . . . . . . Why Bother to Use Probability? . . . . . . . . . . . . The Dutch Book Argument is Rejected by Bayesians . Insurance Against Everything . . . . . . . . . . . . . . No Need to Collect Data . . . . . . . . . . . . . . . . . Empty Promises . . . . . . . . . . . . . . . . . . . . . The Meaning of Consistency . . . . . . . . . . . . . . . Interpreting Miracles . . . . . . . . . . . . . . . . . . . Science, Probability and Subjectivism . . . . . . . . . Apples and Oranges . . . . . . . . . . . . . . . . . . . Arbitrage . . . . . . . . . . . . . . . . . . . . . . . . . Subjective Theory and Atheism . . . . . . . . . . . . . Imagination and Probability . . . . . . . . . . . . . . . A Misleading Slogan . . . . . . . . . . . . . . . . . . .

79 . . . . . . . . . . . . . . . . . . . . . . . . . . . .

80 81 83 84 85 86 88 90 91 92 93 94 95 96 98 100 100 101 102 103 104 105 107 108 109 110 111 113

page xiii

April 29, 2016

9:7

xiv

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-fm

Contents

5.22 5.23 5.24 5.25

Axiomatic System as a Magical The Meaning of Subjectivity . Probability and Chance . . . . Conflict Resolution . . . . . .

Trick . . . . . . . . . . . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

6. The Logical Philosophy of Probability 6.1 6.2

Resonance Downloaded from www.worldscientific.com by 80.229.203.126 on 06/24/16. For personal use only.

6.3

125

Falsifiability . . . . . . . . . . . . . . . . . . . . . . . . . Why Do Scientists Ignore the Logical Philosophy of Probability? . . . . . . . . . . . . . . . . . . . . . . . Probabilities of Propositions and Events . . . . . . . . .

7. Common Issues 7.1 7.2 7.3 7.4 7.5 7.6 7.7

Law Enforcement . . . . . . . . . . . The Value of Extremism . . . . . . . Common Elements in Frequency and Theories . . . . . . . . . . . . . . . . Common Misconceptions . . . . . . . Shattered Dreams of Perfection . . . What Exists? . . . . . . . . . . . . . Abuse of Language . . . . . . . . . . 7.7.1 Expected value . . . . . . . . 7.7.2 Standard deviation . . . . . 7.7.3 Subjective opinions . . . . . 7.7.4 Optimal Bayesian decisions . 7.7.5 Confidence intervals . . . . . 7.7.6 Significant difference . . . . 7.7.7 Consistency . . . . . . . . . 7.7.8 Objective Bayesian methods 7.7.9 Prior . . . . . . . . . . . . . 7.7.10 Non-informative prior . . . .

114 114 119 122

125 127 128 131

. . . . . . . . . . . . . . Subjective . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . .

131 133

. . . . . . . . . . . . . . .

133 134 136 137 137 138 138 138 138 138 138 139 139 140 140

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

Epistemology

141

8. Epistemology

143

8.1

The Problem of Induction . . . . . . . . . . . . . . . 8.1.1 An ill posed problem . . . . . . . . . . . . . 8.1.1.1 On intuitively obvious propositions 8.1.2 Induction is a law of nature . . . . . . . . .

. . . .

. . . .

143 144 146 147

page xiv

April 29, 2016

9:7

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-fm

Contents

Resonance Downloaded from www.worldscientific.com by 80.229.203.126 on 06/24/16. For personal use only.

8.2

8.3 8.4

8.5 8.6

8.1.3 Anthropic principle . . . . . . . . . . Resonance . . . . . . . . . . . . . . . . . . . . 8.2.1 Information and knowledge . . . . . . 8.2.2 Resonance complexity . . . . . . . . . 8.2.3 Facts . . . . . . . . . . . . . . . . . . 8.2.4 Learning resonance . . . . . . . . . . 8.2.5 Resonance level reduction . . . . . . . 8.2.6 Properties of resonance . . . . . . . . 8.2.7 Physical basis of resonance . . . . . . 8.2.8 Information and knowledge revisited . 8.2.9 Resonance and subjectivity . . . . . . 8.2.10 Raw resonance . . . . . . . . . . . . . 8.2.11 Resonance and philosophy of Hume . 8.2.12 Is resonance a new concept? . . . . . Consciousness . . . . . . . . . . . . . . . . . . Intelligence . . . . . . . . . . . . . . . . . . . 8.4.1 Artificial Intelligence . . . . . . . . . 8.4.2 Social Context of Intelligence . . . . . Free Will . . . . . . . . . . . . . . . . . . . . From Philosophy to Science . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . .

xv

148 149 149 150 151 153 155 156 157 159 160 161 162 163 164 166 169 170 171 174

9. Religion

177

10. Science

181

10.1 10.2

Science as a Communication System . . . . . . Some Attributes of Science . . . . . . . . . . . 10.2.1 Interpersonal character . . . . . . . . . 10.2.2 The role and limitations of resonance . 10.2.3 Convergence to the truth? . . . . . . . 10.2.4 Science control . . . . . . . . . . . . . . 10.2.5 Beyond simple induction . . . . . . . . 10.2.6 Science as a web . . . . . . . . . . . . . 10.3 Science for Scientists . . . . . . . . . . . . . . . 10.4 Alien Science . . . . . . . . . . . . . . . . . . . 10.4.1 A lonely alien . . . . . . . . . . . . . . 10.5 Sources and Perils of Loyalty . . . . . . . . . . 10.5.1 Science as an antidote to manipulation 10.5.2 Dependent information sources . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

181 184 184 185 185 186 186 187 187 188 189 190 193 194

page xv

April 29, 2016

9:7

xvi

9in x 6in

b2422-fm

Contents

10.6 10.7 10.8 10.9 10.10 10.11 10.12

Resonance Downloaded from www.worldscientific.com by 80.229.203.126 on 06/24/16. For personal use only.

Resonance: From Probability to Epistemology and Back

10.13

Falsificationism and Resonance . . . . . . . . . . . . . Falsificationism as a Religion . . . . . . . . . . . . . . 10.7.1 Sainthood in science . . . . . . . . . . . . . . . Technology . . . . . . . . . . . . . . . . . . . . . . . . Multiple Personality Disorder . . . . . . . . . . . . . . Reality, Philosophy and Science . . . . . . . . . . . . . Decision Making . . . . . . . . . . . . . . . . . . . . . Major Trends in Philosophy of Science . . . . . . . . . 10.12.1 Real science — the big picture . . . . . . . . . 10.12.2 Probabilism and Bayesianism in philosophy of science . . . . . . . . . . . . . . . . . . . . . 10.12.3 Levels of philosophical analysis . . . . . . . . . 10.12.4 Position of my theory in philosophy of science Circularity . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . .

195 197 198 200 200 202 204 205 207

. . . .

208 210 211 213

Science of Probability 11. The Science of Probability 11.1

11.2 11.3

Interpretation of (L1)–(L6) . . . . . . . . . . . . . . . . 11.1.1 Events . . . . . . . . . . . . . . . . . . . . . . . 11.1.2 Symmetry . . . . . . . . . . . . . . . . . . . . . 11.1.3 Enforcement . . . . . . . . . . . . . . . . . . . . 11.1.4 Limits of applicability . . . . . . . . . . . . . . . 11.1.5 (L1)–(L6) as a starting point . . . . . . . . . . . 11.1.6 Approximate probabilities . . . . . . . . . . . . 11.1.7 Statistical models . . . . . . . . . . . . . . . . . 11.1.8 The Bayes theorem . . . . . . . . . . . . . . . . 11.1.9 Probability of past events . . . . . . . . . . . . . 11.1.10 Purely mathematical independence . . . . . . . 11.1.11 Ruelle’s view of probability . . . . . . . . . . . . Scientific Verification of (L1)–(L6) . . . . . . . . . . . . Predictions . . . . . . . . . . . . . . . . . . . . . . . . . 11.3.1 Predictions at various reliability levels . . . . . . 11.3.2 Predictions in existing scientific and philosophical theories . . . . . . . . . . . . . . . . . . . . . . . 11.3.3 Predictions, conditioning and hypothesis tests . 11.3.4 Prediction examples . . . . . . . . . . . . . . . .

215 217 218 218 219 221 221 222 223 224 225 225 226 226 226 229 230 232 233 233

page xvi

April 29, 2016

9:7

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-fm

Contents

Resonance Downloaded from www.worldscientific.com by 80.229.203.126 on 06/24/16. For personal use only.

11.4 11.5 11.6 11.7

11.8 11.9 11.10 11.11 11.12 11.13 11.14 11.15 11.16 11.17 11.18 11.19 11.20 11.21 11.22 11.23 11.24 11.25

11.3.5 Histograms and image reconstruction . . . . . . 11.3.6 Contradictory predictions . . . . . . . . . . . . . 11.3.7 Multiple predictions . . . . . . . . . . . . . . . . Symmetry, Independence and Resonance . . . . . . . . . Symmetry is Relative . . . . . . . . . . . . . . . . . . . Moderation is Golden . . . . . . . . . . . . . . . . . . . Applications of (L1)–(L6): Some Examples . . . . . . . 11.7.1 Poisson process . . . . . . . . . . . . . . . . . . 11.7.2 Laws (L1)–(L6) as a basis for statistics . . . . . 11.7.3 Long run frequencies and (L1)–(L6) . . . . . . . 11.7.4 Life on Mars . . . . . . . . . . . . . . . . . . . . Symmetry and Data . . . . . . . . . . . . . . . . . . . . Probability of a Single Event . . . . . . . . . . . . . . . On Events that Belong to Two Sequences . . . . . . . . Events Are More Fundamental Than Random Variables Deformed Coins . . . . . . . . . . . . . . . . . . . . . . . Are Coin Tosses i.i.d. or Exchangeable? . . . . . . . . . Mathematical Foundations of Probability . . . . . . . . Axioms versus Laws of Science . . . . . . . . . . . . . . Objective and Subjective Probabilities . . . . . . . . . . Physical and Epistemic Probabilities . . . . . . . . . . . Can Probability Be Explained? . . . . . . . . . . . . . . Propensity . . . . . . . . . . . . . . . . . . . . . . . . . . Countable Additivity . . . . . . . . . . . . . . . . . . . . Yin and Yang . . . . . . . . . . . . . . . . . . . . . . . . Are Laws (L1)–(L6) Necessary? . . . . . . . . . . . . . Quantum Mechanics . . . . . . . . . . . . . . . . . . . . The History of (L1)–(L6) in Philosophy of Probability . Symmetry and Theories of Probability . . . . . . . . . .

12. Decision Making 12.1 12.2

Common Practices . . . . . . . . . . . . . . . . . . . Decision Making in the Context of (L1)–(L6) . . . . 12.2.1 Maximization of expected gain . . . . . . . . 12.2.2 Maximization of expected gain as an axiom . 12.2.3 Stochastic ordering of decisions . . . . . . . 12.2.4 Generating predictions . . . . . . . . . . . . 12.2.5 Intermediate decision problems . . . . . . . .

xvii

234 236 238 242 243 244 246 246 247 247 248 250 251 252 253 254 256 257 258 259 260 261 263 264 265 266 266 267 268 271

. . . . . . .

. . . . . . .

271 272 272 275 275 278 279

page xvii

April 29, 2016

9:7

xviii

Resonance: From Probability to Epistemology and Back

b2422-fm

Contents

12.3 12.4 12.5

12.6 Resonance Downloaded from www.worldscientific.com by 80.229.203.126 on 06/24/16. For personal use only.

9in x 6in

Decision Making and Resonance . . . . . . . . Events with No Probabilities . . . . . . . . . . Utility . . . . . . . . . . . . . . . . . . . . . . . 12.5.1 Variability of utility in time . . . . . . 12.5.2 Nonlinearity of utility . . . . . . . . . . 12.5.3 Utility of non-monetary rewards . . . . 12.5.4 Unobservable utilities . . . . . . . . . . 12.5.5 Can utility be objective? . . . . . . . . 12.5.6 What is the utility of gazillion dollars? Identification of Decisions and Probabilities . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

13. Frequency Statistics 13.1

13.2

13.3

13.4 13.5 13.6

291

Confidence Intervals . . . . . . . . . . . . . . . . . . . . 13.1.1 Practical challenges with statistical predictions . 13.1.2 Making predictions is necessary . . . . . . . . . Estimation . . . . . . . . . . . . . . . . . . . . . . . . . 13.2.1 Estimation and (L1)–(L6) . . . . . . . . . . . . 13.2.2 Unbiasedness — a concept with a single application . . . . . . . . . . . . . . . . . . . . Hypothesis Testing . . . . . . . . . . . . . . . . . . . . . 13.3.1 Hypothesis tests and collectives . . . . . . . . . 13.3.2 Hypothesis tests and the frequency interpretation of probability . . . . . . . . . . . . . . . . . . . 13.3.3 Hypothesis testing and (L1)–(L6) . . . . . . . . 13.3.3.1 Sequences of hypothesis tests . . . . . . 13.3.3.2 Single hypothesis test . . . . . . . . . . Hypothesis Testing and (L6) . . . . . . . . . . . . . . . Hypothesis Testing and Falsificationism . . . . . . . . . Does Frequency Statistics Need the Frequency Philosophy of Probability? . . . . . . . . . . . . . . . . .

14. Bayesian Statistics 14.1

14.2

Two Faces of Subjectivity . . . . . . . . . . . . . 14.1.1 Non-existence versus informal assessment 14.1.2 Are all probabilities subjective? . . . . . 14.1.3 Conditioning versus individuality . . . . 14.1.4 Non-existent decisions . . . . . . . . . . . Elements of Bayesian Analysis . . . . . . . . . .

280 282 284 284 285 287 288 288 289 290

291 292 294 294 297 297 299 300 301 301 301 302 303 304 304 307

. . . . . .

. . . . . .

. . . . . .

. . . . . .

307 307 308 308 309 309

page xviii

April 29, 2016

9:7

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-fm

Contents

14.3

14.4

Resonance Downloaded from www.worldscientific.com by 80.229.203.126 on 06/24/16. For personal use only.

14.5 14.6 14.7 14.8 14.9 14.10 14.11 14.12 14.13

Models . . . . . . . . . . . . . . . . . . . . . . . . . 14.3.1 Bayesian models are totally objective . . . 14.3.2 Bayesian models are totally subjective . . . Priors . . . . . . . . . . . . . . . . . . . . . . . . . 14.4.1 Objective priors . . . . . . . . . . . . . . . 14.4.2 Bayesian statistics as an iterative method . 14.4.3 Truly subjective priors . . . . . . . . . . . Resonance at Work . . . . . . . . . . . . . . . . . . Data . . . . . . . . . . . . . . . . . . . . . . . . . . Posteriors . . . . . . . . . . . . . . . . . . . . . . . 14.7.1 Non-convergence of posterior distributions Bayesian Statistics and (L1)–(L6) . . . . . . . . . . Spurious Predictions . . . . . . . . . . . . . . . . . Who Needs Subjectivism? . . . . . . . . . . . . . . Preaching to the Converted . . . . . . . . . . . . . Constants and Random Variables . . . . . . . . . . Criminal Trials . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . .

Miscellanea

On Ideologies and Their Photo-Negatives . . . . . Experimental Statistics — A Missing Science . . . Statistical Time Capsules . . . . . . . . . . . . . . Is Statistics a Science? . . . . . . . . . . . . . . . Psychoanalytic Interpretation of Philosophy of Probability . . . . . . . . . . . . . . . . . . . . . 15.6 From Intuition to Science . . . . . . . . . . . . . . 15.7 Science as Service . . . . . . . . . . . . . . . . . . . 15.8 The Three Aspects of Probability . . . . . . . . . . 15.9 Is Probability a Science? . . . . . . . . . . . . . . . 15.10 Are Probability and Logic Experimental Sciences?

333 . . . .

. . . .

. . . .

333 337 340 341

. . . . . .

. . . . . .

. . . . . .

343 344 346 347 348 349

16. Paradoxes, Wagers and Rules 16.1 16.2

310 310 311 311 312 313 315 317 318 318 319 321 321 322 323 326 328

331

15. On Ideologies 15.1 15.2 15.3 15.4 15.5

xix

St. Petersburg Paradox . . . . . . . . . . . . . . . . . . Pascal’s Wager . . . . . . . . . . . . . . . . . . . . . . . 16.2.1 Scientific aspects of Pascal’s wager . . . . . . . .

353 353 355 356

page xix

April 29, 2016

9:7

xx

9in x 6in

b2422-fm

Contents

16.3

16.4 16.5 Resonance Downloaded from www.worldscientific.com by 80.229.203.126 on 06/24/16. For personal use only.

Resonance: From Probability to Epistemology and Back

16.6 16.7

16.2.1.1 Two kinds of infinity . . . . . . . . 16.2.1.2 Minor sins . . . . . . . . . . . . . . 16.2.1.3 On the utility of eternal life in hell 16.2.1.4 Exponential discounting . . . . . . 16.2.2 A sociological analysis of Pascal’s wager . . Cromwell’s Rule . . . . . . . . . . . . . . . . . . . . 16.3.1 Cromwell’s rule: practical implementation . 16.3.2 Cromwell’s rule: philosophical problems . . Principal Principle . . . . . . . . . . . . . . . . . . . A New Prisoner Paradox . . . . . . . . . . . . . . . . 16.5.1 Analysis of the new prisoner paradox . . . . Ellsberg Paradox . . . . . . . . . . . . . . . . . . . . The Probability of God . . . . . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

17. Teaching Probability 17.1 17.2 17.3

375

Teaching Independence . . . . . . . . . . . . . . . . . . . Probability and Frequency . . . . . . . . . . . . . . . . . Undergraduate Textbooks . . . . . . . . . . . . . . . . .

18. Mathematical Methods of Probability and Statistics 18.1

18.2 18.3 18.4

Probability . . . . . . . . . . . . . . . . . . . . . . . 18.1.1 Law of large numbers, central limit theorem and large deviations principle . . . . . . . . 18.1.2 Exchangeability and de Finetti’s theorem . . Frequency Statistics . . . . . . . . . . . . . . . . . . Bayesian Statistics . . . . . . . . . . . . . . . . . . . Contradictory Predictions . . . . . . . . . . . . . . .

356 357 358 358 359 361 361 364 366 367 368 369 372

378 379 380 383

. .

383

. . . . .

385 385 386 387 388

. . . . .

Bibliography

391

Index

397

page xx

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch01

Chapter 1

Introduction

My earlier book on philosophy of probability, [Burdzy (2009)], was focused on finding the best scientific formulation of the foundations of probability. This book presents much of the same program but it also tries to determine the main sources of the success of probability. This analysis will take me well beyond the narrow topic of philosophy of probability, to the frightening depths of epistemology.

1.1 Knowledge “There is no truth” — this claim, in different forms, was made by a number of philosophers. What is surprising to me is that philosophers as different as Karl Popper and Thomas Kuhn, arguably the best known 20th century philosophers of science, shied away from the clear declaration that science constantly brings us closer to the truth about the objectively existing universe. I cannot prove that the objective universe exists or that we can find the truth about it. Nobody can. I am a 100% skeptic. But skepticism is a dead end in philosophy. The interesting direction in philosophy is to describe how we arrive at statements that we consider true. Once we understand the process, or rather many different processes, we can join other people in pursuing the truth in one of the established ways or we can seek our own alternative way. Scientists arrive at the truth in their own way. The prevalence of religion proves that there is not even a slightest chance, in the present society, for a consensus on how to find the truth. Despite the obvious lack of consensus on the truth and ways of attaining it, I believe that there is an important and universal element of knowledge acquisition (I will call it “resonance”) that received too little attention from philosophers. I will argue that resonance is the missing link in the known philosophical 1

page 1

April 29, 2016

9:6

2

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch01

Resonance: From Probability to Epistemology and Back

theories of probability. If resonance proves to be a viable concept outside philosophy of probability, I will consider this a welcome bonus.

1.2 Probability Two and two makes four. Imagine a mathematical theory which says that it makes no sense to talk about the result of addition of two and two. Imagine another mathematical theory which says that the result of addition of two and two is whatever you think it is. Would you consider any of these theories a reasonable foundation of science? Would you think that they are relevant to ordinary life? If you toss a coin, the probability of heads is 1/2. According to the frequency philosophy of probability, it makes no sense to talk about the probability of heads on a single toss of a coin. According to the subjective philosophy of probability, the probability of heads is whatever you think it is. Would you consider any of these theories to be a reasonable foundation of science? Would you think that they are relevant to ordinary life? The frequency philosophy of probability is usually considered to be the basis of the “frequency” statistics and the subjective philosophy of probability is often regarded as the basis of the “Bayesian” statistics (readers unfamiliar with these terms should consult Chapter 18). According to the frequency philosophy of probability, the concept of probability is limited to long runs of identical experiments or observations, and the probability of an event is the relative frequency of the event in the long sequence. The subjective philosophy claims that there is no objective probability and so probabilities are subjective views; they are rational and useful only if they are “consistent,” that is, if they satisfy the usual mathematical probability formulas. Von Mises, who created the frequency philosophy, claimed that ([von Mises (1957), p. 11]), We can say nothing about the probability of death of an individual [within a year] even if we know his condition of life and health in detail.

De Finetti, who proposed the subjective philosophy, asserted that ([de Finetti (1974), p. x]), Probability does not exist.

The standard education in probability and statistics is a process of indoctrination in which students are taught, explicitly or implicitly, that

page 2

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch01

Introduction

3

individual events have probabilities, and some methods of computing probabilities are scientific and rational. An alien visiting our planet from a different galaxy would have never guessed from our textbooks on probability and statistics that the two main branches of statistics are related to the philosophical claims cited above. I believe that the two cited philosophical claims are incomprehensible to all statisticians except for a handful of aficionados of philosophy. I will try to explain their meaning and context in this book. I will also argue that the quoted claims are not mere footnotes but they constitute the essence of the two failed philosophical theories.

1.3 Summary of the Main Claims 1.3.1 Resonance The acquisition of information and creating knowledge (this includes both facts and theories) can be divided into two steps. I will call the first step “resonance” for reasons explained later in the book. This process is very fast in most cases, subconscious and very reliable in a great variety of situations. My guess is that resonance is not based on logic in any reasonable sense of the word “logic.” Resonance is a crude but reasonably reliable filter of information arriving at our senses. Resonance is fallible in many situations recognized as significant to individual lives, society and science. Logic, probability, induction and all other named and unnamed ingredients of science provide the second filter, much more refined and reliable than resonance. Traditionally, philosophy was focused on the second filter because we have almost no access to resonance via our consciousness. This situation created various misconceptions concerning the sources of reliable truth. One of these is a tendency to ignore resonance despite the fact that resonance is at least as important to science and general knowledge as logic. Another common misconception is that resonance (under the name of “subjectivity”) is unreliable or not needed. Some other myths go in the opposite direction and invest intuition, subjective opinions and mystical experiences in powers that these sources of opinion and information do not have. Resonance is a necessary first filter because it is impossible to process all information available to us in a logical way in a timely manner. This should not be interpreted as a claim that resonance was created deliberately by humans. Quite the opposite, resonance is the result of the blind evolution process selecting the fittest individuals.

page 3

April 29, 2016

9:6

4

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch01

Resonance: From Probability to Epistemology and Back

I will illustrate the role of resonance in the knowledge acquisition process by analyzing four classical philosophical problems: induction, consciousness, intelligence and free will. I will argue that the classical problem of induction is ill-posed. Our “knowledge” of facts is based on the same ontological and epistemological assumptions as our predictions of “unknown” events. The reliability of induction is a law of nature or rather the confluence of several laws of nature, including evolution. These laws of nature are specific to our universe so no general logical justification of induction can exist. Consciousness is (among other things) an ability to observe, memorize and analyze one’s own information processing. Resonance is inaccessible to consciousness so it may appear to be irrational. This is a misleading impression. A process that does not follow the classical logic and is inaccessible to conscious analysis does not have to be arbitrary, subjective or unreliable. Intelligence is the highest form of resonance. It does not have roots in observations of repetitive phenomena. Its essence is the ability to select facts or highly probable theories that are relevant to the current interests of the individual or society from the practically infinite amount of information and potential explanations of observations. The analysis of free will cannot profit from relating free will to the deterministic or stochastic nature of our universe. Free will is an impression of one individual about another one due to the inability of a highly complex mind to create a model of another equally complex mind that would generate reliable predictions.

1.3.2 Critique of frequency and subjective philosophies of probability In a nutshell, each of the two most popular philosophies of probability, frequency and subjective, failed in two distinct ways. First, both theories are very weak. The frequency philosophy of von Mises, developed in the first half of the 20th century, provides analysis of long sequences of independent and identical events only. The subjective philosophy of de Finetti (developed in parallel to that of von Mises, more or less) offers an argument in support of the mathematical rules of probability, with no hint on how the rules can be matched with the real world. Second, each of the two philosophical theories failed in a “technical” sense. The frequency

page 4

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch01

Introduction

5

theory is based on “collectives,” a notion that was completely abandoned by the scientific community long time ago. The subjective theory is based on an argument which fails to give any justification for the use of the Bayes theorem. Even one of the two types failures would be sufficient to disqualify these theories. The double failure makes each of the theories an embarrassment for the scientific community. The philosophical contents of the theories of von Mises and de Finetti may be split into (i) positive philosophical ideas, (ii) negative philosophical ideas, and (iii) innovative technical ideas. There is nothing new about the positive philosophical ideas in either theory. The negative philosophical ideas are pure fantasy. The technical ideas proved to be completely useless. I will now discuss these elements of the two theories in more detail.

1.3.2.1 Positive philosophical ideas The central idea in the frequentist view of the world is that probability and (relative) frequency can be identified, at least approximately, and at least in propitious circumstances. It is inevitable that, at least at the subconscious level, von Mises is credited with the discovery of the close relationship between probability and frequency. Nothing can be further from the truth. At the empirical level, one could claim that a relationship between probability and frequency is known even to animals, and was certainly known to ancient people. The mythical beginning of the modern probability theory was an exchange of ideas between Chevalier de Mere, a gambler, Pierre de Fermat and Blaise Pascal, two mathematicians, in 1654. It is clear from the context that Chevalier de Mere identified probabilities with frequencies and the two mathematicians developed algebraic formulas. On the theoretical side, the approximate equality of relative frequency and probability of an event is known as the Law of Large Numbers. An early version of this mathematical theorem was proved by Jacob Bernoulli in 1713. The main philosophical and scientific ideas associated with subjectivism and Bayesian statistics are, obviously, the Bayes theorem and the claim that probability is a personal opinion. Once again, one can subconsciously give credit to de Finetti for discovering the Bayes theorem or for inventing the idea that probability is a subjective opinion. The Bayes theorem was proved by Thomas Bayes, of course, and published in 1763 (although it appears that the theorem was known before Bayes). De Finetti was not the first person to suggest that the Bayes theorem should be used in science and

page 5

April 29, 2016

9:6

6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch01

Resonance: From Probability to Epistemology and Back

other avenues of life, such as the justice system. In fact, this approach was well known and quite popular in the 19th century. Between Newton and Einstein, the unquestioned scientific view of the world was that of a clockwise mechanism. There was nothing random about the physical processes. Einstein himself was reluctant to accept the fact that quantum mechanics was inseparable from randomness. Hence, before the 20th century, probability was necessarily an expression of limited human knowledge of reality. Many details of de Finetti’s theory of subjective probability were definitely new but the general idea that probability was a personal opinion was anything but new.

1.3.2.2 Negative philosophical ideas Both von Mises and de Finetti took, as a starting point, a very reasonable observation that not all everyday uses of the concept of probability deserve to be elevated to the status of science. A good example to have in mind is the concept of “work” which is very useful in everyday life but had to be considerably modified to be equally useful in physics. One of the greatest challenges for a philosopher of probability is the question of how to measure the probability of a given event. Common sense suggests observing the frequency of the event in a sequence of similar experiments, or under similar circumstances. It is disappointing that quite often there is no obvious choice of “similar” observations, for example, if we want to find the probability that a given presidential candidate will win the elections. Even when we can easily generate a sequence of identical experiments, all we can get is the relative frequency which characterizes the whole sequence, not any particular event. The observed frequency is not necessarily equal to the true probability (if there is such a thing), according to the mathematical theory of probability. The observed frequency is highly probable to be close to the true probability, but applying this argument seems to be circular — we are using the concept of probability (“highly probable”) before we determined that the concept is meaningful. Von Mises and de Finetti considered philosophical difficulties posed by the measurement of probability of an event and concluded that a single event does not have a probability. This intellectual decision was similar to that of a philosopher coming to the conclusion that God does not exist because the concept of God is mired in logical paradoxes. The atheist

page 6

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch01

Introduction

7

philosophical option has a number of intellectual advantages — one does not have to think about whether God can make a stone so heavy that He cannot lift it himself. More significantly, one does not have to resolve the apparent contradiction between God’s omnipotence and omnibenevolence on one hand, and all the evil in the world on the other. Likewise, von Mises and de Finetti do not have to explain how one can measure the probability of a single event. While the philosophical position of von Mises and de Finetti is very convenient, it also makes their philosophies totally alienated from science and other branches of life. In practical life, all people have to assign probabilities to single events and they have to follow rules worked out by probabilists, statisticians and other scientists. Declaring that a single event does not have a probability has as much practical significance as declaring that complex numbers do not exist. The claim that “God does not exist” is a reasonable philosophical option. The claim that “religion does not exist” is nonsensical. The greatest philosophical challenge in the area of probability is a probabilistic counterpart of the question “What does a particular religion say?” This challenge is deceptively simple — philosophers found it very hard to pinpoint what the basic rules for assigning probabilities are. This is exemplified by some outright silly proposals by the “logical” school of probability. While other philosophers tried to extend the list of basic rules of probability, von Mises and de Finetti removed some items from the list, most notably symmetry. The fundamental philosophical claim of von Mises and de Finetti, that events do not have probabilities, was like a straitjacket that tied their hands and forced them to develop very distinct but equally bizarre theories. Their fundamental claim cannot be softened or circumvented. For a philosopher, it is impossible to be an atheist and believe in God just a little bit. Creating a philosophical theory of God that exists just a little bit is not any easier than creating a theory of God that fully exists. Similarly, creating a philosophy of probability which includes some events with a somewhat objective probability is as hard as inventing a philosophy claiming that all events have fully objective probability. The two philosophies can be considered normative. Then their failure manifests itself in the fact that they are totally ignored. If the two theories are regarded as descriptive then they are complete failures because the two philosophers proved unable to make simple observations.

page 7

April 29, 2016

9:6

8

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch01

Resonance: From Probability to Epistemology and Back

1.3.2.3 Innovative technical ideas Von Mises came to the conclusion that the only scientific application of probability was in the context of long sequences of identical experiments or observations. Nowadays, such sequences are modeled mathematically by “i.i.d.” random variables (i.i.d. is an acronym for “independent identically distributed”). Since individual events do not have probabilities in the von Mises’ view of the world, one cannot decide in any way whether two given elements of the sequence are independent, or have identical distribution. Hence, von Mises invented a notion of a “collective,” a mathematical formalization of the same class of real sequences. Collectives are sequences in which the same stable frequencies of an event hold for all subsequences chosen without prophetic powers. Collectives have been abandoned by scientists long time ago. One of the basic theorems about i.i.d. sequences that scientists like to use is the Central Limit Theorem. I do not know whether this theorem was proved for collectives and I do not think that there is a single scientist who would like to know whether it was. De Finetti proposed to consider probability as a purely mathematical technique that can be used to coordinate families of decisions, or to make them “consistent.” This idea may be interpreted in a more generous or less generous way. The more generous way is to say that de Finetti had nothing to say about the real practical choices between innumerable consistent decision strategies. The less generous way is to say that he claimed that all consistent probability assignments were equally good. In practice, taking the last claim seriously would lead to chaos. The second significant failure of de Finetti’s idea is that in a typical statistical situation, there are no multiple decisions to be coordinated. And finally and crucially, I will show that de Finetti’s theory cannot justify the Bayes theorem — the essence of Bayesian statistics. De Finetti’s theory applies only to a handful of artificial textbook examples, and only those where no data are collected.

1.3.3 Scientific laws of probability I will argue that the following laws are the de facto standard of applications of probability in all sciences. (L1) Probabilities are numbers between 0 and 1 (including 0 and 1), assigned to events whose outcome may be unknown.

page 8

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch01

Introduction

9

(L2) If events A and B cannot happen at the same time then the probability that A or B will occur is the sum of the probabilities of the individual events, that is, P (A or B) = P (A) + P (B). (L3) If events A and B are physically independent then they are independent in the mathematical sense, that is, P (A and B) = P (A)P (B). (L4) If events A and B are symmetric then the two events have equal probabilities, that is, P (A) = P (B). (L5) When an event A is observed then the probability of B changes from P (B) to P (A and B)/P (A). (L6) An event has probability 0 if and only if it cannot occur. An event has probability 1 if and only if it must occur. The shocking aspect of the above laws is the same as in “the Emperor has no clothes.” There is nothing new about the laws — they are implicit in all textbooks. The laws (L1)–(L6) provide a codification of the science of probability at the same level as laws known in some fields of physics, such as thermodynamics or electromagnetism. People familiar with the probability theory at the college level will notice that (L1)–(L6) are a concise summary of the first few chapters of any standard undergraduate probability textbook. It is surprising that probabilists and statisticians, as a community, cling to odd philosophical theories incompatible with (L1)– (L6), and at the same time they teach (L1)–(L6) implicitly, using examples. I will argue that both frequency statistics and Bayesian statistics fit quite well within the framework of (L1)–(L6). The laws (L1)–(L6) include ideas from the “classical” philosophy of probability and Popper’s “falsifiability” approach to science in the probabilistic context. Hence, the laws can hardly be called new. However, I am not aware of any published system of probability laws that are equally simple and match the contents of current textbooks equally well.

1.3.4 Statistics and philosophy I will argue that frequency statistics has nothing (essential) in common with the frequency philosophy of probability and Bayesian statistics has nothing (essential) in common with the subjective philosophy of probability. The two branches of statistics and the two corresponding philosophical theories have roots in the same intuitive ideas based on everyday observations. However, the intellectual goals of science and philosophy pulled the developing theories apart. The basic intuition behind the frequency

page 9

April 29, 2016

9:6

10

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch01

Resonance: From Probability to Epistemology and Back

statistics and the frequency philosophy of probability derives from the fact that frequencies of some events appear to be stable over long periods of time. For example, stable frequencies have been observed by gamblers playing with dice. Stable frequencies are commonly observed in biology, for example, the percentage of individuals with a particular trait is often stable within a population. The frequency philosophy of probability formalizes the notion of stable frequency but it does not stop here. It makes an extra claim that the concept of probability does not apply to individual events. This claim is hardly needed or noticed by frequency statisticians. They need the concept of frequency to justify their computations performed under the assumption of a “fixed but unknown” parameter (implicitly, a physical quantity). Hence, frequency statisticians turned von Mises’ philosophy on its head. Von Mises’ philosophy can be summarized by saying that “If you have an observable sequence, you can apply probability theory.” Frequency statisticians transformed this claim into “If you have a probability statement, you can interpret it using long run frequency.” There are several intuitive sources of Bayesian statistics and the subjective philosophy of probability. People often feel that some events are likely and other events are not likely to occur. People have to make decisions in uncertain situations and they believe that despite the lack of deterministic predictions, some decision strategies are better than others. People “learn” when they make new observations, in the sense that they change their assessment of the likelihood of future events. The subjective philosophy of probability formalizes all these intuitive ideas and observable facts but it also makes an extra assertion that there is no objective probability. The last claim is clearly an embarrassment for Bayesian statisticians so they rarely mention it. Their scientific method is based on a mathematical result called the Bayes theorem. The Bayes theorem and Bayesian statistics are hardly related to the subjective philosophy. Just like frequency statisticians, Bayesian statisticians turned a philosophy on its head. A brief summary of de Finetti’s philosophy is “No matter how much information you have, there is no scientific method to assign a probability to an event.” Bayesian statisticians transformed this claim into “No matter how little information you have, you can assign a probability to an event in a scientifically acceptable way.” Some Bayesian statisticians feel that they need the last claim to justify their use of prior distributions.

page 10

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch01

Introduction

11

I do not see anything absurd in using the frequency and subjective interpretations of probability as mental devices that help people do abstract research and apply probability in real life. Frequency statisticians use probability outside the context of long runs of experiments or observations, but they may imagine long runs of experiments or observations, and doing this may help them conduct research. In this sense, the frequency theory is a purely philosophical theory — some people regard long run frequency as the true essence of probability and this conviction may help them apply probability even in those situations where no real long runs of experiments exist. Some Bayesian statisticians consider probability to be a tool used for coordination of decisions in a rational way, in agreement with the philosophical theory of de Finetti. All Bayesian statisticians apply probability irrespective of whether there is a need to coordinate any decisions. Bayesian statisticians may believe that coordination of decisions is the essence of probability and this purely philosophical belief may help them conduct research.

1.4 Historical and Social Context In order to avoid unnecessary controversy and misunderstanding, it is important for me to say what claims I do not make. The controversy surrounding probability has at least two axes, a scientific axis and a philosophical axis. The two controversies were often identified in the past, sometimes for good reasons. I will not discuss the scientific controversy, that is, I will not take any position in support of one of the branches of the science of statistics, frequency or Bayesian; this is a job for statisticians and other scientists using statistics. I will limit myself to the following remarks. Both frequency statistics and Bayesian statistics are excellent scientific theories. This is not a judgment of any particular method proposed by any of these sciences in a specific situation — all sciences are more successful in some circumstances than others, and the two branches of statistics are not necessarily equally successful in all cases. My judgment is based on the overall assessment of the role of statistics in our civilization, and the perception of its value among its users. A reader not familiar with the history of statistics may be astounded by the audacity of my criticism of the frequency and subjective philosophical

page 11

April 29, 2016

9:6

12

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch01

Resonance: From Probability to Epistemology and Back

theories of probability. In fact, there is nothing new about it, except that some of my predecessors were not so bold in their choice of language. Countless arguments against the frequency and subjective philosophies were advanced in the past and much of the material in this book consists of a new presentation of known ideas. I will be mostly concerned with the substance of philosophical claims and their relationship with statistics. One is tempted, though, to ask why it is that thousands of statisticians seem to be blind to apparently evident truth. Why did philosophical and scientific theories, rooted in the same elementary observations, develop in directions that are totally incompatible? Although these questions are only weakly related to the main philosophical arguments in this book, I will now attempt to provide a brief diagnosis. Statisticians have been engaged for a long time in a healthy, legitimate and quite animated scientific dispute concerning the best methods to analyze data. Currently, the competition is viewed as a rivalry between “frequency” and “Bayesian” statistics but this scientific controversy precedes the crystallization of these two branches of statistics into well defined scientific theories in the second half of the 20th century. An excellent book [Howie (2002)] is devoted to the dispute between Fisher and Jeffreys, representing competing statistical views, at the beginning of the 20th century. The scientific dispute within statistics was always tainted by philosophical controversy. Some statisticians considered understanding philosophical aspects of probability to be vitally important to scientific success of the field. My impression, though, is that philosophy was and is treated in a purely instrumental way by many, perhaps most, statisticians. They are hardly interested in philosophical questions such as whether probability is an objective quantity. They treat ideology as a weapon in scientific discussions, just like many politicians treat religion as a weapon during a war. Most statisticians find little time to read and think about philosophy of probability and they find it convenient to maintain superficial loyalty to the same philosophy of probability that other statisticians in the same branch of statistics profess. Moreover, many statisticians feel that they have no real choice. They may feel that their own philosophy of probability might be imperfect but they do not find any alternative philosophy more enticing. Philosophers and statisticians try to understand the same simple observations, such as more or less stable frequency of girls among babies,

page 12

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch01

Introduction

13

or people’s beliefs about the stock market direction. Philosophy and science differ not only in that they use different methods but they also have their own intellectual goals. Statisticians are primarily interested in understanding complex situations involving data and uncertainty. Philosophers are trying to determine the nature of the phenomenon of probability and they are content with deep analysis of simple examples. It is a historical accident that frequency statistics and the frequency philosophy of probability developed at about the same time and they both involved some frequency ideas. These philosophical and scientific theories diverged because they had different goals and there was no sufficient interest in coordinating the two sides of the frequency analysis — it was much easier for statisticians to ignore the inconvenient claims of the frequency philosophy. The same can be said, more or less, about Bayesian statistics. The roots of Bayesian statistics go back to Thomas Bayes in the 18th century but its modern revival coincides, roughly, with the creation of the subjective philosophy of probability. The needs of philosophy and science pushed the two intellectual currents in incompatible directions but scientists preferred to keep their eyes shut rather than to admit that Bayesian statistics had nothing in common with the subjective philosophy. One of my main theses is that the original theories of von Mises and de Finetti are completely unrelated to statistics and totally unrealistic. So, why bother to discuss them? It is because they are the only fully developed and mostly logically consistent intellectual structures, one based on the idea that probabilities are frequencies, and the other one based on the idea that probabilities are subjective opinions. Both assert that individual events do not have probabilities. Some later variants of these theories were less extreme in their assertions and hence more palatable. But none of these variants achieved the fame of the original theories, and for a good reason. The alternative versions of the original theories are often focused on arcane philosophical points and muddle the controversial but reasonably clear original ideas.

1.5 Disclaimers I will cite many sources in this book but I am not able to trace every one of my philosophical claims to an earlier philosophy or scientific theory. I believe that the concept of “resonance” is my original contribution but my “resonance” theory is clearly an amalgam of various known philosophical

page 13

April 29, 2016

9:6

14

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch01

Resonance: From Probability to Epistemology and Back

and scientific ideas. So, the value of the resonance theory, if any, lies in bringing various elements together. I had doubts about the value of and need for my philosophical project. But I found so much nonsense and hypocrisy in various supposedly respectable philosophical writings that I came to the conclusion that a new dose of common sense in the philosophical literature is needed, even if it is partly repetitive, to buttress the camp of reason. The philosophical material to which I refer is easily accessible and well organized in books and articles. Both de Finetti and von Mises wrote major books with detailed expositions of their theories. These were followed by many commentaries. I felt that these writings were often contradictory and confusing but I had enough material to form my own understanding of the frequency and subjective philosophical theories. Needless to say, this does not necessarily imply that my understanding is correct and my very low opinion about the two theories is justified. If any of my claims are factually incorrect, I have nobody but myself to blame. When it comes to statistics, the situation is much different. On the purely mathematical side, both frequency and Bayesian statistics are very clear. However, the philosophical views of professional statisticians span a whole spectrum of opinions, from complete indifference to philosophical issues to fanatical support for the extreme interpretation of one of the two popular philosophies. For this reason, whenever I write about statisticians’ views or practices, I necessarily have to choose positions that I consider typical. I regret any misrepresentation of statisticians’ philosophical positions, overt or implied. I feel that I have to make another explicit disclaimer, so that I am not considered ignorant and rude (at least not for the wrong reasons). Both von Mises and de Finetti were not only philosophers but also scientists. My claim that their ideas are complete intellectual failures refers only to their philosophical theories. Their scientific contributions are quite solid. For example, de Finetti’s representation of exchangeable sequences as mixtures of i.i.d. sequences is one of the most beautiful and significant theorems in the mathematical theory of probability. I end the introduction with an explanation of the usage of a few terms, because readers who are not familiar with probability and statistics might be confused when I refer to “philosophy of probability” as a foundation for statistics rather than probability. I am a “probabilist.” Among my colleagues, this word refers to a mathematician whose focus is a field of

page 14

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch01

Introduction

15

mathematics called “probability.” The probability theory is applied in all natural sciences, social sciences, business, politics, etc., but there is only one field of natural science (as opposed to the deductive science of mathematics) where probability is the central object of study and not just a tool — this field is called “statistics.” For historical reasons, the phrase “philosophy of probability” often refers to the philosophical and scientific foundations of statistics.

page 15

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch02

Chapter 2

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Main Philosophies of Probability

My general classification of the main philosophies of probability is borrowed from [Galavotti (2005); Gillies (2000); Weatherford (1982)]. Some authors pointed out that even the classification of probability theories is rife with controversy, so the reader should not be surprised to find a considerably different list in [Fine (1973)]. I will present only these versions of popular philosophical theories of probability which I consider clear. In other words, this chapter plays a double role. It is a short introduction to the philosophy of probability for those who are not familiar with it. It is also my attempt to clarify the basic claims of various theories. I think that I am faithful to the spirit of all theories that I present but I will make little effort to present the nuances of their various interpretations. In particular, I will discuss only this version of the frequency theory that claims that probability is not an attribute of a single event. This is because I believe that the need for the concept of a “collective” used in this theory is well justified only when we adopt this assumption. Similarly, I will not discuss the views of philosophers who claim that both objective and subjective probabilities exist. I do not see how one can construct a coherent and convincing theory including both objective and subjective probabilities — see Sec. 5.17. I will pay much more attention to the subjective and frequency theories than to other theories, because these two theories are widely believed to be the foundation of modern statistics and other applications of probability. I will discuss less popular philosophies of probability first.

19

page 19

April 29, 2016

9:6

20

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch02

Resonance: From Probability to Epistemology and Back

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

2.1 The Classical Theory Traditionally the birth of the modern mathematics-based probability theory is dated back to the correspondence between Pierre de Fermat and Blaise Pascal in 1654. They discussed a problem concerning dice posed by Chevalier de Mere, a gambler. In fact, some calculations of probabilities can be found in earlier books (see Chapter 1 of [Gillies (2000)] for more details). The “classical” definition of probability gives a mathematical recipe for calculating probabilities in highly symmetric situations, such as tossing a coin, rolling a die or playing cards. It does not seem to be concerned with the question of the “nature” of probability. In 1814, Laplace stated the definition in these words (English version after ([Gillies (2000), p. 17])): The theory of chance consists in reducing all the events of the same kind to a certain number of cases equally possible, that is to say, to such as we may be equally undecided about in regard to their existence, and in determining the number of cases favorable to the event whose probability is sought. The ratio of this number to that of all the cases possible is the measure of this probability, which is thus simply a fraction whose numerator is the number of favorable cases and whose denominator is the number of all the cases possible.

Since the definition applies only to those situations in which all outcomes are (known to be) equally “possible,” it does not apply to a single toss or multiple tosses of a deformed coin. The definition does not make it clear what one should think about an experiment with a deformed coin — does the concept of probability apply to that situation at all? The classical definition seems to be circular because it refers to “equally possible” cases — this presumably means “equally probable” cases — and so probability is defined using the notion of probability. The “classical philosophy of probability” is a modern label. That “philosophy” was a practical recipe and not a conscious attempt to create a philosophy of probability, unlike all other philosophies reviewed below. They were developed in the 20th century, partly in parallel.

2.2 The Logical Theory The “logical” theory of probability maintains that probabilities are numbers representing relations between sentences. They are weak forms of logical implication, intuitively speaking. According to this theory, the study of

page 20

May 10, 2016

13:19

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch02

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Main Philosophies of Probability

21

probability is a study of a (formal) language. John Maynard Keynes and, later, Rudolf Carnap were the most prominent representatives of this philosophical view. Their main books were [Keynes (1921)] and [Carnap (1950)]. The version of the theory advocated by Keynes allowed for nonnumerical probabilities. The logical theory is based on the “principle of indifference” which asserts that, informally speaking, equal probabilities should be assigned to alternatives for which no reason is known to be different. The principle is also known under the name of the “principle of insufficient reason” and goes back to Laplace. It was criticized as early as in 1854 by Boole who thought that our ignorance is not a good justification for assigning equal probabilities to events. The principle of indifference does not have a unique interpretation. If you toss a deformed coin twice, what is the probability that the results will be different? There are four possible results: HH, T H, HT and T T (H stands for heads, T stands for tails). The principle of indifference suggests that all four results are equally likely so the probability that the results will be different is 1/2. A generalization of this claim to a large number n of tosses says that all sequences of outcomes are equally likely. A simple mathematical argument then shows that the tosses are (mathematically) independent and the probability of heads is 1/2 for each toss. Since this conclusion is not palatable, Keynes and Carnap argued that the probability that the results of the first two tosses will be different should be taken as 1/3. This claim and its generalizations are mathematically equivalent to choosing the “uniform prior” in the Bayesian setting. In other words, we should assume that the tosses are independent and identically distributed with the probability of heads that is itself a random variable — a number chosen uniformly from the interval [0, 1]. The logical theory seems to be almost unknown among mathematicians, probabilists and statisticians. The emphasis on the logical aspect of probability seems to miss the point of the real difficulties with this concept. Statisticians and scientists seem to be quite happy with Kolmogorov’s mathematical theory as the formal basis of probability. Almost all of the controversy is concerned with the implementation of the formal theory in practice. The boundaries between different philosophies are not sharp. For example, Carnap believed in two different concepts of probability, one appropriate for logic, and another one appropriate for physical sciences. The logical theory is also known as a “necessary” or “a priori” interpretation of probability.

page 21

April 29, 2016

9:6

22

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch02

Resonance: From Probability to Epistemology and Back

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

2.3 The Propensity Theory The term “propensity theory” is applied to recent philosophical theories of probability which consider probability to be an objective property of things or experiments just like mass or electrical charge. Karl Popper developed the first version of the propensity theory (see [Popper (1968)]). The following example illustrates a problem with this interpretation of probability. Suppose a company manufactures identical computers in plants in Japan and Mexico. The propensity theory does not provide a convincing interpretation of the statement “This computer was made in Japan with probability 70%,” because it is hard to imagine what physical property this sentence might refer to. Popper advanced another philosophical idea, namely, that one can falsify probabilistic statements that involve probabilities very close to 0 or 1. He said ([Popper (1968), Sec. 68, p. 202]), The rule that extreme improbabilities have to be neglected [...] agrees with the demand for scientific objectivity.

This idea is implicit in all theories of probability and one form of it was stated by Cournot in the first half of the 19th century (quoted after ([Primas (1999), p. 585])): If the probability of an event is sufficiently small, one should act in a way as if this event will not occur at a solitary realization.

Popper’s proposal did not gain much popularity in the probabilistic and statistical community, most likely because it was not translated into a usable scientific “law of nature” or a scientific method. A version of Popper’s idea appears in this book as (L6) — an essential part of my own theory. Popper’s two philosophical proposals in the area of probability, that probability is a physical property, and that probability statements can be falsified, seem to be independent, in the sense that one could adopt only one of these philosophical positions.

2.4 The Subjective Theory Two people arrived independently at the idea of the subjective theory of probability in 1930’s, Frank Ramsey and Bruno de Finetti. Ramsey did not live long enough to develop fully his theory so de Finetti was the founder and the best known representative of this school of thought.

page 22

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch02

Main Philosophies of Probability

23

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

The “subjective” theory of probability identifies probabilities with subjective opinions about unknown events. This idea is deceptively simple. First, the word “subjective” is ambiguous so I will spend a lot of time trying to clarify its meaning in the subjective philosophy. Second, one has to address the question of why the mathematical probability theory should be used at all, if there is no objective probability. The subjective philosophy is also known as the “personal” approach to probability.

2.4.1 Interpreting subjectivity De Finetti emphatically denied existence of any objective probabilistic statements or objective quantities representing probability. He summarized this in his famous saying “Probability does not exist.” This slogan and the claim that “probability is subjective” are terribly ambiguous and lead to profound misunderstandings. Here are four interpretations of the slogans that come naturally to my mind. (i) Although most people think that coin tosses and similar long run experiments displayed some patterns in the past, scientists determined that those patterns were figments of imagination, just like optical illusions. (ii) Coin tosses and similar long run experiments displayed some patterns in the past but those patterns are irrelevant to the prediction of any future event. (iii) The results of coin tosses will follow the pattern I choose, that is, if I think that the probability of heads is 0.7 then I will observe roughly 70% of heads in a long run of coin tosses. (iv) Opinions about coin tosses vary widely among people. Each one of the above interpretations is false in the sense that it is not what de Finetti said or what he was trying to say. The first interpretation involves “patterns” that can be understood in both objective and subjective sense. De Finetti never questioned the fact that some people noticed some (subjective) patterns in the past random experiments. De Finetti argued that people should be “consistent” in their probability assignments (I will explain the meaning of consistency momentarily). That recommendation never included a suggestion that the (subjective) patterns observed in the past should be ignored in making one’s own subjective predictions of the future, so (ii) is not a correct interpretation of de Finetti’s ideas either.

page 23

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

24

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch02

Resonance: From Probability to Epistemology and Back

Clearly, de Finetti never claimed that one can affect future events just by thinking about them, as suggested by (iii). We know that de Finetti was aware of the clustering of people’s opinions about some events, especially those in science, because he addressed this issue in his writings, so again (iv) is a false interpretation of the basic tenets of the subjective theory. I have to add that I will later argue that the subjective theory contains implicitly assertions (i) and (ii). The above list and its discussion were supposed to convince the reader that interpreting subjectivity is much harder than one may think. A more complete review of various meanings of subjectivity will be given in Secs. 5.1 and 5.23. The correct interpretation of “subjectivity” of probability in de Finetti’s theory requires some background. The necessity of presenting this background is a good pretext to review some problems facing the philosophy of probability. Hence, the next section will be a digression in this direction.

2.4.2 Verification of probabilistic statements The mathematics of probability was never very controversial. The search for a good set of mathematical axioms for the theory took many years, until Kolmogorov came up with an idea of using measure theory in 1933. But even before then, the mathematical probability theory produced many excellent results. The challenge always lay in connecting the mathematical results and real life events. In a nutshell, how do you determine the probability of an event in real life? If you make a probabilistic statement, how do you verify whether it is true? It is a good idea to have in mind a concrete elementary example — a deformed coin. What is the probability that it will fall heads up? Problems associated with this question and possible answers span a wide spectrum from practical to purely philosophical. Let us start with some practical problems. A natural way to determine the probability of heads for the deformed coin would be to toss the coin a large number of times and take the relative frequency of heads as the probability. This procedure is suggested by the Law of Large Numbers, a mathematical theorem. The first problem is that, in principle, we would have to toss the coin an infinite number of times. This, of course, is impossible, so we have to settle for a “large” number of tosses. How large should “large” be? Another practical problem is that a single event is often a member of two (or more) “natural” sequences. The experiment of tossing a deformed

page 24

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch02

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Main Philosophies of Probability

25

coin is an element of the sequence of tosses of the same deformed coin, but it is also an element of the sequence of experiments consisting of deforming a coin (a different coin every time) and then tossing it. It is possible that the frequency of heads will be 30% in the first sequence (because of the lack of symmetry) but it will be 50% in the second sequence (by symmetry). People who may potentially donate money to a presidential candidate may want to know the probability that John Smith, currently a senator, will win the elections. The obvious practical problem is that it may be very hard to find a real sequence that would realistically represent Smith’s probability of winning. For example, Smith’s track record as a politician at a state level might not be a good predictor of his success at the national level. One could try to estimate the probability of Smith’s success by running a long sequence of computer simulations of elections. How can we know whether the model implemented in the computer program accurately represents this incredibly complex problem? On the philosophical side, circularity is one of the problems lurking when we try to define probability using long run frequencies. Even if we toss a deformed coin a “large” number of times, it is clear that the relative frequency of heads is not necessarily equal to the probability of heads on a single toss, but it is “close” to it. How close is “close”? One can use a mathematical technique to answer this question. We can use the collected data to find a 95% “confidence interval,” that is, an interval that covers the true value of the probability of heads with probability 95%. This probabilistic statement is meaningful only if we can give it an operational meaning. If we base the interpretation of the confidence interval on the long run frequency idea, this will require constructing a long sequence of confidence intervals. This leads either to an infinite regress (sequence of sequences of sequences, etc.) or to a vicious circle of ideas (defining probability using probability). Another philosophical problem concerns the relationship between a sequence of events and a single element of the sequence. If we could perform an infinite number of experiments and find the relative frequency of an event, that would presumably give us some information about other infinite sequences of similar experiments. But would that provide any information about any specific experiment, say, the seventh experiment in another run? In other words, can the observations of an infinite sequence provide a basis for the verification of a probability statement about any single event? Suppose that every individual event has a probability. If we toss a deformed coin 1,000 times, we will observe only one quantity, the relative

page 25

April 29, 2016

9:6

26

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch02

Resonance: From Probability to Epistemology and Back

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

frequency of heads. This suggests that there is only one scientific quantity involved in the whole experiment. This objection can be answered by saying that all individual events have the same probability. But if we assume that all individual probabilities are the same, what is the advantage of treating them as 1,000 different scientific quantities, rather than a single one?

2.4.3 Subjectivity as an escape from the shackles of verification The previous section should have given the reader a taste of the nasty philosophical and practical problems related to the verification of probability statements. The radical idea of de Finetti was to get rid of all these problems in one swoop. He declared that probability statements cannot be verified at all — this is the fundamental meaning of subjectivity in his philosophical theory. This idea can be presented as a great triumph of thought or as a great failure. If you are an admirer of de Finetti, you may emphasize the simplicity and elegance of his solution of the verification problem. If you are his detractor, you may say that de Finetti could not find a solution to a philosophical problem, so he tried to conceal his failure by declaring that the problem was ill-posed. De Finetti’s idea was fascinating but, alas, many fascinating ideas cannot be made to work. This is what I will show in Chapter 5. I will now offer some further clarification of de Finetti’s ideas. Probability statements are “subjective” in de Finetti’s theory in the sense that “No probability statement is verifiable or falsifiable in any objective sense.” Actually, according to de Finetti, probability statements are not verifiable in any sense, “subjective” or “objective.” In his theory, when new information is available, it is not used to verify any probability statements made in the past. The subjective probabilities do not change at all — the only thing that happens is that one starts to use different probabilities, based on the old and new information. This does not affect the original probability assignments, except that they become irrelevant for making decisions — they are not falsified, according to de Finetti. The observation of the occurrence of an event or its complement cannot falsify or verify any statement about its probability. One of the most important aspects of de Finetti’s interpretation of subjectivity, perhaps the most important aspect, is that his philosophical theory is devoid of any means whatsoever of verifying any probability statement. This extreme position, not universally adopted by subjectivists,

page 26

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch02

Main Philosophies of Probability

27

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

is an indispensable element of the theory; I will discuss this further in Chapter 5. A good illustration of this point is the following commentary of de Finetti on the fact that beliefs in some probability statements are common to all scientists, and so they seem to be objective and verifiable (quoted after ([Gillies (2000), p. 70])): Our point of view remains in all cases the same: to show that there are rather profound psychological reasons which make the exact or approximate agreement that is observed between the opinions of different individuals very natural, but there are no reasons, rational, positive, or metaphysical, that can give this fact any meaning beyond that of a simple agreement of subjective opinions.

A similar idea was expressed by Leonard Savage, the second best known founder of the subjective philosophy of probability after de Finetti. The following passage indicates that he believed in the impossibility of determining which events are probable or “sure” in an objective way (page 58 of [Savage (1972)]): When our opinions, as reflected in real or envisaged action, are inconsistent, we sacrifice the unsure opinions to the sure ones. The notion of “sure” and “unsure” introduced here is vague, and my complaint is precisely that neither the theory of personal probability, as it is developed in this book, nor any other device known to me renders the notion less vague.

The subjective theory is rich in ideas — no sarcasm is intended here. In the rest of this section, I will discuss some of these ideas: the “Dutch book” argument, the axiomatic system for the subjective theory, the identification of probabilities and decisions, and the Bayes theorem.

2.4.4 The Dutch book argument Probability does not exist in an objective sense, according to the subjective theory, so why should we use the probability calculus at all? One can justify the application of the mathematical theory of probability to subjective probabilities using a “Dutch book” argument. A Dutch book will be formed against me if I place various bets in such a way that no matter which events occur and which do not occur, I will lose some money. One can prove in a rigorous way that it is possible to make a Dutch book against a person if and only if the “probabilities” used by the person are not “consistent,” that is, they do not satisfy the usual formulas of the mathematical probability theory.

page 27

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

28

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch02

Resonance: From Probability to Epistemology and Back

I will illustrate the idea of a Dutch book with a simple example. See Sec. 18.1 for the definition of expectation and other mathematical concepts. Consider an experiment with only three possible mutually exclusive outcomes A, B and C. For example, these events may represent winners of a race with three runners. The mathematical theory of probability requires that the probabilities of A, B and C are non-negative and add up to 1, that is, P (A) + P (B) + P (C) = 1. The complement of an event A is traditionally denoted Ac , that is, Ac is the event that A did not occur, and the mathematical theory of probability requires that P (Ac ) = 1 − P (A). Suppose that I harbor “inconsistent” views, for example, my personal choice of probabilities is P (A) = P (B) = P (C) = 0.9, so that P (A) + P (B) + P (C) > 1. Since I am 90% sure that A will happen, I am willing to pay someone $0.85, assuming that I will receive $1.00 if A occurs (and nothing otherwise). Placing this bet seems to be rational because the expected gain is equal to $0.15 · P (A) − $0.85 · P (Ac ) = $0.15 · 0.9 − $0.85 · 0.1 = $0.05, a strictly positive number. A similar calculation shows that I should also accept two analogous bets, with A replaced by B and C. If I place all three bets, I will have to pay $0.85 + $0.85 + $0.85 = $2.55. Only one of the events A, B or C may occur. No matter which event occurs, A, B or C, I will receive the payoff equal to $1.00 only. In each case, I am going to lose $1.55. A Dutch book was formed against me because I did not follow the usual rules of probability, that is, I used “probabilities” that did not satisfy the condition P (A) + P (B) + P (C) = 1. Consistency protects me against creating a situation resulting in certain loss so I have to use the mathematics of probability in my judgments, the subjective theory advises. Note that the claim here is not that inconsistency will necessarily result in a Dutch book situation (in a given practical situation, there may be no bets offered to me), but that consistency protects me against the Dutch book situation under all circumstances. The essence of the Dutch book argument is that one can achieve a deterministic and empirically verifiable goal using probability calculus, without assuming anything about existence of objective probabilities. Savage proposed that consistency is the essence of probability (page 57 of [Savage (1972)]): According to the personalistic view, the role of the mathematical theory of probability is to enable the person using it to detect inconsistencies in his own real or envisaged behavior.

page 28

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch02

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Main Philosophies of Probability

29

The idea of a “Dutch book” seems to be very close to the idea of “arbitrage” in modern mathematical finance. An arbitrage is a situation in financial markets when an investor can make a positive profit with no risk. The definition refers to the market of financial instruments, such as stocks, bonds and options. Financial theorists commonly assume that there is no arbitrage in real financial markets. If a person has inconsistent probabilistic views then someone else can use a Dutch book against the person to make a profit with no risk — just like in a market that offers arbitrage opportunities. For a more complete discussion of this point, see Sec. 5.18.

2.4.5 The axiomatic system The subjective theory of probability is sometimes introduced using an axiomatic system, as in [DeGroot (1970)] or [Fishburn (1970)]. This approach gives the subjective theory of probability the flavor of a mathematical (logical, formal) theory. The postulates are intuitively appealing, even obvious — just what one would expect from axioms. One could argue that logical consistency is a desirable intellectual habit with good practical consequences but there exist some mathematical theories, such as non-Euclidean geometries, which do not represent anything real (at the human scale in ordinary life). Hence, adopting a set of axioms does not guarantee a success in practical life — one needs an extra argument, such as empirical verification, to justify the use of any given set of axioms. The subjective theory claims that statements about probability values cannot be verified (because probability does not exist in an objective sense) so this leaves the Dutch book argument as the only subjectivist justification for the use of the mathematical rules of probability and the implementation of the axiomatic system. The importance of the axiomatic system to (some) subjectivists is exemplified by the following challenge posed by Dennis Lindley in his review [Math Review MR0356303 (50 #8774a), 1975] of [DeGroot (1970)]: Many statisticians and decision-theorists will be out of sympathy with the book because it is openly Bayesian. [...] But they would do well to consider the argument dispassionately and consider whether the axioms are acceptable to them. If they are, then the course is clear; if not, then they should say why and then develop their own and the deductions from them.

page 29

April 29, 2016

9:6

30

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch02

Resonance: From Probability to Epistemology and Back

Lindley clearly believed that Bayesian statistics can be derived from a simple set of axioms. I will show that almost nothing useful can be derived from these axioms.

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

2.4.6 Identification of probabilities and decisions If a theory of probability is developed in the decision theoretic context, it is clear that one needs to deal with the question of the “real” value of money and of the value of non-monetary rewards, such as friendship. An accepted way to deal with the problem is to introduce a utility function. One dollar gain typically has a different utility for a pauper and for a millionaire. It is commonly assumed that the utility function is increasing and convex, that is, people prefer to have more money than less money (you can always give away the unwanted surplus), and the subjective satisfaction from the gain of an extra dollar is smaller and smaller as your fortune grows larger and larger. The ultimate subjectivist approach to probability is to start with a set of axioms for rational decision making in the face of uncertainty and derive the mathematical laws of probability from these axioms. This approach was developed in [Savage (1972)], but it was based in part on the von NeumannMorgenstern theory (see ([Fishburn (1970), Chapter 14])). If one starts from a number of quite intuitive axioms concerning decision preferences, one can show that there exists a probability measure P and a utility function such that a decision A is preferable to B if and only if the expected utility is greater if we take action A rather than action B, assuming that we calculate the expectation using P . If a probability distribution and a utility function are given then the decision making strategy that maximizes the expected utility satisfies the axioms proposed by Savage. Needless to say, deriving probabilities from decision preferences does not guarantee that probability values are related in any way to reality. One can only prove theoretically a formal equivalence of a consistent decision strategy and a probabilistic view of the world.

2.4.7 The Bayes theorem The subjective theory is implemented in Bayesian statistics in a very specific way. The essence of statistics is the analysis of data so the subjective theory has to supply a method for incorporating data into a consistent set of opinions. On the mathematical side, the procedure is called “conditioning,”

page 30

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch02

Main Philosophies of Probability

31

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

that is, if some new information is available, the holder of a consistent set of probabilistic opinions is supposed to start using the probability distribution conditional on the extra information. The mathematical theorem that shows how to calculate the conditional probabilities is called the Bayes theorem (see Sec. 18.3). The consistent set of opinions held before the data are collected is called the “prior distribution” or simply the “prior” and the probability distribution obtained from the prior and the data using the Bayes theorem is called the “posterior distribution” or the “posterior.”

2.5 The Frequency Theory The development of the foundations of the mathematical theory of probability at the end of the 17th century was related to observations of the stability of relative frequencies of some events in gambling. In the middle of the 19th century, John Venn and other philosophers developed a theory identifying probability with frequency. At the beginning of the 20th century, Richard von Mises formalized this idea using the concept of a collective. A collective is a long (ideally, infinite) sequence of isomorphic events. Examples of collectives include casino-type games of chance, repeated measurements of the same physical quantity such as the speed of light, and measurements of a physical quantity for different individuals in a population, such as blood pressure of patients in a hospital. Von Mises defined a collective using its mathematical properties. For a sequence of observations to be a collective, the relative frequency of an event must converge to a limit as the number of observations grows. The limit is identified with the probability of the event. Many observations related to weather show seasonal patterns and the same is true for some business activities. Von Mises did not consider such sequences to be collectives so he imposed an extra condition that relative frequencies of the event should be equal along “all” subsequences of the collective. The meaning of “all” was the subject of a controversy and some non-trivial mathematical research. One of the subsequences is the sequence of those times when the event occurs but it is clear that including this subsequence goes against the spirit that the definition is trying to capture. Hence, one should limit oneself to subsequences chosen without prophetic powers but, as I said, this is harder to clarify and implement than it may seem at the first sight. The issue is further complicated by the fact that in real life only finite sequences are available, and then the restriction to all sequences chosen without prophetic powers is not helpful at all.

page 31

April 29, 2016

9:6

32

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch02

Resonance: From Probability to Epistemology and Back

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Another, perhaps more intuitive, way to present the idea of a collective is to say that a collective is a sequence that admits no successful gambling system. This is well understood by owners of casinos and roulette players — the casino owners make sure that every roulette wheel is perfectly balanced (and so, the results of spins are a collective), while the players dream of finding a gambling system or, equivalently, a pattern in the results. Von Mises ruled out applications of probability outside the realm of collectives (page 28 of [von Mises (1957)]): It is possible to speak about probabilities only in reference to a properly defined collective.

Examples of collectives given by von Mises are very similar to those used to explain the notions of independent identically distributed (i.i.d.) random variables, or exchangeable sequences (see Chapter 18). Both the definition of an i.i.d. sequence and the definition of an exchangeable sequence include, among other things, the condition that the probabilities of the first two events in the sequence are equal. In von Mises’ theory individual events did not have probabilities so he could not define collectives in the same way as i.i.d. or exchangeable sequences were defined. Instead, he used the principle of “place selection,” that is, he required that frequencies were stable along all subsequences of a collective chosen without prophetic powers. Some commentators believe that von Mises’ collectives are necessarily deterministic sequences. In other words, some people believe that von Mises regarded collectives as static large populations or sequences, not sequences of random variables with values created in a dynamic way as the time goes on. Although this distinction may have a philosophical significance, I do not see how it could make a practical difference, because von Mises clearly allowed for future “real” collectives, that is, he thought that a scientist could legitimately consider a collective that does not exist at the moment but can reasonably be expected to be observed in real life at some future time. Quite often, the frequency theory of von Mises and the subjective theory of de Finetti are portrayed as the antipodal options in the philosophical discourse. This is sometimes expressed by labeling the two theories as “objective” and “subjective.” In fact, the fundamental claims of both ideologies make them sister theories. De Finetti claimed that probability of an event is not measurable in any objective sense and so did von Mises. These negative claims have profound consequences in both

page 32

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch02

Main Philosophies of Probability

33

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

scientific and philosophical arenas. Von Mises argued that there is an objectively measurable quantity that can be called “probability” but it is an attribute of a long sequence of events, not an event. De Finetti thought that it can be proved objectively that probabilities should be assigned in a “consistent” way. Hence, both philosophers agreed that there are no objective probabilities of events but there are some objectively justifiable scientific practices involving probability.

2.6 Summary of Philosophical Theories of Probability A brief list of major philosophical theories of probability is given below. The list also includes my own theory, denoted (L1)–(L6), to be presented in Chapter 11. I consider my theory to be a scientific, not philosophical theory, for reasons to be explained later. However, I think that it should be included in the list for the sake of comparison. Each philosophy is accompanied by the main intuitive idea that underlies that philosophy. (1) (2) (3) (4) (5) (6)

The The The The The The

classical theory claims that probability is symmetry. logical theory claims that probability is “weak” implication. frequency theory claims that probability is long run frequency. subjective theory claims that probability is personal opinion. propensity theory claims that probability is physical property. system (L1)–(L6) claims that probability is search for certainty.

Of course, there is some overlap between theories and between ideas. A striking feature of the current intellectual atmosphere is that the two most popular philosophical theories in statistics — frequency and subjective — are the only theories that deny that the concept of probability applies to individual events (in an objective way). The philosophical ideas of von Mises and de Finetti were as revolutionary as those of Einstein. Einstein and other physicists forced us to revise our basic instincts concerning the relationship between the observed and the observer, the role of space and time, the relationship between mass and energy, the limits of scientific knowledge, etc. The idea that events do not have probabilities is equally counterintuitive. Of the four well crystallized philosophies of probability, two chose the certainty as their intellectual holy grail. These are the theories of von Mises and de Finetti. The other two philosophies of probability, logical and propensity, seem to be concerned more with the philosophical essence

page 33

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

34

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch02

Resonance: From Probability to Epistemology and Back

or nature of probability and do not propose that achieving certainty is the main practical goal of the science of probability. In von Mises’ theory, the certainty is achieved by making predictions about limiting frequencies in infinite collectives. De Finetti pointed out that one can use probability to avoid the Dutch book with certainty. Traditionally, we think about scientific statements as being experimentally testable. Both von Mises and de Finetti appealed to empirical observations and made some empirically verifiable claims. Logic is not traditionally represented as an experimentally testable science, although logical statements can be tested with computer programs. Hence, the logical philosophy of probability does not stress the empirical verifiability of probabilistic claims.

2.7 From Ideas to Theories The most pronounced limitation of most philosophical ideas about probability is their incompleteness. This in itself is not a problem but their proponents could not stop thinking that their favorite ideas described probability completely, and hence made silly claims. Examples of complete scientific theories include Newton’s laws of motion, laws of thermodynamics, and Maxwell’s equations for electromagnetic fields. Each of these theories provided tools for the determination of all scientific quantities in the domain of its applicability, at least in principle. Each of the following philosophical theories or ideas about probability is incomplete in its own way. The classical definition of probability was used until the end of the 19th century, long after probability started to be used in situations without “all cases equally possible.” The logical philosophy of probability is based on the principle of indifference. The principle cannot be applied in any usable and convincing way in practical situations which involve, for example, unknown quantities that can take any positive value (and only positive values), such as mass. The theory of von Mises claims that collectives are the domain of applicability of the probability theory. Leaving all other applications of probability out of the picture is totally incompatible with science in its present form. The advice given to statisticians by de Finetti is to use the Bayes theorem. Without any additional advice on how to choose the prior distribution, the directive is practically useless.

page 34

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch02

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Main Philosophies of Probability

35

Kolmogorov’s axioms (see Chapter 18) are sometimes mistakenly taken as the foundation of the science of probability. In fact, they provide only the mathematical framework and say nothing about how to match the mathematical results with reality. Popper’s idea of how to falsify probabilistic statements is incomplete in itself, but we can make it more usable by adding other rules. Many scientists and philosophers have an unjustified, almost religious, belief that the whole truth about many, perhaps all, aspects of reality springs from a handful of simple laws. Some cases where such a belief was or is applied are mathematics and fundamental physics. The belief was quashed in mathematics by G¨ odel’s theorems (see [Hofstadter (1979)]). Some physicists apply this belief to the string theory, with little experimental support, and with a fervor normally reserved for religious fanatics. In my opinion, there is nothing to support this belief in general or in relation to probability theory. The foundations of probability are awash with interesting ideas. The first memorable ideas were proposed at least three centuries ago by Bernoulli in relation to St. Petersburg paradox. The Achilles heel of foundations of probability is the construction of a complete philosophical theory of probability. A number of philosophers ([Carnap (1950)], [von Mises (1957)], [Hacking (1965)], [Gillies (1973)], [de Finetti (1974, 1975)], to name just a few) constructed more or less complete and logically consistent philosophical theories of probability. When one reads any of these theories, the first impression is that of professionalism. The books are logical, the arguments are detailed and go in many directions, diverse aspects of the problem are considered, pro and con arguments are given, etc. When I finished reading any of these books, I asked myself whether I would use any of these theories as an explanation of the concept of probability for undergraduate students. Did these theories represent faithfully the science of statistics in its present form? The answers to these questions were invariably “no.”

2.8 Popular Philosophy The most popular philosophical theories in statistics, frequency and subjective, are more sophisticated and less intuitive than most people realize. Many examples in popular literature trying to explain these theories are misleading — they have little if anything to do with the theories. In this sense, such examples do more harm than good because they

page 35

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

36

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch02

Resonance: From Probability to Epistemology and Back

suggest interpretations inconsistent with the fundamental ideas of the two philosophies. We should not be surprised by the poor knowledge of advanced theories among general population. Many people display a shocking ignorance of some basic facts. According to [Columbia Law Survey (2002)], “Almost two-thirds of Americans think Karl Marx’s maxim, ‘From each according to his ability, to each according to his needs’ was or could have been written by the framers and included in the [American] Constitution.” [ABC News (2014)] reported that quarter of Americans gave the wrong answer to the question “Does the Earth go around the Sun, or does the Sun go around the Earth?” in a survey conducted in 2012. A popular view of the frequency interpretation of probability is that “in repeated experiments, the frequency of an event tends to a limit.” This can mean, for example, that there are examples of real life sequences where such a tendency has been observed. Another interpretation of this statement is that the Law of Large Numbers (a mathematical theorem) is true. The problem with this “frequency interpretation” is that it is accepted by almost all scientists, so it does not characterize frequentists. All people agree that frequencies of some events seem to converge in some sequences. And people with sufficient knowledge of mathematics do not question the validity of the Law of Large Numbers. Similarly, the “subjective interpretation” of probability has the following popular version: “people express subjective opinions about probabilities; whenever you hold subjective opinions, you should be consistent.” Again, this interpretation hardly characterizes the subjective approach to probability because it will not be easy to find anyone who would argue that people never express subjective opinions about probability, or that it is a good idea to be inconsistent. The main philosophical claims of the frequency and subjective theories are negative. According to the frequency theory, one cannot apply the concept of probability to individual events, and according to the subjective theory, objective probability does not exist at all, that is, it is impossible to verify scientifically any probability assignments. The knowledge of the “negative” side of each of these philosophical theories hardly spread in the scientific community and even fewer people seem to be willing to embrace wholeheartedly these negative claims. A good illustration of the thoroughly confused state of the popular view of the philosophical foundations of probability can be found in one of the

page 36

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch02

Main Philosophies of Probability

37

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

most authoritative current sources of the popular knowledge — Wikipedia, the free cooperative online encyclopedia. The article [Wikipedia (2006a)] on “Bayesian probability,” accessed on July 6, 2006, starts with In the philosophy of mathematics Bayesianism is the tenet that the mathematical theory of probability is applicable to the degree to which a person believes a proposition. Bayesians also hold that Bayes’ theorem can be used as the basis for a rule for updating beliefs in the light of new information — such updating is known as Bayesian inference.

The first sentence is a definition of “subjectivism,” not “Bayesianism.” True, the two concepts merged in the collective popular mind but some other passages in the same article indicate that the author(s) can actually see the difference. The reference to the the belief in a “proposition” would be more appropriate in an article on the logical philosophy of probability. The second sentence of the quote suggests that non-Bayesians do not believe in the Bayes theorem, a mathematical result, or that they believe that the Bayes theorem should not be applied to update beliefs in real life. In fact, all scientists believe in the Bayes theorem. I am not aware of a person who would refuse to update his probabilities, objective or subjective, using the Bayes theorem. The introduction to the Wikipedia article is misleading because it lists beliefs that are almost universal, not exclusive to subjectivists or Bayesians. The article [Wikipedia (2006b)] on “Frequency probability,” accessed on July 6, 2006, contains this passage Frequentists talk about probabilities only when dealing with well-defined random experiments. The set of all possible outcomes of a random experiment is called the sample space of the experiment. An event is defined as a particular subset of the sample space that you want to consider. For any event only two things can happen; it occurs or it occurs not. The relative frequency of occurrence of an event, in a number of repetitions of the experiment, is a measure of the probability of that event. [...] Frequentists can’t assign probabilities to things outside the scope of their definition. In particular, frequentists attribute probabilities only to events while Bayesians apply probabilities to arbitrary statements. For example, if one were to attribute a probability of 1/2 to the proposition that “there was life on Mars a billion years ago with probability 1/2” one would violate frequentist canons, because neither an experiment nor a sample space is defined here.

page 37

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

38

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch02

Resonance: From Probability to Epistemology and Back

The idea that the main difference between frequentists and non-frequentists is that the former use a sample space and events and the latter do not and must have originated in a different galaxy. Here, on our planet, all of statistics at the research level, frequency and Bayesian, is based on Kolmogorov’s mathematical approach and, therefore, it involves a sample space and events. Moreover, the remarks about the probability of life on Mars are closer to the logical theory of probability, not the subjective theory of probability. To be fair, not everything in Wikipedia’s presentation of philosophies of probability is equally confused and confusing — I have chosen particularly misleading passages.

2.9 Is There Life Beyond Von Mises and De Finetti? The foundations of probability look completely different from different perspectives. I am not even talking about disagreements between philosophers on the nature of probability. Different accounts of the foundations of probability offer strikingly different descriptions of the most significant directions in and contributions to this part of philosophy. Among all philosophical theories of probability, I have chosen to focus almost entirely on two theories, those of von Mises and de Finetti. One of the top results of the online search for “philosophy of probability” is an article [H´ ajek (2007)] which presents a rainbow of philosophical ideas. Von Mises and de Finetti are featured in that article but they by no means dominate the review. So, why do I focus on von Mises and de Finetti? The first and the main reason is paradoxical in nature. I would like to understand the foundations of statistics and the two main branches of statistics, frequency and Bayesian, seem to be most strongly connected with the philosophies of von Mises and de Finetti. I call this reason paradoxical because one of the main theses of this book is that there is practically no relationship between the two branches of statistics and the two philosophical theories. However, I do not know of any other philosophical theories of probability that would match statistics better in the minds of statisticians and other users of statistics and probability. A good reason to limit the discussion to the theories of von Mises and de Finetti is that they dominate certain areas of the public discussion. Although [Galavotti (2005); H´ ajek (2007); Mellor (2005)] and other publications prove that professional philosophers have a broad view of their own

page 38

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch02

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Main Philosophies of Probability

39

field, there are clear signs showing that much of the scientific and general public does not know much or does not care about probabilistic philosophical ideas other than “frequency” and “subjective.” I will support this claim with several examples. The textbook [Hacking (2001)] does not focus on just two philosophers, de Finetti and von Mises, but Chapters 13–19 present a sympathetic view of subjectivism and frequentism, leaving no doubt in student’s mind that these are the two main directions in philosophy of probability. The titles of the chapters in [von Plato (1994)] contain only three names — those of Kolmogorov, von Mises and de Finetti. A recent book [Heller (2012)] on “philosophy of chance” is very different from the mainstream monographs on philosophy of probability. Its review of the main philosophical theories of probability is limited to those of von Mises and de Finetti (Secs. 6.7 and 6.8). Two popular undergraduate textbooks, [Pitman (1993)] and [Ross (2006)], discuss only frequency and subjective interpretations of probability (see Sec. 17.3). This handful of examples was enough to convince me that my main intellectual adversaries in the area of philosophy of probability are de facto von Mises and de Finetti. I did not find any other philosophical theory of probability more attractive than those of von Mises and de Finetti. Many theories sound more reasonable than these two but they fail to answer crucial questions to my satisfaction.

page 39

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch03

Chapter 3

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Skepticism

3.1 How Do You Prove that You are Not a Camel? The title of this section is a popular saying in some East European countries. It apparently originates in the following joke. A rabbit was running for his life. When asked by another rabbit what he was running from, he said a lion had threatened to eat any camel he met. “But we are rabbits!” “Yes, but how do I prove that I am not a camel?”

The joke circulated mostly in (post-) communist societies so it seems that its brunt was directed at the absurdities of the communist bureaucracy but I will give it an epistemological meaning. The saying “How do you prove that you are not a camel?” points to a paradox. It seems that if a claim sounds reasonable and is close, in some sense, to the truth but it is in fact false then it should be hard to disprove the claim. In other words, it seems that it is hard to discriminate between two assertions, a true one and a false one, if they are close to each other. The same intuition suggests that if a claim is obviously false then it should be easy to disprove the claim. The saying quoted in the title of this section indicates that in reality the opposite may be true — nonsensical claims may be the hardest to disprove. The paradox may be explained as follows. If people only slightly disagree about a claim then this indicates that they possess similar knowledge bases and they accept similar rules of reasoning. Hence, it should be relatively easy for them to identify the sources of disagreement and determine the truth. If a claim made by one person appears to be obviously and utterly false to another person then it is likely that the disagreement is much deeper than just the matter of the claim at hand. The two people might have anchored their beliefs in considerably different knowledge bases or 41

page 41

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

42

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch03

Resonance: From Probability to Epistemology and Back

they might use different standards of quality in reasoning. Even worse, one or both people may harbor prejudices that totally eliminate the possibility of rational communication between them. Philosophers made a number of claims that sounded as outrages as the one about a rabbit being a camel. Zeno claimed that motion did not exist. Leibnitz claimed that the universe was made of infinitesimally small monads that never interacted but were perfectly synchronized with one another by God. De Finetti claimed that probability did not exist. Strange and radical claims force an intellectual adversary to develop a deep analysis of the problem. No amount and no depth of analysis can assure the eventual reconciliation of contradictory opinions. But such analysis will allow the holders of contradictory opinions to assess the foundations of their own convictions. I would like to say that de Finetti’s claim that probability does not exist is nonsensical (and indeed I say so) and leave it at that. But the popularity of his philosophical theory forces me to descend into the depths of epistemology. I doubt that I will convince his devotees that they are totally wrong. But I will supply ammunition to people who feel that de Finetti’s ideas are crazy.

3.2 Skepticism I enter the discussion of skepticism with reluctance because I feel that I am drifting away from the main topic of the book — analysis of foundations of probability. However, the circumstances force me to address skepticism. Recall that de Finetti claimed that probability did not exist. His claim can be interpreted in different ways, more scientific or more philosophical, but its skeptical character is evident. A rational person has to be a total skeptic. Ancient Greek philosophers are responsible for making skeptics of all of us. They established logic as the basis of rational opinion. One could go further and say that logic has become synonymous with rationality. But logic can only generate true statements from other true statements so nothing can be proved because every assumption can be questioned. The first notable skeptic was Pyrrho of Elis in the third century BC. Since then skepticism took many forms. I see (at least) three good justifications for the skeptical philosophical position — logical, transcendental and practical. (i) We use logic to justify and analyze logic. This is a vicious circle of arguments. It is possible that our logic is invalid or unusable. Recall

page 42

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch03

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Skepticism

43

that logic can only generate new truths from some assumed truths. Hence, to accept any truth, we have to assume some truths as selfevident. Adopting some axioms is not automatically more rational than a consistently skeptical position about all truths. (ii) The “transcendental” reason for skepticism is that the world known to us may be a part of a much larger structure, unknown or incomprehensible to us. For example, there may exist an evil demiurge, or we may be a part of a simulation similar to that in the film “Matrix.” Physicists seriously talk about multiverse. Not knowing the full context of our existence severely limits our ability to analyze our universe. (iii) There are practical reasons for being skeptical. We believe that individual people and large groups of people may be mistaken in various ways. Here are some examples. We do not trust intoxicated people. How do we know that our brains normally or typically function in a more reliable state? How do we know that our current view of the world is not a dream? The experience of contradictory witness accounts, in the justice system and outside, inspires a skeptical attitude towards our ability to observe facts. There are huge discrepancies between religious beliefs of various groups of people. Large groups of people used to believe in “self-evident truths,” such as that the Earth was flat, or that the Earth was stationary and the Sun moved around it. I fully accept the most radical version of skepticism. I believe that we cannot know anything with certainty and that it is possible that everything that we think we know is totally wrong. At the same time I believe that skepticism is an intellectual and practical dead end. On the intellectual side, philosophers and scientists try to find the truth about reality. These may be accounts of facts or proposals for theories. No matter how tentative or uncertain these intellectual products are, they are infinitely more interesting than the skeptical position. On the practical side, skepticism is de facto synonymous with hypocrisy. I am not aware of a philosophical skeptic who actually successfully implemented his skeptical position in real life. I cannot even imagine totally skeptical behavior. Radical claims made in [Feyerabend (1975)] will serve as an illustration (see Sec. 3.3 for a short review of the book). Feyerabend claimed in Appendix 4 (p. 221) that there was no rational reason for not jumping out of a 50th floor window. He attributed not jumping himself to his cowardice. I do not believe his argument for

page 43

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

44

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch03

Resonance: From Probability to Epistemology and Back

a second because people who are cowards but who believe that certain actions are rational (or at least not obviously irrational) and desirable try to overcome their cowardly feelings and they advise other people to take these actions. For example, people with aviophobia (fear of flying) sometimes force themselves to fly and visit relatives when the need arises. If we believe that various actions are equally rational and beneficial then we alternate between different options out of boredom or curiosity. For example, we may order different dishes on consecutive visits to a restaurant for no obvious rational reason. There is no evidence that Feyerabend tried to jump from a skyscraper window just to have this experience. Would Feyerabend advise other people, especially those who are not cowards, to jump out of the window? Feyerabend’s claim was intellectually empty. People do not jump from windows because they harbor some beliefs and studying these beliefs and their sources is a fascinating subject for philosophy (not so much for science because scientists know perfectly well what jumping from a window would result in). Skepticism is necessarily self-contradictory. A skeptic has to make some non-skeptical assumptions to make a skeptical claim. For example, Feyerabend’s claim that he does not jump from a window of a skyscraper because he is a coward entails that cowardice exists, that he is a coward, that cowardice is the true reason why he does not jump, etc. I do not see any reason why we should consider all of these assertions to be less controversial than the assertion that a person who jumps out of a window will fall to the ground, hit the pavement with a high velocity and die. Another radical claim of Feyerabend was his praise for the Chinese Communist Party for suppressing Western medicine and replacing it with the traditional Chinese medicine ([Feyerabend (1975)], pp. 50, 51). Feyerabend did not propose to suppress the Western medicine throughout the world and replace it with the Chinese medicine. Why not? If the Chinese medicine worked objectively better than the Western medicine for the Chinese then it should have done the same good job in the rest of the world. Does the Chinese medicine work only in China? Are the Chinese significantly genetically different from all other humans? If the improvement in China was only subjective then perhaps we should replace Western medicine with narcotics that will give people an illusion of wellbeing. I doubt that skeptics really believe in their own theories. The illustrations that they provide are either irrelevant to most people (Feyerabend

page 44

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch03

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Skepticism

45

claims that some Nile tribes have a different sense of time from ours; see page 251 of [Feyerabend (1975)]) or sound outright silly, like the discussion of jumping from the 50th floor. Religion is a good example of selective skepticism. Religious people accept some observations, testimony and intellectual arguments in favor of their own religion and reject analogous arguments of the more or less same strength in favor of other religions. Philosophical theories of probability proposed by von Mises and de Finetti are examples of skeptical thinking, hypocrisy and intellectual failure. Both philosophers claimed that the scientific concept of probability did not apply to individual events — that was the skeptical part. I will provide evidence that von Mises and de Finetti did not abide by their own rules — this is the hypocritical part. Finally, the intellectual failure of von Mises and de Finetti was their inability to describe popular methods of assigning probabilities to individual events. They also failed to understand the reasons why people assigned these and not other probabilities to individual events. Even if you do not believe in God, it is your intellectual failure if you do not understand why other people do. The reason why we can assert with confidence that the skeptical claims of Zeno (“Motion does not exist”), von Mises, de Finetti and Feyerabend are nonsensical is because their essence is quite scientific, not just purely philosophical. The claims of all four philosophers are based on empirical observations or make empirical predictions. Von Mises’ book contains a discussion of statistics, de Finetti’s book contains a discussion of applications of probability, and Feyerabend’s book contains a practical suggestion for teaching voodoo at schools. Hence, the skeptical claims of all four philosophers have operational interpretations. Even if one of these sets of claims were implemented in real life, the civilization would collapse in a short time. We all have no choice but to adopt a certain level of skepticism. It is totally uninteresting to reject all claims and equally uninteresting to accept all claims. A philosophical theory is convincing if it is more or less equally skeptical in all of its claims. Otherwise, jumping from one level of skepticism to another suggests that the philosopher classified claims as true or false a priori, according to his instinct and not any rational reasoning, and then applied skeptical arguments only to those claims that were marked for rejection.

page 45

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

46

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch03

Resonance: From Probability to Epistemology and Back

I am not sure whether I can precisely describe my level of skepticism but I will try to maintain the same level as much as I can. I will now list a few beliefs which I will assume without proof because I would not find a more skeptical position interesting. I will assume that (the standard) logic is a reliable tool for information processing. I will assume that the universe exists in the objective sense and that it is governed by objective laws of nature. I will not assume that we know any of these laws or that we can discover any of these laws. Instead, I will assume that we can build an image of the universe and its laws in our minds in such a way that this “knowledge” can be an effective tool for achieving practical goals. I will also assume that people can recognize many relevant facts with great reliability if they focus appropriately or devote sufficient resources (material and theoretical) to the task. Note that von Mises and de Finetti operated at my level of skepticism most of the time, just like most people in many practical situations.

3.2.1 Frustration of philosophers A number of times philosophers who could not come up with a good theory or explain a phenomenon or answer a philosophical question gave up rationality and made nonsensical claims, usually totally or partly skeptical. Examples of such philosophical contributions include Zeno’s claim that motion does not exist, de Finetti’s claim that probability does not exist and Feyerabend’s claim that anything goes ([Feyerabend (1975)]). These claims are absurd not because they are strange or they are the views of a minority. They are intellectual dead ends — they do not lead to interesting theories, beliefs or practical methods. Nonsensical claims are tolerated in philosophy for good reasons. Philosophers cannot label any theory as absurd outright. That would lead to worse consequences than tolerance towards nonsense. One of the main roles of philosophy is to examine the foundations of our knowledge. If a claim is in fact absurd then a philosopher should be able to explain why. Doing this may require a certain dose of analysis. I realize and accept that philosophy cannot reject an absurd claim in a summary trial. On the other hand, science has to do just that out of practical necessity. In this way, philosophy is less efficient than science. This book testifies to this. I have to go to great lengths to show that de Finetti’s theory makes no sense although this should be evident to every rational person.

page 46

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch03

Skepticism

47

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

I do not think that it is utterly impossible to improve quality control in philosophy. A step in the right direction would be to identify those philosophical claims and theories that have a clear operational meaning. Then one should test them and in case of failure label them as such with the same definiteness as this is done in science. The skeptical claims of Zeno, von Mises, de Finetti and Feyerabend were failed scientific theories that gained recognition and respect by pretending to be philosophical theories. The four philosophers exploited a loophole in the intellectual life.

3.3 Anything Goes This section is devoted to a short review of a book [Feyerabend (1975)] titled Against Method: Outline of an Anarchistic Theory of Knowledge. The book has no relation to probability. I decided to write a few paragraphs about it partly because it is an excellent example of skepticism and hypocrisy taken to the extreme and partly for the amusement of the reader. Feyerabend’s book shows that de Finetti and von Mises were neither exceptional in their skepticism nor extreme in this respect. First, let me present a little bit of the general background. Paul Feyerabend worked as a professor of philosophy at the University of California, Berkeley, from 1958 to 1989. “Against method” means “against Popper’s method.” The title of this section refers to Feyerabend’s claim that “The only principle that does not inhibit progress is: anything goes” (p. 23). Feyerabend did everything that he could to annoy and offend mainstream scientifically minded readers. He mentioned “special professions such as science or prostitution” (p. 217). He praised the Chinese Communist Party for suppressing Western medicine (pp. 50, 51). He claimed that there is no rational reason why a person should not walk out of a window of the fiftieth floor (p. 221). He called philosophy of science a “bastard subject” (p. 301). He proposed to teach “myths of ‘primitive’ societies” in parallel with science (p. 308). On page 299, Feyerabend wrote The separation between state and church must therefore be complemented by the separation between state and science. We need not fear that such a separation will lead to a breakdown of technology. [...] We shall develop and progress with the help of the numerous willing [emphasis in original] slaves in universities and laboratories who provide us with pills, gas, electricity, atom bombs, frozen dinners [...].

page 47

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

48

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch03

Resonance: From Probability to Epistemology and Back

Note that Feyerabend did not see any rational reason why jumping from a 50th floor window would kill you but he somehow knew that willing slaves at universities would provide electricity. Conspicuous by its absence in the book is the controversy between Creationism and the evolution theory. I would like to know whether Feyerabend considered Creationism to be “incommensurate” with the evolution theory and whether he thought that fossils of dinosaur bones were not facts because there were no facts whatsoever. Some of his claims sounded less insane. He claimed that in antiquity people had a different concept of an object, especially human body — it was a sum of its parts, not a whole (p. 233). He also claimed that not all people had the same sense of time (p. 251). At the other extreme, Feyerabend demonstrated his sanity when he offered a painstakingly detailed analysis of Galileo’s manipulation of telescopic observations. After reading his pontifications on the separation of state and science and jumping from a skyscraper, one has to wonder whether there are any reasons to believe Feyerabend when he says that Galileo existed, that telescope is an optical instrument and that the Moon in the sky is a real object and not a figment of imagination. The book is far from silly. Aficionados of philosophy will find some gold nuggets — I feel that I did. Some of the arguments given by Feyerabend were very subtle and he definitely proved to be erudite. Feyerabend’s skepticism was so extreme that it was more amusing than annoying. A reader with a minimal dose of intelligence would notice that Feyerabend was skeptical only when it suited him. He painted the word “hypocrisy” on his forehead like a clown. One has to wonder whether the whole book or at least parts of it were a hoax. It is possible that Feyerabend wrote the book as a provocation or for amusement of some of his colleagues. In the end, reading the book was like watching drunken Einstein. I saw a genius who decided to make a fool of himself.

3.4 Von Mises’ Brand of Skepticism Recall that, according to von Mises (page 28 of [von Mises (1957)]), It is possible to speak about probabilities only in reference to a properly defined collective.

This should be interpreted as saying that probability is an attribute of a “collective” and not of an event. A collective is an infinite sequence of

page 48

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch03

Skepticism

49

observations such that the relative frequency of an event converges to the same number along every subsequence chosen without prophetic powers. The common limit is called the probability (of the event in this collective).

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

3.4.1 The smoking gun The quote of von Mises presented in Sec. 3.4 is subject to interpretation, just like everything else in philosophy. I will now argue that the concept of a collective necessarily leads to a radical interpretation of von Mises’ theory. Collectives are equivalent to physical evidence in a court trial. The concept of a collective is a conspicuous element of the frequency theory and there is no justification for its use except the radical interpretation of the theory. Examples of real collectives given by von Mises are of the same type as the ones used to illustrate the ubiquitous concept of independent identically distributed (i.i.d.) random variables. On the mathematical side, i.i.d. sequences are much more convenient than collectives (see Sec. 4.13). The mathematical concept of i.i.d. random variables was already known in the 19th century, even if this name was not always used. Why is it that von Mises chose to formalize a class of models of real phenomena using a considerably less convenient mathematical concept? The reason is that the definition of an i.i.d. sequence of random variables X1 , X2 , . . . includes, among other things, a statement that the following two events {X1 is equal to 0} and {X2 is equal to 0} have equal probabilities (I have chosen 0 as an example; any other value would work as well). According to von Mises’ philosophy, a single event, even if it is a part of a well defined collective, does not have its individual probability. In other words, there is no theoretical or empirical method that could be used to determine whether P (X1 = 0) = P (X2 = 0). According to von Mises, the notion of an i.i.d. sequence involves non-existent quantities, that is, probabilities of individual events. As I said, the fact that collectives are the most prominent part of von Mises’ theory proves that the only interpretation of this theory that is compatible with the original philosophical idea of von Mises is that individual events do not have probabilities.

3.4.2 Inconsistencies in von Mises’ theory The claim of von Mises quoted at the beginning of Sec. 3.4 refers to collectives. What constitutes a collective? Are imaginary collectives

page 49

April 29, 2016

9:6

50

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch03

Resonance: From Probability to Epistemology and Back

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

acceptable from the scientific point of view? The difficulties with von Mises’ philosophical theory are well illustrated by inconsistencies in his own book [von Mises (1957)]. His philosophical ideas require that only “real” collectives are considered. When his ideas are applied to science, “imaginary” collectives are used. On page 9 of [von Mises (1957)] we find, Our probability theory has nothing to do with questions such as: ‘Is there a probability of Germany being at some time in the future involved in a war with Liberia?’

On page 10, von Mises explained that “unlimited repetition” was a “crucial” characteristic of a collective and gave real life examples of collectives with this feature, such as people buying insurance. He also stated explicitly that The implication of Germany in a war with the Republic of Liberia is not a situation which repeats itself.

Needless to say, we could easily imagine a long sequence of planets such as the Earth, with countries such as Germany and Liberia. And we could imagine that the frequency of wars between the pairs of analogous countries on different planets is stable. It is clear that von Mises considered such imaginary collectives to be irrelevant and useless. Later in the book, von Mises discussed hypothesis testing. On page 156, he said that hypothesis testing could be approached using the Bayes method. This takes us back to pages 117 and 118 of his book. There, he constructed a collective based on the observed data. For example, in the case when the data were observations of a Bernoulli sequence (that is, every outcome was either a “success” or “failure”) of length n, and we observed n1 successes, he constructed a collective using a “partition.” That is, he considered a long sequence of data sets, such that in every case, the ratio n1 /n was the same number a. In practice, this construction yields a purely imaginary collective. Except for a handful of trivial applications of statistics, data sets never repeat themselves in real life, even if we look only at “sufficient” statistics (that is, special general numerical characteristics of the data sets). Hence, von Mises saw nothing wrong about imaginary collectives in the scientific context. We see that inconsistencies in von Mises’ book match perfectly my general criticism of the skeptical philosophical position. Skepticism is inextricably linked to hypocrisy.

page 50

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch03

Skepticism

51

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

3.5 De Finetti’s Brand of Skepticism This section is analogous to Sec. 3.4.1 in that I will argue that my extreme interpretation of de Finetti’s theory is the only interpretation compatible with the most prominent technical part of that theory — the decision theoretic foundations. An axiom in the mathematical theory of probability says that if we have two events A and B that cannot occur at the same time and C is the event that either A or B occurs, then the probability of C is equal to the sum of the probabilities of A and B. In symbols, the axiom says that P (A ∪ B) = P (A) + P (B) if P (A ∩ B) = ∅. The Central Limit Theorem, a much more complicated mathematical assertion, is another example of a probabilistic statement (see Sec. 18.1.1). A standard way to verify scientific statements such as F = ma, one of Newton’s laws of motion, is to measure all quantities involved in the statement and check whether the results of the measurements satisfy the mathematical formula. In the case of Newton’s law, we would measure the force F , mass m and acceleration a. It is important to realize that it is impossible to check the law F = ma in all instances. For example, we cannot measure at this time any physical quantities characterizing falling rocks on planets outside of our solar system. Moreover, scientific measurements were never perfect, and perfection was never a realistic goal, even before quantum mechanics made the perfect measurement theoretically unattainable. If the statements that P (A ∪ B) = P (A) + P (B) for mutually exclusive events A and B, and the Central Limit Theorem, are scientific laws then the course of action is clear — we should measure probability values in the most objective and accurate way in as many cases as is practical, and we should check whether the values satisfy these mathematical formulas, at least in an approximate way. It has been pointed out long time ago by David Hume that, on the philosophical side, this procedure cannot be considered as a conclusive proof of a scientific statement. However, verifying probability statements in this way would put them in the same league as other scientific statements. Instead of following the simple and intuitive approach described above, de Finetti proposed to derive probability laws from postulates representing rational decision making (see page 87 of [de Finetti (1974)] for the Dutch book argument). He proposed to limit rational decision strategies to a family nowadays referred to as “consistent” or “coherent” strategies.

page 51

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

52

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch03

Resonance: From Probability to Epistemology and Back

And then he showed that choosing as a consistent decision strategy is mathematically equivalent to choosing a probability distribution to describe the (unknown) outcomes of random events. This seems to be an incredibly roundabout way to verify the laws of probability. Would anyone care to derive Newton’s laws of motion from axioms describing rational decision making? There is only one conceivable reason why de Finetti chose to derive probability laws from decision theoretic postulates — he did not believe that one could measure probabilities in a fairly objective and reliable way. This agrees perfectly with his famous (infamous?) claim that “Probability does not exist.” His claim is not just a rhetorical slogan — it is the essence of his philosophy. De Finetti made a very interesting discovery that probability calculus can be used to make desirable deterministic predictions. He also declared that objective probability does not exist. The two philosophical claims are logically independent but the first one is truly significant only in the presence of the second one.

3.5.1 How to eat the cake and have it too It is clear from the way de Finetti presented his theory that he wanted to eat the cake and have it too. On one hand, it appears that he claimed that the choice of probability values should be based on the available information about the real world, and on the other hand he vehemently denied that probability values can be chosen or verified using objective evidence, such as symmetry or frequency. On page 111, Chapter II, of his essay in [Kyburg and Smokler (1964)], he approvingly writes about two standard ways of assigning probabilities — using symmetry and using long run frequencies. However, he does not invest these methods with any objective meaning. In this way, he is absolved from any philosophical responsibility to justify them. This magical trick can be used to provide philosophical foundations for any theory, for example, the theory of gravitation. On one hand, one can take a vaguely positive view towards Einstein’s equations for gravitation. At the same time, one can declare that the available evidence of gravitation’s existence is not sufficiently objective and scientific. De Finetti has this to say about symmetry (page 7 of [de Finetti (1974)]): [...] let us denote by O statements often made by objectivists, and by S those which a subjectivist (or, anyway, this author) would reply.

page 52

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch03

Skepticism

53

O: Two events of the same type in identical conditions for all the relevant circumstances are ‘identical’ and, therefore, necessarily have the same probability.

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

S: Two distinct events are always different, by virtue of an infinite number of circumstances (otherwise how would it be possible to distinguish them?!). They are equally probable (for an individual) if — and so far as — he judges them as such (possibly by judging the differences to be irrelevant in the sense that they do not influence his judgment).

On the philosophical side, de Finetti dismissed any connection between objective symmetry and probability. On the scientific side, he discussed fair coin tosses in Chapter 7 of [de Finetti (1975)]. De Finetti’s book is like a treatise on tuberculosis, in which the author asserts early in the book that eating garlic has no effect on tuberculosis, but later writes several chapters on growing, storing and cooking garlic. De Finetti defined “previsions” as follows ([de Finetti (1974)], p. 72): Prevision, in the sense we give to the term and approve of (judging it to be something serious, well founded and necessary, in contrast to prediction), consists in considering, after careful reflection, all the possible alternatives, in order to distribute among them, in the way which will appear most appropriate, one’s own expectations, one’s own sensations of probability.

Later in the book, de Finetti makes these remarks on the relationship between “previsions” and observed frequencies ([de Finetti (1974)], p. 207): Previsions are not predictions, and so there is no point in comparing the previsions with the results in order to discuss whether the former have been ‘confirmed’ or ‘contradicted’, as if it made sense, being ‘wise after the event’, to ask whether they were ‘right’ or ‘wrong’. For frequencies, as for everything else, it is a question of prevision not prediction. It is a question of previsions made in the light of a given state of information; these cannot be judged in the light of one’s ‘wisdom after the event’, when the state of information is a different one (indeed, for the given prevision, the latter information is complete: the uncertainty, the evaluation of which was the subject under discussion, no longer exists). Only if one came to realize that there were inadequacies in the analysis and use of the original state of information, which one should have been aware of at that time (like errors in calculation, oversights which one noticed soon after, etc.), would it be permissible to talk of ‘mistakes’ in making a prevision.

De Finetti’s “prevision” is a family of probabilities assigned to all possible events, that is, a (prior) probability distribution. De Finetti claimed that

page 53

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

54

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch03

Resonance: From Probability to Epistemology and Back

the prior distribution cannot be falsified by the data, and neither can it be confirmed by the data. Hence, according to de Finetti, the scientific (practical) success of theories of Markov processes and stationary processes, presented in Chapter 9 of [de Finetti (1975)], was never “confirmed” by any observations. Just like von Mises (see Sec. 3.4.2), de Finetti was (logically) inconsistent, in the sense that he endorsed models for which, he claimed, was no empirical support. I will now go back to the question of my interpretation of de Finetti’s theory. De Finetti made some conspicuous and bold statements, for example, he fully capitalized and displayed at the center of the page his claim that “Probability does not exist” (page x of [de Finetti (1974)]). He also called this claim “genuine” in the same paragraph. On the other hand, he used language that I find disturbingly ambivalent and confusing. For example, in the dialog between “O” and “S” quoted above, S seems to reject in the first sentence the idea that there exist objective symmetries that would necessitate assigning the same probability value to two events. However, in the second sentence, de Finetti uses the phrase “he judges” in reference to “events ... equally probable.” Very few people would interpret the phrase “he judges” as saying that an individual can assign probabilities in an arbitrary way, as long as they satisfy Kolmogorov’s axioms. So de Finetti seems to suggest that probabilities should be chosen using some information about the real world. This opinion of mine is confirmed by the following quotes. De Finetti says that subjectivists follow an approach (quoted after ([Galavotti (2005), p. 218])) leaving to a second (extra-mathematical) phase the discussion and the analysis of motives and criteria for the choice of a particular one amongst all these possible evaluations [consistent probability distributions].

Here is another de Finetti quote (after ([Galavotti (2005), p. 219])), as for myself, though maintaining the subjectivist idea that no fact can prove or disprove belief, I find no difficulty in admitting that any form of comparison between probability evaluations (of myself, of other people) and actual events maybe an element influencing my further judgment, of the same status as any other kind of information.

I find this claim of de Finetti totally incomprehensible. If a belief cannot be disproved by any facts, what benefit will I derive from comparing my beliefs with those of others?

page 54

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch03

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Skepticism

55

I have chosen to interpret de Finetti’s theory as saying that probabilities can be chosen in an arbitrary way, as long as they satisfy the usual mathematical formulas. I consider this the only interpretation consistent with the most significant elements of de Finetti’s theory. Looking into the past, he asserted that there is no objective symmetry that can be used to find probabilities. Looking into the future, he claimed that observed frequencies cannot falsify any prevision. Hence, a subjective probability distribution may appear to be rational and anchored in reality to its holder, but no other person can prove or disprove in an objective way that the distribution is correct. For all practical purposes, this is the same as saying that probability distributions can be chosen in an arbitrary way. I have already commented on logical inconsistencies in the dialog between O and S. The quote on the frequencies is similarly annoying because it contains statements that nullify the original assertion. Suppose that at the end of clinical trials, a subjectivist statistician concludes that the side effect probability for a given drug is 2%. Suppose further that when the drug is sold to a large number of patients in the general population, the observed side effect rate is 17%. It would be very natural for the statistician to review the original study. Suppose that he concludes that the original estimate of 2% did not match the 17% rate in the general population because the patients in the clinical trials were unusually young. This seems to fit perfectly into the category of “oversights” that de Finetti mentions parenthetically. Hence, de Finetti’s supporters can claim that his theory agrees well with scientific practice. De Finetti labeled as “oversight” every case where the data falsified the prevision. In this way, he did not have to discuss the philosophical problem of how observed frequencies falsify probability statements — something to which Popper devoted many pages in [Popper (1968)].

3.6 On Approximate Theories It is a great mistake to treat the theories of von Mises and de Finetti as approximations or idealizations in the scientific sense. According to Newton’s theory, falling bodies move along parabolas. This is false on our planet because of the presence of atmosphere which induces drag (air resistance). Hence, Newton’s theory is an approximation or idealization of reality. Newton’s theory provides a very good approximation for trajectories of falling bodies on the Moon because the Moon has no atmosphere.

page 55

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

56

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch03

Resonance: From Probability to Epistemology and Back

The theories of von Mises and de Finetti are not idealizations or approximations in any conceivable sense. Frequency statisticians apply the concept of probability in situations where there are no collectives (or i.i.d. sequences). Bayesians apply their methods in situations where there is no need to coordinate decisions. People who support either philosophy of probability make a subconscious mistake. They learn purely philosophical arguments and they like them. They do realize that the arguments are impractical. Nevertheless they accept the philosophical arguments because they subconsciously believe that the discrepancy between the theories of von Mises and de Finetti on one hand and reality on the other hand is like the discrepancy between any scientific theory and reality. The result is that the requirement for being realistic is waived for the bizarre philosophical arguments of von Mises and de Finetti because they are mistakenly and subconsciously classified as approximate or idealized.

3.7 Temperature, Beauty and Probability I will illustrate differences between the concepts of subjectivity used in de Finetti’s theory and Bayesian statistics. People have sensations of temperature, beauty and probability. All of these sensations are subjective in more than one sense: (i) They are all experienced by humans. (ii) There are differences between how different people experience these sensations. (iii) The sensations can vary over time, even over short periods of time. (iv) The sensations are inaccurate and unreliable in the sense that even the person who experiences these sensations may be uncertain about their relation to reality. Scientists developed thermodynamics, a field of physics, that is concerned with temperature, among other things. There seems to be a consensus that human sensations of temperature correspond to an objective quantity with the same name, and the correspondence is reasonably close. On the other hand, beauty has never become a scientific concept. Psychologists do study human perception of beauty but scientists did not find an objective property or quantity that could be a part of a scientific theory and at the same time it would correspond, more or less, to the human sensation of beauty. It is clear that the attitude of Bayesian statisticians towards probability is the same as the attitude of physicists towards temperature. They believe that human sensations of probability correspond to an objective quantity with the same name.

page 56

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch03

Skepticism

57

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

3.8 Latter Day Subjectivism A book [Lindley (2006)] “Understanding Uncertainty” was written by Dennis Lindley, a prominent statistician and a fanatical subjectivist. After years of the debate between subjectivists and frequentists (and representatives of other philosophical directions in the foundations of probability), the least one could expect from an author of this stature and experience would be a clear exposition of the basic ideas of his ideology. Alas, the book contains the same simple logical contradictions that one can find in a much older book by de Finetti. The main representatives of the subjective philosophy of probability could not provide a clear account of their ideology. I will expose some glaring contradictions in Lindley’s philosophy. On page 12, Lindley quotes de Finnetti’s famous (infamous?) saying “Probability does not exist” with approval. He also writes “... probability does not exist as a property of the world in the way that distance does, for distance between two points, properly measured, is the same for all of us, it is objective, whereas probability depends on the person looking at the world, on you, as well as on the event ...” On page 44, Lindley writes “Some people have put forward the argument that the only reason two persons differ in their beliefs about an event is that they have different knowledge bases, and if the bases were shared, the two people would have the same beliefs, and therefore the same probability. [...] We do not share this view ...” Fast forward to page 55: “... a storm in April can cause several failures, revealing that independence of [electricity supply] interruption events is not a sensible assumption.” Not sensible to whom? To Dennis Lindley? To me? To every rational person? If I happen to have the same knowledge base as that of Dennis Lindley, can I think that the interruptions are independent and still be a rational person? Page 58: “There is a well-established association between heavy consumption of saturated fat and heart disease found by observing people, but it does not follow that reducing the amount of fat [...] will reduce deaths from heart disease, although more recent evidence suggests that it may be true.” Given the body of facts alluded to in Lindley’s statement, am I still a rational person if I think that I will greatly diminish the probability of my heart disease by increasing my consumption of saturated fat by a factor of five?

page 57

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

58

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch03

Resonance: From Probability to Epistemology and Back

Lindley is a strict subjectivist when it suits his philosophical needs — he does not have to discuss the thorny question of what the objective probability is and how to measure it. Once he moves to applied arena, he sounds perfectly rational because his statements obviously refer to objective probabilities. De Finetti and Lindley proved to be complete failures as philosophers because they did not realize that the real philosophical problem with subjectivity extends in the direction exactly opposite to their theories. People believe that there is a solid dose of personal or subjective element in science, including probability, but this subjective or personal element is not detrimental to science and perhaps it is even helpful. Some form of subjectivity may be even a necessary ingredient of science — this seems to be the main message of [Polanyi (1958)]. I completely agree with Polanyi’s diagnosis but I find the medicine that he prescribed too bitter. His proposed solution of the problem of subjectivity is too philosophical in nature and leaves me unsatisfied so I will develop my own “resonance” theory to tackle the philosophical problem of “personal knowledge.”

page 58

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch04

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Chapter 4

The Frequency Philosophy of Probability

This chapter is devoted to a detailed critique of the frequency theory. I have already started this analysis in Sec. 3.4. The frequency theory illustrates well a natural tension between philosophy and science. In one area of intellectual activity, the weakest possible claims are the most convenient, while in the other, the strongest possible claims are the most practical. I will argue that there is an unbridgeable gap between the philosophical theory of collectives and the needs of science.

4.1 Collective as an Elementary Concept Scientific theories involve quantities and objects, such as mass, electrical charge, and sulfur. For a theory to be applicable as a science, these quantities and objects have to be recognizable and measurable by all people, or at least by experts in a given area of science. Some of these concepts are considered to be elementary or irreducible. In principle, one could explain how to recognize sulfur using simpler concepts, such as yellow color. However, the reduction has to stop somewhere, and every science chooses elementary concepts at the level that is convenient for this theory. For example, sulfur is an elementary concept in chemistry, although it is a complex concept in physics. An elementary concept is a quantity, object or situation that can be recognized via resonance, to be discussed in Sec. 8.2.

59

page 59

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

60

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch04

Resonance: From Probability to Epistemology and Back

The frequency theory is based on an elementary concept of “collective.” This theory does not offer any advice on what one can say about the probability of an event if there is no collective that contains the event. Hence, according to the frequency philosophy, students and scientists have to learn how to recognize collectives, just like children have to recognize cats, trees and colors. Once you can recognize collectives, you can apply probability theory to make predictions concerning relative frequencies of various events in the same collective, or different collectives. Von Mises pointed out that simple probabilistic concepts, such as conditioning, require that we sometimes use several collectives to study a single phenomenon. I think that the meaning of the above remarks can be appreciated only if we contrast them with the following common misinterpretation of the frequency theory. In this false interpretation, the point of departure is an i.i.d. sequence (I will argue in Secs. 4.12 and 4.13 that collectives cannot be identified with i.i.d. sequences in von Mises’ theory). Next, according to the false interpretation of the frequency theory, we can use the Law of Large Numbers to make a prediction that the relative frequency of an event will converge (or will be close) to the probability of the event. In von Mises’ theory, the convergence of the relative frequency of an event in a collective is a defining property of the collective and thus it cannot be deduced from more elementary assumptions or observations. Another way to see that an application of the Law of Large Numbers is a false interpretation of von Mises’ theory is to note that once we determine in some way that a sequence is i.i.d., then the convergence of relative frequencies is a consequence of the Law of Large Numbers, a mathematical theorem. Hence, the same conclusion will be reached by supporters of any other philosophy of probability, including logical and subjective, because they use the same mathematical rules of probability. A concept may be sometimes applied to an object or to a small constituent part of the object. For example, the concept of mass applies equally to the Earth and to an atom. Some other concepts apply only to the whole and not to its parts. For instance, tigers are considered to be aggressive but the same adjective is never applied to atoms in their bodies. The theory of von Mises requires a considerable mental effort to be internalized. Most people think about probability as an attribute of a single event. In the frequency theory, probability is an attribute of a sequence, and only a sequence.

page 60

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch04

The Frequency Philosophy of Probability

61

4.2 Applications of Probability Do Not Rely on Collectives

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

I will present three classes of examples where the frequency theory fails to provide a foundation for established scientific methods. In this section, I will not try to distinguish between collectives and i.i.d. or exchangeable sequences because I will not be concerned with the differences between these concepts. Instead, I will discuss their common limitations.

4.2.1 Stochastic processes A large number of sequences of random variables encountered in scientific practice and real life applications are not i.i.d. or exchangeable — it is a tradition to call them “stochastic processes.” Some of the best known classes of stochastic processes are Markov processes, stationary processes and Gaussian processes. Markov processes represent randomly evolving systems with short or no memory. Stationary processes are invariant under time shifts, that is, if we start observations of the process today, the sequence of observations will have the same probabilistic characteristics as if we started observations yesterday. Gaussian processes are harder to explain because their definition is somewhat technical. They are closely related to the Gaussian (normal) distribution which arises in the Central Limit Theorem and has the characteristic bell shape. One can make excellent predictions based on a single trajectory of any of these processes. Predictions may be based on various mathematical results such as the “ergodic” theorem or the extreme value theory. In some cases, one can transform a stochastic process mathematically into a sequence of i.i.d. random variables. However, even in cases when this is possible, this purely mathematical procedure is artificial and has little to do with von Mises’ collectives. The frequency philosophy is useless when we try to apply it to stochastic processes because the predictions mentioned above do not correspond to frequencies within any real collectives. As a slightly more concrete example, consider two casinos that operate in two different markets. Suppose that the amount of money gamblers leave in the first casino can be reasonably modeled as an i.i.d. sequence. In the case of the second casino, assume that there are daily cycles, because the types of gamblers that visit the casino at different times of the day are different. Hence, the income process for the second casino cannot be modeled by an i.i.d. process. Suppose that in both cases, we were able to

page 61

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

62

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch04

Resonance: From Probability to Epistemology and Back

confirm the models and estimate the parameters using data from the past. In each case we can make a prediction for earnings of the casino over the next year. The theory of von Mises can be applied directly to make an income prediction in the case of the first casino. In the case of the second casino, the frequency theory says that we have to find an i.i.d. sequence (collective), presumably a long sequence of similar casinos, to apply the probability theory. This is totally unrealistic. In practice, predictions for both casinos would be considered equally valuable, whatever that value might be. Nobody would even think of finding a sequence of casinos in the second case.

4.2.2 Unlikely events Another class of examples where the frequency theory is miles apart from the real science are situations involving very small probabilities. Suppose someone invites you to play the following game. He writes an arbitrarily chosen 20-digit integer on a piece of paper, without showing it to you. You have to pay him $10 for an opportunity to win $1,000, if you guess the number. Anyone who has even a basic knowledge of probability would decline to play the game because the probability of winning is a meager 9 · 10−19 . According to the frequency theory, we cannot talk about the probability of winning as long as there is no long run of identical games. The frequency philosophy has no advice to offer here although no scientist would have a problem with making a rational choice. Practical examples involve all kinds of very unlikely events, for example, natural disasters. Some dams are built in the US to withstand floods that may occur once every 500 years. We would have to wait many thousands of years to observe a sequence of floods sufficiently long so that the observed frequency could be matched, more or less, with the theoretical probability. In that time, several new civilizations might succeed ours. According to the frequency theory, it makes no sense to talk about the probability that a dam will withstand floods for the next 100 years. Even more convincing examples arise in the context of changing technology. Suppose that scientists determine that the probability that there will be a serious accident at a nuclear power plant in the US in the next 100 years is 1%. If the estimate is correct then one needs to wait for 10,000 years to observe one or a handful of accidents, or wait for 100,000 years to get a solid statistical confirmation of the probability value. This is totally unrealistic because the technology is likely to change in a drastic way much sooner than that, say, in 100 years. I do not think

page 62

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch04

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

The Frequency Philosophy of Probability

63

that anyone can imagine now what nuclear power plant technology will be 1,000 years from now. The 1% estimate has obviously a lot of practical significance but it cannot be related to any “long run” of observations that could be made in reality. There are many events that are not proven to be impossible but have probability so small that they are considered impossible in practice, and they do not fit into any reasonable long run of events. It is generally believed that yeti does not exist and that there is no life on Venus with very high probability. If we take the frequency theory seriously, we cannot make any assertions about probabilities of these and many other equally unlikely events — this is a sure recipe for the total paralysis of life as we know it.

4.2.3 Graphical communication It is common in science to present the results of research using graphs such as in Figures 4.1–4.3. Figures of this kind often represent simulations. In a typical case, the figure is supposed to make a visual statement that would be hard to convey by words. Most of the time the figure shows a “typical” shape of a random function or some other random mathematical object. Figures 4.1–4.3 show simulations of fractional Brownian motion with three different values of the “Hurst parameter.” The graph becomes smoother as the parameter becomes larger. In each of the three cases, one of the possible shapes of the fractional Brownian motion is a straight line, depicted in

1

0.2

0.4

0.6

0.8

1.0

−1

−2

−3

Fig. 4.1

Fractional Brownian motion with Hurst parameter 0.1.

page 63

April 29, 2016

9:6

64

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch04

Resonance: From Probability to Epistemology and Back

0.5

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

0.2

0.4

0.6

0.8

1.0

−0.5

Fig. 4.2 Fractional Brownian motion with Hurst parameter 0.5 (standard Brownian motion).

0.1

0.2

0.4

0.6

0.8

1.0

−0.1

−0.2

−0.3

Fig. 4.3

Fractional Brownian motion with Hurst parameter 0.9.

Figure 4.4. Strictly speaking, the trajectory cannot be perfectly straight but it can be straight with any given accuracy. In particular, it can appear to be straight to humans, with strictly positive probability. The probability of seeing a straight trajectory of fractional Brownian motion with parameter less than, say, 0.95 is extremely small. The message conveyed by Figures 4.1–4.4 is that fractional Brownian motion processes take the shape of a straight line with negligible probability. Moreover, the paths of fractional Brownian motions with different values of the Hurst parameter differ in

page 64

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch04

The Frequency Philosophy of Probability

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

0

65

1

Fig. 4.4 The straight line in this figure is a possible trajectory of a discrete approximation to fractional Brownian motion with any Hurst parameter. The probability of such a trajectory is strictly positive but extremely small.

roughness with overwhelming probability. We do not have to (although we can) look at a long sequence of such simulations to “get the picture.” One look at the graphs in Figures 4.1 and 4.3 is all that one needs to tell which of the two processes has a larger Hurst parameter. For readers who have some technical knowledge of stochastic processes I would like to point out that I used fractional Brownian motion as my illustration because its increments are not i.i.d. Suitable transformations can be used to represent the process as a function of an i.i.d. sequence but this has nothing to do with von Mises’ collectives.

4.3 Collectives in Real Life The concept of a “collective” invented by von Mises is an awkward attempt to formalize the idea of repeated experiments or observations. Two alternative ways to formalize this idea are known as an i.i.d. sequence and “exchangeable” sequence, the latter favored by de Finetti. Exchangeability is a form of symmetry — according to the definition of an exchangeable sequence, any rearrangement of a possible sequence of results is as probable as the original sequence. The idea of an i.i.d. sequence stresses independence of one experiment in a series from another experiment in the same series, given the information about the probabilities of various results for a single experiment. In interesting practical applications, this information is missing, and then, by de Finetti’s theorem, an i.i.d. sequence can be equivalently thought of as an exchangeable sequence (see Sec. 18.1.2). A fundamental problem with collectives is that they would be very hard to use, if anybody ever tried to use them. Scientists have to analyze data collected in the past and also to make predictions. For the concept of a collective to be applicable to the past data, a scientist must be able

page 65

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

66

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch04

Resonance: From Probability to Epistemology and Back

to recognize a collective in an effective way. The definition of a collective suggests that one can determine the lack of patterns in the data either at the intuitive level, by direct examination of the sequence, or using some more formalized but practical procedure. I will discuss scientific methods of detecting patterns below. On the informal side, I do not think that people can effectively determine whether a sequence contains non-i.i.d. patterns. A convincing support for my position is provided by a “java applet” prepared by Susan Holmes and available online [Holmes (2007)]. The program generates two binary sequences, one simulating i.i.d. events, and the other one representing a (non-trivial) Markov chain, that is, dependent events. Many people find it hard to guess whether an unlabeled sequence is i.i.d. or non-i.i.d. The definition of a collective requires that for a given event, the relative frequency of that event in the sequence (collective) converges, and the same is true for every subsequence of the collective chosen without clairvoyant powers (the limit must be always the same). I believe that people can effectively recognize symmetries and independence via resonance (to be discussed in Sec. 8.2). Our ability to recognize the lack of patterns along “all” subsequences, postulated by von Mises, is an illusion. The requirement that the relative frequencies have the same limits along “all” subsequences is especially hard to interpret if one has a finite (but possibly long) sequence. In this case, we necessarily have limits 1 and 0 along some subsequences, and it is hard to find a good justification for eliminating these subsequences from our considerations. The purpose of the requirement that the limit is the same along all subsequences is to disallow sequences that contain patterns, such as seasonal or daily fluctuations. For example, temperatures at a given location show strong daily and seasonal patterns so temperature readings do not qualify as a collective. Surprisingly, this seemingly philosophically intractable aspect of the definition of a collective turned out to be tractable in practice in quite a reasonable way. Some of the most important tools used in modern statistics and other sciences are random number generators. These are either clever algebraic algorithms (generating “pseudo-random” numbers) or electronic devices generating random numbers (from thermal noise, for example). From the practical point of view, it is crucial to check that a given random number generator does not produce a sequence that contains patterns, so there is a field of science devoted to the analysis of random number generators. The results seem to be very satisfactory in the sense

page 66

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch04

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

The Frequency Philosophy of Probability

67

that most statisticians and scientists can find random number generators sufficiently devoid of patterns to meet their needs. In this special sense, von Mises is vindicated — it is possible to check in practice if a sequence is a collective. However, widely used and accepted methods of checking whether a sequence is “truly” random, such as George Marsaglia’s battery of tests [Marsaglia (1995)], do not even remotely resemble von Mises’ idea of checking the frequency of an event along every subsequence. Mathematics is used in science to reduce the number of measurements and to make predictions, among other things. A scientist makes a few measurements and then uses mathematical formulas appropriate for a given science to find values of some other quantities. If we adopt the frequency view of probability, the only predictions offered by this theory are predictions involving limits of long run relative frequencies. According to the frequency theory, even very complex mathematical results in probability theory should be interpreted as statements about long run frequencies for large collections of events within the same collective. In certain applications of probability, such as finance, this is totally unrealistic. The frequency view of the probability theory as a calculus for certain classes of infinite sequences is purely abstract and has very few real applications.

4.4 Collectives and Symmetry A scientific theory has to be applicable in the sense that its laws have to be formulated using terms that correspond to real objects and quantities observable in some reasonable sense. There is more than one way to translate the theory of collectives into an implementable theory. If we use a collective as an observable, we will impose a heavy burden on all scientists, because they will have to check for the lack of patterns in all potential collectives. This is done for random number generators out of necessity and in some other practical situations when the provenance of a sequence is not fully understood. But to impose this requirement on all potential collectives would halt the science. An alternative way is to identify a collective with an exchangeable sequence. The invariance under permutations (that is, the defining feature of an exchangeable sequence) can be ascertained in a direct way in many practical situations — this eliminates the need for testing for patterns. This approach is based on symmetry, and so it implicitly refers to (L4) and more generally, to (L1)–(L6). Hence, either it is impossible to implement the concept of a collective or the concept is redundant.

page 67

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

68

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch04

Resonance: From Probability to Epistemology and Back

There is another, closely related, reason why the concept of a collective is almost useless without (L1)–(L6). Typically, when a scientist determines a probability by performing a large number of experiments or collecting a large number of observations, she wants to apply this knowledge in some other context — one could even say that this is the essence of science. Consider the following routine application of statistics. Suppose a group of 1,000 patients were given a drug and improvement occurred in 65% of cases. A statistician can make a prediction that out of 2 million people afflicted by the same ailment, if they all take this drug, about 1.3 million will experience improvement of their condition. The statistician must be able to select a part of the general population to which the prediction can be applied. Obviously, the statistician cannot observe any limits along any subsequences until the drug is actually widely used. Making a prediction requires that the statistician uses symmetry to identify the relevant part of the population — here, applying symmetry means identifying people with similar medical records. One can analyze the performance of the drug a posteriori, and look at the limits along various subsequences of the data on 2 million patients. Checking whether there are any patterns in the data may be useful but this does not change in any way the fact that making a prediction requires the ability to recognize symmetries. If we base probability theory on the concept of a collective, we will have to apply knowledge acquired by examining one collective to some other collective. A possible way to do that would be to combine the two collectives into one sequence and check if it is a collective. This theoretical possibility can be implemented in practice in two ways. First, one could apply a series of tests to see if the combined sequence is a collective — this would be a solid but highly impractical approach, because of its high cost in terms of labor. The other possibility is to decide that the combined sequence is a collective (an exchangeable sequence) using (L4), that is, to recognize the invariance of the combined sequence under permutations. This is a cost-efficient method but since it is based on (L4), it makes the concept of a collective redundant.

4.5 Frequency Theory and the Law of Large Numbers It is clear that many people think that a philosophy of probability called the “frequency theory” is just a philosophical representation of the mathematical theorem and empirical fact known as the “Law of Large

page 68

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch04

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

The Frequency Philosophy of Probability

69

Numbers.” Paradoxically, the philosophical approach to probability chosen by von Mises makes it practically impossible to apply the Law of Large Numbers in real life. A simple version of the Law of Large Numbers says that if we have a sequence of independent experiments such that each one can result in a “success” or “failure,” and the probability of success on each trial is equal to the same number p then the proportion of successes in a long sequence of such trials will be close to p. Probabilists call such a sequence “Bernoulli trials.” In von Mises’ philosophical theory, probabilities are not assigned to individual events. Hence, to give a meaning to the Law of Large Numbers, we have to represent the “probability of success on the kth trial” as a long run frequency. If we want to apply the Law of Large Numbers in the context of the frequency theory, we have to consider a long sequence of long sequences. The constituent sequences would represent individual trials in the Bernoulli sequence. To apply the Law of Large Numbers in real life, you have to recognize an i.i.d. sequence and then apply the Law of Large Numbers to make a prediction. The von Mises theory says that you have to recognize a collective, that is, a sequence that satisfies the Law of Large Numbers. The frequency theory failed to recognize the real strength of the Law of Large Numbers — one can use the Law of Large Numbers to make useful predictions starting from simple assumptions and observations.

4.6 Why is Mathematics Useful? I guess that there are many unrelated reasons why mathematics is useful. I will discuss only one of them in relation to the frequency philosophy of probability. Consider a scientific law that involves a mathematical formula, such as F = ma, one of Newton’s laws of motion. Here F is the force, m is the mass and a is the acceleration. Suppose that we verified that the law is correct with sufficient certainty. What practical benefits can we derive from the knowledge that F = ma? If we can measure two of these quantities, say, F and a, then we can calculate the third one using the formula, that is, m = F/a. And then we do not have to measure the third quantity, that is m. We see that mathematics can eliminate some measurements. If we could measure accurately and easily every physical quantity then the need for mathematics would be greatly diminished.

page 69

April 29, 2016

9:6

70

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch04

Resonance: From Probability to Epistemology and Back

An elementary probabilistic formula says that

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

P (A or B) = P (A) + P (B) − P (A and B).

(4.1)

According to the frequency philosophy of probability, this formula can be applied only to observed or observable frequencies. The same remark applies to all other formulas in the mathematical theory of probability. In other words, according to the frequency theory of probability, the only useful application of mathematics to probability is to calculate frequencies of some events from the information about frequencies of some other events. Knowing frequencies is useful but in some of the best known practical examples only one frequency matters. For instance, frequency statisticians claim that the null hypothesis is incorrectly rejected (“Type I error”) with a small frequency, given appropriate circumstances. Knowing this frequency is useful but the utility of this information does not have anything to do with the relationship between frequencies expressed in (4.1) or any other mathematical formula. The probability of Type I error is mathematically related to other probabilities because it can be derived from the statistical model. The mathematical relationships used in the derivation of the probability of Type I error do not correspond to any useful relationships between observed or observable frequencies. This is especially clear when we deal with long sequences of non-isomorphic hypothesis tests, a situation common in academic setting. The frequency of Type I error is the only observable and useful frequency in this context. The frequency theory of probability fails to explain why it is beneficial to use the mathematical theory of probability to derive the probability (frequency) of Type I errors.

4.7 Benefits of Imagination and Imaginary Benefits A possible argument in defense of the frequency approach to probability is that even though long runs of experiments or observations do not exist in some situations, we can always imagine them. What can be wrong with using our imagination? I will first examine the general question of the benefits of imagination, before discussing imaginary collectives. One of the human abilities that makes us so much more successful than other animals is the ability to imagine complex future sequences of events, complex objects, objects that have not been made yet, etc. What is the practical significance of imagining a car? After all, you cannot drive an imaginary car. Everything we imagine can be used to make rational

page 70

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch04

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

The Frequency Philosophy of Probability

71

choices and take appropriate actions. In this sense, imagining a spaceship traveling twice as fast as the speed of light is as beneficial as imagining a spaceship traveling at the speed of 10 kilometers per second. We can use the conclusions that we arrive at by imagining both spaceships to design and build a spaceship that will actually reach Mars. It is important to distinguish between the benefits of imagination and imagined benefits. We can imagine benefits of building a spaceship that can travel to Mars but we will actually benefit only if we build the spaceship and it reaches Mars. In the context of frequency statistics, imagination can be invoked to justify most popular statistical methods, such as unbiased estimators or hypothesis testing. Suppose that we try to estimate the value of a physical quantity, such as the density of a material, and we make a series of measurements. Under some assumptions, the average of the measurements is an unbiased estimate of the true value of the density. The statement that the estimator is unbiased means that the expected value of the average is equal to the true density. The frequentist interpretation of this statement requires that we consider a long sequence of sequences of measurements of the density. Then the average of the sequence of estimates (each based on a separate sequence of measurements) will be close to the true value of the density. This is almost never done in reality. One good reason is that if a sequence of sequences of identical measurements of the same quantity were ever done, the first thing that statisticians would do would be to combine all the constituent sequences into one long sequence. Then they would calculate only one estimate — the overall average. I do not see any practical justification for imagining a sequence of sequences of measurements, except for some vague help with creative thinking.

4.8 Imaginary Collectives The last section was concerned with practical implications of imagination. I will now point out some philosophical problems with imagined collectives. Since we do not have direct access to anyone’s mind, imaginary collectives have no operational meaning. In other words, we cannot check whether anyone actually imagines any collectives. Hence, we can use imagination in our own research or decision making but our imagined collectives cannot be a part of a meaningful scientific theory. Contemporary computers coupled with robots equipped with sensors can do practically everything that humans can do (at least in principle) except for mimicking human mind functions. In other words, we can program a computer or robot

page 71

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

72

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch04

Resonance: From Probability to Epistemology and Back

to collect data, analyze them, make a decision and implement it. We cannot program a computer to imagine collectives and it is irrelevant whether we will be ever able to build computers with an imagination — the imagination would not make them any more useful in this context. A different problem with imagined collectives is that in many (perhaps all) cases one can imagine more than one collective containing a given event. In many such cases, the probability of the event is different in the two imagined collectives. Consider a single toss of a deformed coin. This single event can be imagined to be a part of a collective of tosses of the same deformed coin, or a part of a collective of experiments consisting of deforming different coins and tossing each one of them once. Both collectives are quite natural and one can easily perform both types of experiments. The long run frequency of heads may be different in the two collectives (see Sec. 11.12). The frequency interpretation of probability is like the heat interpretation of energy. The “essence” of energy can be explained by saying that energy is something that is needed to heat a sample of water from temperature 10◦ C to 20◦ C (the amount of energy needed depends on the amount of water). If we drop a stone, its potential energy is converted to the kinetic energy and the heat energy is not involved in this process in any way. One can still imagine that the potential and kinetic energies can be converted to heat that is stored in a sample of water. In real life this step is not necessary to make the concept of energy and its applications useful. Similarly, one can always imagine that the probability of an event is exemplified by finding an appropriate i.i.d. sequence and observing the long run relative frequency of the event in the sequence. In real life this step is not necessary to make the concept of probability and its applications useful.

4.9 Computer Simulations It is quite often that we cannot find a real sequence that could help us find the probability of an event by observing the relative frequency. One of my favorite examples is the probability that a given politician is going to win the elections. I do not see a natural i.i.d. sequence (or collective) into which this event would fit. Many people believe (see [Ruelle (1991), p. 19], for example) that computer simulations provide a modern answer to this philosophical and scientific problem. I will argue that this is not the case. Computer simulations are an excellent scientific tool, allowing scientists to calculate probabilities with great accuracy and great reliability

page 72

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch04

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

The Frequency Philosophy of Probability

73

in many cases. But they cannot replace a non-existent real sequence. On the philosophical side, computer simulations contribute very little to the discussion. Contemporary computers can simulate very complex random systems. Every year the speed and memory size of computers increase substantially. However, computer simulations generate only an estimate of probability or expectation that under ideal circumstances could be obtained in the exact form, using pen and paper. In this way, computer simulations play the same role as numerical calculations (that is, deterministic computations generating an estimate of a mathematical quantity). Consider a statistician who does not have a full understanding of a real phenomenon. Computer simulations may yield a very accurate probability estimate, but this estimate pertains to the probability of an event in the statistician’s model. If the statistician does not understand the real situation well then there is no reason to think that the result of simulations has anything to do with reality. There is a huge difference between estimating what people think about the mass of the Moon, and estimating the mass of the Moon (although the two estimates can be related). Recall that an event may belong to more than one “natural” sequence (see Sec. 11.12). One could simulate all these sequences and obtain significantly different estimates of the probability of the event. The philosophical and practical problem is to determine which of the answers is relevant and simulations offer no answer to this question. Computer simulations will not turn global warming into a problem well placed in the framework of von Mises’ frequency theory. The problem is not the lack of data. According to von Mises, a single event does not have a probability. No matter how many atoms you simulate, you cannot determine whether an atom is aggressive. This is because the concept of aggression does not apply to atoms. No matter how many global warmings you simulate, you cannot determine the probability of global warming in the next 50 years. This is because the concept of probability does not apply to individual events, according to von Mises.

4.10 Frequency Theory and Individual Events Scientists who deal with large data sets or who perform computer simulations consisting of large numbers of repetitions might have hard time understanding what is wrong with the frequency theory of probability. Isn’t the

page 73

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

74

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch04

Resonance: From Probability to Epistemology and Back

theory confirmed by empirical evidence? The problem with the frequency theory is that it is a philosophical theory and so its primary intellectual goal is to find the true essence of probability. For philosophical reasons, the theory denies the possibility of assigning probabilities to individual events. Can we alter the frequency theory and make it more realistic by admitting that individual events have probabilities? Suppose that a philosopher takes a position that individual events do have probabilities. It is natural to assume that in his theory, one could assign probabilities to all possible outcomes in a sequence of two trials. Similarly, the theory would cover sequences of trials of length three, four, ... , one million. Hence, there would be no need to provide a separate philosophical meaning to long sequences and relative frequencies of events in such sequences. The Law of Large Numbers, a mathematical theorem, says that if an event has probability p, then the frequency of such events in a sequence of i.i.d. trials will be close to p with high probability. This is the statement that frequentists seem to care most about. The statement of the Law of Large Numbers does not contain any elements that need the philosophical theory of collectives, if we give a meaning to probabilities of individual events. Once a philosopher admits that individual events have probabilities, the theory of collectives becomes totally redundant. I have to mention that Hans Reichenbach, a frequentist respected by some philosophers even more than von Mises (see [Weatherford (1982), p. 144]), believed that the frequency theory can be applied to individual events. I have to admit that I do not understand this position. Moreover, Reichenbach’s philosophy seems to be closer to the logical theory than frequency theory.

4.11 Collectives and Populations Von Mises’ collectives were not intended to be interpreted as fixed populations. We can see this from two features of von Mises’ theory. (i) The frequency interpretation of probability holds in all fixed populations, of any size. Von Mises clearly thought about collectives as large sets. (ii) The order of events in a fixed population does not matter — the usual probabilistic formulas hold for frequencies in a population whether it is ordered or not. The fundamental property of von Mises’ collectives is that their elements are arranged in a way that shows no patterns. Suppose that you have a box of sand with 108 grains of sand. One sand grain has been marked using a laser and a microscope. If you pay $10, you

page 74

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch04

The Frequency Philosophy of Probability

75

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

can choose “randomly” a grain of sand. If it has the mark, you will receive $1,000. Just after a grain is sampled, all the sand will be dumped into the sea. I doubt that anyone would play this game. The number of grains of sand in the box is enormous. Does the frequency theory support the decision not to play the game? In other words, does the frequency theory say that the probability of finding the marked grain is 10−8 ? I will argue that it does not, despite von Mises’ claim to the contrary (page 11 of [von Mises (1957)]): ... concept of probability ... applies ... to problems in which either the same event repeats itself again and again, or a great number of uniform elements are involved at the same time.

The frequency theory assigns probabilities to long sequences of events. The above game is concerned with only one event. It does involve a large number, 108 , but that number represents the size of the population (collection) of sand grains, not the length of any sequence of events. A collective is a family of events that does not contain patterns. This condition can apply to a sequence, that is, an ordered set. A population is unordered. To bring it closer to the notion of a collective, one has to endow it with an order. In some cases, for example, a sand box, there seems to be no natural order for the elements of the population. Some orderings of a finite population will obviously generate patterns. If we decide to consider only those orderings that do not generate patterns, the procedure seems to be tautological in nature — an ordering might be a collective in the sense of having no patterns because we have chosen an ordering that has no patterns. I doubt that it is worth anyone’s time to try to find a fully satisfactory version of the theory of collectives that includes populations.

4.12 Are All i.i.d. Sequences Collectives? Consider the following sequence of events. (A1 ) There will be a snowstorm in Warsaw on January 4th next year. (A2 ) There will be at least 300 car accidents in Rio de Janeiro next year. (A3 ) There will be at least 80 students in my calculus class next spring. Suppose that we know that each one of these events has probability 70%. Assume that the sequence is not limited to the three events listed above but that it continues, so that it contains at least 1,000 events, all of them ostensibly unrelated to each other. Assume that each of the 1,000 events

page 75

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

76

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch04

Resonance: From Probability to Epistemology and Back

is 70% certain to happen. Then the sequence satisfies the mathematical definition of an “i.i.d.” sequence, that is, all events are independent and have the same probability. Is this sequence a collective? I will argue that the answer is no. Standard examples of collectives, such as tosses of a deformed coin or patients participating in medical trials, are characterized by convergent frequencies of specified events. In these examples, we believe that the frequencies converge to a limit no matter whether we can determine what the limit is or not (prior to observing the sequence). For example, we believe that if we are given a deformed coin to analyze, the frequency of heads will converge to a limit, although we do not know what the limit might be. We believe that the events in the sequence A1 , A2 , A3 , . . . described above will occur with frequency close to 70% only because we determined this probability separately for each element of the sequence, in some way. Hence, we implicitly assume that individual events, such as A1 , have probabilities, the existence of which von Mises denied. The above philosophical analysis has some practical implications. Sequences such as A1 , A2 , A3 , . . . can be used to make predictions. For example, suppose that employees of a company make thousands of unrelated decisions, and somehow we are able to determine that each decision results in “success” with probability lower than 80%. Then we can make a verifiable prediction that decisions will be successful at a (not necessarily stable) rate lower than 80%. One can make successful predictions for long run frequencies even if the theory of collectives does not apply.

4.13 Are Collectives i.i.d. Sequences? In the next two sections, I will present two attempts at a rigorous definition of a collective. The first representation is my own and is no more than the original von Mises definition expressed in the language of the modern theory of stochastic processes. The second representation interprets a collective as an appropriately “random” deterministic sequence. I will use some advanced concepts, typically introduced at the Ph.D. level. It has been pointed out that von Mises was close to inventing the concept of a stopping time, fundamental to the modern theory of stochastic processes. The following mathematical definition tries to capture the concept of a collective using the idea of a stopping time. Suppose that X1 , X2 , X3 , . . . are random variables taking values 0 or 1. Let Fn denote the σ-field generated by X1 , . . . , Xn . We call T a predictable stopping

page 76

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch04

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

The Frequency Philosophy of Probability

77

time if T is a random variable taking values in the set of strictly positive integers and for every n ≥ 2, the event {T = n} is Fn−1 -measurable. We will call X1 , X2 , X3 , . . . a stochastic collective if for some p ∈ [0, 1] and every sequence T1 , T2 , T3 , . . . of predictable stopping times such that
T1 < T2 < T3 < . . . a.s., we have limn→∞ nk=1 XTk /n = p, a.s. Stochastic collectives defined above are not the same as i.i.d. sequences, although every i.i.d. sequence is a stochastic collective. I will now briefly justify the first of these claims. Note that for any stochastic collective X1 , X2 , X3 , . . . and any fixed n, we can arbitrarily modify X1 , . . . , Xn and obtain in this way a new collective. It is obvious (to mathematically trained people) that if the sequence X1 , X2 , X3 , . . . is i.i.d. then the modified sequence does not have to be i.i.d. I do not know whether sequences of random variables that I call “stochastic collectives” were studied. In particular, I do not know whether the Central Limit Theorem has been ever proved for stochastic collectives, and I doubt that it holds. I also doubt that the scientific community has much interest in proving or disproving the Central Limit Theorem for stochastic collectives.

4.14 Martin-L¨ of Sequences Some commentators believe that von Mises had deterministic sequences in mind when he defined collectives. The somewhat vague concept of a deterministic collective (or deterministic random sequence) was formalized by Per Martin-L¨of in 1966. There are several equivalent definitions of a Martin-L¨ of sequence (see [Wikipedia (2014i)]). Roughly speaking, we call a sequence “Martin-L¨of” if it passes a countably infinite family of tests for randomness. The tests were chosen to be reasonable (“computable”) from the computer theoretical point of view. See [Wikipedia (2015l)] for more information on algorithmic randomness. The Martin-L¨ of sequence is an unquestionably interesting theoretical concept that spurred significant research activity in logic and theoretical computer science. At the same time, the concept is totally useless from the practical point of view, for two independent reasons. First, just like in the case of my “stochastic collective” (see Sec. 4.13), any modification of a finite number of elements of a Martin-L¨ of sequence turns it into another Martin-L¨ of sequence. Imagine that a statistician collected a random sample consisting of 1,000 data. It is clear that replacing the first 100 of the data with arbitrary numbers would greatly

page 77

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

78

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch04

Resonance: From Probability to Epistemology and Back

affect the value of the data for statistical analysis. The invariance of the family of Martin-L¨ of sequences under finite modifications is mathematically convenient but fails to capture the essence of statistical samples. The second problem with Martin-L¨of sequences, from the point of view of science and philosophy of probability, is that they cannot be effectively matched with real sequences. I will illustrate this claim using the Law of Iterated Logarithm (LIL). This law is a well known result in the classical probability theory asserting that, with probability 1, properly defined fluctuations in an i.i.d. sequence of 0’s and 1’s eventually stay within certain deterministic bounds. The bounds involve the square root function, similar to that in the Central Limit Theorem, and a “correction” consisting of “iterated logarithm.” I will argue in Sec. 11.4 that our “resonance” abilities are sufficient to let us recognize symmetric and independent events, and, therefore, i.i.d. sequences. I doubt that we have the ability to recognize sequences with convergent relative frequencies along all subsequences chosen without prophetic powers (see Sec. 4.3). In other words, I do not think that we are able to implement von Mises’ definition of a collective in real life. Since Martin-L¨ of sequences have to satisfy the LIL, a property much more subtle than convergence of relative frequencies, there is no even remote possibility that anyone could match the definition of a Martin-L¨of sequence with a sequence of real data in the same intuitive way that i.i.d. or exchangeable sequences are recognized. On the theoretical side, Martin-L¨of sequences may have some appeal for logicians and theoretical computer scientists but they are totally unappealing to probabilists, including the author of this book. Probability theory cannot be developed on the basis of Martin-L¨ of sequences. No part of statistical theory has been formulated in terms of Martin-L¨of sequences and I do not see any reason to expect that it will ever be. The lack of success of Martin-L¨of sequences underscores the failure of the original von Mises’ program.

page 78

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch05

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Chapter 5

The Subjective Philosophy of Probability

“Motion does not exist.”

Zeno of Elea (c. 450 B.C.)

“Probability does not exist.”

Bruno de Finetti (c. 1950 A.D.)

The subjective theory of probability is by far the most confused theory among all scientific, mathematical and philosophical theories of probability. This is a pity, because several beautiful and interesting ideas went into its construction (no sarcasm here). A lesson for thinkers is that even the most innovative and promising ideas may lead to an intellectual dead end. Contrary to the popular opinion, de Finetti was not a subjectivist — he was an antisubjectivist. He believed that some knowledge related to probability was objective and some was subjective. His theory denied any value to the personal probabilistic opinions. This constitutes a fundamental and complete discord between de Finetti’s philosophy and the mainstream subjective approach to probability. Pragmatic subjectivists are correct in that they see some value in “subjective” (personal, subconscious) opinions about probability. I will explain the value of “personal” knowledge in my theory of resonance, see Sec. 8.2. Lindley, a prominent subjectivist, definitely recognized value of subjective probabilistic opinions in [Lindley (2006)] but he could not describe this value and was confused about how we coordinate subjective opinions. See Sec. 3.5 for the initial part of the analysis of de Finetti’s theory.

79

page 79

April 29, 2016

9:6

80

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch05

Resonance: From Probability to Epistemology and Back

5.1 “Subjective” — A Word with a Subjective Meaning

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

I am convinced that 99% of the support for the subjective interpretation of probability stems from the confusion concerning the meaning of the word “subjective.” I believe that most people consider subjectivity of probability to be one of the following two phenomena. (1) Different people have different knowledge bases so their assessments of probabilities are different. (2) In practical situations we often lack information or theories that would yield precise probability values so we have to content ourselves with approximate values based on our intuition. I will argue that de Finetti’s theory has absolutely nothing to do with the above unquestionably real phenomena. (1) It is easy to see that for some events A, B and C, the conditional probability of A given B is different from the conditional probability of A given C. Formally, P (A | B)
= P (A | C)

(5.1)

for some A, B and C. Informally speaking, different people may estimate the probability of an event A in different ways because they have different prior information represented in the formula by B and C. Formula (5.1) is a part of the standard mathematical theory of probability based on Kolmogorov’s axioms. Since this mathematical theory is accepted by virtually all scientists and philosophers, it is not controversial at all. If believing in (5.1) means that one is a subjectivist then we are all subjectivists — not a very interesting philosophical thought. It is absurd to think that de Finetti invented (5.1) or that (5.1) is the core of his philosophical theory. That would make his theory much worse than wrong — it would make his theory trivial. The main claim of de Finetti’s philosophical theory can be represented by a different formula, namely, P1 (A | B)
= P2 (A | B). De Finetti claimed that two people possessing the same information might differently assess the probability of a future event (P1 versus P2 ) and there was no objective way to verify who was right and who was wrong. (2) It is obviously true that, quite often, we do not have sufficient information about facts or knowledge of reliable theories to make accurate probability assessments. It is equally obvious that, quite often, we do not

page 80

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch05

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

The Subjective Philosophy of Probability

81

have sufficient information about facts or knowledge of reliable theories to make accurate assessments of temperature, distance, electrical charge, etc. In everyday life, we may have no access to measuring instruments. Nobody will refrain from saying “My coffee is cold” (if she believes that this is the case) just because she has no thermometer to measure the temperature in an accurate way. Scientists cannot measure many physical quantities with great accuracy for a whole spectrum of reasons, from profoundly theoretical (Heisenberg’s uncertainty principle) and practical (some galaxies are too distant) to trivial (lack of money). I am not aware of “subjective philosophies” of temperature, distance, electrical charge, etc. Of course, psychologists study human perceptions of probability, temperature, distance, electrical charge, etc. but this does not indicate in any way anyone’s belief that these concepts are subjective. If “subjective” means that a quantity is sometimes informally evaluated then obviously probability is subjective but so are temperature and distance. De Finetti did not create a subjective philosophy for all scientific quantities. So his idea of “subjective probability” must have been different from informal assessment of objective probability. I will continue the review of various meanings of the word “subjective” in Sec. 5.23.

5.2 The Subjective Theory of Probability is Objective The labels “personal probability” and “subjective probability” used for de Finetti’s theory are highly misleading. His theory is objective and has nothing to do with any personal or subjective opinions. De Finetti says that certain actions have a deterministic result, namely, if you take actions that are coordinated in a way called “consistent” or “coherent” then there will be no Dutch book formed against you. This argument has nothing to do with the fact that the decision maker is a person. The same argument applies to other decision makers: businesses, states, computer programs, robots and aliens living in a different galaxy. In practice, probability values in de Finetti’s theory have to be chosen by people but this does not make his theory any more subjective than Newton’s laws of motion. A person has to choose a body and force to be applied to the body but this does not make acceleration predicted by Newton’s laws subjective. Newton’s claim is that the acceleration depends only on the mass of the body and the strength of the force and has nothing

page 81

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

82

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch05

Resonance: From Probability to Epistemology and Back

to do with the personal or any other way in which the body and force have been chosen. Similarly, de Finetti’s predictions are objective. If a monkey chooses a consistent decision strategy by randomly pounding on the keyboard, de Finetti’s theory makes an objective and verifiable prediction that no Dutch book will be formed against the beneficiary of monkey’s decisions. This brings us to a closely related issue that needs to be clarified. In de Finetti’s theory, one should talk about a beneficiary and not a decision maker. The two can be different, for example, a computer can be the decision maker and a human can be the beneficiary, or an employee can be the decision maker and her employer can be the beneficiary. De Finetti’s analysis makes predictions concerning the beneficiary. The decision maker is in the background, he is almost irrelevant. The use of adjectives “personal” and “subjective” is misleading because it suggests that the decision maker is the main protagonist of the theory. In fact, the theory is concerned exclusively with predictions concerning the beneficiary. De Finetti is sometimes portrayed as a person who tried to smuggle a non-scientific concept (subjectivity) into science, like an alchemist or astrologer. Nothing can be further from the truth. De Finetti wanted to purge a non-scientific concept — probability — from science. De Finetti did not promote the idea that one should use personal or subjective probability because doing so would be beneficial. He was saying that choosing probabilities in a consistent way was necessary if one wanted to avoid deterministic losses. Choosing probabilities by applying personal preferences is practical but there is nothing in de Finetti’s theory which says that using any other way of choosing probabilities (say, using computer software) would be less beneficial. A chemist or engineer could propose a “theory of painting” by claiming that “painting a wooden table increases its durability.” The following practical choices have no effect on this prediction: (i) the color of the paint (as long as the paint chemical composition satisfies appropriate quality standards), (ii) the gender of the painter, (iii) the day of the week. De Finetti made a scientific and verifiable prediction that you can avoid a Dutch book situation if you use probability. The following choices have no effect on de Finetti’s prediction: (i) the probability values (as long as they satisfy Kolmogorov’s axioms), (ii) the gender of the decision maker, (iii) the day of the week. De Finetti’s theory is no more subjective than the “theory of painting.”

page 82

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch05

The Subjective Philosophy of Probability

83

5.3 A Science without Empirical Content

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

De Finetti’s theory fails one of the basic tests for a scientific theory of probability — it does not report any probabilistic facts or patterns observed in the past. I will illustrate this claim with some examples from physics and probability. Facts and patterns can be classified according to their generality. Consider the following facts and patterns. (A1) John Brown cut a branch of a tree on May 17, 1963 and noticed that the saw was very warm when he finished the task. (A2) Whenever a saw is used to cut wood, its temperature increases. (A3) Friction generates heat. (A4) Mechanical energy can be transformed into heat energy. (A5) Energy is always preserved. I might have skipped a few levels of generality but I am sure that the example is clear. Here are probabilistic counterparts of the above facts and patterns. (B1) John Brown flipped a coin on May 17, 1963. It fell heads up. (B2) About 50% of coin flips in America in 1963 resulted in heads. (B3) Symmetries in an experiment such as coin tossing or in a piece of equipment such as a lottery machine are usually reflected by symmetries in the relative frequencies of events. (B4) Probabilities of symmetric events, such as these in (B3), are identical. I consider the omission of (B4) from the subjective theory to be its fatal flaw that destroys its claim to be a scientific theory representing probability. I will examine possible excuses for the omission. Some sciences, such as paleontology, report individual facts at the same level of generality as (A1) or (B1) but I have to admit that the theory of probability cannot do that. One of the reasons is that the number of individual facts relevant to probability is so large that they cannot be reported in any usable way, and even if we could find such a way, the current technology does not provide tools to analyze all the data ever collected by the humanity. The omission of (B2) by the subjective theory is harder to understand but it can be explained. It is obvious that most people consider this type of information useful and relevant. Truly scientific examples at the level of generality of (B2) would not deal with coin tosses but with repeated

page 83

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

84

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch05

Resonance: From Probability to Epistemology and Back

measurements of scientific constants, for example, the frequency of side effects for a drug. It is a legitimate claim that observed patterns at this level of generality belong to various fields of science such as chemistry, biology, physics, etc. They are in fact reported by scientists working in these fields and so there is no need to incorporate them into the theory of probability. One could even say that such patterns do not belong to the probability theory because they belong to some other sciences. Finally, we come to (B3) and (B4). Clearly, these patterns do not belong to any science such as biology or chemistry. If the science of probability does not report these patterns, who will? If you roll a die, the probability that the number of dots is less than three is 1/3; this is a concise summary of some observed patterns. Every theory of probability reported this finding in some way, except for the subjective theory. Needless to say, de Finetti did not omit such statements from his theory because he was not aware of them, the omission was a conscious choice. De Finetti’s choice can be easily explained. If he reported any probabilistic patterns, such as the apparent stability of relative frequencies in long runs of experiments, his account would have taken the form of a “scientific law.” Scientific laws need to be verified (or need to be falsifiable, in Popper’s version of the same idea). Stating any scientific laws of probability would have completely destroyed de Finetti’s philosophical theory. The undeniable strength of his theory is that it avoids in a very simple way the thorny question of verifiability of probabilistic statements, it denies that there are any objectively true probabilistic statements. The same feature that is a philosophical strength, is a scientific weakness. No matter how attractive the subjective theory may appear to the philosophically minded people, it has nothing to offer on the scientific side.

5.4 If Probability does not Exist, Everything is Permitted The most famous message in Dostoevsky’s novel The Brothers Karamazov is “If God does not exist, everything is permitted.” One of the most extraordinary claims ever made in science and philosophy is that consistency alone is the sufficient basis for a science, specifically, for the science of probability and Bayesian statistics. I feel that people who support this claim lack imagination. I will try to help them by presenting an example of what may happen when consistency is indeed taken as the only basis for making probability assignments — then “everything is permitted.”

page 84

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch05

The Subjective Philosophy of Probability

85

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

5.4.1 Creating something out of nothing Dyslexia is a mild disability which makes people misinterpret written words, for example, by rearranging their letters, as in “tow” and “two.” Let us consider the case of Mr. P. Di Es, an individual suffering from a probabilistic counterpart of dyslexia, a “Probabilistic Dysfunctionality Syndrome.” Mr. P. Di Es cannot recognize events which are disjoint, physically independent or invariant under symmetries, and the last two categories are especially challenging for him. Hence, Mr. P. Di Es cannot apply (L1)–(L6) to make decisions. Here are some examples of Mr. P. Di Es’ perceptions. He thinks that the event that a bird will come to the bird feeder in his yard tomorrow is not physically independent from the event that a new war will break out in Africa next year. At the same time, Mr. P. Di Es does not see any relationship between cloudy skies in the morning and rain in the afternoon. Similarly, Mr. P. Di Es has problems with sorting out which sequences are exchangeable. When he reads a newspaper, he thinks that all digits printed in a given issue form an exchangeable sequence, including those in the weather forecast and stock market analysis. Mr. P. Di Es buys bread at a local bakery and is shortchanged by a dishonest baker about 50% of the time. He is unhappy every time he discovers that he was cheated but he does not realize that the sequence of bread purchases in the same bakery can be considered exchangeable and so he goes to the bakery with the same trusting attitude every day. Some mental disabilities are almost miraculously compensated in some other extraordinary ways, for example, some autistic children have exceptional artistic talents. Mr. P. Di Es is similarly talented in a very special way — he is absolutely consistent in his opinions, in the sense of de Finetti. Needless to say, a person impaired as severely as Mr. P. Di Es would be as vulnerable as a baby. The ability of Mr. P. Di Es to assign probabilities to events in a consistent way would have no discernible positive effect on his life. The example is clearly artificial — there are very few, if any, people with this particular combination of disabilities and abilities. This is probably the reason why so many people do not realize that consistency alone is totally useless. Consistency is never applied without (L1)–(L6) in real life. It is amazing that the subjective philosophy, and implicitly the consistency idea, claim all the credit for the unquestionable achievements of the Bayesian statistics.

page 85

April 29, 2016

9:6

86

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch05

Resonance: From Probability to Epistemology and Back

5.4.2 The essence of probability

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

I will formalize the example given in the last section. First, it will be convenient to talk about “agents” rather than people. An agent may be a person or a computer program. It might be easier to imagine an imperfect or faulty computer program, rather than a human being, acting just as Mr. P. Di Es does. Consider four agents, applying different strategies in face of uncertainty. (A1) Agent A1 assigns probabilities to events without using the mathematics of probability, without using consistency and without using (L1)–(L6). He does not use any other guiding principle in his choices of probability values. (A2) Agent A2 is consistent but does not use (L1)–(L6). In other words, he acts as Mr. P. Di Es does. (A3) Agent A3 uses (L1)–(L6) in his probability assignments but does not use the mathematical rules for manipulating probability values. (A4) Agent A4 applies both (L1)–(L6) and the mathematical theory of probability (in particular, he is “consistent”). Let me make a digression. I guess that agent A3 is a good representation for a sizeable proportion of the human population. I believe that (L1)–(L6) are at least partly instinctive (see Sec. 8.2 on resonance) and so they are used by most people but the mathematical rules of probability are not easy to apply at the instinctive level and they are mostly inaccessible to people lacking education. Whether my guess is correct is inessential since I will focus on agents A1, A2 and A4. Before I compare the four agents, I want to make a comment on the interpretation of the laws of science. Every law contains an implicit assertion that the elements of reality not mentioned explicitly in the law do not matter. Consider the following example. One of the Newton’s laws of motion says that the acceleration of a body is proportional to the force acting on the body and inversely proportional to the mass of the body. An implicit message is that if the body is green and we paint it red, doing this will not change the acceleration of the body. (This interpretation is not universally accepted — some young people buy red cars and replace ordinary mufflers with noise-making mufflers in the hope that the red color and noise will improve the acceleration of the car.) It is quite clear that agents A1 and A4 lie at the two ends of the spectrum when it comes to the success in ordinary life, but even more so

page 86

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch05

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

The Subjective Philosophy of Probability

87

in science. Where should we place agent A2? I have no doubt that A2 would have no more than 1% greater success rate than A1. In other words, consistency can account for less than 1% of the overall success of probability theory. I guess that A3 would be about half-way between A1 and A4, but such a speculation is not needed for my arguments. Now I am ready to argue that the subjective theory of probability is false as a scientific theory. The theory claims that probability is subjective, there is no objective probability, and you have to be consistent. An implicit message is that if you assign equal probabilities to symmetric events, as in (L4), you will not gain anything, just like you cannot increase the acceleration of a body by painting it red. Similarly, the subjective theory claims that using (L3) cannot improve your performance. In other words, the subjective theory asserts that agent A2 will do in life as well as agent A4. I consider this assertion absurd. De Finetti failed in a spectacular way by formalizing only this part of the probabilistic methods which explains less than 1% of the success of probability — he formalized only the consistency, that is, the necessity of applying the mathematical rules of probability. I do not see any way in which the subjective science of probability can be repaired. It faces the following alternative: either it insists that (L1)–(L6) can give no extra advantage to people who are consistent, and thus makes itself ridiculous by advocating Mr. P. Di Es-style behavior; or it admits that (L1)–(L6) indeed provide an extra advantage, but then it collapses into ashes. If (L1)–(L6) provide an extra advantage, it means that there exists a link between the real universe and beneficial probability assignments, so the subjective philosophy is false. The subjective philosophy is walking on a tightrope. It must classify some decision families or probability distributions as “rational” and some as “irrational.” The best I can tell, only inconsistent families of probabilities are branded irrational. All other families are rational. Moving some consistent families of decisions, that is, some probability distributions, to the “irrational” category would destroy the beauty and simplicity of the subjective philosophy. Leaving them where they are, makes the theory decoupled from the scientific practice. When designing a theory one can either choose axioms that are strong and yield strong conclusions or choose axioms that are weak and yield few conclusions. There is a wide misconception among statisticians concerning the strength of de Finetti’s theory. The axioms of his theory are chosen

page 87

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

88

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch05

Resonance: From Probability to Epistemology and Back

to be very weak, so that they are acceptable to many people. The price that you have to pay for this intellectual choice is that the conclusions are incredibly weak. The widespread belief that de Finetti justified the use of subjective priors in Bayesian statistics is based on a simple logical mistake, illustrated by the following proof that number seven is lucky. (i) The concept of “lucky” does not apply to numbers. (ii) Hence, one cannot say that seven is unlucky. (iii) It follows that seven is lucky. The corresponding subjectivist reasoning is the following. (i) Probability does not exist, that is, prior probabilities are neither correct nor incorrect. (ii) Hence, one cannot say that subjective priors are incorrect. (iii) It follows that subjective priors are correct.

5.5 De Finetti’s Ultimate Failure This section is devoted to the most profound failure of the subjectivist approach to probability. It is clear that the main claim to fame of the subjective philosophy of probability is that it (allegedly) justifies Bayesian statistics. I have already shown that de Finetti’s consistency is practically useless without further significant postulates. I will now show that de Finetti’s theory collapses like a house of cards in the presence of data. This section was inspired by the ideas presented in [Ryder (1981)], as quoted in [Gillies (2000), p. 173]. My argument is related to the “Objection” on page 258 of [Hacking (2001)]. On page 259, Hacking writes that he did not find responses to the objection “convincing.” I am grateful to Jim Pitman (private communication) for a discussion of the original version of my argument published in [Burdzy (2009)] but I am the only person to be blamed for any flaws in the version presented below.

Consistency — the exclusive goal Let us recall some basic concepts and ideas from the foundations of the subjective philosophy of probability. De Finetti proposed to classify decision strategies as consistent or inconsistent. A consistent decision strategy cannot result in a Dutch book situation. The most fundamental philosophical idea of de Finetti is that Probability is beneficial only because it can be used to achieve a desirable deterministic goal.

page 88

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch05

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

The Subjective Philosophy of Probability

89

I call a goal “deterministic” if it can be achieved with certainty. De Finetti proved in a rigorous way that people who apply the usual mathematical rules of probability to maximize the expected gain will never find themselves in a Dutch book situation. Hence, they will achieve a desirable deterministic goal. Why do we need the Bayes theorem? It is the key to computing the posterior distribution that would “agree” with the prior distribution. Why do we need prior and posterior distributions? De Finetti claimed that there was no objective probability but we could use these distributions to coordinate our decisions. Why does the posterior distribution have to “agree” with the prior distribution? The two distributions should agree so that decisions made on the basis of the prior distribution are well coordinated with the decisions made on the basis of the posterior distribution. What does it mean for two decisions to be well coordinated? In de Finnetti’s philosophy it means that a desirable deterministic goal can be achieved with certainty. My understanding of de Finetti’s idea of consistency is that the concept is objective. I do not see how consistency could be subjective. In that case we all could individually label any and all of our favorite decision strategies as “consistent” and we all would be happily consistent ever after. If consistency is represented as an abstract concept then it cannot provide a decisive support for the subjectivist philosophy. Axiomatic systems, such as those in [DeGroot (1970)] or [Fishburn (1970)], appeal only to some people. But the real reason why an abstract axiomatic system supporting consistency cannot succeed is that subjectivists claim that it implies Bayesian statistics. Frequency statisticians have no incentive to accept subjectivist axioms without proof (as in the case of any set of axioms) since the axioms supposedly show the inadequacy of frequency methods. This means that the Dutch book argument, an empirical type of reasoning, remains the only candidate for the foundations of the subjectivist ideology. It would be hard to find anyone who would willingly sign a collection of contracts resulting in sure financial loss to him. So the prevention of a Dutch book situation, although not as attractive as “arbitrage” (see Sec. 5.18), makes an impression of an incontrovertible argument in support of the subjectivist ideology. I am going to show that the Dutch book argument crumbles in the presence of data, that is, when the Bayes theorem is invoked.

page 89

April 29, 2016

9:6

90

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch05

Resonance: From Probability to Epistemology and Back

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

5.5.1 Lazy decision maker Consider the following textbook-type example. Suppose that Susan was shown two urns, the first one with two white balls and one black ball, and the other one with two black balls and one white ball. Someone tossed a coin without showing it to Susan and noted the result on a piece of paper. Susan assumed that the result of the coin toss was heads with probability 1/2. Susan was offered and accepted the following bet (Bet I) on the result of the coin toss. She would collect $8 if the result was tails; otherwise she would lose $7. Then someone looked at the result of the coin toss and sampled a ball from the first urn if the result was heads; he sampled a ball from the other urn otherwise. Suppose Susan was told that a white ball had been sampled but she was not told which urn the ball had come from. Susan used the Bayes theorem to calculate the posterior probability of heads, it turned out to be 2/3. Next Susan was offered Bet II, identical to Bet I, that is, the second bet would pay $8 if the result of the coin toss was tails; otherwise she would lose $7. Susan declined the second bet because its expected gain was negative. Now suppose that Peter participated in the same type of experiment with urns with the same composition as those used in the case of Susan. It happened that the color of the sampled ball shown to Peter was also white. Peter accepted the first bet, just like Susan did. He also accepted the second bet. Peter knew the mathematical theory of probability well so he could have applied the Bayes theorem and declined the second bet just like Susan did. He said he accepted the second bet because doing so did not create a Dutch book situation. He did not want to waste his time to compute the posterior distribution via an application of the Bayes theorem. The wording of my example suggests that all relevant probabilities were objective but this is irrelevant — a subjectivist may use my probability values as her subjective probabilities and my guess is that most subjectivists would do just that. The example illustrates the fundamental difference between the Dutch book argument in the setting without data (see Sec. 2.4.4) and the Dutch book argument in a situation when some data are collected. In the former setting, a simple example given in Sec. 2.4.4 has shown that following the standard rules of probability would lead to a desirable deterministic goal. In that case, the subjectivist approach would eliminate a Dutch book. The situation is completely different in the present example. Most people

page 90

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch05

The Subjective Philosophy of Probability

91

would agree that Susan behaved in a rational way and Peter behaved in an irrational way. But Peter will not lose money with certainty. So if we want to argue that Peter should have chosen a different strategy, the argument cannot be based on elimination of a deterministic undesirable outcome.

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

5.5.2 Interpreting Dutch book I will argue that the idea of a Dutch book is considerably less clear and convincing than popular presentations might suggest. I will use two examples, a version of the one presented earlier in this section and a simplified version of the same example. Let us start with a simple example where no data are involved. Consider an experiment with only two possible mutually exclusive outcomes A and B. In other words, A is a complement of B. Suppose that Susan was offered Bet A which paid $8 if the event A occurred. Otherwise she would lose $7. She accepted the bet. Her decision was not inconsistent because it did not create a Dutch book. Next Susan was offered Bet B which paid $6 if the event B occurred. Otherwise she would lose $9. She accepted Bet B (in addition to Bet A). Susan put herself in a Dutch book situation. She will lose $1 no matter which of the events A or B is going to occur. Susan’s acceptance of Bet B would be considered irrational in the subjectivist world. The acceptance of Bet A was rational according to subjectivist dogmas because Susan could have gained some money. Accepting the second bet created a situation in which the loss was certain. Next consider the following version of the example given earlier in this section. Suppose that Susan was shown two urns, the first one with two white balls and one black ball, and the other one with two black balls and one white ball. Someone tossed a coin without showing it to Susan and noted the result on a piece of paper. Susan assumed that the result of the coin toss was heads with probability 1/2. Susan was offered and accepted the following bet (Bet I) on the result of the coin toss. She would collect $8 if the result was tails; otherwise she would lose $7. Then someone looked at the result of the coin toss and sampled a ball from the first urn if the result was heads; he sampled a ball from the other urn otherwise. Suppose Susan was told that a white ball had been sampled but she was not told which urn the ball had come from. Susan used the Bayes theorem to calculate the posterior probability of heads, it turned out to be 2/3. Next Susan was offered Bet II which would pay $6 if the result of the coin toss was heads;

page 91

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

92

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch05

Resonance: From Probability to Epistemology and Back

otherwise she would lose $9. Susan accepted the second bet because it had positive expected gain. Just after observing the color of the sampled ball, Susan was aware that she could win $8 or lose $7, just like in the case of Bet A. And just like in the case of Bet A, she chose to accept a bet that transformed her situation radically, the uncertainty about her gain was removed and she resigned herself to the certain loss of $1. Yet we do not consider acceptance of Bet II, in addition to Bet I, to be an irrational action. The subjectivist philosophical challenge is to show that accepting Bet II in addition to Bet I was rational but a seemingly similar combination of Bets A and B was irrational. The challenge is far from trivial because we cannot refer to probability or expectation. In de Finetti’s world probability does not exist. For example, we cannot say that Susan lowered her expected loss by accepting Bet II. She did lower her subjective expected loss but we are now concerned with consistency of her decisions, an objective notion. It seems to me that the only way to justify the acceptance of Bet II (without using the concepts of probability and expectation) is to refer to the fact that the color of the sampled ball might have been different and the world could have developed in a different way. Such a justification might exist but I do not have any incentive to search for it. Whatever justification along these lines exists, it is a far cry from the “self-evident” Dutch book argument which formalizes the idea that nobody would like to lose money with certainty. I note parenthetically that a frequentist would have no problem justifying the irrationality of Susan’s decision to accept bets A and B, and the rationality of acceptance of Bets I and II. Of course, the frequentist justification requires that the decision problems would or at least could be repeated a large number of times, and this assumption may be questioned.

5.5.3 Dutch book with a lapse of time It is quite often, perhaps always, that when we make two decisions then the decisions are separated by some time, perhaps as small as a few seconds. Within this short time interval millions of events take place in our universe. They may include a car accident in Paris and an avalanche on a planet in a distant galaxy. We have two opposite choices. (i) We can assume that an event is relevant to the question of whether two decisions form a Dutch book only if the outcome of the event may

page 92

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch05

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

The Subjective Philosophy of Probability

93

change the value of the payoffs related to the two decisions or change the family of events on which the payoffs are based. (ii) We can assume that a subjectivist may consider an event to be relevant to the question of whether two decisions form a Dutch book even if the outcome of the event does not change the value of the payoffs related to the two decisions and does not change the family of events on which the payoffs are based. The point of departure for de Finetti’s approach is a world without probability. This is why a supporter of de Finetti’s philosophy must make a choice between (i) and (ii) without referring to probability. He has to describe a rule specifying when an event can be considered to be relevant to a potential Dutch book using only decision theoretic concepts such as events and payoffs. First assume (i) and recall Susan’s decision problem concerning Bets I and II from Sec. 5.5.2. Susan accepted Bet II. At this point, she knew that she would lose $1 no matter what the result of the coin toss was. The sampling of the ball and the observation of its color did not change Susan’s decision problem (Bets I and II). The possible outcomes and payoffs related to Bets I and II remained unchanged whether a ball was sampled or not. Hence, assuming (i), accepting Bets I and II created a Dutch book whether Susan had an opportunity to observe the color of the sampled ball or not. I am sure that most people will find this conclusion unpalatable so I will examine the consequences of (ii) next.

5.5.3.1 The butterfly effect According to Wikipedia, “In chaos theory, the butterfly effect is the sensitive dependence on initial conditions in which a small change in one state of a deterministic nonlinear system can result in large differences in a later state.” I will use a different “butterfly effect” in the following argument. In this section, I will adopt assumption (ii) from the last section: A subjectivist may consider an event to be relevant to the question of whether two decisions form a Dutch book even if the outcome of the event does not change the value of the payoffs related to the two decisions and does not change the family of events on which the payoffs are based. Consider the following version of the basic example — this time with no urns or balls. Suppose that someone tossed a coin without showing it to Susan and noted the result on a piece of paper. Susan was offered Bet I

page 93

May 10, 2016

13:19

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

94

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch05

Resonance: From Probability to Epistemology and Back

on the result of the coin toss. She would collect $8 if the result was tails; otherwise she would lose $7. Susan was also offered Bet II that would pay $6 if the result of the coin toss was heads; otherwise she would lose $9. Susan signed the contract with Bet I and was looking outside the window for a few seconds before the second contract, with Bet II, was placed in front of her. It is possible that a butterfly appeared in Susan’s view. Let’s call this event B. Let H denote the event “heads.” It was consistent for Susan to believe that P (H) = 1/2, P (B | H) = 2/3 and P (B | H c ) = 1/3 (I am constructing an example that is mathematically equivalent to the original example with urns and balls). These probabilities are absurd from the objectivist point of view but we are now working in the subjectivist world where consistency is the only constraint. If the butterfly indeed appeared then the posterior probability of heads was 2/3 and it was rational for Susan to accept Bet II in addition to Bet I. But what if no butterfly appeared? Consider Peter, another subjectivist, whose subjective probabilities were P (H) = 1/2, P (B c | H) = 2/3 and P (B c | H c ) = 1/3. If Peter accepted Bet I and observed no butterflies, it was rational for him to sign Bet II, in addition to Bet I. Since it was rational for one subjectivist to sign Bets I and II in the presence of butterflies and it was rational for another subjectivist to sign Bets I and II in the absence of butterflies, it follows that simultaneous acceptance of Bets I and II is rational, period. But there is a catch. If you are a rational person and you want to accept Bets I and II, you must delay acceptance of Bet II by a few seconds and look outside the window to check if there are any butterflies. You also have to have the right thoughts about butterflies and coins in your mind.

5.5.4 Rule of conditionalization The scientific law (L5) is called the “rule of conditionalization” in [Hacking (2001), p. 259]. The rule is introduced as a way to deal with the “Objection” on page 258 of [Hacking (2001)]. Hacking proposes to treat the rule of conditionalization as a moral rule. I find this proposal incomprehensible. Most people adhere to some moral rules but these rules are not arbitrary. More accurately, those rules that are universally recognized, such as the prohibition of homicide, are typically justified by practical considerations. Making a moral rule out of the ban on homicide ensures that all people feel safer than they would otherwise. Why should anyone adopt the rule of conditionalization as a moral guideline? If

page 94

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch05

The Subjective Philosophy of Probability

95

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

it brings no measurable or at least discernible benefit to anyone then it is a philosophical “pure abstract nonsense” with no moral appeal. If the rule of conditionalization does bring some benefit on some occasions then the benefit should be explicated. Once this is done, people will adopt the rule because they will know that it is beneficial, not because it is “moral.” In my system (L1)–(L6), the rule of conditionalization (that is, (L5)) is easily justified. Its application leads to reliable predictions. One can easily perform simple experiments proving this point.

5.6 All Sequential Decisions are Consistent Consider the following simple sequential decision problem involving statistical data. Suppose that a doctor prescribes a dose of a drug to patients with high blood pressure in his care. To make things simple, let us assume that each patient receives only one dose of the drug and patients come to the doctor sequentially. The doctor records three pieces of data for each patient — the dose of the drug and the blood pressures before and after the drug is taken. The drug is a recent arrival on the market and the doctor feels that he has to learn from his observations what doses are best for his patients. The standard Bayesian analysis of this problem is the following. The doctor should start with a prior distribution describing his opinion about the effect of the drug on patients with various levels of blood pressure. As he collects the data, he should use the Bayes theorem to update his views, hence generating a new posterior distribution after the treatment of each patient. This new posterior distribution should be used to determine the best drug dose for the next patient. Let us examine the subjectivist justification for the above procedure. According to the subjectivists, objective probability does not exist but one can use the probability calculus to distinguish between consistent and inconsistent strategies. A decision strategy is consistent if and only if it is represented by a probabilistic view of the world. The Bayes theorem is a part of the probability calculus so one should apply the Bayes theorem when some new data are collected. It turns out that in the above example every decision strategy applied by the doctor can be represented using a probabilistic prior. The proof of this claim is routine, similar to some well known constructions, such as Tulcea’s theorem on Markov processes ([Ethier and Kurtz (1986), Appendix 9]). Hence, all strategies available to the doctor are consistent in the

page 95

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

96

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch05

Resonance: From Probability to Epistemology and Back

subjectivist sense. It follows that the doctor does not have to do any calculations and can prescribe arbitrary doses of the drug to his patients — whatever he does, he is consistent. Tulcea’s theorem alluded to above is a somewhat technical result so it is unfair for me to ask the reader to consult [Ethier and Kurtz (1986)] and verify that the method applies in our situation. I cited Tulcea’s theorem because it can be used to generate “interesting” probability models (Markov processes). I will describe a much simpler mathematical model (an “uninteresting” one) to show that all sequential strategies are consistent. Suppose that, before the first application of the new drug, the doctor builds his prior distribution describing his beliefs about the future patients and the effectiveness of the drug. The doctor may assume that health conditions of all patients are independent from each other and the effect of the drug on any patient is independent from that for any other patient. Constructing a family of independent random variables is a totally elementary procedure in mathematics. The resulting prior distribution says that the doctor cannot learn anything from the past observations of patients and drug effects about future patients and drug effects. Hence, every sequence of drug doses is consistent. Needless to say, the prior that I have just described makes no sense from the objectivist point of view. All I am saying is that de Finetti’s consistency is a totally useless concept in the case of sequential decisions. Of course, no Bayesian statistician would suggest that the doctor should abandon the Bayes theorem. This shows that whatever it is that Bayesians are trying to achieve in cases like this, it has absolutely nothing to do with de Finetti’s commandment to avoid inconsistency. In my example, at every point of time, there are no decisions to be coordinated in a consistent way (because there is only one decision) and the doctor has the total freedom of choice. If the doctor had to make several decisions between any two batches of data, he would have to coordinate them in a consistent way but the same mathematical argument would show that the doctor would not have to coordinate decisions made at different times.

5.7 Honest Mistakes Another practical illustration of problems with the subjectivist philosophy involves our attitude to past mistakes. Suppose that some investors can buy stocks or bonds only once a day. Assume that they receive new economic and financial information late in the day, too late to trade on the same day. Consider three investors whose

page 96

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch05

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

The Subjective Philosophy of Probability

97

priors may be different but the following is true for each one of them. The prior and the information that arrived on Sunday night are such that for any information that may become available on Monday night, the following strategies are consistent: (i) buy stocks on Monday and buy stocks on Tuesday, or (ii) buy bonds on Monday and buy bonds on Tuesday. It is inconsistent to (iii) buy stocks on Monday and buy bonds on Tuesday, or (iv) buy bonds on Monday and buy stocks on Tuesday. Suppose that Investor I bought some bonds on Monday and some bonds on Tuesday. He was consistent and rational. Investor II wanted to buy stocks on Monday but his computer malfunctioned and he ended up buying bonds on Monday. On Tuesday, the same investor realized that a mistake had been made by the computer program on the previous day. He considered this an accidental loss and bought stocks on Tuesday, in agreement with his original investment strategy. Investor II could have followed the example of Investor I and he could have bought bonds on Tuesday, that would have made his investments consistent. Investor III bought some bonds on Monday and some stocks on Tuesday, although he knew very well that this was an inconsistent strategy. He confided to a friend that he bought stocks on Tuesday because he was bored of buying bonds every day. The actions taken by Investors II and III were identical but most people would brand Investor II rational and Investor III irrational. It is hard to see how the subjective theory could justify the behavior of Investor II. The subjective philosophy says that being consistent is objectively beneficial and being inconsistent is objectively suboptimal. What practical benefits did Investor II reap that would not apply to Investor III, who was blatantly irrational and inconsistent but took the same actions as Investor II? An objectivist would have no problem analyzing the “mistake” made on Monday. The mistake was taking an action incompatible with the objective probabilities. In the objectivist view, a mistake can be discovered on Monday, on Tuesday, at some later time, or never. After the discovery of the objective mistake, all new actions have to take into account the true objective probabilities and the results of all past actions, including the errors. Mistakes are made not only by computers but also by humans, needless to say. Suppose that Investor II mistakenly bought some bonds on Monday for one of the following reasons: (a) He read a newsletter and missed the crucial word “not” in a sentence. (b) He did some calculations in his mind but made a mistake in them.

page 97

April 29, 2016

9:6

98

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch05

Resonance: From Probability to Epistemology and Back

(c) He was angry at a person handling stock transactions and so he bought some bonds instead of stocks.

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Not everybody would consider (c) a “good excuse” but I think that most people would agree that if (a), (b) or (c) happened, then it would be a rational course of action for Investor II to buy stocks on Tuesday. Now consider the following possible causes of the mistaken purchase of bonds on Monday: (d) Headache. (e) Insufficient attention to detail. (f) Poor judgment. The reasons for the mistake become more and more vague as we move down the list. The last item, “poor judgment,” is so general that it applies to practically every situation in which a decision maker is unhappy with one of his actions taken in the past. If we accept (f) as a legitimate excuse and we commend Investor II for buying stocks on Tuesday because he came to the conclusion that buying bonds on Monday was a “poor judgment,” then we effectively nullify any subjectivist justification for the consistency of decision making before and after collecting data. It seems that we have to draw the line somewhere, but where should the line be?

5.8 Cohabitation with an Evil Demiurge A good way to visualize the futility of the subjectivist philosophy of decision making in the presence of data is to use a concept that we are all more or less familiar with a supernatural being. Imagine a world governed by an evil demiurge bent on confusing people and making their lives miserable. The demiurge changes the laws of nature in arbitrary and unpredictable ways. Sometimes he announces the timing and nature of changes. Sometimes he cheats by making false announcements, and sometimes he makes changes without announcing them. Some of the changes are permanent, for example, he changed the speed of light at some point. Some changes are temporary, for example, coffee was a strong poison for just one day. Some changes may affect only one person, for example, John Smith was able to see all electromagnetic waves with his own eyes for one year, but nobody else was affected similarly.

page 98

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch05

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

The Subjective Philosophy of Probability

99

In a universe governed by the evil demiurge, no knowledge of the past events can be used to make a deterministic prediction of any future event. Likewise, statistical predictions do not have the same value as in our universe. For example, the demiurge once changed the properties of a new drug being developed by a pharmaceutical company. The drug had been 80% effective in medical trials but it became 1% effective when the company started selling it. The idea of a world with an evil demiurge is very similar to an an idea of a chaotic and unpredictable world invented by David Hume as a part of an argument exposing a logical weakness in the principle of induction. I will now analyze the value of the subjectivist strategy in an uncooperative world. Given any past observations, the future may take any shape whatsoever. People living in that strange world may have a great variety of opinions about the future and it is hard to argue that one opinion is more rational than another because the demiurge is completely unpredictable from the human point of view. Consistency (in de Finetti’s sense) does place restrictions on families of probabilistic opinions and decision strategies. But consistency does not place any restriction on the relationship between future events and past observations in that world — anything can happen in the future, no matter what happened in the past. Hence, Bayesian statisticians may choose arbitrary posterior distributions without need to use the Bayes theorem. Some Bayesian statisticians may choose to use the Bayes theorem to coordinate their prior and posterior distributions according to the standards of Bayesian statistics. However, their efforts cannot be empirically verified in a universe governed by an evil and unpredictable demiurge. Even if some past observations support the Bayesian approach, the demiurge may manipulate the nature in such a way that all future Bayesian decisions will lead to huge losses. It is obvious that Bayesian statisticians believe that the past performance of Bayesian methods in our universe is excellent. Yet the subjective philosophy does not make any argument that is specific to our universe and would not apply to the universe governed by the evil demiurge. Bayesian statistics would be useless in that strange universe but it is not useless in ours. De Finetti failed to notice that consistency is blind to the stability of the laws of nature. This stability is the foundation of statistics just as it is the foundation of all science. Statistics that does not acknowledge the stability of the laws of nature is an empty shell of a theory.

page 99

April 29, 2016

9:6

100

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch05

Resonance: From Probability to Epistemology and Back

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

5.9 Why Bother to Use Probability? De Finetti claimed that the mathematical rules for calculating probabilities can be used to make sure that our decisions are consistent and that no other rational rules for probability assignments exist or are needed. De Finetti proved a theorem showing that one can achieve decision theoretic consistency if and only if a probabilistic representation of the problem is consistent (that is, it agrees with the usual mathematical axioms). Whenever scientists invent two methods to achieve the same practical goal, they test both methods to see which one is more efficient, less expensive, faster, etc. In other words, they discriminate between the methods on the basis of secondary characteristics. One would expect that scientists checked which of the two methods of attaining consistency was more efficient: (i) checking consistency of decisions in a direct way, using only decision-theoretic axioms and deductions from them, or (ii) via probabilistic calculations. As far as I know, nobody ever bothered to make the comparison, either in theory or in practice. Moreover, I have never seen a suggestion by anyone that such a comparison should be made. This includes de Finetti. This proves beyond reasonable doubt that nobody, including the inventor of the theory himself, ever believed that probability is used exclusively to achieve decision theoretic consistency. I believe that in most practical cases, if there is a need to eliminate inconsistent decisions, this can be done in a very efficient way using only decision theoretic concepts.

5.10 The Dutch Book Argument is Rejected by Bayesians In this section I will develop some ideas from Sec. 5.5. Let us consider a case of Bayesian decision analysis based on some data. To make the argument simple, I will assume that all the contracts that the decision maker can sign have payoffs that may depend only on events that will be observed in the future, that is, after the data are collected and the posterior distribution is computed. A deterministic goal that one can achieve in a statistical situation, that is, a situation when data are collected, is to avoid signing contracts such that at a certain point of time, before the payoff of any of the contracts, it is already known that the combined effect of all contracts is a sure loss for the decision maker. Whether we call this a “Dutch book” argument or

page 100

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch05

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

The Subjective Philosophy of Probability

101

not is not important. The goal stated above can be achieved with certainty by setting the posterior distribution equal to the prior one. Clearly, if all contracts based on the prior distribution do not form a Dutch book, then neither those based on the posterior distribution will form a Dutch book. The two sets of contracts form one large consistent family of contracts, because all of them are based on the same probability distribution. Bayesian statisticians never take the posterior distribution to be the prior distribution, except perhaps in some trivial cases. Hence, the deterministic goal that I presented above is rejected by the Bayesians although no other deterministic goal seems to replace it. This proves that Bayesian statisticians prefer to achieve some other goal rather than the deterministic goal of avoiding the Dutch book. The above reasoning undermines the whole idea of the Dutch book argument, both in the statistical setting, and in situations where no data are collected. I have demonstrated that there is a practical situation (actually, a commonplace occurrence) when a deterministic goal is rejected in favor of some other unspecified potential gain, presumably of probabilistic (random) nature. This in turn implies that the original Dutch book argument is far from obvious. De Finetti presented the Dutch book argument as a selfevident choice of all rational people, supposedly because every rational person would like to achieve a deterministic and beneficial goal. To complete his argument, de Finetti would have to show that no random goal can be considered more valuable than a deterministic goal. The behavior of Bayesian statisticians shows that rational people do not consider every deterministic goal to be more desirable than every “randomly achievable” goal.

5.11 Insurance Against Everything De Finetti’s idea that we should use probability to protect ourselves against a Dutch book is effectively an insurance against everything — the least practical insurance ever invented. Engineers designing an airplane make computations for one particular design or several competing designs but not all possible airplane designs. They do only as much of mathematical calculations as is needed for the purpose at hand. This is to save time, manpower, money, etc. The idea that Bayesians should make extensive calculations to protect themselves against all possible Dutch books is totally impractical. If Bayesians really wanted to protect themselves against a Dutch book, they would find it

page 101

April 29, 2016

9:6

102

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch05

Resonance: From Probability to Epistemology and Back

much easier to review the relevant contracts or decisions, typically a small number of these and see whether they form a Dutch book or not.

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Tabula rasa I will now discuss a situation when data are available and may change the prior distribution to the posterior distribution. This will provide an opportunity to amplify the argument given earlier in this section. Consider a paranoid decision maker who wants to protect himself against all possible Dutch book situations. De Finetti has a piece of good advice for him — use probability to coordinate your decisions. Now suppose that the decision maker obtained some new data. According to the standard Bayesian procedure he will replace his prior distribution with a posterior distribution. At this point he knows that he will never use his prior distribution again. From now on he will use exclusively his new posterior distribution and its successors. Hence, he can review all decisions that he made using the prior distribution. Suppose that there were none. No matter how paranoid the decision maker is, he is now sure that he does not need to coordinate any prior decisions with the posterior decisions. It follows that he can take any probability distribution as his posterior distribution. The data erased the slate clean. The situation outlined above is common in scientific research. Typically no decisions are based on the prior distribution. This distribution usually plays a strictly technical role in statistical analysis and is not the basis for any decisions.

5.12 No Need to Collect Data In many practical situations, scientists are not limited to studying existing data sets but they can choose whether to collect some data or not. As long as they have interest in the subject and sufficient resources, such as money, manpower and laboratories, they inevitably choose to collect data. De Finetti’s theory fails to explain why they do so, and does so at many different levels, so to speak. The easiest way to see that there is no need to collect data is to notice that, according to de Finetti, the only purpose of the probability theory is to eliminate inconsistent decision strategies. In every situation, no matter what information you have, there is at least one consistent strategy. Hence, if you do not collect any data, you can still act in a consistent way. Quite

page 102

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch05

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

The Subjective Philosophy of Probability

103

often, not collecting data would result in substantial financial savings, so the choice is obvious — stop collecting data. I will look at the need to collect data in two other ways. Suppose a person is given a chance to play the following game with a deformed coin, previously unknown to him. The coin will be tossed once. The player will receive $100 if the result is heads and he will lose $1 otherwise. His only choice is to play the game or not. The coin will be destroyed after the game is played, or immediately after the player decides not to play the game. Now suppose that the person can examine the deformed coin by tossing it for 10 minutes before making a decision whether to play or not. Every rational person would toss the coin for a few minutes to collect some data, before deciding whether to play the game. A simple intuitive explanation for collecting the data is that the information thus collected may show that the game is highly advantageous to the player, so collecting the data may open an opportunity for the player to enrich himself with minimal risk. On the other hand, the process of collecting the data itself cannot result in any substantial loss except some time. According to de Finetti’s theory, there is no point in collecting the data because there are no decisions to be coordinated — the person has only one decision to make. Suppose that we have a situation when multiple decisions need coordination and there is plenty of opportunity for inconsistent behavior. Let us limit our considerations to a case of statistical analysis when potential gains and losses do not depend on the data, only on some future random events. The decision maker may generate an artificial data set, say, by writing arbitrary numbers or using a random number generator. Then he can find a consistent set of decisions based on this artificial data set. The resulting decision strategy will be consistent because the payoffs do not depend on the data, so they do not depend on whether the data are genuine or not. The cost of collecting real data can be cut down to almost nothing, by using artificial data. A different way to present this idea is this. Suppose that you learn that a scientist falsified a data set. Will you be able to find inconsistencies in his or anyone else’s decisions? Scientists insist on using only genuine data sets, but this is not because anyone ever found himself in a Dutch book situation as a result of someone publishing a fake data set.

5.13 Empty Promises De Finetti’s postulates contain a pseudo-scientific implicit claim that following a consistent strategy will generate the maximal expected gain.

page 103

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

104

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch05

Resonance: From Probability to Epistemology and Back

This is highly misleading. From the purely mathematical point of view, a consistent decision strategy is equivalent to a strategy that maximizes the expected gain. However, there are infinitely many totally incompatible consistent decision strategies and each one of them maximizes the expected gain. Clearly, something is wrong with this logic. The problem is that a consistent decision strategy maximizes the expected gain assuming that the expectation is calculated using some probability distribution. The mathematical theory does not and cannot say whether the probability assignments representing a consistent decision strategy have anything to do with reality. The claim about the maximized expected gain is a purely abstract statement that often applies equally to two contradictory (but individually consistent) strategies. To see this, consider the following example. Consider a game with two players that involves repeated tosses of a deformed coin. The game requires that one side of the coin should be marked before the tosses start. The players will play the game only if they agree beforehand on the side to be marked. Once the coin is marked, a single round of the game consists of a toss of the coin. The first player pays $1.00 to the second player if the coin lands with the marked side up, and otherwise the second player pays $1.00 to the first one. A consistent set of beliefs for the first player is to assume that the tosses are independent with the probability of heads being 90%. A consistent set of beliefs for the second player is to assume that the tosses are independent with the probability of heads equal to 10%. If the two players adopt these views then they will agree that the mark should be made on the side of the tails. If the coin is highly biased and the players repeat the game many times, one of the players will do much better than the other one. Yet according to the subjectivist postulates both players will be always consistent and each one of them will always maximize his expected gain. The following remark is for readers familiar with the concept of exchangeability. Note that the example is based on the assumption that both players believe that coin tosses are i.i.d., not merely exchangeable. It is consistent to believe that tosses of a deformed coin are i.i.d., even if you see the coin for the first time.

5.14 The Meaning of Consistency The following remarks belong to Sec. 7.7 on abuse of language but I consider them sufficiently important to be repeated twice, in a somewhat different way.

page 104

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch05

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

The Subjective Philosophy of Probability

105

The word “consistent” has a different meaning in everyday life than in the subjective philosophy or science. A common everyday practice is to use logic in a non-scientific way. “If you eat your broccoli then you can have ice-cream,” a mother may say to her child. This really means “If you do not eat your broccoli then you will not have ice-cream, and if you eat your broccoli then you will have ice-cream.” I do not consider the equivalence of the two sentences in everyday speech to be “illogical.” Every convention is acceptable as long as all parties agree on the rules. Similar conventions are known as “implicatures” in linguistics, see [Wikipedia (2015m)]. Just like everyday logic is not identical with the formal logic, the everyday meaning of “consistent” is not the same as the meaning of “consistent” in the subjective theory. Consider the following conversation. Mr A.: “My child goes to Viewridge Elementary School. The school administration is unfriendly and the teachers are incompetent.” Mr B.: “What you have just said is consistent with what I have heard from other parents.” Mr C.: “I also have a child in the same school and I disagree. The school administration is friendly and the teachers are competent.” Mr B.: “This is also consistent with what I have heard from other parents.” This imaginary conversation strikes us as illogical. Two contradictory statements are never described as simultaneously consistent with the same body of evidence. Consider the following claims. “Smoking increases the chance of cancer” and “Smoking decreases the chance of cancer.” Both statements are consistent with the data, in the sense that for any of these statements, a statistician may choose a prior such that his posterior distribution can be summarized by the designated statement. My guess is that most Bayesian statisticians believe that smoking increases the chance of cancer. Many people believe that this is due to consistency of the claim with the data. In fact, the subjectivist notion of consistency neither supports nor falsifies any of the two contradictory statements about smoking and cancer. “Consistency” is a stronger notion in everyday speech than in the subjective theory. In the first case, it encourages people to choose between contradictory claims made by others. In the second case, it does not.

5.15 Interpreting Miracles A popular definition of a “miracle” is that it is a very unlikely event that actually occurred (in theology, a miracle is a “sign from God,” a

page 105

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

106

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch05

Resonance: From Probability to Epistemology and Back

substantially different concept). In my theory based on (L1)–(L6), a miracle is the opposite of a successful prediction. Both von Mises and de Finetti misunderstood the role of miracles in probability but in different ways. Von Mises associated predictions and therefore, miracles, only with collectives, that is, long sequences of identical observations. In the subjective theory, the concept of consistency puts a straitjacket onto a subjectivist decision maker, as noticed, for example, in [Weatherford (1982)]. Miracles are expected to affect the mind of a rational person but consistency removes any flexibility from decision making. Consider, for example, two friends who have strong trust in each other. Suppose that for some reason, one of them betrays the other (this is a “miracle”), and keeps betraying him on numerous occasions. Common sense dictates that the repeated breach of one’s trust should be reciprocated with the more cautious attitude towards the offender. Many people would argue that continued loyalty of the betrayed party could be only rationally explained by “irrational emotions.” In the subjectivist scheme of things, there is no such thing as irrational loyalty. For some prior distributions, no amount of disloyalty can change the mind of a person. De Finetti’s theory does not provide any philosophical arguments that would support eliminating such extreme priors. I do not argue that one should abandon all ethical and human considerations and I do not advocate swift adjustment of one’s attitude according to circumstances, that is, a form of opportunism. My point is that the subjectivist philosophy does not provide an explanation or theoretical support for common patterns of behavior — no consistent attitude, even the most insensitive to environmental clues, is irrational in de Finetti’s theory. The problem of extreme inflexibility of the subjectivist theory is well known to philosophers. In the statistical context, a subjectivist statistician who considers a future sequence of events exchangeable will never change his mind about exchangeability of the sequence, no matter how nonexchangeable data seem to be. Some philosophers proposed an ad hoc solution to the problem — one should not use a prior that is totally concentrated on one family of distributions. This is a perfectly rational and practical advice except that it runs against the spirit and letter of subjectivism. If some priors are (objectively) better than some other priors then there must be an objective link between reality and probability assignments, contrary to the philosophical claims of the subjective theory.

page 106

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch05

The Subjective Philosophy of Probability

107

5.16 Science, Probability and Subjectivism

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Recall from Sec. 2.4.3 what de Finetti had to say about the fact that beliefs in some probability statements are common to all scientists (quote after [Gillies (2000), p. 70]): Our point of view remains in all cases the same: to show that there are rather profound psychological reasons which make the exact or approximate agreement that is observed between the opinions of different individuals very natural, but there are no reasons, rational, positive, or metaphysical, that can give this fact any meaning beyond that of a simple agreement of subjective opinions.

This psychological explanation for the agreement of opinions is vacuous and dishonest. It is vacuous because it can explain everything, so it explains nothing. Note that the statement does not even mention probability. It applies equally well to gravitation. The statement is dishonest, because it suggests that individuals are free to hold views that disagree with the general sentiment. You may privately hold the view that smoking cigarettes decreases the probability of cancer but you are not allowed to act accordingly — the sale of tobacco products to minors is prohibited by law. The fact that all physicists agree on the probability that the spin of an electron will be positive under some experimental conditions is neither subjective nor objective — this agreement is the essence of science. No branch of “deterministic” science has anything to offer besides the “simple agreement of subjective opinions” of scientists. Nobody knows the objective truth, unless he or she has a direct line to God — even Newton’s physics proved to be wrong, or at least inaccurate. The agreement of probabilistic opinions held by various scientists is as valuable in practice as their agreement on deterministic facts and patterns. Consensus on an issue cannot be identified with the objective truth. But consensus usually indicates that people believe that a claim is an objective truth. De Finetti correctly noticed (just like everybody else) that the evidence in support of probabilistic laws, such as (L1)–(L6), is less convincing than that in support of deterministic laws (but I would argue that this is true only in the purely philosophical sense). Hence, users of probability have the right to treat the laws of probability with greater caution than the laws of the deterministic science. However, I see no evidence that they exercise this right; laws (L1)–(L6) are slavishly followed even by the most avowed supporters of the subjectivist viewpoint.

page 107

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

108

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch05

Resonance: From Probability to Epistemology and Back

De Finetti did not distinguish between the account of the accumulated knowledge and the application of the same knowledge. Science has to summarize the available information the best it can, so the science of probability must consist of some laws such as (L1)–(L6). The same science of probability must honestly explain how the laws were arrived at. A user of probability may choose to consider all probabilistic statements to be subjective, as proposed by de Finetti, but there is nothing peculiar about the probability theory here — quantum physics and even Newton’s laws of motion can be considered subjective as well, because one cannot provide an unassailable proof that any given law is objectively true.

5.17 Apples and Oranges Can you mix objective and subjective probabilities in one theory? The reader might have noticed that many of my arguments were based on the same categorical assumption that was made by de Finetti, that no objective probability can exist whatsoever. It may seem unfair to find a soft spot in a theory and to exploit it to the limit. One could expect that if this one hole is patched in some way, the rest of the theory might be quite reasonable. Although I disagree with de Finetti on almost everything, I totally agree with him on one point — it is not possible to mix objective and subjective probabilities in a single philosophical theory. This was not a fanatical position of de Finetti, but his profound understanding of the philosophical problems that would be faced by anyone trying to create a hybrid theory. I will explain what these problems are in just a moment. First, let me say that the idea that some probabilities are subjective and some are objective goes back at least to Ramsey, the other co-inventor of the subjective theory, in 1920’s. Carnap, the most prominent representative of the logical theory of probability, talked about two kinds of probability in his theory. And even now, Gillies and Hacking seem to advocate a dual approach to probability in [Gillies (2000)] and [Hacking (2001)]. Other people made similar suggestions but all of this is pure handwaving. If you assume that both objective and subjective probabilities exist, your theory will be a Frankenstein monster uniting all the philosophical problems of both theories and creating some new problems of its own. You will have to answer the following questions, among others. (i) If some probabilities are objective, how do you verify objective probability statements? Since subjective probability statements cannot be

page 108

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch05

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

The Subjective Philosophy of Probability

109

objectively verified, do they have the same value as objective statements or are they inferior? If the two kinds of probability are equally valuable, why bother to verify objective probability statements (one could use exclusively subjective probabilities)? If the two kinds of probability are not equally valuable, how do you define and measure the degree of inferiority of subjective statements? (ii) If you multiply an objective probability by a subjective probability, is the result objective or subjective? (iii) Are all probabilities totally objective or totally subjective, or can a probability be, say, 70% subjective? If so, question (ii) has to be modified: If you multiply a 30% objective probability by a 60% objective probability, to what degree is the result objective? How do you measure the degree to which a probability is objective? There is no point in continuing the list. I am not aware of any theory that would be able to give even remotely convincing answers to (i)–(iii). The ideas that “probability is long run frequency” and “you should be consistent” are perfectly legitimate within the scientific context, because they are not exclusive — they are some of many good ideas used in practice, in some circumstances. Doctors do not expect any drug to be a panaceum and similarly statisticians and probabilists cannot expect their field to be based on just one good idea. The peaceful coexistence of these ideas in the scientific context cannot be emulated in the philosophical context. This is because each of the frequency and subjective philosophies has to claim that its main idea is all there is to say about probability. Otherwise, the two ideologies become marginal philosophical theories, formalizing and justifying only those aspects of probability that were never controversial.

5.18 Arbitrage Paradoxically, an excellent illustration of the failure of de Finetti’s philosophy is provided by the only example of its successful scientific application. The Black–Scholes option pricing theory is a mathematical method used in finance. It incorporates two ideas: (i) one can achieve a desirable practical goal (to duplicate the payoff of an option on the maturity date) in a deterministic way in a situation involving randomness, and (ii) the “real probabilities”, whether they exist or not, whether they are objective or not, do not matter. These two ideas are stunningly close, perhaps identical, to the two main philosophical ideas of de Finetti.

page 109

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

110

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch05

Resonance: From Probability to Epistemology and Back

When the Black–Scholes theory is taught, students have to be indoctrinated to internalize ideas (i) and (ii). These ideas are totally alien to anyone who has previously taken any class on probability or statistics. Ideas (i) and (ii) are not taught in any other scientific context. The Black–Scholes theory is based on the concept of “arbitrage,” that is, a situation on the market when an investor can make a profit without risk. This is, in a sense, the opposite of a Dutch book situation in which a decision maker sustains a loss with certainty. The standard theoretical assumption is that real financial markets do not admit arbitrage. I am amazed that de Finetti is not given any credit for providing the basic philosophy for the Black–Scholes theory. My guess is that this is because de Finetti’s supporters think that de Finetti designed the foundations for all of probability and statistics — so why bother to mention a particular application of probability to option trading? Sadly, this is the only practical application of de Finetti’s philosophy.

5.19 Subjective Theory and Atheism The concept of God presents a number of difficult philosophical puzzles, just like the concept of probability. Theological paradoxes depend on the specific religion. Here are some examples related to Catholicism. If God is omnipotent, can He make a stone so heavy that He cannot lift the stone himself? How is it possible that there is only one God but there is also the Holy Trinity? If God is omnipotent and omnibenevolent, why is there so much evil in the world? (This problem is know as “theodicy.”) Theologians have some answers to these questions but their arguments are far from convincing. There is one philosophical attitude towards God that provides easy answers to all of these and similar questions — atheism. An atheist philosopher can answer all these questions in only four words: “God does not exist.” Atheism may be considered attractive from the philosophical point of view by some people but it has a very inconvenient aspect — it is totally inflexible. An atheist philosopher must deny the existence of God in any form and in any sense. The reason is that if a philosopher admits that God might exist in some sense then the same philosopher must answer all inconvenient questions concerning God. Constructing a philosophical theory of “partly existing God” is not any easier than constructing a theory of 100% existing God.

page 110

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch05

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

The Subjective Philosophy of Probability

111

De Finetti was an atheist of probability. His fundamental philosophical idea was that “probability does not exist.” This claim instantly solved all “paradoxes” involving probabilities. I respect de Finetti for bravely admitting the total lack of flexibility of his philosophy. His statement quoted in Sec. 2.4.3 may appear to be silly to people who have no patience for philosophical subtleties. In fact, the statement is the proof that de Finetti understood the essence of his own theory — something that cannot be said about scores of his followers. A common view among “non-extremist” subjectivists and Bayesian statisticians is that some probabilities are at least partly objective. This view cannot be adopted by a subjectivist philosopher. Building a philosophical theory of probability in which some probabilities are 1% objective is not any easier than building a philosophical theory of probability in which all probabilities are 100% objective. In either case, all questions concerning probability are equally hard to answer.

5.20 Imagination and Probability The idea that probability is mainly used to coordinate decisions so that they are not inconsistent is merely unrealistic, if we assume that probability is objective. If we assume that probability is subjective, the same idea is self-contradictory. A common criticism of the frequency theory coming from the subjectivist camp is that the frequency theory applies only to long sequences of i.i.d. events; in other words, it does not apply to individual events. Ironically, subjectivists fail to notice that a very similar criticism applies to their own theory, because the subjective theory is meaningful only if one has to make at least two distinct decisions — the Dutch book argument is vacuous otherwise. Both theories have problems explaining the common practice of assigning a probability to a unique event in the context of a single decision. The subjective theory is meaningless even in the context of a complex situation involving many events, as long as only one decision is to be made. Typically, for any given event, its probability can be any number in the interval from 0 to 1, for some consistent set of opinions about all future events, that is, for some probability distribution. If a single decision is to be made and it depends on the assessment of the probability of such an event, the subjective theory has no advice to offer. There are spheres of human activity, such as business, investment and warfare, where multiple decisions have to be coordinated for the optimal result. However, there

page 111

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

112

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch05

Resonance: From Probability to Epistemology and Back

are plenty of probabilistic situations, both in everyday life and scientific practice, when only isolated decisions are made. Let us have a look at a standard statistical problem. Suppose a scientist makes repeated measurements of a physical quantity, such as the speed of light, and then he analyzes the data. If he is a Bayesian then he chooses a prior and calculates the posterior distribution of the speed of light, using the Bayes theorem. Quite often, the only action that the scientist takes in relation to such measurements is the publication of the results and their statistical analysis in a journal. There are no other actions taken, so there can be no families of inconsistent actions, in de Finetti’s sense. It is possible that some other people may take actions that would be inconsistent with the publication of the data and their analysis, but this is beyond scientist’s control. Of course, the value of the physical constant published by the scientist may be wrong, for various reasons. However, de Finetti’s theory is concerned only with consistency and does not promise in any way that the results of the Bayesian analysis would yield probability values that are “realistic.” In simple statistical situations, there is no opportunity to be inconsistent in de Finetti’s sense in real life. One can only imagine inconsistent actions. A careful analysis of imaginary decisions shows that de Finetti’s theory is self-contradictory. I will use a decision-theoretic approach to probability to arrive at a contradiction. A decision maker has to take into account all possible decision problems, at least in principle. Some of possible actions may result in inconsistencies, and hence losses. Some of possible decision problems may or may not materialize in reality. For every potential decision problem, one should calculate the expected utility loss due to inconsistencies that may arise when the relevant decisions are not coordinated with other decisions. The number of possible decision problems is incredibly large, so we have to make most decisions at the intuitive level, or otherwise we would not be able to function. For example, a doctor advising a patient must subconsciously disregard all the facts that he knows about planets and spaceships. For any potential decision problem, one has to decide intuitively whether the expected utility loss due to informal decision making process is greater than the value of time and effort spent on the calculations that are needed for the formal Bayesian decision analysis. If the cost of time is higher, the decision problem should be approached in an informal way. The crux of the matter is that in de Finetti’s theory, objective probability does not exist. If one chooses any probability values that satisfy the usual mathematical

page 112

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch05

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

The Subjective Philosophy of Probability

113

rules, the corresponding decision strategy is consistent. Hence, one can choose probabilities in such a way that for each decision problem the expected utility loss resulting from solving it informally is smaller than the value of the time that would be needed to do the usual Bayesian analysis. Thus it is consistent to approach all decision problems informally, assuming that time of the decision maker has at least a little bit of utility. It follows that the probability theory is useless — one can be consistent by simply never doing any probabilistic calculations. The subjective theory of probability suffers from the dependence on imagined entities, just like the frequency theory. In the case of the frequency theory, one has to imagine non-existent collectives; in the case of the subjective theory of probability, one has to imagine non-existent collections of decisions, subjective probabilities are just a way of encoding consistent choices between various imaginary decisions.

5.21 A Misleading Slogan Ideologies use slogans that often are not interpreted literally but are used as guiding principles. For example, “freedom of speech” is not taken literally as an absolute freedom — it is illegal to reveal military secrets and nobody suggests that it should be otherwise. Many Christians support military forces and do not think that this necessarily contradicts the commandment that “you shall not kill.” The frequentist idea that “probability is a long run frequency” can be defended as saying that one should determine values of probabilities by repeated experiments or observations whenever practical. Or one could interpret this slogan as saying that observing long run frequencies is the “best” way of finding probability values. I have a feeling that most frequency statisticians and other frequentists try to live by these rules. In other words, they may not interpret their slogan literally but their interpretation is more or less in line with practices concerning other slogans. I cannot be equally lenient toward the subjectivist ideology. Its slogans, “probability is subjective” and “probability does not exist,” are never interpreted as “one should remove any objectivity from probabilities.” All statistical practice is concerned with finding probabilities that are as objective as possible. Subjectivity is considered to be a necessary evil. Bayesian statistics can be described as a miraculous transmutation of the subjective into the objective (see Chapter 14).

page 113

April 29, 2016

9:6

114

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch05

Resonance: From Probability to Epistemology and Back

5.22 Axiomatic System as a Magical Trick

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

The axiomatic system in the subjective theory is a magical trick. It is designed to draw attention of the audience to something that is totally irrelevant. The axiomatic system may be used to justify only the following statement about probabilities: (S) Probabilities are non-negative, not greater than 1, and if two events cannot occur at the same time, the probability that one of them is going to occur is the sum of probabilities of the two events.

De Finetti successfully formalized (S). However, (S) is trivial from the philosophical point of view, because (S) was never the subject of a scientific or philosophical controversy. Every mathematical, scientific and philosophical theory of probability contains (S), either explicitly or implicitly. The subjectivist axiomatic system draws attention away from the truly significant claim of de Finetti that “probability does not exist.” Needless to say, this claim would be hard to sell to most scientists without any magical tricks. It is perplexing to see that some people still try to place Bayesian inference on axiomatic foundations (see Lindley’s quote in Sec. 2.4.5) although the axiomatic program seems to be in retreat in those places where one would expect it to succeed. Some people believe that G¨ odel’s theorems (see [Hofstadter (1979)]) killed the axiomatic dream in mathematics. Actually, the collapse of the axiomatic program in mathematics has little to do with G¨odel. The currently most popular axiomatic system for mathematics (“ZFC,” see [Jech (2003)]) does not consist of “obvious” statements. They may seem obvious only to (some) people with the mathematic knowledge at the Ph.D. level. See more on this topic in [Lakatos (1978b), Chapters 1–2]. One would expect physics, of all non-deductive sciences, to be the most amenable to axiomatic representation, in the sense that one would hope to be able to derive all physics from a set of fundamental reasonably simple laws. The current state of physics is very far from this ideal. Moreover, reductionism in physics is highly impractical.

5.23 The Meaning of Subjectivity I continue the discussion of the meaning of the word “subjective” started in Sec. 5.1. One of the reasons why the subjective theory of probability is so successful is because the word “subjective” has numerous meanings and

page 114

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch05

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

The Subjective Philosophy of Probability

115

everyone can choose a meaning that fits his own understanding of the theory. I will review some of the meanings of the word “subjective” in the hope that this will help the discussion surrounding the subjective theory — one cannot expect a substantial convergence of opposing philosophical views if their holders use the same word in different ways. Dictionaries contain long lists of different meanings of the word “subjective” but many of those meanings are not relevant to this discussion, and vice versa, some meanings used in the specialized probabilistic context cannot be found in the dictionaries. The meaning of “subjective” in de Finetti’s theory is presented in (v) below. When Bayesian statisticians talk about subjective probability, they use the word “subjective” as in (vi), (vii), (ix) or (xiii) on the following list. I start my review by repeating verbatim four possible interpretations of the statement that “probability is subjective” and their discussion from Sec. 2.4.1. (i) Although most people think that coin tosses and similar long run experiments displayed some patterns in the past, scientists determined that those patterns were figments of imagination, just like optical illusions. (ii) Coin tosses and similar long run experiments displayed some patterns in the past but those patterns are irrelevant to the prediction of any future event. (iii) The results of coin tosses will follow the pattern I choose, that is, if I think that the probability of heads is 0.7 then I will observe roughly 70% of heads in a long run of coin tosses. (iv) Opinions about coin tosses vary widely among people. Each one of the above interpretations is false in the sense that it is not what de Finetti said or what he was trying to say. The first interpretation involves “patterns” that can be understood in both objective and subjective sense. De Finetti never questioned the fact that some people noticed some (subjective) patterns in the past random experiments. De Finetti argued that people should be “consistent” in their probability assignments and that recommendation never included a suggestion that the (subjective) patterns observed in the past should be ignored in making one’s own subjective predictions of the future, so (ii) is not a correct interpretation of de Finetti’s ideas either. Clearly, de Finetti never claimed that one can affect future events just by thinking about them, as suggested by (iii). We know that de

page 115

April 29, 2016

9:6

116

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch05

Resonance: From Probability to Epistemology and Back

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Finetti was aware of the clustering of people’s opinions about some events, especially those in science, because he addressed this issue in his writings, so again (iv) is a false interpretation of the basic tenets of the subjective theory. (v) I continue the review with the meaning that was given to the word “subjective” by de Finetti. According to him, a probability statement cannot be proved or disproved, verified or falsified. In other words, “probability” does not refer to anything that can be measured in an objective way in the real universe. (vi) The word “subjective” is sometimes confused with the adjective “relative,” see Secs. 5.1 and 11.5. Different people have different information and, as is recognized in different ways by all theories, the probability of an event depends on the information possessed by the probability assessor. One can deduce from this that probability is necessarily subjective, because one cannot imagine a realistic situation in which two people have identical knowledge. This interpretation of the word “subjective” contradicts in a fundamental way the spirit of the subjective theory. The main idea of the subjective theory is that two rational people with access to the same information can differ in their assessment of probabilities. If the differences in the probability assessments were attributable to the differences in the knowledge, one could try to reconcile the differences by exchanging the information. No such possibility is suggested by the subjective theory of probability, because that would imply that probabilities are a unique function of the information, and in this sense they are objective. De Finetti was not trying to say that the impossibility of perfect communication between people is the only obstacle preventing us from finding objective probabilities. (vii) Another meaning of subjectivity is that information is processed by human beings so it is imperfect for various reasons, such as inaccurate sensory measurements, memory loss, imprecise application of laws of science, etc. Humans are informal measuring devices. A person can informally assess the height of a tree, for example. This is often reasonable and useful. Similarly, informal assessment of probabilities is only an informal processing of information, without explicit use of probabilistic formulas. This is often reasonable and useful. But this is not what de Finetti meant by subjective probability, although this is considered to be the subjective probability by many people.

page 116

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch05

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

The Subjective Philosophy of Probability

117

In order to implement (L1)–(L6) in practice, one has to recognize events that are disjoint, independent or symmetric. This may be hard for a number of reasons. One of them is that no pair of events is perfectly symmetric, just like no real wheel is a perfect circle. Hence, one has to use a “subjective” judgment to decide whether any particular pair of events is symmetric or not. Even if we assume that some events are perfectly symmetric, the imperfect nature of our observations makes it impossible to discern such events and, therefore, any attempt at application of (L1)–(L6) must be subjective in nature. This interpretation of the word subjective is as far from de Finetti’s definition as the interpretation in (vi). In de Finetti’s theory, real world symmetries are totally irrelevant when it comes to the assignment of probabilities. In his theory, probability is subjective in the sense that numbers representing probabilities are not linked in any way to the observable world. Probability values chosen using symmetries are not verifiable, just like any other probability values, so symmetry considerations have no role to play in the subjective theory. (viii) “Subjective” opinion can mean “arbitrary” or “chaotic” in the sense that nobody, including the holder of the opinion, can give any rational explanation or guiding principle for the choice of the opinion. This meaning of subjectivity is likely to be attributed to subjectivists by their critics. In some sense, this critical attitude is justified by the subjective theory — as long as the theory does not explicitly specify how to choose a consistent opinion about the world, you never know what a given person might do. I do not think that de Finetti understood subjectivity in this way. It seems to me that he believed that an individual may have a clear, well organized view of the world. De Finetti argued that it is a good idea to make your views consistent, but he also argued that nothing can validate any specific set of such views in a scientific way. (ix) “Subjective” can mean “objectively true” or “objectively valuable” but “varying from person to person.” For example, my appreciation of Thai food is subjective because not all people share the same taste in food. However, my culinary preferences are objective in another sense. Although my inner feeling of satisfaction when I leave a Thai restaurant is not directly accessible to any other person, an observer could record my facial expressions, verbal utterances and

page 117

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

118

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch05

Resonance: From Probability to Epistemology and Back

restaurant choices to confirm in quite an objective way that Thai food indeed gives me pleasure and is among my favorite choices. There is no evidence that this interpretation of the word “subjective” has anything to do with de Finetti’s theory. In many situations, such as scientific research, the consequences of various decisions are directly observable by all interested people and there is a universal agreement on their significance. In such cases, a result of a decision cannot be “good” or “true” for one person but not for some other person. (x) One may interpret “subjectivity” as an attitude. A quantity is “objective” if scientists attempt to measure it in more and more accurate ways, by designing better equipment and developing new theories. In case of subjective preferences, one can measure the prevailing attitudes in the population (for example, popularity of different restaurants), but no deeper meaning is given to such statistics. In other words, subjectivity may be considered to be the antonym of objectivity, and objectivity may be identified with consensus. For most people, the only way to know that quantum mechanics is true is that there is a consensus among physicists on this subject. Hence, objectivity may be identified with consensus in the operational sense. (xi) We may define subjectivity using its antonym — objectivity, but this time we may choose a different definition of objectivity. A quantity may be called objective if it may exist without human presence, knowledge or intervention, for example, the temperature on the largest planet in the closest galaxy to the Milky Way is objective. (xii) One can try to characterize subjectivity or objectivity operationally, in terms of the attitude of the society toward attempts at changing someone’s mind. Consider the following statements: “blond hair is beautiful,” “lions eat zebras” and “smoking tobacco increases the probability of cancer.” Suppose, for the sake of argument, that most people believe in each of these statements. Cosmetics companies might want to sell more dark hair dye and so they might start an advertising campaign trying to convince women that dark hair has a lot of sex appeal with no reference to solid empirical evidence for the last claim. Although people have mixed feelings toward advertising, nobody would complain that such an advertising campaign is fundamentally unethical. As for lions and zebras, it would

page 118

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch05

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

The Subjective Philosophy of Probability

119

be unthinkable for anyone to start a campaign trying to convince people that zebras eat lions, without any striking new evidence. It is clear that the statement about smoking and cancer belongs to the category of scientific facts rather than subjective opinions from the point of view of advertising. A tobacco company trying to convince people that smoking is healthy would draw public wrath, unless its claims were supported by very solid scientific data. (xiii) Some critics of Bayesian statistics assert that the use of subjective priors makes the theory unscientific because there is no place for subjectivity in science. Bayesians retort that frequentist models and significance levels are also chosen in a subjective way. I will not comment here on the merits of either argument. I want to make a linguistic point. I believe that when some people criticize the use of “subjective” priors in statistics, what they really mean is that the source of (some) priors is not explicitly known and amenable to scientific scrutiny, unlike frequentist and Bayesian statistical models. I will try to clarify my point by discussing a case when a prior is subjective in some sense but not subjective in the sense that I have just defined. Suppose that a Bayesian statistician claims that in a certain situation there is no prior information available so it is best to use a non-informative prior which happens to be the uniform distribution in this particular case. This prior is subjective in the sense that an individual chose to use the uniform prior based on his personal judgment. But this prior is not subjective in the sense that it is based on unknown and unknowable information processed in an unknown and unknowable way.

5.24 Probability and Chance Subjectivists keep trying to perform an impossible balancing act. On one hand, they claim that all probability is subjective (or personal). On the other hand, they have to account for all the existing scientific evidence for objective probability, so that they do not appear to be totally irrational. Let us see how successful Dennis Lindley, a leading subjectivist, was in this endeavor. He proposed to use two terms, “probability” and “chance,” in his book [Lindley (2006)] titled “Understanding uncertainty.” In my opinion, Lindley’s “probability” means “subjective probability” and his “chance”

page 119

April 29, 2016

9:6

120

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch05

Resonance: From Probability to Epistemology and Back

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

means “objective probability,” although I am sure that he would not accept my interpretation because he believed in chance but he did not believe in objective probability. On page 115 of his book Lindley wrote Another feature of the Bernoulli θ is that it has a degree of objectivity in the sense that if Peter and Mary both judge a series to be exchangeable, then the value of θ, as a limiting frequency, will be common to them both, though unknown to them both. The objectivity is limited though because if Paul disagrees with exchangeability, θ may not have a meaning for him. Experience shows that there is a massive agreement about some series being exchangeable, so that the objectivity can be at least a convenient approximation. The upshot of these considerations is that θ, while it obeys the rules of the probability calculus, is not a probability in the sense of a belief. As a result, we prefer to give it a different name and it is often referred to as a chance.

The passage refers to repeated tosses of a thumbtack. The sequence of results, if it is exchangeable, can be represented as a mixture of i.i.d. sequences with the mixing measure identified with the distribution of θ. (a) Let’s start with Lindley’s remark that “θ, as a limiting frequency, will be common to them both.” If this remark refers to the subjective limiting frequency, that is, if the value of θ (that is, the limiting frequency) is what Peter and Mary imagine then there is no reason why the value of θ has to be “common to both,” either as a guess about a single specific value, or in the sense of the common probability distribution of θ on the interval [0,1]. Hence, the only possible interpretation of “will be common to them both” is that in an infinite series of experiments we would objectively observe a limiting frequency. In real life there are no infinitely long sequences so we have to find an interpretation of the claim that would apply if we had access only to a finite, possibly long, sequence of trials. A possible interpretation is that each sequence of exchangeable experiments corresponds to an objective θ and the value of this θ can be effectively estimated from a long finite sequence. This is, of course, an objective interpretation of probability, hence, I am quite sure that Lindley would reject this interpretation. If objective θ does exist but it cannot be effectively estimated then I consider it a totally uninteresting philosophical invention. Finally, Lindley might believe that θ does not exist in the objective sense. Then his claim that “θ, as a limiting frequency, will be common to them both” becomes totally mysterious to me. Perhaps Lindley is saying that if both Peter and Mary believe that the sequence is exchangeable then they have to believe that

page 120

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch05

The Subjective Philosophy of Probability

121

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

in a world where infinite sequences existed, they would have observed the same limiting frequency. But what does this claim say about our world? If Peter and Mary observe a very long but finite sequence, both of them will observe the same frequency of successes. Needless to say, the last claim is trivially true. No matter how I look at the claim “θ, as a limiting frequency, will be common to them both,” the only interpretation that makes sense to me is that θ is an objective number characterizing the experiments in the sequence. (b) Next, let’s talk about Lindley’s remark that “Experience shows that there is a massive agreement about some series being exchangeable, so that the objectivity can be at least a convenient approximation.” There are massive agreements on a number of issues. The agreements fall into (at least) two categories illustrated by the following examples. (i) Most people in the US believe that pink clothes are appropriate for baby girls and blue clothes are appropriate for baby boys. (ii) Most people believe that the Earth circles the Sun, not vice versa. Example (i) shows that a massive agreement can be reached about an issue of taste. Such a massive agreement is not accompanied by any kind of objectivity. Example (ii) shows that people may reach an agreement about a question concerning an objective fact. It seems that there was a massive agreement on the statement opposite to (ii) centuries ago, so a massive agreement does not prove that the subject of the agreement must be an objective truth. But it is quite clear that both in the remote past and now, people believed and believe that the motion of the Earth and the Sun is a question of objective truth and the current agreement is about the best available astronomical theory of the solar system. Hence, Lindley’s remark that “Experience shows that there is a massive agreement about some series being exchangeable, so that the objectivity can be at least a convenient approximation” makes sense only if objective probability exists. If objective probability does not exist then the massive agreement on exchangeability belongs to the same category as Example (i) and it is not a “convenient approximation” of objectivity. (c) Lindley says “if Paul disagrees with exchangeability, θ may not have a meaning for him.” This indicates to me that Lindley does not believe that exchangeability is an objective property of a sequence. This impression is strongly reinforced by the sentence: “Experience shows that there is a

page 121

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

122

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch05

Resonance: From Probability to Epistemology and Back

massive agreement about some series being exchangeable.” If it is impossible to know objectively whether a sequence is exchangeable, it is also impossible to know whether the limiting frequency exists. So the remark “θ, as a limiting frequency, will be common to them both” has to be interpreted as saying “if the sequence is exchangeable then θ, as a limiting frequency, will be common to them both.” This, in turn, can be interpreted in two different ways. First, this could mean “if the sequence is objectively exchangeable then θ, as a limiting frequency, will be common to them both.” So once again we arrive at a claim that exchangeability is an objective property of some sequences. Or we could interpret the claim as “if the sequence is subjectively exchangeable then θ, as a limiting frequency, will be common to them both.” I find it hard to interpret this statement. Going back to my earlier remarks, the claim could mean that if both Paul and Mary believe that the sequence is exchangeable then they have to believe that in a world where infinite sequences existed, they would have observed the same limiting frequency. But what does this claim say about our world? Of course, if Paul and Mary observe a very long but finite sequence, both of them will observe the same frequency of successes. In conclusion, Lindley’s strained attempts to avoid objective probability seem totally unconvincing. You cannot be a subjectivist and a scientist at the same time.

5.25 Conflict Resolution The following quote comes from the very end of [Lindley (2006), pp. 241–242]: Any sound appreciation of the thesis presented in this book must recognize a severe limitation of that thesis [...] The thesis is personal. [centered in original] That is to say, it is a method for “you”. [...] Our theory only admits one probability and one utility. When co-operation is present [...] a “you” is not unreasonable, but not with conflict. [...] My hope is that today, somewhere in the world, there is a young person with sufficient skill and enthusiasm to be given at least five years to spend half their time thinking about decision making under conflict. They will need to be a person with considerable skill in mathematics, for only a mathematician has enough skill in reasoning and abstraction to capture what hopefully is out there, waiting to be discovered. Conflict is the most important mathematical and social problem of the present time.

page 122

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch05

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

The Subjective Philosophy of Probability

123

The above text is followed by only one short paragraph on an unrelated topic. Delegating these ideas to the very end of the book indicates that Lindley attached a great importance to conflict resolution. I find Lindley’s invocation incomprehensible. The civil law goes back to antiquity, most notably to the Roman Empire. The system of civil law is, in part, a system for conflict resolution. Another old and well known method of conflict resolution is consensual compromise via negotiations. Various ideologies include ideas on conflict resolution. An example of such an idea comes from the utilitarian philosophy: “The good of a society is the sum of happiness of the individuals in that society.” This may lead to the “optimal” solution of conflicts. Or one could remove the source of the conflict. Religious suggestions along these lines include “If someone slaps you on one cheek, turn to them the other also” (Bible) and “Desire is the root of evil” (Buddha). In view of the long history of philosophical research on various ideas related to conflict resolution, Lindley’s suggestion that a mathematician could make substantial contributions to this field after a five year effort is truly off the wall. Lindley’s intentions are commendable but he misses the fundamental solution to his quandary. The incredibly powerful and successful solution to conflict resolution is objective science. People of different persuasions will agree on science, if they agree on anything. Consider an example of enemy states who signed a treaty limiting the number of tanks on each side to 5,000. Each side may try to cheat by hiding tanks in the woods, calling tanks “armored vehicles,” etc. But if one state documents presence of 4,000 tanks in one part of the other state’s territory and 3,000 tanks in some other part, the offending state will not claim that 4,000 + 3,000 = 5,000. Why not, if all other methods of cheating seem to be acceptable or at least practiced? The laws of science are so fundamental that if you deny them, no trust between the parties is left. Saying that 4,000 + 3,000 = 5,000 effectively removes any and all basis for cooperation or negotiation. If the laws of science are removed then nothing is left except naked force. In the last paragraph of his book, Lindley denies that he is a religious subjectivist and claims to be rational. But only religious zeal can make someone blind to the fact that the science of statistics is a collective attempt to find as much objective probability in uncertain situations as possible. Bayesian statisticians are not any less objective than the frequentists. The best known and widely practiced method of finding objective values of probabilities is based on relative frequencies in long

page 123

April 29, 2016

9:6

124

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch05

Resonance: From Probability to Epistemology and Back

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

sequences of independent and identical trials. Lindley dismisses the method by calling it “chance” (to distinguish it from probability) and devotes to it a small section of his book. I agree with Lindley that it is a good idea to have reliable practical methods for conflict resolution under uncertainty. They are called objective probability. I hope that no mathematician wastes five years of his life searching for something that is already well known.

page 124

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch06

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Chapter 6

The Logical Philosophy of Probability

6.1 Falsifiability My discussion of the logical philosophy of probability will be brief because this direction in philosophy did not achieve much recognition among scientists. This could change in the future (I hope it does not) because the book [Jaynes (2003)] seems to enjoy some popularity. The reader may be amused to learn that according to [Lyon (2010), p. 112], “Nearly every philosopher now agrees that the logical interpretation of probability is fundamentally flawed.” Lyon refers to the philosophical program developed in [Carnap (1950)]. Carnap tried to create a universal formal language for all of science that would include the concepts of degree of confirmation and probability. That was a monumental intellectual undertaking but it had no direct impact on science and it does not appear that it had much impact on our understanding of science. Jaynes proposed in his book [Jaynes (2003)] to consider a robot and used this concept to formulate his “basic desiderata.” For example, Desideratum (IIIc) on page 19 starts with The robot always represents equivalent states of knowledge by equivalent plausibility statements.

The idea of the robot seems to me to be completely misguided. Why do probability theory and statistics need a robot if Newton’s laws of motion do not include a robot in their statements? I guess Jaynes wanted to emphasize the objective nature of his desiderata in contrast to the subjectivist approach.

125

page 125

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

126

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch06

Resonance: From Probability to Epistemology and Back

A major practical and philosophical difficulty with Jaynes’ philosophy is that people do not agree on what constitutes “equivalent states of knowledge.” So even if a robot acting according to (IIIc) exists somewhere in an imaginary universe, we do not know what it plans to do. Or rather different people are convinced that the robot will act according to their own opinions, which often happen to be different from the opinions of other people. Let me illustrate my point by the famously incorrect banner headline “Dewey Defeats Truman” on the front page of the Chicago Tribune on November 3, 1948. The following quote comes from [Wikipedia (2014j)]. On election night, the Chicago Tribune printed the headline “Dewey Defeats Truman,” which turned out to be mistaken. In the morning the grinning president-elect, Harry S. Truman, was photographed holding a newspaper bearing this headline. The reason the Tribune was mistaken is that their editor trusted the results of a phone survey. Survey research was then in its infancy, and few academics realized that a sample of telephone users was not representative of the general population.

Imagine that a robot existed somewhere in the universe and knew that the phone survey did not generate a uniform sample of all eligible voters. That was irrelevant because people who did the data analysis had no access to the robot. The idea of a robot is similar to the idea of God and it is a philosophical failure for the same reasons. Philosophers often invoke God if they lack any solid argument. Recall Dostoevsky’s claim that “If God does not exist, everything is permitted.” It is obvious to me that Dostoevsky was implicitly saying that “If God does exist, some things are forbidden.” The recipient of Dostoevsky’s message is supposed to accept the claim because it allegedly makes no sense to question God’s will or to try to understand God’s motives and reasoning. Similarly, the idea of a robot is supposed to eliminate the need for the search for an explanation of why one should represent “equivalent states of knowledge by equivalent plausibility statements.” To be fair, the idea of invoking a higher authority to silence opponents is not limited to theology. Communist propagandists treated writings of Marx, Lenin and Stalin like holy texts. The American Constitution is often treated in the same way in political propaganda. The famous “Dewey Defeats Truman” statistical failure points in a different direction than including a robot in the foundations of probability. We should be able to detect mistakes in probability assignments so that

page 126

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch06

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

The Logical Philosophy of Probability

127

we can improve our methods. What I am missing in [Jaynes (2003)] is a careful discussion of falsifiability of probability statements. People can use wrong probability values for all kinds of reasons, including trivial ones, such as typing errors. Jaynes discusses hypothesis testing (of course) but this is not the same as the philosophical discussion of the means of falsifying probability statements. Without such a discussion, one has to presume that, in Jaynes theory, the only falsification methods are those of the frequency philosophy or the subjective philosophy. I have argued in the previous chapters that none of these is acceptable or realistic. Jaynes’ robot will look even more awkward in view of the resonance theory, to be developed in Sec. 8.2. Humans use their resonance abilities to recognize symmetries relevant to probability. If there is anything that robots lack at this point, relative to humans, it is precisely the higher level resonance abilities. Most of the time we complain that contemporary robots lack intelligence, the highest level of resonance, but they also lack some of the more basic resonance abilities, such as those used in building simple probability models.

6.2 Why Do Scientists Ignore the Logical Philosophy of Probability? Mathematicians and statisticians are satisfied with Kolmogorov’s axioms and do not need an alternative formal (mathematical, algebraic) foundations of probability theory. Some alternative theories to Kolmogorov’s axioms have been proposed and developed, such as finitely additive probability, “non-standard” probability and fuzzy sets, but none of these theories made much impact on the field. In logical theories of probability a numerical probability is assigned to a proposition, not event. My guess is that most scientists cannot tell the difference between the claims that “the probability of rain in Seattle on January 7, 2020, is 78%” and “the probability that the proposition ‘there will be rain in Seattle on January 7, 2020,’ is true is 78%.” Moreover, my guess is that even if they could tell the difference, they would be at a loss why to bother to use the second version. Mathematicians do not like logic, contrary to popular opinion. Outside the field of “mathematical logic,” mathematicians know very little logic and show little interest in its development and methods. A typical mathematician knows only a handful of non-trivial logical arguments: the

page 127

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

128

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch06

Resonance: From Probability to Epistemology and Back

induction principle, the “diagonal method” of Cantor, the axiom of choice (but often he does not realize when he uses it implicitly) and knows about G¨ odel’s theorems (but he might not be able to state them correctly). The “non-standard analysis,” a sophisticated logical theory, is used by some mathematicians, although it is shied away by most. Despite my negative initial remark, these examples show that at least some non-trivial logical methods found a way into mathematics and so, indirectly, into science. In contrast, I am not aware of any applications of the formal methods developed by the logical school of philosophy of probability to mathematics, statistics or other sciences.

6.3 Probabilities of Propositions and Events I will use a paradox discovered by [Lewis (1976)] to argue that assigning probability values to propositions is a bad choice. In short, Lewis showed that “probabilities of conditionals are not conditional probabilities.” I will present my argument in the standard Kolmogorov framework. Suppose that we have a very simple probability space partitioned into three elementary events A, B and C, that is, these events are pairwise exclusive and they exhaust all possibilities. Suppose that P (A) = 0.4, P (B) = 0.3 and P (C) = 0.3. Then the conditional probability of A given A or B is 4/7, that is, P (A | A or B) = P (A)/P (A or B) = 0.4/(0.4 + 0.3) = 4/7. It follows that there is no event F in this probability space with the property that P (F ) = P (A | A or B) because probabilities of all events are multiples of 0.1. None of the above presents a problem to the Kolmogorov theory because the expression “A | A or B” was never supposed to represent an event. However, in the logical philosophy of probability, the function P is supposed to take propositions as arguments. A natural guess is that, in probability statements, the expression “A | A or B” should represent the proposition “(A or B) implies A.” If the proposition “(A or B) implies A” has a probability in our probability space then it cannot be equal to the conditional probability P (A | A or B) because the latter is equal to 4/7 and no proposition has this probability in our probability space. I have to confess that I greatly simplified and perhaps distorted Lewis’ argument so if the reader is interested in the full story then they should

page 128

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch06

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

The Logical Philosophy of Probability

129

consult the original paper. But my point seems to be somewhat different from that of Lewis. Scientists must assign probabilities to some entities which can take values of the binary type (0 or 1) at least in some circumstances. Events are such entities. Scientists do not need to assign value 1 or 0 (“occurred” or “not occurred”) to “A | A or B” because, in practical applications, it is enough to assign binary values to “A or B” and “A.” It is possible that this observation may lead to a solution of “Lewis’ paradox” but I doubt that it is worth the effort. If we make a prediction P (A | B) = 0.99 then we can falsify it only if we determine that B occurred but A ∩ B did not occur (see Sec. 11.3 for the discussion of probabilistic predictions and their falsification). The conditional predictions cannot be falsified by observing any single event because they are statements about the ratio P (A ∩ B)/P (B) which does not imply any assertion about the values of the two individual probabilities. Conditional probabilities are treated as probabilities in mathematics and science because both families obey the same algebraic rules but conditional probabilities are relations between probabilities, not statements about a single probability. As far as I can tell, the logical philosophy of probability does not contain a convincing theory of falsifiability of conditional probabilities.

page 129

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch07

Chapter 7

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Common Issues

7.1 Law Enforcement The area of law enforcement provides excellent opportunities to document the total disconnection between the two most popular philosophies of probability and the real applications of probability. Consider the following two criminal cases. In the first case, a house was burgled and some time later, Mr. A.B., a suspect, was arrested. None of the stolen items were ever recovered but the police found a piece of evidence suggesting his involvement in the crime. The owners of the house kept in their home safe a 10-letter code to their bank safe. The home safe was broken into during the burglary. The search of Mr. A.B. yielded a piece of paper with the same ten letters as the code stored in the home safe. In court, Mr. A.B. maintained that he randomly scribbled the 10 letters on a piece of paper, out of boredom, waiting at a bus stop. The prosecution based its case on the utter improbability of the coincidental agreement between the two 10-letter codes, especially since the safe code was generated randomly and so it did not contain any obvious elements such as a name. The other case involved Mr. C.D. who shot and killed his neighbor, angered by a noisy party. In court, Mr. C.D. claimed that he just wanted to scare his neighbor with a gun. He admitted that he had pointed the gun at the neighbor from the distance of 3 feet and pulled the trigger but pointed out that guns not always fire when the trigger is pulled, and the target is sometimes missed. Under questioning, Mr. C.D. admitted that he had had years of target practice, that his gun fired about 99.9% of the time, and he missed the target about 1% of the time. Despite his experience with guns, Mr. C.D. estimated the chances of hurting the neighbor as 1 in a billion. 131

page 131

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

132

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch07

Resonance: From Probability to Epistemology and Back

I am convinced that no jury would believe the explanations offered by the two defendants. Each of the defendants, though, could invoke one of the official philosophies of probability to strengthen his case. In the case of Mr. A.B., the frequency theory says that no probabilistic statements can be made because no long run of isomorphic observations (“collective”) was involved. Specifically, a sequence of only 10 letters cannot be called long. Likewise, the police did not record a long run of burglaries involving stolen codes. One could suggest running computer simulations of ten random letters, but Mr. A.B. would object — in his view, computer simulations are completely different from the workings of his brain, especially when he is “inspired.” Mr. A.B. could also recall that, according to von Mises, nothing can be said about the (im)probability of a specific event, even if it is a part of a well defined collective. Mr. C.D. could invoke the subjective theory of probability. No matter what his experience with guns had been, his assessment of the probability of killing the neighbor was as good as any other assessment, because probability is subjective. Hence, the killing of the neighbor should have been considered an “act of God” and not a murder, according to Mr. C.D. He could even present an explicit Bayesian model for his gun practice and a prior distribution consistent with his assertion that the chance of hurting the neighbor was 1 in a billion. Needless to say, societies do not tolerate and cannot tolerate interpretations of probability presented above. People are required to recognize probabilities according to (L1)–(L6) and when they fail, or when they pretend that they fail, they are punished. A universal (although implicit) presumption is that (L1)–(L6) can be effectively implemented by members of the society. If you hit somebody on the head with a brick, it will not help you to claim that it was your opinion that the brick had the same weight as a feather. The society effectively assumes that weight is an objective quantity and requires its members to properly assess the weight. The society might not have explicitly proclaimed that probability is objective but it effectively treats the probability laws (L1)–(L6) as objective laws of science and enforces this implicit view on its members. There are countless examples of views — scientific, philosophical, religious, political — that used to be almost universal at one time and changed completely at a later time. The universal recognition or implementation of some views does not prove that they are true. One day, the society may cease to enforce (L1)–(L6). However, neither frequentists nor subjectivists object

page 132

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch07

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Common Issues

133

to the current situation in the least. I see no evidence that any statistician would have much sympathy for the probabilistic arguments brought up by the two defendants in my examples. Supporters of the frequency and subjective interpretations of probability use their abstract philosophical arguments only in purely academic discussions and otherwise they use common sense — something I am trying to formalize as (L1)–(L6). All societies enforce probability values of certain events. Democratic societies do it via elected governments. Various branches of the government enforce safety and security regulations, implicitly saying that certain actions decrease the probability of death, injury or sickness. For example, manufacturers have to print warning labels on household chemicals (detergents, cleaners, paint), motorists have to fasten seat belts, companies have to obey regulations concerning the size and shape of toys for babies and small children, drug companies have to follow certain procedures when developing, testing and submitting drugs for approval. Social pressure can be as effective in enforcing probability values as the justice system. Unconventional probabilistic opinions, for example, “We are likely to be late for the meeting because two butterflies appeared in my garden,” may result in social ostracism — you may lose, or not acquire, friends, a job, an investment opportunity, etc.

7.2 The Value of Extremism In defense of de Finetti and von Mises, I have to say that they realized that their theories could not accept compromises. This put them far above many other authors who tried to fix the most striking problems with the frequency and subjective theories by taking compromise positions on some issues. Such fixes inevitably destroyed the foundations of the two philosophical theories.

7.3 Common Elements in Frequency and Subjective Theories Despite enormous differences between the frequency and subjective theories of probability, they share some similar ideas. Both von Mises and de Finetti tried to find certainty in the universe full of uncertainty. Von Mises identified probabilities with frequencies in infinite sequences (collectives). According to the strong Law of Large Numbers, the identification is perfect, that is, it occurs with certainty. The fact that real sequences are finite may be dismissed as an imperfect match between theory and

page 133

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

134

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch07

Resonance: From Probability to Epistemology and Back

reality — something that afflicts all scientific theories. De Finetti argued that probability can be used to achieve a different practical goal with certainty — if you use the mathematical theory of probability to coordinate your decisions then you will not find yourself in a Dutch book situation. None of the two philosophers could think of a justification for the scientific uses of probability that would not involve perfect, that is, deterministic, predictions. One may present the above philosophical choice of von Mises and de Finetti as their common belief that in any scientific theory, probability should be a physical quantity measurable in the same way as mass, electrical charge or length. In other words, one should have an effective way of measuring the probability of any event. Von Mises defined probability in an operational way, as the result of a specific measurement procedure — the observation of the limiting relative frequency of an event in an infinite (or very long) sequence of isomorphic experiments. He unnecessarily denied the existence of probability in other settings. De Finetti could not think of any scientific way to achieve the goal of measuring probability with perfect accuracy so he settled for an unscientific measurement. In the subjective theory, the measurement of probability is straightforward and perfect — all you have to do is to ask yourself what you think about an event. The incredibly high standards for the measurement of probability set by von Mises and de Finetti have no parallel in science. Take, for example, temperature. A convenient and reliable way to measure temperature is to use a thermometer. However, if the temperature is defined as the result of this specific measurement procedure, then we have to conclude that there is no temperature at the center of the Sun. At this time, it seems that we will never be able to design a thermometer capable of withstanding temperatures found at the center of our star. Needless to say, physicists do not question the existence of the temperature at the center of the Sun. Its value may be predicted using known theories. The value can be experimentally verified by combining observations of the radius, luminosity, and other properties of the Sun with physical theories. Von Mises and de Finetti failed for the same reason — they set unattainable goals for their theories.

7.4 Common Misconceptions For reference, I list common misconceptions about the frequency and subjective philosophies of probability. All items are discussed in some

page 134

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch07

Common Issues

135

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

detail in other sections of this book. The list is eclectic. It contains a quick review of some of my main claims, and some widespread elementary misconceptions. (i) The main claims of the frequency and subjective philosophies are positive. In fact, they are negative: “individual events do not have probabilities.” (ii) The two philosophies are at the opposite ends of the intellectual spectrum. In fact, they are the only philosophies of probability that claim that individual events do not have probabilities. (iii) The frequency philosophy is based on the notion of an i.i.d. sequence. In fact, it is based on the notion of a collective. A collective is a given deterministic sequence, not a sequence of random variables. Hence, it is hard to talk about independence of its elements. (iv) According to the frequency theory, an event may have two (or more) probabilities because it may belong to two different sequences. In fact, according to the frequency theory, a single event does not have a probability at all. (v) De Finetti’s theory is subjective. In fact, it is objective. (vi) The theory of von Mises justifies hypothesis testing. In fact, it cannot be applied to sequences of non-isomorphic hypothesis tests. Elements of collectives have everything in common except probabilities. Some sequences of hypothesis tests have nothing in common except probabilities. (vii) Statistical priors are the same as philosophical priors. In fact, a philosophical “prior” is equivalent to the conjunction of a statistical “prior” and “model.” (viii) The frequency theory endows every probability with a meaning via a relative frequency in some, perhaps imagined, sequence. In fact, von Mises thought that probability cannot be applied to some events although we can always imagine a corresponding collective. (ix) Computer simulations supply the missing collective in case there is no real one. In fact, they contribute nothing on the philosophical side. They are just a very effective algorithm for calculations. (x) “Subjective” means “informally assessed.” In fact, in de Finetti’s theory “subjective” means “does not exist.” (xi) According to de Finetti, some probabilities are subjective. In fact, according to de Finetti, all probabilities are subjective (do not exist).

page 135

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

136

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch07

Resonance: From Probability to Epistemology and Back

(xii) According to the subjective theory of probability, two people may have different subjective opinions about probabilities if they have different information. In fact, according to the subjective theory of probability, two people may have different subjective opinions about probabilities even if they have identical information. (xiii) Bayesian statisticians use probability to coordinate decisions. In fact, in most cases there are no decisions that need to be coordinated. (xiv) The statement that “smoking decreases the probability of cancer” is inconsistent. Actually, it is neither consistent nor inconsistent because the concept of consistency applies only to families of statements involving probability. This particular statement about smoking is a part of a consistent view of the world. (xv) Posterior distributions converge when the amount of data increases. This is not true. If one person thinks that a certain sequence is exchangeable and someone else has the opposite view then their posterior distributions might never be close. (xvi) De Finetti’s theory, unlike the frequency theory, endows probabilities of individual events with a meaning. In fact, his theory only says that the probabilities of the event and its complement should sum up to 1.

7.5 Shattered Dreams of Perfection At the end of the 19th century many scientists and philosophers thought that perfect knowledge was attainable, at least in some narrow fields. Lord Kelvin said that “There is nothing new to be discovered in physics now. All that remains is more and more precise measurement.” Bertrand Russell thought that one can place all of mathematics on absolutely solid foundations. Some trends in philosophy were devoted to finding an unshakable framework for all of knowledge. The dreams of physicists were shattered by Einstein and other 20th century physicists. The dreams of mathematicians were broken by G¨ odel. The dreams of philosophers never materialized. De Finnetti’s dream amazingly survived in the distorted belief that a rational person has to be Bayesian. De Finetti’s theory was supposed to be a method for finding, in a totally automatic manner, a perfectly rational behavior. Von Mises did not go that far but his proposal of limiting probability applications to collectives was a step towards perfection via rejection of seemingly imperfect applications of probability.

page 136

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch07

Common Issues

137

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

7.6 What Exists? Scientists are accustomed to conflicting claims concerning existence or nonexistence of an object. For example, in basic algebra, square root of a negative number does not exist. On the other hand, the field of complex analysis is based on the notion of the imaginary unit, which is the square root of −1. A more arcane example is the derivative of Brownian motion. It can be proved that Brownian trajectories do not have derivatives. On the other hand, scientists and mathematicians routinely use the notion of “white noise,” which is precisely that — the derivative of Brownian motion. In the end, the only thing that matters is the operational definition of an object that “exists.” For example, you can add two complex numbers but there is no useful ordering of complex numbers, similar to the ordering of real numbers. There is a useful notion of independence of two white noise processes but there is no useful notion of the value of a white noise at a fixed time. For a scientist, what really matters is the operational meaning of the theories of von Mises and de Finetti. The two philosophers agreed that one cannot measure the probability of a single event in a scientific way. This claim is in sharp contrast to reality. Scientists go to great lengths to find consensus on the value of probability of important events, such as climate change. At the operational level, there is no fundamental difference between efforts of scientists to find an accurate value of the speed of light, the age of the universe, or the probability that New Orleans will be devastated by a hurricane in the next 50 years. In this sense, probability of a single event does exist.

7.7 Abuse of Language Much of the confusion surrounding probability can be attributed to the abuse of language. Some ordinary words were adopted by probabilists and statisticians following the custom popular in all of science. In principle, every such word should acquire a meaning consistent with the statistical method using it. In fact, the words often retain much of the original colloquial meaning. This is sometimes used in dubious philosophical arguments. More often, the practice exploits subconscious associations of users of probability and statistics. The questionable terms often contain hidden promises with no solid justification. I will now review terms that I consider confusing.

page 137

April 29, 2016

9:6

138

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch07

Resonance: From Probability to Epistemology and Back

7.7.1 Expected value The “expected value” of the number of dots on a fair die is 3.5. Clearly, this value is not expected at all. In practice, the “expected value” is hardly ever expected. See Sec. 12.2.1 for a more complete discussion.

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

7.7.2 Standard deviation The “standard deviation” of the number of dots on a fair die is about 1.7. The possible (absolute) deviations from the mean are 0.5, 1.5 and 2.5, so 1.7 is not among them. Hence, the phrase “standard deviation” is misleading for the same reason that “expected value” is misleading. In my opinion, “standard deviation” does much less damage than “expected value.”

7.7.3 Subjective opinions See Sec. 5.1 for a long list of different meanings of “subjectivity” in the probabilistic context. Only one of them, (v), fits well with the philosophical theory invented by de Finetti. This special meaning is rarely, if ever, invoked by statisticians and users of statistics.

7.7.4 Optimal Bayesian decisions Subjectivist decisions cannot be optimal, contrary to the implicit assertion contained in the title of [DeGroot (1970)], “Optimal Statistical Decisions.” Families of decisions can be consistent or inconsistent according to the subjective theory. One can artificially add some criteria of optimality to the subjective philosophy, but no such criteria emanate naturally from the theory itself.

7.7.5 Confidence intervals Confidence intervals are used by frequency statisticians. The word “confidence” is hard to comprehend in the “objective” context. It would make much more sense in the subjectivist theory and practice. A similar concept in Bayesian statistics is called a “credible interval.” I do not think that the last term is confusing but I find it rather awkward.

7.7.6 Significant difference When a statistical hypothesis is tested by a frequency statistician, a decision to reject or accept the hypothesis is based on a number called

page 138

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch07

Common Issues

139

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

a “significance level.” The word “significant” means in this context “detectable by statistical methods using the available data.” This does not necessarily mean “significant” in the ordinary sense. For example, suppose that the smoking rates in two countries are 48.5% and 48.7%. This may be statistically significant, in the sense that a statistician may be able to detect the difference, but the difference itself may be insignificant from the point of view of the health care system.

7.7.7 Consistency The word “consistent” is applied in de Finetti’s theory to decision strategies that satisfy a certain system of axioms. The word has a different meaning in everyday life. For example, a scientist may say that “The only conclusion consistent with the data on smoking and cancer is that smoking cigarettes increases the probability of lung cancer.” In this case, “consistent” means “rational” or “scientifically justifiable.” In fact, the statement that “smoking cigarettes decreases the probability of lung cancer” is also consistent in de Finetti’s sense. More precisely, there exists a consistent set of probabilistic views that holds that smoking is healthy. There is more than one way to see this but the simplest one is to notice that the posterior distribution is determined in part by the prior distribution. If the prior distribution is sufficiently concentrated on the mathematical equivalent of the claim that smoking is healthy than even the data on 10100 cancer patients will not have much effect on the posterior distribution — it will also say that smoking is healthy. On the top of the problem described above, the word “consistent” has a different meaning in logic. For this reason, many philosophers use the word “coherent” rather than “consistent” when they discuss de Finetti’s theory. To be consistent in the sense of de Finetti is not the same as to be logical. Subjectivist consistency is equivalent, by definition, to acceptance of a set of axioms.

7.7.8 Objective Bayesian methods A field of Bayesian statistics adopted the name of “objective Bayesian methods.” The name of the field is misleading because it suggests that other Bayesians choose their probabilities in a subjective way. In fact, nobody likes subjectivity and all Bayesian statisticians try to choose their probabilities in the most objective way they can.

page 139

April 29, 2016

9:6

140

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch07

Resonance: From Probability to Epistemology and Back

7.7.9 Prior In the subjective philosophy, a prior is a complete probabilistic description of the universe, used before the data are collected. In Bayesian statistics, the same complete description of the universe is split into a “model” and a “prior.”

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

7.7.10 Non-informative prior The phrase is used in the Bayesian context and, therefore, it is associated in the minds of many people with the subjective theory. The term suggests that some (other) priors are informative. The last word is vague but it may be interpreted as “containing some objective information.” In the subjective theory, no objective information about probabilities exists.

page 140

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch08

Chapter 8

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Epistemology

The first part of the book focused on the critique of two dominant philosophies of probability. The remaining part of the book will be constructive. I will present my own vision of the science of probability but before I do that, I will examine some problems in (general) epistemology. A thorough discussion of the concept of probability, especially of the claim that probability is subjective, necessitates a detailed analysis of the general foundations of knowledge creation. The main purpose of the present chapter is to introduce and analyze a new concept of “resonance.” To test it and place it in a suitable context, I will discuss several well known philosophical problems, namely, the problems of induction, consciousness, intelligence and free will.

8.1 The Problem of Induction The simplest version of the problem of induction is concerned with a sequence of identical observations made in the past. Why is it rational to believe that similar events will occur in the future? For example, suppose that we observed that ripe apples fall from the tree to the ground. Why is it rational to expect that ripe apples will fall to the ground in the future? Is it possible that apples will fly to the sky instead? A more general form of the problem of induction asks why it is rational to hold beliefs about the unknown on the basis of the known. For example, scientists currently predict climate change (earlier referred to as global warming). This event is not believed to be identical to any event in the past. The climate did change in the past, for example, the Earth went through an ice age that ended about 12,000 years ago. But none of the past

143

page 143

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

144

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch08

Resonance: From Probability to Epistemology and Back

climate changes occurred concurrently with human industrial activity. So, why do many people believe that the climate will change in the near future? On the philosophical side, the problem of induction stems from the limitations of the deductive logic. This type of logic can only transform a true proposition or a family of true propositions into a new true proposition. Deductive logic is traditionally and widely recognized as the most solid tool that can generate reliable knowledge. One could dismiss the problem of induction as an artificial problem of our own making, created by our irrational attachment to one particular intellectual technique of attaining truth. I do not dismiss the problem of induction because rejecting deductive logic or arbitrarily supplementing it with other methods does not lead to a deeper understanding of the problem. It is possible that no inductive logic will ever succeed for social reasons. Based on the history of the philosophical research of induction so far, I guess that there is little chance that a universal agreement of all philosophers on a single version of inductive logic will ever materialize. The problem of induction is not only a philosophical problem — it is also a practical problem. We use information retrieved from our own memory or obtained from other people. Since some of this information is false and some of it is unreliable, we want to understand how to tell the difference between reliable and unreliable mechanisms of generating knowledge. Ideally, inductive logic should be similar to deductive logic — mechanically verifiable and hence very reliable.

8.1.1 An ill posed problem I will argue that the traditional problem of induction is ill posed. A brief summary of my argument is the following. The problem of induction is to provide a convincing justification for why we can pass from our information about the known to our beliefs about the unknown. We cannot pass from the known to the unknown because there is nothing that we really know. We “know” facts only because we make a number of assumptions about our universe and our knowledge. These are the same assumptions that we need to adopt to make predictions. These assumptions cannot be proved but most people are comfortable making them, consciously or subconsciously. The assumptions apply equally to the known and to the unknown so the problem of induction is an illusion on the philosophical side. Of course, we have to distinguish between the known and unknown in everyday life and science — I will discuss this point later.

page 144

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch08

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Epistemology

145

The first of the implicit assumptions that we make when we talk about known facts is that our universe is governed by stable laws. This is a reasonable assumption in my opinion because otherwise there is nothing interesting that we can say about anything. But the stability of our universe is nonetheless an assumption, not a fact. There are several ways in which our universe might not be governed by stable laws. It is possible that our universe is governed by a capricious demiurge who may change the laws of nature at any time. So, starting tomorrow, all physics may be wrong. It is also possible that the whole universe was created by the demiurge 5,000 years ago together with humans and dinosaur fossils. The reader will recognize here the actual belief of a large religious community — the belief dismissed by science but impossible to dismiss on purely logical grounds. Modern technology provides inspiration for other scenarios. The film “Matrix” was concerned with an artificial world, very much like ours, existing in the memory of a computer. Scientists tell us that our physical universe extends enormously in space and time and speculate that there are many other universes (see [Tegmark (2014)]). This lead some people to speculate that some matter might have assembled randomly somewhere in the universe and formed a mind with the thoughts that one of us (the reader of this book?) experiences. Needless to say, these thoughts do not have to have anything in common with the existing universe. The common aspect of all these scenarios is that they all allow for the universe to have been completely different from the current universe at any time in the past. It follows that the “fact” that all apples fell to the ground in the past is based on the assumption that the world was stable until now. While the first assumption, discussed above and made implicitly when we assert “facts,” has ontological nature, the second one, to be discussed next, is epistemological. Let us take for granted that there exists objective universe governed by stable laws. Do we have the ability to observe the universe and correctly recognize facts and laws? There are many practical situations when we fail to do so. They include intoxication with alcohol or drugs, tiredness and some mental diseases. We believe that our dreams do not represent reality. The unreliability of our observations is dramatically illustrated by contradictory testimonies in courts. Large religious communities of people disagree about facts from lives of their founding figures and more recent miracles. The famous experiment in which people did not notice a gorilla in the middle of a basketball game (see “Invisible Gorilla Test” in [Wikipedia (2014k)]) is yet another case of

page 145

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

146

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch08

Resonance: From Probability to Epistemology and Back

disconnection between reality and our “knowledge.” Going back to falling apples, their downward motion is a “fact” only because we assume that we can trust our senses and information processing in our mind. To apply induction and make a prediction we have to assume that the universe is stable and that we can correctly recognize the relevant facts and laws. These are the same assumptions that we make when we talk about “known facts.” Logic cannot justify induction because induction does not work in some universes that we can imagine without violating the rules of logic. One could wonder whether the asymmetry between the known and the unknown is a matter of degree. Is it true that predictions require a stronger form of the two assumptions than what we need to recognize past facts? To put it in a different way, can we be more certain of the past facts than of predictions, even if we accept that both are based on unverifiable assumptions? I believe that we typically derive our certainty from the web of various facts and laws which support our belief either in a past fact or in a prediction of a future event. On the practical side, we are typically aware of the past facts that are well confirmed in multiple ways. Our attention is often drawn to those future events whose outcome cannot be predicted well using known facts and theories. This distinction between the past and the future is not universal, though. I am much more confident that the Sun will rise tomorrow than that Shakespeare wrote the dramas attributed to him (see [Wikipedia (2015g)]). The distinction between the known and unknown is a matter of the strength of the connection of an event to other events and laws of nature.

8.1.1.1 On intuitively obvious propositions The reader may be surprised by my claim that our belief in facts is based on the exactly same set of implicit assumptions that also underlie our predictions and so facts are not any more “solid” than predictions concerning the future. This reminds me of a somewhat amusing situation in mathematics. There exist three mathematical axioms that are logically equivalent. One of them, the axiom of choice, is usually considered to be obviously true. The second of these axioms, the Kuratowski–Zorn Lemma does not seem to be obviously true and neither it seems to be obviously false. Finally, the third equivalent axiom, the well-ordering theorem, seems to be obviously false (see [Wikipedia (2015c, 2015d, 2015e)]). These intuitive feelings about the logical values of the axioms are widespread among

page 146

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch08

Epistemology

147

mathematicians. Hence, I do not find it surprising that people (including me) consider past facts to be solid while predictions of many future events appear to us to be speculative at best, despite the fact that our knowledge of the future rests on the same implicit assumptions as our knowledge of the past.

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

8.1.2 Induction is a law of nature Declaring the problem of induction to be ill posed and leaving it at that would be hardly a satisfying answer to the puzzle. I will propose a new formulation of the problem and a solution to the new version of the challenge. The starting point of my analysis is the observation that induction does not work. More precisely, sometimes it works and sometimes it does not. There are numerous examples of predictions that failed. Some of the best known examples are weather forecasts. People like to make predictions concerning functioning of the society, including wars, political campaigns and economy. So many of these predictions fail that they often are not taken seriously. Induction failed a number of times when people tried to understand laws of nature. Many people believed that the Sun went around the Earth, that the Earth was flat and that heavier objects fell faster than light objects. On the scientific side, the theories of phlogiston and ether proved to be false, as far as we can tell. The standard version of the problem of induction asks for an explanation of why induction works. This is a misleading question. A much better question is why induction works in so many cases that it is an effective tool in science and everyday life. One can also ask whether all possible universes are capable of supporting sentient beings that can develop science. Observations of our own universe on very small scales and very large scales suggest the negative answer. There are many physical systems that are not sufficiently complex to support sentient beings using induction. So, induction is a law of nature in our universe. I do not know a good reason why induction would have to work in all possible universes. I will base my analysis of induction on the concept of “resonance,” to be introduced in Sec. 8.2. My proposed “solution” of the induction problem is no more than a reduction. Let us recall a well known case of reduction to illustrate the concept. In antiquity, people believed that celestial objects moved along circles (with the annoying exception of planets). One of the explanations was that celestial objects were perfect and the most

page 147

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

148

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch08

Resonance: From Probability to Epistemology and Back

perfect shape for a trajectory was a circle. Newton explained the circular trajectories in terms of gravitation. Newton’s idea was only a reduction or push of the philosophical problem to a lower level because existence of the universal gravitation had to be assumed as an unexplained fact. This does not mean that that reduction was useless — quite the opposite, the theory of gravitation was later used to interpret astronomical observations, predict existence and positions of new planets and design and launch satellites. Science seems to be an activity always looking for a reduction of the current set of unexplained assumptions and phenomena to a more fundamental level. Newton’s theory was followed by a deeper analysis by Einstein, in his relativity theory. Currently physicists try to probe nature even deeper; one of their attempts is the string theory. I hope that my theory of resonance, despite being “only” a reduction, could explain some phenomena, make new predictions and inspire technology, for example, computers and other devices with “artificial intelligence” capabilities. But even if the theory of resonance does not achieve any of these goals, it will serve as an alternative look at the problem of induction. I think that the analysis of induction as a law of nature generates answers that are more interesting than those coming from attempts to create an inductive logic. The philosophical analysis of probability can be regarded as a specialized part of the analysis of induction.

8.1.3 Anthropic principle The anthropic principle is presented in this way by [Wikipedia (2015b)], The anthropic principle [...] is the philosophical consideration that observations of the physical Universe must be compatible with the conscious and sapient life that observes it. Some proponents of the anthropic principle [...] believe it is unremarkable that the universe’s fundamental constants happen to fall within the narrow range thought to be compatible with life.

I believe in the following instance of the anthropic principle: “We necessarily live in a universe where induction works.” Nelson Goodman introduced the idea of “grue” (see [Wikipedia (2015a)]). An object is grue if it is green until January 1, 2050, and it is blue at all later times. The philosophical problem with grue is that all available evidence supports the claim that grass is grue as much as the claim that grass is (permanently) green. So, why should we think that grass is green? My answer is that the natural laws of our universe made us think

page 148

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch08

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Epistemology

149

that grass is green. So far, we are doing quite well applying induction as formed in our minds by the evolution. Some scientists (see [Tegmark (2014)]) believe that there are many universes, including some with completely different laws of nature from our own. There is absolutely no way that one can prove that grass is not grue because there exist universes, real or imagined, where grass is indeed grue. The incredible effectiveness and reliability of induction is partly an illusion. We celebrate all cases where induction works well and we seem to underestimate the extent of failures in the philosophical discussions. Induction fails, for example, in weather prediction. The reason for this failure is not (only) that scientists have not developed reliable methods for weather prediction so far. The failure is due to a special mathematical character of some systems that was given a technical name of “chaos.” In particular, weather is “chaotic.” Chaos can be considered a law of nature — to save induction we call a case of non-induction a law of nature. Once we move away from science then it is hard to find stable laws, even of the chaotic sort. And yet nobody complains that induction does not work in our world.

8.2 Resonance The process of knowledge acquisition consists of two main parts. The second part was the traditional focus of epistemology so I will give this part of the process the label “logical.” The existence and preeminence of the first part of the process is the main new claim of this book. I will refer to this part of the epistemological process as “resonance” because I believe that it has some common elements with the eponymous natural phenomenon. Roughly speaking, resonance is a crude first filter in the process of acquisition of knowledge. The logical processing is the much finer second filter.

8.2.1 Information and knowledge I have used the words “information” and “knowledge” in an informal way and I will continue to do so in the rest of the book. I will give a special meaning to information and knowledge in this section so that these concepts help us understand the phenomenon of resonance. Every physical system is acted upon by a large number of diverse forces. Examples include gravitation and electromagnetic waves. It is customary to give labels to many of these forces even if in principle they are examples of or can be reduced to other physical forces. For example, we speak about

page 149

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

150

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch08

Resonance: From Probability to Epistemology and Back

friction, air resistance, and visible light. The totality of forces acting on a system can be called “information.” Knowledge is an ability of a system to process information in a way that is beneficial to it. The “beneficial” aspect of knowledge is actually quite subtle. When it comes to primitive animals, “beneficial” should be understood in the most direct way. But the beneficial aspect of knowledge can apply to the whole species or it can be interpreted as improvement of DNA survival probability. It is even more subtle when it comes to humans, where knowledge can be used for purposes that are destructive or evil. Then knowledge is beneficial to the goal set by a person. The knowledge possessed by primitive organisms (animals and plants) is beneficial in the strict sense of the word because these organisms developed the simplest knowledge of the world via Darwinian evolution and selection. In this context, knowledge is the ability of the primitive organisms to react to the simplest chemical, light and gravitational stimuli in a way that increases their chances of survival and reproduction. The functioning of the most primitive organisms is the simplest example of resonance. Simple organisms developed senses able to react to only some of the forces acting upon them. Resonance is the ability to filter only useful information from the enormous amount of the physical stimuli acting on a system.

8.2.2 Resonance complexity I believe that a wide spectrum of phenomena can be reasonably classified as instances of the same general process of resonance. To put some order within this family of phenomena, I will propose a classification of resonance types based on the complexity of the stimulus and the action following it. At the lower end, resonance detects a simple signal from the environment. Consider, for example, a primitive living organism that can react to light and move appropriately. An equally simple example of resonance is provided by (man made) lights which are automatically turned on at night by a photosensitive switch. The stimulus can be simple even though the reaction can be very complex, for example, a single sensor measuring the depth of water may trigger a major response to the danger of flooding. Even when the signal is simple, the process that leads from the stimulus to the reaction may be very complicated. The route from a light stimulus to the motion of an organism, no matter how primitive, is a complex chain of biochemical reactions.

page 150

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch08

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Epistemology

151

I am not sure how many clearly different levels of resonance complexity one can recognize. I will indicate a few levels that seem to be distinct. Animals react to danger. Danger can take different forms. It includes, depending on the animal and circumstances, cliffs, fire, and predators. Recognizing each of these dangers requires more than the detection of a single type of stimulus. At the next level of complexity of resonance we find reactions to social situations. Animals are capable of recognizing complex social situations in their groups and reacting in an appropriate way. Social behavior requires an ability to detect signals with complex structure and process them in a sophisticated way. Needless to say, humans are capable of reacting to social situations and we believe that our level of sophistication in this area is much higher than that attained by any other species. The highest level of resonance complexity has been attained only by humans. In this case, resonance is the reaction to and processing of very diverse and large body of information about the world. It involves sophisticated concepts such as marriage, democracy, war, God, and climate. To do science, people have to recognize facts and construct theories, known in statistics as models. They have to select a small number of viable theories (models) because the number of theories is infinite and one cannot analyze them all. The ability to select viable theories is what makes people much more intelligent than computers at this time. People have a picture of reality in their minds and can generate models that are likely to be true within this picture. The best scientists are capable of proposing theories that go well beyond the ordinary science but still have a good chance of being successful. Generating scientific theories (for example, statistical models) seems to be the highest form of resonance because of its complexity.

8.2.3 Facts Our ability to recognize “facts” is the most elementary but fundamental example of human resonance. Facts do not exist in isolation. A fact is a detected pattern. To say that a table is white is to say that an object is like other objects which we recognize as tables (that is, we recognize a symmetry in the universe) and it is also like other objects which are white (another symmetry). The representation of a fact as a pattern is especially significant in the social context. We do not know whether other people perceive tables as tables

page 151

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

152

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch08

Resonance: From Probability to Epistemology and Back

and white as white but we can match our patterns of behavior (including speech) with those of other people and thus we can agree on facts. A fact is a distilled piece of information from an ocean of sensations, matched with memories in the process of resonance. Hence, the ability to determine and communicate facts is non-trivial and is subject to learning and failure. It is quite often the case that failures help to illustrate the success. Many machines are equipped with sensors and can perceive light and temperature and detect motion. All such equipment ages and fails at some point. People “fail” in similar ways. Some people are blind or deaf and this limits their perception but they can still use other senses to greatly make up for these deficiencies. A more subtle and more instructive is the case of people with autism. They fail to notice some social facts and cues — these are fundamental to the functioning in the society. Facts in our minds are not necessarily objective facts. Resonance is a part of information processing that takes input and generates output. The evolution made us, animals, good information processors in the sense that we can individually survive and multiply as species. This success does not prove that facts in our minds match facts in the objective universe. Quite the opposite, there are good indications that our image of the universe may be incomplete or even distorted. Physicists consider quantum physics and relativity theory to be very solid scientific theories. Yet our intuitive understanding of facts is considerably different from “facts” in these theories. On the top of that, physics is ripe with more speculative theories which talk about higher dimensions, multiple universes, etc. Our perception of and belief in “facts” developed for the benefit of entities of our size and need not be universal. There might be sentient beings on other planets processing information in a different way. Just like religion evolved to help the survival of individuals and communities despite its distortion of facts (see Sec. 8.6), our fact perception might have evolved to help survival of large communities of cells (human bodies). None of this means that we should abandon our belief in facts. Our civilization rests on the foundation of facts and nothing can change that. No matter how artificial our mental facts are, we have to rely on them. The other option is skepticism, with little theoretical or practical appeal. The fundamental mistake of de Finetti and some positivists was that they thought that scientists could recognize facts and only facts. There exists a view of science according to which science collects facts and then

page 152

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch08

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Epistemology

153

arranges them into theories. De Finetti thought that observing facts was sufficient to do probability. He failed to recognize the fundamental role of the human ability to recognize relationships. In probability, symmetry and independence are fundamental relationships. Actually, these two relationships are fundamental to all of science. Other relationships are also used but their position is less fundamental than those of symmetry and independence. De Finetti implicitly accepted our ability to recognize one type of relationship — disjointness of some events. This is what you need and all that you need to match Kolmogorov’s axioms with reality. Von Mises’ theory of collectives was based on our ability to recognize a specific relationship — the relationship of identity between elements of a collective. Von Mises missed some other relationships by restricting his concept of probability only to collectives.

8.2.4 Learning resonance The most primitive animals cannot learn much at the level of an individual. Their process of learning consists of mutation and selection at the level of species. The consecutive generations respond better and better to the environment due to this process. The higher we go on the ladder of life, the more developed is the process of learning at the individual level. It seems that the ability to learn at the individual level is one of the main reasons for the crushing success of the human population relative to other animal populations. A well known proverb says “Give a man a fish and you feed him for a day; teach a man to fish and you feed him for a lifetime.” The evolution made a big step forward when it equipped humans with an excellent ability to learn at the individual level, hugely exceeding the learning abilities of even most advanced apes. Humans developed a whole system of learning for individuals based on schools and universities but much, perhaps most, of the individual learning is totally informal. A part of of the learning process, both in animals and in humans, is learning resonance. This consists of moving learned skills from the conscious level to the subconscious level. [Polanyi (1958), p. 49], points out that human ability to ride a bicycle is mysterious. Although this ability can be described in abstract scientific terms and, to certain degree, it can be explained to people learning how to ride a bicycle, the learning process and the ability are mostly unavailable to scientific analysis at this point. The cyclist’s ability to detect changes in the position and velocity of the

page 153

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

154

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch08

Resonance: From Probability to Epistemology and Back

bicycle, a prelude to taking corrective turns of the handlebar, is an example of learned resonance. The part of the process of learning of resonance most relevant to my analysis of probability belongs to the formal education system. Students of all ages listen to theoretical explanations and practice related skills by doing numerous examples. Both [Kuhn (1970)] and [Polanyi (1958)] point out that the learning process is really based on practicing a skill by doing many examples. Understanding of theoretical principles underlying all the examples comes later. The structure of most traditional and current textbooks is the opposite — the presentation of a topic starts with the theory and followed by examples. A successful learning process turns even complex theoretical concepts and advanced skills into instances of resonance. Students who mastered the material do not have to “think” about how to apply the theory which they have learned. The practice establishes the corresponding skills in the subconscious. From now on, students apply the learned skills and ideas according to the principles of resonance — instantaneously and subconsciously. The ability to read is a good basic example. Reading a text is not processing of individual letters but processing of whole words or even sentences. The learning process builds new resonance abilities on the basis of already established abilities. Let us have a look at an example of such a hierarchy in learning mathematics. Students first learn fractions. This skill is used to learn differentiation and integration. These skills in turn are the basis for learning how to solve differential equations. People are trained to recognize very complex facts because it is virtually impossible to pass from simple facts to successful decision strategies either in science or in everyday life. Scientists and other professionals must recognize, for example, sulfur, lasers, supernova explosions, and cancer. Kuhn’s “scientific revolutions” (see [Kuhn (1970)]) are, among other things, changes in resonance abilities. Scientists learn resonance according to a theory and resist the change for practical reasons — they find it hard or impossible to apply resonance according to a new theory. In more elementary language, they lack intuition needed for applications of the new theory. This may be one of the reasons for the resistance to change and the need for revolutions to replace old theories with better ones. Kuhn recognizes the important role of “normal” science but he seems to underestimate its training role. Learning science is far from learning facts

page 154

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch08

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Epistemology

155

from textbooks. Scientists have to develop resonance abilities at the highest level. The only practical way of learning resonance at this level is to conduct “normal” research. The scientific laws of probability (L1)–(L6) (see Sec. 1.3.3) are the starting point for teaching of probability. At this level, most examples in probability textbooks are combinatorial in nature. They often deal with highly symmetric objects and situations, such as coins, dice, playing cards, etc. The remaining probabilistic material is taught in the hierarchical style mentioned before. For example, students learn at some point about the normal distribution. Then they learn about Gaussian processes. This can be followed by the theory of stochastic partial differential equations (SPDEs) with Gaussian solutions. Each of these topics relies on the previous one. Some philosophical theories of science (see Sec. 10.12) emphasize positive reinforcement — this is how one can interpret Inductivism. Popper’s type of Falsificationism is rather rigid in that it demands rejection of theories disproved by experiments. The Bayesian method advocates a reasonable attitude by letting the data (and the Bayes theorem) to decide which way to go. I find all of these descriptions of the real learning process oversimplified. We try to learn new theories and skills. Of course, successes reinforce our beliefs and skills while failures push us in the opposite direction. But quite often what actually happens is neither reinforcement nor abandonment of a theory. Instead we learn how to modify our approach by reducing the scope of the applicability of a theory. In other cases we broaden the scope and in some others we modify both assumptions and conclusions. I doubt that the Bayesian method is a good description of this process because the Bayes theorem presupposes a prior set of candidates for a new theory. These are typically unknown in every conceivable sense, theoretical and practical. The theory of resonance suggests a mechanism for the learning process. We use our innate skills (resonance) to move from one theory to another when we have access to new empirical data. Identifying this process with Bayesian learning is a scientific conjecture favored by some researchers. I do not think that we have sufficient evidence in support of this conjecture.

8.2.5 Resonance level reduction Resonance level can be reduced in practice. In statistical context, if someone cannot recognize that observations come from a normal distribution at the intuitive level, there are methods to check if the distribution is in fact

page 155

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

156

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch08

Resonance: From Probability to Epistemology and Back

normal. This process requires some data and some theory so it is applied only in those situations where it is really needed. But resonance level cannot be reduced in some other cases. A good practical example of irreducible resonance are sequences called “independent identically distributed” (i.i.d.) by some statisticians and “exchangeable” by others. One has to recognize a symmetry in this case — further reduction of resonance level is not possible. Statisticians have methods that can be used to determine whether a given sequence is i.i.d. or exchangeable but these methods are based on the ability to recognize some other symmetries (i.e., facts and models). This illustrates the possibility of moving the target of resonance within the same level of complexity to achieve a practical goal. A standard method to reduce the resonance level, both in everyday life and scientific practice, is to use a measuring instrument such as a thermometer or scale. These instruments act as resonance “antennas” in that they filter the relevant information from the environment. Then they convert it to the form that is easily recognizable by humans using a lower level or more reliable form of resonance. A measuring instrument must be based on a theory — a set of assumptions about how the world works.

8.2.6 Properties of resonance Examples of resonance given in previous sections should have helped the reader understand the concept. I will now list the essential attributes of resonance to clarify its meaning. (i) Resonance is a crude filter of information. (ii) Resonance is the first step in the knowledge creation process. It comes before logical analysis (if there is any). (iii) Low level resonance is instantaneous. It is the fastest of all information processing steps. High level resonance (for example, some resonance involved in scientific research) need not be fast — see Sec. 8.4. (iv) Resonance is atomic. It is not a process consisting of steps. (v) Resonance resembles a “finite dimensional projection” in mathematical terminology. All forces acting on a system, no matter how small the system is, can be reasonably represented as an infinite dimensional family of stimuli. Resonance cuts dimensionality to one or a few dimensions. Most of the potentially available information is discarded.

page 156

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch08

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Epistemology

157

(vi) Resonance, especially at the higher levels of complexity, detects information that is not normally considered to be elementary in science, in terms of the fundamental forces of nature. Quite the opposite, at the highest level of resonance complexity, humans instantaneously detect anger, happiness and aggression. In the statistical language, this resembles the detection of “principal components” which are combinations of simpler quantities (stimuli). (vii) Resonance is not conscious. People are aware of the results of resonance and can be aware of the resonance taking place. For example, they may notice their own detection of someone’s anger and, obviously, they are aware of their own reaction to someone’s anger. But people have no insight into the nature of their own resonance and they cannot analyze this part of their information processing. This does not mean that people cannot correct the results of resonance — logical and scientific analysis is the second, more accurate, step of information processing. People are also capable of manipulating their own resonance. For example, some people make a conscious effort to eliminate their racial prejudices and reactions. (viii) The results of resonance, especially at the high level of complexity, come with a feeling of their (high or low) reliability. This is in contrast with the lowest level resonance which is either acted upon or not. The feeling of degree of reliability is what many people would call “subjective probability.” I object to this misnomer because the feeling of reliability is informal but not arbitrary. The informal assessment of reliability is a crucial element of intelligent information processing. The results of resonance that are considered somewhat unreliable can be subject to logical processing. They can also inspire efforts to collect more relevant information and thus generate a more reliable assessment of reality.

8.2.7 Physical basis of resonance I do not know the physical basis of epistemological resonance. It is possible that scientists working on human brain have already determined the physical basis of resonance or will determine it in the near future. This book is devoted to the philosophical aspects of resonance, not its physical basis, so I do not feel that I have to do more than to present my guess concerning a physical process that could be the basis of epistemological resonance. It will not be much of a surprise for anyone that I have chosen

page 157

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

158

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch08

Resonance: From Probability to Epistemology and Back

the label “resonance” for the epistemological process because I believe that it might be related to a physical phenomenon called “resonance.” Resonance occurs when a system reacts to a periodic force. For resonance to occur, the period of the force must belong to a restricted family of numbers characterizing the system. Radio receivers are designed to pick up the signal of only one frequency at a time. All other frequencies in the electromagnetic wave spectrum are ignored. Undesirable resonance occurred in some faulty engineering designs, for example, some bridges and airplanes. Some people can break a wine glass by singing a note that perfectly matches the fundamental frequency of glass vibrations. On the mathematical side, physical resonance could be represented as filtering of all eigenfunctions except the first one (or perhaps the top few eigenfunctions). In other words, resonance is a form of mathematical projection. One could speculate that the epistemological resonance is, at least sometimes, based on a similar projection on an eigenfunction subspace in some high dimensional or complicated space. I apologize to readers without scientific background for the technical jargon used in this paragraph. I can assure you that this is no more than speculation — I have no solid evidence linking epistemological resonance to these mathematical concepts. Our low level resonance abilities are limited to events which are manifestations of physics laws describing objects at our magnitude. We can access other levels of physical universe, for example quantum and relativity phenomena, via experiments involving resonance (observations) at our level. It is an interesting question whether other sentient beings could observe facts represented by physics theories for very small or very large objects in a direct way using some other resonance capabilities. I guess that this could be the case if not in ours then perhaps in other physical universes. There has been much talk about and some progress on quantum computing in recent years. The advantage of quantum computing over digital computing is that some calculations are done in parallel, in an infinite dimensional space, in a sense. Resonance has a similar advantage over logical processing of data. In the resonance process, a single mode can be filtered in a single step from data that are infinite dimensional. In other words, we can instantly and intuitively judge which data (aspects of reality) are irrelevant. A less sophisticated way to speed up computations is parallel computing. I do not think that parallel computing is a good analogy for

page 158

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch08

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Epistemology

159

resonance, in contrast to quantum computing. Parallel computing speeds up the computing time by simply assigning the same type of task to multiple processors. Only one candidate for the physical basis of higher levels of resonance in intelligent animals and humans comes to my mind — neural networks. The currently available man-made neural networks are very impressive if you start with very low expectations concerning performance of “mechanical” systems. They are hardly impressive if you compare their performance to that of a typical animal. Yet they seem to be the best hope for artificial intelligence, necessarily including high level resonance.

8.2.8 Information and knowledge revisited If induction is a law of nature, one would expect that information and knowledge are scientific concepts, that is, we should be able to represent them as elements or manifestations of physical systems. I do not think that a detailed analysis of information and knowledge is needed for or would contribute much to the understanding of the resonance theory but I will offer some tentative ideas on this matter. My intention is to give the concept of resonance a very broad meaning so I will propose interpretations of information and knowledge with equally broad meanings. Note that the definitions of information and knowledge proposed in this section are not the same as in Sec. 8.2.1. I am just trying to present some food for thought. Information is the ability of a subsystem of one system to hold an isomorphic image of a subsystem of some other system. The isomorphism has to be understood in a very broad sense so that it covers both human memory and computer memory. These two examples show that the phenomenon of information arises when an agent can access an isomorphic image of a system (held in, for example, computer memory) instead of observing directly the “source” system (say, wildflowers). The mere isomorphism of parts of two systems does not seem to be significant in itself. An agent who can benefit from the isomorphism in some way gives the meaning to the concept of information. The last condition is needed to eliminate coincidental and meaningless similarities, such as roundness of oranges and the Moon. We can consider knowledge to be the ability of one system to respond to the state of another system in a reproducible way. This is a very broad definition because it covers, among other things, ocean tides reacting to the changing position of the Moon. So once again one needs more structure to

page 159

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

160

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch08

Resonance: From Probability to Epistemology and Back

make knowledge an interesting concept. I believe that knowledge becomes interesting only in the context of living animals or other complex structures, such as robots. Knowledge shows its potential when it is applied to improve the chances of the system to achieve a goal. Individual animals try to prolong their lives and so do species. Their ability to use knowledge is the result of the evolution. It is often hard to find any direct benefit of what we consider human knowledge. Human knowledge can be described as the resonance process ran amok. The evolution process created human mind and favored individuals with abilities increasing survival probability. The ability to gain knowledge was a greatly practical skill at the earlier stages of the evolution of humans but now it partly lost its direct relevance to survival.

8.2.9 Resonance and subjectivity Resonance is suspect but it is not subjective or irrational. My main source of motivation for developing an epistemology and especially the theory of resonance is my desire to understand the subjective component of the concept of probability. I believe that the main reason why people call probability subjective is a rather simple case of confusion (see Sec. 5.1). But one cannot dismiss summarily all claims about subjectivity of probability made by people who devoted much thought to the problem. I came up with the concept of resonance because this is the only way, in my opinion, in which the claims about subjectivity of probability can be made at least a little bit rational. I think that calling resonance subjective would do more harm than good. I strongly believe that the word “subjective” has connotations of freedom. In my opinion, neither Mona Lisa was beautiful nor the painting of her is beautiful. I do not like honey, beans and whiskey. I like red wine but I do not like white wine. I like jazz and opera but I do not like country and western music. All of the above are my subjective opinions. I identify the subjectivity of these opinions with freedom because I feel that I can hold these beliefs no matter what other people think. It is unlikely that I will suffer any consequences by expressing these beliefs. I can change these beliefs at will and without justification (in fact, I changed some of them at some point in my life). Resonance does not resemble or involve freedom in any conceivable sense. Resonance refers to the fact that a part of our mind (or that of more primitive animals or machines) acts as a black box. It accepts some input

page 160

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch08

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Epistemology

161

and it generates some output but we cannot examine the inner workings of the black box in real time. The black box may be very accurate or not but it is not arbitrary. We mostly care about those cases of resonance that seem to be very accurate. From the practical point of view, people can recognize basic facts with great accuracy despite all types and cases of failure. Modern computers act like black boxes for most people because most people know very little about the details of computers’ hardware and software. Yet nobody would apply the word “subjective” to the information processing done by computers. I think that the word “suspect” characterizes the resonance phenomenon better than the word “subjective.” Since we do not have easy access (or we have none) to the process of resonance, it is automatically suspect, on the purely intellectual side. We can only develop some degree of trust in our resonance ability by observing many successful applications of resonance. Of course, this is somewhat circular, because “observing success” requires an ability to recognize facts and this in turn requires resonance. But one can say that resonance fits well into a certain logically consistent theory of the universe.

8.2.10 Raw resonance Resonance can be wrong. This is not necessarily bad news — let me start with a few positive examples. Color television and computer displays are based on the successful simulation of colors; this is a polite way of saying that our senses are fooled. Color TV and computer monitors display only mixtures of three colors: red, green and blue. We have an illusion of seeing all possible pure colors in the spectrum and all their possible mixtures. Actually, some people can easily tell the difference between the colors on a computer screen and the colors of the real object but this ability varies from person to person. Not only TV and computers are based on the same mental inability to distinguish precisely between various mixtures of primary colors. This phenomenon is behind the successes of color photography, color print and even the venerable art of painting. Note that painters have only a finite number of dyes to choose from. If we pursue this direction of thought, we realize that the success of the classical painting is based, in part, on our ability to “see” three dimensional objects in two-dimensional plane. Likewise, the film is based on our inability to separate individual frames into a discrete sequence — we see a continuous motion on the screen.

page 161

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

162

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch08

Resonance: From Probability to Epistemology and Back

Next we go to the dark side. According to [Wikipedia (2015f)], “Approximate statistics on the number of trials for witchcraft and executions in various regions of Europe in the period 1450–1750” give a lower estimate for the number of executions of witches at 35,000. About 75 to 80% of those were women. Witch-hunt was a complex social phenomenon with multiple roots, including economic, political and religious causes. But, at a certain level, every individual case involved resonance — the ability to recognize a witch in a more or less direct way from the available empirical evidence (including gossip, etc.). I do not believe that all people involved in witch-hunt (on the side opposite to the “witches”) were evil or had hidden motives. I think that many of them genuinely believed in their ability to recognize witchcraft and witches. There were innumerable cases of false beliefs of various caliber. I have chosen witch-hunt as my illustration because of its drastic character and clear, from the modern perspective, lack of understanding of the laws of nature. I am sure that the scourges of racism, antisemitism, discrimination of women, etc., involve some dose of mistaken resonance. The evolution created a human propensity to recognize “facts” via resonance in many areas of life. In some of the areas, the “facts” are almost never confirmed by the scientific process of empirical verification. Religion comes first to my mind as an example of this tendency but it is definitely not the only example. Political ideologies enjoy significant loyalty. Many people have a tendency to follow and believe strong leaders. If resonance is misleading in so many cases, how do we know that it is not equally misleading in scientific research? Actually, resonance is misleading in science. Our normal instinctive picture of the physical world agrees with Newtonian physics. This is a false belief according to quantum mechanics and theory of relativity. Newton’s physics is the most effective way of interpreting the world for human-sized organisms. Resonance does not necessarily lead to the truth about the world. Most of the time (we hope) it leads to the most beneficial, from the point of evolution, interpretation of the world.

8.2.11 Resonance and philosophy of Hume The name most associated with the problem of induction is that of David Hume. He was skeptical about the possibility of justifying induction but he

page 162

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch08

Epistemology

163

did not suggest that we should abandon it. I quote after [Galavotti (2005), p. 32]:

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Wherever the repetition of any particular act or operation produces a propensity to renew the same act or operation, without being impelled by any reasoning or process of the understanding; we always say, that this propensity is the effect of custom. By employing that words [sic] we pretend not to have given the ultimate reason of such a propensity. We only point out a principle of human nature.

My epistemology goes one step beyond Hume’s. I combine Hume’s observations with the theory of evolution, which was not available until about a century after his death. I propose to use the evolution as an explanation for the observed success of the induction in the past. This understanding of the past successes of induction as a law of nature, going beyond Hume’s “custom,” may inspire confidence in our innate abilities to successfully implement induction in the future. I have to emphasize that “inspire confidence” is as far as I am willing to go because nothing can be proved beyond any doubt — see Sec. 3.2. In the simplest situations, induction is based on observations of repeated events, and hence on resonance. In other words, the simplest case of induction is the simplest level of resonance and can be described as a perception of symmetries in the world.

8.2.12 Is resonance a new concept? Is resonance just a new name for an already well established concept? There are several candidates for a precursor of resonance: subconsciousness, filtering, pattern recognition and instinct. One could say that resonance is subconscious information processing and logical reasoning is conscious information processing. This identification is a useful rough idea but it is not totally accurate. I do not know the full extent of subconscious information processing (and it appears that nobody knows at this time) but subconsciousness is certainly not limited to resonance, that is, a specific and rather narrow in scope process of filtering of relevant information. Similarly, conscious information processing does not have to be limited to what we call logic. The epistemological resonance (that is, the resonance in my sense of the word) is closer to the physical resonance than to signal filtering — this justifies my choice of terminology. Filtering signal from noise presupposes

page 163

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

164

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch08

Resonance: From Probability to Epistemology and Back

the existence of signal and noise. At the highest level, resonance includes the human ability to discern patterns in the available information. This applies equally to witty remarks at a party and new proposals for historical theories. Although the posterior analysis of such cases of resonance could possibly represent them as signal filtering, I doubt that this does the justice to the idea of resonance. We can classify components of an information stream as signal and noise either if we understand the source of the information and noise, or as a result of the a posteriori analysis. I do not think that we can identify resonance with pattern recognition. It is conceivable that all resonance can be represented in some abstract way as pattern recognition but I doubt that it is useful to think about resonance this way. Some forms of resonance are much more complex than what we normally call pattern recognition. Instinct is an instance of resonance but the concept of resonance is broader than the concept of instinct. We typically talk about instinct only when it is innate, that is, it is not acquired via learning. I do not limit resonance in this way because the origin of resonance abilities does not necessarily affect resonance effectiveness. I think that it is best to stretch the concept of resonance well beyond the domain of behavior typically referred to as instinctive. Thus I include the simplest reflexes of the most primitive organisms and the most impressive examples of human intelligence into the concept of resonance. The uniting element of all of these examples is the lack of logical or complex preprocessing of information and the highly selective and successful filtering of stimuli.

8.3 Consciousness I recommend a book by [Revonsuo (2010)] as an excellent review of philosophical and scientific theories of consciousness and related research on the brain. Consciousness is a very complicated phenomenon. I will limit my discussion to its very narrow aspect — its role in information processing. The idea of subconsciousness is connected with the name of Sigmund Freud who developed psychoanalysis at the end of the 19th century. While Freud’s theory was very broad, I am focused on a very narrow aspect of epistemology. For my analysis, the crucial aspect of consciousness is the ability of a conscious being to remember its own information processing. In addition, the information about one’s own reasoning stored in the memory can be

page 164

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch08

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Epistemology

165

easily and quickly recalled and communicated to other people or to the person himself (in the process of reflective thinking). One of the features of consciousness is that a person remembers not only a piece of information but also remembers that it remembered the information. This extra bit is useful, for example, if one wants to recall that a certain piece of information could have been communicated to other people or could have been analyzed by the person himself. Our ability to remember our own conscious thoughts is the fundament of scientific objectivity (or our illusion thereof). Since our conscious reasoning is memorized, it can be repeated many times and the results of all iterations can be checked for agreement. Moreover, our conscious reasoning can be described to other people who can then repeat the same reasoning and compare the results. Even more significant is our ability to analyze our memorized reasoning. We can recall our logical process and analyze it step by step. The analysis can be done by the owner of the memories or it can be done by the community with whom the memories are shared. The two actions that we can apply to memorized reasoning, repetition and analysis, are the best (and only, I believe) methods of quality control of our personal information processing. The utility of consciousness stems in part from its selective nature. We are not conscious of all our sensory data. Moreover, we seem to be aware of (and add to our memories) mostly high level transformations of or extracts from the raw data; for example, we see a robbery — we do not see colorful shapes moving at a distance that represent the robbery. Here consciousness cooperates with resonance. Resonance acts as a filter and data compression device which allows us to see (only) a robbery within a huge amount of data. Remembering all the raw data would be very hard and processing them at a later time would be very time consuming. Human consciousness combined with imagination and memory gives us the ability to make mental experiments, for example, to consider various vacation scenarios. The value of this ability is hard to overestimate but I do not think that it is directly related to resonance. Consciousness seems to be beneficial for many reasons. It seems quite obvious that the process of evolution, especially selection, reinforced consciousness in humans because of its beneficial effects for both individuals and groups of people. An individual cannot observe or analyze his own process of resonance via conscious reasoning. We can repeat observations based on resonance, for example, we can look at the same flower many times to make sure that

page 165

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

166

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch08

Resonance: From Probability to Epistemology and Back

we correctly recognized its color — red or yellow. But our subconscious application of the resonance process is not directly observable so we cannot be sure that we are repeating the same process (although it is reasonable to assume that we do). Even more significant is that we cannot analyze the exact physical mechanism of the resonance process behind our recognition of the color of the flower. Here I do not mean the scientific analysis which I believe will be available sooner or later. What cannot be analyzed is the instance of resonance that occurred in recognition of flower’s color. The hypothetical scientific analysis cannot be performed at a cost and in time that would make it relevant to everyday life. To appreciate my point, compare resonance to logical analysis — we can retrace the steps of a logical argument and pinpoint a mistake in it, if there was one. We cannot pinpoint a mistake in a specific case of resonance, even if we somehow know that there was a mistake. We can compare the results of an instant of resonance with everything else that we know and we can conclude that resonance failed in this particular case but this is not the same as finding a specific error in the resonance process. Is it possible that resonance will be amenable to direct logical analysis just like our conscious reasoning? Lord Kelvin said that “heavier-thanair flying machines are impossible” in 1895 so I should be careful with making impossibility claims. But resonance seems to be impossible to analyze in the following sense. It is likely that for every specific instance of resonance, scientists will be able to design a method of analyzing the process in arbitrary detail. But the analysis will have to be based on other instances of resonance, either in humans or machines or both. I believe that resonance has a character similar to a logical proposition. We can prove every proposition by designing a logic and an axiomatic system in which the proposition can be derived from the axioms. But this is only pushing the problem further back. One cannot prove all propositions, including the axioms, and I believe that we will not be able to circumvent the inaccessibility of resonance to direct analysis.

8.4 Intelligence Just like in the case of consciousness, I will not attempt to describe the phenomenon called “intelligence” in detail. I just want to point out a close relationship between intelligence and resonance. Intelligence is primarily an ability to apply resonance at the highest level. This ability comes at

page 166

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch08

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Epistemology

167

different sublevels and occurs in a variety of situations. I believe that we apply the term “intelligence” to those instances of resonance that stand out in terms of quality and complexity. Animals apply resonance to recognize objects and react accordingly. So do humans. People apply resonance in much more demanding contexts, for example, to recognize quickly social messages conveyed by facial expressions. Examples of intelligent behavior include the ability to recognize investment opportunities. Some people react quickly to slightest hints of social nature. Yet other people may be able to notice natural phenomena not so obvious to others. The kind of intelligence that is most relevant to this book is the ability to construct new scientific theories. Although the story about an apple falling on Newton’s head is not quite accurate (apparently he had a great idea sitting under a tree; there is no evidence that an apple fell on his head), it illustrates well intelligence as a form of resonance. All people could see the Moon moving in the sky and apples falling from trees. But it was only Newton who realized that the two phenomena might have something in common. Einstein likewise did not base his relativity theory on his own new experiments or observations. Many physicists knew the same scientific theories and facts that became the inspiration for Einstein’s theory. Newton and Einstein had antennas in their heads able to resonate at the frequencies of fundamental laws of nature. In this sense they were exceptionally intelligent. The simplest type of induction consists of drawing conclusions or conjectures from repeated observations of the same event. An example of intelligence is to draw a conclusion or a conjecture from the observation of a single event. This could be a coincidence — the simultaneous occurrence of unrelated events, especially if they are both highly improbable. Simple mechanical and biological systems cannot acquire knowledge in this way because an intelligent agent has to have a model of the world in his mind, according to which the two events have small probabilities. Moreover, he has to single out only two relevant aspects of an observation from billions of irrelevant facts. Finally, he must be able to come up with an alternative explanation, linking the two aspects of the observed phenomenon into a single theory which gives reasonably high probability to the observed “coincidence.” People are inclined to see coincidences where there are none. This is typical of superstition — a good example is astrology. Science has

page 167

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

168

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch08

Resonance: From Probability to Epistemology and Back

to weed out false theories based on (real) coincidences. It also needs intelligent (and perhaps lucky) scientists who can notice coincidences and conjecture theories whose reasonably large proportion will turn out to be (approximately) true. Since finding new significant scientific theories is hard and cannot be easily taught or programmed, the society puts a premium on new interesting ideas. The discoverers of new significant theories are rewarded by prestige or money. This creates a problem. Some people create highly speculative theories in the hope that they will prove to be correct and will earn the inventor recognition. This trend is present in modern physics, parts of which have become very speculative. One can see the same trend in an area as distant from science as modern art where artists try to outdo themselves in inventing more and more strange forms of modern art. Most science is “normal” in the sense of [Kuhn (1970)] and not revolutionary like that of Newton and Einstein. Constructing scientific models is sometimes routine and one can imagine that properly programmed contemporary computers could play this scientific role. But such routine science, no matter how much it is needed and appreciated, is not the kind that draws the greatest admiration. Nobel prizes go to people who applied intelligence (resonance) in ways that are well beyond capabilities of most of other scientists. One of the most popular and maligned contemporary expressions, to “think out of the box,” points in the same direction, towards innovation. Having poured praise on geniuses I have to point out that normal science needs normal intelligence and normal resonance. Mainstream scientific theories are not invented as a result of logical reasoning although once they are invented, they are often justified (in a preliminary way) using logical arguments bringing a number of ideas together. I have proposed in Sec. 8.2.6 that resonance is very fast, almost instantaneous. Intelligence is an exception to this rule on some occasions. Simple resonance is unquestionably fast — we react quickly to dangers such as fire, falling objects, etc. But the “eureka moment” might build on a long, most likely subconscious, process leading to the final discovery. A number of scientists reported a period of incubation for their ideas. At this point scientists are not able to observe mental activity in the brain with sufficient accuracy to determine the length and structure of processes leading to intelligent behavior.

page 168

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch08

Epistemology

169

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

8.4.1 Artificial intelligence I find philosophical arguments against the possibility of artificial intelligence incomprehensible. Some people believe that consciousness and intelligence can reside in an object made of meat (a human being) but not in an object made of copper and silicon (a computer). I believe that most people are disappointed with the current state of artificial intelligence for the following two reasons. First, computers have a disappointingly weak consciousness in comparison to their raw computing power. The aspect of consciousness that I refer to is the ability to observe one’s own reasoning, memorize it and later recall it and analyze it. The first three of these tasks seem to be routine and programmable in the current computers. In fact, many operating systems have “log files” recording (some) activity of the computer. Computers are missing the ability to select information for log files. Recording all processes in the computer main processor is impractical because it would overwhelm the memory. Even if all processes could be fully recorded, logical reprocessing of all of this stored information could not be done in real time. Reading the contemporary computer log files does not resemble even remotely conversing with an intelligent agent. To be fair, there are many programs that try to make the interaction with computers user friendly, for both professional and lay users. Second, so far computers do not have an ability to apply resonance to those aspects of reality that we humans care about most. When intelligent humans interact, they quickly asses the expectations of the other person (this is an application of resonance) and react accordingly. Even a poorly trained representative of a “call center” can typically assess the needs of a caller who has an unusual problem (the representative can apply intelligence — a special case of resonance). Explaining the same problem to an automated online service system is impossible. According to [Wikipedia (2014v)], “The Turing test is a test of a machine’s ability to exhibit intelligent behavior equivalent to, or indistinguishable from, that of a human.” Some people are impressed by the results of recent Turing test competitions to the point that they claim that the test has already been passed by a computer. I will agree with this assessment only when call centers routinely employ computers in place of humans. Turing’s test is somewhat misleading. It tests whether a given computer has the ability to communicate with people just like a person. It is natural for us to focus on the human aspects of a conversation such as

page 169

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

170

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch08

Resonance: From Probability to Epistemology and Back

fluency, natural passage from one topic to another, witty observations, etc. Intelligence is the highest form of resonance and one can easily imagine a highly intelligent robot that cannot easily communicate with people. On the other hand, there already exist computer programs that can reasonably well emulate human conversations. The Turing test is in principle unlimited so it can include conversations on topics that would reveal the extent of robot’s intelligence. I feel that this is somewhat concealed by the form of the test. Turing is best known in popular culture for two ideas — one of them is the Turing test. The other one is a theorem saying that all digital computers are equivalent. Some people seem to have subconsciously combined the two ideas and concluded that Turing proved that one day digital computers will be functionally equivalent to people. In principle, one could approximate every physical process by a program running on a digital computer but such brute force approach is still very far from a successful imitation of human reasoning. There is no reason to expect that machines will be limited to digital information processing in the future. The idea of neural networks is an alternative approach (although, paradoxically, it is currently implemented in digital computers). Few people would classify the ability of cars to navigate a very difficult road in a DARPA test (see [Wikipedia (2014w)]) as “artificial intelligence.” But the fast filtering and processing of information implemented in those cars is a clear step towards “artificial resonance.” A number of people argued that a computer program cannot be intelligent. This position may be defended in the following way. Any behavior that is fully understood and can be fully analyzed is not considered intelligent in our tradition. Computer programs consist of step by step instructions so they cannot be intelligent, by definition. Computers will become intelligent once they show the signs of resonance, that is, the ability to detect quickly the most relevant elements of the situation and act accordingly. I do not see any reason why this goal cannot be achieved by an algorithm whose complexity makes it completely opaque to a human being.

8.4.2 Social context of intelligence Intelligence is a controversial concept because of its social ramifications. It played a role in racist theories in the past. It was the focus of a recent debate started in [Herrnstein and Murray (1994)]. For one of the critical responses to that book, see [Gould (1996)]. My discussion of the concept of

page 170

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch08

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Epistemology

171

intelligence is narrow. I do not have anything to contribute to the general debates about IQ and its implications. The general discussion of intelligence leads to some natural and significant questions. One of them is the question about measurability of intelligence and specific methods of measuring it. Another good question is whether there exists a single trait that we should consider to be intelligence or whether intelligence is better understood as a family of different traits. I do not have easy answers to these questions. I believe that intelligence is a concept that can help us understand knowledge acquisition from the philosophical perspective. I do not think that measuring intelligence would contribute in a significant way to my epistemology. It is natural to expect that there are different types of intelligence because there are different types of practical and intellectual tasks where resonance plays a major role. Classifying these situations and types of intelligence may be an interesting project but it goes beyond the scope of this book.

8.5 Free Will My analysis of free will will be partly an illustration of the concept of resonance. The analysis will also anticipate my discussion, in Secs. 11.1.4 and 12.4, of an important distinction between situations when we face uncertainty and can build a realistic probabilistic model representing unknown events, and those situations when we do not have tools to construct such a probabilistic model. Roughly speaking, from the time of Newton to Einstein, the predominant view of the universe was that of a clockwise mechanism. This seemed to indicate that we, people, had no free will. If the current state of the universe completely and accurately determines whether a person living 200 years from now will make a good deed or commit a crime, how can we possibly believe that the person will take one of these actions as an exercise of his free will? The 20th century physics, specifically quantum mechanics, makes us believe that our world is inherently probabilistic. I find this hardly reassuring as an argument in favor of free will. Does a die have free will when we throw it and it “chooses” its side randomly? Suppose that we know that a person gives money to beggars 30% of the time. Is this an evidence of his exercise of free will? If yes then we can easily design a robot with free will. It is standard to equip a computer with a random number generator. The robot could be placed in the street and dispense coins to

page 171

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

172

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch08

Resonance: From Probability to Epistemology and Back

beggars randomly, at the rate of 30%. Would you think that such a robot had free will? My thesis is that free will has nothing to do with the deterministic or random nature of our universe. Free will is our inability to model another person. We try to “understand” all natural objects by constructing their simplified models in our minds and then we use these models to make deterministic or probabilistic predictions. We cannot apply this strategy to other humans. To support my claim, I will first discuss an adversarial situation such as a game of chess or war. To make accurate predictions, we need to have a fairly complete and reliable model of the adversary including his thoughts. We should try to understand not only his analysis of the general aspects of the situation but also his understanding of our own thinking. In other words, we should try to understand how the other person understands our understanding of the situation. This is the beginning of an infinite alternating sequence of mutual models constructed by the adversaries, trying to include all of the analysis done so far and to add one more level at each stage. The infinite sequence cannot be effectively built and analyzed in real time. In practice, the game of chess and similar games seem to be the only well known examples where the players try to predict several moves of the adversary at a time. In all other circumstances we seem to give up at a much earlier stage of the mutual model construction. The result is that if two roughly equally intelligent people meet then none of them can construct a reliable model of the other person and so he cannot make reliable predictions, deterministic or probabilistic, of the other person’s actions. We perceive our failure as “free will” of the other person. People who can easily and effectively predict actions of other people are often successful and admired for their skill. We often talk about free will in the context of ethical choices, not adversarial situations, so my argument has to take a more general shape. I see at least three related but separate reasons why we cannot build an effective deterministic or probabilistic model of another human mind (or our own thinking and decision process, for that matter). (i) Our mind seems to be incredibly complicated and it combines sensory inputs that are equally complicated. These “sensory inputs” encompass, among other things, information obtained from other people, including all science, common knowledge and general culture. Psychologists and sociologists try to describe laws governing our

page 172

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch08

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Epistemology

173

thinking processes, both in the statistical and descriptive sense. Yet despite enormous progress in these fields and general attempts to understand human psyche, quite often we cannot build a reliable model predicting the outcome of a decision making process of a concrete person in a concrete situation. (ii) There is solid anecdotal evidence that our decision process is chaotic, that is, it is very sensitive to small changes in the input. Sometimes seemingly minor and irrelevant events can change the mind of a person in a significant way or concerning a significant matter. As far as we know, the behavior of a chaotic system cannot be predicted beyond certain limits, specific to the system. (iii) Let us assume for the sake of the argument that one day we will be able to construct measuring devices able to monitor all inputs of a person with arbitrary accuracy and the chaotic nature of the thinking process will be overcome by some incredibly sophisticated theory and a computer program based on it. Suppose that this system will be able to predict human behavior with perfect accuracy in deterministic cases, and will reliably asses probabilities in those situations when the decision is made randomly. This system would have practical applications, obviously, although I doubt that people would be comfortable knowing that it exists and is watching them. I believe that this system would still leave us with a strong feeling of the existence of free will. The reason is that we already have such a system. It is called God or Nature, depending on your religious position. God supposedly knows the future, including your future decisions based on your free choices. Nature does not care about our sense of time so it “knows” what you are going to do. From the human perspective, the divine or natural knowledge of the future does not seem to destroy the feeling of the existence of free will. The reason is that the mere constatation that there is a system that “knows” our free decisions cannot compensate for our lack of understanding of the process. I posit that if we ever design a system as powerful as to make perfect predictions of free decisions, we will not be able to understand it in our human terms and it will have no effect on our perception of free will, just like the widespread belief in God does not. The three reasons why we cannot build an effective theory of human mind have different roots. Reason (i) may be described as a limitation of contemporary science. Reason (ii) appears to be a permanent scientific

page 173

April 29, 2016

9:6

174

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch08

Resonance: From Probability to Epistemology and Back

limitation. And reason (iii) seems to be a permanent limitation of our mind. My analysis of free will does not include any elements that would limit it to human beings. It applies equally well to nations, big corporations and future intelligent robots.

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

8.6 From Philosophy to Science What is the value of my resonance theory? I doubt that the resonance theory will provide a complete solution to the problems of induction, consciousness, intelligence and free will. These phenomena seem to be too complicated to be amenable to simple modeling. But the resonance theory gives us the following. (i) It helps to analyze the philosophical foundations of probability (see Chapter 11). This was the original motivation for my epistemological speculations. This is also an application of the resonance theory that I am most confident about — the resonance theory generates reasonable answers to difficult questions about the nature of probability. (ii) The resonance theory removes the aura of mystical enigma from the four philosophical problems mentioned at the beginning of this section. It shows that one can reduce these problems to questions that are comprehensible and amenable to analysis. (iii) The resonance theory reduces the four philosophical problems to four scientific problems. The resonance theory can be tested (at least partly and at least in theory) using standard scientific methods. One can also develop more detailed variants of the theory and test them. (iv) The resonance theory indicates that the four philosophical problems are amenable to scientific analysis and hence it may spur development of other scientific models for the problems. Personally, I would find any attempt to represent the four philosophical problems in a scientific way more interesting than their purely philosophical analysis. The idea of resonance and the analysis of induction, intelligence, consciousness and free will might apply, at least to some extent, to complex systems other than a human being such as nations, smaller groups of people such as political parties and to the largest group — the humanity. Is resonance really a new idea? I do not know because it is impossible to know the whole philosophical literature. But I know that resonance did

page 174

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch08

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Epistemology

175

not appear under any name or in any form in the writings of de Finetti, Feyerabend, Kuhn, Lakatos, Polanyi, Popper and von Mises, and it seems to be unknown in the popular philosophy. The reason why resonance was overlooked might have been that it was mistaken for “intuition” or “mystical experience.” Scientifically minded philosophers might have considered it a way of smuggling religious concepts into “rational” philosophy. Religious philosophers are unlikely to accept my theory because it stresses the idea of empirical verification of claims (although this part of my theory is a well established philosophical idea). The emergence of the phenomenon of resonance is attributed in my theory to Darwinian evolution. Hence, resonance does not provide a back door to a transcendental vision of the world. It explains popular beliefs in transcendental theories as a survival strategy.

page 175

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch09

Chapter 9

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Religion

I reluctantly bring up the subject of religion in this book. I will not try to discuss this topic in depth because I have nothing new to say. My earlier book [Burdzy (2009)] presented my views on probability. This book presents, in addition, my views on epistemology. In both cases I felt that I had some new ideas and I wanted to share them. This is not the case of religion. I feel that it is better to talk about religion, no matter how briefly and tentatively, rather than to follow an ostrich approach. It is impossible to fully discuss science and not to mention its intellectual competitors. In the middle of the 20th century many philosophers felt that Marxism was one of the major players in the field. The collapse of communism as a political force removed Marxism as a serious competitor from many discussions. As of today, religion is entrenched in Poland, my native country, and the United States, my adopted country. Religion is playing a major role in many current political developments — various wars and unrest in the Middle East and Muslim countries, the terrorist movement, political tensions in Europe, etc. Lakatos classified some philosophical theories of science as “demarcationist” ([Lakatos (1978a), p. 109]). For some philosophers, that meant delineating the border between science and religion, and for a good reason. Both religion and science have been playing the roles of major pillars of the society and it is hard to avoid philosophical analysis of their relationship in a book on the foundations of science. Some philosophers, who will remain unnamed here, were so afraid of offending religious people that they assigned the role of the main intellectual competition for science to superstition, voodoo, etc. These phenomena exist, of course, but they play a marginal role compared to religion. It is religion

177

page 177

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

178

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch09

Resonance: From Probability to Epistemology and Back

and only religion that is a viable competition for science in the area of epistemology. It is possible that some philosophers refrained from discussing or attacking religion out of conscious or subconscious cowardice. But I will be more generous. Another good explanation is that these philosophers did not want their philosophical theories of science to become embroiled in a heated exchange focused on religion because that would draw attention away from the main elements of their theories. I believe that it is better to discuss religion openly and clearly rather than attack it via proxies such as superstition or voodoo. I think that it is best if I state explicitly my own position on the question of God. I am an atheist. I think that people believe in God for two different reasons. One is that some people were born with a “God gene” or a set of such genes (see [Hamer (2004)]). Although existence of such a gene is a speculative conjecture at this point, it agrees with my guess as to why some people have a great need for spiritual expression and activity. Traditionally this inclination manifested itself in religion. I find it easier to believe that the widespread religiosity of people can be explained by genetic predisposition rather than by social phenomena (a classical materialist explanation). The second reason why some people believe in God is indoctrination. Clearly, I lack the God gene and religious indoctrination failed in my case. Despite Catholic upbringing, I lost my faith when I was about 20. I have no intention of using this book as an instrument of atheist propaganda. I am not interested in spreading atheist propaganda and I am not capable of doing that. But some of my arguments will contain clear elements of my atheist position. I hope that this will not distract religious readers from the main message of the book. I will now make some remarks on the very narrow subject of the interface of religion and epistemology. It is a reasonable philosophical position to maintain that religion makes no verifiable claims. For example, the Catholic dogma that there is one God and at the same time there is the Holy Trinity does not seem to be verifiable in any reasonable way and, perhaps more importantly, it is not intended to be a verifiable claim. If we could agree that religion consisted exclusively of claims that are not verifiable and are not intended to be verified then I would have no problem with religion just like I do not have a problem with poetry and science fiction literature. The latter two make some unusual claims but clearly these claims are not meant to be verified or verifiable in any normal sense of the word.

page 178

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch09

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Religion

179

The sentiment expressed in the last paragraph is no more than my wishful thinking. Religion traditionally made specific and detailed claims about our observable universe. The best known examples include Christian objections to the Copernican and Darwin theories. The official recognition of miracles, for example, those related to the recent canonization of a few modern saints, also belongs to the same category. On the top of that, religious beliefs of a large proportion of the population clearly contain explicit beliefs in divine observable intervention in our universe. This trend in the “applied religion” contrasts with the attempts of philosophers, starting at least with Spinoza, to make religion immune to any attacks based on empirical observations. Spinoza is credited with the invention of pantheism — the idea that God and nature are one. Paul Tillich, one of the most influential theologians of the 20th century, is quoted approvingly by [Polanyi (1958), p. 283], Knowledge of revelation, although it is mediated primarily through historical events, does not imply factual assertion, and it is therefore not exposed to critical analysis by historical research.

Michal Heller — a scientist, a priest and a member of the Papal Academy— pointed out to me in an exchange of opinions [Heller (2013)] that [...] there is a traditional theological doctrine, called “apophatic theology,” reminding us that if we say something about God, we should be certain that we are only expressing our ignorance (docta ignorantia).

The double (ordinary and philosophical) nature of religion is a form of hypocrisy. On one hand, if nuns pray for the recovery of one of them and she does recover then this can be taken as evidence of God’s benevolence and omnipotence (see [Wikipedia (2014o)]). On the other hand, no case of genocide can be taken as evidence against God’s benevolence and omnipotence, according to Christian theologians. I consider direct argument with theologians to be a futile and uninteresting undertaking. I refer to religion in various arguments only to underscore certain elements of my epistemology. My complaints are not limited to religion. Other ideologies distinguished themselves in similar hypocrisy as well. Marxists made very detailed and specific predictions about observable reality. When the predictions did not bear out, communists had no problem finding philosophers willing to certify that every aspect of “real communism” was totally consistent with the Marxist dogma.

page 179

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

180

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch09

Resonance: From Probability to Epistemology and Back

In my opinion, religion is the result of Darwinian evolution because it is an effective way to increase the survival probability for individuals and communities. It gives a motivation to act to individuals and coherence to group action. It helps the government to channel efforts towards common goals. Religion seems to be an antidote to consciousness. When people developed consciousness, they understood the apparently irrational character of their own instinctive behavior. The instinct of life preservation seems to contradict the futility of individual life in the face of death and altruistic social behavior seems to be irrational from the point of an individual. Religion calms the resulting anxiety by offering the concept of the meaning of life, promise of life after death and the divine and unquestionable source of morality. Needless to say, these remarks are my atheist retelling of the biblical story of eating the fruit of the tree of knowledge of good and evil by Adam and Eve. Infallibility seems to be an essential aspect of religion — it addresses a deep seated need of many people. Just like Descartes was looking for a source of ultimate certainty, so do ordinary people. They often find it in God. I will argue that fallibility is the essence of science in Sec. 10.7. I postulate that the main difference between religion and science is that the first one is the embodiment of infallibility and the latter one is the embodiment of fallibility. One of the reasons why I feel intellectually secure in my atheist position is that I do not think that there exists any broadly accepted scientific or philosophical technique that would discriminate between the true religion (if there is such) and all other religions — those that are mistaken and those that are deliberately fraudulent. Of course, every popular religion developed theological arguments in its own support. The difference with science is striking — it is hard to imagine that scientists could claim existence of an object or phenomenon but would have no theoretical or empirical methods that could determine its attributes, at least in principle, to the satisfaction of the whole scientific community.

page 180

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch10

Chapter 10

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Science

I will limit my analysis of science to several topics which are not necessarily most significant but are most relevant to the subject of this book. I have already related science to resonance in Sec. 8.4. I will refer to this relationship occasionally throughout this chapter and the remaining part of the book. I will analyze science as a communication system because I consider my scientific laws of probability (L1)–(L6) to be the de facto basis for the communication system for the probability and statistics ideas. I will also argue that science is a defense against intellectual fraud because this topic is closely related to the claims of subjectivity of probability. Finally, I will try to find a place for my theory among existing philosophical theories of science.

10.1 Science as a Communication System A unique feature of humans among all species is our ability to communicate using language. Individuals of many other species, from insects to mammals, can exchange some information between each other, but none of these cases comes even close to the effectiveness of human oral and written communication. The language gives us multiple sets of eyes and ears. Facts observed by other people are accessible to us via speech, books, radio, etc. The wealth of available facts is a blessing and a problem. We often complain that we are overwhelmed with information. A simple solution to this problem emerged in human culture long time ago — data compression. Families of similar facts are arranged into patterns and only patterns are reported to other people. Pattern recognition is not only needed for

181

page 181

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

182

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch10

Resonance: From Probability to Epistemology and Back

data compression, it is also the basis of successful predictions. People generally assume that patterns observed in the past will continue in the future so knowing patterns gives us an advantage in life. An important example of “patterns” are laws of science. Some people are not as good at pattern recognition as others so communication gives them not only access to multiple sets of eyes and ears but also access to multiple minds. Communication enables exchange of information not only between concurrently living individuals but between generations as well. The result is accumulation, one could say explosion, of knowledge, including scientific knowledge. The process described above is not perfect. Our senses are imperfect, our memory is imperfect and our ability to recognize patterns is imperfect. On the top of that, communication adds its own errors. Some of them are random but some of them are typically human. What we say may be colored by our political or religious beliefs, for example. Some people pursue their goals by spreading misinformation, that is, they lie. Experience taught people to be somewhat skeptical about information acquired from other people. Information is categorized and different batches of information are considered to be reliable to different degree. Science may be defined as the most respected and most reliable knowledge that people offer to other people. The distinguishing feature of science is its method. Science achieved its high status in various ways, for example, scientific claims are often repeatedly verified, the ethical standards imposed in science are much higher than in politics, assault on established theories is approved, facts rather than feelings are stressed, the simplest theory is chosen among all that explain known facts, etc. Religion seems to lie at the other extreme of major ideologies. The utterly counterintuitive claims of the quantum theory and relativity theory are widely accepted by populations as diverse as democratic societies and communist societies, Catholics and Muslims. On the other hand, the humanity seems to have reconciled itself to the coexistence of various religions without any hope for the ultimate coordination of their beliefs. In other words, religious information conveyed from one person to another may be met with total skepticism, especially if the two people are followers of different religions. To maintain its elevated status, science has to present facts and patterns in the most reliable way. The most general patterns are called “laws” in natural sciences. The history of science showed that we cannot fully trust any laws, for example, the highly successful gravitation theory discovered

page 182

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch10

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Science

183

(or invented) by Newton was later fundamentally revised by Einstein. The laws of science are the most reliable information on facts and patterns available at this time, but they are not necessarily absolute truths. The success of science (and human communication in general) depends very much on universality of certain basic perceptions. In other words, almost all people agree on certain basic facts, such as numbers and colors. When I look at five red apples, I am quite sure that any other person would also see five red apples, not seven green pears. Of course, we do make counting mistakes from time to time. The further we are from numbers, the harder it is to agree on directly perceived facts. If two people cannot agree on an answer to a question such as “Is the water in the lake cold or warm?”, they can use a scientific approach to this question by translating the problem into the language of numbers. In this particular case, one can measure the water temperature using a thermometer. Numbers displayed by thermometers and other measuring devices are a highly reliable way to relay information between people. One has to note, however, that no scientific equipment or method can be a substitute for the prescientific agreement between different people on some basic facts. We have to assume that our sensations generated by resonance are analogous to those generated by the resonance process in the minds of other people. For example, suppose that a distance is measured and it is determined that it is 8 meters. A person may want to communicate this information to another person in writing. This depends on the ability of the other person to recognize the written symbol “8” as the number “eight.” The problem cannot be resolved by measuring and describing the shape of the symbol because a report on the findings of such a procedure would contain words and symbols that might not be recognized by another person. The example seems to be academic but it is less if we think about pattern recognition by computers. One of the main reasons for the success of natural sciences is that most of their findings are based on directly and reliably recognizable facts, such as numbers, or they can be translated into such language. Measuring the spin of an electron is far beyond the ability of an ordinary person (and even most scientists) but the procedure can be reduced to highly reliable instructions and the results can be represented as numbers. The further we go away from natural sciences, the harder it is for people to agree, in part because they cannot agree even on the basic facts and perceptions. A statement that “Harsh laws lead to the alienation of the people” contains the words

page 183

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

184

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch10

Resonance: From Probability to Epistemology and Back

“harsh” and “alienation” whose meaning is not universally agreed upon. A very precise definition, legal-style, may be proposed for these words but such a definition need not be universally accepted. I used [Feyerabend (1975)] in Sec. 3.3 to illustrate pitfalls of skepticism. The book deserves another honorable mention for the incredible distortion of science. In Chapter 17 and many other parts of his book, Feyerabend claims that some scientific theories are “incommensurable” with others. This means that the claims of pairs or families of theories are neither in agreement nor in contradiction. Feyerabend is wrong — the essence of science is that it is commensurable. People must have been always aware of frequent incompatibility of their observations, arguments, opinions, etc., with those of other people. The discovery of the scientific methodology was represented by some philosophers as a new relationship between people and nature. At least as significant is the fact that science was a new relationship between people. Because of science, large groups of people developed trust in one another concerning various matters of fact and theory. All Feyerabend’s examples of incommensurability were extracted from the fringes of science. They came from antiquity, from infancy of some branches of science and from sciences that are under development, such as medicine. Incommensurability is a figment of imagination — every person with at least average education knows that copper is a good conductor of electricity and glass is not.

10.2 Some Attributes of Science 10.2.1 Interpersonal character Science is a set of facts that can be described and theories that can be implemented by everyone. These collections span the spectrum from sensory observations to advanced scientific theories that have been determined to be reliably intersubjective. This means that information passed on from one person to another can be reliably verified by the receiving side. Of course, a single person does not have time, energy, knowledge, etc. to verify all scientific claims. Many advanced scientific claims can be verified only by a handful of highly trained experts. But every scientific claim can be verified by everybody in the following sense — any person or organization with sufficient resources (money, time, energy, materials, buildings) can hire people who have the relevant expertise and can verify the claim. Good

page 184

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch10

Science

185

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

examples of collections of advanced claims are modern physics and its technical implementations such as space rockets, smart phones and nuclear plants. The elements of science that are interpersonal and reliable are not necessarily as basic as the ability to count or to recognize colors. Highly trained experts can easily recognize more complex objects or situations such as zebras and supernova explosions. In other words, resonance comes at different levels.

10.2.2 The role and limitations of resonance Scientific theories are often presented as models with parameters. The model is obtained through the process of resonance. It can be analyzed by the usual scientific process of falsification/verification based on logic. The space of models is discrete, in a sense. If a model is rejected, there is no presumption that a better model for the same phenomenon should be “close” to the original one, although it could. The parameters turn each model into a vector space or manifold (a continuous structure). The process of resonance is not sufficiently effective to provide the values of the parameters. But resonance gives bounds on the values of parameters. Consider Newton’s Law of Universal Gravitation. It states that the force of attraction between two bodies with masses m1 and m2 at the distance r is equal to Gm1 m2 /r2 . Here, G is a constant that is a parameter. Its value can be obtained from experiments but not from resonance. The number 2 in r2 may be interpreted as a parameter to be measured in experiments or as a resonance-derived constant because 2 is the only power of r which makes some systems based on gravitation stable.

10.2.3 Convergence to the truth? I wish I could say that science gets closer and closer to the objective truth about reality but my skepticism does not permit me to go that far. It is interesting to notice that neither Kuhn nor Popper, the two best known philosophers of science in the 20th century, claimed that science brought us closer to the truth. Science is an attempt to understand the reality using a certain set of intellectual tools. The tools are designed to select these claims that are practical and also to get rid of manipulation by unscrupulous individuals and ideologies. The practical success of science does not prove

page 185

April 29, 2016

9:6

186

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch10

Resonance: From Probability to Epistemology and Back

that it is a faithful representation of objective reality; see Sec. 3.2 and the last paragraph of Sec. 8.2.10.

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

10.2.4 Science control An important characteristic of scientific research is that it can be directed to and focused on the most significant topics, gaps, anomalies, etc. This gives science a huge advantage over those fields and practices where information is collected randomly or arbitrarily. Asbestos, lead and cigarettes are examples of substances and products which have been scientifically proved to be harmful. Since powerful industries made profits from selling these products, misinformation and political influence were (and in some cases still are) used to keep the manufacturing of these products legal. Science used one of its methods to establish the truth about the harmful effects of these substances — overwhelming amounts of data were collected and analyzed by independent researchers. Religion is an example of ideology where directed or controlled empirical research is not practiced. These activities seem to be incompatible with the nature of religion. Economics is a good intermediate example. Economists collect enormous amounts of data but they have limited ability to perform controlled experiments, especially on the “macro” scale.

10.2.5 Beyond simple induction The effectiveness of science is greatly due to reliance on scientific theories. They enable scientists to make predictions in situations that do not repeat. In other words, theories let scientists make reliable predictions that are not directly based on repetitive observations or experiments. Technology is a good illustration of this feature of science. The first airplane was a scientific prediction, in a sense. The Wright brothers “predicted” that the machine constructed by them will fly. Needless to say, that prediction was not based on a long sequence of observations of similar flying machines. The design was based on repeated observations of some phenomena, for example, the plane had wings, just like birds do. Also, the Wright brothers did experiments with various designs of the propeller. Nevertheless, the airplane itself was not identical to and not even very similar to any other object.

page 186

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch10

Science

187

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

10.2.6 Science as a web Science is a web of highly interrelated facts, theories and fields. For example, the evolution theory is a field of biology but it has to agree with biochemistry (via research on DNA and related molecules), geology (dinosaur bones) and astronomy (evolution of the solar system). In contrast, religion quite often offers isolated claims, such as the Creationist claim that the universe was created in its present shape in the last 10,000 years. Changing a significant part of the evolution theory would affect a number of constituent theories. Creationists can easily adjust their claim and move the purported date of creation without the danger of creating a logical inconsistency. The necessity to coordinate diverse scientific theories creates an opportunity for multiple attempts at falsification of assertions. This applies also to predictions made using probabilistic methods.

10.3 Science for Scientists Science for scientists is a considerably different intellectual construct from science for philosophers. This can be attributed to different ambitions of the two intellectual activities. Scientists need to communicate with each other to verify theoretical claims. Scientific theories have to be expressed in terms that are easily recognizable by all people or at least can be learned by scientists. The terms may refer to or be close to everyday concepts such as distance and time. Some terms have roots in normal life but acquire new meanings in science, for example, the scientific concept of work is only loosely related to work in normal life. I would even say that the chemical concept of water is different from the normal concept of water. Some other scientific terms did not originate in normal life, for example entropy in thermodynamics and eigenvector in quantum mechanics. The basic terms in any science have to be recognizable via the resonance process. More advanced and complex terms may also be recognized via resonance by some scientists but these scientists need special training to move these terms from explicit conscious processing to their subconsciousness. The role of resonance in science is illustrated by the following example of a failure of the process. One of the physics laws says that work is equal to force times distance. This sounds very simple but many students in my calculus classes lack any background in physics and are incapable of solving word problems that involve (the scientific version of) the concept

page 187

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

188

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch10

Resonance: From Probability to Epistemology and Back

of work. These students did not have an opportunity to learn how to recognize, via resonance, those situations when work is done and to identify the components of word problems with the theoretical concepts of work, distance and force. In this book, the primary examples of science for scientists and applications of resonance are the scientific laws of probability (L1)–(L6). Scientists have to learn the concepts of “probability” and “event.” They also have to learn (L1)–(L6) so that they can apply these laws at the resonance level. Philosophers can spend their time arguing about whether symmetry is objective or subjective. Scientists have to develop the subconscious ability to recognize symmetries relevant to probability to be able to implement the probability theory in practice. Scientists can safely ignore the discussion of resonance if all they want to do is to apply probability in statistics and other sciences. Engineers who want to build intelligent machines should incorporate resonance in their vision of intelligence because resonance is one of the main reasons for effectiveness of human actions and, conjecturally, of similar effectiveness of intelligent computers and robots.

10.4 Alien Science Some estimates put the number of habitable planets in our universe at 1022 . It is hard to imagine that none of these supports intelligent life (although [Tegmark (2014)] does not think that it is impossible that we are the only civilization in this universe). Suppose that we receive a long message from an alien civilization and we are smart enough to decipher it. It might contain some scientific information. How do we tell whether the information contains true facts and true scientific theories? Here, on the Earth, science shares the intellectual space with religion, myths, fairy tales, science fiction and individual opinions. It is possible that the extraterrestrial message is one of the science alternatives. We can classify philosophical theories of science by examining their potential applications to alien belief systems. The theories of [Kuhn (1970)] and [Polanyi (1958)] (see Sec. 10.12.1) clearly apply only to terrestrial science. Let me put it in a different way — if these theories apply to a typical extraterrestrial science then both Kuhn and Polanyi had incredible insight and luck to provide the universal (in the sense of universe-wide) description of science.

page 188

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch10

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Science

189

In comparison to Kuhn’s and Polanyi’s theories, various other recent philosophical attempts seem awkward. Logical positivism seems too sterile (see [Wikipedia (2014y)]). Popper’s falsificationism seems too distant from real science where positive verification is highly prized. Carnap’s dream of a universal language and rational confirmation rules guiding the knowledge acquisition process sound like a pipe dream. De Finetti’s philosophy and other forms of Bayesianism seem equally artificial because they envision an automatic process guiding our search for the truth. But all these theories would do much better in confrontation with the alien science than those of Kuhn and Polanyi because they are seeking the essence of science with little reference to the human side of the scientific enterprise. Would my resonance theory apply to an extraterrestrial science? I find it hard to imagine that one can acquire knowledge at a practical rate without the benefit of preprocessing, that is, without using resonance. My question has an empirical nature. If and when we either contact alien civilizations or build a variety of intelligent robots then we will be able to say with confidence whether resonance is an indispensable element of science.

10.4.1 A lonely alien It is possible (although very unlikely in my opinion) that some other planet is inhabited by a lonely sentient being and science was developed on that planet by this individual. He does not have to communicate with other individuals but he has to communicate with himself, via his own memory or written notes (or some other form of external memory), so the part of my theory concerned with science as a communication system (see Sec. 10.1) remains relevant to that alien science. But some other parts of my theory, such as defense against deception (see Sec. 10.12.4), are less relevant, even though self-delusion is a real threat for humans and presumably poses a danger for the alien individual. Let me mention parenthetically that one can go in the opposite direction and replace individuals with communities in my theory, instead of replacing the community of scientists with a lonely alien scientist. In my communication theory of science, the role of individuals can be played by research institutes, universities, nations, etc.

page 189

April 29, 2016

9:6

190

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch10

Resonance: From Probability to Epistemology and Back

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

10.5 Sources and Perils of Loyalty The success of science has not escaped the attention of various individuals and groups. A number of ideologies — political, philosophical and religious — tried to steal the prestige of science by presenting themselves as scientific. This is a part of a much wider trend. People are intelligent animals and they easily discover and learn new ways of achieving their goals. One of the ways to achieve some practical goals is to enlist the help of other people. This can be done in many ways. A simple and honest way is to exchange goods or services in a consensual way. A different way is to make other people think that helping you is beneficial to them. This can be implemented in various ways and can exploit various elements of our psyche — intellectual, emotional and spiritual. At the most sophisticated level, the method uses grand theories such as political systems, economic ideologies and religions. Many of these ideologies occasionally spread false information because doing so is beneficial to some groups of people. Darwinian evolution seems to benefit animals more than their individual cells — many cells are killed in the normal life of an animal. It is only natural to expect that, for similar reasons, the evolution process shaped animals so that species benefit at the cost of individuals, at least occasionally. An example of this process taken to the extreme are exploding ants (see [Wikipedia (2014l)]). In humans, loyalty seems to be a mechanism with similar roots. Loyalty is a strong positive emotion towards another individual, a group of people, an institution or a general concept such as an art trend or a field of science. The essence of loyalty is the reluctance to develop a negative opinion about the object of the loyalty in situations when a totally unbiased observer would do just that. The same evolution and selection process that created resonance also froze some human knowledge using emotions such as love and loyalty. There are a number of areas of knowledge where beliefs about facts and theories cannot be changed at all, can be changed with great difficulty, or change only accidentally. These areas include religion, romantic love, love for (loyalty to) your close and not so close relatives (parents, children, siblings, cousins, etc.), patriotism, local patriotism (loyalty to a small geographical area or group of people), loyalty to a profession, loyalty to a research program or art school. The list is likely to be much longer. The damping effect of love and loyalty helps maintain high spirits of an individual in face of adversity (Christians do not stop believing that God is merciful no matter what misfortunes happen to them or even to whole nations).

page 190

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch10

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Science

191

Loyalty helps to maintain stable groups, from marriages and friendships to large nations. The mixture of self-interest, ability to communicate (possibly false) information to other people and loyalty is extremely powerful. Political ideologies and religion explore the combined power of these phenomena to the limit. Science was sometimes hit by a ricochet originating from such ideologies. On other occasions, these ideologies tried to enslave science outright. An example of an intellectual ricochet was the attempt of the Catholic church to suppress the Copernican theory. At its core, the Catholic dogmas are totally independent from the beliefs about the relative motion of the Sun and the planets but the Copernican theory might have given Catholics wrong ideas in areas far from astronomy. At the other extreme, science was treated like a slave in the Soviet Union under Stalin. Since human science is a social activity, it is under a constant threat of being distorted by old and new ideologies. Needless to say, it may be also distorted by petty manipulations of some unscrupulous scientists. At the intermediate level it may be distorted by institutions such as universities or laboratories. So, science had to develop defensive mechanisms. Its main defensive mechanism is the requirement of amenability of scientific theories to empirical verification or falsification. A different danger to science comes not from attacks on science but from creation of alternative ideologies. Among those, religion is unquestionably most successful. But the competition comes in different guises, from pseudo-science to postmodern philosophy. See [Sokal (2008)] for examples of pseudo-science and his hoax exposing the silly face of postmodern philosophy. Alternative ideologies sometimes challenge scientific findings in a direct way. Creationism and its modern version, “Intelligent Design,” are examples of a direct challenge (see [Wikipedia (2014x)]). Challenges mounted by pseudo-science and postmodern philosophy are partly direct and partly indirect — these ideologies promote highly suspicious modes of intellectual activity and present them as valid alternatives to the scientific method. Overall, science is an activity concerned not only with a neutral nature. It is also directed against malicious opponents with various motives who continually try to subvert scientific efforts to find the most objective truth. The claims of religion are not reliably interpersonal — the existence of multitude of religions proves this point. This does not imply, according to the standard logic, that any given religion is false. The main point of

page 191

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

192

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch10

Resonance: From Probability to Epistemology and Back

science is not that its claims are closer to the truth than those of religion. The claims of science are accessible to all people and reproducible by all people (in principle). Hence, they are less likely to be intentionally false. The possibility of verification discourages falsification of experimental results, creation of false theories and publication of false claims. The scientific claims that are considered the most significant are experimentally verified multiple times in multiple ways, unlike religious claims. I will now propound a speculative idea that there cannot be a single scientific method because science has to defend itself from various existing and potential new directions of assault. These include religion, government (communism, fascism), superstition, modern pseudoscience, political theories, postmodern and related philosophical ideologies, and formally correct but uninteresting scientific theories. Just like medical researchers have to invent new antibiotics to fight bacteria that developed resistance to old antibiotics, science has to develop methods of differentiating itself from other activities that learned how to mimic science. I am far from adopting Feyerabend’s position that “anything goes” but I am willing to admit that he might have pointed to a significant problem that science has to face. The idea that science needs empirical verification goes back at least to Francis Bacon in the 17th century. He is even believed to have advocated to “torture” the nature but this quote seems to be an inadequate translation according to [Pesic (1999)]. The question of who was the first philosopher to require empirical verification of all scientific theories is a good illustration of how philosophy and science diverged. It is common among scientists to believe that the idea was invented by Popper in the 29th century (see [Tegmark (2014), pp. 124, 281 and 300]). In fact, Popper developed a somewhat awkward theory based only on falsification (and “corroboration” in place of verification) as a philosophical device to evade the notorious problem of induction. But Popper deserves full credit for reminding the 20th century intellectuals (well, only those that were not deaf to the voice of reason) that the search for the truth without empirical verification can easily result in unbridled fantasy. Although Popper’s version of falsification was intended to solve the problem of induction and I do not believe that he succeeded in this endeavor, I consider falsification absolutely fundamental to science because this is the best way to discriminate between science on one hand and political ideologies, religion and pseudo-science on the other.

page 192

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch10

Science

193

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

10.5.1 Science as an antidote to manipulation Science is partly a result of developments in information processing in societies. First, people learned how to communicate, pass information on to others, receive information and use information to their benefit. Then they noticed that they could lie and manipulate other people. And then they noticed that other people lie and manipulate. Science is an answer to this problem — a system which ensures that the effects of other people’s lies and manipulation are minimized. This system has a number of elements. (i) Scientific theories are supposed to be expressed in terms that either are or can be reduced to those facts that are most widely recognized as true. These are variably referred to as qualia, basic facts, sense perceptions, etc. (ii) Most successful and solid scientific theories are based on multitude of experimental evidence. Science has methods of improving reliability of observations and measurements. (iii) Science is expressed in terms that give clear operational instructions to people who want to verify factual and theoretical claims. Some theories are too advanced to be verified and some experiments are too costly (in terms of money, energy and labor) to be performed by an ordinary person but they can be repeated at least by experts or teams of experts. (iv) Publishing false scientific claims results in a level of opprobrium far exceeding the level of criticism of dubious claims made by representatives of other ideologies (political, religious, etc.). (v) Despite the presence of all the usual evils of social interactions, such as conformism and suppression of criticism, in the scientific community, there is a great degree of support for new and even revolutionary ideas. Einstein and Darwin were role models for generations of scientists. Searching for errors within established theories is a part of the scientific ethos. (vi) Scientific claims are repeatedly tested. The standing of a theory depends on the severity and number of tests that were performed or that can be performed. Hence, there are substantial differences between the reliability of physics, chemistry, biology, medicine, economy and humanities. (vii) Science is a network with fractal growth, that is, it grows in unexpected directions, it is sometimes circular in the sense that it

page 193

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

194

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch10

Resonance: From Probability to Epistemology and Back

looks at the same facts and theories from different angles, magnifies some parts and fills some gaps. For this reason, trying to negate a scientific fact has the same effect as removing a card from a house of cards — the whole structure may collapse. Lindley suggested that the Moon is made of green cheese with strictly positive (personal) probability (see Sec. 16.3). If the cheesy composition of the Moon is ever confirmed, most of the contemporary science will collapse. At the other extreme, many key theological claims can be modified individually without much harm to the rest of a given theology. (viii) Prediction is stressed in science because it is hard and so it is a severe test of a given theory. Explanations of existing facts are easier to come by and harder to eliminate by logical analysis.

10.5.2 Dependent information sources Standard Bayesian information processing in our minds increases our degree of belief in a claim if and when some new piece of supporting evidence arrives. The evidence may have the form of hearsay. It would not be wise to reject all evidence that is indirect. The problem with this mode of opinion forming is that we are apparently not very good at recognizing situations when multiple information channels are highly correlated. We are all bombarded by propaganda of political, religious and social nature. Many instances of the same message are often highly correlated. Despite the high correlation of information sources people feel that a claim is corroborated if they hear multiple expressions of the same opinion. Science is not immune from this danger. But science is the only ideology which placed multiple independent verifications of every claim on a high pedestal. This ideal is an element of “scientific religion” in addition to those described in Sec. 10.7. If people were totally gullible and succumbed to all propaganda then the society would likely collapse. So, a compensating behavior developed and many people became immune to any amount of propaganda in some specific fields or coming from specific sources. There are several unfortunate results of these developments in the social communication system. The first one is the war between propaganda sources which encourages information providers to use more and more extreme methods of communication and indoctrination. Another negative effect is a polarization within minds of individuals (but also groups of people). The whole universe of information is often divided into reliable sources, no matter how dependent and

page 194

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch10

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Science

195

deceitful they might be, and unreliable sources and arguments, no matter how reasonable they might be. A common result is an almost complete ossification of opinions among some people and groups, especially on emotionally charged subjects, such as religion, politics and social issues. Going back to science, one may say that science is an attempt to counteract these obviously counterproductive practices and habits. Ideally, scientists are supposed to listen to all reasonable arguments, set aside emotions and loyalties, and test all claims multiple times, independently, and in diverse ways.

10.6 Falsificationism and Resonance Popper invented falsificationism (see [Popper (1968)]) to solve the problem of induction. Roughly speaking, according to Popper, progress in science is based on falsification, a form of deductive reasoning. Hence, the problem of induction goes away. I believe that the problem of induction is ill-posed (see Sec. 8.1.1) so this automatically makes Popper’s theory redundant in my epistemology. But I will try to defend Popper later in this section and in Sec. 10.7. My resonance theory reflects my deep belief that the process of selecting scientific theories for verification or falsification is at least as important as the second stage, that is, attempts to falsify the theories. Popper did not seem to be concerned with the selection problem. If we set aside philosophical controversies surrounding subjectivity and focus on routine scientific research, the Bayesian approach makes it clear that empirical evidence can increase or decrease the probability of validity of a theory. It is totally unrealistic to represent science as a process favoring only one of these directions. In fact, some of the most famous and feted experiments confirmed rather than falsified scientific theories. A good example is the recent detection of the Higgs boson (see [Wikipedia (2014z)]). Typically, an experimental falsification of a specific claim is not the end of a theory. It only indicates that a certain theory may be limited. In other words, we may have to impose extra assumptions that were so far ignored or unknown. For example, suppose that someone claims that water always boils at 100◦ Celsius. This is not true in high mountains. The reason is that the boiling temperature depends on the pressure. In this case falsification leads to a larger (stricter) set of assumptions. In other situations, a case of falsification may lead to a new set of assumptions so strict that the law

page 195

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

196

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch10

Resonance: From Probability to Epistemology and Back

becomes useless, for example, it could effectively be that the assumptions were satisfied only once in the whole history of the universe. When scientists encounter “anomalies,” that is, experimental results contradicting a current theory, they may try to explain the discrepancy using ad hoc methods. Human inventiveness knows no bounds so here is where Popper’s idea of a crucial experiment is needed. If a part of the scientific community feels that the discrepancies between the current theory and anomalies form an unreasonable gap then it is legitimate to ask the supporters of the current theory to propose a crucial experiment and promise to abandon the theory if the results of the experiment do not match the predictions of the theory. Many authors ([Feyerabend (1975)], [Kuhn (1970)], [Lakatos (1978a)]) pointed out that science does not evolve in such a clean way. But in my opinion Popper’s idea should not be taken as a literal description of the scientific enterprise but as an ideal which scientists should enforce when controversies in a field of science reach an unacceptably high threshold. Popper’s falsificationism is not an accurate representation of statistics and a big chunk of science concerned with estimation (determination) of parameters. This is a search for the best matching theory in a continuous space and hence has little to do with falsification. Falsification of a theory via a single crucial experiment is fully reliable only in the case of a deductive-type theory where a large body of knowledge is derived from a handful of assumptions. Only mathematical theories have such structure and these are the same theories that are not amenable to empirical verification. This inadequacy of falsificationism can be illustrated in an alternative way by Descartes’ famous “Cogito ergo sum” (“I think, therefore I am”). The apparent truth of the claim was questioned by a number of philosophers but to see how little value it has, it is best to assume that the claim is true. Then, we realize that Descartes’ claim is worthless because nothing consequential can be derived from it. It is an isolated philosophical thought of a person desperately looking for certainty. Science and everyday life are based on huge collections of assumptions, usually called facts and theories, generated by resonance. These facts and theories are independent from each other to large extent, that is, typically one can falsify one of them without affecting many of the other claims. The distinguishing feature of science is that facts and theories form a web that imbues science with the appearance of reliability and stability. Falsificationism works well in those cases where a single experiment can

page 196

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch10

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Science

197

affect our belief in a large number of facts or theories and hence can have a significant effect on science. Popper’s idea of falsification of a theory does not presuppose existence of an alternative theory waiting in the wings. This is fundamentally different from the Bayesian philosophy of science because the Bayes theorem requires the knowledge of alternative theories and their consequences. The best known revolutions in science that produced the quantum physics and relativity theory can hardly be described as a Bayesian transformation of the prior distribution over the collection of all theories into a posterior distribution over the same space. While the Bayesian approach may be a reasonably accurate description of some developments in science, Popper’s philosophy is at least as accurate in some other cases.

10.7 Falsificationism as a Religion Science is an ideology with an organization, not unlike religion. One cannot assign policemen to watch over activities of all scientists so the proper behavior (following the best scientific practices) has to be enforced by an ideology — a system of beliefs that are not necessarily rational but greatly depend on the emotional side of our minds. Science is party based on the emotions of honor and loyalty. Other elements of enforcement are scientific organizations (universities, research institutes, learned societies) and various forms of recognition. Scientists seem to be motivated by vanity much more than by money. Scientific organizations provide rewards that satisfy the craving for vanity. Science is vulnerable to the same problems that apply to other organizations and ideologies: authoritarianism, herd instinct, enforced conformism, etc. See [Polanyi (1958)] for a detailed review of various nonrational (“personal” in Polanyi’s language) aspects of science. Popper’s falsificationism plays a special role in the scientific ideology. Recall that Popper tried to solve the philosophical problem of induction by reducing it to falsification, a type of deductive procedure. I have great doubts about this proposal, just like many other people. But Popper’s idea became extremely important for a different reason. Falsificationism became an article of scientific faith. One can trace the roots of the idea to Francis Bacon in the 17th century but it was Popper’s somewhat dogmatic presentation that somehow appealed to many minds. It was pointed out by many authors that Popper’s idea of crucial experiments that can falsify a theory is totally unrealistic — this is not how

page 197

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

198

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch10

Resonance: From Probability to Epistemology and Back

science works (see [Feyerabend (1975)], [Kuhn (1970)], [Lakatos (1978a)]). But the idea morphed into a scientific Decalogue. Just like in the case of the Ten Commandments, falsification is more of a moral aspiration than a rule that is strictly followed by the scientists. Scientists are sinners because they breach the rules of falsificationism for various reasons. These include petty desires for grants, recognition and power in the scientific community. The list of reasons also includes deep seated prejudices and well intentioned skepticism towards new and untested theories. Falsificationism is based on doubt. No scientific claim is immune from falsification in Popper’s ideology. The entrenched position of doubt in the modern philosophy was criticized by [Polanyi (1958), Chapter 9]. He proposed “commitment” as an antidote for doubt in Chapter 10 of his book. I think that Polanyi missed the fact that doubt is directed not only against abstract and subtle intellectual claims but also, and perhaps primarily, against active subversion of the truth. Popper’s extremist falsificationism might not be the formal basis of my law (L6) but it is definitely the basic source of my inspiration. Probabilistic claims have to be verified in some way. Philosophers working in the area of probability before Popper either were unable to describe the process of verification or falsification, or ignored that process, or had philosophical ideas that were awkward and unrealistic. Popper insisted that falsifiability can be applied to probability. The analysis of what can be falsified according to the theories of von Mises and de Finetti makes it perfectly clear why they are inadequate. According to von Mises, only frequency observations may lead to falsification of probability statements. In de Finetti’s theory, only inconsistency can be observed and provide a form of falsification.

10.7.1 Sainthood in science Great scientists play the same role in science as saints in religion. Christians found the source of their moral aspirations in the Ten Commandments and the rest of the Bible. But most Christians find it very hard to stick to all moral teachings all the time. Hence, most Christians are sinners. The concept of a saint is used to show by example that it is possible to obey moral teachings. More significantly, saints put a human face on dry teachings of the catechism and make them more memorable and palatable.

page 198

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch10

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Science

199

The fundamental moral teaching of science is openness to change. If a theorem or law of nature is falsified then it should be abandoned. Implementing this principle is very hard for scientists who invested much of their life in a theory and they are asked by the moral code to abandon it. So they can interpret the results of the falsifying experiment as “anomaly.” This, of course, is partly legitimate because anomalies do occur — they are observations that can be explained away without making major modifications to the reigning theory. Kuhn pointed out in his book [Kuhn (1970)] that science is different from, say, philosophy, because it changes. There are still Aristotelians among philosophers although they may be different from the followers of Aristotle in antiquity. I agree with Kuhn but I would put it differently. Philosophy and religion did change and do change but they are not necessarily proud of this process. It may sound preposterous but it is conceivable that philosophers and theologians will arrive at the ultimate theories one day and they will never change them again. Scientists are the only community where change has been elevated to the status of the supreme moral commandment. To inspire greater devotion among its believers, the religion of falsificationism has its saints. Great scientists such as Darwin and Einstein play the role of saints because they symbolize the change in science. No matter how resistant the community might have been to the change at the time of birth of the theories of evolution and relativity, the change was eventually accepted and embraced. Einstein was joined in the falsificationist heaven by a number of scientists who developed the quantum theory (Bohr, Heisenberg, Dirac, etc.) and many other lesser saints. The scientific community reveres these people as their own saints because they proved through their actions that no scientific theory is immune to falsification. Some of the most deep seated beliefs about our universe were overthrown at the beginning of the 20th century by the relativity theory and quantum physics. The analogy with Catholic saints can be continued by pointing out that Einstein did not have a perfect scientific judgment, just like many saints used to be sinners. Einstein added a “cosmological constant” to his equations (and later regretted doing it) and did not believe that “God played with dice” (see [Wikipedia (2014m, 2014n)]). Both of these beliefs were controversial and their scientific status does not seem to be ultimately resolved at this time.

page 199

April 29, 2016

9:6

200

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch10

Resonance: From Probability to Epistemology and Back

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

10.8 Technology Polanyi presents in Sec. 6.8 of his book [Polanyi (1958)] a profound discussion of technology as an activity fundamentally distinct from science. I will make only a few remarks on technology. Animals apply simple induction just like people. Primitive animals “learn” beneficial behavior via the selection process (a part of evolution). More advanced animals learn at the level of individuals, that is, individuals can notice patterns and learn beneficial behavior as a reaction to repeated external conditions. But very few animals can make tools. This ability is limited to a handful of species, mostly apes and some birds. Moreover, animal tools are rudimentary. Technology is where the unbridgeable gap between humans and animals is most apparent. One does not have to point to the space shuttle to prove our superiority. Animals have not developed anything even as simple as a cart. The highly complex structures built by ants, termites and some birds are results of pure instinct. Technology is based on scientific theories, not generalization from repeated observations. Inventors build new equipment in their minds using novel applications and combinations of the known laws of nature. Applications to technology, a form of positive verification, provide a much more convincing proof of validity of a scientific theory than any laboratory experiment.

10.9 Multiple Personality Disorder According to [Wikipedia (2014t)], Dissociative identity disorder (DID), previously known as multiple personality disorder (MPD), is a mental disorder on the dissociative spectrum characterized by at least two distinct and relatively enduring identities or dissociated personality states that alternately control a person’s behavior [...].

This condition should not be confused with schizophrenia; see [Wikipedia (2014u)]. People have an amazing ability to combine scientific and religious beliefs and behavior in their lives. Most of the time they avoid making really basic scientific mistakes, no matter how strong their religious beliefs are. This provides strong empirical evidence that a balanced mix of religion and science is the best recipe for maximizing survival probability.

page 200

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch10

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Science

201

People are adept at dividing their thoughts and actions into the scientific and the spiritual. In ordinary life, they follow the scientific method to achieve practical goals. They turn to religion if science fails them. Health issues are a good example. Medicine made a great progress over the centuries but we neither fully understand the human body nor can cure many of the diseases, injuries, etc. So, many people pray for the health of their relatives, friends and their own. Similar remarks apply to other aspects of reality that significantly affect our lives, for example, weather and war. Once a scientific fact is established with a sufficient certainty, it moves from the spiritual to the scientific part of the mind. For example, despite the biblical precedent, nobody seems to pray for the multiplication of bread to feed the hungry in a famine zone. Technology seems to be the real triumph of scientific thinking. Some scientific theories can be ignored but technology cannot be ignored easily. I find it amusing and satisfying to see that people dressed in totally different ways (mainstream Westerners, Hindus, Muslims, Sikhs, orthodox Jews, etc.) use cell phones as naturally as they drink water. The same people who sometimes kill each other for religious reasons, all implicitly accept quantum mechanics hidden inside cell phones. We learn from [Wikipedia (2014x)] that According to a 2014 Gallup poll, about 42% of Americans believe that “God created human beings pretty much in their present form at one time within the last 10,000 years or so.”

The enduring and amazing popularity of Creationism in the US illustrates the fact that one can ignore science as long as doing this makes no difference in practice. Creationists do not dispute those scientific laws that affect everyday life. There are some groups of people, specifically, followers of Christian Science and the Jehovah’s Witnesses, who reject either all medical attention or at least some significant medical procedures. This fact could be used to dispute my remarks about established scientific facts moving from the spiritual to the scientific part of our psyche. But note that this rejection of science is in the area of medicine which, as I have already noted, is only partly successful. There are very few, if any, people who reject the most basic scientific facts affecting everyday life. The spiritual approach to the truth evolved to take over our mode of thinking in situations when the scientific approach does not work. There

page 201

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

202

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch10

Resonance: From Probability to Epistemology and Back

are two basic types of situations where this arises. First, there are practical needs where science failed. To this day this applies very much to medical problems. Second, there are situations where science has a clear advice but the advice is pessimistic. It may be an advantage to the species to maintain false hopes in individuals as this may improve the survival rates by pure chance. Individuals that strive to overcome difficulties against all odds may have higher probability of success in those cases when a rare chance event gives an unexpected boost. The United States accidentally embraced a good model of separating science in schools from religion in churches. Originally this was meant to prevent religious tensions in the society. But it has other beneficial effects. The separation of church and state in the organization of the society parallels the separation of various modes of information processing in our minds. The scientific mode is open to processing based on logic and empirical evidence. The religious mode is closed to scientific-style verification.

10.10 Reality, Philosophy and Science Quite often, philosophy and science start with the same basic observations, such as “2 apples and 2 apples makes 4 apples.” The bulk of research in mathematics and science consists of building more and more sophisticated theories dealing with more and more complex real phenomena. Philosophy, on the other hand, often goes in the opposite direction and analyzes the foundations of our knowledge, questioning the “obvious” truths. I will illustrate the above claims with a brief description of where the analysis of “2 + 2 = 4” can take mathematicians and philosophers. Mathematicians developed a theory of numbers that includes not only addition but also subtraction, multiplication and division. They developed interest in “prime” numbers. A prime number cannot be divided by any other number except itself and 1. Then mathematicians asked whether there exist infinitely many prime numbers and whether there exist infinitely many pairs of prime numbers which differ only by 2. They proved that there are infinitely many prime numbers but they still do not know (at the time of this writing) whether there are infinitely many pairs of prime numbers that differ by 2. A philosopher may start with a few examples which seem to contradict the assertion that 2 + 2 = 4. If we place two zebras and two lions in the same cage, we will soon have only two animals in the cage, hence, 2 + 2 = 2

page 202

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch10

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Science

203

in this case. Two drops of water and two drops of water can combine into a single drop of water, so 2 + 2 = 1 in some situations. If we place two male rabbits and two female rabbits in a cage, we may have soon 37 rabbits, so 2 + 2 = 37 under some circumstances. I am sure that all people would feel that all these examples were misleading. Pinpointing what exactly is wrong with these examples is not quite easy. Is it that these examples are “dynamic” in nature, so 2 + 2 = 4 does not apply? The answer cannot be that simple, because we can envision a dynamic experiment of placing two apples and two oranges in a basket. The result will be that we will have 4 fruit in the basket, vindicating the claim that 2 + 2 = 4. Mathematicians and scientists have to assume that 2 + 2 = 4. This is not because they have an ultimate philosophical or scientific proof that this statement is objectively true but because doing otherwise would paralyze science. Philosophers discovered long time ago that some standard scientific practices (induction, for instance) seem to be shaky on the philosophical side. Scientists have no choice but to ignore these objections, even if they seem to be justifiable. In order to apply the statement “2 + 2 = 4,” children have to learn to recognize situations when this law holds, by example. There is a wide spectrum of situations that can be reliably recognized by most people where the law 2 + 2 = 4 applies. In other words, resonance at this level seems to work in a very reliable and interpersonal way. A similar remark applies to probability. There is a wide spectrum of situations, easily recognized by most people, where probabilities can be assigned using standard recipes. The role of the science of probability, at the most elementary level, is to find these situations and present them as scientific laws. One of the greatest mistakes made by von Mises and de Finetti was an attempt to mix philosophical objections into scientific research. The statement that “if you toss a coin, the probability of heads is 1/2” has the same scientific status as the statement “2 + 2 = 4.” Both statements summarize facts observed in the past and provide a basis for many actions taken by scientists. The idea advanced by von Mises and de Finetti alike, that probability cannot be assigned to a single event, is a purely philosophical objection that can only confuse scientists and, especially, students. There is no science without 2 + 2 = 4 and there is no probability theory without P (heads) = 1/2. The philosophical objections have to be disregarded in probability and statistics for the same reason why they are ignored in number theory and physics. Of course, statistics has not been

page 203

April 29, 2016

9:6

204

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch10

Resonance: From Probability to Epistemology and Back

paralyzed by the philosophical claims of von Mises and de Finetti. The statement that P (heads) = 1/2 is treated as an objective fact in statistics and one has to wonder why some people believe that the philosophical theories of von Mises and de Finetti have anything to do with science.

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

10.11 Decision Making The unique characteristic of statistics among all natural sciences is that decision theory is embedded in it in a seemingly inextricable way. I will try to separate the inseparable in Chapters 11 and 12. Here, I will outline my philosophy of decision making in relation to my philosophy of science. In deterministic situations, decision making is not considered a part of science. For example, it is up to a chemist to find the melting temperature of gold (1064◦ C) but it is left to potential users of chemistry to implement this piece of scientific knowledge. If anybody needs to work with melted gold, he or she has to heat it to 1064◦ C. The decision to heat or not to heat a piece of gold is not considered a part of chemistry. The laws of deterministic sciences can be presented as instructions or logical implications: if you heat gold to 1064◦ C then it will melt. If you want to achieve a goal, all you have to do is to consult a book, find a law which explains how to achieve that goal, and implement the recipe. This simple procedure fails when a decision problem involves probability because the goal (the maximum possible gain, for example) often cannot be achieved with certainty. It is standard to assume in decision theory that the decision maker would like to maximize his or her gain. If no decision maximizes the gain with certainty, the decision maker has to choose among available decisions using some criterion not based on the infallible attainability of the goal. The choice is not obvious in many practical situations. Historically, statistics had a strong decision theoretic component — scientists felt that it would be unfair to leave this matter in the hands of lay people, who might not be sufficiently knowledgeable about decision making. The decision making problem is not scientific in nature. Science can predict the results of different decisions, sometimes with certainty and sometimes with some probability, but it is not the business of science to tell people what decisions they should make. The identification of decision making and probability assignments by the subjective theory of probability is misleading (see Sec. 12.6). The identification is only a mathematical trick. The subjectivist claim that your decision preferences uniquely determine your probabilities (and vice

page 204

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch10

Science

205

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

versa) refers to nothing more than a purely abstract way of encoding your preferences using mathematical probabilities. This part of the subjective theory shows only a mathematical possibility of passing from probabilities to decisions and the other way around, using a well defined mathematical algorithm. If probability is objective, it is not obvious at all that decision preferences and probabilities should be identified (see Chapter 12).

10.12 Major Trends in Philosophy of Science I doubt that I can find the precise place of my philosophy of science among existing theories. Nevertheless I decided to present a very sketchy review of a few trends in this area and supplement it with some general observations of mine. I start my presentation with several highly schematic classifications of philosophical theories of science. The first one is borrowed from [Lakatos (1978a)]. (i) Inductivism ([Lakatos (1978a), p. 103]). According to inductivism only those propositions can be accepted into the body of science which either describe hard facts or are infallible inductive generalizations from them. When the inductivist accepts a scientific proposition, he accepts it as provenly true; he rejects if it is not. His scientific rigour is strict: a proposition must be either proven from facts, or — deductively or inductively — derived from other propositions already proven.

(ii) Conventionalism ([Lakatos (1978a), p. 105]). Conventionalism allows for the building of any system of pigeon holes which organizes facts into some coherent whole. The conventionalist decides to keep the centre of such a pigeonhole system intact as long as possible: when difficulties arise through an invasion of anomalies, he only changes and complicates the peripheral arrangements. But the conventionalist does not regard any pigeonhole system as provenly true, but only as ‘true by convention’ (or possibly even as neither true nor false). In revolutionary brands of conventionalism one does not have to adhere forever to a given pigeonhole system: one may abandon it if it becomes unbearably clumsy and if a simpler one is offered to replace it.

(iii) Falsificationism ([Lakatos (1978a), p. 108]). In the code of honour of the falsificationist a theory is scientific only if it can be made to conflict with a basic statement; and a theory must be eliminated if it conflicts with an accepted basic statement. Popper also indicated a further condition that a theory must satisfy in order

page 205

April 29, 2016

9:6

206

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch10

Resonance: From Probability to Epistemology and Back to qualify as scientific: it must predict facts which are novel, that is, unexpected in the light of previous knowledge.

[Lakatos (1978b), Chapter 6], discussed a closely related problem of appraising scientific theories. Lakatos classified the main schools of thought in this area as follows.

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

(a) Skepticism ([Lakatos (1978b), p. 107]). Scepticism regards scientific theories as just one family of beliefs which rank equal, epistemologically, with the thousands of other families of beliefs.

(b) Demarcationism ([Lakatos (1978b), p. 109]). According to ‘demarcationists,’ the products of knowledge can be appraised and compared on the basis of certain universal criteria.

(c) Elitism ([Lakatos (1978b), p. 111]). [...] science can only be judged by case law, and the only judges are the scientists themselves.

Lakatos proposed his own philosophy of science — see Sec. 10.12.1. The following classification of philosophical theories of science is orthogonal, in a sense, to the one given above. I do not strive here for completeness. (1) Constructive empiricism. According to [Wikipedia (2014s)], Constructive empiricism states that scientific theories are semantically literal, that they aim to be empirically adequate, and that their acceptance involves, as belief, only that they are empirically adequate. A theory is empirically adequate if and only if everything that it says about observable entities is true. A theory is semantically literal if and only if the language of the theory is interpreted in such a way that the claims of the theory are either true or false [...].

(2) Scientific Realism. According to Branden Fitelson lectures, Scientific realism is a two-part thesis: 1. Science aims to give us, in its theories [...] a literally true story of what the world is like. 2. Acceptance of a scientific theory involves the belief that it is true.

(3) Instrumentalism. I quote Branden Fitelson’s lectures again, According to instrumentalism, the aim of science is to give us theories which are useful in various ways (e.g., useful for making predictions of certain kinds or for building bridges, etc.).

page 206

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch10

Science

207

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

10.12.1 Real science — the big picture A major trend in the philosophy of science does not fit easily into the classifications described in Sec. 10.12 — a number of philosophers tried to present the big picture of science as it really happened. The best known representative of this trend was Thomas Kuhn. His book [Kuhn (1970)] presented science as periods of “normal science” punctuated by scientific revolutions. While Kuhn’s philosophy is by far most popular, I believe that philosophical theories presented in [Feyerabend (1975)], [Lakatos (1978a)] and [Polanyi (1958)] fit into the same category. Lakatos called his theory the “methodology of scientific research programmes” ([Lakatos (1978a), Chapter 1]). Lakatos’ theory is a sophisticated version of falsificationism based on the idea of problemshift. The theory might be a reasonably accurate picture of science on a big scale. In the “normal science” and everyday life we have to deal with simple observations, simple problems and simple decisions. The theory of Lakatos may apply to science with the big “S” but it is an overkill for the normal science. Lakatos showed a great respect for Popper but in the end I find his theory closer to Kuhn’s in that Lakatos tried to capture the “big picture” of science, on the largest scale. The book [Polanyi (1958)] is titled Personal knowledge but the title may be misinterpreted. Polanyi did not limit his investigation of knowledge acquisition to the level of an individual person. He tried to describe the process in the broad context including even law and religion. I recognize significant contributions coming from Kuhn, Lakatos and Polanyi but I find Lakatos’ theory the least inspiring. Kuhn and Popper are considered the most significant 20th century philosophers of science and their theories are considered to be antipodal. I think that their theories address completely different questions. Consider the scientific claim that copper is a good conductor of electricity. Kuhn may be able to place this claim in the history of science. But this is a remote history and at this point the claim that copper is a good conductor seems to be as obvious as the one that grass is green. Popper tries to determine the basis on which we can place our belief in the conductance properties of copper no matter what historical process brought us to this point. Here, my interpretation of Popper’s philosophy is much more narrow than his own but it is the one that I need in this book. Epistemology can be roughly divided into two trends. One direction is normative, in a sense, as it aims at finding the truth and methods

page 207

April 29, 2016

9:6

208

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch10

Resonance: From Probability to Epistemology and Back

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

of achieving this goal. The other direction is descriptive, that is, it tries to describe how people, especially scientists, arrive at what they believe is the truth. Describing what scientists actually do may seem like a secondary intellectual goal compared to the haughty search for the truth. But describing how science actually works proved extremely hard, especially in the area of statistics. Describing the scientific mode of operation seems to be far more practical than the abstract search for the truth, for example, from the point of view of artificial intelligence research.

10.12.2 Probabilism and Bayesianism in philosophy of science There are attempts in philosophy of science to solve some philosophical problems, for example, the problem of induction, using probabilistic tools. Bayesian analysis seems to be popular in this context. This approach fails to explain the success of science because there are infinitely many candidates for scientific laws. If all of them are assigned equal prior probabilities then the Bayes formula will generate a useless answer. I will present a somewhat silly example to illustrate my claim. From the scientific point of view, a rainbow has infinitely many colors but, in agreement with a tradition going back to Newton, we often list only seven colors: red, orange, yellow, green, blue, indigo and violet. Consider 49 scientific theories concerning the color of grass in the next two days. According to the first theory, the color of grass will be red tomorrow and red the day after tomorrow. According to the second theory, the color of grass will be red tomorrow and orange the day after tomorrow. According to the third theory, the color of grass will be red tomorrow and yellow the day after tomorrow. Continuing in this way, we can construct 49 theories because 7 times 7 is 49. Assume that all these theories are given equal prior probabilities and suppose that we observe that grass is green tomorrow. Then, according to the Bayes theorem, grass may have any of the seven colors the day after tomorrow, all with equal probabilities. The conclusion is hardly palatable. Consider a more practical example. Einstein came up with the idea of relativity. Did it exist in his mind prior to his conscious discovery? If so, what was its prior probability? Did the idea exist in the minds of other physicists? If yes, what was its probability? If not, why not? Why did the Bayesian processing put a high probability on the relativity theory in Einstein’s mind but not in the minds of other physicists? I doubt that we

page 208

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch10

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Science

209

will ever have even remotely convincing answers to these questions. The Bayesian representation of scientific research explains absolutely nothing. I will now make a technical digression for the benefit of readers familiar with basic statistic. Suppose that we want to estimate the unknown mean of the normal distribution with variance 1. In the Bayesian approach, we can start with an improper prior distribution, uniform on the whole real line. Once we collect a few observations from the distribution with the unknown mean, the Bayes theorem will yield a perfectly reasonable posterior (proper) probability distribution for the unknown mean. This seems to indicate that the Bayes theorem can generate reasonable answers even if we start with a uniform distribution on the set of all “scientific theories,” that is, all possible values of the unknown parameter. This is an illusion. The Bayes theorem and the improper prior work well together in the context of mean estimation because the joint prior distribution of the unknown mean and the values of the observations is far from uniform in any reasonable sense. Given a value of the mean, the observations are likely to be close to the mean, according to the prior distribution. I have argued that probabilities assigned to different potential laws of science or to different specific predictions have to be unequal. In other words, some theories of the world have to have higher prior probabilities than others. How should we choose theories and events that deserve higher prior probabilities? This question is essentially equivalent to the original problem of how to find reliable knowledge. We are at square one. The Bayes theorem cannot create something out of nothing. Of course, the Bayes theorem is an excellent way of processing information in the case when the model and the prior distribution are well chosen. My resonance theory explains why we should believe that some models and priors can and in fact are well chosen in practical life. Carnap’s ambitious program was not Bayesian but it seems to have suffered from similar problems. Carnap attempted to find a probabilistic logic which would assign a unique rational and objective probability to a proposition for each knowledge base. This was related to an attempt at constructing a universal language for all of science. Needless to say, the project was hopelessly unrealistic. But even if we accept Carnap’s project as a philosophical rather than practical undertaking, the program seemed to ignore the basic problem of scientific research and perhaps all knowledge creation. One of the fundamental challenges is to state reasonable conjectures about theories and future observations based on the

page 209

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

210

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch10

Resonance: From Probability to Epistemology and Back

current knowledge. In practice, the challenge is tackled with the help of human intelligence, a form of resonance. Carnap obfuscated the role of intelligence by introducing a “degree of confirmation” which appeared to be a purely formal procedure, conceivably implementable in unintelligent computer programs. The same critique applies to Popper’s program. One of the reasons why his program was not realistic is that it stressed logical analysis of existing theories. It mostly ignored the question of how to generate good candidates for scientific theories that could pass falsificationist tests with high probability.

10.12.3 Levels of philosophical analysis In this section I will propose my own classification of different philosophical approaches to science. I will call different categories in the classification “levels.” They are not necessarily ordered in the sense of logical implication or containment. All the levels are legitimate, significant, and illuminating. (i) The lowest level is that of positivism or conventionalism. At this level, it is assumed that we can observe facts and construct theories based on these facts or representing these facts. Popper’s theory belongs to this level. This description of science is appropriate for the task of recognizing science as opposed to non-science. It can represent our human attitude towards artificial intelligence and alien civilizations. We would say that they can do science if they recognize facts the way we do and build theories the same way we do. According to a philosophy of science of this type, a non-human agent, for example, a robot or an alien, can succeed in science and be nominated for the Nobel prize in physics or chemistry. At this level, facts are not analyzed — it is assumed that we can effectively and reliably recognize them. (ii) The second level of analysis of science tries to determine technical and scientific components of any agent capable of doing science. My concept of resonance is an example of an element of such analysis. This concept is not needed at the previous level of analysis. The present level of analysis grapples with questions that could enter the process of design of a robot with artificial intelligence. Theories at this level may contain guesses about how science must work in every conceivable civilization in our universe.

page 210

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch10

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Science

211

(iii) The next level is the analysis of how human science actually works. This is the subject of books by authors as diverse as Feyerabend, Kuhn, Lakatos, and Polanyi. This analysis can be very interesting, deep, incisive and informative. But I am not sure how universal it is. Polanyi points out that emotions and morality (among other things) are inextricable elements of human science. If science of every civilization in this universe works in a similar way then Polanyi’s insight is truly remarkable. But I am not willing to grant this to Polanyi without empirical evidence (which is not likely to arrive in my lifetime or even in foreseeable future). (iv) The last item on my list is the “full” philosophical analysis of science. This type of analysis is not limited in any way. It may include the fundamental questions of truth, knowledge, logic, etc. One of the biggest challenges of this type of analysis is consistency of its skepticism. De Finetti was skeptical about our ability to recognize symmetries but was not skeptical about our ability to recognize facts. In de Finetti’s world, you could recognize five tosses of a coin and three throws of a die. You could tell the difference between a coin and a die and you could recognize operations applied to the same object as symmetric for the purpose of fact recognition but you could not recognize any probabilistic symmetry. This book contains analysis at level (ii) because this level is most relevant to the discussion of the controversies in philosophy of probability. My first book [Burdzy (2009)] belonged to level (i) because it was concerned with the formulation of probability appropriate for scientific applications, where facts were assumed.

10.12.4 Position of my theory in philosophy of science I find it hard to pinpoint the exact place of my epistemological theory in the big scheme of things. The reason is that my motivation is very narrow — I want to determine what part of the knowledge acquisition process may have given rise to the popular feeling that probability is subjective. Hence, I answer only these questions that are related to this narrow project or are byproducts of my main arguments. My theory seems to have some affinity to every philosophical direction listed in (i)–(iii) of Sec. 10.12. I claim that the problem of induction is ill-posed but my theory effectively tries to justify induction so it is an

page 211

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

212

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch10

Resonance: From Probability to Epistemology and Back

unusual form of inductivism. On the other hand, the law (L6) stated Sec. 1.3.3 provides a link to falsificationism but of rather weak nature. The resonance theory stresses the evolutionary character of the roots of our knowledge and this fits well with conventionalism because evolution may guide sentient beings (at the genetic or theoretical level) from one theory to another, depending on the current needs, human abilities, collected facts, etc. Among approaches (a)–(c) listed in Sec. 10.12, I am definitely against (a). I feel the greatest affinity to (b) but my resonance theory may fit best into (c). I did not spend much time trying to pinpoint the place of my theory in the group of theories marked (1)–(3); I would place mine in the category of instrumentalism or empiricism. I feel the greatest affinity with Polanyi’s philosophy, not because I agree with his conclusions but because I like his passionate but fully rational description of personal aspects of science. Polanyi was an accomplished scientist so his description and analysis of science were based on his personal experience, hence highly convincing and reliable. At the same time, I could not agree less with Polanyi’s solution to the problem that he considered: Why can we trust knowledge that is largely personal? His solution stresses personal commitment, is somewhat vague and leads in some twisted way to the Christian faith and God. His book contains only a moderate dose of theology but the last sentence of the book mentions Christianity and the last word of the book is “God.” This could not have been a random choice. In contrast, my theory is an attempt to explain the success of science in scientific terms, despite its personal side. My resonance theory is an amalgam of various ideas and has roots in a number of existing philosophical theories, scientific theories and isolated facts. To give an example of a source of my inspiration, I quote from an article [Wilson (2014)] on John Stewart Mill: [Mill] argues that the rules of scientific method evolve out of the spontaneous inductions about the world that we make as embodied creatures. As we investigate the world to find the best means to satisfy our natural needs and aims, some patterns maintain themselves, others turn out to be false leads. The former guide us in our anticipations of nature, and in our plans; they enable us to infer what will be and what would be if we were to do certain things or if other things were to happen. These patterns that we accept as guides we come to think of as laws: a law is a regularity that we accept for purposes of prediction and contrary-to-fact inference.

page 212

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch10

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Science

213

People always had a feeling that there were different sources of knowledge, not only logical or rational reasoning. In the religious context, revelation and mystical experiences were treated as legitimate sources of the truth. The importance of intuition in scientific research was widely acknowledged. The reader might have noticed that I did not spend any time analyzing and justifying some of my most fundamental assumptions listed in Sec. 3.2. This neglect does not represent my forgetfulness. I delved into epistemology much deeper than I had ever thought I would. I do not believe that an even more careful examination of the foundational issues would make my resonance theory any more meaningful, convincing or attractive.

10.13 Circularity My resonance theory is an attempt to construct a scientific theory of science. The circularity of this reasoning is obvious. But circularity is unavoidable in philosophy. The analysis of deductive logic uses deductive logic. And the only alternative to circularity is infinite regress. A philosophical problem P1 may be analyzed using a method M2 , the method M2 may be analyzed using a method M3 , the method M3 may be analyzed using a method M4 , etc. Either the chain will form a circle at some point (that is Mj = Mk for some j
= k) or there will be an infinite regress of methods. There is another way in which my theory is circular. Recall my claim that induction is a law of nature. Our ability to apply induction is a result of the process of evolution. The process of evolution was discovered by scientists who used induction as their intellectual tool. It is conceivable that the evolution created sentient beings inclined to view the process of evolution in a biased way.

page 213

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch11

Chapter 11

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

The Science of Probability

The science of probability must provide a recipe for assigning probabilities to real events. I will argue that the following list of six “laws of probability” is a good representation of our accumulated knowledge related to probabilistic phenomena and that it is a reasonably accurate representation of the actual applications of probability in science. (L1) Probabilities are numbers between 0 and 1 (including 0 and 1), assigned to events whose outcome may be unknown. (L2) If events A and B cannot happen at the same time then the probability that A or B will occur is the sum of the probabilities of the individual events, that is, P (A or B) = P (A) + P (B). (L3) If events A and B are physically independent then they are independent in the mathematical sense, that is, P (A and B) = P (A)P (B). (L4) If events A and B are symmetric then the two events have equal probabilities, that is, P (A) = P (B). (L5) When an event A is observed then the probability of B changes from P (B) to P (A and B)/P (A). (L6) An event has probability 0 if and only if it cannot occur. An event has probability 1 if and only if it must occur. The laws (L1)–(L6) are implicit in all textbooks, of course. The contents of probability textbooks was selected by a mechanism not unlike Darwinian evolution — the topics most relevant to science established themselves as the core curriculum. I consider it an embarrassment for the scientific community that (L1)–(L6) are presented in textbooks only in an implicit way, while some strange philosophical ideas are presented as the essence of probability. 217

page 217

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

218

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch11

Resonance: From Probability to Epistemology and Back

A system of laws similar to (L1)–(L6) appeared on page 17 of [Ruelle (1991)] but it was missing (L4)–(L6) (see Sec. 11.1.11). In my earlier book, [Burdzy (2009)], I proposed a smaller system of laws of probability (L1)–(L5). The current law (L5) was called (L6) and was discussed separately as a potential addition to the system. I believe that the enlarged system (L1)–(L6) is a more faithful representation of the foundations of probability than the former version. I especially like the following minimalist interpretation of (L1)–(L6). The laws (L1)–(L6) are an account of facts and patterns observed in the past — they are the best compromise (that I could find) between accuracy, objectivity, brevity, and utility in description of the past situations involving uncertainty. The discussion of (L1)–(L6) will be divided into many sections, dealing with various scientific and philosophical aspects of the laws.

11.1 Interpretation of (L1)–(L6) The laws (L1)–(L6) should be easy to understand for anyone who has any experience with probability but nevertheless it is a good idea to spell out a few points.

11.1.1 Events An “event” is something that can be determined to have occurred or not. In general philosophy or philosophy of science, the occurrence of an event would be referred to as a “fact.” This interpretation of an event applies and is fundamental to not only (L1)–(L6) but also to both frequentist and subjectivist approaches to probability. One must be able to determine whether the relevant event occurred to be able to apply (L6). In the frequency approach, one needs to determine frequencies, so one has to be able to determine which events in a sequence occurred. In the subjectivist approach, the stress is on the Bayes formula. This is a special case of conditioning so we have to be able to tell which events (conditions) occurred. Sometimes it is hard to determine whether an event occurred or not. To deal with this problem, we often use random variables instead of events. Consider, for example, the event that tomorrow will be a hot day. The notion of “hot” is quite useful in everyday life but not very useful in science. Instead, scientists use temperature, a physical quantity represented

page 218

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch11

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

The Science of Probability

219

by a “real” number. Tomorrow’s temperature is a random variable. On the technical side, random variables can be reduced to events in the following sense. Saying that a random variable X is equal to a is equivalent to saying that events {X ≤ b} occurred for all b ≥ a and they did not occur for any b < a. It is easy to object to the claim that an event must occur or not — life is much more complicated. Yes, but an idealization is what we need as long as it is helpful. Some readers may be surprised that I spend so much time discussing events. The reason is that in the logical philosophy of probability, probability values are assigned to propositions. This idea never gained any traction in the scientific community and this may be one of the reasons why the logical theory of probability is almost unknown among scientists. Scientists do not have patience and use for the pedantic distinction between the event that an apple fell from a tree and the proposition “an apple fell from a tree.”

11.1.2 Symmetry The word “symmetry” should be understood as the invariance under any transformation preserving the structure of the outcome space (model) and its relation to the outside world. Although I have just attempted to define “symmetry,” I doubt that any such definition has any value because the concept of symmetry cannot be reduced to simpler concepts. If someone does not understand the concept of symmetry, there is no reason to believe that he would understand the concept of “invariance” used in my definition. The mirror symmetry (left and right hands are symmetric in this sense) is an elementary example of non-probabilistic symmetry. A simple example of (L4) is the assertion that if you toss a coin (once) then the probability of heads is 1/2. The classical definition of probability (see Sec. 2.1) is often applied in situations that involve physical and spatial symmetries, for example, games using dice, playing cards, etc. From the statistical point of view, the most important examples to which (L4) applies are sequences of i.i.d. (independent, identically distributed) events or random variables and exchangeable events or random variables (see Chapter 18 for definitions). When observations are ordered chronologically, the i.i.d. property and exchangeability can be thought of as symmetries in time. Our ability to recognize symmetries relevant to probability is the primary example of resonance in this book. This ability is partly innate,

page 219

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

220

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch11

Resonance: From Probability to Epistemology and Back

that is, the result of evolution. It is partly taught by example. Without our ability to recognize probabilistic symmetries science would collapse. It is fundamentally important to realize that (L4) does not refer to the symmetry in a gap in our knowledge but it refers to the physical (scientific) symmetry in the problem. This point was made as early as in 1854 by Boole who thought that our ignorance was not a good justification for assigning equal probabilities to events (see [Galavotti (2005), p. 141]). We know that the ordering of the results of two tosses of a deformed coin does not affect the results. But we do not know how the asymmetry of the coin will affect the result of an individual toss. Hence, if we toss a deformed coin twice, and T and H stand for “tails” and “heads,” then T H and HT have equal probabilities according to (L4), but T T and HH do not necessarily have the same probabilities. I will discuss this example in greater depth in Sec. 11.12. The above remark about the proper application of (L4) is closely related to the perennial discussion of whether the use of the “uniform” distribution can be justified in situations when we do not have any information. In other words, does the uniform distribution properly formalize the idea of the total lack of information? The short answer is “no.” The laws (L1)–(L6) formalize the best practices when some information is available and have nothing to say when there is no information available. I will try to explain this in more detail. A quantity has the “uniform probability distribution” on [0, 1] if its value is equally likely to be in any interval of the same length, for example, it is equally likely that the quantity is in any of the intervals (0.1, 0.2), (0.25, 0.35) and (0.85, 0.95). Some random quantities can take values in an interval, for example, the percentage of vinegar in a mixture of vinegar and water can take values between 0% and 100%, that is, in the interval [0, 1]. If we have a sample of vinegar solution in water and we do not know how it was prepared, there is no symmetry that would map the percentage of vinegar in the interval (0.25, 0.35) onto the interval (0.85, 0.95). In this case, (L4) does not support the use of the uniform distribution. If we record the time a phone call is received at an office with the accuracy of 0.1 seconds then the number of seconds the call was made after the last whole minute is between 0 and 59.9. The time when a phone call is made is close to being stationary (that is, invariant under time shifts) on time intervals of order of one hour. We can use (L4) to conclude that the number of seconds after the last whole minute when a phone call is received is uniformly distributed between 0 and 59.9.

page 220

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch11

The Science of Probability

221

The uniform distribution can be used as a “seed” in a Bayesian iterative algorithm that generates objectively verifiable predictions. This does not imply that the probabilities described by the uniform prior distribution are objectively true. See Sec. 14.4.2 for further discussion of this point.

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

11.1.3 Enforcement The laws (L1)–(L6) are enforced in science and in the life of the society. They are enforced not only in the positive sense but also in the negative sense. A statistician cannot combine two unrelated sets of data, say, on blood pressure and supernova brightness, into one sequence. She has to realize that the combined sequence is not exchangeable, that is, not symmetric. People are required to recognize events with probabilities far from 1 and 0. For example, people are required to recognize that drowning in a swimming pool unprotected by a fence has a non-negligible probability. They have to take suitable precautions, for example, they should fence the pool. Equally simple examples show that people are supposed to recognize both independent and dependent events.

11.1.4 Limits of applicability An implicit message in (L1)–(L6) is that there exist situations in which one cannot assign probabilities in a scientific way. Actually, laws (L1)–(L6) do not say how to assign values to probabilities, they only specify some conditions that the probabilities must satisfy. Only in some cases, such as seven tosses of a symmetric coin, (L1)–(L6) uniquely determine probabilities of all events. It is quite popular to see the philosophy of probability as an investigation of rational behavior in situations when our knowledge is incomplete. The essence of the science of probability, as embodied in (L1)–(L6), is to present the rules of rational behavior when some information is available. In other words, the science of probability is not trying to create something out of nothing, but delineates the boundaries of what can be rationally asserted and verified in situations when some information is available. The logical, subjective and frequency philosophies and Kolmogorov’s axioms contain a misleading implicit message that one has to know probabilities of all events in the sample space to be able to proceed. In fact, knowledge of some restrictions on probability values is enough to generate useful predictions. For example, it is not necessary to know the probability

page 221

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

222

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch11

Resonance: From Probability to Epistemology and Back

of heads for a deformed coin to be able to generate a useful prediction that the relative frequencies of heads in two long sequences of tosses of the same deformed coin will be very close to each other. Of course, predictions for deformed coins are not particularly useful, but the same idea applies to truly useful predictions, such as the rate of side effects for a drug. Laws (L1)–(L6) can be applied only when we are able to apply resonance to recognize the relevant properties of events such as symmetry or physical independence. We also need resonance to determine whether an event occurred or not. When need arises, one may replace direct resonance with a method that relies on a lower level resonance or an alternative resonance (see Sec. 8.2.5).

11.1.5 (L1)–(L6) as a starting point The relationship between (L1)–(L6) and the real probabilistic and statistical models is analogous to the relationship of, say, Maxwell’s equations for electromagnetic fields and a blueprint for a radio antenna. The laws (L1)–(L6) are supposed to be the common denominator for a great variety of methods, but there is no presumption that it should be trivial to derive popular models, such as linear regression in statistics or geometric Brownian motion in finance, from (L1) to (L6) alone. One may find other useful conditions for probabilities besides (L1)–(L6) in some specific situations but none of those extra relations seems to be as fundamental or general as (L1)–(L6). Some widely used procedures for assigning probabilities are not formalized within (L1)–(L6). These laws are similar to the periodic table of elements in chemistry — a useful and short summary of some basic information, with no ambition for being exhaustive. Consider the following textbook example from frequency statistics. Suppose that we have a sequence of i.i.d. normal random variables with unknown mean and variance equal to one. What is the best estimator of the unknown mean? This model involves “normal” random variables so for this mathematical model to be applicable, one has to recognize in practice normal random variables. There are at least two possible approaches to this practical task. First, one can try to determine whether the real data are normally distributed using resonance (common sense or intuition). Alternatively, one can test the data for normality in a formal way, using (L1)–(L6). The last option is more

page 222

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch11

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

The Science of Probability

223

scientific in spirit but it is not always practical — the amount of available data might be too small to determine in a convincing way whether the measurements are indeed normal. A simple model for stock prices provides another example of a practical situation when probabilities are assigned in a way that does not seem to follow directly from (L1) to (L6). A martingale is a process that has no overall tendency to go up or down. According to some financial models, stock prices are martingales. The reason is that the stock price is the current best guess of the value of the stock price at some future time, say, at the end of the calendar year. According to the same theory, the current guess has to be the conditional expectation of the future price given the current information. Then a mathematical theorem shows that the stock price has to be a “Doob martingale.” In fact, nobody seems to believe that stock prices are martingales. Nevertheless, this oversimplified model makes a prediction that stock prices should be non-differentiable functions of time. This is well supported by the empirical evidence. My point is that the martingale-like properties of stock prices would be hard to derive from (L1)–(L6) in a direct way. Finally, let me mention the Schr¨ odinger equation — the basis of quantum mechanics. A solution to the equation can be interpreted as a probability distribution. The laws (L1)–(L6) were involved in the experimental research preceding the formulation of the Schr¨ odinger equation but I do not see how the probabilities generated by the equation can be derived from (L1) to (L6) alone.

11.1.6 Approximate probabilities All of science is an approximate match between theories (quite often mathematical theories or formulas) and reality. The science of probability is not any different. The laws (L1)–(L6) will never be matched perfectly with real phenomena. That much is quite obvious. But even very crude probability estimates can be useful for some practical purposes. For example, it may happen that the knowledge of approximate symmetry is sufficient. It may suffice to know that the probability of an event is between 40% and 60%. If the corresponding experiments or observations can be repeated then the long run frequency may verify or falsify the claim about the approximate value of the probability of the event.

page 223

April 29, 2016

9:6

224

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch11

Resonance: From Probability to Epistemology and Back

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

11.1.7 Statistical models Many statistical models and, more generally, probabilistic scientific models are examples of “higher level” resonance. Some simple models are almost direct implementations of (L1)–(L6). Constructing such models requires only the basic level of resonance, that is, skills that are probably innate. Some other models are very complex. The ability to conjecture a good complex model is an acquired skill. The model must be sufficiently realistic to generate probabilities that are reasonably reliable. It must be also sufficiently simple so that it is tractable using currently known mathematical methods and computer simulations. Statistics deals with situations when the model is only partly specified by resonance and logic. Its unspecified parts are represented as unknown “parameters.” These can be determined, with some accuracy, from the data. Traditionally, prior probabilities in Bayesian statistics had the reputation for being subjective but nobody seemed to have raised a similar objection in the case of models, Bayesian or frequentist. Paradoxically, the resonance theory indicates that models are more “subjective” than (some) prior distributions. Resonance is not subjective in any sense discussed in Sec. 5.23 except for interpretation (xiii). Resonance is a mental process which takes sensory data and generates an action or a belief. This process is subconscious so it cannot be directly verified for correctness no matter how one understands “correctness.” Hence, the value of the results of resonance has to be ascertained in some other way. One may compare proposals obtained via resonance by different people and use consensus as a substitute for objective truth. One may observe the consequences of actions based on resonance and accept or reject the theory implicit in the actions. By comparison, many Bayesian priors are “objective” because they require no resonance. For example, consider estimating the probability of heads for a deformed coin (“deformed coin” should be understood as a toy model representing many truly practical situations). A standard Bayesian model for this statistical problem is to assume that the sequence of tosses is exchangeable and, because of de Finetti’s theorem, it is a mixture of i.i.d. sequences. The prior distribution of the unknown probability of heads can be taken to be uniform because this gives a very reasonable posterior distribution no matter what the “true” probability of heads is (I put “true” in the quotation marks because only objectivists believe that the probability of heads is an objective number). Note that a group of Bayesian statisticians may agree on using the uniform prior for this particular model under all

page 224

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch11

The Science of Probability

225

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

circumstances, even if they have some personal feelings about the coin. In this case, the prior distribution is fixed in a way that has some scientific justification. But the main point is that the choice of this prior distribution and its justification are fully accessible to conscious analysis and in this sense this particular prior distribution, or rather the agreement to use it, is objective.

11.1.8 The Bayes theorem The law (L5) is a form of the Bayes theorem. The reader familiar with the classical version of the Bayes formula (18.1) may be surprised by my claim because (L5) is usually written as P (B | A) = P (A and B)/P (A) and is considered to be the definition of conditional probability. The last formula is indeed the mathematical definition of conditional probability. But (L5) is a law of science and it says that the conditional probability is a mathematical quantity that represents real life probability when we acquire new information. The classical Bayes theorem (18.1) is just one of many mathematical ways of calculating conditional probabilities. Of course, (18.1) and its various variants play the crucial role in Bayesian statistics. But I believe that (L5) is the essence of the Bayes theorem in the scientific and philosophical sense.

11.1.9 Probability of past events Laws (L1)–(L6) make no distinction between events that will happen in the future and events that happened in the past. Probability can be meaningfully attributed to an event if that event can be eventually determined (at least in principle) to have occurred or not. For example, it makes sense to talk about the probability that a scientific theory is correct, the probability that there was life on Mars and the probability that dramas attributed to Shakespeare were indeed written by Shakespeare. In each case someone can make a prediction, that is, make a claim that the probability in question is very close to 0 or 1. In each case it is conceivable that we will acquire new information at some point that will settle the matter beyond reasonable doubt. In this way, probabilistic predictions related to these examples can be falsified. Hence, it makes sense to talk about the probability of a scientific theory or a past event.

page 225

April 29, 2016

9:6

226

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch11

Resonance: From Probability to Epistemology and Back

11.1.10 Purely mathematical independence

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

In relation to (L3), one should note that there exist pairs of events which are not “physically independent” but independent in the mathematical sense. If you roll a fair die, the event A that the number of dots is even is independent from the event B that the number is a multiple of 3, because P (A and B) = 1/6 = 1/2 · 1/3 = P (A)P (B).

11.1.11 Ruelle’s view of probability I have already mentioned that David Ruelle gave in Chapter 3 of [Ruelle (1991)] a list of probability laws similar to (L1)–(L6), but missing (L4)– (L6). The absence of the last three laws from that system makes it significantly different from mine. However, this observation should not be interpreted as a criticism of Ruelle’s list — he was not trying to develop a complete scientific codification of probability laws. At the end of Chapter 3 of his book, Ruelle gave a frequency interpretation of probability. In the case when the frequency cannot be observed in real life, Ruelle suggested that computer simulations can serve as a scientific substitute. This approach does not address the classical problem of determining the probability of a single event belonging to two different sequences — see Sec. 11.10. As for computer simulations, they are a great scientific tool but they seem to contribute little on the philosophical side — see Sec. 4.9.

11.2 Scientific Verification of (L1)–(L6) A major difference between probability theory and deterministic branches of science lies in the different method of verifying its assertions. We assign values to probabilities just like we find a value of any scientific quantity. At the most elementary level we use resonance directly to determine probabilities. We may use innate resonance or learned resonance. At the scientific level, probabilities are typically determined via a logical process based on information (acquired via resonance). From time to time specific probability values and probabilistic theories have to be verified to ascertain the reliability of our methods of finding probabilities. The verification process in probability theory is based on (L6). I will call a claim which gives an event a probability close to 1, a prediction. One can scientifically verify probabilistic predictions. One cannot verify values of probabilities which are far from 0 and 1 in a direct way but there are practical ways to deal with this problem, described later in this section.

page 226

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch11

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

The Science of Probability

227

The fact that only some probability values (0 and 1) can be directly verified does not diminish the value of the probability theory. In practice, this does not stop scientists from determining values of probabilities far from 0 and 1 and verifying theoretical formulas. Probability is not the only science which uses verification methods different from those of traditional deterministic sciences. Quantum mechanics is an obvious example of a field with similar epistemological challenges. The difficulties with verification inherent in quantum mechanics are embodied in a number of laws and have popular-science representations: the Heisenberg uncertainty principle ([Wikipedia (2015j)]), the wave function collapse ([Wikipedia (2015k)]), the double slit experiment ([Wikipedia (2015h)]), and the Schr¨ odinger cat ([Wikipedia (2015i)]). And, of course, the probabilistic interpretation of the wave function. I wonder if any philosopher ever claimed that “the wave function does not exist.” Before I turn to the question of the scientific verification of (L1)–(L6), I will discuss the idea of a scientific proof. The following is a simplified version of the discussion in [Lakatos (1978a)]. The idea of a “proof” in mathematics is this: you start with a small set of axioms and then you use a long chain of logical deductions to arrive at a statement that you consider interesting, elegant or important. Then you say that the statement has been proved. Physicists have a different idea of a “proof” — you start with a large number of unrelated assumptions, you combine them into a single prediction, and you check if the prediction agrees with the observed data. If the agreement is within 20%, you call the assumptions proved. The procedure for verification of (L1)–(L6) that I advocate resembles very much the “physics’ proof.” Consider a real system and assign probabilities to various events using (L1)–(L5), before observing any of these events. Then use the mathematical theory of probability to find an event A with probability very close to 1 and make a prediction that the event A will occur. The occurrence of A can be treated as a confirmation of the assignment of probabilities and its non-occurrence can be considered its falsification. A very popular scientific method of verifying probability statements is based on repeated trials — this method is a special case of the general verification procedure described above. It has the same intuitive roots as the frequency theory of probability. Suppose that A is an event and we want to verify the claim that P (A) = 0.7. Then, if practical circumstances allow, we find events A1 , A2 , . . . , An such that n is large and the events

page 227

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

228

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch11

Resonance: From Probability to Epistemology and Back

A, A1 , A2 , . . . , An are i.i.d. Here, “finding events” means designing an experiment with repeated measurements or finding an opportunity to make repeated observations. Let me emphasize that finding repeated observations cannot be taken for granted. The mathematics of probability says that if P (A) = 0.7 and A, A1 , A2 , . . . , An are i.i.d. then the observed relative frequency of the event in the whole sequence will be very close to 70%, with very high probability. If the observed frequency is indeed close to 70%, this can be considered a proof of both assertions: P (A) = 0.7 and A, A1 , A2 , . . . , An are i.i.d. Otherwise, one typically concludes that the probability of A is different from 0.7, although in some circumstances one may instead reject the assumption that A, A1 , A2 , . . . , An are i.i.d. Recall the discussion of the two interpretations of the “proof,” the mathematical one and the physical one. Traditionally, the philosophy of probability concerned itself with the verification of probability statements in the spirit of the mathematical proof. One needs to take the physics’ attitude when it comes to the verification of probability assignments based on (L1), (L6), or (L1)–(L6) themselves — it is not only the probability statements but also assumptions about symmetries or lack of physical influence that can be falsified. The general verification method described above works at (at least) two levels. It is normally used to verify specific probability assignments or relations. However, the combined effect of numerous instances of application of this procedure constitutes a verification of the whole theory, that is, the laws (L1)–(L6). The method of verification of (L1)–(L6) described above works only in the approximate sense, for practical and fundamental reasons — no events in the universe are absolutely “physically independent,” no symmetry is perfect, mathematical calculations usually do not yield interesting events with probabilities exactly equal to 1, and the events of probability “very close” to 1 occur “almost” always, not always. The actual implementation of experiments or observations designed to verify (L1)–(L6) is superfluous, except for didactic reasons. Scientists accumulated an enormous amount of data over the centuries and if someone thinks that the existing data do not provide a convincing support for (L1)–(L6) then there is little hope that any additional experiments or observations would make any difference. Since Popper was the creator and champion of the propensity theory of probability, one may reach a false conclusion that his idea incorporated

page 228

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch11

The Science of Probability

229

in (L6) turns (L1)–(L6) into a version of propensity theory of probability. In fact, (L1)–(L6) make no claims about the true nature of probability, just like Newton’s laws of motion do not make any claims about the nature of mass or force.

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

11.3 Predictions I attribute the success of probability theory and statistics to their ability to generate predictions as good as those of deterministic sciences. In practice, no deterministic prediction is certain to occur, for various reasons, such as human errors, natural disasters, oversimplified models, limited accuracy of measurements, etc. Some predictions offered by the probability theory in “evidently random” experiments, such as repeated coin tosses, are much more reliable than a typical “deterministic” prediction. Law (L6) refers to events of probability 0 or 1. In the context of probabilistic phenomena, practically no interesting events have such probabilities — this is almost a tautology. However, there exist many important events whose probabilities are very close to 0 or 1. In other words, the only non-trivial applications of (L6), as an account of the past observations or as a prediction of future events, are approximate in nature. One may wonder whether this undermines the validity of (L6) as a law of science. I believe that (L6) does not pose a philosophical problem any deeper than that posed by the concept of “water.” There is no pure water anywhere in nature or in any laboratory and nobody tries to set a universal level of purity for a substance so that it can be called “water” — this is done as needed in scientific and everyday applications. The concept of temperature applies to human-size bodies, star-size bodies and bacteria-size bodies but it does not apply to individual atoms. It does not apply to molecules consisting of three atoms. The temperature of an atom is not a useful concept, and the same applies to the temperature of a three-atom molecule. How many atoms should a body have so that we can talk about its temperature? As far as I know, the critical number of atoms was never defined. Moreover, doing so would not contribute anything to science. Similarly, it would not be useful to set a number close to 1 and declare that a probabilistic statement is a prediction if and only if it involves a probability greater than that number. I note parenthetically that the same remarks apply to von Mises’ concept of a “collective.” In principle, the concept refers to an infinite sequence. In practice, all sequences are

page 229

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

230

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch11

Resonance: From Probability to Epistemology and Back

finite. Trying to declare how long a sequence has to be so that it can be considered a “collective” would not contribute anything to science. I define a prediction as an event which has probability “very close to 1.” This definition may raise some philosophical concerns: (i) a “probabilistic prediction” is not a prediction at all, and (ii) the definition of a prediction is vague and so “prediction” means different things to different people. These concerns are legitimate so, from the purely philosophical point of view, one could reject the concept of a probabilistic prediction because of (i) and (ii). However, no science has equally high intellectual standards. If we accept the standards implicit in (i) and (ii) then social sciences and humanities are almost worthless intellectual endeavors. Moreover, even natural sciences do not offer, in practice, anything more solid than probabilistic predictions (although they do in theory). If the high standards hinted at in (i) and (ii) are adopted, then we will have to reject quantum physics, an inherently probabilistic field of science, because it makes no predictions at all.

11.3.1 Predictions at various reliability levels The statement of law (L6), “events of probability zero cannot happen,” does not distinguish between small but significantly different probability values, say, 0.001 and 10−100 . This poses philosophical and practical problems. A person reporting the failure of a prediction to another person does not convey a clear piece of information if the probability of the event was not specified. I believe that a crude rule is needed to make fine distinctions. I will elaborate on this idea using a deterministic example. Consider the concepts of “black” and “white.” The ability to distinguish between black and white is needed in everyday life and science. Not all white objects are equally white, for example, not all white pieces of paper are equally bright. Hence, one could argue that the concept of “white” is insufficiently accurate to be acceptable in science. There are at least two answers to this philosophical problem. The first one is that the crude concept of “white” is sufficiently accurate to be useful. The second answer is more delicate. One can measure degrees of whiteness but the measurement process hinges on the human ability (resonance) to distinguish between black and white in a crude way. Imagine a very precise instrument measuring the brightness of “white” paper. The result of the measurement can be displayed using a traditional gauge with a black arrow on the white background or on the modern computer screen, using black digits on the white background. Hence our ability to measure

page 230

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch11

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

The Science of Probability

231

brightness of the paper with great accuracy depends on our ability to read the gauge or the numbers on the computer screen. This in turn depends on our ability to distinguish between white and black in a crude way. In other words, a fine distinction between degrees of whiteness depends on the crude distinction between white and black. Measuring physical quantities with great accuracy or measuring extreme values of these quantities is possible but it usually requires sophisticated scientific theories and superb engineering skills. For example, measuring temperatures within a fraction of a degree from absolute zero, or the (indirect) measurement of the temperature at the center of the Sun require sophisticated theories and equipment. Similarly, to measure very small probabilities with great accuracy, one needs either sophisticated theories, or excellent data, or both. In theory, we could use relative frequency to estimate a probability of the order of 10−1,000,000 , but I doubt that we will have the technology to implement this idea any time soon. The only practical way to determine a truly small probability is first to find a good model for the phenomenon under consideration using observations and statistical analysis, and then apply a theorem such as the Large Deviations Principle (see Sec. 18.1.1). The statistical analysis needed in this process involves an application of (L6) in a simple and crude form. For example, one has to reject the possibility that the observed patterns in the data were all created by a faulty computer program. This is effectively saying that an event of a small probability, a computer bug, did not happen. Typically, we do not try to determine the order of magnitude of this event in a formal or accurate way — a simple and rough application of (L6) seems to be sufficient and constitutes a part of a very accurate measurement of very small probability. The above remarks on the relationship between the rough law (L6) and accurate scientific predictions apply also to other elements of the system (L1)–(L6). We have to recognize symmetries in a rough way, via simple resonance, to apply probability theory and all other scientific theories. Sometimes this is not sufficient, so scientists measure various physical quantities with great accuracy to determine, among other things, whether various quantities are identical (symmetric). Similarly, applications of probability require that we recognize independent events in a rough way, which is again a case of low level resonance. In some situations this is not sufficient and statisticians measure correlation (a number characterizing the degree of dependence) in an accurate way.

page 231

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

232

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch11

Resonance: From Probability to Epistemology and Back

It is good to keep in mind some examples of events that function in science as predictions. In the context of hypothesis testing (see Sec. 13.3), events with probability 0.95 are treated quite often as predictions. In other words, the “significance level” can be chosen to be 0.05. If an event with probability 0.05 occurs then this is considered to be a falsification of the underlying theory, that is, the “null hypothesis” is rejected. At the other extreme, we have the following prediction involving a frequency. If we toss a fair coin 10,000 times, the probability that the observed relative frequency of heads will be between 0.47 and 0.53 is about 0.999999998. This number and 0.95 are vastly different so it is no surprise that the concept of prediction is not easy to recognize as a unifying idea for diverse probabilistic and statistical models.

11.3.2 Predictions in existing scientific and philosophical theories Predictions are well known in the subjective philosophy, Bayesian statistics, and frequency statistics. The subjective philosophy and Bayesian statistics agree on decision theoretic consequences of dealing with an event which has very high probability. When we calculate the expected value of utility related to a “prediction,” that is, an event of very high probability, the expected value will be very close to that if the event had the probability equal to 1. Hence, the decision maker should make the same decision, no matter whether the probability of the event is very close to 1, or it is exactly equal to 1. In other words, in practice, probabilistic “predictions” are treated in the subjective philosophy and Bayesian statistics in the same way as deterministic predictions. There is a fundamentally important difference, though, between probabilistic and deterministic predictions in the subjective philosophy. If a deterministic event predicted by some theory did not occur in reality, all people and all theories seem to agree that something must have gone wrong — either the theory was false or its implementation was erroneous. The subjective philosophy does not grant probabilistic predictions (that is, events of high probability) any special philosophical status, relative to events with moderate probabilities. If you believe that you will win a lottery with probability 99% and you do not win, the subjective theory has no advice on what you should do, except to stay consistent. You may choose to believe that next time you will win the same lottery with probability 99%.

page 232

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch11

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

The Science of Probability

233

An important branch of frequency statistics is concerned with hypothesis testing. A hypothesis is rejected if, assuming that the hypothesis is true, the probability that a certain event occurs is very small, and nevertheless the event does occur. In other words, a frequency statistician makes a prediction, perversely hoping that it will fail, and if indeed the predicted event does not occur, the statistician concludes that the assumptions on which the prediction was based must have been false. The relationship between (L6) and the formal theory of hypothesis testing is similar to that between crude informal measurements and high accuracy scientific measurements described earlier in this section. The sophisticated scientific theory is needed for advanced scientific applications but it is based on a crude principle (L6) at its foundations.

11.3.3 Predictions, conditioning and hypothesis tests The law (L6) is close in spirit to Fisher’s interpretation of hypothesis testing. In his approach, if an event predicted by a model does not occur, we reject the model. Both branches of statistics, frequency and Bayesian, seem to suggest that this method of dealing with unlikely events is too crude. On the frequency side, the Neyman–Pearson theory of hypothesis testing stresses the need for an alternative hypothesis. In other words, we reject the null hypothesis (the theory that makes the prediction) if the observation falls into a certain region defined by both hypotheses, null and alternative. In the Bayesian approach, nothing is ever rejected. An observation of a failed prediction simply modifies our probability distribution, that is, an application of the Bayes theorem generates the posterior distribution from the prior distribution and the observation of the failed prediction. I believe that the Neyman–Pearson hypothesis testing and the Bayesian approach are meaningful only if we have a good alternative theory, generated by resonance. If the alternative hypothesis is chosen in either theory in an arbitrary way then there is no reason to think that the Neyman–Pearson or Bayesian methods will yield a result that is useful.

11.3.4 Prediction examples I will present a few examples of predictions. I will start by discussing long run frequencies because this is undoubtedly the most fundamental example of a real life application of a prediction. I have already mentioned that if we toss a fair coin 10,000 times then the probability that the observed relative

page 233

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

234

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch11

Resonance: From Probability to Epistemology and Back

frequency of heads will be between 0.47 and 0.53 is about 0.999999998. The close relationship between frequencies and probabilities is so profoundly embedded in the scientific psyche that many people fail to notice that this relationship is just one of many practical cases of predictions. In many situations what matters is not the long run average but a catastrophically large value of a process. The process may be i.i.d. or have i.i.d. increments or it may have a completely different structure. Examples of catastrophically large values of random variables include large rainfall resulting in flooding and high velocity of wind during hurricanes. Both natural phenomena cause huge losses for insurance companies. In this context, predictions say that catastrophically large values of the relevant quantities are very unlikely. In other cases, what matters is not a particularly large value of a random variable but a complex catastrophic event such as a nuclear plant meltdown. This event may be related to unusually large values of some quantities, such as the level of radiation in the air, but reducing the catastrophe to these large values seems to be missing the point. A prediction related to such an event is a claim that the catastrophe has a very small probability. Some of the most significant and practical examples of probabilistic predictions are never represented as such in everyday life. Everyone knows that an egg will never unscramble itself and water placed on a hot stove will not freeze. Yet these events are not impossible — according to modern physics they are merely highly improbable. There are, roughly speaking, two sources of randomness at the microscopic level that often appear to generate deterministic phenomena on the macroscopic level — statistical physics and quantum physics. The egg and water examples illustrate some statistical physics effects. Tunneling diodes are quantum mechanical devices in which electrons jump (“tunnel”) over a seemingly insurmountable barrier. On a macroscopic scale, a cat could “tunnel” through a door but this event is extremely unlikely to occur so people install cat doors. The next section contains more examples of predictions.

11.3.5 Histograms and image reconstruction The rise of computers and their ability to generate pictures representing data illustrates predictions in a far more interesting way than the classical predictions of stable frequencies. A popular example of graphical data representation is a histogram of the empirical distribution (more precisely,

page 234

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch11

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

The Science of Probability

235

its density). The shape of the graph is a prediction. For example, if the empirical distribution comes from the normal distribution, the graph will have the characteristic bell shape with high probability. A more exotic example is that of fractional Brownian motion. Figures 4.1–4.3 show simulations of fractional Brownian motion with three different values of the “Hurst parameter.” The graph becomes smoother as the parameter becomes larger. One look at the graphs in Figures 4.1 and 4.3 is all one needs to tell which of the two processes has a larger Hurst parameter. The roughness of the graph is a prediction — it occurs with very high probability, not certainty. Sometimes photographs are transmitted through noisy channels, for example, when they are sent from space. Different techniques of reconstruction make the photographs look much better to human eye; see an example in Figure 11.1. The prediction here is that the reconstructed photograph is very close to the true representation of the object with very high probability. The idea that the reconstructed image is “very close” to the original can be

Fig. 11.1 The picture on the left is a photograph corrupted by placing 10,000 white pixels in a random (i.i.d. uniform) fashion. The corrupted photograph was reconstructed using the “MedianFilter” Mathematica function. The result is on the right.

page 235

April 29, 2016

9:6

236

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch11

Resonance: From Probability to Epistemology and Back

expressed in rigorous mathematical terms but in practice it means that the objects in the reconstructed photograph are easily recognizable by humans.

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

11.3.6 Contradictory predictions In a practical situation, two rational people may disagree about the value of a probability, for example, opinions of two political scientists about chances that a given presidential candidate has in the next elections may differ. If these probability estimates are 40% and 60%, there seems to be no reason to think that either opinion is irrational. Hence, one may conclude that probability is subjective in the sense that perfectly rational people may have reasonably well justified but different opinions. The attitude to contradictory probabilistic claims is different when they are predictions, that is, if the pundits assign very high probabilities to their claims. Imagine, for example, that one political commentator says that a candidate has 99.9% probability of winning elections, and another commentator gives only 0.1% chance to the same candidate. Only a small fraction of people would take the view that there is nothing wrong with the two opinions because “rational people may have different opinions.” The mathematical theory of probability comes to the rescue, in the form of a theorem proved in Sec. 18.4. The theorem says that people who base their opinions on the standard mathematical theory of probability are unlikely to make contradictory predictions even if they have different information sources. More precisely, suppose that two people consider an event A. Assume that each person knows some facts unknown to the other person. Let us say that a person makes a prediction when she says that either an event A or its complement is a prediction, or, more precisely, the probability of A is either smaller than δ or greater than 1 − δ, where δ > 0 is a small number, chosen to reflect the desired level of confidence. The two people make “contradictory predictions” if one of them asserts that the probability of A is less than δ and the other one says that the probability of A is greater than 1 − δ. The theorem in Sec. 18.4 says that the two people can make the probability of making contradictory predictions smaller than an arbitrarily small number 2δ > 0, if they agree on 1 − δ as the probability value which turns a probabilistic claim into a prediction. This result may be interpreted as saying that, at the operational level, predictions can be made objective, if people choose to cooperate. This interpretation may be even reconciled with the belief that opinions about moderate probabilities are subjective. It is best not to overestimate the philosophical or scientific significance

page 236

May 10, 2016

13:19

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch11

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

The Science of Probability

237

Fig. 11.2 Rain forecasts provided by Bing Weather for Seattle, WA, USA. According to forecasts made on July 22, 2014, by WDT (Weather Decision Technologies) and Foreca, the probability of rain on July 24, 2014 was 0% and 95%, resp.

of the theorem in Sec. 18.4, but I have to say that I find it reassuring. On the negative side, the theorem appears to be somewhat circular. The assertion of the theorem, that the probability of contradictory predictions is small, is itself a prediction. Hence, the theorem is most likely to appeal to the converted, that is, people who already have a positive attitude toward predictions, similar to mine. The reader may be amused by the radically different rain forecasts in Figure 11.2. Similarly radically different forecasts occur quite often — I did not have to wait for many days to take a computer screen snapshot illustrating my point. It is possible that the “probability of rain” is interpreted in different ways by different companies. Below is a quote from the explanation provided by the National Weather Service/NOAA, Department of Commerce. The probability of precipitation forecast is one of the most least understood elements of the weather forecast. The probability of precipitation has the following features: ..... The likelihood of occurrence of precipitation is stated as a percentage ..... A measurable amount is defined as 0.01” (one hundredth of an inch) or more (usually produces enough runoff for puddles to form)

page 237

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

238

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch11

Resonance: From Probability to Epistemology and Back ..... The measurement is of liquid precipitation or the water equivalent of frozen precipitation ..... The probability is for a specified time period (i.e., today, this afternoon, tonight, Thursday) ..... The probability forecast is for any given point in the forecast area To summarize, the probability of precipitation is simply a statistical probability of 0.01 inch or more of precipitation in the given forecast area in the time period specified. Using a 40% probability of rain as an example, it does not mean (1) that 40% of the area will be covered by precipitation at given time in the given forecast area or (2) that you will be seeing precipitation 40% of the time in the given forecast area for the given forecast time period. Here are two examples giving the same statistical result: (1) If the forecaster was 80% certain that rain would develop but only expected to cover 50% of the forecast area, then the forecast would read “a 40% chance of rain” for any given location. (2) If the forecaster expected a widespread area of precipitation with 100% coverage to approach, but he/she was only 40% certain that it would reach the forecast area, this would, as well, result in a “40% chance of rain” at any given location in the forecast area.

11.3.7 Multiple predictions When it comes to deterministic predictions, all predictions are supposed to hold simultaneously so the failure of a single prediction may falsify a whole theory. By nature, probabilistic predictions may fail even if the underlying theory is correct. If a large number of predictions are made at the same time then it is possible that with high probability, at least one of the predictions will fail. This seems to undermine the idea that a failed prediction falsifies the underlying theory. In practice, the problem is dealt with using (at least) three processes that I will call selection, aggregation and amplification. Before I explain these concepts in more detail, I will describe a purely mathematical approach to the problem of multiple predictions. Consider a special case of independent predictions A1 , A2 , A3 , . . .. Suppose that all these events have probability 99%. Then in the long run, these predictions will fail at an approximate rate of 1%, even if the theory behind these predictions is correct. The last statement is a prediction itself and can be formalized as follows, using the Law of Large Numbers. There exists a large number n0 such that for every number n greater than n0 , the percentage of predictions Ak that fail among the first n predictions will be between 0.9% and 1.1%, with probability greater than 99%. The last probability can be increased, but then the value of n0 has to be adjusted.

page 238

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch11

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

The Science of Probability

239

The process of selection of predictions is applied mostly subconsciously. We are surrounded with a very complex universe, full of unpredictable events. Most of them are irrelevant to our lives, such as whether the leaf at the top of my linden tree will fall to the north or to the south. We normally think about a small selection of events that can influence our lives and we try to make predictions concerning these events. The fewer the number of predictions, the fewer the number of failed predictions. A lottery provides an example of the aggregation procedure. Typically, the probability that a specific person will win a given lottery is very small. In other words, we can make a prediction that the person will not win the lottery. The same prediction applies to every person playing the same lottery. However, it is not true that the probability that nobody will win the lottery is small. We never make one hundred thousand predictions, each one saying that a different specific person will not win the lottery. The number of predictions that are actually made is reduced by combining large families of related predictions into a smaller number of “aggregate” predictions. For some lotteries, one can make a single “aggregate” prediction that somebody will win the lottery. For some other lotteries, the probability that someone will win the lottery may be far from 1 and 0 — in such a case, no aggregate prediction can be made. When a specific prediction is very significant to us, we can amplify its power by collecting more data. This is a standard practice in science. For example, most people believe that smoking increases the probability of cancer. Let me represent this claim in a somewhat artificial way as a statement that the cancer rate among smokers will be higher than the cancer rate among non-smokers in 2035 with probability p. We believe that p is very close to 1, but to make this prediction even stronger, new data on smokers and cancer victims are continually collected. The more data are available, the higher value of p they justify. Both amplification and aggregation are used in the context of hypothesis testing (see Sec. 13.3.3). Even after applying selection and aggregation, a single person will generate a large number of predictions over his lifetime. If the predictions are reasonably independent then one can prove, just like indicated earlier in this section, that only a small proportion of predictions will fail, with high probability. This claim is a single “aggregate” prediction. Such a single aggregate prediction can be constructed from all predictions made by a single physical person, or by a group of people, for example, scientists working in a specific field.

page 239

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

240

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch11

Resonance: From Probability to Epistemology and Back

I will now address a few more scientific and philosophical points related to multiple predictions. First, it is interesting to see how mathematicians approach very large families of predictions. Consider, for example, Brownian motion. This stochastic process is a mathematical model for a chaotically moving particle. Let B(t) denote the position of a Brownian particle moving along a straight line at time t. It is known that for a fixed time t, with probability one, B(t) is not equal to 0. It is also known that for a fixed time t, with probability one, the trajectory B(t) has no derivative, that is, it is impossible to determine the velocity of the Brownian particle at time t. The number of times t is infinite and, moreover, it is “uncountable.” Can we make a single prediction that all of the above predictions about the Brownian particle at different times t will hold simultaneously? It turns out that, with probability one, for all times t simultaneously, there is no derivative of B(t). However, with probability one, there exist t such that B(t) is equal to 0. These examples show that infinitely many (uncountably many) predictions can be combined into a single prediction or not, depending on the specific problem. On the technical side, this is related to the fact that the product of zero and infinity is not a well defined number. In practice, nobody makes infinitely many separate predictions about B(t) for all values of t. One can make either several simultaneous predictions for a few specific values of t, or an aggregate prediction concerning the behavior of the whole Brownian trajectory. Scientists make large numbers of diverse predictions, and the same holds for ordinary people, except that in the latter case, the predictions are informal. Transforming a family of predictions into a single aggregate prediction may be hard for several reasons. First, using mathematics to combine multiple predictions into a single aggregate prediction may be easier said than done when individual predictions are not independent. Second, the single combined prediction can be verified only at the end of a possibly long period of time. Third, on the philosophical side, a single combined prediction is not an attractive idea either. The falsification of a single aggregate prediction only indicates that there is something wrong with the theory underlying all of the constituent predictions. A single falsified aggregate prediction provides little specific information on what might have gone wrong because it is based on a very complex theoretical structure. Rather than to combine multiple predictions into a single aggregate prediction, a more practical way to go is to treat an individual

page 240

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch11

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

The Science of Probability

241

falsified probabilistic prediction not as a proof that the underlying theory is wrong but as an indication that it may be wrong and hence it merits further investigation. The amplification procedure described above, that is, collecting more data, can generate a new prediction with probability very close to 1. A large number of predictions will hold simultaneously with very high probability if the constituent predictions hold with even greater probability. It is instructive to see how the problem of multiple predictions affects other philosophical theories of probability. First, consider the frequency theory. This theory is concerned with collectives (sequences) that are infinite in theory but finite in practice. The relative frequency of an event converges to a limit in an infinite collective. All we can say about the relative frequency of an event in a finite collective is that it is stable with a very high probability. For example, we can say that the relative frequency in the first half of the sequence will be very close to the relative frequency in the second part of the sequence with very high probability; this probability depends on the length of the sequence and it is very high for very long sequences. The last claim is an example of a prediction. Clearly, some of these predictions will fail if we consider a large family of finite collectives. Hence, the frequency theory has to tackle the problem of multiple predictions, just like my own theory. Next, let me suggest an operational definition of a prediction in the context of the subjective theory. An event is a prediction if its probability is so high that changing this probability to 1 would not change any decisions which the decision maker might make. For example, a subjectivist decision maker, who is a potential lottery player, can implement a prediction that a given number is not the winning lottery number by not buying a specific lottery ticket. The same prediction and the same decision may be applied by the same person to every number on every ticket. However, if the same subjectivist decision maker becomes the operator of the lottery, he is not likely to make a prediction that no number will be the winning number. The apparent paradox can be easily resolved on the mathematical side using the theory of i.i.d. sequences, and it can be resolved on the philosophical side using ideas described earlier in this section. I end this section with some remarks of purely philosophical nature. Some predictions occasionally fail. I have explained that one way to deal with this problem is to combine a family of predictions into a single prediction — an aggregate of predictions. However, the process of

page 241

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

242

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch11

Resonance: From Probability to Epistemology and Back

aggregation of predictions has to stop. It may stop at the personal level, when a person considers the collection of all predictions that are significant to him, throughout his life. Or we could consider a purely theoretical aggregate of all predictions ever made in the universe by all sentient beings. The ultimate prediction cannot be combined with any other predictions, by definition. And it can fail, even if the underlying theory is correct. Hence, probability is a form of non-scientific belief that the universe that we live in (or our personal universe) is such that this single aggregate prediction will hold. This may sound like an unacceptable mixture of an almost religious belief and science. In fact, deterministic scientific theories are also based on non-scientific beliefs. For example, the belief that the laws of science are stable is not scientific. In other words, there is no scientific method that could prove that all laws of science discovered in the past will hold in the future. I will try to rephrase the idea of aggregating all predictions into a single one. One could say that the philosophical essence of the science of probability is to generate a single prediction (for example, by aggregating many simple predictions). Once the prediction is stated, a person or a group of people who generated it act on the belief that Nature (or the Creator) will grant the wish of the person or the people, and make the prediction come true. My resonance theory suggests that we believe that predictions will come true because we were shaped to hold such beliefs by the evolution. Animals perceive probabilities in an instinctive way. The selection process made animals, including humans, believe in positive outcomes of highly probable events because such beliefs increased the survival probability.

11.4 Symmetry, Independence and Resonance Our ability to recognize symmetries and independence is a special case of resonance. The success of probability theory (that is, good predictions in the sense of (L6)) stems from reliable information about some aspects of the real world. My theory asserts implicitly that symmetries and physical independence are objective and that they can be effectively recognized. If one adopts the philosophical position that objective symmetries do not exist or cannot be effectively recognized then the theory of probability becomes practically useless. I have shown in Chapters 4 and 5 that the frequency theory and the subjective theory are meaningless without (L3)

page 242

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch11

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

The Science of Probability

243

and (L4). Hence, if there is a genuine problem with these two laws, all philosophies of probability are severely affected. The problem is not unique to the probability theory and (L1)–(L6). Let me reiterate some claims from Chapters 8 and 10, this time specialized to symmetry and independence. The ability to recognize events which are symmetric or physically unrelated is a fundamental element of any scientific activity. The need for this ability is so basic and self-evident that scientists almost never talk about it. Suppose a biologist wants to find out whether zebras are omnivorous. He has to go to Africa and observe a herd of zebras. He has to find a collection of symmetric objects characterized by black and white stripes. In particular, he must not mistake lions for zebras. Moreover, the zoologist must disregard any information that is unrelated to zebras, such as data on snowstorms in Siberia in the 17th century or car accidents in Brazil in the last decade. Skills needed to recognize symmetry in the probabilistic context are precisely the same as the ones needed if you want to count objects. People are expected and required to recognize symmetries, for example, shoppers and sellers are expected by the society to agree on the number of apples in a basket — otherwise, commerce would cease to exist. Every scientific experiment in deterministic science is based on symmetry and independence. Symmetry may be exemplified by identity of repeated trials or it may be the identity of the model and a single experiment. The concept of independence is implicit in attempts to isolate the experiment from all noises and irrelevant influences. The laws (L1)–(L6) are based on principles taken for granted elsewhere in science, if not in philosophy. One cannot prove beyond any doubt that people can effectively recognize events that are disjoint, symmetric or physically independent. But it is clear that if people cannot do that then the probability theory cannot be implemented with any degree of success.

11.5 Symmetry is Relative Much of the confusion surrounding the subjective philosophy of probability is caused by the fact that the word “subjective” may be mistakenly interpreted as “relative.” I believe that when de Finetti asks “But which symmetry?” in [de Finetti (1974)] (page xi of Preface), he refers to the fact that symmetry is relative. Some (perhaps all) physical quantities are relative — this is a simple observation, at a level much more elementary than Einstein’s “relativity

page 243

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

244

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch11

Resonance: From Probability to Epistemology and Back

theory.” For example, if you are traveling on a train and reading a book, the velocity of the book is zero relative to you, while it may be 70 miles per hour relative to someone standing on the platform of a railway station. Each of the two velocities is real and objective — this can be determined by experiments. Passengers riding on the train can harmlessly throw the book between each other, while the same book thrown from a moving train toward someone standing on the platform can harm him. Suppose that two people are supposed to guess the color of a ball — white or black — sampled from an urn. Suppose that one person has no information besides the fact that the urn contains only white and black balls, and the other person knows that there are 10 white balls and five black balls in the urn. The two people will see different symmetries in the experiment and the different probability values they assign to the event of “white ball” can be experimentally verified by repeated experiments. The symmetry used by the first person, who does not know the number of white and black balls in the urn, can be applied in a long sequence of similar experiments with urns with different contents. He would be about 50% successful in predicting the color of the ball, in the long run (see Sec. 11.12 for more details). The other person, knowing the composition of the urn, would have a rate of success for predicting the color of the ball equal to 2/3, more or less, assuming that the samples are taken from the same urn, with replacement. Note that the experiments demonstrating the validity of the two symmetries and the two probability values are different, just like the experiments demonstrating the validity of two different velocities of a book traveling in a train are different. The fact that symmetry is relative does not mean that it is arbitrary, just like velocity is relative to the observer but not arbitrary. In the subjective philosophy of probability, the word “subjective” does not mean “relative.”

11.6 Moderation is Golden I have already mentioned in Sec. 11.1 that (L1)–(L6) do not cover some popular and significant ways of assigning probabilities to events, for example, these laws do not provide a (direct) justification for the use of the normal distribution in some situations. Should one extend (L1)–(L6) by adding more laws and hence make the set more complete? Or perhaps one could remove some redundant laws from (L1)–(L6) and make the system more concise?

page 244

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch11

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

The Science of Probability

245

I believe that the fundamental criterion for the choice of a system of laws for the science of probability should have a utilitarian nature. Can the laws, in their present shape, be a useful didactic tool? Would they help scientists memorize the basic principles of probability? Would they provide clear guidance towards empirically verifiable assertions? I could not find a set of laws less extensive or more extensive than (L1)–(L6) that would be more useful. It is my opinion that adding even one extra statement to (L1)–(L6) would necessitate adding scores of similar ones and the laws would experience a quantum jump — from a concise summary, they would be transformed into a ten-volume encyclopedia. I propose a somewhat speculative argument in support of not extending the laws any further. Any probability assignment that is not specified by (L1)–(L6) can be reduced to (L1)–(L6), at least under the best of circumstances. For example, suppose that a statistical model contains a statement that a quantity has a normal distribution. A scientist might not be able to recognize in an intuitive and reliable way (via resonance) whether the result of a measurement has the normal distribution. But it might be possible to generate a sequence of similar (“exchangeable”) measurements and use this sequence to verify the hypothesis that the measurement has the normal distribution. Let me repeat that this is practical only in some situations. This simple procedure that verifies normality can be based only on (L1)–(L6). It is clear that even the most complex models can be verified in a similar way, using only (L1)–(L6), at least under the best possible circumstances. Hence, on the philosophical side, (L1)–(L6) seem to be sufficient to derive the whole science of probability, including statistics and all other applications of probability theory. On the other hand, assignments of probabilities made on the basis of any one of (L1)–(L6), especially (L2)–(L4), cannot be reduced to the analysis using only a subset of these laws. To see this, suppose that we want to verify a claim that the probability of a certain event A is equal to p, where p is not close to 0 or 1. Suppose further that we can generate a sequence of events exchangeable with A and record the frequency with which the event occurs in the sequence. The relative frequency can be taken as an estimate of p. The proper execution of the experiment and its analysis require that we are able to effectively recognize the elements of the sample space, that is, the events that cannot happen at the same time. This is routine, of course, but it means that we have to use (L2). Next, we have to be able to eliminate from our considerations all irrelevant information,

page 245

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

246

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch11

Resonance: From Probability to Epistemology and Back

such as the current temperature on Mars (assuming that A is not related to astronomy). This is an implicit application of (L3). And finally we have to be able to identify an exchangeable sequence of events, which requires using (L4). A really good reason for keeping all laws (L1)–(L6) in the system is not philosophical but practical. Even if a philosopher can demonstrate that one of these laws can be derived from the others, removing that law from the system would not make the system any more practical as a teaching tool. To see that this is the case, it suffices to consult any undergraduate probability textbook.

11.7 Applications of (L1)–(L6): Some Examples Anyone who has ever had any contact with real science and its applications knows that (L1)–(L6) are a de facto scientific standard, just like Newton’s laws of motion. Nevertheless, I will give a few, mostly simple, examples. Some of them will be derived from real science, and some of them will be artificial, to illustrate some philosophical points. First of all, (L3) and (L4) are used in probability is the same way as in the rest of science. Recall the example involving zebras in Sec. 11.4. When a scientist wants to study some probabilistic phenomena, he often finds collections of symmetric objects. For example, if a doctor wants to study the coronary heart disease, he has to identify people, as opposed to animals, plants and rocks. This is considered so obvious that it is never mentioned in science. More realistically, physicians often study some human subpopulations, such as white males. This is a little more problematic because the definition of race is not clear-cut. Doctors apply (L3) by ignoring data on volcano eruptions on other planets and observations of Moon eclipses.

11.7.1 Poisson process A non-trivial illustration of (L4) is a “Poisson process,” a popular model for random phenomena ranging from radioactive decay to telephone calls. This model is applied if we know that the number of “arrivals” (for example, nuclear decays or telephone calls) in a given interval of time is independent of the number of arrivals in any other (disjoint) interval of time. The model can be applied only if we assume in addition a symmetry, specifically the “invariance under time shifts” — the number of arrivals in a time interval

page 246

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch11

The Science of Probability

247

can depend on the length of the interval but not on its starting time. It can be proved in a rigorous way that the independence and symmetry described above uniquely determine a process, called a “Poisson process,” except for its intensity, that is, the average number of arrivals in a unit amount of time.

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

11.7.2 Laws (L1)–(L6) as a basis for statistics Laws (L1)–(L6) are applied by all statisticians, frequency and Bayesian. A typical statistical analysis starts with a “model,” that is, a set of assumptions based on (L1)–(L6). Here (L2), (L3) and (L4) are the most relevant laws, as in the example with the Poisson process. The laws (L1)–(L6) usually do not specify all probabilities or relations between probabilities, such as the intensity in the case of the Poisson process. The intensity is considered by frequency statisticians to be an “unknown but fixed” parameter that has to be estimated using available data. Bayesian statisticians treat the unknown parameter as a random variable and give it a distribution known as a prior. I will discuss the frequency and Bayesian branches of statistics in much more detail later in the book. The point that I want to make now is that (L1)–(L6) are used by both frequency and Bayesian statisticians. The application of these laws, especially (L3) and (L4), has nothing to do with the official philosophies adopted by the two branches of statistics. Frequency statisticians apply (L3) and (L4) even if the available samples are small. Models (but not all priors) used by Bayesian statisticians de facto follow the guidelines given in (L1)–(L6) and so they attract very little philosophical controversy.

11.7.3 Long run frequencies and (L1)–(L6) I will show how the long run frequency interpretation of probability fits into the framework of (L1)–(L6). I have already criticized, in Chapter 4, the von Mises theory of “collectives” formalizing the idea of the long run frequency. I will now argue that this staple scientific application of probability agrees well with (L1)–(L6). To be concrete, consider a clinical test of a new drug. For simplicity, assume that the result of the test can be classified as a “success” or “failure” for each individual patient. Suppose now that you have a “large” number n of patients participating in the trial. There is an implicit other group of patients of size m, comprised of all people afflicted by the same malady

page 247

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

248

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch11

Resonance: From Probability to Epistemology and Back

in the general population. We apply the law (L4) to conclude that all n + m patients form an “exchangeable” sequence. Choose an arbitrarily small number δ > 0 describing your error tolerance and a probability p, arbitrarily close to 1, describing the level of confidence you desire. One can prove that for any values of δ > 0 and p < 1, one can find n0 and m0 such that for n > n0 and m > m0 , the absolute value of the difference between the success rate of the drug among the patients in the clinical trials and the success rate in the general population will be smaller than δ with probability greater than p. One usually assumes that the general population is large and so, implicitly, m > m0 , whatever the value of m0 might be. If the number of patients in the clinical trial is sufficiently large, that is, n > n0 , one can apply (L6) to treat the clinical trial results as a predictor of the future success rate of the drug. In other applications of the idea of the long run frequency, the counterpart of the group of patients in the clinical trial may be a sequence of identical measurements of an unknown constant. In such a case, the general population of patients has no explicit counterpart — this role is played by all future potential applications of the constant.

11.7.4 Life on Mars The question of the probability that there was or there is life on Mars is of great practical importance. If the reader is surprised by my claim, he should think about the enormous amount of money — billions of dollars — spent by several nations over the course of many years on spacecraft designed to search for a sign of (past or present) life on the surface of Mars. A good way to present the probabilistic and philosophical challenge related to life on Mars is to pose the following question: Why is it rational to send lifeseeking robots to Mars but not to Venus? The frequency theory of probability suggests that we should look for a long sequence of “identical” events that incorporates life on Mars, and another sequence for Venus. A natural idea would be to look at a long sequence of planets similar to Mars and see what percentage of them ever supported life. There are several problems with this idea. The first is somewhat philosophical, but it cannot be ignored from the scientific point of view either. Which planets are “similar” to Mars? Should we consider all planets that have the same size and the same distance from their star? Or should we insist that they also have a similar atmosphere and a similar chemistry of the rocks on the surface? If we specify too many similarities

page 248

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch11

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

The Science of Probability

249

then our “sequence” will consist of a single planet in the universe — Mars. A much more practical problem is that at this time, we cannot observe even a small sample of planets “similar” to Mars and verify whether they support life. Even if the “long run frequency of life on planets similar to Mars” is a well-defined concept, it is of no help to scientists and politicians trying to decide whether they should spend money on life-seeking spacecraft. The subjective theory does not offer much in terms of practical advice either. This theory stresses the need of being consistent. Trivially, it is consistent to believe that there was life on Mars but not on Venus with high probability, and it is also consistent to believe that there was life on Venus but not on Mars with high probability. De Finetti’s general position expressed in the quote in Sec. 2.4.3 implies that sending life-seeking spacecraft to Mars but not to Venus is just a current fad and it cannot be scientifically justified any more than sending life-seeking spacecraft to Venus but not to Mars. Laws (L1)–(L6) can be used to justify not sending life-seeking spacecraft to Venus in the following way. Multiple observations of and experiments with different life forms on the Earth show that life known on the Earth can survive only in a certain range of conditions. The spectrum of environments that can support life is enormous, from ocean depths to deserts, but there seem to be limits. The environment on Venus is more or less similar to (“symmetric with”) some environments created artificially in laboratories. Since no life survived in laboratory experiments in similarly harsh conditions, we believe that life did not and does not exist on Venus. We use an approximate symmetry to make a prediction that there was no life on Venus. The argument uses laws (L4) and (L6). The question of life on Mars illustrates well the “negative” use of (L6). What we seem to know about Mars suggests that there might have been times in the past when the environment on Mars was similar to (“symmetric with”) environments known on the Earth or created artificially in laboratories, in which life was sustained. Hence, we cannot conclude that the probability of life on Mars is very small. That does not mean that the probability is large. The only thing that we can say is that we cannot make the prediction that signs of life on Mars will never be found. Hence, it is not irrational to send life-seeking spaceship to Mars. It is not irrational to stop sending life-seeking spaceship to Mars either. In a situation when neither an event nor its complement have very small probabilities, no action can be ruled out as irrational and the decision is a truly subjective choice.

page 249

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

250

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch11

Resonance: From Probability to Epistemology and Back

The last assertion needs a clarification. Decision makers often attach significance not to the outcome of a single observation or experiment but only to the aggregate of these. For example, stores are typically not interested in the profit made on a single transaction but in the aggregate profit over a period of time, say, a year. A decision maker has to choose an aggregate of decisions that is significant for him. One could argue that one should consider the biggest aggregate possible but that would mean that we would have to consider all our personal decisions in our lifetime as a single decision problem. This may be the theoretically optimal decision strategy but it is hardly practical. Thus most decision makers consider “natural” aggregates of decisions. An aggregate may consist of a single decision with significant consequences. My suggestion that sending of life-seeking space probes to Mars and not sending them are both rational decisions, was made under assumption that this action was considered in isolation. In fact, politicians are likely to consider many spending decisions as an aggregate and so one could try to make a prediction about the cumulative effect of all such decisions. The decision to send spacecraft to Mars may be a part of an aggregate of decisions which is definitely rational or irrational as an aggregate, without specifying the rationality status of the individual decisions in the aggregate.

11.8 Symmetry and Data Application of symmetry in statistics presents a scientific and philosophical problem. Once the data are collected, they cannot be symmetric with the future observations. The reason is that the values of the past observations are already known and the values of the future observations are unknown. This is a difference between the past and future data that can be hardly considered irrelevant. It seems that according to (L4), one cannot use any data to make predictions. Recall a simple statistical scheme based on (L1)–(L6) from Sec. 11.7.3. One chooses a symmetric (exchangeable) group of patients in the population. A small subgroup is invited to participate in a drug trial and given a new medication. We can use (L4) prior to the commencement of the trials to conclude that the percentage of patients in the whole population whose condition is going to improve is more or less the same as the percentage of patients participating in the trials who show improvement. Strictly speaking, (L4) does not allow us to make the same claim after the data are collected.

page 250

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch11

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

The Science of Probability

251

On the practical side, it would be silly to discard the data just because someone had forgotten to apply a standard statistical procedure before the data were collected. However, the problem with the broken symmetry is not purely theoretical or philosophical — it is the basis of a common practice of manipulation of public opinion using statistics. The principal idea of this highly questionable practice is very simple. In some areas, huge amounts of data are collected and many diverse statistics (that is, numbers characterizing the data) are computed. Some and only some of these statistics may support someone’s favorite view on social, economic or political matters. For example, the government may quote only those statistics that support the view that the economy is doing well. This practice is an example of broken symmetry. The government implicitly says that the economic data in the last year and the data in the future are symmetric. Since the economic data in the last year were positive, so will be the data in the future. In fact, the past data chosen for the public relations campaign may have been selected a posteriori, and the non-existent symmetry was falsely used for making implicit predictions. Predictions made by people using the data in an “honest” way and those that manipulate the data can be confronted with reality, at least in principle. The manipulation can be documented by empirically detected false predictions.

11.9 Probability of a Single Event In some cases, such as tosses of a symmetric coin, laws (L1)–(L6) not only impose some relationships between probabilities but also determine probabilities of individual events. If an event has probability (close to) 0 or 1, this value can be verified or falsified by the observation of the event. An implicit message in (L6) is that if an event has a probability (much) different from 0 or 1, this value cannot be verified or falsified. One has to ask then: Does this probability exist in an objective sense? Let us see what may happen when a probability value is chosen in an arbitrary way. Suppose someone thinks that if you toss a coin then the probability of heads is 1/3 and not 1/2. If that person tosses a coin only a few times in his lifetime, he will not be able to make a prediction related to the tosses and verify or falsify his belief about the probability of heads. Now suppose that there is a widespread belief in a certain community that the probability of heads is 1/3, and every individual member of the community tosses coins only a few times in his or her lifetime. Then no individual in

page 251

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

252

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch11

Resonance: From Probability to Epistemology and Back

this population will be able to verify or falsify his beliefs, assuming that the members of the community do not discuss coin tosses with one another. Suppose an anthropologist visits this strange community, interviews the people about their probabilistic beliefs and collects the data on the results of coin tosses performed by various people in the community. She will see a great discrepancy between the aggregated results of coin tosses and the prevalent probabilistic beliefs. This artificial example is inspired by some real social phenomena. It has been observed that lotteries are more popular in poor communities than in affluent communities. There are many reasons why this is the case, but one of them might be the lack of understanding of probability among the poorer and supposedly less educated people. A single poor person is unlikely to be able to verify his beliefs about the probability of winning a lottery by observing his own winnings or losses (because winnings are very rare). But someone who has access to cumulative data on lottery winnings may be able to determine that members of poor communities overestimate the probability of winning. The point of these examples is that a statement that may be unverifiable by a single person, might be an element of a collection of beliefs of individuals that yield verifiable (and falsifiable) predictions. A single person choosing a single probability value may not know how this probability value will be used. The discussion in Secs. 11.5, 11.10, 11.12 and 16.5 shows that a single event can be embedded in different sequences or some other complex random phenomena. Hence, it is impossible to find a unique scientific value for every probability. Nevertheless, in many practical situations, most people may agree on the context of an event and, therefore, they may agree on a unique value of the probability of the event.

11.10 On Events that Belong to Two Sequences A good way to test a philosophical or scientific theory is to see what it has to say about a well-known problem. Suppose that an event belongs to two exchangeable sequences. For example, we may be interested in the probability that a certain Mr. Winston, smoking cigarettes, will die of a heart attack. Suppose further that we know the relevant statistics for all smokers (men and women combined), and also statistics for men (smokers and non-smokers combined), but there are no statistics for smoking men. If the long run frequencies are 60% and 50% in the two groups for which statistics are available, what are the chances of death from a heart attack for Mr. Winston?

page 252

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch11

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

The Science of Probability

253

Laws (L1)–(L6) show that the question does not have a natural scientific answer. One needs symmetry to apply (L4), the most relevant law here. However, Mr. Winston is unique because we know something about him that we do not know about any other individual in the population. For all other individuals included in the data, we either do not know their gender or whether they smoke. Statisticians and scientists would not give up that easily. They might propose a model, based on a scientific theory, describing the relationship between smoking, heart disease and gender. Then observations of some frequencies may be used to estimate probabilities that do not correspond to any of the observed frequencies. The success of this procedure would be greatly dependent on the validity of the scientific theory used to construct the probabilistic model. Hence, the apparent contradiction between probability values suggested by two observed frequencies can be resolved only if there is a scientific theory which combines the frequencies and all other available information into a single model. Let me make a digression to present a common error arising in the interpretation of the frequency theory of von Mises. Some people believe that the frequency theory is flawed because it may assign two different values to the probability of an event because the event belongs to two different sequences. This is a misconception — according to the frequency theory, a single event does not have a probability at all.

11.11 Events are More Fundamental than Random Variables One should stress events rather than random variables in the foundations of probability. This is because, most of the time, events are less controversial than random variables. The philosophical status of unknown quantities is unclear — we could think about them as unknown constants or random variables. For example, consider the following probabilistic claim about the speed of light L. Suppose a scientist collected some data, used them to calculate some numbers x and y, and claims that P (x < L < y) = 0.99.

(11.1)

In frequency statistics, L is an unknown constant and the endpoints x and y of the “confidence interval” (x, y) in (11.1) are random variables (functions of data). In Bayesian statistics, L is a random variable and x and y are

page 253

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

254

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch11

Resonance: From Probability to Epistemology and Back

known constants. In Bayesian setting, (11.1) is a formal representation of a “credible interval.” Hence, there is no agreement on the philosophical status of the numbers L, x and y. At some point in the future, using a better technology than that currently available, we may be able to improve the accuracy of measurement of L many times, say 1000 times, and then we will effectively know whether it is true that x < L < y or not. There will be little controversy about the philosophical meaning of the statement that “x < L < y.” In other words, the event {x < L < y} invites much less philosophical controversy than numbers (unknown constants or random variables) L, x and y. In real life, it may be hard to determine whether an event actually occurred — a typical example is provided by criminal trials. But all practical difficulties encountered while trying to determine whether an event happened apply equally when we try to determine the value of a random variable.

11.12 Deformed Coins Suppose that you will have to bet on the outcome of a single toss of a deformed coin but you cannot see the coin beforehand. Should you assume that the probability of heads is equal to 1/2? There are two simple arguments, one in favor and one against the statement that the probability of heads is 1/2. The first argument says that since we do not know what effect the deformation might have, we should assume that the probability of heads is 1/2, by symmetry. The other argument says that our general experience with asymmetric objects strongly suggests that the probability of heads is not equal to 1/2. The probability of heads is unknown but this does not mean that it is equal to 1/2. The problem is resolved using (L1)–(L6) as follows. If we toss the coin only once, we cannot generate a prediction, that is, there is no event associated with the experiment with probability very close to 1. In practice, this means that we will never know with any degree of certainty what the probability of heads is for this particular coin. If the coin is physically destroyed after a single toss, no amount of statistical, scientific or philosophical analysis will yield a reliable or verifiable assertion about the probability of heads. The single toss of the deformed coin might be an element of a long sequence of tosses. If all the tosses in the sequence are performed with the

page 254

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch11

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

The Science of Probability

255

same deformed coin, we cannot generate the prediction that the long run frequency of heads will be 1/2. Hence, in this setting, one cannot assume that the probability of heads is equal to 1/2. But we can make a prediction that the relative frequency of heads will converge to a limit. Another possibility is that the single toss of the given coin will be an element of a long sequence of tosses of deformed coins, and in each case one will have to try to guess the outcome of the toss without inspecting the coin beforehand. In this case, one may argue that one should assume that the probability of heads is 1/2. This is because whatever decision related to the toss we make, we assign our beliefs to heads and tails in a symmetric way. In other words, the coin is not symmetric but our thoughts about the coin are symmetric. The prediction that in the long run we will be able to guess correctly the outcome of the toss about 50% of the time is empirically verifiable. In case of many physical systems, we have excellent support for our intuitive beliefs about symmetry, provided by the past statistical data and scientific theories, such as statistical physics, chaos theory, or quantum physics. My opinion is that symmetry in human thoughts has reasonable but not perfect support in statistical data and, unfortunately, very little, if any, theoretical support. The “deformed coin” may seem to be a purely philosophical puzzle with little relevance to real statistics. It is, therefore, a good idea to recall a heated dispute between two important scientists, Fisher and Jeffreys, described in [Howie (2002)]. Consider three observations of a continuous quantity, that is, three observations coming from the same unknown distribution. The assumption that the quantity is “continuous” implies that, in theory, there will be no ties between any two of the three observed numbers. What is the probability that the third observation will be between the first two? Jeffreys argued that the probability is 1/3, by symmetry, because any of the three observations has the same probability of being the middle one. Fisher did not accept this argument (see Chapter 5 of [Howie (2002)]). It is easy to see that this problem that captured the minds of these applied scientists is a version of the “deformed coin” problem. If you collect only three observations, no scientifically verifiable prediction can be made. If you continue making observations from the same distribution, the long run proportion of observations that fall between the first two observations will not be equal to 1/3 for some distributions — this is a verifiable prediction. Similarly, if we consider a long run of triplets of observations coming from physically unrelated distributions, the long run proportion of cases with

page 255

April 29, 2016

9:6

256

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch11

Resonance: From Probability to Epistemology and Back

the third observation being in the middle will be about 1/3; this is also an empirically verifiable prediction.

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

11.13 Are Coin Tosses i.i.d. or Exchangeable? Consider tosses of a deformed coin. One may argue that they are independent (and so i.i.d., by symmetry and (L4)) because the result of any toss cannot physically influence any other result, and so (L3) applies. Note that (L1)–(L6) cannot be used to determine the probability of heads on a given toss. Frequency statisticians would refer to the sequence of results as “i.i.d. with unknown probability of heads.” An alternative view is that results of some tosses can give information about other results, so the coin tosses are not independent. For example, if we observe 90 heads in the first 100 tosses, we are likely to think that there will be more heads than tails in the next 100 tosses. The obvious symmetry and (L4) make the tosses exchangeable. There are many exchangeable distributions and, by de Finetti’s theorem (see Sec. 18.1.2), they can be all represented as mixtures of i.i.d. sequences. Since the mixing distribution is not known either in practice or in theory, a Bayesian statistician may call the sequence of results an “exchangeable sequence with unknown (or subjective) prior.” De Finetti’s theorem shows that both ways of representing coin tosses are equivalent because they put the same mathematical restrictions on probabilities. Hence, it does not matter whether one thinks about tosses of a deformed coin as i.i.d. with unknown probability of heads or regards them as an exchangeable sequence. Independence is relative, just like symmetry is relative (see Sec. 11.5). Coin tosses are independent or not, depending on whether we consider the probability of heads on a single toss to be an unknown constant or a random variable. Both assumptions are legitimate and can be used to make successful predictions. The fact that independence is relative does not mean that we can arbitrarily label some events as independent. I have to mention a subtle mathematical point involving the equivalence of exchangeability and i.i.d. property for a sequence. In reality, all coin tossing sequences are finite. The exchangeability of a finite sequence is not equivalent to the i.i.d. property, in the sense of de Finetti’s theorem. Hence, the ability to recognize properly an i.i.d. sequence is a different ability from the ability to recognize invariance of a finite sequence under permutations. Suppose that someone properly recognized a finite exchangeable sequence.

page 256

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch11

The Science of Probability

257

To conclude that it is a mixture of i.i.d. sequences, one has to imagine an infinite sequence which is an appropriate extension of the real finite sequence and recognize that the extended sequence is exchangeable.

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

11.14 Mathematical Foundations of Probability Customarily, Kolmogorov’s axioms (see Sec. 18.1) are cited as the mathematical basis for the probability theory. In fact, they are not axioms in the ordinary (mathematical) sense of the word. It is hard to overestimate the influence and importance of Kolmogorov’s idea for the probability theory, statistics and related fields. Simple random phenomena, such as casino games or imperfect measurements of physical quantities, can be described using very old mathematical concepts, borrowed from combinatorics and classical analysis. On the other hand, modern probability theory, especially stochastic analysis, depends in a crucial way on measure theory, a fairly recent field of mathematics. It was Kolmogorov who realized that measure theory was a perfect framework for all rigorous theorems that represent real random phenomena. In addition, measure theory provided a unified treatment of “continuous” and “discrete” models, adding elegance and depth to our understanding of probability. A few alternative rigorous approaches to probability, such as “finitely additive probability” and “non-standard probability” (a probabilistic counterpart of a strangely named field of mathematics, “non-standard analysis”), have only a handful of supporters. None of the above means that Kolmogorov’s “axioms” are axioms. Currently published articles in mathematical journals specializing in probability contain concepts from other fields of mathematics, such as complex analysis and partial differential equations, to name just two. I do not think that anybody would propose to relabel a mathematical theorem containing an estimate of the probability of an event as “non-probabilistic” only because its proof contains methods derived from complex analysis or partial differential equations. As far as I know, Kolmogorov’s axioms cannot generate mathematical theorems proved in complex analysis and partial differential equations. All mathematical theorems in probability and statistics are based on the same system of axioms as the rest of mathematics — the current almost universal choice for the axioms seems to be the “ZFC,” the Zermelo–Fraenkel system with the axiom of choice (see [Jech (2003)]). The philosophical status of Kolmogorov’s axioms is

page 257

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

258

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch11

Resonance: From Probability to Epistemology and Back

really strange. They are neither mathematical axioms nor scientific laws of probability. The lack of understanding of the role that Kolmogorov’s axioms play in probability theory might be caused, at least in part, by poor linguistic practices. The mathematical theory based on Kolmogorov’s axioms uses the same terminology as statistics and related sciences, for example, the following terms are used in both mathematics and science: “sample space,” “event” and “probability.” The equivalent mathematical terms, “measurable space,” “measurable set” and “normalized measure” are not popular in statistics and only occasionally used in mathematical research papers in the area of probability. The linguistic identification of mathematical and scientific concepts in the field of probability creates an illusion that Kolmogorov’s axioms constitute a scientific theory. In fact, they are only a mathematical theory. For comparison, let us have a brief look at the mathematical field of partial differential equations. Nobody has any doubt that the “second derivative” is a mathematical term and in certain situations it corresponds to “acceleration,” a physical concept. A result of this linguistic separation is that scientists understand very well that the role of a physicist is to find a good match between some partial differential equations (purely mathematical objects) and reality. For example some partial differential equations were used by Maxwell to describe electric and magnetic fields, some other equations were used by Einstein to describe space and time in his relativity theory, and yet a different one was used by Schr¨ odinger to lay foundations of quantum physics. The role of probabilists, statisticians and other scientists is to find a good match between elements of Kolmogorov’s mathematical theory and real events and measurements. The misconception that Kolmogorov’s axioms represent a scientific or philosophical theory is a source of much confusion.

11.15 Axioms Versus Laws of Science Some scientific theories, mostly mathematics, are summarized using “axioms.” Many natural sciences are summarized using “laws of science.” Axioms are statements accepted without proof. Laws of science can be falsified by experiments and observations. Axioms work well in mathematics where most researchers agree on the advanced parts of the theory so axioms are needed only to provide solid rigorous foundations and clarify some subtle points. If scientists do

page 258

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch11

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

The Science of Probability

259

not agree on advanced techniques in a field of science then there is no reason why they should agree on the axioms. For this reason, trying to axiomatize probability or statistics is a bad intellectual choice. Axioms are accepted without justification — this is the meaning of axioms. Since statistics is riddled with controversy, an opponent of the subjective theory has the intellectual right to reject subjectivist axioms with only superficial justification. Probability should be based on laws of science, not axioms. The susceptibility of the laws of science to refutation by experiments or observations is built into their definition, at least implicitly. True, the standards of refutation are subject to scientific and philosophical scrutiny. Hence, a description of the verification procedure should be included explicitly or at least implicitly in the given set of laws. “Self-evident” axioms would clearly fail in some highly non-trivial fields of science. Most people would choose axioms of Newton’s physics over Einstein’s physics because Newton’s physics is “self-evident.” Yet the 20th century physics theories are considered superior to Newton’s because they agree with experimental data and make excellent predictions. Only successful predictions can validate a scientific theory. Axioms are appropriate only for a mathematical theory.

11.16 Objective and Subjective Probabilities The concept of subjectivity does not belong to science. Scientists argue about whether their results, claims and theories are true of false, correct or incorrect, exact or approximate, rigorous or heuristic. The statement that “zebras are omnivorous” may be true or false but scientists do not spend any time arguing whether it is objective or subjective. A new theory in physics known as the “string theory” may be called speculative but I do not think that anybody suggests that it is subjective. The idea of bringing subjectivity into the scientific foundations of probability created only confusion. The laws of probability (L1)–(L6) are enforced by the society. Examples of enforcement include all safety regulations, such as obligatory seat belt use, to lower the probability of death in an accident. Societies enforce laws that can be regarded as subjective, such as driving on the right hand side of the road in the US, and some laws that can be regarded as objective, such as the rules of arithmetic used in the calculation of taxes. In principle, all laws can be changed but I would expect much resistance if anyone proposed to abandon enforcement of “objective” laws. As far as I can tell, changing

page 259

April 29, 2016

9:6

260

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch11

Resonance: From Probability to Epistemology and Back

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

the implicit enforcement of (L1)–(L6) would require arguments similar to those that would be needed to stop the enforcement of “objective” laws, such as the laws of physics used in building codes. Hence, (L1)–(L6) are treated by the society just like the laws that are considered unquestionably objective. The word “subjective” has completely different meanings in de Finetti’s theory and Bayesian statistics, compounding the difficulties of the discussion (see Sec. 5.1 and Chapter 14).

11.17 Physical and Epistemic Probabilities I made hardly any attempt to distinguish between physical and epistemic probabilities although this seems to be one of the important questions in the philosophy of probability. One can describe “physical” probabilities as those that have nothing to do with the presence or absence of humans, and they have nothing to do with imperfections of human knowledge. Probabilities of events in quantum physics are the primary (and only “true”?) example of physical probabilities. The standard interpretation of quantum mechanics says that it is impossible to improve probabilistic claims made by quantum physics about some events by collecting more data, improving the accuracy of measurements, or developing more sophisticated theories. In other words, some probabilities in the microscopic world seem to be a part of the physical reality, unrelated to human presence. However, even this most agreed upon example of physical probabilities is not as solid as some commentators would have it; see remarks on Tegmark’s interpretation of quantum probabilities in Sec. 11.18. Most probabilities that scientists and ordinary people are concerned with pertain to macroscopic objects and situations, such as weather, patients, stock market, etc. In many, perhaps all, situations, one can imagine that we can collect more data, perform more accurate measurements, or develop better theories to analyze these situations. Hence, a probability in one of these situations can be attributed to a gap in the human knowledge rather than to the real physical impossibility to predict the result of an experiment or observation. For example, the result of a coin toss can be predicted with great accuracy given sufficient knowledge about the initial position and velocity of the coin. The law (L4) sometimes refers to the true physical symmetry and sometimes to the symmetry in our knowledge and information processing. We have to recognize symmetries to be able to function and the question

page 260

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch11

The Science of Probability

261

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

of whether these symmetries are physical or whether they represent a gap in our knowledge does not affect the effectiveness of (L1)–(L6). I am not aware of a set of scientific laws for probability that would make an effective use of the fact that there are both physical and epistemic probabilities, I do not think that any such system would be more helpful than (L1)–(L6), or that it would represent the current state of the sciences of probability and statistics in a more accurate way.

11.18 Can Probability be Explained? Can the phenomenon of probability be reduced to or explained by a more fundamental law of physics, in the same sense as temperature can be reduced to the average energy of molecules and biological processes can be reduced to chemical processes? I do not believe that reductions of this type completely explain the higher level phenomena but scientific benefits of known reductions are unquestionable. It was proposed that probability has two sources. One of them is sensitivity of some deterministic dynamical systems to initial conditions. The other source is the irreducibly probabilistic character of quantum physics. While I totally agree that it is highly plausible that all random macroscopic events are manifestations of these two physical phenomena, I find neither of the two avenues of reduction capable of “explaining” probability. “Chaos” and “butterfly effect” are popular terms for sensitivity of deterministic dynamical systems to initial conditions. A small change in the initial conditions can cause a large change on the macroscopic scale. Tossing a coin is a good illustration of this effect. Figure 11.3 shows regions in the initial conditions plane which correspond to heads (black stripes) and tails (white stripes) in a simplified coin tossing model discussed in [Keller (1986)]. The picture illustrates the fact that a small change in the initial velocity or rotation speed may move the point from a black stripe to a white stripe; then the result of the coin toss will change from heads to tails. Since controlling the initial position and velocity of the coin or measuring them with high precision is almost impossible in everyday practice, coin tosses seem to be random to us, with both sides of the coin equally likely. So far, I totally agree with this scientific analysis. But the argument given above assumes tacitly that the initial conditions are random and have a distribution in the plane which is spread out more or less evenly over black

page 261

April 29, 2016

9:6

262

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch11

Resonance: From Probability to Epistemology and Back 10

8

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

6

4

2

0

0

2

4

6

8

10

Fig. 11.3 The axes represent the initial velocity and rotation speed of a coin. The plane is divided into black and white regions representing initial conditions resulting in heads and tails, resp. Adapted from Figure 2 in [Keller (1986)].

and white regions. Common sense indeed suggests that the distribution of the initial conditions has a smooth density with a small slope and it is spread over a region that contains many black and white stripes. Hence, we have reduced the assumption that the coin falls heads or tails with equal probabilities because of an “obvious” symmetry to rather technical assumptions on the probability distribution of the initial conditions. Our analysis is an unquestionable case of reduction but I do not think that it is an “explanation” of macroscopic probability. We still have not found the “source” of probability. The lower level model (random initial conditions) is much more complicated than the traditional representation of coin tossing as a simple experiment with an obvious probabilistic symmetry. My knowledge of quantum physics is very limited so I have to rely on experts. I like the “Everett many-worlds” interpretation of quantum mechanics, presented in [Tegmark (2014), p. 186]. In this interpretation, the wave function never collapses. So the evolution of the universe is deterministic. Tegmark argues, in a section starting on page 191, that we experience an illusion of randomness. We live in one of many universes. Since a typical

page 262

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch11

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

The Science of Probability

263

deterministic parallel universe contains deterministic sequences that possess properties that we normally associate with random sequences, such as stable frequencies, we have the illusion of randomness. I am not competent to evaluate the scientific content of this theory so all I can say is that I find it very attractive. But there is a catch. On page 193, Tegmark uses the word typical in relation to our own universe in the family of all parallel universes and he italicizes it. Why should we live in a “typical” universe? The idea that we live in a typical universe seems to be equivalent to the claim that our universe was chosen randomly according to the uniform distribution over all parallel universes. This is a version of the “indifference principle” which says that equal probabilities should be assigned to events if we do not have any information that would favor one of them. While the principle may sound attractive, it is obviously false and harmful in such a great generality. We are able to recognize symmetries in random models due to resonance. But resonance evolved to make us well adjusted to terrestrial challenges. I see no obvious reason why our feelings about “typical” events should extend to the situation as exotic as parallel universes. Let me make it clear that I actually believe that we live in a typical universe but I do not have any solid argument in support of this purely intuitive feeling. Just like in the case of chaotic dynamical systems, the explanation of probability in quantum physics seems to be a reduction of one probabilistic model to a different probabilistic model. I do not feel that this provides an “explanation” of the phenomenon of probability.

11.19 Propensity I have great respect for Popper’s approach to probability because my law (L6) is an implementation of his falsificationist ideas. On the other hand, I do not like the other major idea of Popper in the area of probability. Popper believed that probability is “propensity,” a physical quantity such as mass or distance. I do not think that the idea of propensity is useful. Consider the event that there was life on Mars or the event that Shakespeare did not write the dramas attributed to him. It is hard to imagine that our current world has a propensity to agree with one or the other outcome of these events because this is contrary to our normal intuition concerning propensity, as a physical property that can generate some results in the future. Hence, propensity seems to be useless as a practical intellectual tool that could help scientists do their job. Propensity does not seem to be helpful on the philosophical side either. If the probability of a future event

page 263

April 29, 2016

9:6

264

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch11

Resonance: From Probability to Epistemology and Back

is 0.7 given the current state of the universe then we could say that the universe has 70% propensity to generate this event. This explains absolutely nothing. Propensity seems to be just a new name for something that we are trying to understand.

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

11.20 Countable Additivity The question of σ-additivity (also known as countable additivity) of probability is only weakly related to the main theme of this book but the discussion of this question will allow me to illustrate one of my fundamental philosophical claims — that probability is a science, besides having mathematical and philosophical aspects, and so it can and should be empirically tested. First I will explain the concept of σ-additivity. Formally, we say that probability is σ-additive if for any countably infinite sequence of mutually exclusive events A1 , A2 , . . ., the probability of their union is the sum of probabilities of individual events. A probability is called finitely additive if the last statement is assumed only for finite sequences (of any length) of disjoint events A1 , A2 , . . . , An . To illustrate the definition, let us consider a sequence of tosses of a deformed coin. The coin is deformed in my example to stress that the symmetry of the coin is irrelevant. Let A1 denote the event that the first result is tails. Let A2 be the event that the first toss results in heads and the second one yields tails. In general, let An be the event that the first n − 1 results are heads and the n-th result is tails. These events are mutually exclusive, that is, no two of these events can occur at the same time. The union of these events, call it B, is the event that at least one of the tosses results in tails. In other words, at least one of the results is tails if and only if one of the events Ak , k = 1, 2, . . ., occurs. The σ-additivity of probability is the statement that the probability of B is the same as the sum of probabilities of events Ak , k = 1, 2, . . . The widely accepted axiomatic system for the mathematical probability theory, proposed by Kolmogorov, assumes that probability is σ-finite. My guess is that the main reason why σ-additivity is so popular is that it is very convenient from the mathematical point of view. Not everybody is willing to assume this property in real applications but finitely-additive probability has never attracted much support. I claim that σ-additivity is an empirically testable scientific law. A statement belongs to science if it can be empirically falsified according to Popper. Recall the example with the deformed coin. One can estimate

page 264

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch11

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

The Science of Probability

265

probabilities of events B, A1 , A2 , . . ., for example, using long run frequencies. Suppose that for some deformed coin, the values are P (B) = 1, P (A1 ) = 1/4, P (A2 ) = 1/8, P (A3 ) = 1/16, etc. Then the sum of probabilities P (Ak ) is equal to 1/2, which is not the same as the probability of B and this (hypothetical) example provides a falsification of σ-additivity. Of course, probability estimates obtained from long run experiments would be only approximate and one could only estimate a finite number of probabilities P (Ak ). But these imperfections would not be any different from those of any other scientific measurement. One could not expect to obtain an indisputable refutation of σ-additivity but one could obtain a strong indication that it fails. I have to make sure that readers who are not familiar with probability theory are not confused by the probability values presented in the last example. According to the standard mathematical theory of probability, one cannot have P (B) = 1, P (A1 ) = 1/4, P (A2 ) = 1/8, P (A3 ) = 1/16, etc. for any deformed or symmetric coin. I made up these values to emphasize that it is possible, in principle, that empirical values do not match the currently accepted mathematical theory. I should also add that I believe that σ-additivity is strongly supported by empirical evidence. Scientists accumulated an enormous amount of observations of random phenomena and nobody seems to have noticed patterns contradicting σ-additivity of probability. Arguments against σ-additivity seem to have purely philosophical nature. However, we must keep our minds open on this question — probability is a science and one cannot make ultimate judgments using pure reason.

11.21 Yin and Yang The relationship between the philosophy of probability and statistics is analogous to that between mathematics and physics. In those fields of mathematics and physics which can be directly compared to each other, mathematicians accomplished very little because they insist on rigorous proofs. Physicists often perform mathematical operations that are not justified in a rigorous way. Physicists generate many claims because they prefer to generate all possible true claims at the price of generating some false claims. Similarly, all best known philosophies of probability (classical, logical, frequency and subjective) strike us as very limited in scope, because they insist on detailed and profound analysis of even the most obvious claims

page 265

April 29, 2016

9:6

266

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch11

Resonance: From Probability to Epistemology and Back

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

about probability. Statisticians are willing to accept every method that is reasonably reliable in practice. There is a difference, though, between the two pairs of fields. Both mathematicians and physicists are very well aware of strengths and weaknesses of their own and their colleagues’ approaches to science. Historically speaking, there is a much stronger tendency in statistics to mix science with philosophy, despite philosophy’s conspicuously different methods and goals.

11.22 Are Laws (L1)–(L6) Necessary? First, let me explain what I mean by “necessary.” I consider the statement “11 is a prime number” necessary because I cannot imagine a universe in which one could arrange 11 apples in some number of rows, with the same number of apples in each row, except one row or 11 rows. I do not consider (L1)–(L6) necessary in the same sense. I believe that a universe with all of the following properties is logically possible. (1) Some laws of nature are probabilistic, like quantum mechanics in our world. (2) All the laws of nature are known to people living in that world. (3) All scientific quantities needed for making probabilistic predictions are effectively measurable in that world. In such an imaginary world, symmetry and independence, and the ability to recognize them, would play no special role in the foundations of its probability theory. Sentient beings would observe (measure) all relevant quantities with ease and perfect accuracy and make probabilistic predictions, as needed. Although a universe satisfying (1)–(3) seems to be logically possible, there are reasons to doubt its existence. A sentient being in such a universe would have access to the full knowledge of itself. A brain or some other physical object would have to contain in its memory all the information about its own state and, on the top of that, the state of the rest of the universe. This seems to be self-contradictory.

11.23 Quantum Mechanics From the time of Newton and Leibnitz until early 20th century, the standard scientific view of the universe was that of a clockwise mechanism. Probability was a way to express and quantify human inability to predict the

page 266

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch11

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

The Science of Probability

267

future, despite its deterministic character. Quantum physics brought with it a fundamental change in our understanding of the role of randomness. Some physical processes are now believed to be inherently random, in the sense that the outcome of some events will be never fully predictable, no matter how much information we collect, or how accurate our instruments might become. The philosophical interpretation of the mathematical principles of quantum physics has been a subject of much controversy and research. To this day, some leading scientists are not convinced that we fully understand this theory on the philosophical side — see [Penrose (2005)]. As far as I can tell, the laws (L1)–(L6) apply to quantum physics just like to macroscopic phenomena. Physicists implicitly apply (L3) when they ignore Pacific storms in their research on electrons. Similarly, (L4) is implicitly applied when physicists use their knowledge of electrons acquired in the past in current experiments with electrons. Finally and crucially, (L6) is applied in the context of quantum mechanics, just like in all of science, to make predictions using long run frequencies. I am far from claiming that the system (L1)–(L6) is sufficient to generate all probabilistic assertions in quantum physics. Quite the opposite, my guess is that the Schr¨odinger equation and probability values that it generates cannot be reduced to (L1)–(L6) in any reasonable sense. This, however, does not diminish the role of (L1)–(L6) as the basis of the science of probability in the context of quantum physics. The laws (L1)–(L6) can and should be supplemented by other laws, as needed.

11.24 The History of (L1)–(L6) in Philosophy of Probability The laws (L1)–(L2) were known since the very beginning of the mathematical theory of probability. They were never controversial so they never attracted much attention on the philosophical side. For some reason, (L3) was never raised to the status of an important philosophical principle. Mathematicians say that the probability theory is the measure theory plus independence. Independence is sometimes considered to be a purely mathematical concept, not a law of science. Independence is a part of the definition of an i.i.d. sequence, one of the most fundamental concepts in probability and statistics. De Finetti preferred to treat such sequences as exchangeable and exchangeability does not

page 267

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

268

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch11

Resonance: From Probability to Epistemology and Back

refer to independence (at least not directly). Von Mises did not believe in probabilities of individual events so independence plays no role in the definition of a collective (an alternative formalization of an i.i.d. sequence). I will discuss the history of (L4) in Sec. 11.25. The law (L5) has a strange position in the philosophical literature. For historical reasons, the philosophical discussion of conditioning was mostly focused on the Bayes theorem, a special case of conditioning, and mostly ignored the general principle (L5). There are exceptions — the book [Hacking (2001)] presents (L5) as the “rule of conditionalization” on page 259. Nevertheless, quite often, the law (L5) seems to be represented as a mathematical definition, not a law of science (recall a similar remark about (L3)). It is clear that people who believe in the fundamental importance of the Bayes theorem also believe in (L5) but they prefer to concentrate on the special case (the Bayes theorem) rather than on the general principle (L5). The law (L6) was proposed by a number of people but never reached a high status in the philosophy of probability. Standard short reviews of Popper’s philosophy do not mention his opinions equivalent to (L6).

11.25 Symmetry and Theories of Probability The law (L4) is the most conspicuous link of the system (L1)–(L6) to statistics because it is the basis of i.i.d. and exchangeable models. The importance of exchangeable events has been recognized by each of the main philosophies of probability, under different names: “equally possible cases” in the classical theory, “principle of indifference” in the logical theory, “collective” in the frequency theory and “exchangeability” in the subjective theory. None of these philosophies got it right. The classical theory of probability was based on symmetry although the term “symmetry” did not appear in the classical definition of probability. Since the definition used the words “all cases possible,” it was applicable only in highly symmetric situations, where all atoms of the outcome space had the same probability. I do not think that we could stretch the classical definition of probability to derive the statement that in two tosses of a deformed coin the events HT and T H have the same probability. The classical philosophy missed the important point that symmetry is useful even if not all elements of the outcome space have the same probability. Since the classical philosophy was not a conscious attempt to build a complete philosophical theory of probability but a byproduct of scientific

page 268

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch11

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

The Science of Probability

269

investigation, one may interpret the shortcomings of the classical theory as incompleteness rather than as an error. The law (L4) is built into the logical theory under the name of the “principle of indifference.” This principle seems to apply to situations where there is inadequate knowledge, while (L4) must be applied only in situations when some relevant knowledge is available, and according to what we know, the events are symmetric. For example, we know that the ordering of the results of two tosses of a deformed coin does not affect the results. But we do not know how the asymmetry of the coin will affect the results. Hence, T H and HT have equal probabilities, but T T and HH do not. According to some versions of the logical theory, the probability of T T is 1/4 or 1/3. It can be empirically proved that these probability assignments lead to some false predictions, as follows. Consider a long sequence of deformed coins and suppose that each coin is tossed twice. Assume that we do not know anything about how the coins were deformed. They might have been deformed in some “random” way, or someone might have used some “nonrandom” strategy to deform them. It seems that the logical theory implies that in the absence of any knowledge of the dependence structure, we should assume that for every coin, the probability of T T is either 1/4 or 1/3, depending on the version of the logical theory. This and the mathematical theory of probability lead to the prediction that the long run frequency of T T ’s in the sequence will be 1/4 or 1/3. This can be empirically disproved, for some sequences of deformed coins. The problem, at least with some versions of the logical theory, is that they extend the principle of indifference to situations with no known physical symmetry. The frequency theory made the “collective” (long sequence of events) its central concept. Collectives are infinite in theory and they are presumed to be very large in practice. The law (L4) is implicit in the definition of the collective because the collective seems to be no more than an awkward definition of an exchangeable sequence of events. To apply the frequency theory in practice, one has to be able to recognize long sequences invariant under permutations (that is, exchangeable sequences), and so one has to use symmetry as in (L4). The frequency theory fails to recognize that (L4) is useful outside the context of collectives, that is, very long exchangeable sequences. To see this, recall the example from the previous paragraph, concerned with a sequence of deformed coins. The coins are not (known to be) “identical” or exchangeable, so one cannot apply the idea of collective to make a verifiable prediction that the long run frequency of HT ’s will

page 269

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

270

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch11

Resonance: From Probability to Epistemology and Back

be the same as the frequency of T H’s. Of course, one can observe long sequences of HT ’s and T H’s and declare the whole sequence of results a collective, but that would be a retrodiction, not prediction. The problem with the logical theory of probability is that it advocates using symmetry in some situations when there is no symmetry and so it makes some extra predictions which are sometimes false. The problem with the frequency theory of probability is the opposite one — the theory does not support using symmetry in some situations when symmetry exists and so it fails to make some verifiable predictions. The subjective theory’s attitude towards (L4) is the most curious among all the theories. Exchangeability is clearly a central concept, perhaps the central concept, in de Finetti’s system of thought, on the scientific side. These healthy scientific instincts of de Finetti gave way to his philosophical views, alas. His philosophical theory stresses absolute subjectivity of all probability statements and so deprives (L4) of any meaning beyond a free and arbitrary choice of an individual. All Bayesian statisticians and subjectivists use symmetries in their probability assignments just like everybody else. Yet the subjective theory of probability insists that none of these probability assignments can be proved to be correct in any objective sense.

page 270

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch12

Chapter 12

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Decision Making

Decision making is not a part of science (see Sec. 10.11). Science can (try to) predict the consequences of various decisions but it is not the role of science to tell people what they should do. I will divide my discussion of decision making into several parts. Section 12.2 will deal with decision making options when all the relevant probabilities are determined by (L1)–(L6) or in some other way. Section 12.4 will address the question of what to do when not all probabilities needed to make a decision are known. I will ignore the utility function in the initial part of the chapter but I will discuss some aspects of this concept in Sec. 12.5.

12.1 Common Practices Some common beliefs and practices concerning decision making involving uncertainty seem to be almost uncontroversial. These beliefs act as an inspiration for various scientific and philosophical proposals for formal theories of decision making. Decision making principles pose a problem when a decision making situation does not fall squarely into one of the following two categories. If an event of interest has very high probability then it is natural to assume that it will happen with certainty. This common belief is the essence of my law (L6). People typically ignore events of small probability, such as the possibility of dying in a car accident on a given trip. Probabilistic decision making is thus reduced to deterministic decision making in some situations. One could even go as far as to say that all “deterministic” decision problems arise in this way because nothing is ever totally certain.

271

page 271

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

272

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch12

Resonance: From Probability to Epistemology and Back

The other type of decision problem which does not seem to pose much of a practical or intellectual challenge is when the decision maker is faced with a long sequence of similar decisions. Then it is natural to maximize the expected value of the gain or the expected value of the utility in each individual case and hope that the Law of Large Numbers will generate the maximum average gain over a long time. The details of what should be maximized, what probabilities should be used, etc., are the subject of a philosophical and practical debate but it is clear that most people are willing to choose individual decisions on the basis of long run average predictions. I will return to this discussion in Sec. 12.2.5.

12.2 Decision Making in the Context of (L1)–(L6) I will now present a semi-formal description of a simple probabilistic decision problem. Very few real life decision problems are that simple but readers unfamiliar with the formal decision theory might get a taste of it. Suppose that one has to choose between two decisions, D1 and D2 . Suppose that if decision D1 is made, the gain may take two values G11 and G12 , with probabilities p11 and p12 . Similarly, D2 may result in rewards G21 and G22 , with probabilities p21 and p22 . Assume that p11 + p12 = 1 and p21 + p22 = 1, all four probabilities are strictly between 0 and 1, and G11 < G21 < G22 < G12 so that there is no obvious reason why D1 should be preferable to D2 or vice versa. Recall that, in this section, I assume that the four probabilities, p11 , p12 , p21 and p22 , are determined by (L1)–(L6) or in some other way. How can we decide in a “rational” way which of the two decisions D1 or D2 is “better”? I will start by criticizing the most popular answer to this question and then I will propose two other decision making philosophies.

12.2.1 Maximization of expected gain A standard decision making philosophy is to choose a decision which maximizes the expected gain. This decision making philosophy is quite intuitive but I will show that it is profoundly flawed. Recall the decision problem from the previous section. If we make decision D1 then the expected gain is G11 p11 + G12 p12 and if we make decision D2 then the expected gain is G21 p21 + G22 p22 . Hence, if we want to maximize the expected gain, we should make decision D1 if G11 p11 + G12 p12 > G21 p21 + G22 p22 , and we should choose D2 if the

page 272

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch12

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Decision Making

273

inequality goes the other way. The decisions are equally preferable if the expected values are equal. The above strategy sounds rational until we recall that, typically, the “expected value” is not expected at all. If we roll a fair die, the “expected number” of dots is 3.5. Of course, we do not expect to see 3.5 dots. I have a feeling that most scientists subconsciously ignore this simple lesson. To emphasize the true nature of the “expected value”, let me use an equivalent but much less suggestive term “first moment.” Needless to say, “maximizing the first moment of the gain” sounds much less attractive than “maximizing the expected value of the gain.” Why should one try to maximize the first moment of the gain and not minimize the third moment of the gain? I will address this question from both frequency and subjective points of view. The frequency theory of probability identifies the probability of an event with the limit of relative frequencies of the event in an infinite sequence of identical trials, that is, a collective. Similarly, the expected value (first moment) of a random variable may be identified with the limit of averages in an infinite sequence of i.i.d. random variables, by the Law of Large Numbers. If we want to use the frequency theory as a justification for maximizing of the first moment of the gain, we have to assume that we face a long sequence of independent and identical decision problems and the same decision is made every time. Only in some practical situations one decision maker deals with a sequence of independent and identical decision problems. In everyday life, various decision problems may have completely different structure. In science and business, the form of a decision problem may sometimes remain the same but the information gained in the course of analyzing earlier problems may be applied in later problems and so the decision problems may not be independent. The frequency theory of probability provides a direct justification for the practice of maximizing of the expected gain only in some cases. Maximizing of the expected gain within the subjective theory of probability seems to be a reasonable strategy for the same reason as in the case of the frequency theory — linguistic. The subjective theory says that the only goal that can be achieved by a decision maker is to avoid a Dutch book situation, by choosing a consistent decision strategy. There are countless ways in which one can achieve consistency and none of them is any better than any other in any objective sense, according to the subjective theory. A mathematical theorem says that if you choose any consistent strategy then you maximize the expected gain, according to

page 273

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

274

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch12

Resonance: From Probability to Epistemology and Back

some probability distribution. The idea of “maximizing of expected gain” clearly exploits subconscious associations of decision makers. They think that their gain will be large, if they choose a decision which maximizes the expected gain. The subjective theory says that the gain can be large or small (within the range of possible gains corresponding to a given decision) but one cannot prove in any objective sense that the gain will be large. Moreover, the subjective theory teaches that when the gain is realized, its size cannot prove or disprove in the objective sense any claim about optimality or suboptimality of the decision that was made. Hence, maximizing the expected gain really means maximizing the subjective feelings about the gain. This sounds like a piece of advice from a “selfhelp” book rather than science. I will present the above claims in a somewhat different way. Within the subjective philosophy, the idea of maximizing of subjective gain is tautological. The prior distribution can be presented in various formal ways. One of them is to represent the prior as a set of beliefs containing (among other statements) conditional statements of the form “if the data turn out to be x then my preferred decision will be D(x).” Since, in the subjective theory, probabilities and expectations are only a way of encoding consistent human preferences, an equivalent form of this statement is “given the data x, the decision D(x) maximizes the expected gain.” Hence the question of why you would like to maximize the expected gain is equivalent to the question of why you think that the prior distribution is what it is. In the subjective philosophy, it is not true that you should choose the decision which maximizes the expected gain; the decision that maximizes the expected gain was labeled as such because you said you preferred it over all other decisions. The strict subjectivist version of Bayesian statistics is a process that successfully obfuscates the circularity of the subjectivist preference for the maximization of the expected gain. A Bayesian subjectivist starts with a prior distribution (prior opinion), then collects the data, combines the prior distribution and the data to derive the posterior distribution, and finally makes a decision that maximizes the expected gain, according to the posterior distribution. The whole multistage process, often very complex from the mathematical point of view, is a smokescreen that hides the fact that the maximization of the expected gain according to the posterior distribution is nothing but the execution of the original (prior) preference, not assumed by subjectivists to have any

page 274

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch12

Decision Making

275

objective value. I will later argue that the above subjectivist interpretation of Bayesian statistics is never applied in real life.

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

12.2.2 Maximization of expected gain as an axiom Before I propose my own two alternative decision making philosophies, I have to mention an obvious, but repugnant to me, philosophical choice — one can adopt the maximization of the expected gain as an axiom. I have argued in Sec. 10.11 that the choice of a decision strategy is not a part of the science of probability so this axiom cannot be shown to be objectively correct or incorrect, except in some special situations. Hence, I am grudgingly willing to accept this choice of the decision philosophy, if anyone wants to make this choice. At the same time I strongly believe that the choice is based on a linguistic illusion. If the same axiom were phrased as “one should maximize the first moment of the gain,” most people would demand a good explanation for such a choice. And I have already shown that the justifications given by the frequency and subjective theories are unconvincing. The real answer to the question “Why is it a good idea to maximize the expected gain?” seems to be more technical than philosophical in nature. A very good technical reason to use expectation is that it is additive, that is, the expectation of the sum of two random variables is the sum of their expectations, no matter how dependent the random variables are. This is very convenient in many mathematical arguments. The second reason is that assigning a single value to each decision makes all decisions comparable, so one can always find the “best” decision. Finding the “optimal” decision is often an illusion based on a clever manipulation of language, but many people demand answers, even poor answers, no matter what. The maximization of the expected gain can be justified, at least in a limited way, within each of the two decision making philosophies proposed below. I find that approach much more palatable than the outright adoption of the expected gain maximization as an axiom.

12.2.3 Stochastic ordering of decisions The first of my own proposals for a decision philosophy is based on an idea that probability is the only quantity that distinguishes various events within the probability theory. I will use an analogy to clarify this point.

page 275

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

276

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch12

Resonance: From Probability to Epistemology and Back

Consider two samples of sulfur, one spherical and one cubic in shape. If they have the same mass, they are indistinguishable from the point of view of chemistry. Similarly, two balls made of different materials but with the same radii and the same density would be indistinguishable from the point of view of the gravitation theory. Consider two games, one involving a fair coin and the other involving a fair die. Suppose that you can win $1 if the coin toss results in heads, and lose $2 otherwise. You can win $1 if the number of dots on the die is even, and otherwise you lose $2. Since the probabilities are the only quantities that matter in this situation, there is no rational reason to prefer one of the games over the other. Now consider two games whose payoffs are known and suppose that they are stochastically ordered, that is, their payoffs G1 and G2 satisfy P (G1 ≥ x) ≥ P (G2 ≥ x) for all x. It is elementary to see that there exist two other games with payoffs H1 and H2 such that Gk has the same distribution as Hk for k = 1, 2, and P (H1 ≥ H2 ) = 1. The game with payoff H1 is obviously more desirable than the one with payoff H2 , and, therefore, the game with payoff G1 is more desirable than the one with payoff G2 , by the equivalence described in the previous paragraph. In other words, the decision making philosophy proposed here says that a decision is preferable to another decision if and only if its payoff stochastically majorizes the payoff of the other decision. Here are some properties of the proposed decision making strategy. (i) Consider two decisions and suppose that each one can result in a gain of either $a or $b. Then the gain distributions are comparable. In this simple case, the proposed decision algorithm agrees with the maximization of the expected gain. (ii) Two decisions may be comparable even if their expected gains are infinite (that is, equal to plus or minus infinity), or undefined. (iii) If two decisions are comparable and the associated gains have finite expectations, a decision is preferable to another decision if and only if the associated expected gain is larger than the analogous quantity for the other decision. (iv) Suppose that in a decision problem, two decisions D1 and D2 are comparable and D1 is preferable. Consider another decision problem, consisting of comparable decisions D3 and D4 , with D3 being preferable. Assume that all random events involved in the first decision problem are independent of all events involved in the second problem.

page 276

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch12

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Decision Making

277

If we consider an aggregate decision problem in which we have to make two choices, one between D1 and D2 , and another choice between D3 and D4 , then the aggregate decision D1 and D3 is comparable to the aggregate decision D2 and D4 , and the first one is preferable. Unfortunately, the same conclusion need not hold without the assumption of independence of the two decision problems. (v) One can justify expected gain maximization (under some circumstances) using the idea of stochastic ordering of decisions as follows. Suppose that one has to deal with n independent decision problems, and the kth problem is a choice between two decisions whose gains are random variables G1k and G2k , respectively. If EG1k − EG2k ≥ 0 for every k, the difference EG1k − EG2k is reasonably large, n is not too small, and the variances of Gjk ’s are not too large then G11 + · · · + G1n is either truly or approximately stochastically larger than G21 + · · · + G2n . This conclusion is a mathematical theorem which requires precise assumptions, different from one case to another. Since G11 + · · · + G1n is stochastically larger than G21 +· · ·+G2n , we conclude that it is beneficial to maximize the expected gain in every of the n decision problems. This justification of the idea of maximizing of the expected gain does not refer to the Law of Large Numbers because it is not based on the approximate equality of (Gj1 + · · · + Gjn )/n and its expectation. The number n of decision problems does not have to be large at all — the justification works for moderate n but the cutoff value for n depends significantly on the joint distribution of G1k ’s and G2k ’s. (vi) An obvious drawback of the proposed decision making philosophy is that not all decisions are comparable. Recall the utility function used by the subjective theory. I will make a reasonable assumption that all utility functions are non-decreasing. It is easy to show that two decisions are comparable if and only if one of the decisions has greater expected utility than the other one for every non-decreasing utility function. Hence, the proposed ordering of decisions is consistent with the subjective philosophy in the following sense. In those situations in which the probabilities are undisputable, two decisions are comparable if and only if all decision makers, with arbitrary non-decreasing utility functions, would make the same choice. Let me use the last remark as a pretext to point out a weakness in the subjective philosophy of probability. The comparability of all decisions in the subjective theory is an illusion because the ordering of decisions is strictly subjective,

page 277

April 29, 2016

9:6

278

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch12

Resonance: From Probability to Epistemology and Back

that is, it depends on the individual decision maker. He or she can change the ordering of decisions by fiat at any time, so the ordering has hardly any meaning.

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

12.2.4 Generating predictions My second proposal for a decision making strategy is better adapted to laws (L1)–(L6), especially (L6), than the “stochastic ordering” presented in the previous subsection. The basic idea is quite old — it goes back (at least) to Cournot in the first half of the 19th century (quoted after [Primas (1999), p. 585]): If the probability of an event is sufficiently small, one should act in a way as if this event will not occur at a solitary realization.

Cournot’s recommendation contains no explicit message concerning events which have probabilities different from 0 or 1. My proposal is to limit the probability-based decision making only to the cases covered by Cournot’s assertion. I postulate that probabilistic and statistical analysis should make predictions its goal. In other words, I postulate that decision makers should try to find events that are significant and have probabilities close to 1 or 0. To illustrate the idea, I suppose that one faces a large number of independent decision problems, and at the kth stage, one has a choice between decisions with payoffs G1k and G2k , satisfying EG1k = x1 , EG2k = x2 < x1 , VarGjk ≤ 1. If one chooses the first decision every time, the average gain for the first n decisions will be approximately equal to x1 . The average will be approximately x2 , if one chooses the second decision every time. A
consequence of the Large Deviations Principle is that the n 1 probability P k=1 Gk /n ≤ (x1 + x2 )/2 goes to 0 exponentially fast as n goes to infinity, and so it can be assumed to be zero for all practical purposes, even for moderately large n. This and a similar estimate for
n 2 P k=1 Gk /n ≥ (x1 + x2 )/2 generate the following prediction. Making the first decision n times will yield an average gain greater than (x1 +x2 )/2, and making the second decision n times will result in an average gain smaller than (x1 + x2 )/2, with probability pn very close to 1. Here, “very close to 1” means that 1 − pn is exponentially small in n. Such a fast rate of convergence is considered excellent in the present computer science dominated intellectual climate. √ The traditional curse of statistics is the slow rate (that is, 1/ n) of convergence of approximations to the “true value,” as indicated by the

page 278

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch12

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Decision Making

279

Central Limit Theorem. At the intuitive level, this means that to improve the accuracy of statistical analysis 10 times one needs 100 times more data. The Large Deviations Principle, when used as in the above example, yields a much better rate of convergence to the desirable goal. I conjecture that decision makers base their confidence in statistical methods because they subconsciously rely on predictions generated by the Large Deviations Principle rather than on claims derived from the Central Limit Theorem. The decision making strategy proposed in this section is partly based on the observation that in the course of real life we routinely ignore events of extremely small probability, such as being hit by a falling meteor. Acting otherwise would make life unbearable and would be doomed to failure, as nobody could possibly analyze all events of extremely small probability. Applying the Large Deviations Principle (consciously or subconsciously) can reduce the uncertainty to levels that are routinely ignored in normal life out of necessity. Clearly, the decision making strategy proposed in this section yields applicable advice in fewer situations than the one proposed in the previous section. This strategy should be adopted by those who think that it is better to set goals for oneself that can be realistically and reliably attained rather than to deceive oneself into thinking that one can find a good recipe for success under any circumstances.

12.2.5 Intermediate decision problems The “intermediate” in the title of this section refers to problems that do not fit into any of the following two templates. Some events have probabilities so small that we assume that they will never occur, period. This is especially true if the possible losses and gains are moderate and so the expected value of gain or loss is negligibly small. A typical example of such an event is damage to one’s property from a falling meteor. I do not think that anyone takes precautions to protect his property against meteors. An almost universal decision is to ignore all very unlikely events. Some decision situations can result in small gains or losses. A good example may be a choice of a restaurant for dinner. The gain or loss of utility due to lucky or unlucky choice of a restaurant is usually considered small in relation to other gains or losses in one’s life. Individuals, groups of people, companies, etc. face very large numbers of such low key decision problems in their lifetimes. Hence, one can apply the Law of Large

page 279

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

280

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch12

Resonance: From Probability to Epistemology and Back

Numbers in all such decision situations and hope that it gives a reasonably accurate prediction despite informal and approximate match between this mathematical theorem and reality. Maximization of the expected value of the gain in every case of decision problem at this level will result in the maximal gain over the lifetime of the decision maker (more precisely: decision beneficiary). In both types of decision problems discussed above, the widely used strategies can be easily justified using (L6) because they both involve predictions, that is, events with very high probabilities. “Intermediate” decision problems are those where we do not deal with isolated events of very small probability and we do not deal with large numbers of decisions with small payoffs. Examples of intermediate decision problems include choices of college, profession, job, spouse and house, pertaining to and viewed by an individual. A typical person makes only a handful of such choices in his lifetime. Each of such decisions may have a major impact on one’s wealth, success and happiness. An intermediate decision problem cannot be reduced to finding a prediction (an event of very high probability) that would involve only the decision maker. If many people use a decision strategy that maximizes the chances for lifelong happiness, the Law of Large Numbers implies that the percentage of happy people in the population will be maximized. For this to happen, all or at least many individuals have to identify their interests (or feelings) with those of a group of people. Such identification is known and common in the form of loyalty. But loyalty is not universal and I do not see any purely logical argument in favor of loyalty in decision making. Loyalty seems to be the result of evolution so this discussion is closely related to the topic of Sec. 12.3. I will illustrate my thoughts on intermediate decision problems by a discussion of St. Petersburg paradox in Sec. 16.1.

12.3 Decision Making and Resonance Since resonance is a significant component of science, it is only natural to expect that a mechanism similar to resonance can explain our preferences in making decisions. It seems that animals do not make decisions in the sense relevant to our discussion. Obviously, animals have to make all kinds of decisions concerning food, shelter, safety, etc. But most animals seem to make all these decisions subconsciously. So far, researchers documented rather

page 280

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch12

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Decision Making

281

limited animal ability to consciously analyze information before making decisions — this applies to only a few species, mainly apes. We humans are animals and, just like other animals, we make many decisions in a purely instinctive manner. Resonance is an ability that evolved through the Darwinian selection process. Our ability to make optimal subconscious decisions must have evolved through the same selection process because resonance that is not followed by good decisions is useless from the point of view of survival. This implies that our instinctive decision making skills are fine tuned, at least in the range of challenges that appeared during the process of evolution of human beings. Our instinctive decision choices are often imperfect, just like the process of resonance is not perfect. Psychologists and economists collected a multitude of examples of suboptimal human behavior. Conscious information processing geared towards decision making is needed for at least two major reasons. First, even “primitive” human societies were so complex that many decision problems were far outside the range that intuitive decision making could handle well. The second reason is the constant pressure from individuals and groups of people who try to exploit instinctive decision making of other individuals for their own benefit, often to the detriment of the exploited individual. I do not think that psychologists understand well our decision making instincts. My pessimism is supported by a review of psychological research in [Nickerson (2004)]. So I will speculate that the evolution process shaped our minds so that we have a tendency to make decisions that maximize the survival probability of the genealogical tree of descendants of the decision maker. Or perhaps we subconsciously try to maximize the expected value of the number of descendants at a future time. I admit that both conjectures are speculative and heuristic. It is possible that the evolution process made us maximize some other functional of the universe. Assuming that the first of my conjectures is true, it would be natural to represent our subconscious decision making preferences in terms of the utility function which is simply the probability of survival — either survival of the individual or survival of the tree of descendants. Even if all my speculations concerning the evolution of our subconscious decision making tendencies are correct, they do not provide a solution to the philosophical problem of choosing the best decision strategy. The fact that a specific decision strategy was favored by the evolution does not imply

page 281

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

282

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch12

Resonance: From Probability to Epistemology and Back

in any way that a rational person has to follow the same decision strategy. Consider the following analogy — if it is proved by psychologists and brain scientists beyond any doubt that evolution made us naturally aggressive, I doubt that anyone would consider this scientific finding a strong argument in favor of aggressive behavior. Despite my apprehensive attitude towards adoption of maximization of the expected utility as the “rational” decision strategy, I realize that many people may be convinced of the value of this decision strategy by the fact that the evolution favored it and hence built it into our minds. But this is no more than a choice. No logical argument can lead from the strategy favored by the evolution to our own free and conscious choice of the decision strategy.

12.4 Events with No Probabilities So far, I implicitly assumed in my discussion of decision making that the relevant probabilities were known. This section examines decision making options in situations where (L1)–(L6) and available information do not determine the relevant probabilities. One of the great and indisputable victories of the subjectivist propaganda machine is the widespread belief that there is always a way to choose a rational (or beneficial or the best) action in any situation involving uncertainty. Many of the people who otherwise do not agree with the subjective theory of probability, seem to think that it is a genuine intellectual achievement of the subjective theory to provide a framework for making decisions in the absence of relevant and useful information. What can other sciences offer in the absence of information or relevant theories? A physicist cannot give advice on how to build a plane flying at twice the speed of light or how to make a room temperature superconductor. Some things cannot be done because the laws of science prohibit them, and some things cannot be done because we have not learned how to do them yet (and perhaps we never will). Nobody expects a physicist to give an “imperfect but adequate” advice in every situation — nobody knows how to build a plane which flies at “more or less” twice the speed of light or make a superconductor which works at “more or less” room temperature. No such leniency is shown towards probabilists and statisticians by people who take the subjectivist ideology seriously — if probability is subjective then there is no situation in which you lack anything to make

page 282

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch12

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Decision Making

283

probability assignments. And, moreover, if you are consistent, you cannot be wrong. What should one do in a situation involving uncertainty if no relevant information is available? An honest and rather obvious answer is that there are situations in which the probability theory has no scientific advice to offer because no relevant probability laws or relations are known. This is not anything that we, probabilists, should be ashamed of. The form of the laws (L1)–(L6) may shed some light on the problem. The laws do not give a recipe for assigning values to all probabilities. They only say that in some circumstances, the probabilities must satisfy some conditions. If no relevant relations, such as lack of physical influence or symmetry, are known then laws (L1)–(L6) are not applicable and any assignment of values to probabilities is arbitrary. Note that every event is involved in some relation listed in (L1)–(L6), for example, all events on the Earth are physically unrelated to a supernova explosion in a distant galaxy (except for some astronomical observations). Hence, strictly speaking, (L1)– (L6) are always applicable but the point of the science of probability is to find sufficiently many relevant relations between events so that one can find good estimates for probabilities of relevant events. One could argue that in a real life situation, one has to make a decision and hence one always (implicitly) assigns values to probabilities — in this limited sense, probability always exists. However, the same argument clearly fails to establish that “useful relations between events can be always found.” A practical situation may force a person to make a decision and, therefore, to make implicitly probability assignments, but nothing can force the person to make predictions that will eventually agree with observations. This reminds me of one of the known problems with torture (besides being inhumane): you can force every person to talk, but you do not know whether the person will be saying the truth. On the practical side of the matter, it is clear that people use a lot of science in their everyday lives in an intuitive or instinctive way. Whenever we walk, lift objects, pour water, etc., we use laws of physics, more often than not at a subconscious level. We are quite successful with these informal applications of science although not always so. The same applies to probability — a combination of intuition, instinct, and reasoning based on analogy and continuity can give very good practical results. This however cannot be taken as a proof that one can always assign values to all probabilities and attack every decision problem in a scientifically justified way.

page 283

April 29, 2016

9:6

284

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch12

Resonance: From Probability to Epistemology and Back

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

12.5 Utility The utility function does not play an essential role in my own philosophy of probability and philosophy of decision making. Nevertheless, I will make some remarks on utility because it is an important element of the subjective philosophy of probability. A mathematical theorem proved in an axiomatic version of the subjective theory (see [Fishburn (1970)]) says that a consistent decision strategy is equivalent to the existence of a probability measure and a utility function such that every decision within the consistent strategy is chosen to maximize the expected utility, computed using these probability distribution and utility function. The important point here is that the same theorem cannot be proved without the utility function. In other words, if we assume that the “real” utility of x dollars is x for every x, then some decision strategies will appear to be inconsistent, although the intention of the inventors of the theory was to consider these strategies rational. The utility function has to be used when we want to apply mathematical analysis to goods that do not have an obvious monetary value, such as friendship and art. However, much of the philosophical and scientific analysis of the utility function was devoted to the utility of money. It is universally believed that the “real” value of x dollars is u(x), where u(x) is not equal to x. A standard assumption about u(x) is that it is an increasing function of x, because it is better to have more money than less money (if you do not like the surplus, you can give it away). A popular but less obvious and far from universal assumption is that u(x) is a concave function, that is, the utility of earning an extra dollar is smaller and smaller, the larger and larger your current fortune is.

12.5.1 Variability of utility in time A standard assumption about the utility function is that it represents personal preferences and, therefore, it is necessarily subjective. In other words, science cannot and should not tell people what various goods are really worth. If the utility function is supposed to be a realistic model of real personal preferences, it has to account for the real changes in such preferences. A 20-year old man may put some utility on (various sums of) money, friendship, success, and adventure. It would be totally unrealistic to expect the same man to have the same preferences at the ages of 40 and 60, although the preferences may remain constant for some individuals.

page 284

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch12

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Decision Making

285

The variability of utility presents the following alternative to the decision theory. First, one could assume that the utility function is constant in time. While this may be realistic in some situations, I consider it wildly unrealistic in some other situations, even on a small time scale. The second choice is to assume that the utility function can change arbitrarily over time. This would split decision making into a sequence of unrelated problems, because it would be impossible to say anything about compatibility of decisions made at different times. The middle road is an obvious third choice, actually taken by some researchers. One could assume that utility can change over time but there are some constraints on its variability. This is definitely a sound scientific approach, trying to model real life in the best possible way. But this approach destroys the philosophical value of the utility function. The more conditions on the utility function one imposes, the less convincing the axioms of the decision theoretical version of the subjective theory are.

12.5.2 Nonlinearity of utility I will start this section with an example concerned with multiple decisions that have to be made before observing the gain or loss resulting from any one of them. Suppose that someone’s worth is $100,000 and this person is offered the following game. A fair coin will be tossed and the person will win $1.10 if the result is heads; otherwise the person will lose $1.00. It is usually assumed that the utility function is (approximately) differentiable. This implies that if the person wins the game, the utility of his wealth will be about a + 1.1c in some abstract units. In the case he loses the game, the utility of his wealth will be a − c. Hence, if the person wants to maximize his expected utility, he should play the game. Now imagine that the person is offered to play 100,000 games, all identical to the game described above, and all based on the same toss of the coin. In other words, if the coin falls heads, he will collect $1.10 one hundred thousand times, and otherwise he will lose $1.00 the same number of times. Of course, this is the same as playing only one game, with a possible gain of $110,000 and a possible loss of $100,000. After the game, either the person will be bankrupt or he will have $210,000. Recall that a typical assumption about the utility function is that it is concave — this reflects the common belief that $1.00 is worth less to a rich person than to a pauper. In our example, it is possible, and I would even say quite realistic,

page 285

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

286

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch12

Resonance: From Probability to Epistemology and Back

that the person would consider the utility of his current wealth, that is, the utility of $100,000, to be greater than the average of utilities of $0 and $210,000. Hence, the person would not play the game in which he can win $110,000 or lose $100,000 with equal probabilities. It follows that he would not play 100,000 games in which he can win $1.10 or lose $1.00. This seems to contradict the analysis of a single game with possible payoffs of $1.10 and −$1.00. The example is artificial, of course, but the problem is real. If we consider decision problems in isolation, we may lose the big picture and we may make a sequence of decisions that we would not have taken as a single aggregated action. On the mathematical side, the resolution of the problem is quite easy — the expected utility is not necessarily additive. Expectation is additive in the sense that the expected value of the total monetary gain in multiple decision problems is the sum of expectations of gains in individual problems, even if the decisions are not independent. The same assertion applied to utility is false — in general, it is not true that the expected value of utility increment resulting from multiple decisions is the sum of expectations of utility increments from individual decisions. The expectation of utility increment is additive only under some very restrictive assumptions on the utility function. The above argument shows that one cannot partition a large family of decision problems into individual problems, find the optimal solution for each individual problem separately, and obtain in this way the optimal strategy that would apply to the original complex problem. Theoretically, a person should consider all decision problems facing him over his lifetime as a single decision problem. Needless to say, this cannot be implemented even in a remotely realistic way. In some practical cases, such as multiple simultaneous decisions made within a big company, decision makers face an unpleasant choice. Theoretically, they should analyze outcomes of all possible combinations of all possible actions and all outcomes of all random events — this may be prohibitively expensive, in terms of money and time. Or they can analyze various decision problems separately, effectively assuming that the utility function is (approximately) linear, and thus ignoring the standard assumption that the utility function is curved. As I indicated before, one of the main reasons for using expectation in decision making is technical. The additivity of expectation is a trivial

page 286

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch12

Decision Making

287

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

mathematical fact but it is an almost miraculous scientific property — I do not see any intuitive reason why the expectation should be additive in case of dependent gains (random variables). The inventors of the axiomatic approach to the subjective theory overlooked the fact that the utility function destroys one of the most convincing arguments in support of maximization of the expected gain as the most rational decision strategy.

12.5.3 Utility of non-monetary rewards The problems with utility outlined in the previous sections are even more acute when we consider utility of non-monetary awards. Eating an ice cream on a hot summer day may have the same utility as $3.00. Eating two ice creams on the same day may have utility of $5.00 or $6.00. Eating 1,000 ice creams on one day has a significantly negative utility, in my opinion. Similar remarks apply to one glass of wine, two glasses of wine, and 100 glasses of wine. This does not mean that utility is a useless concept when we consider non-monetary rewards. People have to make choices and their choices define utility, at least in an implicit way. The problem is that the utility of a collection of decisions is a complicated function of rewards in the collection. In many situations, the utility of the collection cannot be expressed in a usable way as a function of utilities of individual rewards. While it is theoretically possible to incorporate an arbitrarily complex utility function into a decision theoretic model, the applicability of such a theory is highly questionable. Either the theory has to require specifying the utility for any combination of rewards, which is far beyond anything that we could do in practice, or the theory must assume that utility is approximately additive, which limits the theory to only some practical situations. My philosophical objections to utility for non-monetary rewards are similar to my objections to the “fuzzy set” theory. The fuzzy set theory tries to model human opinions in situations when an object cannot be easily classified into one of two categories. In an oversimplified view of the world, a towel is either clean or dirty. In the fuzzy set theory, a towel belongs to the set of clean towels with a degree between 0% and 100%. While the fuzzy set theory is clearly well rooted in human experience, its main challenge is to model human opinions in complex situations. On one hand, the algebra of fuzzy sets should be a realistic model for real human opinions, and on the other hand it should be mathematically tractable. Many scientists, including me, are skeptical about the fuzzy set theory because they believe that the two goals are incompatible.

page 287

April 29, 2016

9:6

288

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch12

Resonance: From Probability to Epistemology and Back

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

12.5.4 Unobservable utilities The decision theoretic approach to statistics consists of expressing the consequences of statistical analysis as losses, usually using a utility function. For example, suppose that a drug company wants to know the probability of side effects for a new drug. If the true probability of side effects is p and the statistical estimate is q, we may suppose that the drug company will incur a loss of L dollars, depending on p and q. A common assumption is that the loss function is quadratic, that is, for some constant c, we have L = c(p − q)2 . While the utility loss can be effectively observed in some situations, it is almost impossible to observe it in some other situations. For example, how can we estimate the loss incurred by the humanity caused by an error in the measurement of the atomic mass of carbon done in a specific laboratory in 1950? The result of such practical difficulties is that much of the literature on decision theory is non-scientific in nature. Researchers often advocate various loss functions using philosophical arguments, with little empirical evidence of their actual relationship to real losses. A sound scientific approach to utility functions that are not observable is to make some assumptions about their shape, derive mathematical consequences of the assumptions, and then compare mathematical predictions with observable quantities — all this in place of making direct measurements of the utility function. Such approach is used, for example, in modeling of investor preferences and financial markets. In my opinion, the results are mixed, at best. There is no agreement between various studies even on the most general characteristics of utility functions, such as convexity.

12.5.5 Can utility be objective? A utility function is interesting from the philosophical point of view only if it is subjective. If the utility function can be effectively measured in an objective and scientific way then it constitutes, in part, an objective rescaling of the real line, from currency units to utility units. An objective utility function also assigns value to each non-monetary reward, such as friendship. On the mathematical side, an objective utility function can play a significant role in finding optimal decisions, but there is nothing interesting about it from the philosophical point of view. An objective utility is like any other objectively measurable quantity — mass, energy or temperature.

page 288

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch12

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Decision Making

289

There are two obvious problems with objective utility functions. First, no solid scientific methods of measuring objective utility functions were developed. There are no such methods for measuring the utility of financial rewards and there are none for measuring non-monetary rewards, such as friendship. The second problem with objective utility functions is their relationship with human feelings. It is not clear why people should maximize objective utility (if there is such a thing) if it does not agree with their subjective feelings. Only some people are willing to subordinate their feelings to objective calculations. And there is no obvious reason why people who try to maximize their subjective satisfaction are irrational.

12.5.6 What is the utility of gazillion dollars? I will argue that the subjective utility of very large sums of money is not bounded but undefined. Let N = 101,000,000,000,000,000,000,000,000 . Some people may say that the utility of 2N dollars is not smaller than the utility of N dollars. The standard explanation for this position is that if you prefer to have N dollars rather than 2N dollars then you can give away N dollars. When was it last time that someone donated N dollars to a charity? The utility of N dollars is undefined because we have no experience with sums of money that are close to N . We can ask an individual what his or her feelings about N dollars are but whatever the answer might be, it is a figment of imagination. Since nobody has N dollars, no claims about the utility of N dollars can be verified in any scientifically acceptable way, such as observations of individuals making decisions involving N dollars. The utility function was introduced by Bernoulli as a realistic measure of the utility of a given sum of money (at least more realistic than the nominal value of the money). A bounded utility function could explain the St. Petersburg paradox (see Sec. 16.1), in which a person is willing to pay only a finite fee for an opportunity to play a game with infinite payoff. This explanation works only if the utility of all sums of money is well defined and bounded. In my opinion, the utility function of large sums of money is not well defined.

page 289

April 29, 2016

9:6

290

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch12

Resonance: From Probability to Epistemology and Back

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

12.6 Identification of Decisions and Probabilities The axiomatic approach to the subjective theory of probability identifies decisions and probabilities (see, for example, [Fishburn (1970)]). Every set of consistent decisions corresponds to a probability distribution, that is, a consistent (probabilistic) view of the world, and vice versa, any probability distribution defines a consistent set of decisions. If we adopt this position then the discussion of decision making in this chapter is redundant. This is the case only if we assume that objective probabilities do not exist. If objective probabilities (or objective relations between probabilities) exist then the identification of probabilities and decisions is simply not true. If objective probabilities exist, decision makers can use them in various ways. The subjectivist claim that your decisions uniquely determine your probabilities is nothing more than a way of encoding your decisions, of giving them labels. In some cases these labels may have nothing to do with objective probabilities.

page 290

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch13

Chapter 13

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Frequency Statistics

It is a common view that frequency statistics is justified by the “frequency philosophy.” It is pointless to write a chapter proving that frequency statistics is not related to the philosophical theory of von Mises. His theory of collectives was abandoned long time ago. I guess that less than 1% of frequency statisticians know what a collective is. However, I cannot summarily dismiss the idea that the “frequency interpretation” of probability is the basis of frequency statistics. There is a difficulty, though, with the analysis of the “frequency interpretation” — unlike von Mises’ theory, the frequency interpretation is a mixture of mathematical theorems and intuitive feelings, not a clearly developed philosophy of probability. Despite this problem, I will try to give a fair account of the relationship between the frequency statistics and the frequency interpretation of probability. Three popular methods developed by frequency statisticians are estimation, hypothesis testing and confidence intervals.

13.1 Confidence Intervals The concepts of an estimator and a hypothesis test seem to be more fundamental to frequency statistics than the concept of a confidence interval. I will discuss confidence intervals first because I will refer to some of this material in the section on estimation. Suppose that a parameter θ, presumably an objective physical quantity, is unknown, but some data related to this quantity are available. A “95%confidence interval” is an interval constructed by a frequency statistician on the basis of the data, and such that the true value of the parameter θ is covered by the interval with probability 95%. More precisely, statisticians

291

page 291

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

292

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch13

Resonance: From Probability to Epistemology and Back

prove that if the value of the unknown parameter θ is θ0 , then the 95% confidence interval will contain θ0 with probability 95%. Among the basic methods of frequency statistics, confidence intervals fit (L1)–(L6) best because they are probability statements, unlike estimators and hypothesis tests. Hence, a single confidence interval is a prediction in the sense of (L6), if the probability of coverage of the unknown parameter is chosen to be very high. Personally, I would call a confidence interval a prediction only if the probability of the coverage of the true value of the parameter were 99% or higher. This does not mean that we cannot generate predictions when we use confidence intervals with lower probability of coverage — we can aggregate multiple cases of confidence intervals and generate a single prediction. A scientist or a company might not be interested in the performance of a confidence interval in a single statistical problem, but in the performance of an aggregate of statistical problems. Suppose that n independent 95% confidence interval are constructed. If n is sufficiently large, then one can make a prediction that at least 94% of the intervals will cover the true values of the parameters with probability 99.9% or higher. No matter where we draw the line for the confidence level for a single prediction, we can generate a prediction at an arbitrarily high confidence level by aggregating a sufficiently large number of confidence intervals. Another way to analyze scientific performance of confidence intervals is to express losses due to non-coverage errors using the units of money or utility. We can apply, at least in principle, one of probabilistic techniques to find the distribution of the aggregate loss due to non-coverage errors for a family of confidence intervals and generate a corresponding prediction, for example, a single 99.9% confidence interval for the combined loss.

13.1.1 Practical challenges with statistical predictions The methods of generating predictions from confidence intervals outlined above may be hard to implement in practice for multiple reasons. It is best to discuss some of the most obvious challenges rather than to try to sweep the potential problems under the rug. Generating a prediction from a single confidence interval requires very solid knowledge of the tails of the distribution of the random variable used to construct the confidence interval. If the random variable in question

page 292

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch13

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Frequency Statistics

293

is, for example, the average of an i.i.d. sequence, then the Central Limit Theorem becomes questionable as an appropriate mathematical tool for the analysis of the tails. We enter the domain of the Large Deviations Principle (see Sec. 18.1.1). On the theoretical side, typically, it is harder to prove a theorem that has the form of a Large Deviations Principle than a version of the Central Limit Theorem. On the practical side, the Large Deviations Principle-type results require stronger assumptions than the Law of Large Numbers or the Central Limit Theorem — checking or guessing whether these assumptions hold in practice might be a tall order. Only in some situations, we can assume that individual statistical problems that form an aggregate are approximately independent. Without independence, generating a prediction based on an aggregate of many cases of statistical analysis can be very challenging. Expressing losses due to errors in monetary terms may be hard or subjective. If the value of a physical quantity is commonly used by scientists around the world, it is not an easy task to assess the combined losses due to non-coverage of the true value by a confidence interval. At the other extreme, if the statistical analysis of a scientific quantity appears in a specialized journal and is never used directly in real life, the loss due to a statistical error has a purely theoretical nature and is hard to express in monetary terms. Another practical problem with aggregates is that quite often, a statistician has to analyze a single data set, and he has no idea what other confidence intervals that were constructed in the past, or will be constructed in the future, should be considered a part of the same aggregate. There is a very convenient mathematical idea that seems to solve this problem — expectation. The expectation of the sum of losses is equal to the sum of expectations of losses. Hence, if we want to minimize the expected loss for an aggregate problem, it suffices to minimize the expected loss for each individual statistical problem. This approach works well only if either we ignore the utility function or we assume that it has a special form — see Sec. 12.5.2. A somewhat different problem with aggregates is that one of the statistical errors in the aggregate may generate, with some probability, a loss much greater than the combined losses due to all other statistical errors. In some situations involving potential catastrophic losses, if we limit our analysis only to the expectation of losses, then we may reach an unpalatable conclusion that confidence intervals capable of generating

page 293

April 29, 2016

9:6

294

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch13

Resonance: From Probability to Epistemology and Back

only small losses can be more or less arbitrary because their contribution to the total expected loss is minuscule.

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

13.1.2 Making predictions is necessary The long list of practical problems with predictions given in the previous section may suggest that the idea of generating and verifying a prediction is totally impractical. However, a moment’s thought reveals that most of these practical problems would apply to every method of validating statistical analysis, and they are already well known to statisticians. Predictions have to be verified, at least in some cases, to provide empirical support for a theory. Statistical predictions that have the form of confidence intervals can be verified if and when we find the true value of the estimated quantity. It is not impossible to verify statistical predictions generated by confidence intervals. Theoretically, we will never know the value of any scientific quantity with perfect accuracy. However, if we measure the quantity with an accuracy much better than the current accuracy, say, 1,000 times better, then we can treat the more accurate measurement as the “true value” of the quantity, and use it to verify the statistical prediction based on the original, less accurate measurement. The more accurate measurement might be currently available at a cost much higher than the original measurement, or it might be available in the future, due to technical progress. Frequency statisticians may use their own techniques to evaluate usefulness of confidence intervals and this is fine as long as the end users of confidence intervals are satisfied. However, statistics is riddled with controversy so frequency statisticians must (occasionally) generate predictions in the sense of (L6) so that their theory is falsifiable, and their critics have a chance to disprove the methods of frequency statistics. If the critics fail to falsify the predictions then, and only then, frequency statisticians can claim that their approach to statistics is scientific and properly justified.

13.2 Estimation The theory of estimation is concerned with unknown quantities called parameters. Examples of such quantities include the speed of light, the

page 294

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch13

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Frequency Statistics

295

probability of a side effect for a given drug, and the volatility in a financial market. Let us consider a simple example. If you toss a deformed coin, the results may be represented by an independent identically distributed (i.i.d.) sequence of heads and tails. The probability of heads (on a single toss) is an unknown constant (parameter) θ and the goal of the statistical analysis is to find a good estimate of the true value of θ. If n tosses were performed and k of them resulted in heads then one can take k/n as an estimate of θ. This estimator (that is, a function generating an estimate from the data) is unbiased in the sense that the expected value of the estimate is the true value of θ. The ultimate theoretical goal of the estimation theory is to find an explicit formula for the distribution of a given estimator for each value of the parameter θ. This, theoretically, allows one to derive all other properties of the estimator because the distribution encapsulates all the information about the estimator. Quite often, an explicit or even approximate formula for the distribution is impossible to derive. In such cases, one can try to prove a weaker property of the estimator, for example, that it is unbiased, which means that its expected value is equal to the true value of the parameter. Suppose that an estimator is unbiased. The long run interpretation of this statement requires that we collect a long sequence of data sets, all in the same manner, and independently from each other. A crucial assumption is that the parameter (presumably, a physical quantity) is known to have the same value every time we collect a set of data. Suppose, moreover, that we apply the same estimator every time we collect a data set. Then, according to the Law of Large Numbers, the average of the estimates will be close to the true value of the unknown parameter with a high probability. The scenario described above is purely imaginary. There are many practical situations when one of the conditions described above holds. For example, multiple estimates of the same quantity are sometimes made (think about estimating the speed of light). There are also applications of the estimation theory when the same estimator is applied routinely over long stretches of time; for example, a medical laboratory may estimate the level of a hormone for a large number of patients. It hardly ever happens that data sets are repeatedly collected in an identical way and a long sequence of isomorphic estimates are calculated, assuming that the estimated quantity is known to have the same value in each case. In this scenario, pooling

page 295

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

296

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch13

Resonance: From Probability to Epistemology and Back

all the data together and calculating a single estimate would be a better strategy obviously. The long run frequency interpretation of unbiasedness is unrealistic for another, somewhat different, reason. Statisticians often work with only one data set at a time and there is no explicit or implicit expectation that isomorphic data sets will be collected in the future. I do not think that any statistician would feel that the value of a given (single) estimate diminished if he learned from a clairvoyant person that no more similar estimates will be ever made. It will not help to imagine a long sequence of similar estimation problems. The frequency interpretation of probability or expectation based on imagination and the Law of Large Numbers suffers from practical and philosophical problems discussed in Secs. 11.10 and 11.12. Imagination is an indispensable element of research and decision making but imaginary sequences are not a substitute for real sequences. I have already explained why the frequency interpretation does not explain why a specific property of estimators, unbiasedness, is useful in practice. I will now give a number of reasons, of various nature, why the frequency interpretation does not support the statistical theory of estimation in general. When multiple data sets, measurements of the same physical quantity, are collected, they often differ by their size (number of data) — this alone disqualifies a sequence of data sets from being an i.i.d. sequence, or a collective in the sense of von Mises. Frequency statisticians do not require that estimators are applied only to very large data sets. The accuracy of the estimator depends on the amount of data, and, of course, the larger the number of data, the better the accuracy. This is never taken to mean that the estimator cannot be used for a small number of data. It is left to the end user of statistical methods to decide whether the estimator is useful for any particular amount of data. This indicates that the idea of von Mises that only very long sequences should be considered is mostly ignored. Frequency statisticians do not hesitate to analyze models which are far from von Mises’ collectives, for example, stationary processes and Markov processes. Some stochastic processes involve complex dependence between their values at different times and so they are far from collectives. Some families of stochastic processes can be parameterized, for example, distinct members of a family of stationary processes may be labeled by one or several real numbers. There is nothing that would prevent a frequency statistician

page 296

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch13

Frequency Statistics

297

from estimating the parameters of a process in this family. A frequency statistician would feel no obligation to find an i.i.d. sequence of trajectories of the process. A single trajectory of a stochastic process can be the basis for estimation in frequency statistics.

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

13.2.1 Estimation and (L1)–(L6) The theory of estimation can be justified in several ways using (L1)–(L6), just like the theory of confidence intervals. Consider a single statistical problem of estimation. Recall that, according to (L6), a verifiable statistical statement is a prediction, that is, an event of probability close to 1. The only practical way to generate a prediction from an estimator is to construct a confidence interval using this random variable. When we estimate several parameters at the same time, or in the infinite-dimensional (“non-parametric”) case, a confidence interval has to be replaced with a “confidence set,” that is, a subset of a large abstract space. Another justification based on (L1)–(L6) for the use of estimators comes from aggregate statistical problems. Suppose that n independent estimates are made, and each one is used to construct a 95% confidence interval. If n is sufficiently large then one can make a prediction that at least 94% of the intervals will cover the true values of the parameters, with probability 99.9% or higher. Finally, just like in case of confidence intervals, we may apply the decision theoretic approach. We may express losses due to errors using the units of money or utility. Once the losses are expressed in the units of money or utility, one can apply, at least in principle, one of probabilistic techniques to find the distribution of the aggregate loss and generate a corresponding prediction, for example, a 99.9% confidence interval for the combined loss. Predictions generated by estimators face the same practical challenges as those based on confidence intervals, and are equally needed — see Sec. 13.1.

13.2.2 Unbiasedness — a concept with a single application There is only one practical situation where the notion of unbiasedness has a natural frequency interpretation. It is estimation of the current time by

page 297

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

298

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch13

Resonance: From Probability to Epistemology and Back

owners of various clocks and watches. In this example, estimates, that is, times shown by different clocks, are (hopefully) unbiased. With some stretch of reality, the estimates are not very far from being identically distributed and not very far from being independent. The estimated quantity — time — may be considered to be an unknown constant. The unbiasedness of the estimates has a practical meaning — the overall loss experienced by the society due to time estimate errors is likely to be minimized if the estimates are unbiased. Combining different estimates offered by different clocks is not practical in everyday life. In all other practical situations, one of the following problems makes the frequency interpretation of unbiasedness unrealistic. (i) Sometimes, it is unrealistic to consider the unknown parameter to be an unknown constant because it is quite clear that it has an objective prior distribution in the Bayesian sense. This may be the case of laboratory measurements of cholesterol level in blood. Available data may give rise to an objective prior for a cholesterol level for a random patient. Data collected for an individual may generate a prior distribution for cholesterol level of this specific person. (ii) Sometimes the unknown quantity is a constant, for example, when scientists measure the speed of light. But different measurements are not identical. Equipment used to estimate the speed of light evolves over time. (iii) If there is a large number of identical and independent measurements done by various experimenters, it is best to combine the data and derive one estimate of the unknown quantity. It makes no sense to derive a large number of i.i.d. estimates. Unbiasedness is a simple statistical concept that can be introduced early on in statistical education but it does not seem to have strong following among professional statisticians. This does not diminish the value of my remarks about the lack of frequency interpretation for unbiasedness. I used unbiasedness as an elementary example of a property of an estimator computed under the assumption that the parameter is an unknown constant. Any other property of an estimator or an asymptotic property of a sequence of estimators, such as consistency, calculated under the assumption that the parameter is an unknown constant is almost impossible to interpret using frequencies, just like unbiasedness.

page 298

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch13

Frequency Statistics

299

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

13.3 Hypothesis Testing The theory of estimation, a part of frequency statistics, has nothing in common with von Mises’ theory of collectives. It cannot be justified by the “frequency interpretation” of probability either, except in very few special cases. The relationship between hypothesis testing (see Sec. 18.2) and von Mises’ collectives, and the relationship between hypothesis testing and the frequency interpretation of probability are more subtle. Hypothesis tests are prone to two kinds of errors. One can make a Type I error also known as “false positive”, that is, the incorrect rejection of the null hypothesis. The other possible error is called Type II error or “false negative”, that is, the incorrect acceptance of the null hypothesis. There are at least three common settings for hypothesis testing. First, in some industrial applications of testing, one has to test large numbers of identical items for defects. The probabilities of false negatives and false positives have a clear interpretation as long run frequencies in such situations. Another good example of a long run of isomorphic hypothesis tests is a series of medical tests, say, for HIV. In a scientific laboratory, a different situation may arise. A scientist may want to find a chemical substance with desirable properties, say, a new drug. He may perform multiple hypothesis tests on various substances, and the models involved in these testing problems may be very different from one another. A statistical prediction in this case would involve the percentage of false rejections or false acceptances of a hypothesis. Just like in the case of estimation, one cannot use the von Mises theory of collectives in this case because the tests are not necessarily identical. Nevertheless, if we assume that they are independent then we can make a prediction about the aggregate rate of false rejections or aggregate rate of false acceptances. The third type of situation when hypothesis tests are used is when a single hypothesis is tested, with no intention to relate this test to any other hypothesis test. A good example is a criminal trial. In the US, the guilt of a defendant has to be proved “beyond reasonable doubt.” Since the jury has only two choices — guilty or not guilty — criminal trials are examples of hypothesis testing, even if rarely (likely never) are they formalized using the statistical theory of hypothesis testing. If a jury votes “guilty,” it effectively makes a prediction that the defendant committed the crime. In this case, the prediction refers to an event in the past. In a criminal trial, a prediction is the result of a single case of hypothesis testing. A long run frequency

page 299

April 29, 2016

9:6

300

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch13

Resonance: From Probability to Epistemology and Back

interpretation of hypothesis testing in case of criminal trials is possible but has a questionable ethical status. Presumably, in a given criminal trial, the society expects the jury to make every effort to arrive at the right decision in this specific case.

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

13.3.1 Hypothesis tests and collectives Hypothesis testing has a split personality. There is a limited number of industrial and medical applications of hypothesis testing that fit very well into von Mises’ framework of collectives. However, collectives are not used in any way except to make a prediction about the long run frequencies of testing errors. In practice, one can make the same prediction using the much more convenient concept of an i.i.d. sequence. The other two common applications of hypothesis testing are individual tests and sequences of non-isomorphic hypothesis tests. None of them fit into the von Mises theory. A good way to tell the difference between the various approaches to hypothesis testing is to analyze the reaction of the person in charge of the test to an erroneous testing decision. If an erroneous rejection of the null hypothesis is met with great concern then we have the case of an individual hypothesis test. If the error is considered a random fluctuation, a normal price that has to be paid, we have the situation best represented as a sequence of tests. In the first case, if a testing error is discovered, the testing procedure might be criticized and an improvement proposed. In the latter case, there would be no compelling reason to try to improve the testing procedure. Long sequences of hypothesis tests do not automatically fit into the von Mises philosophy. First, a statistician may use different significance levels for various tests in a sequence. Even if the same significance level is used for all tests in a sequence and, therefore, the tests form an i.i.d. sequence, they do not necessarily form a collective. The intention of von Mises was to reserve the concept of a collective for sequences of events that were physically identical, not only mathematically identical. In the scientific setting, where large numbers of completely different hypothesis tests are performed, such a long sequence is the opposite of a von Mises’ collective. Elements of a collective have everything in common, except probability (because a single event does not have a probability). A long sequence of hypothesis tests may have nothing in common except the significance level (the probability of an error).

page 300

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch13

Frequency Statistics

301

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

13.3.2 Hypothesis tests and the frequency interpretation of probability The frequency interpretation of probability does apply to hypothesis tests and, in theory, provides a good philosophical support for this method. Recall the two situations where long runs of hypothesis tests are applied. First, we may have a long run of isomorphic tests, with “simple” null and alternative hypotheses. For example, a machine part can be declared to be either defective or not defective. Then one can predict the frequency of false positives, that is, false classifications of parts as defective. Similarly, we can predict the frequency of false negatives. Both frequencies can be observed in practice in some situations, and this can be easily turned into a prediction in the sense of (L6). Hence, the theory of hypothesis testing is a well justified science. A subtle point here is that both frequencies, of false positives, and false negatives, are within sequences that are unknown to the statistician. This is because the statistician does not know in which tests the null hypothesis is true. Hence, in many situations the predicted frequencies are not directly observable. One could argue that the frequencies of false positives and false negatives may be observable in the future, when improved technology allows us to revisit old tests and determine which null hypotheses were true. Strictly speaking, this turns the predicted frequencies of false positives and false negatives into scientifically verifiable statements. But this does not necessarily imply that predictions verifiable in this sense are useful in practice. Hence, it is rational to recognize the theory of hypothesis testing as a scientific theory (in the sense given above), but to reject it in some practical situations because its predictions are not useful.

13.3.3 Hypothesis testing and (L1)–(L6) Generally speaking, hypothesis testing fits into the framework of (L1)-(L6) just like confidence intervals and estimators do. One can try to generate predictions in the sense of (L6) based on a single hypothesis test, or on a sequence of hypothesis tests. There are some differences, though.

13.3.3.1 Sequences of hypothesis tests Consider a sequence of hypothesis tests and, for simplicity, assume that the significance level is 5% in each case. Moreover, suppose that the tests are independent, but not necessarily isomorphic. Then, in the long run, the

page 301

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

302

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch13

Resonance: From Probability to Epistemology and Back

percentage of false positives, that is, cases when the null hypothesis is true but it is rejected, will be about 5%. This is (or rather can be transformed into) a scientific prediction, in the sense of (L6). Let me repeat some remarks from the last section to make sure that my claim is not misunderstood. The statistician does not know which null hypotheses are true at the time when he performs the tests. He may learn this later, for example, when a better technology is available. In case of a medical test for a virus, the condition of a patient may change in the near future so that it will be known whether he has the virus, or not. The time in the future when we learn whether the null hypothesis is true or not with certainty may be very distant, depending on the specific problem. Hence, the prediction, although scientifically verifiable in the abstract sense, may be considered to be irrelevant, because of the time delay. A prediction concerning false negatives can be generated in a similar way, but this is a bit more complicated — I will try to outline some difficulties below. There is a number of practical situations when the null hypothesis and the alternative hypothesis are simple, for example, a company may want to classify parts as defective or non-defective. However, there are also practical situations where hypotheses are not simple, for example, when the rates of side effects for two drugs are either identical (the null hypothesis) or they differ by a real number between −1 and 1 (the alternative). Now the issue of generating a prediction for false negatives in the long run setting is complicated by the fact that it is not obvious how to choose the appropriate sequence of tests in which a predicted frequency of false negatives will be observed. Should we consider all cases when the two rates of side effects are different? In theory, this would mean all tests, as it is virtually impossible that the difference between the rates is exactly zero. Or should we consider only those cases when the difference of rates is greater than some fixed number, say, 5%?

13.3.3.2 Single hypothesis test A single hypothesis test can be considered as an application of (L6) in its pure form. We incorporate the null hypothesis into our model. Then we make a prediction that a certain random variable (the “test statistic”) will not take a value in a certain range. We observe the value of the test statistic. If the value is in the “rejection region,” we conclude that something was wrong with the theory that generated the prediction, and usually this means that we reject the null hypothesis.

page 302

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch13

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Frequency Statistics

303

The above algorithm is a correct application of (L6), and shows that a single hypothesis test is a scientific procedure. There remain some issues to be addressed, though. The first one is the question of the significance level. A popular significance level is 5%. In other words, a hypothesis test implicitly assumes that an event is a prediction if its probability is 95% or higher. The choice of probability which makes an event a prediction is subjective. Personally, I find 95% too low. I would hesitate to draw strong conclusions if an event of probability 95% failed. I would prefer to raise the level to 99%, but this choice makes effective hypothesis testing more costly. Second, the interpretation of hypothesis testing described above, as a pure application of (L6), may be too crude for scientific purposes. Consider applications of geometry. For some practical purposes, we can consider an automobile wheel to be a circle. For some other purposes, engineers have to describe its shape with much greater precision. They would conclude that a wheel is not a circle when measured with great accuracy. In the simplest setting, the result of testing a hypothesis is its rejection or acceptance. Hence, hypothesis testing makes an impression of a crude method. In practice, it is often supplemented by the “p-value,” an extra piece of information for the statistician and end users of the test. The p-value is the probability that we observe the actual data or “more extreme” data if the null hypothesis is actually true. The p-value does not have a simple frequency interpretation although it is used by frequentists. It is very unlikely that a sequence of isomorphic hypothesis tests will yield identical or very similar p-values. To put it differently, it would be very unusual for a collection of hypothesis tests grouped together because they yielded very similar p-values to have a practical application.

13.4 Hypothesis Testing and (L6) I have already pointed out a similarity between my interpretation of (L6) and testing of statistical hypotheses. Despite unquestionable similarities, there are some subtle but significant philosophical differences — I will outline them in this section. Testing a statistical hypothesis often involves a parametric model, that is, some probability relations are taken for granted, such as exchangeability of the data, and some parameters, such as the expected value of a single measurement, are considered unknown. The hypothesis to be tested usually refers to the parameter, whose value is considered to be “unknown.” Hence, in hypothesis testing, only one part of the model can be falsified by the

page 303

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

304

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch13

Resonance: From Probability to Epistemology and Back

failure of a probabilistic prediction. In general, the failure of a prediction in the sense of (L6) invalidates some assumptions adopted on the basis of (L2)–(L5). There is no indication in (L6) which of the assumptions might be wrong. One of the standard mathematical models for hypothesis testing involves not only the “null hypothesis,” which is often slated for rejection, but also an alternative hypothesis. When a prediction made on the basis of (L1)–(L6) fails, and so one has to reject the model built using (L2)– (L5), there is no alternative model lurking in the background. This is in agreement with general scientific practices — a failed scientific model is not always immediately replaced with an alternative model; some phenomena lack good scientific models, at least temporarily. One can wonder whether my interpretation of (L6) can be formalized using the concept of hypothesis testing. It might, but doing so would inevitably lead to a vicious circle of ideas, on the philosophical side. Hypothesis testing needs a scientific interpretation of probability and so it must be based on (L1)–(L6) or a similar system. In any science, the basic building blocks have to remain at an informal level, or otherwise one would have to deal with an infinite regress of ideas.

13.5 Hypothesis Testing and Falsificationism Hypothesis testing is a perverse version of falsificationism. In the standard version of falsificationism, a theory is weeded out if it appeared to be true but made a prediction inconsistent with the result of an experiment. In applications of hypothesis testing, one hopes for the falsification of a straw-man hypothesis. So, it is not true that scientists always try to formulate theories that are likely to pass empirical tests. Sometimes scientists formulate theories with the intention and the hope of rejecting them. Of course, the final goal is the creation of a true scientific theory — only the means involve a perverse version of falsificationism.

13.6 Does Frequency Statistics Need the Frequency Philosophy of Probability? Why do frequency statisticians need the frequency philosophy of probability, if they need it at all? They seem to need it for two reasons. First, some of the most elementary techniques of frequency statistics agree very well with the theory of collectives. If you have a deformed coin with an unknown

page 304

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch13

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Frequency Statistics

305

probability of heads, you may toss it a large number of times and record the relative frequency of heads. This relative frequency represents probability in the von Mises theory, and it is also the most popular unbiased estimator in frequency statistics. A closely related method is to average measurement results because this reduces the impact of measurement errors. For some scientists, the long run frequency approach is the essence of probability because it is arguably the most widely applied probabilistic method in science. Frequency statisticians use the frequency theory in an implicit way to justify the use of expectation in their analysis. An estimator is called “unbiased” if its expected value is equal to the true value of the unknown parameter. Many people subconsciously like the idea that the expected value of the estimator is equal to the true value of the parameter even if they know that the mathematical “expected value” is not expected at all in most cases. An implicit philosophical justification for unbiased estimators is that the expected value is the long run average, so even if our estimator is not quite equal to the true value of the unknown parameter, at least this is true “on average.” The problem here, swept under the rug, is that, in a typical case, there is no long run average to talk about, that is, the process of estimation of the same unknown constant is not repeated by collecting a long sequence of isomorphic data sets. Overall, applications of the frequency interpretation of probability are limited in frequency statistics to a few elementary examples and some confusion surrounding expectation.

page 305

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch14

Chapter 14

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Bayesian Statistics

Bayesian statistics is a very successful branch of science because it is capable of making excellent predictions, in the sense of (L6). It is hard to find anything that de Finetti’s philosophy and Bayesian statistics have in common. I will list and discuss major differences between the two in this chapter. The general structure of Bayesian analysis is universal — the same scheme applies to all cases of statistical analysis. One of the elements of the initial setup is a “prior,” that is, a prior probability distribution, a consistent view of the world. The data from an experiment or observations are the second element. A Bayesian statistician then applies the Bayes theorem to derive the “posterior,” that is, the posterior probability distribution, a new consistent view of the world. The posterior can be used to make decisions — one has to find the expected value of the gain associated with every possible decision and take the action that maximizes this expectation.

14.1 Two Faces of Subjectivity There are several fundamental philosophical differences between de Finetti’s theory and Bayesian statistics.

14.1.1 Non-existence versus informal assessment See Secs. 5.1 and 5.23 for the discussion of various meanings of the word “subjective.” In de Finetti’s philosophical theory, the word means “nonexistent” because this is the only meaning that fits de Finetti’s main philosophical idea that probability is a quantity that is not measurable in any objective or scientific way. In Bayesian statistics, the word “subjective” 307

page 307

April 29, 2016

9:6

308

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch14

Resonance: From Probability to Epistemology and Back

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

means “informally assessed.” Typically, the word refers to the informal summary of scientist’s prior knowledge in the form of the prior distribution. Some priors are considered to be subjective in the sense that statisticians cannot justify their beliefs in an explicit way. That does not mean that Bayesian scientists consider prior distributions to be completely arbitrary. Quite the opposite, a prior distribution that is based on some solid empirical evidence, for example, some observations of long run frequencies, is considered preferable to a purely informal prior distribution.

14.1.2 Are all probabilities subjective? In de Finetti’s theory, it is necessary to assume that all probabilities are subjective. If de Finetti admits that some probabilities are objective then his theory collapses, on the philosophical side. If even a single probability in the universe is objective then the philosopher has to answer all relevant philosophical questions, such as what it means for the probability to exist in the objective sense or how to measure the probability in the objective sense (see Secs. 3.5 and 5.17). Bayesian statisticians are willing to use subjective prior distributions but they never consider their posterior distributions to be equally subjective. One could even say that the essence of Bayesian statistics is the transformation of subjective priors into objective posteriors.

14.1.3 Conditioning versus individuality Individuals, ordinary people and scientists alike, sometimes express probabilistic views different from those of other individuals. In Bayesian statistics, different statisticians or users of statistics may want to use different prior distributions because the priors reflect their personal knowledge and knowledge varies from person to person. We can represent this as P (A | B)
= P (A | C). Here B and C stand for different information that different people have and the mathematical formula says that different people may estimate the probability of an event A in a different way, because they have different prior information. De Finetti was not the first person to claim that different people may have different opinions about the future because they have different information about the past. This claim is a part of every scientific and philosophical theory of probability and it is not controversial at all; it is a simple mathematical fact expressed by the mathematical formula given above. The main claim of de Finetti’s philosophical theory can be represented by a different formula, namely, P1 (A | B)
= P2 (A | B).

page 308

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch14

Bayesian Statistics

309

De Finetti approved discrepancy between assessments of the probability of a future event by two people even if they had the same information because, according to him, there was no objective way to verify who was right and who was wrong.

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

14.1.4 Non-existent decisions De Finetti’s theory is based on the idea that probability does not exist but we can use the calculus of probability to coordinate decisions so that they are rational, that is, consistent. Hence, according to de Finetti, the Bayes theorem in Bayesian statistics is not used to calculate any real probabilities because they do not exist. The Bayes theorem is a mathematical tool used to coordinate decisions made on the basis of the prior and posterior distributions. This philosophical idea does not match standard Bayesian practices at all. Bayesian statisticians see nothing wrong with collecting data first and starting the statistical analysis later. The prior distribution is chosen either to represent the prior knowledge of the scientist or in a technically convenient way. There is no attempt to choose the prior distribution so that it represents decisions made before the start of data collecting. I have never heard about a Bayesian statistician trying to coordinate the mythical prior decisions with the posterior decisions.

14.2 Elements of Bayesian Analysis Recall the general structure of Bayesian analysis from the beginning of this chapter. One of the elements of the initial setup is a “prior,” that is, a prior probability distribution, a consistent view of the world. The data from an experiment or observations are the second element. A Bayesian statistician then applies the Bayes theorem to derive the “posterior,” that is, the posterior probability distribution, a new consistent view of the world. The posterior can be used to make decisions — one has to find the expected value of the gain associated with every possible decision and take the action that maximizes this expectation. This simple and clear scheme conceals an important difference between philosophical “priors” and Bayesian “priors.” In the subjective philosophy, the prior is a complete probabilistic description of the universe, used before the data are collected. In Bayesian statistics, the same complete description of the universe is split into a “model” and a “prior” (see example below). These poor linguistic practices lead to considerable confusion. Some people

page 309

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

310

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch14

Resonance: From Probability to Epistemology and Back

believe that using (statistical) subjective priors is a legitimate scientific practice, but the same people would not be willing to accept the use of subjective models. On the philosophical side, there is no demarcation line between subjective priors and subjective models. Consider tosses of a deformed coin — this simple example will illustrate the statistical usage of the words “prior” and “model.” The prior distribution will be specified in two steps. First, a “model” will be found. The model will involve some unknown numbers, called “parameters.” The term “prior” refers in Bayesian statistics only to the unknown distribution of the “parameters.” In a sequence of coin tosses, the results are usually represented mathematically as an exchangeable sequence. According to de Finetti’s theorem, an exchangeable sequence is equivalent (mathematically) to a mixture of “i.i.d.” sequences. Here, an “i.i.d. sequence” refers to a sequence of independent tosses of a coin with a fixed probability of heads. The assumption that the sequence of tosses is exchangeable is a “model.” This model does not uniquely specify which i.i.d. sequences enter the mixture and with what weights. The mixing distribution (that is, the information on which i.i.d. sequences are a part of the mixture, and with what weights), and only this distribution, is customarily referred to as the “prior” in Bayesian statistics.

14.3 Models 14.3.1 Bayesian models are totally objective In Bayesian analysis, models are treated as objective representations of objective reality. One of the common misunderstandings about the meaning of the word “subjective” comes here to play. Bayesian statisticians may differ in their opinions about a particular model that would fit a particular real life situation — in this sense, their views are subjective. For example, some of them may think that the distribution of a given random variable is symmetric, and some others may have the opposite opinion. This kind of subjectivity has nothing to do with de Finetti’s subjectivity — according to his theory, symmetry in the real world, even if it is objective, is not linked in any way to probabilities, because probability values cannot be objectively determined. Hence, according to the subjective theory, differences in views between Bayesian statisticians on a particular model are totally irrelevant from the point of view of the future success of the statistical analysis — no matter what happens, nothing will prove that any particular model is

page 310

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch14

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Bayesian Statistics

311

right or wrong. I do not find even a hint of this attitude among Bayesian statisticians. The importance of matching the model to the real world is taken as seriously in Bayesian statistics as in frequency statistics. Bayesian statisticians think that it is a good idea to make their mathematical model symmetric if the corresponding real phenomenon is symmetric. In other words, they act as if they believed in some objective probability relations. Bayesian models are based on (L1)–(L6) and other laws, specific to each case of statistical analysis. See Sec. 14.7.1 for further discussion of Bayesian models.

14.3.2 Bayesian models are totally subjective See Sec. 11.1.7 for the initial part of this discussion. Bayesian models are subjective because the prior can be corrected by the data but the model typically is not corrected by the data. The model arises from resonance and we can only have faith that it is correct. Of course, models can be tested and are tested in statistical practice in some cases but then untested resonance is a component of the verification procedure for the given model. In this specific and narrow sense, Bayesians developed inverted beliefs about what (prior or model) is objective and subjective in their analysis.

14.4 Priors One could expect that of all the elements of the Bayesian method, the prior distribution would be the most subjective. Recall that in practice, the term “prior distribution” refers only to the opinion about the “unknown parameters,” that is, that part of the model which is not determined by (L1)–(L6) or some other considerations specific to the problem, in a way that can be considered objective. There are strong indications that priors are not considered subjective and that they do not play the role assigned to them by the subjective theory. One of them is a common practice to choose the prior after collecting the data. This disagrees with the subjective ideology in several ways. From the practical point of view, one can suspect that the prior is tailored to achieve a particular result. From the subjectivist point of view, the prior is meant to represent decisions made before collecting the data — the fact that the prior is often chosen after collecting the data shows that there were no relevant decisions made before collecting the data and so there is no need to coordinate anything.

page 311

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

312

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch14

Resonance: From Probability to Epistemology and Back

Surprisingly, Bayesian statisticians discuss the merits of different prior distributions. This suggests that they do not believe in the subjectivity of priors. If the prior is subjective in the personal sense, that is, if it reflects one’s own opinion, then there is nothing to discuss — the prior is what it is. Moreover, deliberations of various properties of priors indicate that some priors may have some demonstrably good properties — this contradicts the spirit and the letter of the subjective theory. According to the subjective theory, no subjective prior can be shown to be more true than any other prior. Hence, one could try to derive some benefits by simplifying the prior. Many priors can save money and time by reducing the computational complexity of a problem. For example, we could assume that the future events are independent from the data and the past. Then the Bayes theorem implies that the posterior distribution is equal to the prior distribution. Hence, one does not have to do any calculations — the savings of time and money can be enormous. In the context of deformed coin tossing, a very convenient prior is the one that makes the sequence of coin tosses i.i.d. with probability of heads equal to 70%. This subjective opinion does not require Bayesian updating when the results of coin tossing are observed — the posterior is always the same as the prior. Needless to say, in Bayesian statistics, priors are never chosen just on the basis of their technical complexity. Bayesian statisticians clearly believe that they benefit in an objective way by rejecting simplistic but computationally convenient priors. Bayesian statisticians choose priors to obtain the most reliable predictions based on the posterior distributions. The matter is somewhat complicated by mathematical and technical limitations. Not all priors lead to tractable mathematical formulas and some priors require enormous amount of computer time to be processed. Setting these considerations aside, we can distinguish at least three popular ways of choosing priors in Bayesian statistics: (i) an application of (L1)–(L6), (ii) a technically convenient probability distribution, not pretending to represent any real probabilities, and (iii) informal summary of statistician’s knowledge. I will discuss these choices in more detail.

14.4.1 Objective priors The adjective “objective” in the title of this subsection indicates that some priors involve probability assignments that can be objectively verified, for

page 312

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch14

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Bayesian Statistics

313

example, by long run repetitions that do not involve any Bayes-related reasoning. Textbook examples show how one can choose a prior using (L1)–(L6). Suppose that there are two urns, the first contains 2 black and 7 white balls, and the second one contains 5 black and 4 white balls. Someone tosses a coin and samples a ball from the first urn if the result of the toss is heads. Otherwise a ball is sampled from the second urn. Suppose that the color of the sampled ball is black. What is the probability that the coin toss resulted in tails? In this problem, the prior distribution assigns equal probabilities to tails and heads, by symmetry. More interesting situations arise when long run frequencies are available. For example, suppose that 1% of the population in a certain country is infected with HIV, and an HIV test generates 1% false negatives and 10% false positives. (A false positive is when someone does not have HIV but the test says he does.) If someone tests positive, what is the probability that he actually has HIV? In this case, the prior distribution says that a person has HIV with probability 1%. This is based on the long run frequency and, therefore, implicitly on (L1)–(L6). In other words, a symmetry is applied because the tested person is indistinguishable, to the tester, from other people in the population.

14.4.2 Bayesian statistics as an iterative method Some priors play the role of the “seed” in an iterative method. Such methods are popular in mathematics and numerical analysis. Suppose that we want to find a solution to a differential equation, that is, a function that solves the equation. An iterative method starts with a function S1 , that is, a “seed.” Then one has to specify an appropriate transformation S1 → S2 that takes S1 into a function S2 . Usually, the same transformation is used to map S2 onto S3 , and so on. The method works well if one can prove that the sequence S1 , S2 , S3 , . . . converges to the desirable limit, that is, the solution of the differential equation. The convergence has been proved in many cases. This iterative method is used in both pure mathematics and applied numerical methods. The seed S1 is not assumed or expected to be a solution to the differential equation or to be even close to such a solution. Not all seeds will generate a sequence of Sk ’s converging fast to the desirable limit, and the number of iterations needed for a good approximation of the solution depends on the problem itself and on the seed. The choice of an efficient transformation Sk → Sk+1

page 313

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

314

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch14

Resonance: From Probability to Epistemology and Back

and the seed S1 is a non-trivial problem with no general solution — the answer depends on the specific situation. The Bayesian method can be interpreted as an iterative scheme when the number of data is reasonably large. The following two procedures are equivalent from the mathematical point of view. The standard algorithm is to combine the prior distribution with all of the available data, using the Bayes theorem, to obtain the posterior distribution. An alternative, mathematically equivalent representation, is to start by combining the prior distribution with a single piece of data to obtain an intermediate distribution. This new distribution can be combined with another single piece of data to obtain another intermediate distribution, and so on. When we finish the process by including the last piece of the data, the resulting distribution will be the same as the one obtained in one swoop. The general success of iterative methods suggests that Bayesian statistics might be successful as well, because it can be represented as an iterative scheme. This is indeed the case in typical situations. I will illustrate the claim with a simple example. Consider tosses of a deformed coin. It is popular to use the “uniform” prior, that is, to assume that the sequence of tosses is a mixture of i.i.d. sequences, each i.i.d. sequence represents a coin with the probability of heads equal to p, and p itself is a random variable which lies in any subinterval [a, b] of [0, 1] with probability b − a. If there were k heads in the first n trials then the probability of heads on the (n + 1)st toss is (k + 1)/(n + 2). For large n, this is very close to k/n, a value that many people would consider a good intuitive estimate of the probability of heads on the (n + 1)st toss. The reliability of the estimate (k + 1)/(n + 2) can be confirmed by real data. The uniform prior in the last example does not represent any “subjective opinion” and it does not represent any “objective probability” either. It is a seed in an iterative method, and it works in an almost magical way — no matter what your coin is, you can expect the posterior distribution to yield excellent probability estimates. I said “magical” because in mathematics in general, there is no reason to expect every iterative method and seed to be equally efficient. The practice of using the prior distribution as an abstract seed in an iterative method is perfectly well justified by (L1)–(L6), because one can empirically verify predictions implicit in the posterior distribution, in the spirit of (L6). An important lesson from the representation of some Bayesian algorithms as an iterative method is that they may yield little useful information

page 314

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch14

Bayesian Statistics

315

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

if the data set is not large. What this really means depends, of course, on the specific situation. It is clear that most people assume, at least implicitly, that the value of the posterior is almost negligible when the model and the prior are not based on (L1)–(L6) or the data set is small. The subjective philosophy makes no distinction whatsoever between probability values arrived at in various ways — they are all equally subjective and unverifiable.

14.4.3 Truly subjective priors A prior may represent scientist’s prior knowledge in an informal way. Using such priors is a sound scientific practice but this practice might be the most misunderstood element of Bayesian statistics. Statisticians know that using personal prior distributions often yields excellent results. Hence, a common view is that subjective priors “work” and hence the subjective theory of probability is vindicated. It is clear that Bayesian statisticians expect posterior distributions to generate reliable predictions. Hence, the question is why and how personal priors can be the basis of objectively verifiable predictions? Resonance is the source and the explanation of reliable personal priors. In the Bayesian context, resonance is needed to construct reliable models and it is also needed to generate reasonable prior distributions. I will discuss two scenarios of statistical analysis — one with small amount of data and the other one with a large amount of data. Before discussing these scenarios, let me recall an elementary fact from Bayesian statistics — in a typical statistical model, no matter what data are, a suitable prior distribution can totally overwhelm the data and almost completely determine the posterior distribution. Hence, the choice of the prior distribution cannot be arbitrary — that would lead to nonsensical statistical inference. If the amount of data is small, the prior distribution determines the posterior distribution to large extent and hence has to be chosen with great care. The prior opinion of a scientist may be based on pure resonance. The prior distribution may be also the result of informal processing of information that, under the best of circumstances, could have been processed in an explicit and formal way. The prior distribution generated in this way may be somewhat different from the distribution that would have been generated from the same information by formal mathematical calculations. But that does not mean such an informal prior is completely different from the results of a formal calculation. This is similar to the informal assessment of temperature. We cannot precisely measure the

page 315

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

316

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch14

Resonance: From Probability to Epistemology and Back

outside temperature using our own senses but that does not mean that our subjective opinions about temperature are hopelessly inaccurate or useless. Hence, some personal priors work because they are not much different from “objective priors” discussed in Sec. 14.4.1. Our resonance abilities, at different levels and acquired in different ways, explain our success in choosing priors that lead to reliable posterior distributions and predictions based on them. Another reason why priors representing personal knowledge often work well is that sometimes they are combined with large amounts of data. In such a case, the posterior distribution is not very sensitive to the choice of the prior. In other words, in such cases, the personal prior plays the role of the seed in an iterative method and quite often its intrinsic value is irrelevant. This does not mean that the personal prior can be arbitrary. Recall that every data set can be overwhelmed by some prior distribution. Hence, resonance is needed to specify a prior that will let the data determine the posterior distribution. If the amount of data is large, the influence of the prior on the posterior is small. This means that resonance is needed to eliminate only the most extreme priors. The collection of reasonable priors is large and resonance is needed only to find a prior in this large collection. Priors that summarize personal knowledge in an informal way are not guaranteed to generate reliable posterior distributions. They summarize what cannot be built into the model — this alone is suspicious. They are prone to prejudices of various kinds, from scientific to social and personal. The subjective ideology is harmful to Bayesian statistics because it muddles the distinction between priors which represent information gathered and processed in an informal way and priors that are objective or are seeds of an iterative algorithm. It has been proposed that some priors, for example, uniform priors, represent the “lack of knowledge.” This may be an intuitively appealing idea but it is hard to see why a prior representing the “lack of knowledge” is useful in any way. The sole test of the value of a prior is the quality of predictions generated by the posterior distribution corresponding to the prior. To see the true role of the prior, consider the following example. Suppose that a scientist has to make a decision. She decides to use the Bayesian approach, chooses a prior, and collects some data. When she is finished, the computer memory fails, she loses all the data and she has no time to collect any more data before making a decision. According to the subjective philosophy, the prior represents the basis for the best course of

page 316

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch14

Bayesian Statistics

317

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

action in the absence of any additional information. Hence, according to the subjective theory, the statistician has to make the decision on the basis of her prior. In reality, nobody seems to be trying to choose a prior taking into account a potential disaster described above. If anything like this ever happened, there would be no expectation that the prior chosen to fit with the whole statistical process (including data collection and analysis) would be useful in any way in the absence of data.

14.5 Resonance at Work Bayesian statistics suffers from the confusion caused by the lack of understanding of resonance but provides an excellent opportunity to illustrate how resonance works in practice in a non-trivial situation. Statistical and in general scientific theories are often presented as models with parameters. The model is obtained through the process of resonance. Once the model is constructed, it can be analyzed by the usual scientific process of falsification/verification based on logic. The space of models is discrete in the following sense. If a model is rejected, there is no reason to believe that a better model for the same phenomenon should be “close” to the original one, although it could. The parameters turn each model into a vector space or manifold (a continuous structure). The process of resonance is not sufficiently precise to provide the values of the parameters. But resonance gives bounds for the values of the parameters. The Bayesian prior is a way to encode beliefs based on resonance about the part of the parameter space where the parameter values reside. Sometimes no useful bounds can be found and the support of the prior distribution is the whole parameter space, as in the case of the uniform distribution. Some priors, for example Jeffrey’s priors, try to encode some general beliefs obtained through resonance, for example beliefs about scaling and symmetry. Models are discrete and parameters are continuous in the following sense. Models are supposed to be accepted or rejected. Acceptance does not mean perfection, just sufficient applicability. Parameters are supposed to be adjusted. The best values are chosen among infinitely many values in the space. A model is always based on resonance. Sometimes the model can be based on raw data with no understanding of the laws of nature underlying the phenomenon. This was the case of diffusion models for stock market

page 317

April 29, 2016

9:6

318

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch14

Resonance: From Probability to Epistemology and Back

prices in the past. More recently, similar models were derived from basic assumptions about the behavior of investors.

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

14.6 Data It happens sometimes that the observed data do not seem to fit the model at all. In its pure subjectivist version, the Bayesian approach is totally inflexible — the posterior distribution must be derived from the prior and the data using the Bayes theorem, no matter what the prior and the data are. In practice, when the data do not match the model, the model is modified. This practice is well justified by (L1)–(L6), as an attempt to improve the reliability of predictions. The subjective philosophy provides no justification for changing the model or the prior, once the data are collected. Paradoxically, the subjective philosophy provides no support for the idea that it is better to have a lot of data than to have little data — see Sec. 5.12. Since the subjective philosophy is only concerned with consistency of decisions, collecting data is not needed. Collecting data will not make it any easier for the decision maker to be consistent than in the case when he has little or no data. Needless to say, Bayesian statisticians believe that collecting data is beneficial.

14.7 Posteriors The posterior has the least subjective status of all elements of Bayesian statistics, mainly because of the social pressure. Business people, scientists, and ordinary people would have nothing to do with a theory that emphasized the subjective nature of its advice. Hence, the subjectivity of (some) priors is mentioned occasionally but posterior distributions are implicitly advertised as objective. Take, for example, the title of a classical textbook [DeGroot (1970)] on Bayesian statistics, “Optimal Statistical Decisions.” Optimal? According to the subjective theory of probability, your opinions can be either consistent or inconsistent — they cannot be true or false, and hence your decisions cannot be optimal or suboptimal. Of course, you may consider your own decisions optimal, but this does not say anything beyond the fact that you have not found any inconsistencies in your views — the optimality of your decisions is tautological. Decisions may be also optimal in some purely mathematical sense, but I doubt that that was the intention of DeGroot when he chose the title for his book. The

page 318

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch14

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Bayesian Statistics

319

title was chosen, consciously or subconsciously, to suggest some objective optimality of Bayesian decisions. The posterior distribution is the result of combining the prior and the model with the data. Quite often, the prior is not objective (see Sec. 14.4.1) so the posterior is not based on (L1)–(L6) alone. This is one reason why posterior probability assignments are not always correct, in the sense of predictions, as in (L6). The weakest point of the philosophical foundations of Bayesian statistics is that they do not stress the necessity of a proof (in the sense of (L6)) that the posterior distribution has desirable properties. The subjective philosophy not only fails to make such a recommendation but asserts that this cannot be done at all. Needless to say, Bayesian statisticians routinely ignore this part of the subjective philosophy and verify the validity of their models, priors and posteriors in various ways. From time to time, somebody expresses an opinion that the successes of Bayesian statistics vindicate the claims of the subjective philosophy. The irony is that according to the subjective theory itself, nothing can confirm any probabilistic claims — the only successes that the Bayesians could claim are consistency and absence of Dutch book situations — this alone would hardly make much of an impression on anyone.

14.7.1 Non-convergence of posterior distributions One of the most profound misconceptions about de Finetti’s theory and Bayesian statistics is the claim that even if two people have two different prior distributions then their posterior distributions will be closer and closer to each other, as the number of available data grows larger and larger. This misunderstanding is firmly rooted in the ambiguity of the word “prior.” The philosophical “prior” includes the statistical “prior” and “model.” The basic example of a situation when Bayesian posterior distributions of two people will converge when more and more data are available is when the data sequence is objectively exchangeable. This claim is supported by empirical data. But it seems to me that this argument is used by some people who declare themselves to be subjectivists. Such an intellectual position of a subjectivist is an example of hypocrisy, of course. Consider the following example. Let Xk be the number of heads minus the number of tails in the first k tosses of a coin. Let Yk be 0 if Xk is less than 0 and let Yk be equal to 1 otherwise. The sequence X1 , X2 , X3 , . . . is known as a simple random walk, and the sequence Y1 , Y2 , Y3 , . . . is known not to be exchangeable. Suppose that two people are shown a number of values

page 319

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

320

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch14

Resonance: From Probability to Epistemology and Back

of Y1 , Y2 , Y3 , . . . , but only one of them knows how this sequence of zeroes and ones was generated. The other person might assume that Y1 , Y2 , Y3 , . . . is exchangeable, not knowing anything about its origin. If he does so, the posterior distributions of the two observers will not converge to each other as the number of observations grows. The above elementary example is quite typical. If two statisticians do not agree on the model then there is no reason to think that their posterior distributions will be close to each other, no matter how much data they observe. Conversely, if two statisticians adopted the same model but used two different priors then their posterior distributions will be closer and closer as the number of data grows, under mild technical assumptions. I propose to turn the above observations into the following “axiomatic” definition of a model in Bayesian statistics. A “model” consists of a family of probability distributions describing the future (“philosophical priors”), and a sequence of random variables Z1 , Z2 , Z3 , . . . (“data”), such that for any two probability distributions in this family, with probability one, the posterior distributions will converge to each other as the number of observed values of Z1 , Z2 , Z3 , . . . grows to infinity. In the context of (L1)–(L6), a model represents objective reality. Opinions of Bayesian statisticians who adopted the same model will converge to each other, as the number of available data grows. One is tempted to assert that the opinions will actually converge to the objective truth but my discussion of skepticism dictates caution on this point. From the practical point of view, the convergence of posterior distributions is the most we can hope for. The identification of the limiting opinion with the objective truth is a pragmatic philosophical position, even if it is vulnerable to criticism from the skeptical direction. The above discussion of Bayesian models paints a picture of the world that is far too optimistic. According to the above vision, rational people should agree on an objective model, and collect enough data so that their posterior distributions are close. Thus achieved consensus would be a reasonable substitute for the objective truth. The catch is that my definition of a model (and all limit theorems in probability and statistics) assume that the number of data grows to infinity. In practice, the number of data is never infinite. We may be impressed by numbers such as “trillion” but even trillion data can be easily overridden by a sufficiently singular prior distribution. In other words, for a typical Bayesian model, and an arbitrarily large data set, one can find a prior distribution which will totally determine the posterior distribution, pushing all the data aside. One may dismiss this

page 320

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch14

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Bayesian Statistics

321

scenario as a purely theoretical possibility that never occurs in practice. I beg to differ. When it comes to religion, politics and social issues, some people will never change their current opinions, no matter what arguments other people may present, or what new facts may come to light. Some of this intransigence may be explained away by irrationality. In the case of dispute between rational people, the irreconcilable differences may be explained by the use of different models. But I believe that at least in some cases, groups of rational people use the same model but start with prior distributions so different from one another that no amount of data that could be collected by our civilization would bridge the gap between the intellectual opponents.

14.8 Bayesian Statistics and (L1)–(L6) Methods of Bayesian statistics can be justified just like the methods of frequency statistics — see Secs. 13.1 and 13.2.1. Briefly, Bayesian statisticians can generate predictions in the sense of (L6). These can take the form of confidence intervals (called “credible intervals” in the Bayesian context). Predictions can be also based on aggregates of statistical problems, or can be made using decision theoretic approach. The general discussion of practical problems with predictions given in Sec. 13.1 applies also to predictions based on Bayesian posterior distributions. Recall why these somewhat impractical predictions are needed. Bayesian statisticians may use their own methods of evaluating their techniques, as long as users of Bayesian statistics are satisfied. Predictions, in the sense of (L6), are needed because of the existing controversy within the field of statistics. Critics of Bayesian statistics must be given a chance to falsify predictions made by Bayesian statisticians. If the critics fail to falsify them then, and only then, Bayesian statisticians may claim that their methods form a solid branch of science.

14.9 Spurious Predictions The Bayesian approach to statistics has some subtle problems that are understood well by statisticians at the intuitive level but are rarely discussed explicitly. Recall that some priors are used only as seeds in an iterative method. These priors introduce probabilities which are not meant to be used. For example, suppose that a statistical consultant analyzes many problems dealing with parameters in the interval [0, 1] over his career. Typically,

page 321

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

322

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch14

Resonance: From Probability to Epistemology and Back

such priors represent unknown probabilities. Suppose further that he always uses the uniform prior on [0, 1], a mathematically convenient distribution. Assuming that the consultant treats different statistical problems as independent, his choice of priors generates a prediction (in the sense of (L6)) that about 70% of time, the true value of the parameter will lie in the interval [0, 0.7]. It is clear that very few people would make such a prediction. In fact, the uniform prior is commonly considered to be “uninformative,” and so it is not supposed to be used in any direct predictions.

14.10 Who Needs Subjectivism? There are (at least) two reasons why some Bayesian statisticians embrace the subjective philosophy. One is the mistaken belief that in some cases, there is no scientific justification for the use of the prior distribution except that it represents the subjective views of the decision maker. I argued that the prior is sometimes based on (L1)–(L6) and other forms of resonance, sometimes it is the seed of an iterative method, and sometimes it is the result of informal processing of information. The justification for all of these choices of the prior is quite simple — predictions based on posterior distributions seem to be reliable. Another reason for the popularity of the subjective philosophy among some Bayesians is that the subjective theory provides an excellent excuse for using the expected value of the (utility of) gain as the only determinant of the value of a decision. As I argued in Secs. 12.2.1, 12.5.2 and 12.6, this is an illusion based on a clever linguistic manipulation — the identification of decisions and probabilities is true only by a philosopher’s fiat. If probabilities are derived from decisions, there is no reason to think that they represent anything in the real world. The argument in support of using the expected value is circular — probabilities and expectations are used to encode a rational choice of decisions and then decisions are justified by appealing to thus generated probabilities and expectations. Bayesian statisticians often point out that their methods “work” and this proves the scientific value of the Bayesian theory. Clearly, this is a statement about the methods and about the choice of prior distributions. It is obvious that if prior distributions had been chosen in a considerably different way, the results would not have been equally impressive. Hence, prior distributions have hardly the status of arbitrary opinions. They are

page 322

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch14

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Bayesian Statistics

323

subjective only in the sense that a lot of personal effort went into their creation. One could say that (some) Bayesian statisticians are victims of the frequency statisticians’ propaganda. They believe in the criticism directed at Bayesian statistics that subjectivity and science do not mix. As a reaction to this criticism, they try to justify their usage of subjective priors by invoking de Finetti’s philosophical theory. In fact, using subjective priors is just fine because the whole science is subjective in the same sense as subjective priors are. Science is about matching idealized theories with the real world, and the match is necessarily imperfect and subjective. Bayesian priors are not any more subjective than, for example, assumptions made by physicist explicitly and implicitly in the Big Bang theory. The only thing that matters in all sciences, including statistics, is the quality of predictions.

14.11 Preaching to the Converted Many of the claims and arguments presented in this chapter are known to and accepted by (some) Bayesian statisticians. It will be instructive to see how philosophical issues were addressed in two books on Bayesian analysis, [Berger (1985)] and [Gelman et al. (2004)]. I start with a review of a few statements made in [Gelman et al. (2004)], a graduate level textbook on Bayesian statistics. On page 13, the authors call the axiomatic or normative approach “suggestive but not compelling.” On the same page, they refer to the Dutch book argument as “coherence of bets” and they say that “the betting rationale has some fundamental difficulties.” At the end of page 13, they say about probabilities that “the ultimate proof is in the success of the applications.” What I find missing here is an explanation of what the “success” means. In my theory, the success is a prediction, in the sense of (L6), that is fulfilled. The authors seem to believe in predictions because they say the following on page 159, More formally, we can check a model by external validation [emphasis in original] using the model to make predictions about future data, and then collecting those data and comparing to their predictions. Posterior means should be correct on average, 50% intervals should contain the true values half the time, and so forth.

The above may suggest that the only probabilistic predictions that can be made are based on long run frequencies. Frequency based predictions are just an example of probabilistic predictions, that is, events of very high probability. The only special thing about long run frequencies is that, quite

page 323

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

324

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch14

Resonance: From Probability to Epistemology and Back

often, they are the shortest path to predictions with very high probabilities, thanks to the Large Deviations Principle. I am not sure how to interpret remarks of the authors of [Gelman et al. (2004)] on subjectivity on pages 12 and 14. It seems to me that they are saying that subjectivity is an inherent element of statistical analysis. In my opinion, all their arguments apply equally well to science in general. As far as I know, standard textbooks on chemistry do not discuss subjectivity in their introductions, and so statistical textbooks need not to do that either (except to present historical misconceptions). Overall, I consider the discussion of philosophical issues in Sec. 1.5 of [Gelman et al. (2004)] level headed and reasonable. However, the fundamental philosophical problem of verification of probability statements is swept under the rug. On pages 12 and 13, the authors show that the frequency approach to the problem of confirmation of probability values has limitations, but they do not present an alternative method, except for the nebulous “success of the applications” at the top of page 14. Berger’s book [Berger (1985)], a monograph on decision theory and Bayesian analysis, is especially interesting because Berger does not avoid philosophical issues, discusses them in detail, and takes a pragmatic and moderate stance. All this is in addition to the highest scientific level and clarity of his presentation of statistical techniques. I will argue that Berger completely rejects de Finetti’s philosophy but this leaves his book in a philosophical limbo. I could not find a trace of de Finetti’s attitude in Berger’s book. The Dutch book argument is presented in Sec. 4.8.3 of [Berger (1985)] and given little weight, on both philosophical and practical sides. The axiomatic approach to Bayesian statistics is described in Sec. 4.1. IV of [Berger (1985)]. Berger points out that axiomatic systems do not prove that “any Bayesian analysis is good.” Berger clearly believes that (some) objective probabilities exist and he identifies them with long run frequencies; see, for example, the analysis of Example 12 in Sec. 1.6.3 of [Berger (1985)]. Berger calls subjective probability “personal belief” in Sec. 3.1 of his book. The best I can tell, Berger does not mean by “personal belief” an arbitrary opinion, but an informal assessment of objective probability. In Sec. 1.2, he writes that Bayesian analysis “seeks to utilize prior information.” I interpret this as scientifically justified (but possibly partly informal) processing of objective information.

page 324

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch14

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Bayesian Statistics

325

The philosophical cracks are shown in Berger’s book at several places. Berger is a victim of the frequentist propaganda — he believes that frequency, when available, is an objective measurement of objective probability, but otherwise we do not have objective methods of verifying statistical methods. For example, in Sec. 3.3.4, Berger points out that in some situations there appear to be several different “non-informative priors.” Berger’s discussion of this problem is vague and complicated. He does mention a “sensible answer” without defining the concept. In my approach, the problem with competing non-informative priors is trivial on the philosophical side. One should use these priors in real applications of statistics, generate useful predictions in the sense of (L6), and then see how reliable the predictions are. One can generate a prediction on the basis of a single case of Bayesian statistical analysis (a credible interval), or one can use an aggregate of independent (or dependent!) cases of Bayesian analysis to generate a prediction. It might not be easy to verify a given prediction, but this problem affects all sciences, from high energy particle physics to human genomics. I am troubled by the unjustified use of expectation in Berger’s exposition. In Sec. 1.6.2 of [Berger (1985)], we find a standard justification of the use of expectation — if we have repeated cases of statistical analysis then the long run average of losses is close to the expectation of loss. However, it is clear that Berger does not believe that statistical methods are applicable only if we have repeated cases of statistical analysis. Berger’s rather critical and cautious attitude towards the Dutch book argument and axiomatic systems indicates that he does not consider them as the solid justification for the use of expectation. This leaves, in my opinion, the only other option — expected value is implicitly presented as something that we should expect to observe. I have already expressed my highly negative opinion about this intuitive justification of expectation in Sec. 12.2.1 of this book. Berger gives seven justifications for the use of Bayesian analysis in Sec. 4.1 of [Berger (1985)]. This alone may raise a red flag — why is not any one of them sufficient? Do seven partial justifications add up to a single good justification? In fact, all these justifications are perfectly good, but we have to understand their role. The only scientific way for Bayesian analysis to prove its worth is to generate reliable predictions, as described in the present book. Berger’s seven justifications can be used before we verify any predictions, to justify the expense of labor, time and money. Once we

page 325

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

326

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch14

Resonance: From Probability to Epistemology and Back

determine that Bayesian analysis generates useful and reliable predictions, the seven justifications may be used to explain the success of Bayesian analysis, to make improvements to the existing methods, and to search for even better methods to make predictions. Overall, [Berger (1985)] overwhelms the reader (especially the beginner) with an avalanche of detailed philosophical analysis of technical points. What is lost in this careful analysis is a simple message that statistical theories can be tested just like any other scientific theory — by making and verifying predictions.

14.12 Constants and Random Variables One of the philosophical views of frequency and Bayesian statistics says that one of the main differences between the two branches of statistics is that the same quantities are treated as constants by frequency statistics, and as random variables by Bayesian statistics. For example, suppose that a statistician has some data on tosses of a deformed coin. A frequency statistician would say that the probability of heads is an unknown constant (not a random variable). The data are mathematically represented as random variables; of course, once the data are collected, the values of the random variables are known. A Bayesian statistician considers the data to be known as constants. The probability of heads on any future toss of the same coin is an unknown number and, therefore, it is a random variable. The reason is that a subjectivist decision maker must effectively treat any unknown number as a random variable. I consider the above distinction irrelevant from the point of view of (L1)–(L6). A statistical theory can be tested only in one way — by verifying its predictions. Predictions are events which have high probabilities. Generally speaking, frequency and Bayesian statisticians agree on what can be called an “event” in real life. They can point out events that they think have high probabilities. The users of statistics can decide whether predictions are successful or not. A statistical theory may be found to be weak if it makes very few predictions. Another statistical theory may be found erroneous if it makes many predictions that prove to be false. It is up to the users of statistics to decide which theory supplies the greatest number of reliable and relevant predictions. Whether a statistician considers parameters constants or random variables, and similarly whether data are considered constants or random variables, seems to be irrelevant

page 326

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch14

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Bayesian Statistics

327

from the user’s point of view. These philosophical choices do not seem to be empirically verifiable, unlike predictions. Let us consider a specific example. Is the speed of light an unknown constant or a random variable? One can use each of these assumptions to make predictions. First, let us assume that the speed of light is an unknown constant. Suppose that, in the next five years, the accuracy of measurements of the speed of light will not be better than 10−10 in some units and that in the same interval of time, about 100 (independent) 90%-confidence intervals will be obtained by various laboratories. Assume that 100 years from now, the accuracy of measurements will be much better, say, 10−100 , so, by today’s standards, the speed of light will be known with perfect accuracy. All confidence intervals obtained in the next five years could be reviewed 100 years from now and one could check if they cover the “true value” of the speed of light, that is, the best estimate obtained 100 years from now. We can assert today that more than 85% of confidence intervals obtained in the next five years will cover the “true value” of the speed of light — this is a prediction obtained by combining frequency statistical methods and (L6). In other words, the last statement describes an event and gives it a very high probability. Next, I will argue that we can generate scientific predictions if we assume that the speed of light is a random variable. Suppose that over the next 100 years, the speed of light is estimated repeatedly with varying accuracy. The results of each experiment are analyzed using Bayesian methods and a posterior distribution for the speed of light is calculated every time. If the posterior distributions are used for some practical purposes, one can calculate the distribution of combined losses (due to inaccurate knowledge of the speed of light) incurred by our civilization over the next 100 years. One can use this distribution to make a prediction that the total accumulated losses will exceed a specific value with probability less than, say, 0.1%. This prediction is obtained by combining Bayesian methods and decision theoretic ideas with (L6). Verifying predictions described in the last two paragraphs may be very hard in practice. But the idea may be applied with greater success when we limit ourselves to a scientific quantity of lesser practical significance than the speed of light. My real goal is not to suggest a realistic scientific procedure for the analysis of various measurements of the speed of light. I want to make a philosophical point — verifiable predictions, in the sense

page 327

April 29, 2016

9:6

328

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch14

Resonance: From Probability to Epistemology and Back

of (L6), are not dependent on whether we consider scientific quantities to be constants or random variables.

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

14.13 Criminal Trials Criminal trials present an excellent opportunity to test a philosophical theory of probability. In the American tradition, the guilt of a defendant has to be proved “beyond reasonable doubt.” First I will discuss criminal trials in the Bayesian framework. Suppose jurors are Bayesians and, therefore, they have to start with a prior opinion before they learn anything about the defendant. Here are some possible choices for the prior distribution. (i) Use symmetry to conclude that the probability of the defendant being guilty is 50%. This is likely to be unacceptable to many people because the prior probability of being guilty seems to be very high, inconsistent with the principle that you are innocent until proven guilty. The appeal to symmetry is highly questionable — what is symmetric about guilt and innocence? The symmetry seems to refer to the gap in our knowledge — using this symmetry to decide someone’s fate does not seem to be well justified. (ii) Suppose that 2% of people in the general population are convicted criminals. Use exchangeability to conclude that the prior probability of the defendant being guilty is 2%. This use of symmetry in the form of exchangeability is questionable because the defendant is not randomly (uniformly) selected from the population. (iii) Suppose that 80% of people charged with committing a crime are found to be guilty. One could argue that this piece of information, just like all information, must be built into the prior because one must always process every bit of data in the Bayesian framework. Hence, one could start with the prior probability of the defendant being guilty equal to 80%, or some other number larger than 50%. Many people would find it totally unacceptable that the mere fact that someone is charged with committing a crime increases the probability that he is guilty. This seems to contradict the presumption of innocence and opens a way for abuse of power. (iv) One can argue that none of the uses of symmetry outlined in (i)–(iii) is convincing so there is no symmetry that can be used to assign the prior probability in an objective way. Hence, one has to use a personal

page 328

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch14

Bayesian Statistics

329

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

prior. This suggestion invites an objection based on the past history — white juries used to have negative prejudices against black defendants. A prior not rooted strongly in the objective reality is suspect. The difficulties in choosing a good prior distribution are compounded by difficulties in choosing the right utility function. What is the loss due to convicting an innocent person? What is the loss due to letting a criminal go free? And who should determine the utility function? The jurors? The society? The unjustly imprisoned person? Jurors may choose to approach their decision problem using a method developed by frequency statistics — hypothesis testing. It is natural to take innocence of the defendant as the null hypothesis. I do not see any obvious choice for the significance level, just like I do not see any obvious choice for the prior distribution in the Bayesian setting. Both approaches to the decision problem, Bayesian and hypothesis testing, can generate predictions in the sense of (L6). Predictions can be made in at least two ways. First, one can make a prediction that a given innocent defendant will not be found guilty or that a given criminal will not be found innocent. One of the problems with this “prediction” is that it might not have a sufficiently large probability, by anyone’s standards, to be called a prediction in the sense of (L6). Another problem is that it may be very hard to verify whether such a “prediction” is true — this is almost a tautology. Jury trials were instituted to deal with the cases when neither guilt nor innocence were totally obvious. Another possible prediction in the context of jury trials can be made about percentages, in a long sequence of trials, of defendants that are falsely convicted and criminals that are found not guilty. This prediction might be verifiable, at least approximately, using statistical methods. And the percentages can be changed by educating jurors and adjusting the legal system in other ways. I consider this prediction to be the most solid of all probabilistic approaches to criminal trials. However, I do see potential problems. If the legal system is tailored to achieve certain desirable percentage targets, defendants may feel that verdicts in their individual cases will be skewed by general instructions given to juries that have nothing to do with their individual circumstances. I do not have an easy solution to the problem of criminal trials in the context of (L1)–(L6). I do not think that the frequency and subjective theories can offer clear and convincing solutions either.

page 329

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch15

Chapter 15

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

On Ideologies

15.1 On Ideologies and Their Photo-Negatives The main thesis of this section is that practitioners of the so-called frequency approach to probability apply probabilistic ideas in ways that are exactly opposite to the theoretical claims of the philosophical theory known as the frequency theory of probability. Similarly, practitioners of the so-called Bayesian approach to probability apply probabilistic ideas in ways that are exactly opposite to the theoretical claims of the philosophical theory known as the subjective theory of probability. I expect that a typical reader will be immediately skeptical about my claims. Most people believe that ideologies are implemented in imperfect ways, for example, no real democratic state is a perfect democracy. But claiming that the real life implementation of an ideology is a photo negative image of its dogmas seems to be no more than a rhetorical device. Not so. Before I discuss philosophies related to probability, I will review some history of two other ideologies, Catholicism and communism, to show that the photo negative effect is real and not uncommon. These two ideologies and the accompanying historical events are known, at least partly, to many people. If the reader agrees with my analysis of these two ideologies, he may be willing to accept my claims about the photo negative effect in the area of philosophy of probability. I will not only analyze the relationship between ideologies and some historical events but I will also try to explain the psychological and social mechanisms that lead to the photo negative effect. Christians chose the cross as the primary symbol of their religion. Of all messages contained in the Bible, the cross singles out the message of the ultimate non-violence. Jesus was God, according to Christians, and as such, he could have defeated his enemies in the physical and immediate sense.

333

page 333

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

334

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch15

Resonance: From Probability to Epistemology and Back

Instead, he chose the terrible and humiliating death on the cross. One cannot invent a more potent symbol than an omnipotent supreme being subjected to the most horrifying death out of his own will. I will match the above element of the Christian ideology with a particular historical event — the conquest of America by the Spaniards (and, to some extent, the Portuguese). There were many examples of non-Christian behavior of (mostly) Christian groups of people in more recent times but I prefer to talk about events that are in the distant past so that we can think about them in a (more or less) dispassionate way. The conquest of America was an act of unprovoked aggression. The major civilizations and states on the continent (Aztecs and Incas) were destroyed. The populations were decimated mainly by the diseases brought by the Europeans but also by direct human actions (combat, etc.). All the valuables (for example, gold) that could be hauled across the Atlantic were robbed. The conquest of America was performed by the agents of the Spanish monarchy. Christianity was the ideology of Spain and of the conquistadores. The contrast between the values symbolized by the Christian cross and the reality of the conquest was striking. Among various elements of the Marxist philosophy, perhaps the most profound one is the claim that the matter controls the spirit and not, as people commonly believe, vice versa. For example, it may seem that a king decides to start a war in his mind and thus greatly affects the material world. In the Marxist view of the world, king’s decision is the consequence of the material conditions in his kingdom via intermediaries such as the social structure. A corollary of this fundamental axiom of Marxist philosophy is that social revolutions can and do come only when the material (economic) conditions are ripe for them. There were a number of communist revolutions but only two of them had a major and lasting impact on the world. These were the Russian and Chinese revolutions. Each of these revolutions occurred in a pre-industrial society, in the obvious conflict with the Marxist theoretical predictions. I have to note parenthetically that Marx himself contemplated a revolution in the pre-industrialized Russian society. Contradictions in philosophical writings are not unusual — the temptation to allow for exceptions whenever convenient is simply too strong. I will now address a couple of natural questions about the contradictions between the two ideologies and their accompanying historical events. The first question is why people ignored the ideologies. The answer seems to be rather obvious. The leaders wanted to achieve some practical goals and they did not care about the consistency of the goals and methods applied to

page 334

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch15

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

On Ideologies

335

achieve them with the dominating ideology, or they self-deluded themselves that there was consistency. The rulers had no choice, in a sense. A conquest of a continent or a major social revolution in a large country are huge undertakings. Rank and file participants in those events needed more than just a utilitarian justification for their actions. Everybody wants to be rich but projects on a cosmic scale require cosmic scale ideologies. The Spanish had only one such ideology at hand, and the same can be said about the communist revolutionaries. The two ideologies were used to support the struggle not because they provided good rational arguments but because they were the only reasonably well established ideologies available to the leaders. The second question is whether it would have been wise or atleast moral for the leaders to tell their subjects or followers that the practical undertakings were inconsistent with the philosophical dogmas. Should they have told the ordinary people that the tenets of the ideologies would be ignored for some time so that the practical goals could be attained, and only then the world would achieve the cosmic balance including the consistency of the ideology and reality? It seems that very few leaders follow this intellectually (and morally?) honest path. Ideology is treated as a weapon and nobody wants to blunt a weapon at a time of struggle. Now I am ready to argue that statistical methods are photo negative images of the philosophical theories that supposedly represent them. First, let me recall a few facts. De Finetti and von Mises created philosophical theories to address certain philosophical questions. Their theories turned out to be radical. In particular, both philosophers claimed that individual events did not have probabilities. Von Mises claimed that individual events did not have probabilities but it was meaningful to apply probability to a sequence of identical observations or experiments (a “collective”). For this theory to be applicable, we must be able to recognize a collective in real life, that is, we have to be able to recognize in some way that the experiments or observations are “identical.” It follows from von Mises’ theory that it is impossible to compare probabilities of individual events because they exist neither in theory nor in practice. Frequency statisticians apply hypothesis testing in various contexts. One of them is scientific, in which various researchers test completely different hypotheses about completely different phenomena. The procedure is justified in the following way. Although there is no guarantee that the

page 335

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

336

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch15

Resonance: From Probability to Epistemology and Back

conclusion of an individual test is correct, the long run frequency of correct conclusions can be made high by appropriate application of mathematical formulas and collection of sufficient amount of data. Statistical tests in the scientific context are not identical so they do not form a “collective.” We believe that the long run frequency of correct conclusions is high only because we calculate the probability of an error for every test separately. In this way, the theory of hypothesis testing, commonly regarded to be the most frequentist of all statistical methods, is a photo negative image of the philosophical frequency theory of probability created by von Mises. The contrast between Bayesian statistics and its closest philosophical ally, the subjective theory of probability invented by de Finetti, is at least as large. The starting point of de Finetti’s philosophy was the claim that probability did not exist. This claim may appear to be plain lunatic but it can be given an interpretation that sounds quite rational. De Finetti said that none of the algorithms developed to measure probability of a single event was sufficiently reliable from the logical and practical point of view to be accepted as a scientific method. De Finetti claimed that the most we could achieve in face of uncertainty was to make our actions “consistent” in an appropriate sense, to avoid losses that were preventable in a specific deterministic way. The common interpretation of Bayesian statistics is that it can deal with randomness in situations where one cannot observe frequencies. In other words, Bayesian approach can be applied to individual events. This is the photo negative version of de Finetti’s theory. In his theory, no event has probability. For a Bayesian, every event has a probability. Are frequency and Bayesian statisticians undereducated? Are they irrational? Don’t they know how to apply logic? Can’t they see that their practices are the photo negative images of their philosophies? I think that statisticians behave just like the Spanish monarchs and revolutionary leaders. They are engaged in a mortal struggle with the representatives of the other branch of statistics. To win, they have to be equipped with a good ideology, among other things. Frequency statisticians could not find a better ideology than the frequency theory of probability. Many Bayesians are not quite happy with the subjective theory of probability or with the identification of Bayesian statistics and subjective thinking. But they did not find a different philosophical theory that could support their branch of statistics. The reality of the scientific struggle is in some respects the same as that of the armed struggle. Just like Catholic monarchs and communist

page 336

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch15

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

On Ideologies

337

leaders, statisticians instinctively feel that it is better to claim that their practical actions are consistent with their ideologies, despite all the evidence to the contrary, than to admit that there is an inconsistency. The winners write history books. In science, the winners will determine the shape of both practice and the dominating scientific theory in the future. I will end this section with a pessimistic conclusion concerning the progress in the ideological struggle between frequency and Bayesian statistics. I have to confess that I believe that my theory of probability, encapsulated in (L1)–(L6), is the best scientific theory of probability available at this point. But even if I abandon this opinion and imagine that some other theory of probability is better than (L1)–(L6), I see little chance that statisticians will adopt the new theory, no matter how convincing it might be. The first group of statisticians who abandon their current philosophical crutches risk losing the war. Only if one branch of statistics scores a clear victory over the other do I envision potential openness of mind among the victors, sufficient to admit that their scientific practice is the photo negative version of their philosophical theory.

15.2 Experimental Statistics — A Missing Science The rules that apply to ordinary followers of a religion do not necessarily apply to high priests. Ordinary Catholic men must remove their hats when they enter a church. This “universal” rule does not apply to bishops. Suppose that a company wants to introduce a new drug. Before the drug is approved, the authorities, doctors and patients expect the company to perform clinical trials, preferably with a control group and using the “double blind” protocol. The collected data have to be analyzed using the best available statistical techniques and only then the drug can be approved. If the company proposed to approve a drug on the basis of general or philosophical arguments, without any supporting data, it would be ridiculed. Or it might be considered to be a representative of “alternative medicine,” where scientific standards are largely ignored. In any case, a proposal to approve a drug without supporting data does not fit into modern science in any way. The above simple and intuitive rules of scientific approach to medicine apparently do not apply to statistics. Statisticians are high priests of data analysis. They analyze data for all other scientists and thus make modern science possible. However, statistics is an exceptional science where methods can be approved and used without any evidence based on hard data.

page 337

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

338

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch15

Resonance: From Probability to Epistemology and Back

General or philosophical arguments are sufficient to justify a method. My sarcastic remarks are inspired by the article [Lecoutre and Poitevineau (2011)]. The article discusses various problems with significance tests and considers several alternative approaches to data analysis. The truly amazing aspect of this article is that the authors do not think that it is necessary to support their claims with data. If, as the authors claim, significance tests are imperfect and confidence intervals or “prep” give better results then there surely must be some empirical evidence to support these claims? In this context, empirical evidence could include a count of correct or incorrect decisions, records of small or large discrepancies between estimated and true values, financial gains or losses incurred by users of statistics, etc. In some sciences data are very hard to collect because of technological limitations, difficult access, financial costs, etc. (consider, for example, collecting data on supernova explosions or human genome). Nevertheless, scientists strive to collect at least some data and refrain from far reaching conclusions until at least some empirical evidence is available. The striking feature of the statistical debate exemplified in the article quoted above is that not only empirical evidence is not presented at all but there is no explanation for why it is missing. The unavoidable impression is that the evidence is missing not because it is hard to collect but because statisticians do not consider empirical evidence to be desirable. An interesting contradiction in psychology was pointed out in [Nickerson (2004)]). A large number of psychological studies of probabilistic reasoning assume that a rational person should use Bayesian approach to decision making. At the same time, many of these research papers in psychology use methods of frequency statistics, such as hypothesis testing. A similar contradiction exists in statistics. Statisticians develop methods to study data collected by scientists working in other fields, such as biology, physics and sociology. Statisticians make little effort to collect data that could verify their own claims. When the results of statistical analysis are revisited many years later, say, 30 years later, then many former statistical hypotheses are currently known to be true or false with certainty (for all practical purposes), and many formerly unknown quantities can be now considered to be known constants (that is, their values are known with accuracy several orders of magnitude greater than what was achievable 30 years ago). Revisiting old statistical studies can generate a wealth of data on how different statistical methods performed in practice. There is too little systematic effort in this

page 338

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch15

On Ideologies

339

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

direction, and whatever various statisticians contributed to this scientific task, their efforts did not take the shape of a well defined, separate field of experimental statistics. There are several reasons why experimental statistics does not exist. (1) Statisticians collect or help to collect a lot of data. Data analysis is the essence and the bread and butter of their profession. Statisticians subconsciously think that this activity fulfills the scientific requirement for collecting data in support of statistical methods. Statisticians do not realize that collecting data to analyze various problems in other sciences is not the same as collecting data to verify the value of statistical methods. (2) There is no intellectual framework for experimental statistics. Hypothesis testing is used precisely because we cannot check whether a given hypothesis is true. This is implicitly taken as meaning that we will never know whether the hypothesis is true. In fact, there are many simple situations when the unknown hypothesis becomes a true or false fact within a reasonable amount of time. For example, it may become clear very soon whether someone is sick or whether a drug has severe side effects. Similarly, many people may have an impression that there is no way to determine experimentally whether an estimator is unbiased because the true value of the unknown quantity will be always unknown. Again, the quantity may be unknown today but it may become known (with great accuracy) in not too distant future. (3) Some statisticians may think that computer simulations are the experimental statistics. Everybody seems to understand the difference between a simulation of a nuclear explosion and an actual nuclear explosion. The same applies to statistical methods. Computer simulations can provide valuable information but they cannot replace analysis of actual data collected in the real universe. (4) If data on performance of statistical methods are ever collected then they will have to be analyzed using statistics. There is a clear danger that if frequency methods are used to analyze the data then they will show the superiority of frequency statistics, and the same applies to the potential misuse of Bayesian statistics. The bias of the analysis can be conscious or subconscious. However, when a sufficiently large amount of data are available then the choice of a statistical method might not matter. Everybody believes that smoking increases the probability of cancer, both frequentists and Bayesians.

page 339

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

340

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch15

Resonance: From Probability to Epistemology and Back

(5) Experimental statistics, or collecting of data on the past performance of statistical methods, may not be considered glamorous. A junior statistician is more likely to analyze a new estimator than to review data from the past because the first option is more likely to advance his academic career. (6) Reviewing past data may be costly. Granting agencies may be unwilling to fund this activity. This is unfortunate. I think that statistics has as much impact on our society as human genome and Big Bang, two well funded directions of research. (7) Poor performance of statistical inference in the past might have some non-statistical roots. For example, inaccurate estimates of the speed of light may be due to systematic bias or inadequate physical theories, none of which can be eliminated even by the best statistical techniques. Disentangling statistical and non-statistical causes of the failed predictions might not be an easy task. I realize that I am not fair by suggesting that statisticians never review the real life performance of their methods. I know that some do. My point is more philosophical than practical. The fact that statistics is an experimental science did not sink into the collective subconsciousness of statisticians. Textbooks and monographs such as [Berger (1985)] and [Gelman et al. (2004)] give too many philosophical arguments in comparison to the number of arguments of experimental kind. Even though the discussion of purely philosophical ideas presented in these books, such as axiomatic systems or the Dutch book argument, is less than complimentary, the reader may be left with an impression that philosophical arguments are on a par with experimental evidence. Solid data on the past performance are the only thing that is needed to justify a statistical method, and no amount of philosophy is going to replace the experimental evidence.

15.3 Statistical Time Capsules Statisticians may help their colleagues working in the distant future evaluate currently used statistical methods. One way to do that would be to include explicit isolated predictions in published results of statistical analysis. Typical results of a statistical analysis include a large number of probabilistic statements, implicit or explicit. A good example is the posterior distribution in Bayesian analysis. The posterior distribution is a collection of a large number of probability values. A prediction is an

page 340

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch15

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

On Ideologies

341

event of high probability. One should choose one or at most a handful of predictions from among all events of high probability present in the results of each case of statistical analysis. The prediction should be concerned with an event that is significant to the users of statistics. It should also be an event that is likely to be known to have happened (or not) in the future. Some events (or their complements), such as death, are likely to be unambiguously observable. Some other events, such as the value of a physical quantity falling outside a confidence intervals, may be observable in the future because of technological progress. Inserting a prediction in a statistical report would be similar to other activities in which people leave information for future generations. Examples include time capsules, to be opened hundreds of years from now, that will help future historians and archaeologists. More common examples of messages to future people are texts carved in tombstones, embedded in castle walls, etc. Explicit predictions inserted in statistical reports would help future generations of statisticians to evaluate various statistical methods. Statisticians can revisit old statistical reports and compare them to currently known facts without help of my proposed system of explicit predictions. But a typical statistical report contains a multitude of implicit predictions so the analysis of the past predictions may be biased by the present statistician’s choice of the past predictions to be confronted with the currently known facts. Inserting a single explicit prediction in a statistical report would remove a bit of subjectivity from the analysis of the results of statistical analysis.

15.4 Is Statistics a Science? I consider a direction in intellectual activity a hard-core science if scientists active in this area either achieved consensus on all essential issues or have a reasonable hope of achieving consensus on all essential issues in the future. Physics is an example of such a science. I realize that, in reality, achieving consensus is far from straightforward. For example, Ernst Mach opposed the atomic theory of matter even after substantial empirical evidence had been collected in favor of this theory. The current status of the string theory provides another example of difficulties on the way to consensus. It is not clear when and how physicists will achieve consensus on the string theory in view of the formidable difficulties in designing experiments that could verify this theory. Nevertheless, it is fair to say that physicists achieved consensus

page 341

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

342

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch15

Resonance: From Probability to Epistemology and Back

on relativity theory and quantum physics, two fundamental theories with highly counterintuitive claims. Let us compare physics to historical research. Some historical facts are controversial — there is no consensus on whether they occurred. But the main reason why historians do not have a consensus similar to that found in physics is that some of the most interesting and significant activities of historians are to select historical facts and interpret them. These are highly subjective activities and there is no reason why they would lead to consensus. Why do I care about consensus? The universe is incredibly complex. I have direct access to a tiny portion of all facts and similarly I am aware of and can understand only a handful of laws governing the universe. Consensus among scientists is the closest thing to the objective truth that exists in practice. I believe that most scientists are smart and honest. I believe in their theories because my only other choice is to be irrational or paranoid. The further we go from natural sciences towards humanities, art and religion, the less consensus we find; the more we have to rely on our own judgment. I do appreciate historical research and art but for qualities much different from intellectual certainty. I believe that statistics can be a science in the same sense as physics. The fact that it is split into two branches, frequency and Bayesian, is unsettling. There is no consensus among statisticians on some fundamental issues. This means that users of statistics must apply their own judgment — they cannot fully rely on results of research performed by statisticians in the same way that they can do it in an idealized science that I described above. Statistics does not generate intellectual material of the same kind as in historical research or art so it should try to evolve in the direction of natural sciences such as physics, in terms of consensus. Scientific controversies are not uncommon; a number of examples are given in [Kuhn (1970); Polanyi (1958)]. One of the best known clashes in science was concerned with the (modern) atomic theory of matter, proposed by John Dalton in 1805. Roughly speaking, chemists believed in the atomic theory of matter throughout the 19th century and physicists did not believe in this theory in the same period. Both sides had solid scientific arguments supporting their positions. At that time, one could imagine a crucial experiment proving the atomic theory. For example, one could imagine constructing a microscope so powerful that we could see individual

page 342

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch15

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

On Ideologies

343

atoms through it. Such a microscope, a “scanning tunneling microscope,” was built in 1981, long after the atomic theory had been accepted by the whole scientific community at the beginning of the 20th century. But the idea of such a microscope existed or could have existed in the 19th century, in the middle of the controversy. My problem with the frequency-subjective controversy within statistics is that it is not clear what kind of evidence could provide overwhelming support for one of the ideologies. The science of statistics is concerned with data. It is hard to imagine that the controversy is waiting to be settled until we collect even more data than the current “big data.” Computers available to statisticians become faster every year but I doubt that computers, no matter how fast, will provide the crucial evidence in favor of one of the theories. I am equally dubious about the ability of new mathematical theorems in the area of statistics to decisively influence the debate. What else is missing? I do not know so this is why I probe the philosophical foundations of statistics and probability in the search for an answer. My laws (L1)–(L6) are (in part) an attempt to turn statistics into a science in the sense outlined in this section. I would like to be able to open a statistical textbook or monograph knowing that all methods presented in the book are endorsed by the majority of statisticians. Such consensus would be a strong assurance for other scientists and laymen that the known statistical methods are reasonably reliable.

15.5 Psychoanalytic Interpretation of Philosophy of Probability Bruno de Finetti, the most prominent representative of the subjective philosophy of probability, made the claim that “Probability does not exist.” Richard von Mises, the most prominent representative of the frequency philosophy of probability, said that “We can say nothing about the probability of death [within a year] of an individual even if we know his condition of life and health in detail.” Both claims sound crazy (to me) but quantum physics also sounds crazy (to me); nevertheless physicists say that quantum physics is true. Radical claims, slogans and documents are natural components of many ideologies. Christians have the Ten Commandments. Communists have the Communist Manifesto. Most Christians do not follow all of the Ten Commandments all the time. Most communists do not follow the Communist Manifesto all the

page 343

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

344

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch15

Resonance: From Probability to Epistemology and Back

time. But true Christians believe in the the Ten Commandments and it is their ambition to follow the rules laid out in the Ten Commandments as much as possible. An analogous remark applies to communists. De Finetti’s and von Mises’ claims can be labeled as “anti-slogans” because they express ideas that their alleged followers try to avoid, not follow. Bayesian statisticians are trying to make their field of science as objective as possible. De Finetti’s slogan reminds them of their original sin, a connection of Bayesian statistics with subjectivity. Likewise, frequency statisticians are trying to make their science relevant in all situations involving randomness, not only those where long sequences of i.i.d. random variables are available. Von Mises’ claim is like a thorn in their body, painfully reminding them about possible limitations of their methods. De Finetti and von Mises were not charismatic figures leading their followers to the barricades. They were psychoanalysts who discovered and brought to light the worst subconscious fears of the two groups of statisticians.

15.6 From Intuition to Science Well developed ideologies have many components, for example, heuristic foundations, formal theory, and practical implementations. Probability also has such components. There are several sources and manifestations of probabilistic intuition. One can try to turn each one of them into a formal or scientific theory. Not all such attempts were equally successful. Here are some examples of probabilistic intuition. (i) Probabilities manifest themselves as long run relative frequencies, when the same experiment is repeated over and over. This observation is the basis of von Mises’ philosophy of probability. Although the stability of long run frequencies was successfully formalized in the mathematical context, as the Law of Large Numbers, the same intuitive idea proved to be a poor material for a philosophical theory (see Chapter 4). (ii) Probabilities appear as subjective opinions, for example, someone may be 90% certain that a (specific) defendant is guilty. This intuitive idea gave rise to the subjective theory of probability of de Finetti. There is an extra intuitive component in this idea, namely, that subjective opinions should be “rational,” that is, it is neither practical nor fair to have arbitrary subjective opinions. This is formalized as

page 344

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch15

On Ideologies

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

(iii)

(iv)

(v)

(vi)

(vii)

(viii)

345

“consistency” in de Finetti’s theory. This philosophical theory does not specify any connections between subjective opinions and the real world and hence it is placed in a vacuum, with no usable advice in most practical situations. Probabilities are relations between logical statements, as a weak form of implication. This idea gave rise to the logical theory of probability. In this theory, the concept of symmetry is embodied in the principle of indifference in a non-scientific way because the principle’s validity is not subject to empirical tests. The logical theory is not popular in science at all because the main intellectual challenge in the area of probability is not to provide a new logical or mathematical structure but to find a usable relationship between the purely mathematical theory based on Kolmogorov’s axioms and real observations. Symmetric events should have identical probabilities. This intuitive idea is incorporated in the classical and logical theories of probability. This assumption and the mathematical laws of probability can be used to calculate effectively some probabilities of interest. These, in turn, can be used to make inferences or decisions. However, symmetry alone is effective only in a limited number of practical situations. Physically unrelated events are mathematically independent. This observation is implicit in all theories of probability but it is not the main basis of any theory. Taken alone, it is not sufficient to be the basis of a complete philosophy of probability. Events whose mathematical probability is very close to 1 are practically certain to occur. Again, this observation alone is too weak to be the basis of a fully developed philosophy of probability but it is implicit in all philosophical theories. Probabilities may be considered physical quantities, just like mass or charge. This intuitive idea is the basis of the propensity philosophy of probability. A deformed coin falls heads up on a different proportion of tosses than an ordinary coin. This seems to be a property of the coin, just like its diameter and weight. This intuitive idea is hard to reconcile formally with the fact that the same experiment may be an element of two (or more) different sequences of experiments (see Sec. 11.10). Probability may be regarded as a quantitative manifestation of uncertainty. This intuitive idea is somewhat different from (ii) because it is less personal. Uncertainty may be objective, in principle.

page 345

April 29, 2016

9:6

346

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch15

Resonance: From Probability to Epistemology and Back

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

This intuitive idea seems to be one of motivations for de Finetti’s theory. (ix) An intuitive idea very close to (viii) is that probability is a way to relate unpredictable events to each other in a way that is better than using arbitrary opinions. Again, this intuitive idea seems to be present in de Finetti’s theory. The intuitive ideas presented in (i), (ii), (viii) and (ix) were transformed beyond recognition in the formal theories of von Mises and de Finetti. The basic philosophical claim common to both theories, that individual events have no probabilities, does not correspond to anything that could be called a “gut feeling.”

15.7 Science as Service Users of probability should have the right to say what they expect from the science of probability and how they will evaluate different theories. People sometimes have non-scientific needs and these should not be totally ignored by scientists. Some needs are rather nebulous, for example, a “profound understanding of the subject.” Here are some possible needs of users of probability. (i) Reliable predictions. This is what my theory, (L1)–(L6), offers (see Chapter 11). My theory of probability puts predictions at the center of the science of probability. This idea was present in some form in [Popper (1968)] and [Gillies (1973)], for example. (ii) A reliable way of calculating probabilities, at least in some situations. This is the essence of the classical philosophy of probability. The classical theory lacks clarity about independent events and the philosophical status of predictions. It is (L1)–(L6) in an embryonic state. (iii) Predictions in the context of long run frequencies. This is a special case of (i). Long run frequency predictions are not reliable any more than any other type of predictions. The frequency theory implicitly assumes that this is all that probability users need. This is not sufficient — users need predictions in other situations as well. (iv) A rational explanation of probability. I would say that the logical and propensity theories pay most attention to this need, among all philosophies of probability. This need is (or at least should be) at the

page 346

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch15

On Ideologies

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

(v)

(vi)

(vii)

(viii)

347

top of the philosophical “to do” list but ordinary users of probability do not seem to place it that high. Coordination of decisions. This is what the subjective theory offers. The problem is that the coordination of decisions offered by the subjective theory (“consistency”) is a very weak property. Most of important decisions in everyday life and science are well coordinated with other decisions, in the sense of consistency. Other theories of probability leave aside, quite sensibly, the decision theoretic questions because they form a separate intellectual challenge. Guidance for making rational decisions in face of uncertainty. This is what the subjective theory is supposed to offer according to some of its supporters. It does not. Its recommendations are weak to the point of being useless. Guidance in situations when a single random experiment or observation is available. Some subjectivists (but not the subjective theory) make empty promises in this area. Interpretation of data from random experiments. The frequency and subjective theories address this need in the sense that statisticians chose to use them as the philosophical foundations of statistics. Needless to say, I believe that the laws (L1)–(L6) address this need much better.

One of my main claims is that probability is a science in the sense that it can satisfy need (i), that is, it can offer reliable predictions, and that users of probability expect reliable predictions from any theory of probability and statistics. This does not mean that (ii)–(viii) should be ignored or that the science of probability cannot satisfy needs listed in (ii)–(viii).

15.8 The Three Aspects of Probability The terribly confused state of the foundations of probability and statistics may be at least partly attributed to the lack of clear recognition that probability has three aspects: mathematical, scientific and philosophical. The mathematics of probability is mostly uncontroversial, in the sense that almost all (but not all) probabilists and statisticians are happy with Kolmogorov’s axioms. However, these axioms are sometimes incorrectly classified as a scientific or philosophical theory (see Sec. 11.14). There is no question that von Mises and de Finetti intended their theories to be foundations of a branch of science — probability and

page 347

April 29, 2016

9:6

348

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch15

Resonance: From Probability to Epistemology and Back

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

statistics. This does not logically imply that these theories are scientific theories. I believe that they are, in the sense that we can express both theories as (families of) falsifiable statements. This is not the standard practice in the field. Instead of presenting various theories of probability as falsifiable, and hence scientific, statements, it is a common practice to state them as axiomatic systems or to use philosophical arguments. Popper stressed falsifiability in his book [Popper (1968)] but, alas, did not create a clear theory that could gain popularity in the scientific community.

15.9 Is Probability a Science? Intellectual activity branched into several major areas including science, mathematics, philosophy and religion. This list is not meant to be exhaustive. For example, one could argue that art belongs to this list and should be a separate entry. The above classification is based on the standards of validation or verification widely accepted in the field. A simplified description of various forms of validation is the following. Religion considers holy books and related texts as the ultimate source of the truth. Science is based on validation of its claims via successful predictions. Mathematics is based on rigid logical deductions from basic axioms. Philosophy is hardest to describe. The interesting part of philosophy is based on ordinary logic. Its theories can be evaluated on the basis of perceived significance, depth and novelty. However, the quality standards for philosophical theories are a legitimate subject of philosophical discussions so the issue is somewhat circular and clouded. Different validation standards for different intellectual activities have an important consequence — a perfectly reasonable and respectable theory in one of these areas may become nonsensical when it is transported to another area in its original form. The clash between religion and science is often the result of using validation methods or claims from one of these fields in the context of the other. The theological claim that there is one God and there is also the Holy Trinity makes no sense in the mathematical context — one is not equal to three. The scientific description of a human being as a bipedal mammal is totally irrelevant in the theological context because it completely misses some important questions such as the one about the meaning of human life. The clash between religion and science is well known. I think that it is even more instructive to consider the clash between mathematics and

page 348

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch15

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

On Ideologies

349

science. The mathematical method of validation of statements based on the absolutely rigid logic is unusable in the scientific context. If we applied this standard to physics, we would have to turn the clock back by 100 years and all of science, technology and civilization would collapse in a day or less. Similarly, introduction of scientific (non-rigorous) reasoning into mathematics would kill mathematics as we know it. Von Mises and de Finetti created theories that are significant, reasonable and respectable as purely philosophical structures. They teach us what one can and what one cannot prove starting from some assumptions. The transplantation of these theories into the scientific context transformed them into laughable fantasies. One could argue that probability should be considered an intellectual activity fundamentally different from science, mathematics, philosophy and religion. The reason for separating probability from mathematics, science and philosophy is that the philosophical research showed that the validation rules for probability are different from the rules used in any of the other fields on the list. My law (L6), based on Popper’s idea, is the validation rule for probabilistic statements. Probability is not mathematics. Of course, there is a huge area of mathematics called “probability” but the real intellectual challenge in the area of probability is concerned with applications of probability in real life. One could argue that probability is not science because it does not make deterministic predictions. The theories of von Mises and de Finetti tried to turn probability into mainstream science by proposing deterministic predictions related to collectives and consistency and based on the mathematics of probability. The failure of both theories shows that trying to turn probability into a deterministic science is like trying to put a square peg into a round hole.

15.10 Are Probability and Logic Experimental Sciences? “I said pig,” replied Alice, “and I wish you wouldn’t keep appearing and vanishing so suddenly: you make one quite giddy.” “All right,” said the Cat; and this time it vanished quite slowly, beginning with the end of the tail, and ending with the grin, which remained some time after the rest of it had gone. “Well! I’ve often seen a cat without a grin,” thought Alice; “but a grin without a cat! It’s the most curious thing I ever saw in all my life!” (Alice’s Adventures in Wonderland, Lewis Carroll)

page 349

April 29, 2016

9:6

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

350

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch15

Resonance: From Probability to Epistemology and Back

Probability theory is like the Cat. It is what remains when all substantive contents is removed from science. Traditionally, this role was ascribed to logic. Probability is similar to logic in the sense that it is distilled science — no concrete facts remain. It was noticed by a number of people that logic can be considered to be a subfield of probability concerned exclusively with propositions whose probability is 0 or 1. While it is hard to object to this view on purely formal grounds, I do not find it very illuminating because it is like calling chemistry a subfield of physics. In practice chemistry and physics are sufficiently different that the reductionist viewpoint is no more than a superficial observation, and the same applies to probability and logic. I believe that substantial and interesting similarities and differences between logic and probability can be found by analyzing their amenability to experimental falsification. Logic was never considered an experimental science. It seemed obvious that the rules governing logic were to be found by pure intellectual analysis. There were also substantial methodological obstacles to experimental testing of logic. First, whatever experimental test of logic one could design, the analysis of the results would necessarily involve logic because logic is indispensable to our thinking. So, the idea of experimental testing of logic seemed to be circular. On the top of that, even the most complicated logical propositions considered in the past were rather simple. One could relatively easily manipulate those propositions in one’s mind. Doing actual experiments representing propositions seemed to be superfluous. Falsifying logic using experimental data was hard to imagine because we process the real data in the same way that we process data in mental experiments. Modern computers seem to have changed the situation. Computer programs are logical functions and they can obviously fail although we believe that all such failures that surfaced so far are due to human and hardware errors and they do not represent problems with the logical theory itself. The imagination of modern intellectuals was captured by the imperfection of logic, from the point of view of human desires and expectations, expressed by G¨odel in his theorems. So, the feeling that logic is (or at least it may be) imperfect came from the pure reason and not experiment. Philosophy of probability contains two clear strains, one of which considers probability an experimental science — these are the frequency and propensity theories of probability. The other direction is represented by the logical and subjective philosophies of probability. While I am not

page 350

April 29, 2016

9:6

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch15

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

On Ideologies

351

a supporter of the frequency philosophy of probability, I recognize that at least it was trying to go in the right direction. It realized that probability was an experimental science because it made numerical predictions about results of real life experiments. One cannot effectively make the same experiments in the mind. The attempts of Carnap on the side of the logical interpretation of probability and de Finetti in the subjective direction proved to be complete failures. Carnap’s theory was never appreciated by scientists and subjectivism is popular among some scientists only because most scientists have no clue that de Finetti claimed that probability did not exist. Going back to the reductionist view of logic as a subfield of probability, I think that it does have a value after all. The experimental nature of probability is an indication of the experimental nature of logic. While I do not see immediate practical advantages of this point of view, I believe that recognizing a (partly) experimental nature of logic is important.

page 351

April 29, 2016

9:7

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch16

Chapter 16

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Paradoxes, Wagers and Rules

There is a number of well known “paradoxes” in probability. They serve as testing grounds for philosophical ideas and shed some light on practical applications of probability.

16.1 St. Petersburg Paradox The following description of the St. Petersburg paradox comes from a Wikipedia article [Wikipedia (2014a)]: A casino offers a game of chance for a single player in which a fair coin is tossed at each stage. The pot starts at 2 dollars and is doubled every time a head appears. The first time a tail appears, the game ends and the player wins whatever is in the pot. Thus the player wins 2 dollars if a tail appears on the first toss, 4 dollars if a head appears on the first toss and a tail on the second, 8 dollars if a head appears on the first two tosses and a tail on the third, 16 dollars if a head appears on the first three tosses and a tail on the fourth, and so on. In short, the player wins 2k dollars, where k equals number of tosses. What would be a fair price to pay the casino for entering the game? [...] Assuming the game can continue as long as the coin toss results in heads and in particular that the casino has unlimited resources, this sum grows without bound and so the expected win for repeated play is an infinite amount of money. Considering nothing but the expected value of the net change in one’s monetary wealth, one should therefore play the game at any price if offered the opportunity. Yet, in published descriptions of the game, many people expressed disbelief in the result. [...] The paradox is the discrepancy between what people seem willing to pay to enter the game and the infinite expected value.

353

page 353

April 29, 2016

9:7

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

354

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch16

Resonance: From Probability to Epistemology and Back

The St. Petersburg paradox shows a common weakness of the theories of von Mises and de Finetti — neither theory can detect a paradox. Note that I am not saying that the theories cannot solve the paradox — they cannot even detect it. In the frequency theory, one has to repeat the game many times until the long run average payoff is more or less equal to the expected payoff. No matter what (fixed) fee you pay per each game, the Law of Large Numbers says that you will make a lot of money in the long run. This is empirically verifiable in principle (although not in practice). So there is no paradox according to the von Mises theory. De Finetti’s theory asks whether there will be a Dutch book created against the decision maker. Or whether axioms for rational decision making will be violated. None of these will occur, no matter what price you pay for the ticket, if you play the game only a finite number of times (it is hard to interpret an infinite sequence of games in practical terms). Once again, this is empirically verifiable so there is no paradox according to de Finetti. The fact that many people consider the game a paradox shows that the two philosophical theories together with their easy explanations of the paradox (in the form of the denial of its existence) are universally ignored. Consider the following variant of St. Petersburg game. In the new game, the payoffs are 2 dollars with probability 1/2, 4 dollars with probability 1/4, 8 dollars with probability 1/8, 100 × 21,000,000,000,000,000,000,000,000 dollars with probability 2−1,000,000,000,000,000,000,000,000 , and with the remaining probability, the reward is 0. The expected value of the reward is 2 × 1/2 + 4 × 1/4 + 8 × 1/8 + 100 × 21,000,000,000,000,000,000,000,000 × 2−1,000,000,000,000,000,000,000,000 = 1 + 1 + 1 + 100 = 103. In my personal opinion, the “true” value of the payoff is 2 × 1/2 + 4 × 1/4 + 8 × 1/8 = 3. In other words, I completely ignore the enormous payoff that has extremely small probability. My guess is that most people would feel the same.

page 354

April 29, 2016

9:7

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch16

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Paradoxes, Wagers and Rules

355

My justification is that the probabilities in the range from 1/2 to 1/8 are common in everyday life. It makes sense to maximize the expected payoff corresponding to such probabilities because of the Law of Large Numbers. Maximizing the expected gain in many decision situations with moderate payoffs and moderate probabilities results in the best possible long term trend. On the other hand, the probability of the enormous payoff in the above game is so small that it should be ignored. Even if we add probabilities of all events relevant to the life of an individual which have similarly small magnitude, we will obtain a very small number. It is safe to assume that none of these events will ever happen. I do not feel that my version of the game presents much of a practical or philosophical challenge. The difference between the original St. Petersburg paradox and my version is that the original paradox involves intermediate rewards and probabilities. Intermediate probabilities are too small to hope that the Law of Large Numbers will translate them into a clear trend in the life of an individual. But intermediate probabilities are too big to be negligible. Intermediate probabilities and rewards do occur in real life. Buying a house or choosing a college can result in substantial gains or losses. The Law of Large Numbers does not apply in such cases to decisions made by a single individual because only a small number of decisions with similarly large rewards occur in a human lifetime. On the other hand, the probabilities involved in these decision problems are too large to be neglected. A possible philosophical choice for an individual is to identify himself with a group of people, for example, all people in the same age group. This allows one to invoke the Law of Large Numbers but the voluntary identification of one’s welfare with that of a group is far from the universal choice.

16.2 Pascal’s Wager According to an article [Wikipedia (2014c)], Pascal’s Wager is an argument in apologetic philosophy devised by [...] Pascal [...]. It posits that humans all bet with their lives either that God exists or not. Given the possibility that God actually does exist and assuming an infinite gain or loss associated with belief or unbelief in said God (as represented by an eternity in heaven or hell), a rational person should live as though God exists and seek to believe in God. If God does not actually exist, such a person will have only a finite loss (some pleasures, luxury, etc.).

page 355

April 29, 2016

9:7

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

356

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch16

Resonance: From Probability to Epistemology and Back

Pascal’s argument does not say anything about which God should be worshiped. Or to put it differently, the same argument can be used in support of worship of any given God. This counterargument was already known to Pascal and was discussed by him in the most unconvincing (to me) manner (see [Wikipedia (2014c)] for more details). As far as I am concerned, this counterargument completely destroys Pascal’s wager. But the wager provides inspiration for a number of observations presented below. I will discuss a few issues that were either not mentioned or not analyzed in detail in [Wikipedia (2014c)].

16.2.1 Scientific aspects of Pascal’s wager 16.2.1.1 Two kinds of infinity We normally think about infinity as a very large number but when someone calls 0 (zero) “infinitely small,” this is likely to be treated as a figure of speech with little scientific content. Zero and infinity play different roles in mathematics. The theory of various types of infinities is much more developed than the theory of infinitely small quantities (although nonstandard analysis is a reasonably successful attempt in the latter direction). In the context of Pascal’s wager, zero and infinity play symmetric roles. According to the standard decision theory, we should choose that decision among all that are available which maximizes the product of the probability and the utility of the reward (utility can be negative). In other words, it is often postulated that the rational decision maker should maximize the expected utility. This formal approach agrees with Pascal’s argument, as far as I can tell. Since there is no single definition of infinity, we have to choose one. I think that the following pragmatic approach is consistent with Pascal’s reasoning. “Infinity” (denoted ∞) is a quantity, such that for any number p strictly between zero and infinity, p × ∞ = ∞. By symmetry, zero (denoted 0) is a quantity, such that for any number u strictly between zero and infinity, 0 × u = 0. The definitions leave open the question of the value of the product 0 × ∞.

page 356

April 29, 2016

9:7

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch16

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Paradoxes, Wagers and Rules

357

In some fields of mathematics, zero times infinity is zero but it is easy to see that in some practical situations, zero times infinity should be interpreted as infinity, and in some other situations it should be considered a number strictly between zero and infinity. Pascal seems to assume that the reward for worshiping God is infinite and the probability of God’s existence is strictly positive. I do not see any obvious reason why only one of these quantities should be considered infinite (infinitely large or infinitely small). In Sec. 16.3, I discuss a proposal by Dennis Lindley that we should never assign zero probability to any event, except possibly logical tautologies or mathematical theorems. By symmetry, one could postulate that no reward should have infinite utility. I am quite skeptical about Lindley’s proposal but I would insist that we either accept existence of both infinitely small probabilities and infinitely large utilities, or none of them. The idea that utility function is bounded seems to have originated with Bernoulli. A bounded utility function easily resolves the St. Petersburg paradox. Of course, this does not prove that the utility function is or may be bounded — there is no way we can test this assumption in any remotely realistic way. I think that many people feel that it is rational to assign a non-zero probability to God’s existence. This vague feeling can be justified in the following way. Assuming that the probability that God exists is zero is equivalent to adding the answer to the assumptions of the wager. Doing this would demolish Pascal’s argument but would not have any philosophical value. If we accept this argument, we see that the assumption that eternal life has infinite utility is also equivalent to inserting the answer into the assumptions of the wager. As I pointed out, infinitely small and infinitely large quantities play symmetric roles so Pascal effectively assumed what he wanted to prove.

16.2.1.2

Minor sins

Let us assume that the utility of eternal life (or some other reward for worshiping God) is infinite. What are then the decision theoretic consequences for minor transgressions of God’s commandments? Suppose that any sin can decrease the probability of eternal life. If the decrease in probability is non-zero then the loss of expected utility is infinite. This seems to be a very harsh punishment for even the slightest disobedience.

page 357

April 29, 2016

9:7

358

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch16

Resonance: From Probability to Epistemology and Back

Another possibility is that minor sins do not decrease the probability of attaining eternal life. This seems to be a wrong message to send to the believers. All sins could be punished in purgatory but I guess that the purgatory has a bounded (negative) utility. Hence it is a rational option to commit minor sins, assuming that they do not decrease the probability of eternal life.

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

16.2.1.3 On the utility of eternal life in hell Since eternal life in heaven is given (positive) infinite utility, it is only natural to assume that eternal life in hell has negative infinite utility. I do not see any reason to assume that the probability of going to hell is zero for a typical believer. Hence, the expected utility for a typical believer is equal to positive infinity plus negative infinity. This is an undefined quantity in mathematics. Pascal seems to be saying that the probability p1 of going to heaven for a believer is strictly greater than the probability p2 of going to heaven for a non-believer. Pascal’s argument translates into p2 × ∞ + (1 − p2 ) × (−∞) < p1 × ∞ + (1 − p1 ) × (−∞). This formula is correct if we replace ∞ with any number that is strictly between zero and infinity. Paradoxically, I would find Pascal’s argument more convincing if he asserted that eternal lives in heaven and in hell had utilities of very large but finite magnitudes.

16.2.1.4

Exponential discounting

A standard assumption made in many economic models is that delayed rewards are subject to exponential discounting. This means that for any person and reward with the current value r, there exists a number c < 1 such that if the reward is delivered after k units of time then the current utility of the delayed reward is rck . Pascal suggested in his discussion of the wager that the eternal life may be considered a sequence of lives, one after another. If we accept exponential discounting with c < 1, represent the eternal life as an infinite sequence of lives, and assume that every single life in the sequence has the same finite utility r (or that the utility of any life in the sequence is bounded above by a common constant r) then the total utility of eternal life is finite, because
∞ the sum of rck from k = 1 to infinity is finite (in symbols, k=1 rck < ∞).

page 358

April 29, 2016

9:7

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch16

Paradoxes, Wagers and Rules

359

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

16.2.2 A sociological analysis of Pascal’s wager Most people do not claim and do not feel that they had direct contact with God or observed any events or phenomena that can be unambiguously interpreted as God’s communication or intervention. What most of us know is that God comes exclusively from other people, in different forms. Hence the classical pure decision theoretic analysis of Pascal’s wager, outside the context of human society, seems to be missing an important, perhaps primary, reason why many people reject the wager (or religion). To start the sociological analysis, let us consider claims made by people about events or phenomena which have probabilities strictly between 0 and 1 and utilities strictly between 0 and infinity. Adding a human factor to the analysis of such a situation is easy. The decision maker has to build a mental model of the person who is the information source, including possible human errors, intentions, etc. This model and the corresponding probabilities have to be combined with the main claim to construct a single model involving the claim and the informant. Then the decision maker can proceed (if he wishes to do so) along the lines of the standard decision theory and take an action which maximizes the expected utility. Needless to say, for practical reasons, the whole procedure has to be based on subjective probabilities because it is impossible to determine the objective probability of, for example, someone’s good intentions, under normal circumstances. The philosophical challenge starts when someone makes a claim that involves quantities that are infinite. For definiteness, let us focus on the claim that involves (positive) infinite utility, because this is the case of Pascal’s wager. One possible way to deal with the problem is to assume that all utilities are finite and, therefore, the philosophical problem can be reduced to the case that has already been discussed. I do not particularly like this solution because it seems to be missing the point. What if a person makes a claim involving a reward that has finite utility but the utility is extremely large? If the decision maker assigns a non-zero probability to the corresponding event then the expected utility may be large enough to induce him to take the same action as in the case when the utility of the reward is truly infinite. Generally speaking, people prefer rewards with large utilities to those with smaller utilities (this is, more or less, the definition of utility). People know that other people like rewards with high utilities so if someone wants to induce another person to take an action that is not particularly attractive, he may offer a reward that has a high utility. Most of the time,

page 359

April 29, 2016

9:7

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

360

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch16

Resonance: From Probability to Epistemology and Back

the two people come to an agreement, an appropriate action is taken by one of the people and the reward is provided by the other person. Since such simple social contracts are very popular, unscrupulous people learned to exploit the system. Criminals of various sorts offer rewards that are not intended to be delivered. The people who are the targets of such fraudulent offers are not devoid of common sense, so they developed resistance to the insincere offers. This started a form of a bidding war, in which criminals increase the utility (value) of the rewards, and the targets of their offers decrease the subjective probability that the offer is genuine. A good example of this kind of activity are certain forms of the so-called “Nigerian scam” delivered by “spam” e-mail. Assuming that people use the standard decision theoretic method of choosing the optimal action by maximizing the expected utility, it is natural to push the scheme to the limit and offer a reward with infinite value. I have not heard about any criminals offering an infinite amount of money but some other rewards of (potentially) infinite value are occasionally offered. These include salvation offered by some “prophets.” People who are the targets of fraudulent offers with an infinitely valuable reward can defend themselves either by abandoning the usual expected utility scheme or by assigning zero subjective probability to the reward (and declaring that zero times infinity is zero). This defensive posture is necessary or otherwise a group of unscrupulous people could easily accumulate most of the wealth of the society. I used criminals as an example of people who offer very high rewards with little if any intention of actually delivering them. This practice is not limited to criminals. It is a widespread belief that politicians routinely make election promises with no intention of fulfilling them. Various other agents, from companies to environmental groups, have been accused of similarly disingenuous claims. Finally, we return to Pascal’s wager. I have no doubt that the claims of God’s existence and eternal life are made in good faith by most of the believers. Yet the good intentions are not a proof of the truth of the claims. It is perfectly rational to reject the conclusion of Pascal’s wager for the same reasons why other offers of rewards with infinite value can be rationally rejected. The core of my argument in this section was based on an idea similar to that in “Goodhart’s law” saying that “When a measure becomes a target, it ceases to be a good measure” (see [Wikipedia (2014g)]) and “Campbell’s

page 360

April 29, 2016

9:7

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch16

Paradoxes, Wagers and Rules

361

law” saying that “The more any quantitative social indicator (or even some qualitative indicator) is used for social decision making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor” (see [Wikipedia (2014h)]).

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

16.3 Cromwell’s Rule The following short introduction to “Cromwell’s rule” is taken from [Wikipedia (2014d)]. Cromwell’s rule, named by statistician Dennis Lindley, states that the use of prior probabilities of 0 or 1 should be avoided, except when applied to statements that are logically true or false. For instance, Lindley would allow us to say that P (2+2 = 4) = 1, where P represents the probability. In other words, arithmetically, the number 2 added to the number 2 will certainly equal 4. [...] As Lindley puts it, assigning a probability should “leave a little probability for the moon being made of green cheese; it can be as small as 1 in a million, but have it there since otherwise an army of astronauts returning with samples of the said cheese will leave you unmoved.”

16.3.1 Cromwell’s rule: practical implementation I have two sets of comments on Cromwell’s rule. First, I believe that trying to apply it in real life may be highly problematic. In other words, the rule may either lead to undesirable results or it may be a meaningless directive. These concerns will be discussed in this section. The next section is devoted to philosophical problems related to Cromwell’s rule. I will try to determine what Lindley was trying to say because the more I think about Cromwell’s rule, the less clear it appears to me. (i) The reader must have received at least some spam messages saying that an employee of a bank in a remote country wanted to transfer an enormous sum of money N (presumably illegally at his disposal) to a bank in another foreign country. He wanted to share a large portion of the money, say N/3, with the recipient of the message, as a reward for helping with this (presumably illegal) transfer. The whole scheme would require the recipient of the message to send $1,000 to the sender of the message to cover the fees needed to start the process.

page 361

April 29, 2016

9:7

362

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch16

Resonance: From Probability to Epistemology and Back

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Let us set the ethical side of the proposal aside and talk about the economic side only. Let p1 denote my personal probability that the offer made in the spam message is genuine and all the steps of the scheme will work as outlined in the e-mail. Let us ignore the utility function for a moment. In other words, let us assume that the utility of money is equal to its nominal value. Then I should send the $1,000 to the spammer if and only if p1 (N/3) > (1 − p1 )1, 000.

(16.1)

Now suppose that I follow Cromwell’s rule and make p1 greater than 0. To succeed, the spammer has to type a large number of zeros at the end of N to make N sufficiently large so that (16.1) becomes true. In other words, the spammer will induce me to send him $1,000 simply by typing a very long sequence of zeros. Needless to say, Cromwell’s rule does not look very appealing in this context. (ii) There are (at least) two possible objections to the argument outlined in (i). The first one is that the larger number N is stated in the e-mail, the less probable it is that the offer is genuine. Hence, for every N , the value of p1 should be strictly positive but so small that (16.1) does not hold. This is a perfectly good solution to the practical problem but it completely nullifies Cromwell’s rule. In this approach, the value of p1 is chosen to be so small as to have the same effect as “p1 = 0,” which seems to completely contradict the spirit of Cromwell’s rule. What is the point of using a non-zero value of p1 if its value is chosen intentionally to be so small that it is functionally equivalent to “p1 = 0”? (iii) Another objection to the argument in (i) is that it ignores the utility function. If we assume that the utility function is bounded then the spammer will not be able to make (16.1) true by typing a large number of zeros. I do not like this solution of the problem for the following reasons. (1) Of all concepts that appear in the philosophy of probability, utility seems to be the most subjective. How can we dictate to anyone that his/her utility function must be bounded? What if a person insists that his utility function is not bounded? Is he automatically irrational? (2) I do not think that there is good empirical evidence that the utility function is bounded. I believe that many (most?) people who have 1,000 dollars would like to have one million dollars. Many (most?) people who

page 362

April 29, 2016

9:7

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch16

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Paradoxes, Wagers and Rules

363

have one million dollars would like to have one billion dollars. Many (most?) people who have one billion dollars would like to have 100 billion dollars. We have no empirical data beyond this point. (3) There are a number of rewards that are non-monetary and seem to have very high utility to many people. The list of these includes life, love, honor, salvation and eternal life. The question is whether the ratio of the utility of any of these rewards to the utility of $1,000 has to be bounded in practical sense. Perhaps it should be but for some people some of the listed rewards are infinitely more valuable than $1,000. (4) Recall Pascal’s wager from Sec. 16.2: “Given the possibility that God actually does exist and assuming the infinite gain or loss associated with belief in God or with unbelief, a rational person should live as though God exists and seek to believe in God. If God does not actually exist, such a person will have only a finite loss (some pleasures, luxury, etc.).” Pascal’s wager is an example of a philosophical problem created by the combination of unbounded (infinite) utility and Cromwell’s rule. The assumption of infinite utility associated with the belief in God may be unpalatable to atheists. But it may be more acceptable to say that the utility associated with the belief in God is unknown and possibly very very large. Cromwell’s rule might lead to the conclusion that a rational person should believe in God given this more modest assumption. What really saves an atheist is existence of many religions — Pascal’s wager applies to many of them so its conclusion leads to a logical inconsistency. (iv) Pascal’s wager presents another problem for Cromwell’s rule. The total number of different religions that ever existed is finite but the total number of religions that can be potentially proposed seems to be infinite. Moreover, I would guess that there exist infinitely many mutually incompatible religions. Hence, the probabilities that they are true must add up to 1 or less. Some of the religions will have to have very small probabilities and I wonder how we can decide which religions should get very small probabilities. There is a theoretical possibility that there are uncountably many mutually incompatible religions. If this is the case, some religions will have to have probability 0. Once again, I wonder how one could select those religions that would have non-zero probability from this collection. Digression: Some readers may wonder how it is possible that there exist uncountably many mutually incompatible religions. It is very simple

page 363

April 29, 2016

9:7

364

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch16

Resonance: From Probability to Epistemology and Back

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

to construct such religions. For every real number T > 0, let T th religion claim that the universe was created exactly T years before January 1, 2000. I conclude that even though Cromwell’s rule was invented as a good practical guideline, its value is questionable precisely in those cases when some people tend to assign zero probability to some events for practical reasons. Lindley’s examples are too artificial to offer a viable practical guide in such situations.

16.3.2 Cromwell’s rule: philosophical problems I will try to analyze in detail what Lindley was trying to say because the more I think about Cromwell’s rule, the less clear it appears to me. I will give labels to Lindley’s suggestions: (*) P (2 + 2 = 4) = 1. (**) “Leave a little probability for the moon being made of green cheese.” (a) The first interpretation of Cromwell’s rule that I am going to discuss is almost certainly (with probability 1?) not what Lindley had in mind. One could say that one can and should assign probability 1 to claims about facts that are directly accessible to our senses. Hence, the event in (**) should not have probability 1 because verifying the composition of the Moon is a realistic task only for the most affluent states. On the top of that, for an individual to be certain about the composition of the Moon, he or she must trust the media controlled by the state, which is not what some people are willing to do (recall that there is a conspiracy theory saying that people never landed on the Moon). Consider the following fact directly accessible to my senses: “I do not have seven hands.” My guess is that it does not qualify for probability 1 according to Lindley’s standards because it is not a logical truth. However, if I consider it possible that my mental condition is such that I cannot verify with certainty the number of my limbs, can I trust my mental condition to be sufficiently solid to verify the laws of arithmetic, even those at the level of “2 + 2 = 4”? It is unfortunate that Lindley chose examples (*) and (**) at two drastically different levels of accessibility to our senses. This difference between (*) and (**) makes it hard to analyze his intentions. (b) I find it somewhat surprising that Lindley, a person who proved mathematical theorems as a part of his job, would assert that we can assign

page 364

April 29, 2016

9:7

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch16

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Paradoxes, Wagers and Rules

365

probability 1 to logical statements. Mathematical journals contain a number of false logical statements, a fact that many mathematicians consider embarrassing. Personally, I would assign probability 1 to “2 + 2 = 4.” I would definitely not assign probability 1 to a logical claim “randomly” chosen from a mathematical journal. A somewhat different presentation of the same philosophical objection is this. Let M = 101,000,000,000,000 . What is the probability that the M th digit of π is even? At this point, it appears (see [Wikipedia (2014e)]) that we might not know the parity of the M th digit of π in our lifetimes. Hence, my personal probability that the M th digit of π is even is 1/2 (see this article [Wikipedia (2014f)] on normal numbers). Saying that either P (the M th digit of π is even) = 1 or P (the M th digit of π is odd) = 1 has no practical meaning and may appeal only to the most abstract intellects. (c) Logical truths do not necessarily represent truths about our universe. The Pythagorean Theorem applies to triangles in the plane but it does not apply to triangles on the sphere (for example, to triangles on the surface of the Earth). So what is the message in (*)? Logical truths are elements of formal logical systems. They are true because we constructed the logical systems so that these claims are true within those systems. In this context, (*) is meaningless. (d) One may interpret (*) as saying that the logical statement “2 + 2 = 4” applies to the real world. No, it doesn’t. If you put two lions and two zebras in a cage then soon you will have two animals in the cage. Hence, 2 + 2 = 2 in this case. You may say, and I would agree, that the claim “2 + 2 = 4” does not apply to the lions and zebras in the cage. Hence, if (*) refers to real world applications of the logical statement “2 + 2 = 4” then (*) really means P (2 + 2 = 4 in those cases when this claim is properly applied) = 1. Needless to say, this probabilistic statement has no significant content. It is a form of tautology. (e) Suppose that the event described in (**) actually occurs. In other words, let us suppose that the Moon is actually made of green cheese. The consequences of such a discovery would dwarf the revolution in physics started by Einstein. Quantum mechanics and relativity theory would look like child’s play in comparison with the fundamental changes needed in astronomy, physics, chemistry and biology to accommodate the fact that

page 365

April 29, 2016

9:7

366

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch16

Resonance: From Probability to Epistemology and Back

the Moon is made of green cheese. If our current science can be that wrong with positive probability, why is it that our current logic cannot be wrong with positive probability? Didn’t G¨ odel’s theorems demonstrate that some of our beliefs about how solid logic is were unfounded?

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

16.4 Principal Principle The Principal Principle was proposed in [Lewis (1980)]. I quote a version stated in [Lewis (1994), p. 475], Chance is objective single-case probability: for instance, the 50% probability that a certain particular tritium atom will decay sometime in the next 12.26 years. [...] if a rational believer knew that the chance of decay was 50%, then almost no matter what else he might or might not know as well, he would believe to degree 50% that decay was going to occur.

Consider the following principles. “If elephants are larger than butterflies then a rational believer should think that elephants are larger than butterflies.” “If 5 + 7 = 12 then a rational believer should think that 5 + 7 = 12.” All three principles — the Principal Principle and my two “principles” — sound equally ridiculous to me. I am not trying to ridicule philosophy; after all, this book proves that I am an aspiring philosopher. I am not trying to ridicule attempts to analyze the foundations of our knowledge; both mathematics and physics benefited greatly from the analysis of their own foundations. I am trying to ridicule the pretentious name “Principal Principle” and amount of attention given to a statement that seems to be an obvious element of any mainstream epistemology. Actually, the Principal Principle is worse than obvious — it is false. It is widely believed and perhaps even supported by evidence that selfconfidence helps achieve practical goals. So, it might be beneficial to a rational believer to choose subjective probabilities different from the objective probabilities. The Principal Principle can be interpreted as saying that betting strategies should be consistent with objectively known probabilities. While rational in this sense, it is hardly a discovery in view of the usual interpretation of expected value as the long run average. If the long run justification is not applicable, my law (L6) and its interpretation provide support for implementing objective probabilities in decision making (see Chapter 12).

page 366

April 29, 2016

9:7

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch16

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Paradoxes, Wagers and Rules

367

The real problem with the Principal Principle is that it is a principle. If using objective probabilities in decision making brings benefits to decision makers then the benefits should be described to potential users (this is what this book is about) and then we will not need any sophisticated philosophical principles to tell us that we should adopt objective probabilities as our subjective opinions. If there is no evidence that objective probabilities exist or there is no evidence that they are useful then no amount of philosophical sophistry will make the Principal Principle anything more than pure abstract nonsense.

16.5 A New Prisoner Paradox This section contains an example, partly meant to illustrate the two decision making philosophies discussed in Sec. 12.2. Imagine that you live in a medieval kingdom. Its ruler, King Seyab, is known for his love of mathematics and philosophy, and for cruelty. As a very young king, 40 years ago, he ordered a group of wise men to take an urn and fill it with 1,000 white and black balls. The color of each ball was chosen by a coin flip, independently of other balls. There is no reason to doubt wise men’s honesty or accuracy in fulfilling king’s order. The king examined the contents of the urn and filled another urn with 510 black and 490 white balls. The contents of the two urns is top secret and the subjects of King Seyab never discuss it. The laws of the kingdom are very harsh, many ordinary crimes are punished by death, and the courts are encouraged to met out the capital punishment. On average, one person is sentenced to death each day. The people sentenced to death cannot appeal for mercy but are given a chance to survive by the following strange decree of the monarch. The prisoner on the death row can sample 999 balls from the original urn. He is told that the second urn contains 1,000 balls, 490 of which are white. Then he can either take the last ball from the first urn or take a single random ball from the second urn. If the ball is white, the prisoner’s life is spared and, moreover the prisoner cannot be sentenced to death on another occasion. No matter what the result of the sample is, all balls are replaced into the urns from which they came, so that the next prisoner will sample balls from urns with the same composition. Now imagine that you have been falsely accused of squaring a circle and sentenced to death. You have sampled 999 balls from the first urn. The sample contains 479 white balls. You have been told that the second urn

page 367

April 29, 2016

9:7

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

368

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch16

Resonance: From Probability to Epistemology and Back

contains 490 white and 510 black balls. Will you take the last ball from the first urn or sample a single ball from the second one? In view of how the balls were originally chosen for the first urn, the probability that the last ball in the first urn is white is 0.50. The probability of sampling a white ball from the second urn is only 0.49. It seems that taking the last ball from the first urn is the optimal decision. However, you know that over the last 40 years, the survival rate for those who took the last ball from the first urn was either 48% or 47.9%. The survival rate for those who sampled from the second urn was about 49%. This frequency based argument suggests that the optimal decision is to sample a ball from the second urn. What would your decision be?

16.5.1 Analysis of the new prisoner paradox My analysis of the new prisoner paradox will be based on the same ideas as the analysis of the modified St. Petersburg paradox presented in Sec. 16.1. I will repeat now some general remarks from Sec. 12.1. People seem to be comfortable making a decision in two types of situations. First, if the event of interest has very high probability then it is natural to assume that it will happen with certainty. This is the essence of (L6). The other type of decision problem when most people seem to be comfortable with making a choice is when very similar decision situations arise multiple times. Then it is natural to maximize the expected value of the gain or the expected value of the utility every time a similar decision problem has to be tackled. The new prisoner paradox is a challenge to our probabilistic intuition because it does not fit easily into any of the above templates. The probabilities of events that are directly relevant to the prisoner are all very close to 1/2, so no useful probabilistic prediction, in the sense of (L6), can be made. The long run frequency interpretation of probability cannot resolve the prisoner’s dilemma in a direct way because there will be no long run of similar decision problems in his life. If someone faces a sequence of independent situations when he may lose life with probability close to 1/2 then the number of lucky survivals has the expectation close to 1. Hence, the person will not be able to take advantage of a long run of decisions, no matter what that hypothetical advantage might be. There are several long run sequences that may be of interest to the prisoner. I have already mentioned two of them — the sequence of prisoners who sampled the last ball from the first urn and the sequence of prisoners

page 368

April 29, 2016

9:7

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch16

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Paradoxes, Wagers and Rules

369

who sampled a ball from the second urn. There is also an imaginary sequence of kingdoms in parallel universes where similar schemes were implemented. All these sequences have one thing in common — the people in these sequences may have no relationship to our prisoner so he may be totally indifferent to their fate. In this case, there seems to be no compelling argument in favor of our prisoner identifying himself with any of the sequences. The essence of the paradox is that if he chooses the last ball from the first urn then he implicitly identifies himself with the sequence of prisoners in parallel universes which seems to be “less real” than the sequence of prisoners (in this world) who chose to take a ball from the second urn. The reader might have noticed that I implicitly asserted that it is objectively true that the probability that the last ball in the first urn is white is 50%. One could argue that the fact that the king placed 490 balls in the second urn is informative and, therefore, the probability that the last ball in the first urn is white is not necessarily 50%, because the symmetry is broken. The subjective-objective controversy is irrelevant here. If the reader does not believe that it is objectively true that the probability in question is equal to 50%, he should consider a prisoner whose subjective opinion is that this probability is 50%.

16.6 Ellsberg Paradox Here are some excerpts from a Wikipedia article [Wikipedia (2014b)] on the Ellsberg paradox. The Ellsberg paradox is a paradox in decision theory in which people’s choices violate the postulates of subjective expected utility. [...] Suppose you have an urn containing 30 red balls and 60 other balls that are either black or yellow. You don’t know how many black or how many yellow balls there are, but that the total number of black balls plus the total number of yellow equals 60. The balls are well mixed so that each individual ball is as likely to be drawn as any other. You are now given a choice between two gambles: Gamble A: You receive $100 if you draw a red ball. Gamble B: You receive $100 if you draw a black ball. Also you are given the choice between these two gambles (about a different draw from the same urn): Gamble C: You receive $100 if you draw a red or yellow ball. Gamble D: You receive $100 if you draw a black or yellow ball.

page 369

April 29, 2016

9:7

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

370

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch16

Resonance: From Probability to Epistemology and Back [...] Utility theory interpretation Utility theory models the choice by assuming that in choosing between these gambles, people assume a probability that the non-red balls are yellow versus black, and then compute the expected utility of the two gambles. Since the prizes are exactly the same, it follows that you will prefer Gamble A to Gamble B if and only if you believe that drawing a red ball is more likely than drawing a black ball (according to expected utility theory). Also, there would be no clear preference between the choices if you thought that a red ball was as likely as a black ball. Similarly it follows that you will prefer Gamble C to Gamble D if, and only if, you believe that drawing a red or yellow ball is more likely than drawing a black or yellow ball. It might seem intuitive that, if drawing a red ball is more likely than drawing a black ball, then drawing a red or yellow ball is also more likely than drawing a black or yellow ball. So, supposing you prefer Gamble A to Gamble B, it follows that you will also prefer Gamble C to Gamble D. And, supposing instead that you prefer Gamble B to Gamble A, it follows that you will also prefer Gamble D to Gamble C. When surveyed, however, most people strictly prefer Gamble A to Gamble B and Gamble D to Gamble C. Therefore, some assumptions of the expected utility theory are violated.

I will offer an explanation of the paradox that seems to be different from those discussed in the Wikipedia article. Assume that the color of each ball in the second urn, black or yellow, was chosen randomly (each color with probability 1/2) and independently of the colors of other balls. (It is enough to assume that this is the subjective belief of the decision maker.) Then Gambles A and B have identical expectations of the reward, $100/3. Gambles C and D also have identical expectations of the reward, $200/3. But Gamble A is “less random” than Gamble B and Gamble D is “less random” than Gamble C. I will try to explain what “less random” might mean. Randomness of a decision situation may involve two different sources, with only one of them potentially repeatable in the time interval relevant to the decision maker. The distinction is best explained by an example. Suppose that a company is looking for a new employee. The employee will have to work on a large number of projects over the next year. The success or failure of each of these projects has two sources of randomness. First, the competence of the employee is a random variable at the time of the hire. Assuming that the company is unlikely to hire a new employee within a

page 370

April 29, 2016

9:7

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch16

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Paradoxes, Wagers and Rules

371

year, this source of randomness does not have a frequency representation. Second, each project presents its own challenges and thus its outcome is random. Now consider two job applicants. Suppose that the company was able to determine the quality of the first applicant accurately and his quality happens to be average. There is another applicant whose qualifications are hard to determine. Suppose that the second applicant may be much better qualified or much worse qualified than the first applicant, with equal probabilities, so that the expected value of his quality is the same as that of the first applicant. To make the example amenable to mathematical analysis, suppose that on every project, the first applicant can generate a profit for the company with mean µ and standard deviation σ. Let us assume that profits derived from different projects are independent. The second applicant may prove to be excellent. In this case, he may generate a profit on every project with the normal distribution with the mean µ + δ and standard deviation σ (with δ > 0). We assume that profits derived from different projects are independent. The second applicant may also prove to be a poor employee and he may generate a profit on a project with the normal distribution with the mean µ−δ and standard deviation σ. Once again, we assume that profits derived from different projects are independent. The profit generated by the second applicant on a single project is a mixture of two normal distributions, with equal weights (that is, equal probabilities, 1/2 each). Next suppose that the utility function of the company is concave and smooth — these are standard assumption. Since the utility function is smooth, it is nearly linear on small intervals. Hence, it is possible that the expected utility gain on a single project is about the same for both applicants. Suppose that the second applicant is slightly preferable for an unrelated reason, say his communication skills, and that adds a small ε > 0 of utility for each project. Then the second applicant is preferable if we consider only a single project. However, if the applicants are expected to work on a large number, say, 100, of unrelated (independent) projects over a year, the calculation is different. The total profit generated by the first applicant over 100 projects will be normal (assuming independence of the projects) with the mean 100µ and variance 100σ 2 . For the second applicant, the total profit will have

page 371

April 29, 2016

9:7

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

372

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch16

Resonance: From Probability to Epistemology and Back

either the mean 100(µ + δ) and variance 100σ 2 or the mean 100(µ − δ) and variance 100σ 2 . Due to concavity of the utility function, the expected utility of hiring the second applicant is lower than that for the first applicant. To see this, let U denote the utility function and recall that U is concave. Then U (µ) > [U (µ + δ) + U (µ − δ)]/2. If σ is sufficiently small then a similar inequality holds not only for the utilities of the expected values but also for the expected utilities for both employees. And the difference may be larger (in favor of the first potential employee) than 100ε. Hence, the first applicant is preferable if he is expected to work on 100 projects. The above example points to a possible explanation of the Ellsberg paradox. Due to Darwinian selection, people might have developed an instinct for classifying sources of randomness as those that lead to frequency manifestation of probability values, and those that do not. The example given above shows that only a part of randomness might manifest itself in frequencies and the optimality of the decision to hire one of the applicants depends on the length of the expected association with the company (one project or 100 projects). I consider it possible that decision makers instinctively assume that only a part of randomness will manifest itself in frequencies. Then standard assumptions about the utility function explain “risk aversion” (that is, preference for what I called the “less random” option above), at least in the case of the Ellsberg paradox. Similar ideas were involved in the arguments presented in Secs. 11.12 and 16.5.

16.7 The Probability of God Stephen Unwin applied the Bayes formula to calculate the probability of God in his book [Unwin (2003)]. I believe that the question of probability of God has much more to do with philosophy of God than philosophy of probability. Hence, personally, I do not find it particularly interesting. But Unwin’s calculation does inspire some thoughts. It is hard to find a common attribute of all Gods of all religions but a good candidate for such an attribute is God’s ability to overrule the laws of nature. For example, God can turn water into wine. The question whether the Bayes theorem applies to God is a little bit more subtle. Some people consider all mathematics (including the Bayes theorem) to be a part of logic. There is no universal agreement on the question of whether God is limited by logic. Another facet of the problem is whether the Bayes theorem is only

page 372

April 29, 2016

9:7

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch16

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Paradoxes, Wagers and Rules

373

a mathematical theorem or whether it is also a law of nature, supported by empirical evidence (in my opinion, it is both). Overall, I do not see why the Bayes theorem should necessarily be applicable to God if the laws of chemistry are not. I think that we can understand applications of probability to natural phenomena better by analyzing problems where probability is applied to the supernatural. There are many objects, systems and phenomena that present the same intellectual challenges as God, except to a smaller degree. Examples include financial markets, human brain and climate. All these systems are complex and our knowledge of them is incomplete. Direct experiments are often hard or expensive or unethical (or all of these). Simplified theories applied to these systems sometimes fail to account for subtle features that manifest themselves in unpredictable ways. Going back to God, failures of some probability estimates applied to natural events make me very dubious about any attempt to calculate the probability of God, if there exists objective probability. Subjective probability obviously exists. The subjective probability of God is exactly what you believe it is, and you are welcome to use the Bayes theorem to organize your subjective probabilities related to God into a consistent belief system — I do not see any harm in doing so. Neither do I see any gain.

page 373

April 29, 2016

9:7

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch17

Chapter 17

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Teaching Probability

I do not have an ambition to reform statistics although I think that statistics might benefit if statisticians abandon the frequency and subjective ideologies and embrace (L1)–(L6). I do have an ambition to reform teaching of probability. I have only one explanation for the remarkable practical successes of statistics and probability in view of the totally confused state of teaching of the foundations of probability — philosophical explanations given to students are so confused that students do not understand almost any of them and they learn the true meaning of probability from examples. I will review the current teaching practices — they illustrate well the disconnection between the frequency and subjective philosophies on one hand and the real science of probability on the other. The current teaching of probability and statistics is unsatisfactory for several reasons. (i) The frequency and subjective philosophical theories are presented in vulgarized versions. It is more accurate to say that they are not presented at all. Instead, only some intuitive ideas related to both theories are mentioned. (ii) Even these distorted philosophical theories are soon forgotten and the sciences of probability and statistics are taught by example. (iii) Implicit explanations of why statistics is effective are false. The unquestionable success of statistics can be explained a little bit by the long run frequencies; it has nothing to do with consistent decision strategies. In the frequency theory, the transition from probability to long run frequency is rather straightforward because it is based on a mathematical theorem, the Law of Large Numbers. Probability textbooks are missing 375

page 375

April 29, 2016

9:7

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

376

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch17

Resonance: From Probability to Epistemology and Back

the real philosophical difficulty — going from sequences of observations to probabilities. A standard approach is to explain that the average of the observations is an unbiased estimator of the mean. From the philosophical point of view, this is already quite a sophisticated claim. The most elementary level of the frequency ideology, the von Mises’ theory of collectives, is completely ignored. And for a good reason, I hasten to add. Except that students end up with no knowledge of what the frequency philosophy of probability is. On the Bayesian side, standard textbooks sweep under the rug some inconvenient claims and questions. If any elements of the Bayesian setup are subjective, can the posterior be fully objective? Is there a way to measure subjectivity? If a textbook is based on an axiomatic system, does it mean that there is no way to verify empirically predictions implicit in the posterior distribution? At the undergraduate college level and at schools, the teaching of probability starts with combinatorial models using coins, dice, playing cards, etc., as real life examples. At the next stage some continuous distributions and models are introduced, such as the exponential distribution and the Poisson process. The models are implicitly based on (L1)–(L6) and are clearly designed to imbue (L1)–(L6) into the minds of students (of course, (L1)– (L6) are not explicitly stated in contemporary textbooks in the form given in this book). Many textbooks and teachers present Kolmogorov’s axioms at this point but this does more harm than good. The elementary and uncontroversial portion of Kolmogorov’s axioms states that probabilities are numbers between 0 and 1, and that probability is additive, in the sense that the probability of the union of two mutually exclusive events is the sum of the probabilities of the events. The only other axiom in Kolmogorov’s system is that probability is countably additive, that is, for any countably infinite family of mutually exclusive events, the probability of their union is the sum of the probabilities of individual events. From the point of view of mathematical research in probability theory, this last axiom is of fundamental importance. From the point of view of undergraduate probability, countable additivity has very limited significance. It can be used, for example, to justify formulas for the probability mass functions of the geometric and Poisson distributions. Kolmogorov’s axioms do not mention independence, suggesting to students that independence does not merit to be included among the most fundamental laws of probability. Kolmogorov’s axioms proved to be a perfect platform for the theoretical

page 376

April 29, 2016

9:7

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch17

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Teaching Probability

377

research in probability but undergraduate students do not have sufficient background to comprehend their significance. In a typical undergraduate probability textbook, the frequency and subjective theories enter the picture in their pristine philosophical attire. They are used to explain what probability “really is.” A teacher who likes the frequency theory may say that the proper understanding of the statement “probability of heads is 1/2” is that if you toss a coin many times, the relative frequency of heads will be close to 1/2. Teachers who like the subjective philosophy may give examples of other nature, such as the probability that your friend will invite you to her party, to show that probability may be given a subjective meaning. In either case, it is clear from the context that the frequency and subjective “definitions” of probability are meant to be only philosophical interpretations and one must not try to implement them in real life. I will illustrate the last point with the following example, resembling textbook problems of combinatorial nature. A class consists of 36 students; 20 of them are women. The professor randomly divides the class into six groups of six students, so that they can collaborate in small groups on a project. What is the probability that every group will contain at least one woman? The frequency theory suggests that the “probability” in the question makes sense only if the professor divides the same class repeatedly very many times. Needless to say, such an assumption is unrealistic, and students have no problem understanding that the frequency interpretation refers to an imaginary sequence of experiments. Hence, students learn to use the frequency interpretation as a mental device that has nothing to do with von Mises’ theory of collectives. As far as I can tell, all “subjectivist” instructors would show students how to calculate the probability that every group will contain a woman using the classical definition of probability. I do not know how many of them would explicitly call the answer “objective” but it is clear to me that students would get the message nevertheless — some probabilities are objective. It seems to me that the only “subjectivity” that students are exposed to is the fact that some probabilities are hard to estimate using simple methods, such as the probability that you will be invited to a birthday party. This has nothing in common with de Finetti’s theory. At the graduate level, the teaching of probability is more sterile. A graduate textbook in probability theory often identifies implicitly the science of probability with the mathematical theory based on Kolmogorov’s axioms. In other words, no distinction seems to be made between mathematical and

page 377

April 29, 2016

9:7

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

378

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch17

Resonance: From Probability to Epistemology and Back

scientific aspects of probability. It is left to students to figure out how one can match mathematical formulas and scientific observations. Students taking a course in frequency statistics can easily understand how the frequency interpretation of probability applies to the significance level in hypothesis testing. It is a mystery to me how one can give a frequency interpretation to one of the most elementary concepts of frequency statistics — unbiased estimator. My guess is that students are supposed to imagine a long sequence of identical statistical problems and accept it as a substitute for a real sequence. A course in Bayesian statistics may start with an axiomatic system for decision making (this is how the author was introduced to the Bayesian statistics). The axioms and the elementary deductions from them are sufficiently boring to make an impression of a solid mathematical theory. The only really important elements of Bayesian statistics, the model and the prior, are then taught by example. The official line seems to be “you are free to have any subjective and consistent set of opinions” but “all reasonable people would agree on exchangeability of deformed coin tosses.” Students (sometimes) waste their time learning the useless axiomatic system and then have to learn the only meaningful part of Bayesian statistics from examples. An alternative way to teach Bayesian statistics is to sweep the philosophical baggage under the rug and to tell the students that Bayesian methods “work” without explaining in a clear way what it means for a statistical theory to “work.”

17.1 Teaching Independence Neither undergraduate nor graduate textbooks try to explain the difference between physical and mathematical independence to students. Typically, at both levels of instruction, the formula P (A and B) = P (A)P (B) is given as the definition of independence. Of course, there is nothing wrong with this definition but my guess is that most students never fully understand the difference between physical and mathematical independence. The physical independence, or lack of relationship, is something that we have to recognize via resonance — intuitively and instantaneously. In cases when the physical independence is not obvious or clear, one has to design an experiment to verify whether independence holds. But there are also simple cases of mathematical independence that have nothing to do with lack of physical relationship. For example, if you roll a die, then the event that the number of

page 378

April 29, 2016

9:7

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch17

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Teaching Probability

379

dots is less than 3 and the event that number of dots is even are independent. This lack of understanding of the difference between the physical and mathematical independence can potentially lead to misinterpretation of scientific data. For example, a scientist may determine that the level of a hormone is (mathematically) independent from the fact that someone has a cancer. This may be misinterpreted as saying that the hormone does not interact with cancer cells. The above problem is related to but somewhat different from the problem of distinguishing between association and causation. A classical example illustrating the difference between association and causation is that there is a positive correlation between the number of storks present in a given season of the year and the number of babies born in the same season. This is an example of association that is not causation.

17.2 Probability and Frequency No matter what ideology the author of a textbook subscribes to, it seems that there can be no harm in introducing students early on to the fact that observed frequencies match theoretical probabilities very well. My own attitude towards presenting this relationship early in the course is deeply ambiguous. On one hand, I cannot imagine a course on probability that would fail to mention the relationship between probability and frequency at the very beginning. This is how I was taught probability, how I teach probability, and how the modern probability theory started, with Chevalier de Mere observing some stable frequencies. On the other hand, I see several compelling philosophical and didactic reasons why the presentation of the relationship between probability and frequency should be relegated to later chapters of textbooks. First, novices have no conceptual framework into which the equality of probability and observed frequencies can be placed. The mathematical framework needed here is that of the Law of Large Numbers. Understanding of the simplest version of the Law of Large Numbers requires the knowledge of the concept of i.i.d. random variables. The simplest proof of the “weak” Law of Large Numbers is based on the so called Chebyshev inequality, which involves concepts of expectation and variance. The presentation of the concepts of random variables, i.i.d. sequences, expectation and variance takes up several chapters of an undergraduate textbook and several months of an undergraduate course.

page 379

April 29, 2016

9:7

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

380

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch17

Resonance: From Probability to Epistemology and Back

If students have no proper background and learn about the approximate equality of probabilities and frequencies, they may develop two false intuitive ideas about sequences of random variables. Students may believe that averages of observations of i.i.d. sequences of random variables must converge to a finite number. This is not the case when the random variables in the sequence do not have finite expectations. Students may also come to the conclusion that stable frequencies are a sure sign of an i.i.d. sequence. This is also false. Roughly speaking, stable frequencies are a characteristic feature of so-called “ergodic” sequences, which include some Markov chains. I would not go as far as to recommend that probability instructors stop teaching about the relationship between probability and frequency early in the course. But I think that they should at least try to alleviate the didactic problems described above.

17.3 Undergraduate Textbooks Explanations of the frequency and subjective interpretations of probability in popular undergraduate textbooks are inconsistent with the philosophical theories of von Mises and de Finetti. My feeling is that the explanations represent textbook authors’ own views and they are not meant to represent faithfully the formal philosophical theories. The problem is that these informal views do not form a well defined philosophy of probability. De Finetti and von Mises had some good reasons why they made some bold statements. These reasons are not discussed in the textbooks. This creates an impression that the frequency and subjective interpretations of probability are easier to formalize than in fact they are. I will illustrate my point by reviewing two standard and popular undergraduate textbooks, [Pitman (1993)] and [Ross (2006)]. Pitman writes the following about the frequency and subjective interpretations of probability on page 11 of his book: “Which (if either) of these interpretations is ‘right’ is something which philosophers, scientists, and statisticians have argued bitterly for centuries. And very intelligent people still disagree.” I could not agree more. But Pitman gives no hint why the interpretations are controversial. In the first part of Sec. 1.2 in [Pitman (1993)], he discusses long run frequency of heads in a sequence of coin tosses, and also presents very convincing data in support of the claim that the probability that a newborn is a boy is 0.513. What might be controversial about these examples? Why would “very intelligent people” disagree?

page 380

April 29, 2016

9:7

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch17

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Teaching Probability

381

The part of the section on “opinions” is even more confusing. On page 16, Pitman discusses the probability of a particular patient surviving an operation. He presents a convincing argument explaining why doctors may reasonably disagree about this probability. But this disagreement is different from the disagreement between philosophers concerning the significance of subjective probabilities. Reasonable people may disagree about the temperature outside. One person may say that it “feels like” 95 degrees, and another one may say that the temperature is 90 degrees. Why do not physicists study “subjective temperature”? Pitman does not explain why we should care about subjective opinions. If I declare that there will be an earthquake in Berkeley next year with probability 88%, why should anyone (including me) care? The Dutch book argument and the decision theoretic axiomatic system are not mentioned. I have a feeling that Pitman is trying to say that “subjective” probabilities are crude intuitive estimates of “objective” probabilities. If this is the case, Pitman takes a strongly objectivist position in his presentation of subjective probabilities. My guess is based on this statement on page 17 in [Pitman (1993)], “Subjective probabilities are necessarily rather imprecise.” The only interpretation of “imprecise” that comes to my mind is that there exist objective probabilities, and differences between subjective probabilities and objective probabilities are necessarily large. A popular textbook [Ross (2006)] confuses philosophy, mathematics and science by comparing the frequency theory and Kolmogorov’s axioms in Sec. 2.3. The frequency theory of probability is a philosophical and scientific theory. The axioms of Kolmogorov are mathematical statements. Comparing them is like comparing apples and oranges. The author explains in Sec. 2.7 of [Ross (2006)] that probability can be regarded as a measure of the individual’s belief, and provides examples that would be hard to interpret using frequency. Hence, all of the three most popular probabilistic ideologies, those invented by von Mises, de Finetti and Kolmogorov, are mentioned in [Ross (2006)]. I will discuss a routine homework problem given in that book and show that it is incompatible with the three ideologies. Problem 53 (b) on page 194 of [Ross (2006)] states that “Approximately 80,000 marriages took place in the state of New York last year. Estimate the probability that for at least one of these couples both partners celebrated their birthday on the same day of the year.” The answer at the end of the book gives the probability as 1 − e−219.18 . It is hard to see how any of the three approaches to probability presented in [Ross (2006)] can help students

page 381

April 29, 2016

9:7

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

382

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch17

Resonance: From Probability to Epistemology and Back

interpret this problem and its solution. It is impossible to derive the answer from Kolmogorov’s axioms unless one introduces some significant extra postulates, such as (L4). There is nothing in Kolmogorov’s axioms that suggests that we should solve this problem using “cases equally possible.” I doubt that Ross would like students to believe that the answer to Problem 53 (b) is “subjective” in the sense that the answer represents only a measure of an individual’s belief, and some rational people may believe that the probability is different from 1 − e−219.18 . A natural frequency interpretation of the problem can be based on a sequence of data for a number of consecutive years. Even if we make a generous assumption that the New York state will not change significantly in the next 10,000 years, a sequence of data for 10,000 consecutive years cannot yield a relative frequency approximately equal to 1 − e−219.18 but significantly different from 1. A more precise formulation of the last claim is the following. It is more natural to consider the accuracy of an estimate of the probability of the complementary event, that is, “none of the partners celebrated their birthday on the same day of the year.” Its mathematical probability is e−219.18 . If we take the relative frequency of this event in a sequence of 10,000 observations (in 10,000 years) as an estimate of the true probability, and the true probability is e−219.18 , then the relative error of the estimate will be at least 100%. If the mathematical answer to the problem, that is, 1 − e−219.18 , has any practical significance, it has nothing to do with any real sequence. It is easy to see that the answer to Problem 53 (b) can be given a simple practical interpretation, based on (L6). For example, if a TV station is looking for a recently married couple with the same birthdays for a TV show, it can be certain that it will find such a couple within the state of New York.

page 382

April 29, 2016

9:7

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch18

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Chapter 18

Mathematical Methods of Probability and Statistics

I will present a review of some mathematical methods of probability and statistics used in the philosophical arguments in this book. This short review is not a substitute for a solid course in probability. Good textbooks at the undergraduate level are [Pitman (1993)] and [Ross (2006)].

18.1 Probability The mathematics of probability is based on Kolmogorov’s axioms. The fully rigorous presentation of the axioms requires some definitions from the “measure theory,” a field of mathematics. This material is not needed in this book, so I will present the axioms in an elementary way. Any probabilistic model, no matter how complicated, is represented by a space of all possible outcomes Ω. The individual outcomes ω in Ω can be very simple (for example, “heads,” if you toss a coin) or very complicated — a single outcome ω may represent temperatures at all places around the globe over the next year. Individual outcomes may be combined to form events. If you roll a die, individual outcomes ω are numbers 1, 2, . . . , 6, that is Ω = {1, 2, 3, 4, 5, 6}. The event “even number of dots” is represented by a subset of Ω, specifically, by {2, 4, 6}. Every event has a probability, that is, probability is a function that assigns a number between 0 and 1 (0 and 1 are not excluded) to every event. If you roll a “fair” die then all outcomes are equally probable, that is, P (1) = P (2) = · · · = P (6) = 1/6. Kolmogorov’s axioms put only one restriction on probabilities — if events A1 , A2 , . . . , An are disjoint, that is, at most one of them can occur, then the probability that at least one of them will occur is the sum of probabilities 383

page 383

April 29, 2016

9:7

384

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch18

Resonance: From Probability to Epistemology and Back

of A1 , A2 , . . . , An . In symbols,

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

P (A1 or A2 or . . . or An ) = P (A1 ) + P (A2 ) + · · · + P (An ). Kolmogorov’s axioms include an analogous statement for a countably infinite sequence of mutually exclusive events — this is called σ-additivity or countable additivity. A curious feature of Kolmogorov’s axiomatic system is that it does not include at all the notion of independence. We call two events (mathematically) independent if the probability of their joint occurrence is the product of their probabilities, in symbols, P (A and B) = P (A)P (B). The intuitive meaning of independence is that the occurrence of one of the events does not give any information about the possibility of occurrence of the other event. If a quantity X depends on the outcome ω of an experiment or observation then we call it a random variable. For example, if the experiment is a roll of two dice, the sum of dots is a random variable. If a random variable X may take values x1 , x2 , . . . , xn with probabilities p1 , p2 , . . . , pn then the number EX = p1 x1 + p2 x2 + · · · + pn xn is called the expected value or expectation of X. Intuitively speaking, the expectation of X is the (weighted) average, mean or central value of all possible values, although each one of these descriptions is questionable. The expected value of the number of dots on a fair die is 1/6 · 1 + 1/6 · 2 + · · · + 1/6 · 6 = 3.5. Note that the “expected value” of the number dots is not expected at all because the number of dots must be an integer. The expectation of (X − EX)2 , that is, E(X − EX)2 is called the variance of X and denoted VarX. Its square √root is called the standard deviation of X and denoted σX , that is σX = VarX. It is much easier to explain the intuitive meaning of standard deviation than that of variance. Most random variables take values different from their expectations and the standard deviation represents a typical difference between the value taken by the random variable and its expectation. The strange definition of the standard deviation, via variance and square root, has an excellent theoretical support — a mathematical result known as the Central Limit Theorem, to be reviewed next.

page 384

April 29, 2016

9:7

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch18

Mathematical Methods of Probability and Statistics

385

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

18.1.1 Law of large numbers, central limit theorem and large deviations principle A sequence of random variables X1 , X2 , X3 , . . . is called i.i.d. if these random variables are independent and have identical distributions. The Strong Law of Large Numbers says that if X1 , X2 , X3 , . . . are i.i.d. and EX1 exists then the averages (X1 + X2 + · · · + Xn )/n converge to EX1 with probability 1 when n goes to infinity. The weak Law of Large Numbers asserts that if X1 , X2 , X3 , . . . are i.i.d. and EX1 exists then for every ε > 0 and p < 1 we can find n so large that P (|(X1 + X2 + · · · + Xn )/n − EX1 | < ε) > p. A random variable √ Y is
ysaid to have the standard normal distribution if P (Y < y) = (1/ 2π) −∞ exp(−x2 /2)dx. At the intuitive level, the distribution of possible values of a standard normal random variable is represented by a bell-shaped curve centered at 0. Suppose that X1 , X2 , X3 , . . . are i.i.d., the expectation of any of these random variables is µ and its standard deviation is σ. The Central Limit  √ Theorem says that for large n, the normalized sum (1/(σ n)) nk=1 (Xk −µ) has a distribution very close to the standard normal distribution. Roughly speaking, the Large Deviations Principle (LDP) says that under appropriate assumptions, observing a value of a random variable far away from its mean has a probability much smaller than a naive intuition might suggest. For example, if X has the standard normal distribution, the probability that X will take a value greater than x is of order (1/x) exp(−x2 /2) for large x. The probability that the standard normal random variable will take a value 10 times greater than its standard deviation is about 10−38 . The Central Limit Theorem suggests that the LDP applies to sums or averages of sequences of i.i.d. random variables. In fact, it does, but the precise formulation of LDP will not be given here. The LDP-type estimates are not always as extremely small as the above example might suggest.

18.1.2 Exchangeability and de Finetti’s theorem A permutation π of a set {1, 2, . . . , n} is any one-to-one function mapping this set into itself. A sequence of random variables (X1 , X2 , . . . , Xn ) is called exchangeable if it has the same distribution as (Xπ(1) , Xπ(2) , . . . , Xπ(n) ) for every permutation π of {1, 2, . . . , n}. Informally, X1 , X2 , . . . , Xn are

page 385

April 29, 2016

9:7

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

386

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch18

Resonance: From Probability to Epistemology and Back

exchangeable if for any sequence of possible values of these random variables, any other ordering of the same values is equally likely. Recall that a sequence of random variables (X1 , X2 , . . . , Xn ) is called i.i.d. if these random variables are independent and have identical distributions. A celebrated theorem of de Finetti says that an infinite exchangeable sequence of random variables is a mixture of i.i.d. sequences. For example, for any given infinite exchangeable sequence of random variables taking values 0 or 1, one can generate a sequence with the same probabilistic properties by first choosing randomly, in an appropriate way, a number p in the interval [0, 1], and then generating an i.i.d. sequence whose elements Xk take values 1 with probability p and 0 with probability 1 − p. Deform a coin and encode the result of the kth toss as Xk , that is, let Xk = 1 if the kth toss is heads and Xk = 0 otherwise. Then X1 , X2 , X3 , . . . is an exchangeable sequence. Some probabilists and statisticians consider this sequence to be i.i.d. with “unknown probability of heads.”

18.2 Frequency Statistics Statistics is concerned with the analysis of data, although there is no unanimous agreement on whether this means “inference,” that is, the search for the truth, or making decisions, or both. One of the methods of frequency statistics is estimation — I will explain it using an example. Suppose that you have a deformed coin and you would like to know the probability p of heads (this formulation of the problem contains an implicit assumption that the probability p is objective). We can toss the coin n times and encode the results as a sequence of numbers (random variables) X1 , X2 , . . . , Xn , with the convention that Xk = 1 if the result of the kth toss is heads and Xk = 0 otherwise. Then we can calculate p = (X1 + X2 + · · · + Xn )/n, an “estimator” of p. The estimator p is our guess about the true value of p. One of its good properties is that it is “unbiased,” that is, its expectation is equal to p. The standard deviation √ of p is npq. Another procedure used by frequency statisticians is hypothesis testing. Consider the following drug-testing example. Suppose that a new drug is expected to give better results than an old drug. Doctors adopt (temporarily) a hypothesis H (often called a “null hypothesis”) that the new drug is not better than the old drug and choose a level of significance, often 5% or 1%. Then they give one drug to one group of patients and the other drug to another group of patients. When the results are collected,

page 386

April 29, 2016

9:7

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch18

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

Mathematical Methods of Probability and Statistics

387

the probability of the observed or “more extreme” results is calculated, assuming that the hypothesis H is true. If the probability is smaller than the significance level, the “null” hypothesis H is rejected and the new drug is declared to be better than the old drug. On the mathematical side, hypothesis testing proceeds along slightly different lines. Usually, at least two hypotheses are considered. Suppose that you can observe a random variable X whose distribution is either P0 or P1 . Let H0 be the hypothesis that the distribution is in fact P0 , and let H1 be the hypothesis that the distribution of X is P1 . An appropriate number c is found, corresponding to the significance level. When X is observed and its value is x, the ratio of probabilities P0 (X = x)/P1 (X = x) is calculated. If the ratio is less than c then the hypothesis H0 is rejected and otherwise it is accepted. The constant c can be adjusted to make one of the two possible errors small: rejecting H0 when it is true or accepting it when it is false. Finally, I will outline the idea of a “confidence interval,” as usual using an example. Suppose a scientist wants to find the value of a physical quantity θ. Assume further that he has at his disposal a measuring device that does not generate systematic errors, that is, the errors do not have a tendency to be mostly positive or mostly negative. Suppose that the measurements are X1 , X2 , . . . , Xn . The average of these numbers, X n = can be taken as an estimate of θ. The empirical (X1 + X2 + · · · + Xn )/n,   standard deviation σn = (1/n) nk=1 (Xk − X n )2 is a measure of accuracy of the estimate. If the number of measurements is large, and some other assumptions are satisfied, the interval (X n − σn , X n + σn ) covers the true value of θ with probability equal to about 68%. If the length of the interval is increased to 4 standard deviations, that is, if we use (X n −2σn , X n +2σn ), the probability of coverage of the true value of θ becomes 95%.

18.3 Bayesian Statistics Bayesian statistics derives its name from the Bayes theorem. Here is a very simple version of the theorem. Let P (A | B) denote the probability of an event A given the information that an event B occurred. Then P (A | B) = P (A and B)/P (B). Suppose that events A1 and A2 cannot occur at the same time but one of them must occur. The Bayes theorem is the following formula, P (A1 | B) =

P (B | A1 )P (A1 ) . P (B | A1 )P (A1 ) + P (B | A2 )P (A2 )

(18.1)

page 387

April 29, 2016

9:7

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

388

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch18

Resonance: From Probability to Epistemology and Back

Intuitively, the Bayes theorem is a form of a retrodiction, that is, it gives the probability of one of several causes (A1 or A2 ), given that an effect (B) has been observed. One of the simplest examples of the Bayesian method is the analysis of tosses of a deformed coin. A popular Bayesian model assumes that coin tosses are exchangeable. According to de Finetti’s theorem, this is mathematically equivalent to the assumption that there exists an unknown number Θ (a random variable), between 0 and 1, representing the probability of heads on a single toss. If we assume that the value of Θ is θ then the sequence of tosses is i.i.d. with the probability of heads on a given toss equal to θ. The Bayesian analysis starts with a prior distribution of Θ. A typical choice is the uniform distribution on [0, 1], that is, the probability that Θ is in a given subinterval of [0, 1] of length r is equal to r. Suppose that the coin was tossed n times and k heads were observed. The Bayes theorem can be used to show that, given these observations and assuming the uniform prior for Θ, the posterior probability of heads on the (n + 1)-st toss is (k + 1)/(n + 2). Some readers may be puzzled by the presence of constants 1 and 2 in the formula — one could expect the answer to be k/n. If we tossed the coin only once and the result was heads, then the Bayesian posterior probability of heads on the next toss is (k + 1)/(n + 2) = 2/3; this seems to be much more reasonable than k/n = 1.

18.4 Contradictory Predictions This section is devoted to a rigorous mathematical proof of a simple theorem formalizing the idea that two people are unlikely to make contradictory predictions even if they have different information sources. More precisely, suppose that two people consider an event A and they may know different facts. In this section, we will say that a person makes a prediction when she says that the probability of A is either smaller than δ or greater than 1 − δ, where δ > 0 is a small number, chosen to reflect the desired level of confidence. The two people make “contradictory predictions” if one of them asserts that the probability of A is less than δ and the other one says that the probability of A is greater than 1 − δ. The theorem proved below implies that two people can make the probability of making contradictory predictions smaller than an arbitrarily small number ε > 0, if they agree on using the same sufficiently small δ > 0 (depending on ε).

page 388

May 10, 2016

13:20

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch18

Mathematical Methods of Probability and Statistics

389

For the notation and definitions of σ-fields, conditional probabilities, etc., see any standard graduate level textbook on probability, such as [Durrett (1996)]. Let A be an event in some probability space (Ω, F, P ), so A ∈ F , and let X = P (A | G) and Y = P (A | H) for two sub-σ-fields G and H of F . Let

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

ε(δ) = sup P (|X − Y |) ≥ (1 − δ), where the supremum is taken over all probability spaces (Ω, F, P ), all events A ∈ F and all sub-σ-fields G and H of F . It was proved in Theorem 14.1 of [Burdzy (2009)] that ε(δ) ≤ 5δ for δ ≤ 1/10. The following stronger result is due to Jim Pitman. I follow his notes, at places verbatim, with his permission. Theorem 18.1 (Pitman (private communication)) For all δ ∈ (0, 1), 2δ ≤ ε(δ) ≤ 2δ. 1+δ

(18.2)

Proof. The lower bound on ε(δ) is attained by the following example. Fix 0 < δ < 1. Let A be an event of probability P (A) = 2δ/(1 + δ), say A = B ∪ C with B and C mutually exclusive events, each with probability δ/(1 + δ). Let G = B ∪ Ac , let H = C ∪ Ac , let G be generated by the partition {G, C} and let H be generated by the partition {H, B}. For any event F , let 1F denote the indicator of F , that is, 1F takes value 1 on F and 0 on F c . It is clear by construction that X = P (A | G) = δ1G + 1C ,

Y = P (A | H) = δ1H + 1B .

It follows that |X − Y | = (1 − δ)1A and hence P (|X − Y |) ≥ 1 − δ) = P (A) = 2δ/(1 + δ). For the upper bound in (18.2), it is enough to consider the case 0 < δ < 1/2. Observe that for any 0 < δ < 1 and any random variables X and

page 389

April 29, 2016

9:7

390

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-ch18

Resonance: From Probability to Epistemology and Back

Y with 0 ≤ X ≤ 1 and 0 ≤ Y ≤ 1 there is the inclusion {|X − Y |) ≥ 1 − δ} ⊂ {X ≤ δ, Y ≥ 1 − δ} ∪ {X ≤ δ, Y ≥ 1 − δ}. (18.3) Since X = E(1A | X), P (X ≤ δ, Y ≥ 1 − δ, A) ≤ P (X ≤ δ, A) = E(1{X≤δ} X) ≤ δP (X ≤ δ).

Resonance Downloaded from www.worldscientific.com by UNIVERSITY COLLEGE LONDON (UCL) on 12/28/17. For personal use only.

We have 1 − Y = E(1Ac | Y ) so P (X ≤ δ, Y ≥ 1 − δ, Ac ) ≤ P (Y ≥ 1 − δ, Ac ) = E(1{1−Y ≤δ} (1 − Y )) ≤ δP (Y ≥ 1 − δ). It follows that P (X ≤ δ, Y ≥ 1 − δ) ≤ δ(P (X ≤ δ) + P (Y ≥ 1 − δ))

(18.4)

and similarly P (Y ≤ δ, X ≥ 1 − δ) ≤ δ(P (Y ≤ δ) + P (X ≥ 1 − δ)).

(18.5)

For 0 < δ < 1/2 the events X ≤ δ and X ≥ δ are disjoint, so P (X ≤ δ) + P (X ≥ 1 − δ) ≤ 1, and the same holds for Y . Add (18.4) and (18.5) and use (18.3) to obtain the upper bound in (18.2).


page 390

Resonance Downloaded from www.worldscientific.com by 106.180.4.152 on 05/26/20. Re-use and distribution is strictly not permitted, except for Open Access articles.

April 29, 2016

9:7

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-bib

Bibliography

ABC News (2014). Quarter of Americans Convinced Sun Revolves Around Earth, Survey Finds. http://abcnews.go.com/US/quarter-americans-convincedsun-revolves-earth-survey-finds/story?id=22542847 [Online; accessed 31January-2015]. Berger, J. (1985). Statistical Decision Theory and Bayesian Analysis. 2nd edn. (Springer, New York). Burdzy, K. (2009). The Search for Certainty. On the Clash of Science and Philosophy of Probability. (World Scientific, Hackensack, NJ). Carnap, R. (1950). Logical Foundations of Probability. (University of Chicago Press, Chicago). Columbia Law Survey (2002). Americans’ Knowledge of the U.S. Constitution. http://www2.law.columbia.edu/news/surveys/survey constitution/ index.shtml [Online; accessed 31-January-2015]. de Finetti, B. (1974). Theory of Probability. A Critical Introductory Treatment. Vol. 1. (Wiley, London). de Finetti, B. (1975). Theory of Probability. A Critical Introductory Treatment. Vol. 2. (Wiley, London). DeGroot, M.H. (1970). Optimal Statistical Decisions. (McGraw-Hill, New York). Durrett, R. (1996). Probability: Theory and Examples. 2nd edn. (Duxbury Press, Belmont, CA). Ethier, S.N. and Kurtz, T.G. (1986). Markov Processes: Characterization and Convergence. (Wiley, New York). Feyerabend, P. (1975). Against Method: Outline of an Anarchistic Theory of Knowledge. (NLB, London; Humanities Press, Atlantic Highlands). Fine, T.L. (1973). Theories of Probability. An Examination of Foundations. (Academic Press, New York). Fishburn, P.C. (1970). Utility Theory for Decision Making. (Wiley, New York). Galavotti, M.C. (2005). Philosophical Introduction to Probability. (CSLI Publications, Stanford, California). Gelman, A., Carlin, J.B., Stern, H.S. and Rubin, D.B. (2004). Bayesian Data Analysis. 2nd edn. (Chapman and Hall/CRC, New York). Gillies, D.A. (1973). An Objective Theory of Probability. (Methuen, London). 391

page 391

April 29, 2016

9:7

Resonance Downloaded from www.worldscientific.com by 106.180.4.152 on 05/26/20. Re-use and distribution is strictly not permitted, except for Open Access articles.

392

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-bib

Resonance: From Probability to Epistemology and Back

Gillies, D.A. (2000). Philosophical Theories of Probability. (Routledge, London). Gould, S.J. (1996). The Mismeasure of Man. (Norton, New York). Hacking, I. (1965). Logic of Statistical Inference. (University Press, Cambridge). Hacking, I. (2001). An Introduction to Probability and Inductive Logic. (Cambridge University Press, Cambridge, U.K.; New York). H´ ajek, A. (2007). Interpretations of Probability. Stanford Encyclopedia of Philosophy. http://plato.stanford.edu/entries/probability-interpret/ [Online; accessed 12-August-2015]. Hamer, D.H. (2004) The God Gene: How Faith is Hardwired into our Genes. (Doubleday, New York). Heller, M. (2012). Philosophy of Chance. A Cosmic Fugue with a Prelude and a Coda. (Copernicus Center Press, Krak´ ow). Heller, M. (2013). Theology for non-believers. Mathematica Applicanda (Matematyka Stosowana), 41, No. 2, 249–254. Herrnstein, R.J. and Murray, C.A. (1994). The Bell Curve: Intelligence and Class Structure in American Life. (Free Press, New York). Hofstadter, D. (1979). G¨ odel, Escher, Bach: An Eternal Golden Braid. (Basic Books, New York). Holmes, S. (2007). The IID sequence applet http://www-stat.stanford.edu/ ∼susan/surprise/IIDnew.html [Online; accessed 06-July-2006]. Howie, D. (2002). Interpreting Probability: Controversies and Developments in the Early Twentieth Century. (Cambridge University Press, New York). Jaynes, E.T. (2003). Probability Theory. The Logic of Science. (Cambridge University Press, Cambridge). Jech, T. (2003). Set Theory: The Third Millennium Edition, Revised and Expanded. (Springer, New York). Keller, J. (1986) The probability of heads. American Mathematical Monthly, 93, 191–197. Keynes, J.M. (1921). A Treatise on Probability. (MacMillan, London). Kuhn, T. (1970). The Structure of Scientific Revolutions. (2nd edition, with postscript; University of Chicago Press, Chicago). Kyburg, H.E. and Smokler, H.E. (eds.) (1964). Studies in Subjective Probability. (Wiley, New York). Lakatos, I. (1978a). The Methodology of Scientific Research Programmes. Volume 1: Philosophical Papers. (Cambridge University Press, Cambridge; New York). Lakatos, I. (1978b). Mathematics, Science and Epistemology. Volume 1: Philosophical Papers. (Cambridge University Press, Cambridge; New York). Lecoutre, B. and Poitevineau, J. (2011). The Significance Test Controversy and the Bayesian Alternative (version 12). StatProb: The Encyclopedia Sponsored by Statistics and Probability Societies. http://statprob.com/encyclopedia / SignificanceTestControversyAndTheBayesianAlternative . html [Online; accessed 12-August-2015]. Lewis, D. (1976). Probabilities of conditionals and conditional probabilities. The Philosophical Review, 85, No. 3, 297–315.

page 392

April 29, 2016

9:7

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-bib

Resonance Downloaded from www.worldscientific.com by 106.180.4.152 on 05/26/20. Re-use and distribution is strictly not permitted, except for Open Access articles.

Bibliography

393

Lewis, D. (1980). A Subjectivist’s Guide to Objective Chance, in Studies in Inductive Logic and Probability, Jeffrey, Richard C. (ed.), Vol. II. (University of California Press, Berkeley and Los Angeles). Lewis, D. (1994). Humean Supervenience Debugged. Mind, 103, 473–490. Lindley, D.V. (2006). Understanding Uncertainty. (Wiley-Interscience, Hoboken, N.J.). Lyon, A. (2010). Philosophy of Probability, in Philosophies of the Sciences: A Guide. F. Allhoff (ed.), (Wiley-Blackwell, Oxford, UK). Marsaglia, G. (1995). The Marsaglia Random Number CDROM with The Diehard Battery of Tests of Randomness. (Supercomputer Computations Research Institute and Department of Statistics, Florida State University). http:// www.stat.fsu.edu/pub/diehard/. Mellor, D.H. (2005). Probability: A Philosophical Introduction. (Routledge, London; New York). Nickerson, R.S. (2004). Cognition and Chance. The Psychology of Probabilistic Reasoning. (Lawrence Erlbaum Associates, Mahwah, New Jersey). Penrose, R. (2005). The Road to Reality: A Complete Guide to the Laws of the Universe. (Knopf, New York). Pesic, P. (1999) Wrestling with Proteus: Francis Bacon and the “torture” of nature. Isis, 90, No. 1, 81–94. Pitman, J. (1993). Probability. (Springer-Verlag, New York). Polanyi, M. (1958). Personal Knowledge; Towards a Post-critical Philosophy. (University of Chicago Press, Chicago). Popper, K.R. (1968). The Logic of Scientific Discovery. (Harper and Row, New York). Primas, H. (1999). Basic elements and problems of probability theory. Journal of Scientific Exploration, 13, 579–613. Revonsuo, A. (2010). Consciousness: The Science of Subjectivity. (Psychology Press, New York). Ross, S. (2006). A First Course in Probability. 7th edn. (Pearson Prentice Hall, Upper Saddle River). Ruelle, D. (1991). Chance and Chaos. (Princeton University Press, Princeton, New Jersey). Ryder, J.M. (1981). Consequences of a simple extension of the Dutch book argument. British Journal of the Philosophy of Science 32, 164–167. Savage, L.J. (1972). The Foundations of Statistics, 2nd revised edn. (Dover, New York). Sokal, A. (2008). Beyond the Hoax: Science, Philosophy, and Culture. (Oxford University Press, Oxford; New York). Tegmark, M. (2014). Our Mathematical Universe: My Quest for the Ultimate Nature of Reality. (Knopf, New York). Unwin, S. (2003). The Probability of God: A Simple Calculation that Proves the Ultimate Truth. (Three Rivers Press, New York). von Mises, R. (1957). Probability, Statistics and Truth, 2nd revised English edn. (Dover, New York).

page 393

April 29, 2016

9:7

Resonance Downloaded from www.worldscientific.com by 106.180.4.152 on 05/26/20. Re-use and distribution is strictly not permitted, except for Open Access articles.

394

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-bib

Resonance: From Probability to Epistemology and Back

von Plato, J. (1994). Creating Modern Probability. (Cambridge University Press, Cambridge). Weatherford, R. (1982). Philosophical Foundations of Probability Theory. (Routledge & K. Paul, London). Wikipedia. (2006a). Bayesian Probability — Wikipedia, The Free Encyclopedia. http://en.wikipedia.org/wiki/Bayesian probability [Online; accessed 06July-2006]. Wikipedia. (2006b). Frequency Probability — Wikipedia, The Free Encyclopedia. http://en.wikipedia.org/wiki/Frequentist [Online; accessed 06-July-2006]. Wikipedia. (2014a). St. Petersburg Paradox — Wikipedia, The Free Encyclopedia. http://en.wikipedia.org/wiki/St. Petersburg paradox [Online; accessed 11December-2014]. Wikipedia. (2014b). Ellsberg Paradox — Wikipedia, The Free Encyclopedia. http: //en.wikipedia.org/wiki/Ellsberg paradox [Online; accessed 15-December2014]. Wikipedia. (2014c). Pascal’s Wager — Wikipedia, The Free Encyclopedia. https://en.wikipedia.org/wiki/Pascal%27s Wager [Online; accessed 15December-2014]. Wikipedia. (2014d). Cromwell’s Rule — Wikipedia, The Free Encyclopedia. https://en.wikipedia.org/wiki/Cromwell%27s rule [Online; accessed 15December-2014]. Wikipedia. (2014e). Approximations of π — Wikipedia, The Free Encyclopedia. https://en.wikipedia.org/wiki/Approximations of %CF%80 [Online; accessed 15-December-2014]. Wikipedia. (2014f). Normal number — Wikipedia, The Free Encyclopedia. http: //en.wikipedia.org/wiki/Normal number [Online; accessed 15-December2014]. Wikipedia. (2014g). Goodhart’s Law — Wikipedia, The Free Encyclopedia. http:// en.wikipedia.org/wiki/Goodhart%27s law [Online; accessed 15-December2014]. Wikipedia. (2014h). Campbell’s Law — Wikipedia, The Free Encyclopedia. http:// en.wikipedia.org/wiki/Campbell%27s law [Online; accessed 15-December2014]. Wikipedia. (2014i). Algorithmically Random Sequence — Wikipedia, The Free Encyclopedia. http://en.wikipedia.org/wiki/Algorithmically random sequence [Online; accessed 15-December-2014]. Wikipedia. (2014j). Sampling Bias — Wikipedia, The Free Encyclopedia. http:// en.wikipedia.org/wiki/Sampling bias [Online; accessed 21-December-2014]. Wikipedia. (2014k). Inattentional Blindness — Wikipedia, The Free Encyclopedia. http://en.wikipedia.org/wiki/Inattentional blindness [Online; accessed 21December-2014]. Wikipedia. (2014l). Camponotus Saundersi — Wikipedia, The Free Encyclopedia. http://en.wikipedia.org/wiki/Camponotus saundersi [Online; accessed 23December-2014].

page 394

April 29, 2016

9:7

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-bib

Resonance Downloaded from www.worldscientific.com by 106.180.4.152 on 05/26/20. Re-use and distribution is strictly not permitted, except for Open Access articles.

Bibliography

395

Wikipedia. (2014m). Cosmological Constant — Wikipedia, The Free Encyclopedia. http://en.wikipedia.org/wiki/Cosmological constant [Online; accessed 25-December-2014]. Wikipedia. (2014n). Hidden Variable Theory — Wikipedia, The Free Encyclopedia. http://en.wikipedia.org/wiki/Hidden variable theory [Online; accessed 25-December-2014]. Wikipedia. (2014o). Beatification of Pope John Paul II — Wikipedia, The Free Encyclopedia. http://en.wikipedia.org/wiki/Beatification of Pope John Paul II [Online; accessed 25-December-2014]. Wikipedia. (2014s). Constructive Empiricism — Wikipedia, The Free Encyclopedia. http://en.wikipedia.org/wiki/Constructive empiricism [Online; accessed 26-December-2014]. Wikipedia. (2014t). Dissociative Identity Disorder — Wikipedia, The Free Encyclopedia. http://en.wikipedia.org/wiki/Dissociative identity disorder [Online; accessed 27-December-2014]. Wikipedia. (2014u). Schizophrenia — Wikipedia, The Free Encyclopedia. http:// en.wikipedia.org/wiki/Schizophrenia [Online; accessed 27-December-2014]. Wikipedia. (2014v). Turing Test — Wikipedia, The Free Encyclopedia. http://en. wikipedia.org/wiki/Turing test [Online; accessed 28-December-2014]. Wikipedia. (2014w). DARPA Grand Challenge — Wikipedia, The Free Encyclopedia. http://en.wikipedia.org/wiki/DARPA Grand Challenge [Online; accessed 28-December-2014]. Wikipedia. (2014x). Creationism — Wikipedia, The Free Encyclopedia. http:// en.wikipedia.org/wiki/Creationism [Online; accessed 28-December-2014]. Wikipedia. (2014y). Logical Positivism — Wikipedia, The Free Encyclopedia. http://en.wikipedia.org/wiki/Logical positivism [Online; accessed 29December-2014]. Wikipedia. (2014z). Higgs Boson — Wikipedia, The Free Encyclopedia. http:// en.wikipedia.org/wiki/Higgs boson [Online; accessed 30-December-2014]. Wikipedia. (2015a). New Riddle of Induction — Wikipedia, The Free Encyclopedia. http://en.wikipedia.org/wiki/New riddle of induction [Online; accessed 1-January-2015]. Wikipedia. (2015b). Anthropic Principle — Wikipedia, The Free Encyclopedia. http://en.wikipedia.org/wiki/Anthropic principle [Online; accessed 1-January-2015]. Wikipedia. (2015c). Axiom of Choice — Wikipedia, The Free Encyclopedia. http://en.wikipedia.org/wiki/Axiom of choice [Online; accessed 4-January-2015]. Wikipedia. (2015d). Zorn’s Lemma — Wikipedia, The Free Encyclopedia. http:// en.wikipedia.org/wiki/Zorn%27s lemma [Online; accessed 4-January-2015]. Wikipedia. (2015e). Well-order — Wikipedia, The Free Encyclopedia. http://en. wikipedia.org/wiki/Well-order [Online; accessed 4-January-2015]. Wikipedia. (2015f). Witch-hunt — Wikipedia, The Free Encyclopedia. http://en. wikipedia.org/wiki/Witch-hunt [Online; accessed 5-January-2015].

page 395

April 29, 2016

9:7

Resonance Downloaded from www.worldscientific.com by 106.180.4.152 on 05/26/20. Re-use and distribution is strictly not permitted, except for Open Access articles.

396

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-bib

Resonance: From Probability to Epistemology and Back

Wikipedia. (2015g). Shakespeare Authorship Question — Wikipedia, The Free Encyclopedia. http://en.wikipedia.org/wiki/Shakespeare authorship question [Online; accessed 29-January-2015]. Wikipedia. (2015h). Double-slit Experiment — Wikipedia, The Free Encyclopedia. http://en.wikipedia.org/wiki/Double-slit experiment [Online; accessed 11-February-2015]. Wikipedia. (2015i). Schr¨ odinger’s Cat — Wikipedia, The Free Encyclopedia. http://en.wikipedia.org/wiki/Schr%C3%B6dinger%27s cat [Online; accessed 11-February-2015]. Wikipedia. (2015j). Uncertainty Principle — Wikipedia, The Free Encyclopedia. http://en.wikipedia.org/wiki/Uncertainty principle [Online; accessed 11-February-2015]. Wikipedia. (2015k). Wave Function Collapse — Wikipedia, The Free Encyclopedia. http://en.wikipedia.org/wiki/Wave function collapse [Online; accessed 11-February-2015]. Wikipedia. (2015l). Algorithmically Random Sequence — Wikipedia, The Free Encyclopedia. https://en.wikipedia.org/wiki/Algorithmically random sequence [Online; accessed 2-August-2015]. Wikipedia. (2015m). Implicature — Wikipedia, The Free Encyclopedia. https:// en.wikipedia.org/wiki/Implicature [Online; accessed 9-August-2015]. Wilson, F. (2014). John Stuart Mill. in The Stanford Encyclopedia of Philosophy (Spring 2014 Edition), Edward N. Zalta (ed.). http://plato.stanford.edu/ archives/spr2014/entries/mill/ [Online; accessed 12-August-2015].

page 396

Resonance Downloaded from www.worldscientific.com by 106.180.4.152 on 05/26/20. Re-use and distribution is strictly not permitted, except for Open Access articles.

April 29, 2016

9:7

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-index

Index

A

axiom, 24, 27, 29–30, 51–52, 87, 89, 100, 114, 139, 146, 166, 227, 257–259, 275, 284–285, 287, 290, 320, 323–325, 334, 340, 348, 354, 376, 378, 381 Kolmogorov’s, 35, 54, 80, 82, 127, 153, 221, 257–258, 264, 345, 347, 376–377, 381–384 of choice, 128, 146, 257 ZFC, 114, 257 Aztecs, 334

Achilles, 35 agent, 86–87, 159, 167, 169, 210, 334, 360 aggregate, 250, 292–293, 297, 299, 321, 325 aggression, 60, 73, 157, 282, 334 alien, 3, 81, 188–189, 210 America, 83, 334 American, 36, 201 American Constitution, 36, 126 ammunition, 42 analysis non-standard, 128, 257, 356 anger, 131, 157 animal, 5, 70, 150–153, 159–160, 167, 190, 200, 202, 242, 246, 280–281, 365 antenna, 156, 167, 222 anthropic principle, 148 apple, 108, 143, 145–146, 167, 183, 202–203, 219, 243, 266, 381 arbitrage, 29, 89, 109–110 Aristotle, 199 art, 161, 168, 190, 284, 342, 348 asbestos, 186 association, 379 astronomy, 187 asymmetry, 146, 220, 254, 269 atheism, 6–7, 110–111, 178, 180, 363 atom, 60, 73, 229, 288, 341–343, 366 atom bomb, 47

B Bacon, F., 192, 197 Bayes’ theorem, 5, 8, 10, 27, 30–31, 37, 89–91, 95–96, 99, 112, 155, 197, 208–209, 225, 233, 268, 307, 309, 312, 314, 318, 372–373, 387–388 Bayes, T., 5, 13 beauty, 56 behavior, 28, 43, 87, 97, 101, 103, 106, 136, 151–152, 167–170, 180, 194, 197, 200, 221, 240, 281–282, 318, 334 beneficiary, 82, 280 Berger, J., 323–325, 340 Bernoulli trials, 50, 69 Bernoulli, J., 5, 35, 120, 289, 357 bet, 27–28, 90–91, 93–94, 254, 323, 355 Bible, 123, 198, 333 Big Bang, 323, 340 397

page 397

April 29, 2016

9:7

Resonance Downloaded from www.worldscientific.com by 106.180.4.152 on 05/26/20. Re-use and distribution is strictly not permitted, except for Open Access articles.

398

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-index

Index

biochemistry, 187 biology, 10, 84, 187, 193, 261, 338, 365 black box, 160–161 Black–Scholes theory, 109 blood pressure, 31, 95, 221 Bohr, N., 199 Boole, G., 21, 220 brain, 43, 132, 157, 164, 168, 266, 282, 373 Brownian motion, 64, 137, 240 fractional, 63–65, 235 geometric, 222 Buddha, 123 business, 15, 31, 81, 111, 273, 318 butterfly, 94, 133, 366 butterfly effect, 93, 261 C calculus, 67, 75, 187 probability, 27–28, 52, 95, 120, 309 camel, 41–42 cancer, 105, 107, 118–119, 136, 139, 154, 239, 339, 379 Carnap, R., 21, 35, 108, 125, 189, 209, 351 casino, 31–32, 61–62, 257, 353 cat, 234, 349 Catholicism, 110, 178, 182, 191, 199, 333, 336–337 causation, 379 Central Limit Theorem, 8, 51, 61, 77, 279, 293, 384–385 certainty, vii, 33–34, 43, 69, 89, 101, 110, 133–134, 146, 180, 201, 204, 235, 254, 271, 302, 338, 342, 364, 368 chaos, 93, 149, 173, 255, 261 Chebyshev inequality, 379 cheese, 361 green, 194, 361, 364–366 chemistry, 59, 84, 193, 204, 210, 222, 248, 276, 324, 350, 365, 373 Chinese Communist Party, 44, 47

Christianity, 113, 179, 190, 198, 212, 333–334, 343–344 cigarette, 107, 139, 186, 252 circularity, 25, 213, 274 civilization, 11, 45, 62, 152, 188–189, 210–211, 321, 327, 334, 349 classical definition of probability, 20 climate, 137, 143–144, 151, 373 coin, 2, 20, 23–24, 53, 83, 90–91, 93–94, 115, 155, 171, 203, 211, 219–221, 229, 232–233, 251–252, 254–256, 260–262, 269, 276, 285, 313–314, 319, 345, 353, 367, 376–377, 380, 383, 388 deformed, 20–21, 24–25, 72, 76, 103–104, 220, 222, 224, 254–256, 264–265, 268–269, 295, 304, 310, 312, 314, 326, 345, 378, 386, 388 coincidence, 167–168 collective, 5, 8, 19, 31–32, 34, 48–50, 56, 59–62, 65–72, 74–77, 106, 113, 132–133, 135–136, 153, 229–230, 241, 247, 268–270, 291, 296, 299–300, 304, 335–336, 349, 376–377 college, 9, 280, 355, 376 communism, 41, 44, 47, 177, 179, 182, 192, 333–336, 343–344 component principal, 157 computer, 22, 25, 34, 71–73, 81–82, 86, 97, 132, 135, 145, 148, 151, 159, 161, 168–171, 173, 183, 188, 210, 224, 226, 230–231, 234, 237, 278, 312, 316, 339, 343, 350 computing digital, 158 parallel, 158–159 quantum, 158–159 conditionalization, 94–95, 268 conditioning, 30, 60, 218, 233, 268, 308

page 398

April 29, 2016

9:7

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-index

Resonance Downloaded from www.worldscientific.com by 106.180.4.152 on 05/26/20. Re-use and distribution is strictly not permitted, except for Open Access articles.

Index confidence interval, 25, 138, 253, 291–294, 297, 301, 321, 327, 338, 341, 387 conflict, 122–124 consciousness, viii, 3–4, 143, 164–166, 169, 174 consensus, 1, 107, 118, 137, 224, 320, 341–343 consistency, 2, 8, 23, 27–28, 30–31, 33, 36, 51, 54, 81–82, 84–88, 94–106, 109, 111–113, 115, 117, 132, 136, 138–139, 198, 232, 249, 273, 283–284, 290, 307, 309, 318–319, 336, 345, 347, 349, 375, 378 consistent decisions, 8 constant, 84, 112, 185, 199, 248, 253–254, 256, 288, 295, 298, 305, 326–328, 338, 358, 387–388 constructive empiricism, 206 contract, 94, 100–102, 360 conventionalism, 205, 210, 212 convexity, 30, 288 Copernican theory, 179, 191 correlation, 194, 231, 379 countable additivity, 264–265, 376, 384 Cournot, A., 22, 278 Creationism, 48, 187, 191, 201 credible interval, 138, 254, 321, 325 criminal trial, 131, 254, 299–300, 328–329 D Dalton, J., 342 Darwin, C., 150, 175, 179–180, 190, 193, 199, 217, 281, 372 data, 8, 12–13, 25, 30–31, 50, 54–55, 62, 65–66, 68, 72–73, 83, 88, 95–96, 98, 100–103, 105–106, 112, 119, 126, 136, 139–140, 155–156, 158, 165, 181–182, 221–224, 227–228, 231, 234, 239, 241, 243, 246–247, 250–253, 255, 259–260, 274, 279, 291, 293, 295–296, 298, 303, 305, 307, 309, 311–312, 314–321, 323,

399

326, 328, 336–340, 343, 347, 350, 363, 379–380, 382, 386 de Finetti’s theorem, 256, 310, 385–386, 388 de Finetti, B., vii–viii, 2, 4–8, 10, 13–14, 22–24, 26–27, 32–35, 38–39, 42, 45–47, 51–58, 65, 79–85, 87–89, 93, 96, 99–103, 106–112, 114–118, 133–139, 152–153, 189, 198, 203–204, 211, 224, 243, 249, 256, 260, 267, 270, 307–310, 319, 323–324, 335–336, 343–347, 349, 351, 354, 377, 380–381 de Mere, Ch., 5, 20, 379 death, 2, 57, 133, 252, 259, 334, 341, 343, 367 decision(s), 8, 26–27, 29–30, 51, 56, 71–72, 76, 81–82, 85, 89, 92–93, 96, 98–100, 102–103, 106, 110–113, 118, 122, 134, 136, 138, 204–205, 232, 241, 249–250, 255, 271, 296, 300, 307, 309, 311, 316, 318–319, 321–322, 324, 326–327, 329, 334, 338, 345, 347, 354–357, 359–361, 366–370, 372, 378, 381, 386 aggregate, 250, 277 sequential, 95 strategy, 8, 10, 30, 51, 82, 87–88, 95, 99, 102, 104, 113, 139, 281, 375 DeGroot, M., 29, 89, 318 demarcationism, 206 demiurge, 43, 98–99, 145 democracy, 133, 151, 182, 333 Descartes, R., 180, 196 dice, 10, 20, 84, 138, 155, 171, 199, 211, 219, 226, 273, 276, 376, 378, 383–384 differential equation, 154, 313 partial, 257–258 stochastic partial, 155 dinosaur, 48, 145, 187 Dirac, P., 199 distribution, 8, 30–31, 52, 55, 87, 89, 101–102, 104, 106, 111, 120, 155,

page 399

April 29, 2016

9:7

Resonance Downloaded from www.worldscientific.com by 106.180.4.152 on 05/26/20. Re-use and distribution is strictly not permitted, except for Open Access articles.

400

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-index

Index

223, 233, 247, 255, 261–262, 274, 276–277, 284, 290, 292, 295, 297, 310, 312, 314–315, 320, 322, 327, 385–387 continuous, 376 empirical, 234 exchangeable, 256 exponential, 376 geometric, 376 mixing, 256, 310 normal, 61, 155, 209, 235, 244–245, 371, 385 Poisson, 376 posterior, 31, 89–90, 95, 99–102, 105, 112, 136, 139, 197, 209, 224, 233, 274, 308–309, 312, 314–316, 318–322, 327, 340, 376 prior, 10, 31, 34, 53–54, 89, 95, 99, 101–102, 106, 132, 139, 197, 209, 221, 224–225, 233, 274, 298, 308–317, 319–322, 328–329, 388 uniform, 119, 209, 220–221, 263, 314, 317, 388 DNA, 150, 187 doctor, 95–96, 109, 112, 246, 337, 381, 386 dollar, 30, 103, 248, 284, 288–289, 353–354, 362–363 Dostoevsky, F., 84, 126 dream, 32, 43, 114, 136, 145, 189 drug, 55, 68, 84, 95–96, 99, 109, 133, 145, 222, 247–248, 250, 288, 295, 299, 302, 337, 339, 386–387 Dutch book, 27–29, 34, 51, 81–82, 88–93, 100–103, 110–111, 134, 273, 319, 323–325, 340, 354, 381 dynamical system, 261, 263 dyslexia, 85 E e-mail, 360, 362 Earth, 36, 43, 50, 60, 121, 143, 147, 188, 249, 283, 365

economics, 186 economy, 193, 251 eigenfunction, 158 Einstein, A., 6, 33, 48, 52, 136, 148, 167–168, 171, 183, 193, 199, 208, 243, 258–259, 365 electromagnetism, 9, 222 elitism, 206 emperor, 9 energy, 33, 72, 83, 184, 193, 261, 288 heat, 83 mechanical, 83 engineer, 82, 101, 188, 303 engineering, 158, 231 environment, 150, 153, 156, 249 ergodic sequence, 380 theorem, 61 error Type I, 70, 299 Type II, 299 estimator, 222, 292, 294–298, 301, 305, 340, 386 unbiased, 71, 295–298, 305, 339, 376, 378, 386 eureka, 168 event, 2–10, 13, 20–28, 31–33, 36–38, 45, 48–49, 51–54, 57, 60, 62–63, 66–67, 69–70, 72–77, 80, 83, 85–87, 92–94, 99–100, 103, 105–106, 111, 114–117, 127–129, 132–137, 143, 146–147, 153, 158, 163–167, 179, 188, 202–203, 209, 217–222, 225–234, 236, 238–239, 241–246, 248–249, 251–254, 256–258, 260–261, 263–265, 267–269, 271, 273, 275–276, 278–280, 282–283, 286, 297, 299–300, 303, 308–309, 312, 323, 326–327, 335–336, 341, 345–346, 355, 357, 359, 364–365, 368, 373, 376, 378–379, 382–384, 387–390 Everett, H., 262 evolution, 3–4, 48, 149–150, 152–153, 160, 162–163, 165, 175, 180, 187,

page 400

April 29, 2016

9:7

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-index

Resonance Downloaded from www.worldscientific.com by 106.180.4.152 on 05/26/20. Re-use and distribution is strictly not permitted, except for Open Access articles.

Index 190, 199–200, 213, 217, 220, 242, 280–282 exchangeability, 14, 32, 61, 65, 67–68, 85, 104, 106, 120–122, 156, 219, 221, 224, 245–246, 248, 250, 252, 256–257, 267–270, 303, 310, 319–320, 328, 378, 385–386, 388 expected value, 28, 30, 71, 73, 103, 138, 232, 272–277, 279–281, 286–287, 293, 295–296, 303, 305, 307, 309, 322, 325, 353–354, 366, 368, 370–372, 379–380, 384–386 conditional, 223 experiment, 2, 6, 8, 11, 20, 22–26, 28, 36–37, 65, 68–70, 72, 83–84, 91, 95, 113, 115, 120–121, 134, 145, 155, 158, 165, 167, 185–186, 193, 195–197, 199–200, 203, 227–229, 243–245, 249–250, 254, 258–260, 262, 265, 267, 304, 307, 309, 327, 335, 341–342, 344–345, 347, 350–351, 373, 377–378, 384 F false negative, 299, 301–302, 313 false positive, 299, 301–302, 313 falsificationism, 155, 189, 195–199, 205, 207, 210, 212, 263, 304 fascism, 192 Fermat, P., 5, 20 Feyerabend, P., 43–48, 175, 184, 192, 196–197, 207, 211 filter, 3, 149–150, 156, 158, 163, 170 finance, 29, 67, 96, 109, 222–223, 288, 295, 373 Fisher, R., 12, 233, 255 Fitelson, B., 206 Frankenstein, 108 free will, viii, 4, 143, 171–174 frequency, 2, 5–6, 8, 10–11, 13, 24–26, 31–34, 36–37, 49–50, 52–53, 55, 60–62, 66–67, 69–70, 72, 74, 76, 83–84, 109, 113, 120–123, 133–135, 198, 218, 222, 226, 228, 231–234, 241, 245, 247–249, 252, 255, 263,

401

265, 269–270, 273, 298–302, 305, 313, 323, 325, 336, 344, 346, 368, 372, 375, 377, 379–382 long run, 10, 25, 52, 247, 267, 324 stable, 10, 12, 234, 380 Freud, S., 164 friendship, 30, 191, 284, 288–289 fuzzy set, 127, 287 G G¨ odel, K., 35, 114, 128, 136, 350, 366 gain, 28, 30, 89, 101, 103–104, 204, 272–280, 285–287, 307, 309, 322, 338, 355, 363, 368, 371 galaxy, 3, 38, 81, 92, 118, 283 Galileo, G., 48 game, 31, 62, 75, 103–104, 145, 172, 219, 257, 276, 285–286, 289, 353–355 garlic, 53 Gelman, A., viii, 323–324, 340 gene, 178 genius, 48, 168 geology, 187 Germany, 50 Gillies, D., 19, 35, 88, 108 God, 6–7, 42, 45, 84, 105, 107, 110, 126, 132, 151, 173, 178–180, 190, 199, 201, 212, 333, 348, 355–357, 359–360, 363, 372–373 Goodman, N., 148 gorilla, 145 government, 133, 180, 192, 251 gravitation, 52, 107, 148–150, 182, 185, 276 grue, 148–149 gun, 131–132 H Hacking, I., 35, 39, 88, 94, 108 happiness, 123, 157, 280 heaven, 199, 355, 358 Heisenberg, W., 81, 199

page 401

April 29, 2016

9:7

Resonance Downloaded from www.worldscientific.com by 106.180.4.152 on 05/26/20. Re-use and distribution is strictly not permitted, except for Open Access articles.

402

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-index

Index

hell, 355, 358 Heller, M., 39, 179 histogram, 234 HIV, 299, 313 hoax, 48, 191 Holmes, S., 66 holy book, 348 Holy Trinity, 110, 178, 348 homicide, 94 honor, 197, 363 human genome, 338, 340 Hume, D., 51, 99, 162–163 hypocrisy, 14, 43, 45, 47–48, 50, 179, 319 hypothesis alternative, 233, 301–302, 304 null, 70, 232–233, 299–304, 329, 386–387 hypothesis test, 50, 70–71, 127, 135, 138, 232–233, 239, 291–292, 299–304, 329, 335–336, 338–339, 378, 386–387 I i.i.d., 8, 14, 32, 49, 56, 60–62, 65–66, 69, 72, 74–77, 104, 111, 120, 135, 156, 219, 222, 224, 228, 234–235, 241, 256, 267–268, 273, 293, 295–298, 300, 310, 312, 314, 344, 379–380, 385–386, 388 ideology, 12, 32, 57, 109, 113, 123, 162, 179, 182, 185, 190–194, 197–198, 282, 311, 316, 333–337, 343–344, 375–376, 379, 381 imagination, 23, 48, 70–72, 84, 111, 113, 115, 165, 184, 289, 296, 350 Incas, 334 incommensurability, 48, 184 independence, 4, 8–9, 21–22, 32, 49, 57, 65–66, 69, 76, 85, 104, 117, 124, 135, 137, 153, 156, 194–195, 217, 219, 221, 226, 228, 231, 238–240, 242–243, 246–247, 256, 266–268, 273, 276–278, 286, 292–293, 295, 297–299, 301, 310, 312, 322, 325,

327, 345–346, 367–368, 370–371, 376, 378–379, 384–386 induction, viii, 3–4, 99, 143–144, 146–149, 159, 162–163, 167, 174, 186, 192, 195, 197, 200, 203, 208, 211, 213 inductivism, 155, 205, 212 infinity, 4, 24–25, 31, 34, 48, 53, 67, 104, 120–122, 133–134, 151, 156, 158, 172, 202, 208, 213, 229, 240–241, 257, 264, 269, 273, 276, 278, 289, 297, 304, 317, 320, 353–360, 363, 376, 384–386 information, 3–4, 10, 25–26, 31, 37, 46, 52–54, 65, 70, 80–81, 83, 96–97, 102–103, 108, 116, 119, 136, 140, 144, 146, 149–152, 156–157, 159, 161, 163–165, 169–170, 181–184, 186, 188, 190–191, 193–194, 202, 209, 220–223, 225–226, 230, 236, 242–245, 256, 260, 263, 266–267, 273, 281–283, 295, 303, 308–310, 314–317, 322, 324, 328–339, 341, 359, 384, 387–388 instinct, 33, 45, 86, 162–164, 180, 197, 200, 242, 270, 281, 283, 337, 372 instrumentalism, 206, 212 insurance, 50, 101, 234 intelligence, viii, 4, 127, 143, 166–171, 174, 188, 210 artificial, 148, 169–170, 208, 210 invariance, 219, 246 iterative method, 313, 321 seed of, 313 J Japan, 22 Jaynes, E., 125–127 Jeffreys, H., 12, 255, 317 Jesus, 333 jury, 132, 299–300, 329 justice system, 6, 43, 133

page 402

April 29, 2016

9:7

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-index

Index

Resonance Downloaded from www.worldscientific.com by 106.180.4.152 on 05/26/20. Re-use and distribution is strictly not permitted, except for Open Access articles.

K Keynes, J., 21 knowledge, vii, 1, 3–4, 6, 33, 36, 46–47, 62, 65, 68–69, 79–81, 99, 108, 114, 116, 118, 125–126, 136, 143–144, 146–147, 149–150, 156, 159–160, 167, 171, 179–180, 182, 184, 189–190, 197, 202, 204, 206–207, 209–213, 217, 220–221, 260–262, 266–267, 269, 292, 308–309, 312, 315–316, 327–328, 366, 373, 376, 379 base, 41, 57, 80, 209 personal, viii, 58, 79, 207, 308, 316 popular, 37 Kolmogorov, A., 21, 24, 35, 39, 54, 82, 127–128, 153, 221, 257–258, 264, 345, 376–377, 381–384 Kuhn, T., 1, 154, 168, 175, 185, 188–189, 196, 198–199, 207, 211, 342 Kuratowski–Zorn Lemma, 146 L Lakatos, I., 114, 175, 177, 196, 198, 205–207, 211, 227 Laplace, P.-S., 20–21 Large Deviations Principle, 231, 278–279, 293, 324, 385 laser, 74, 154 law, 107, 123, 131, 183, 206–207, 259, 367 Campbell’s, 361 Goodhart’s, 360 mathematical, 30, 345 of arithmetic, 364 of Iterated Logarithm, 78 of Large Numbers, 5, 24, 36, 60, 68–69, 74, 133, 238, 272–273, 277, 280, 293, 295–296, 344, 354–355, 375, 379, 385 of motion, 34, 51–52, 69, 81, 86, 108, 125, 229, 246

403

of nature, 4, 22, 46, 98–99, 145–149, 159, 162, 163, 167, 182, 199–200, 213, 266, 342, 372–373 of science, vii, 8–9, 35, 51, 67, 69, 84, 86, 94, 107–108, 114, 116, 123, 132, 155, 181–183, 187–188, 195, 198, 201, 203–204, 208–209, 212, 217–218, 220–223, 225–231, 233, 242–249, 251, 253, 258–261, 263–264, 266–269, 271, 278, 282–283, 311, 317, 343, 347, 349, 366, 373, 376 deterministic, 107, 204 of thermodynamics, 34 strong, 385 weak, 385 Leibnitz, G., 42, 266 Lenin, V., 126 Lewis, D., 128–129, 366 Liberia, 50 Lindley, D., 29–30, 57–58, 79, 114, 119–124, 194, 357, 361, 364 lion, 41, 118–119, 202, 243, 365 logic, 3–4, 21, 34, 42, 46, 105, 127, 139, 144, 146, 149, 157–158, 163, 166, 185, 191, 202, 209, 211, 224, 226–227, 317, 348–351, 357, 364–366, 372 deductive, 144, 213 inductive, 144, 148 Lord Kelvin (Thomson, W.), 136, 166 loss, 28, 82, 97, 99–100, 103, 110, 112–113, 116, 234, 252, 279, 285, 288, 292–294, 297–298, 325, 327, 329, 336, 338, 355, 357, 363 lottery, 83, 232, 239, 241, 252 love, 190, 363, 367 loyalty, 12, 106, 162, 190–191, 195, 197, 280 M Mach, E., 341

page 403

April 29, 2016

9:7

Resonance Downloaded from www.worldscientific.com by 106.180.4.152 on 05/26/20. Re-use and distribution is strictly not permitted, except for Open Access articles.

404

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-index

Index

machine, 83, 152, 160, 166, 169–170, 188, 301 magic, 52, 114, 314 Mars, 71, 246, 248–250 life on, 37–38, 225, 248–249, 263 Marsaglia, G., 67 Martin-L¨ of sequence, 77–78 martingale, 223 Marx, K., 36, 126, 334 Marxism, 177, 179, 334 mathematics, 15, 24, 28, 35–37, 67, 69–70, 86, 114, 122, 128–129, 136, 146, 154, 202, 227–228, 240, 257–258, 265, 313–314, 347–349, 356–358, 366–367, 372, 381, 383 Maxwell, J., 34, 222, 258 measurement, 6, 31, 51, 67, 69, 71, 84, 112, 116, 134, 136, 193, 223, 228–231, 233, 238, 245, 248, 254, 257–258, 260, 265, 288, 294, 296, 298, 303, 305, 325, 327, 387 medicine, 184, 193, 201, 337 alternative, 337 Chinese, 44 Western, 44, 47 memory, 61, 73, 116, 144–145, 159, 164–165, 169, 182, 189, 266, 316 Mexico, 22 microscope, 74, 342–343 tunneling, 343 Mill, J. S., 212 millionaire, 30 mind, 4, 46, 71, 94, 97, 106, 118, 145–146, 149, 151–152, 160, 167, 172–174, 182–183, 194, 197, 200–202, 208, 281–282, 334, 350–351 miracle, 105–106, 145, 179 mixture, 14, 120, 161, 220, 224, 256, 310, 314, 371, 386 model, 4, 25, 49, 54, 62, 70, 73, 119, 132, 135, 140, 151, 156, 167–168, 172, 174, 185, 193, 202, 209, 219, 222–224, 229, 231–232, 240, 243,

245–247, 257, 261–263, 268, 284–285, 287, 296, 299, 302–304, 309–311, 315–321, 323, 358–359, 370, 376, 378, 383, 388 Mona Lisa, 160 monad, 42 monarchy, 334, 336, 367 money, 25, 27, 30, 61, 81, 101–102, 168, 171, 184, 193, 197, 248–249, 284, 286, 289, 292, 297, 312, 325, 353–354, 360–362 Moon, 48, 55, 73, 159, 167, 194, 246, 361, 364–366 multiverse, 43, 262 N nation, 174, 189–190, 248 nature, 99, 148, 157, 179, 184, 191–192, 212, 242 neural network, 159, 170 New York, 382 Newton, I., 6, 34, 51–52, 55, 69, 81, 86, 107–108, 125, 148, 162, 167–168, 171, 183, 185, 208, 229, 246, 259, 266 Neyman–Pearson theory, 233 Nile, 45 Nobel prize, 168, 210 O omnibenevolence, 7, 110 omnipotence, 7, 110, 179, 334 P π, 365 p-value, 303 paleontology, 83 panaceum, 109 paradox, 6, 41, 110–111, 241, 353–355, 369–370 Ellsberg, 369, 372 Lewis’, 128–129 new prisoner, 367–368 prisoner, 367

page 404

April 29, 2016

9:7

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-index

Resonance Downloaded from www.worldscientific.com by 106.180.4.152 on 05/26/20. Re-use and distribution is strictly not permitted, except for Open Access articles.

Index St. Petersburg, 35, 280, 289, 353–355, 357, 368 parameter, 10, 62–65, 185, 196, 209, 224, 235, 247, 291–292, 294–295, 297–298, 303, 305, 310–311, 317, 321–322, 326 Pascal, B., 5, 20, 355–360, 363 patient, 31, 55, 68, 76, 95–96, 112, 139, 247–248, 250, 260, 295, 298, 302, 337, 381, 386 pattern, 23, 31–32, 66–68, 74–75, 83–84, 106–107, 115, 151–152, 164, 181–183, 200, 212, 218, 231, 265 recognition, 163–164, 181–183 pauper, 30, 285 Penrose, R., 267 philosophy, 1, 3–4, 7, 9, 12, 14, 25–26, 42–49, 51–52, 55, 58, 136, 143–144, 148, 155, 162, 174, 177–179, 181, 185, 187–190, 192, 197–199, 202–213, 218, 227, 230, 232, 240–243, 246, 248, 250, 252, 254–255, 258, 267–268, 271–272, 275, 277, 284, 287–288, 324–325, 335, 338, 340, 347–349, 355, 357, 359, 363–364, 366–367, 381 postmodern, 191–192 philosophy of probability, vii–viii, 1–2, 12, 14, 19, 24, 33–36, 38–39, 45, 56–57, 131–132, 211, 221, 228, 241, 243, 247, 260, 265, 267–268, 284, 308, 328, 333, 335, 343, 345, 349–350, 354, 362, 372 a priori, 21 classical, 9, 20, 33–34, 268, 346 frequency, vii, 2, 4, 9–10, 12–14, 31, 33, 35, 59, 133–135, 221, 241, 291, 300, 304, 333, 336, 344, 351, 375–376, 380 logical, 20, 33–34, 125, 219, 221 necessary, 21 personal, 23, 81 propensity, 22, 33 subjective, vii, 2, 4, 9–10, 12–14, 22, 33, 35, 57, 79,

405

133–135, 138, 140, 221, 232, 243–244, 270, 274, 277, 307–309, 315–316, 318–319, 322–324, 333, 336, 345, 375, 377, 380 physics, viii, 6, 9, 35, 56, 59, 83–84, 107, 114, 136, 145, 149, 152, 158, 162, 168, 185, 187, 193, 203, 210, 227–228, 258–261, 265, 267, 283, 325, 338, 341–342, 349–350, 365–366 quantum, 6, 51, 108, 118, 152, 158, 162, 171, 182, 187, 197, 199, 201, 223, 227, 230, 234, 255, 260–263, 266–267, 342–343, 365 statistical, 234, 255 Pitman, J., 380 planet, 3, 38, 50–51, 55, 92, 112, 118, 147, 148, 152, 188–189, 191, 246, 248, 249 plant, 150, 246 Poland, 177 Polanyi, M., viii, 58, 153–154, 175, 179, 188–189, 197–198, 200, 207, 211–212, 342 Popper, K., 1, 9, 22, 35, 47, 55, 84, 155, 175, 185, 189, 192, 195–198, 205, 207, 210, 228, 263–264, 268, 348–349 population, 10, 31–32, 55, 68, 74–75, 86, 118, 126, 153, 182, 248, 250, 252–253, 280, 313, 328, 334 posterior, 31, 89–91, 94–95, 99–102, 105, 112, 136, 139, 197, 209, 224, 233, 274, 307–309, 312, 314–316, 318–323, 327, 340, 376, 388 objective, 308 prediction(s), 4, 23, 25, 34, 45, 53, 60–62, 65, 67–69, 76, 82, 95, 99, 106, 115, 129, 144, 146–149, 172, 179, 182, 186–187, 194, 196, 206, 209, 212, 223, 225–227, 229–242, 249–252, 254–256, 259, 266–267, 269–270, 272, 278–280, 283, 288,

page 405

April 29, 2016

9:7

Resonance Downloaded from www.worldscientific.com by 106.180.4.152 on 05/26/20. Re-use and distribution is strictly not permitted, except for Open Access articles.

406

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-index

Index

292–294, 297, 299–304, 307, 312, 314–316, 318–319, 321–327, 329, 334, 340–341, 346–348, 368, 376, 388 aggregate, 239–242 amplification of, 238 contradictory, 236–237, 388 deterministic, 10, 52, 99, 134, 172, 229, 232, 238, 349 multiple, 238 numerical, 351 reduction of, 238 selection of, 238 spurious, 321 principle of indifference, 21, 268–269 prior, 10, 31, 53, 89, 95, 97, 99, 101–102, 105–106, 112, 119, 132, 135, 139–140, 197, 208–209, 224–225, 233, 247, 256, 274, 307–325, 328–329, 361, 378, 388 improper, 209 Jeffrey’s, 317 non-informative, 119, 140, 325 objective, 224, 298, 312, 316 subjective, 88, 119, 308, 310, 312, 315, 323 uniform, 21, 119, 221, 224, 314, 316, 322 probability, 2, 8, 19, 56, 59, 79, 125, 217, 343, 347–349, 372, 375, 379, 383 conditional, 31, 80, 128–129, 225, 389 epistemic, 260 finitely additive, 127, 257, 264 non-standard, 127, 257 objective, 259 physical, 260 subjective, 259 process Gaussian, 61, 155 Markov, 54, 61, 66, 95, 296, 380 martingale, 223 Poisson, 246–247, 376 stationary, 54, 61, 296

stochastic, 61, 65, 76, 240, 296, 297 proof, 51, 88, 108, 200, 203, 227–228, 241, 257–258, 265, 319, 323, 360, 379, 388 propaganda, 126, 178, 194, 323, 325 propensity, 263–264 prostitution, 47 Pyrrho of Elis, 42 Pythagorean Theorem, 365 R rabbit, 41–42, 203 radio, 158, 181, 222 Ramsey, F., 22, 108 random number generator, 66–67, 103, 171 random variable, 8, 21, 32, 49, 61, 76–77, 135, 218–219, 222, 234, 247, 253–254, 256, 273, 275, 277, 287, 292, 297, 302, 310, 314, 320, 326–328, 370, 379–380, 384–389 Reichenbach, H., 74 relativity, 116, 243, 256 relativity theory, 148, 152, 158, 162, 167, 182, 197, 199, 208, 244, 258, 342, 365 religion, 1, 7, 12, 45, 110, 152, 162, 177–180, 182, 186–188, 190–192, 194–195, 197, 199–202, 207, 242, 321, 333, 337, 342, 348–349, 359, 363, 372 resonance, viii, 1–4, 13, 58–59, 66, 79, 86, 127, 143, 147–152, 154–171, 174–175, 181, 185, 187–190, 195, 203, 209–210, 212–213, 219, 222, 224, 230, 233, 242, 245, 263, 280–281, 311, 315–317, 322, 378 complexity, 150 learning, 153–154 level, 155–156, 185, 224, 226, 231 physical, 157–158

page 406

April 29, 2016

9:7

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-index

Resonance Downloaded from www.worldscientific.com by 106.180.4.152 on 05/26/20. Re-use and distribution is strictly not permitted, except for Open Access articles.

Index revolution, 154, 334, 365 Chinese, 334 Russian, 334 scientific, 154, 197, 207 social, 335 risk, 29, 103, 110, 372 robot, 71, 81, 125–127, 160, 170–172, 174, 188–189, 210, 248 Roman Empire, 123 Ross, S., 381 Ruelle, D., 72, 218, 226 Russell, B., 136 Ryder, J., 88 S σ-field, 76, 389 saint, 179, 198–199 salvation, 360, 363 Savage, L., 27–28, 30 Schr¨ odinger, E., 223, 258, 267 scientific realism, 206 selection, 150, 153, 165, 190, 200, 242, 281, 372 Shakespeare, W., 146, 225, 263 side effect, 55, 222, 288, 302, 339 significance level, 119, 139, 232, 300–301, 303, 329, 378, 386–387 simulations, 25, 43, 63, 65, 72–73, 132, 135, 224, 226, 235, 339 skepticism, 1, 41–48, 50–51, 152, 182, 184, 198, 206, 211 slave, 47–48, 191 slogan, 23, 52, 113, 343–344 smoking, 105, 107, 118–119, 136, 139, 239, 252, 339 Soviet Union, 191 spaceship, 71, 112, 249 spam, 360–362 speed of light, 31, 71, 98, 112, 137, 253, 282, 294–295, 298, 327, 340 Spinoza, B., 179 Stalin, J., 126, 191 standard deviation, 138, 371, 384–387 statistics, 3, 9, 11, 33, 35, 38, 45, 66, 68, 110, 123, 125, 128, 138, 151,

407

181, 188, 196, 203–204, 208, 222, 224, 229, 245, 247, 250–252, 255, 257–259, 261, 265–266, 278, 288, 336–343, 347–348, 375, 383 Bayesian, vii, 2, 5, 8–10, 12–14, 30, 56, 84–85, 88–99, 113, 119, 138–140, 224–225, 232, 253, 260, 274–275, 307, 336, 339, 344, 378, 387 experimental, 337, 339–340 frequency, vii, 2, 9, 12, 14, 71, 222, 232–233, 253, 291, 338–339, 378, 386 stock market, 13, 29, 85, 96–98, 223, 260, 317 stopping time, 76 predictable, 76–77 straitjacket, 7, 106 string theory, 35, 148, 259, 341 student, 2, 35, 39, 60, 75, 110, 154, 155, 187–188, 203, 375–382 subconsciousness, 3, 79, 153, 154, 163–164, 168, 187, 188, 224, 239, 274, 280–281, 283, 344 sulfur, 59, 154, 276 Sun, 36, 43, 121, 134, 146–147, 191, 231 supernova, 154, 185, 221, 283, 338 survival, 150, 152, 160, 175, 180, 200, 202, 242, 281, 368 symmetry, viii, 7, 9, 20, 25, 33, 52–55, 65–68, 83, 85, 87, 117, 151, 153, 155–156, 163, 188, 211, 217, 219–222, 228, 231, 242–244, 246–247, 249–251, 253–256, 260–266, 268–270, 283, 310–311, 313, 317, 328, 345, 356–357, 369 T tautology, 75, 229, 274, 318, 329, 357, 365 Tegmark, M., 145, 149, 188, 192, 262–263 temperature, 56, 66, 72, 81, 83, 118, 134, 183, 195, 204, 218–219, 229,

page 407

April 29, 2016

9:7

Resonance Downloaded from www.worldscientific.com by 106.180.4.152 on 05/26/20. Re-use and distribution is strictly not permitted, except for Open Access articles.

408

Resonance: From Probability to Epistemology and Back

9in x 6in

b2422-index

Index

231, 246, 261, 282, 288, 315–316, 381, 383 textbook, vii, 3, 8–9, 39, 90, 155, 217, 222, 246, 313, 318, 323–324, 340, 343, 375–381, 383, 389 theology, 105, 126, 179, 194, 212 thermodynamics, 9, 34, 56, 187 thermometer, 81, 134, 183 tiger, 60 Tillich, P., 179 Truman, H., 126 truth, 1, 3, 35, 41, 43, 107, 121, 144, 162, 183, 185, 189, 191–192, 198, 201–202, 207–208, 211, 213, 224, 283, 320, 342, 348, 360, 364–365, 386 tuberculosis, 53 Tulcea’s theorem, 95–96 tunneling, 234 Turing test, 169–170 Turing, A., 170 TV, 161, 382 U uncertainty principle, 81, 227 United States, 177, 202 universe, 1, 4, 42–43, 46, 87, 92, 99, 116, 126, 137, 140, 144–149, 151–152, 158, 161, 171–172, 179, 187–188, 194, 199, 210–211, 228, 239, 242, 249, 262–264, 266, 281, 308–309, 339, 342, 364–365, 369 university, 47–48, 153, 189, 191, 197 urn, 367 utility, 30, 112–113, 122, 232, 272, 277, 279, 282, 284–289, 292, 297, 322, 329, 356–360, 362–363, 368–372 function, 30, 271, 277, 281, 284–289, 329, 357, 362, 371–372

V variance, 384 Venn, J., 31 Venus, 63, 248–249 von Mises, R., 2, 4–8, 10, 13–14, 31–35, 38–39, 45–50, 54–56, 60–62, 65, 67, 69, 73–77, 106, 132–137, 153, 175, 198, 203–204, 229, 247, 253, 268, 291, 296, 299–300, 305, 335–336, 343–344, 346–347, 349, 354, 376–377, 380–381 von Neumann–Morgenstern theory, 30 voodoo, 45, 177–178 W war, 12, 50, 85, 147, 151, 172, 177, 194, 201, 334, 337, 360 water, 72, 150, 183, 187, 195, 201, 203, 220, 229, 238, 283, 372 wave electromagnetic, 98, 149, 158 function, 227, 262 weather, 31, 85, 147, 149, 201, 237, 260 well-ordering theorem, 146 white noise, 137 Wikipedia, 37–38, 93, 353, 369–370 witch, 162 work, 6 Z zebra, 118–119, 185, 202, 243, 246, 259, 365 Zeno of Elea, 42, 45–47, 79 zero, 230–231, 240, 244, 278, 302, 356–360, 362–364

page 408