1,319 249 11MB
English Pages xvii, 464 Seiten: Illustrationen, Diagramme [484] Year 2016
Statistical Rethinking
A Bayesian Course with Examples in R and Stan
CHAPMAN & HALL/CRC Texts in Statistical Science Series Series Editors Francesca Dominici, Harvard School of Public Health, USA Julian J. Faraway, University of Bath, UK Martin Tanner, Northwestern University, USA Jim Zidek, University of British Columbia, Canada Statistical Theory: A Concise Introduction F. Abramovich and Y. Ritov
Practical Multivariate Analysis, Fifth Edition A. Afifi, S. May, and V.A. Clark Practical Statistics for Medical Research D.G. Altman Interpreting Data: A First Course in Statistics A.J.B. Anderson
Introduction to Probability with R K. Baclawski
Linear Algebra and Matrix Analysis for Statistics S. Banerjee and A. Roy
Mathematical Statistics: Basic Ideas and Selected Topics, Volume I, Second Edition P. J. Bickel and K. A. Doksum Mathematical Statistics: Basic Ideas and Selected Topics, Volume II P. J. Bickel and K. A. Doksum Analysis of Categorical Data with R C. R. Bilder and T. M. Loughin
Statistical Methods for SPC and TQM D. Bissell Introduction to Probability J. K. Blitzstein and J. Hwang
Bayesian Methods for Data Analysis, Third Edition B.P. Carlin and T.A. Louis Second Edition R. Caulcutt
The Analysis of Time Series: An Introduction, Sixth Edition C. Chatfield Introduction to Multivariate Analysis C. Chatfield and A.J. Collins
Problem Solving: A Statistician’s Guide, Second Edition C. Chatfield
Statistics for Technology: A Course in Applied Statistics, Third Edition C. Chatfield Bayesian Ideas and Data Analysis: An Introduction for Scientists and Statisticians R. Christensen, W. Johnson, A. Branscum, and T.E. Hanson Modelling Binary Data, Second Edition D. Collett
Modelling Survival Data in Medical Research, Third Edition D. Collett Introduction to Statistical Methods for Clinical Trials T.D. Cook and D.L. DeMets
Applied Statistics: Principles and Examples D.R. Cox and E.J. Snell
Multivariate Survival Analysis and Competing Risks M. Crowder Statistical Analysis of Reliability Data M.J. Crowder, A.C. Kimber, T.J. Sweeting, and R.L. Smith An Introduction to Generalized Linear Models, Third Edition A.J. Dobson and A.G. Barnett
Nonlinear Time Series: Theory, Methods, and Applications with R Examples R. Douc, E. Moulines, and D.S. Stoffer Introduction to Optimization Methods and Their Applications in Statistics B.S. Everitt Extending the Linear Model with R: Generalized Linear, Mixed Effects and Nonparametric Regression Models J.J. Faraway
Linear Models with R, Second Edition J.J. Faraway A Course in Large Sample Theory T.S. Ferguson
Multivariate Statistics: A Practical Approach B. Flury and H. Riedwyl Readings in Decision Analysis S. French
Markov Chain Monte Carlo: Stochastic Simulation for Bayesian Inference, Second Edition D. Gamerman and H.F. Lopes
Bayesian Data Analysis, Third Edition A. Gelman, J.B. Carlin, H.S. Stern, D.B. Dunson, A. Vehtari, and D.B. Rubin Multivariate Analysis of Variance and Repeated Measures: A Practical Approach for Behavioural Scientists D.J. Hand and C.C. Taylor Practical Longitudinal Data Analysis D.J. Hand and M. Crowder Logistic Regression Models J.M. Hilbe
Richly Parameterized Linear Models: Additive, Time Series, and Spatial Models Using Random Effects J.S. Hodges Statistics for Epidemiology N.P. Jewell
Stochastic Processes: An Introduction, Second Edition P.W. Jones and P. Smith The Theory of Linear Models B. Jørgensen Principles of Uncertainty J.B. Kadane
Graphics for Statistics and Data Analysis with R K.J. Keen Mathematical Statistics K. Knight
Introduction to Multivariate Analysis: Linear and Nonlinear Modeling S. Konishi
Nonparametric Methods in Statistics with SAS Applications O. Korosteleva Modeling and Analysis of Stochastic Systems, Second Edition V.G. Kulkarni
Exercises and Solutions in Biostatistical Theory L.L. Kupper, B.H. Neelon, and S.M. O’Brien
Exercises and Solutions in Statistical Theory L.L. Kupper, B.H. Neelon, and S.M. O’Brien Design and Analysis of Experiments with R J. Lawson
Design and Analysis of Experiments with SAS J. Lawson A Course in Categorical Data Analysis T. Leonard Statistics for Accountants S. Letchford
Introduction to the Theory of Statistical Inference H. Liero and S. Zwanzig Statistical Theory, Fourth Edition B.W. Lindgren
Stationary Stochastic Processes: Theory and Applications G. Lindgren Statistics for Finance E. Lindström, H. Madsen, and J. N. Nielsen
The BUGS Book: A Practical Introduction to Bayesian Analysis D. Lunn, C. Jackson, N. Best, A. Thomas, and D. Spiegelhalter Introduction to General and Generalized Linear Models H. Madsen and P. Thyregod Time Series Analysis H. Madsen Pólya Urn Models H. Mahmoud
Randomization, Bootstrap and Monte Carlo Methods in Biology, Third Edition B.F.J. Manly Introduction to Randomized Controlled Clinical Trials, Second Edition J.N.S. Matthews
Statistical Rethinking: A Bayesian Course with Examples in R and Stan R. McElreath Statistical Methods in Agriculture and Experimental Biology, Second Edition R. Mead, R.N. Curnow, and A.M. Hasted
Statistics in Engineering: A Practical Approach A.V. Metcalfe Statistical Inference: An Integrated Approach, Second Edition H. S. Migon, D. Gamerman, and F. Louzada
Beyond ANOVA: Basics of Applied Statistics R.G. Miller, Jr.
Decision Analysis: A Bayesian Approach J.Q. Smith
Applied Stochastic Modelling, Second Edition B.J.T. Morgan
Applied Statistics: Handbook of GENSTAT Analyses E.J. Snell and H. Simpson
A Primer on Linear Models J.F. Monahan
Elements of Simulation B.J.T. Morgan
Analysis of Failure and Survival Data P. J. Smith
Probability: Methods and Measurement A. O’Hagan
Applied Nonparametric Statistical Methods, Fourth Edition P. Sprent and N.C. Smeeton
Applied Bayesian Forecasting and Time Series Analysis A. Pole, M. West, and J. Harrison
Generalized Linear Mixed Models: Modern Concepts, Methods and Applications W. W. Stroup
Introduction to Statistical Limit Theory A.M. Polansky
Data Driven Statistical Methods P. Sprent
Statistics in Research and Development, Time Series: Modeling, Computation, and Inference R. Prado and M. West
Survival Analysis Using S: Analysis of Time-to-Event Data M. Tableman and J.S. Kim
Introduction to Statistical Process Control P. Qiu
Sampling Methodologies with Applications P.S.R.S. Rao A First Course in Linear Model Theory N. Ravishanker and D.K. Dey Essential Statistics, Fourth Edition D.A.G. Rees
Stochastic Modeling and Mathematical Statistics: A Text for Statisticians and Quantitative Scientists F.J. Samaniego
Statistical Methods for Spatial Data Analysis O. Schabenberger and C.A. Gotway Bayesian Networks: With Examples in R M. Scutari and J.-B. Denis Large Sample Methods in Statistics P.K. Sen and J. da Motta Singer
Spatio-Temporal Methods in Environmental Epidemiology G. Shaddick and J.V. Zidek
Applied Categorical and Count Data Analysis W. Tang, H. He, and X.M. Tu
Elementary Applications of Probability Theory, Second Edition H.C. Tuckwell Introduction to Statistical Inference and Its Applications with R M.W. Trosset
Understanding Advanced Statistical Methods P.H. Westfall and K.S.S. Henning Statistical Process Control: Theory and Practice, Third Edition G.B. Wetherill and D.W. Brown Generalized Additive Models: An Introduction with R S. Wood
Epidemiology: Study Design and Data Analysis, Third Edition M. Woodward
Practical Data Analysis for Designed Experiments B.S. Yandell
Texts in Statistical Science
Statistical Rethinking
A Bayesian Course with Examples in R and Stan
Richard McElreath Max Planck Institute for Evolutionary Anthropology Leipzig, Germany
CRC Press Taylor & Francis Group 6000 Broken Sound Parkway NW, Suite 300 Boca Raton, FL 33487-2742 © 2016 by Taylor & Francis Group, LLC CRC Press is an imprint of Taylor & Francis Group, an Informa business No claim to original U.S. Government works Version Date: 20150910 International Standard Book Number-13: 978-1-4822-5346-7 (eBook - PDF) This book contains information obtained from authentic and highly regarded sources. Reasonable efforts have been made to publish reliable data and information, but the author and publisher cannot assume responsibility for the validity of all materials or the consequences of their use. The authors and publishers have attempted to trace the copyright holders of all material reproduced in this publication and apologize to copyright holders if permission to publish in this form has not been obtained. If any copyright material has not been acknowledged please write and let us know so we may rectify in any future reprint. Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmitted, or utilized in any form by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying, microfilming, and recording, or in any information storage or retrieval system, without written permission from the publishers. For permission to photocopy or use material electronically from this work, please access www.copyright.com (http:// www.copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC), 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400. CCC is a not-for-profit organization that provides licenses and registration for a variety of users. For organizations that have been granted a photocopy license by the CCC, a separate system of payment has been arranged. Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation without intent to infringe. Visit the Taylor & Francis Web site at http://www.taylorandfrancis.com and the CRC Press Web site at http://www.crcpress.com
Contents
Preface Audience Teaching strategy How to use this book Installing the rethinking R package Acknowledgments
xi xi xii xii xvi xvi
Chapter 1. The Golem of Prague 1.1. Statistical golems 1.2. Statistical rethinking 1.3. Three tools for golem engineering 1.4. Summary
1 1 4 10 16
Chapter 2. Small Worlds and Large Worlds 2.1. The garden of forking data 2.2. Building a model 2.3. Components of the model 2.4. Making the model go 2.5. Summary 2.6. Practice
19 20 28 32 37 45 45
Chapter 3. Sampling the Imaginary 3.1. Sampling from a grid-approximate posterior 3.2. Sampling to summarize 3.3. Sampling to simulate prediction 3.4. Summary 3.5. Practice
49 52 53 61 68 69
Chapter 4. Linear Models 4.1. Why normal distributions are normal 4.2. A language for describing models 4.3. A Gaussian model of height 4.4. Adding a predictor 4.5. Polynomial regression 4.6. Summary 4.7. Practice
71 72 77 78 92 110 115 115
Chapter 5. Multivariate Linear Models 5.1. Spurious association 5.2. Masked relationship 5.3. When adding variables hurts
119 121 135 141 vii
viii
CONTENTS
5.4. 5.5. 5.6. 5.7.
Categorical variables Ordinary least squares and lm Summary Practice
152 159 162 162
Chapter 6. Overfitting, Regularization, and Information Criteria 6.1. The problem with parameters 6.2. Information theory and model performance 6.3. Regularization 6.4. Information criteria 6.5. Using information criteria 6.6. Summary 6.7. Practice
165 167 174 186 188 195 205 205
Chapter 7. Interactions 7.1. Building an interaction 7.2. Symmetry of the linear interaction 7.3. Continuous interactions 7.4. Interactions in design formulas 7.5. Summary 7.6. Practice
209 211 223 225 235 236 236
Chapter 8. Markov Chain Monte Carlo 8.1. Good King Markov and His island kingdom 8.2. Markov chain Monte Carlo 8.3. Easy HMC: map2stan 8.4. Care and feeding of your Markov chain 8.5. Summary 8.6. Practice
241 242 245 247 255 263 263
Chapter 9. Big Entropy and the Generalized Linear Model 9.1. Maximum entropy 9.2. Generalized linear models 9.3. Maximum entropy priors 9.4. Summary
267 268 280 288 289
Chapter 10. Counting and Classification 10.1. Binomial regression 10.2. Poisson regression 10.3. Other count regressions 10.4. Summary 10.5. Practice
291 292 311 322 328 329
Chapter 11. Monsters and Mixtures 11.1. Ordered categorical outcomes 11.2. Zero-inflated outcomes 11.3. Over-dispersed outcomes 11.4. Summary 11.5. Practice
331 331 342 346 351 352
Chapter 12. Multilevel Models 12.1. Example: Multilevel tadpoles 12.2. Varying effects and the underfitting/overfitting trade-off
355 357 364
CONTENTS
12.3. 12.4. 12.5. 12.6.
More than one type of cluster Multilevel posterior predictions Summary Practice
ix
370 376 384 384
Chapter 13. Adventures in Covariance 13.1. Varying slopes by construction 13.2. Example: Admission decisions and gender 13.3. Example: Cross-classified chimpanzees with varying slopes 13.4. Continuous categories and the Gaussian process 13.5. Summary 13.6. Practice
387 389 398 403 410 419 419
Chapter 14. Missing Data and Other Opportunities 14.1. Measurement error 14.2. Missing data 14.3. Summary 14.4. Practice
423 424 431 439 439
Chapter 15.
441
Horoscopes
Endnotes
445
Bibliography
457
Citation index
465
Topic index
467
This page intentionally left blank
Preface
Masons, when they start upon a building, Are careful to test out the scaffolding; Make sure that planks won’t slip at busy points, Secure all ladders, tighten bolted joints. And yet all this comes down when the job’s done Showing off walls of sure and solid stone. So if, my dear, there sometimes seem to be Old bridges breaking between you and me Never fear. We may let the scaffolds fall Confident that we have built our wall. (“Scaffolding” by Seamus Heaney, 1939–2013) This book means to help you raise your knowledge of and confidence in statistical modeling. It is meant as a scaffold, one that will allow you to construct the wall that you need, even though you will discard it afterwards. As a result, this book teaches the material in often inconvenient fashion, forcing you to perform step-by-step calculations that are usually automated. The reason for all the algorithmic fuss is to ensure that you understand enough of the details to make reasonable choices and interpretations in your own modeling work. So although you will move on to use more automation, it’s important to take things slow at first. Put up your wall, and then let the scaffolding fall.
Audience The principle audience is researchers in the natural and social sciences, whether new PhD students or seasoned professionals, who have had a basic course on regression but nevertheless remain uneasy about statistical modeling. This audience accepts that there is something vaguely wrong about typical statistical practice in the early 21st century, dominated as it is by p-values and a confusing menagerie of testing procedures. They see alternative methods in journals and books. But these people are not sure where to go to learn about these methods. As a consequence, this book doesn’t really argue against p-values and the like. The problem in my opinion isn’t so much p-values as the set of odd rituals that have evolved around xi
xii
PREFACE
them, in the wilds of the sciences, as well as the exclusion of so many other useful tools. So the book assumes the reader is ready to try doing statistical inference without p-values. This isn’t the ideal situation. It would be better to have material that helps you spot common mistakes and misunderstandings of p-values and tests in general, as all of us have to understand such things, even if we don’t use them. So I’ve tried to sneak in a little material of that kind, but unfortunately cannot devote much space to it. The book would be too long, and it would disrupt the teaching flow of the material. It’s important to realize, however, that the disregard paid to p-values is not a uniquely Bayesian attitude. Indeed, significance testing can be—and has been—formulated as a Bayesian procedure as well. So the choice to avoid significance testing is stimulated instead by epistemological concerns, some of which are briefly discussed in the first chapter.
Teaching strategy The book uses much more computer code than formal mathematics. Even excellent mathematicians can have trouble understanding an approach, until they see a working algorithm. This is because implementation in code form removes all ambiguities. So material of this sort is easier to learn, if you also learn how to implement it. In addition to any pedagogical value of presenting code, so much of statistics is now computational that a purely mathematical approach is anyways insufficient. As you’ll see in later parts of this book, the same mathematical statistical model can sometimes be implemented in different ways, and the differences matter. So when you move beyond this book to more advanced or specialized statistical modeling, the computational emphasis here will help you recognize and cope with all manner of practical troubles. Every section of the book is really just the tip of an iceberg. I’ve made no attempt to be exhaustive. Rather I’ve tried to explain something well. In this attempt, I’ve woven a lot of concepts and material into data analysis examples. So instead of having traditional units on, for example, centering predictor variables, I’ve developed those concepts in the context of a narrative about data analysis. This is certainly not a style that works for all readers. But it has worked for a lot of my students. I suspect it fails dramatically for those who are being forced to learn this information. For the internally motivated, it reflects how we really learn these skills in the context of our research.
How to use this book This book is not a reference, but a course. It doesn’t try to support random access. Rather, it expects sequential access. This has immense pedagogical advantages, but it has the disadvantage of violating how most scientists actually read books. This book has a lot of code in it, integrated fully into the main text. The reason for this is that doing model-based statistics in the 21st century really requires programming, of at least a minor sort. The code is not optional. Everyplace, I have erred on the side of including too much code, rather than too little. In my experience teaching scientific programming, novices learn more quickly when they have working code to modify, rather than needing to write an algorithm from scratch. My generation was probably the last to have to learn some programming to use a computer, and so coding has gotten harder and harder to teach as time goes on. My students are very computer literate, but they have no idea what computer code looks like.
HOW TO USE THIS BOOK
xiii
What the book assumes. This book does not try to teach the reader to program, in the most basic sense. It assumes that you have made a basic effort to learn how to install and process data in R. In most cases, a short introduction to R programming will be enough. I know many people have found Emmanuel Paradis’ R for Beginners helpful. You can find it and many other beginner guides here: http://cran.r-project.org/other-docs.html To make use of this book, you should know already that y