From GDP to Sustainable Wellbeing: Changing Statistics or Changing Lives? [1st ed.] 9783030530846, 9783030530853

Table of contents :
Front Matter ....Pages i-xix
Setting the Scene (Paul Allin, David J. Hand)....Pages 1-23
Using Statistics to Assess Progress (Paul Allin, David J. Hand)....Pages 25-48
Statistics and Public Policy (Paul Allin, David J. Hand)....Pages 49-81
Wider Audiences for New Measures of Progress (Paul Allin, David J. Hand)....Pages 83-109
Inputs and Outputs: Data Science and the Role of Media (Paul Allin, David J. Hand)....Pages 111-126
Conclusion (Paul Allin, David J. Hand)....Pages 127-140
Back Matter ....Pages 141-144

Citation preview

WELLBEING IN POLITICS AND POLICY

From GDP to Sustainable Wellbeing Changing Statistics or Changing Lives? Paul Allin · David J. Hand

Wellbeing in Politics and Policy

Series Editors Ian Bache University of Sheffield Sheffield, UK Karen Scott Exeter University (Cornwall Campus) Penryn, Cornwall, UK Paul Allin Imperial College London London, UK

Wellbeing in Politics and Policy will bring new lenses through which to understand the significance of the dramatic rise of interest in wellbeing as a goal of public policy. While a number of academic disciplines have been influential in both shaping and seeking to explain developments, the Politics discipline has been relatively silent, leaving important theoretical and empirical insights largely absent from debates: insights that have increasing significance as political interest grows. This series will provide a distinctive addition to the field that puts politics and policy at the centre, while embracing interdisciplinary contributions. Contributions will be encouraged from various subfields of the discipline (e.g., political theory, comparative politics, governance and public policy, international relations) and from those located in other disciplines that speak to core political themes (e.g., accountability, gender, inequality, legitimacy and power). The series will seek to explore these themes through policy studies in a range of settings – international, national and local. Comparative studies – either of different policy areas and/or across different settings – will be particularly encouraged. The series will incorporate a wide range of perspectives from critical to problem-solving approaches, drawing on a variety of epistemologies and methodologies. The series welcomes Pivots, edited collections and monographs.

More information about this series at http://www.palgrave.com/gp/series/15247

Paul Allin · David J. Hand

From GDP to Sustainable Wellbeing Changing Statistics or Changing Lives?

Paul Allin Department of Mathematics Imperial College London London, UK

David J. Hand Department of Mathematics Imperial College London London, UK

Wellbeing in Politics and Policy ISBN 978-3-030-53084-6 ISBN 978-3-030-53085-3 (eBook) https://doi.org/10.1007/978-3-030-53085-3 © The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 This work is subject to copyright. All rights are solely and exclusively licensed by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. This Palgrave Pivot imprint is published by the registered company Springer Nature Switzerland AG The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland

Preface

As you have opened this book, you may be as fascinated as we are by statistics. We are especially interested in how official statistics feature in politics, policy, business decision-making, and in everyday conversations. But whether or not you are already hooked on statistics, we hope you will continue reading because what we want to explore, we believe, affects all of us. Official statistics are those published by governments, usually by their national statistics office. They are intended to be used not only within government but also by businesses, non-governmental organisations such as charities and trade unions, academics, the media, and the public. This puts official statistics in possibly a unique position, certainly different from that of the media. Supreme Court Justice Hugo Black set down a legal interpretation of the role of the media when he declared that “the press was to serve the governed, not the governors”, in a case between the New York Times Company and the United States Government (Legal Information Institute, 1971). Official statistics are intended to serve everyone, both the governed and the government, providing a trusted set of information that all parties and sections of society can use. If we could achieve such a set, this should not of course prevent discussion, debate, and argument about what to do in light of the figures. However, it should at least reduce disagreements on the validity of the numbers themselves. That has been a long-held ambition for official statistics.

v

vi

PREFACE

In the real world, the position may be far from as simple as what we have just suggested. For a start, just the huge volume of official statistics—currently running at some 3570 releases of official statistics a year in the UK—means that any broad-brush picture of official statistics needs to recognise that this is composed of many fine details. What is important, for example, in the measurement of inflation is likely to be different from what is important in describing the ethnic composition of the country. Each topic needs particular scrutiny and understanding. Nevertheless, there are high-level issues that apply to all sets of official statistics, such as how their quality is defined and measured, and how their usefulness is determined, and it will be these kinds of issues that we look at here. We will look at these fundamental issues in terms of the measurement of the wellbeing of a country, by which we mean its current levels and distribution of wellbeing, along with its development and progress, and whether current progress is sustainable into the future. This is all sometimes called beyond GDP. In one sense, the hero of the story is GDP, gross domestic product, the official statistic that stands out as the bottom line in the national accounts that are compiled and published regularly (in some cases, monthly). GDP represents the total value of economic activity in the country in the given time period. GDP is treated heroically because it, and particularly whether it is rising or falling over time, is often taken as the only measure of how the country is doing. This also points to why GDP is also a villain in some people’s eyes: it was not designed to tell us about the state of the nation in any broader sense, and there are many ways in which it is recognised as falling short of that. We will look at the desire to go beyond GDP, with new measures and statistics about the wellbeing of the nation, rather than just how the economy is doing overall. Sustainable development is another phrase that is used to describe aspects of the progress of a country, particularly in political and policy circles. There is much innovation and entrepreneurship aimed at delivering sustainable development, whether in new sources of energy, new forms of transport, or in many, smaller examples of re-engineering existing products and services. There also appears to be greater awareness of sustainability, such as by associating an organisation’s goals and vision with that of the United Nation’s Sustainable Development Goals, as we will discuss in Chapters 3 and 4.

PREFACE

vii

However, it will only be members of future generations who can decide whether or not our development was sustainable. We have found a proliferation of measures and indicators aimed at planning sustainable development and tracking progress. We are less clear about how these are used to build public policy. Moreover, policy is not the only way of affecting behaviour. We suggest that there is still a need to hold conversations about how we make commercial, social, and personal decisions, informed by these wider measures and indicators, rather than the default of only looking for economic growth as the goal. In this book we look at the challenges faced in designing relevant measures, be they of national wellbeing, sustainable development or any other of the myriad aspects of the economy, society, and the environment. More importantly than that, we are also concerned with how to ensure that these measures and indicators are used. We look for lessons from the field of poverty reduction and social protection across Europe, where the effort in compiling indicators has not necessarily been rewarded with progress towards meeting poverty and social exclusion goals. While they may be good statistics in a technical sense, they might not always be as useful—or even as much used—as they could have been. We draw from this example, which is in an established social policy area, some pointers for how official statistics might be used more effectively in what is still an emerging policy area, that of moving beyond GDP, with a new emphasis on wellbeing and sustainable development. Official statisticians appear often to focus on a mission to deliver a particular set of statistics, invariably chosen to meet the needs of government as their main user. This falls short of the full potential of official statistics, to help understand society, the economy, and the environment, to assist in formulating where and what needs to change, and to be important tools in effecting such changes. We stress the teleology of official statistics: what purposes do they serve, rather than just compiling and publishing them as part of the democratic process? The outline of the book is as follows. In Chapter 1 we will discuss in detail four points of departure for the rest of the book that we have already mentioned: the notion of social progress; official statistics; the concept and measurement of a nation’s GDP; and the beyond GDP agenda, which among other things requires the definition and measurement of current wellbeing together with an assessment of the sustainability of current activities for the wellbeing of future generations. Chapter 2 discusses using statistics to assess progress. The key issues are how the statistics are to be

viii

PREFACE

used, and who the intended users are. In Chapter 3 we suggest that public policy creates the framework for going beyond GDP. We explore the role of public policy in effecting change and how official statistics are used in this. Having a new measurement policy is fine, but it is not enough even when it is intertwined with the development of wellbeing policy. So, in Chapter 4 we explore how to effect social change that improves wellbeing and puts us on the track of sustainable development. This is where new measures of progress can help. Chapter 5 recognises that official statistics are delivered through a process that turns data into messages. We look at inputs and outputs of this process, especially the increasing use of data science techniques applied to new sources of big data, and the role of media in bringing statistics into the public arena. Our conclusions in Chapter 6 include a concrete list of recommendations. We have determined these by adopting a teleological approach, asking what are the intended uses of official statistics and how are they used, or not used. Facts are all very well, but it is perhaps even more important to ensure that measures and indicators are actually used. We invite people to either adhere to the recommendations or to say why they are not adopting them, rather than simply saying “that was an interesting perspective”! At the time of writing we are in the throes of the COVID-19 pandemic, and it remains to be seen what long-lasting impact this will have on the global economy and on society. Whatever that impact, the matters discussed in this book will still be of critical importance to global wellbeing and sustainability. In a letter to the UK chancellor, Greenpeace UK (2020) and 25 other civil society groups emphasised that “Public money must be used to address social and environmental priorities, as well as economic needs”. Many conversations about the nature of the recovery from this pandemic will need to take place, conversations that should be informed as far as possible by relevant, timely, and trusted official statistics. Newport, UK London, UK June 2020

Paul Allin David J. Hand

PREFACE

ix

References Greenpeace UK (2020) Open letter to the chancellor on a UK government support package for the aviation industry. https://www.greenpeace.org.uk/ news/the-airlines-industry-wants-a-government-bailout-heres-what-needs-tohappen/. Accessed 11 April 2020 Legal Information Institute (1971) 403 U.S. 713. https://www.law.cornell.edu/ supremecourt/text/403/713. Accessed 19 May 2020

Acknowledgements

This book draws on many stimulating discussions, including at meetings of the Royal Statistical Society and its International Development Section, at conferences in Rome and Cambridge organised by the World Scientific and Engineering Academy and Society, and at an interdisciplinary workshop on the measurement of wellbeing at the Lee Kum Sheung Center for Health and Happiness at the Harvard TH Chan School of Public Health. We gratefully acknowledge the ground-breaking work of two UK organisations, the Office for National Statistics (ONS), especially the extensive material that they have published during the preparation of measures of national wellbeing and of sustainable development, and the What Works Centre for Wellbeing, an independent collaborative centre that is putting high quality evidence on wellbeing into the hands of decision-makers in government, communities, businesses, and other organisations. Although we have worked in or with the ONS and the What Works Centre for Wellbeing, views expressed in this book are the personal views of the authors and do not necessarily represent those of the ONS, the UK Statistics Authority, the What Works Centre for Wellbeing, or others. As always, we have learned much from presentations by, and exchanges and conversations with, many people. Particular thanks go to Fiona Dawe, Dawn Snape, and Claudia Wells at the ONS, Nancy Hey at WWCW, and Deborah Ashby, Eric Bischof, Woody Caan, Phil Crook, Ian Diamond, Myer Glickman, Harvey Goldstein, Jon Hall, John Helliwell, Ed Humpherson, Austin Kennedy, Tom King, Eileen McNeely,

xi

xii

ACKNOWLEDGEMENTS

Ian Plewis, John Pullinger, Sara Selwood, Joss Tantram, Katherine Trebeck, Fiona Underwood, Tyler J VanderWeele, Howard White. We thank the series co-editors Ian Bache and Karen Scott for encouragement, advice, and feedback and we gratefully acknowledge all the editorial and production work at Palgrave, led by Nick Barclay and Balaji Varadharaju. We dedicate this book to Karen and to Shelley as a token of our appreciation for their support and inspiration. Newport, UK London, UK June 2020

Paul Allin David J. Hand

Praise for From GDP to Sustainable Wellbeing

“This is not just another book about Statistics. This remarkable book shows the importance of having reliable and trustworthy statistics, if used and useful, for improving our collective wellbeing. For that purpose, the authors share with us a deep analysis about how to capture the complexity of the world we live in and proposals for having more relevant statistics, helping policy makers, organizations or citizens to make well informed decisions on behalf of a better society.” —Maria João Valente Rosa, Faculty of Social and Human Sciences at Universidade Nova de Lisboa and member of the European Statistical Advisory Committee (ESAC) “Over the last decade there has been growing interest in moving beyond GDP as a measure of society’s welfare. Allin and Hand have made an important contribution to this debate, charting a course for public statistics which would make them better understood and trusted. Their book should be essential reading for statisticians and policy makers alike.” —Nick Macpherson, economist and Permanent Secretary of HM Treasury, 2005–2016 “I liked this: ‘Official statistics are intended to serve everyone, both the governed and the government, providing a trusted set of information that all parties and sections of society can use.’ And this is at the heart of wellbeing measurement and policy. This is information about ‘how xiii

xiv

PRAISE FOR FROM GDP TO SUSTAINABLE WELLBEING

we are doing’ and of social welfare. It’s the shared evidence base of all departments, all public sector bodies, all electoral candidates, all sectors, all communities have to make their choices from, about where we’re going and what we are doing to get there. The book is an essential part of the policy making curriculum.” —Nancy Hey, Executive Director, What Works Centre for Wellbeing, UK “Governments need information on which to develop and evidence policy decisions; and in many countries National Statistical Institutes (NSI) are charged with producing much of that information. Society, and the aspirations of citizens, change over time and so it is essential that NSIs respond by being in a permanent state of change in the production of statistics. Even so, too often the discourse between governments and NSIs is linear rather than interactive and so the impact of statistics on policy is not as great as it should be. This excellent book brings clarity to the need for NSIs to be responsive both to changes in society and to the opportunities that improved technology brings; and reflects impressively on strategies to maximise the impact of statistics on policy.” —Ian Diamond, UK National Statistician

Contents

1

Setting the Scene 1.1 What Is Social Progress? 1.2 What Are Society’s Facts? 1.3 What Are Official Statistics? 1.4 What Is GDP and What Does It Seek to Measure? 1.5 What Lies Beyond GDP? References

1 1 3 9 13 19 21

2

Using Statistics to Assess Progress 2.1 Introduction 2.2 The Role of Official Statistics 2.3 The Stiglitz, Sen, Fitoussi Report 2.4 The Sustainable Development Goals 2.5 Putting Official Statistics to Use 2.6 Conclusion References

25 26 28 35 38 41 45 46

3

Statistics and Public Policy 3.1 Introduction 3.2 Policy-Making in General 3.3 The Use of Evidence in Policy 3.4 Wellbeing Policy and Measures

49 50 53 57 64

xv

xvi

CONTENTS

3.5

Conclusion: Making Better Use of Statistics Around Wellbeing Policy References 4

5

6

Wider Audiences for New Measures of Progress 4.1 Introduction 4.2 The Public Value of Official Statistics 4.3 Can Official Statistics Change Anything? 4.4 The Role of Influencers and Intermediaries 4.5 Trust in Official Statistics 4.6 Conclusion: Better Engagement Between Producers and Users of Official Statistics References

74 77 83 83 85 93 99 102 104 106

Inputs and Outputs: Data Science and the Role of Media 5.1 Introduction 5.2 How Can Data Science Help? 5.3 The Role of the Media 5.4 Statistical Literacy References

111 111 114 121 123 124

Conclusion 6.1 Overview 6.2 Five Broad Conclusions 6.3 Our Recommendations References

127 128 130 134 138

Index

141

About the Authors

Paul Allin is a visiting professor at Imperial College London, researching the use of wellbeing measures. He was previously a statistician, researcher, and policy analyst in various UK government departments and agencies, including directing the Measuring National Wellbeing programme at the Office for National Statistics. He is the author, with David Hand, of The Wellbeing of Nations. David J. Hand is Emeritus Professor of Mathematics and a Senior Research investigator at Imperial College London, where he previously chaired the Statistics Section. He has served as President of the Royal Statistical Society and is on the Board of the UK Statistics Authority. He is the author of over 300 scientific papers and 30 books, including Measurement Theory and Practice, The Improbability Principle, and Dark Data.

xvii

List of Tables

Table 4.1 Table 4.2 Table 4.3

Table 4.4 Table 6.1

Kinds of uses of official statistics (OSR 2019, pp. 2–3, with additions from authors shown in italics ) Users of European official statistics (Vichi et al. 2015, p. 3 and Appendix 1) A taxonomy of individual users according to their frequency of statistical usage and proficiency (Vichi et al. 2015, p. 4) The journeys that an official statistic can take Suggestions for readers wanting further details of methodological developments

86 86

87 93 131

xix

CHAPTER 1

Setting the Scene

Abstract This chapter begins with a discussion of what “progress” means, how it is defined and how it might be measured. This leads us on to examine the nature of statistical facts and of official statistics as the key indicators for monitoring change and progress within society, and we explore the protracted origins and evolution of these measures. One particularly important official statistic, certainly one which attracts a huge amount of attention, is a nation’s gross domestic product (GDP), and we look in detail at its measurement and use. However, GDP is but one measure. It is designed to be a measure of economic activity but displays a number of acknowledged weakneses in meeting that aim. Furthermore, it is clearly not a measure of a wider meaning of progress, other than in economic terms, despite also sometimes being taken as a measure of progress. In the final section of this chapter, we explore the beyond GDP agenda, which seeks to address all these issues. Keywords Progress · GDP · Beyond GDP

1.1

What Is Social Progress?

In everyday terms, talking of progress implies advancement and at least the expectation that this is towards something better. That might reflect a basic human ambition, perhaps driven by nature, nurture, or culture. It © The Author(s) 2021 P. Allin and D. J. Hand, From GDP to Sustainable Wellbeing, Wellbeing in Politics and Policy, https://doi.org/10.1007/978-3-030-53085-3_1

1

2

P. ALLIN AND D. J. HAND

might entail defining a specific goal and plotting a course towards it. But, even within this book’s limitation to social or societal progress, that leaves a lot unsaid. That something is better, or has improved, implies a value judgement: that different conditions can be put in some kind of value order. Sometimes such ordering is straightforward: the human condition was improved by the discovery of general anaesthetics. But sometimes ordering can be questionable: a medical treatment which protects against one disease might increase the chance of another. The problem, of course, is the familiar one of measuring a complex phenomenon with multiple aspects. All of this means that, for society as a whole, progress is probably best defined through indicators, especially where these relate to the things that matter: better education, improved health, enhanced transport networks, increased wealth permitting a wider choice of activities and lifestyle, and so on. Glossing over the difficulty of measuring them, they individually have relatively evident orders, so that improvement or progress is fairly clear for each of them. However, the fact that there are multiple indicators also means that there will be many different overall measures of progress, depending on which sets of indicators are chosen to reflect it. A further complication, to which we return below, is that progress on the small scale or in the short term might be the opposite of that on the large scale or long term. We have witnessed many examples of this. One is the industrial revolution powered by fossil fuels improving the human condition over the twentieth century, only to realise the longer term, adverse consequences of anthropogenic climate change. Another is the dramatic improvement in a wide range of human activities through the use of plastics, ranging from better food preservation and hygiene to a vast range of manufactured products including medical equipment and clothes, only to experience longer term damage as microplastic pollution becomes universal. A third is the development of medicines to treat previously intractable diseases, only to witness the development of antibiotic resistant superbugs. To summarise, these examples illustrate the importance of breadth of scope and the simultaneous use of multiple indicators when evaluating progress. It is not simply a question of optimising one indicator. Different indicators work in different directions, and can sometimes even be opposed, so that focusing on just one indicator is likely to be misleading. Worse still, the dynamic nature of social progress means that

1

SETTING THE SCENE

3

the relevance of indicators can wax and wane—sometimes even as a result of the use of those indicators themselves, in a feedback process. For example, …the infant mortality rate (IMR) is often used as an indicator of health levels in preindustrial societies, where high rates are an important concern, and where reductions can be relatively easily achieved. As infant mortality declines, however, a law of diminishing returns begins to apply, and further reductions require increasingly large expenditure of resources. As the numerator becomes smaller, it also becomes less representative as an indicator of the health of the broader population. (McDowell & Newell, 1996, p. 11)

This point about the need for a range of indicators is important because traditionally social progress has been described in purely financial terms. In particular, as we explore below, GDP (gross domestic product) is the most widely cited measure. Unfortunately, as a measure of wider social progress and welfare, GDP is seriously deficient. Indicators of social progress are constructed by aggregation of data from or about individuals. While this is increasingly done by private organisations, their primary interest is their customer base. Wider measures of society as a whole are typically the domain of official statistics, as created, collated, and published by government agencies or other public bodies. The aim of these statistics is to inform governments, commercial organisations, charities, and citizens. That is, to enable all these stakeholders to see the state of society and how it is changing, and ultimately to benefit the public.

1.2

What Are Society’s Facts?

Historian Mary Poovey begins her 1998 book A History of the Modern Fact by asking “What are facts? Are they incontrovertible data that simply demonstrate what is true? Or are they bits of evidence marshalled to persuade others of the theory one sets out with? Do facts somehow exist in the world like pebbles, waiting to be picked up? Or are they manufactured and thus informed by all the social and personal factors that go into every act of human creation?” (Poovey 1998, p. 1). In short, she asks, are facts immutable and incontrovertible truths, or are they malleable according to the circumstances and aims?

4

P. ALLIN AND D. J. HAND

The notion of fact as an objective truth is certainly attractive. It means we are at least standing on common foundations. As Daniel Patrick Moynihan put it “Everyone is entitled to his own opinion but not to his own facts” (Weisman 2010, p. 2). Objectivity means impartiality and lack of bias and prejudice, but also reproducibility (very topical at the time of writing, when there is an ongoing discussion about reproducibility in science). It means that the value of a measurement taken here, now, by her can, at least in principle, be duplicated when taken there, then, by him. More than this, however, the value of measurements can be communicated, to tell you what you would obtain were you to take the measurement. (All this has qualifications about “other things being equal” or “under identical conditions”). The Royal Statistical Society was founded on the belief in objective fact. It supposed that it was possible to collect data, that is facts, devoid of contamination from “opinion”—or, as one might say, of theory or interpretation. The Society’s Prospectus said it “will consider it the first and most essential rule of its conduct to exclude carefully all Opinions from its transactions and publications – to confine its attention rigorously to facts” (RSS 1934, p. 22). This position was partly due to historical accident around the creation of the Society. At the third meeting of the British Association for the Advancement of Science, held in Cambridge in 1833, the Association had somewhat grudgingly accepted the creation of a new statistical section. Its reservations were based on a concern that the new section’s activities might drift from the scientific to the political. When this new section met, it decided that more was needed and that a permanent statistical body should be established: “Following up the spirit of the instructions received by the Committee at Cambridge, it is advisable to take immediate steps to establish a Statistical Society in London, the object of which shall be the collection and classification of all facts illustrative of the present condition and prospects of Society…” (RSS 1934, p. 10). In a public meeting held in March 1834, one of the attendees, Charles Babbage, proposed “That a Society be established in the name of the Statistical Society of London, the object of which shall be the collection and classification of all facts illustrative of the condition and prospects of Society…”. They moved quickly, and the newly formed Society met on 18 April 1834, and then again on 3 May 1834. (The Statistical Society of London became the Royal Statistical Society in 1887).

1

SETTING THE SCENE

5

The prospectus of the Society describes its aims as the “procuring, arranging, and publishing ‘Facts calculated to illustrate the Condition and Prospects of Society’” (Hill 1984, p. 131). The words condition and prospects are almost synonyms for wellbeing and progress. And then, emulating the constraints adopted by the earlier statistical section of the British Association, the prospectus gave the passage quoted above, that “The Statistical Society will consider it to be the first and most essential rule of its conduct to exclude carefully all Opinions from its transactions and publications”. It further continued “and, as far as it may be found possible, to facts which can be stated numerically and arranged in tables”. Numbers in particular are often seen as prime examples of facts. Again, Poovey: “Numbers have come to epitomize the modern fact, because they have come to seem preinterpretive or even somehow noninterpretive” (Poovey 1998, p. xii). Pure numbers certainly do have an intrinsic objectivity and unambiguity. A 3 is a 3, and nothing more than a 3. It is only within the darker realms of numerology and mysticism that properties and values are attached to numbers beyond the fundamental property of quantity. However, when a number is used to describe something, then more than simply the numerical value of the number is being used. At the least it summarises the results of a (possible variable and ambiguous) process of assigning the number, so that the simple objectivity of the number is lost. This should always be borne in mind: facts represented as numbers are more than pure numbers. More generally than numbers, a doubt that facts can exist out of the context of some theory was early realised: it is “impossible to frame a Statistical exhibition of the present subject [crime] for practically useful purposes, without theorizing” (Proceedings of the Statistical Society of London, Vol 1, p. 194, 1836; quoted in Hill 1984). A comment in the Sixth Report, of 1838–1840, said “it was not to perfect the mere art of ‘tabulating’ that it [the Society] was embodied:- it was not to make us hewers and drawers to those engaged on any edifice of physical science;but it was that we should ourselves be the architects of a science of or sciences, the perfecters of some definite branch or branches of knowledge, which should do honour to ourselves and to our country…” (Statistical Society of London 1840). In a paper in the London and Westminster Review, published in 1838, G. Robertson, a deputy editor, commented on this constraint of the Society to “facts”:

6

P. ALLIN AND D. J. HAND

Opinion is what is most wanted where truth is the object, it is the parent and precursor of truth – opinion is carefully excluded by the Statistical Council … the process of seeking and sifting new opinions is the progress of science – to avoid this process is the most essential rule of the Statistical Council. (Robertson 1838, p. 49) The exclusion of opinions is the exclusion of the only guides which can conduct their researches to any useful end. (p. 50) To separate the facts from the propositions they support, the evidences from the thing they prove, and on their bearing on which their character as witnesses depends, is to destroy and annihilate their nature as evidence – and by stripping them of that for the sake of which they are noticed at all they are rendered utterly meaningless. (p. 69)

In a note which strikes a modern chord, Robertson also quoted a Dr. Cullen: “there are more false facts than false theories”. One hesitates to say that that was its first use, but clearly that was a very early use of the phrase “false facts”, which has become so widespread in recent years. Some thirty year later, in 1865, William Guy, a Vice President of the Statistical Society of London and sometime editor of its journal, pointed out that “if the author could succeed in concealing his opinions, his audience would not be restrained from expressing theirs; and it was surely hard to deny him a liberty which could not be refused to them” (Guy 1865, p. 483). Facts do not exist in isolation, but only in a context. Even something as apparently so isolated as an observation that someone weighs 70 kg is in fact a relationship between the person’s weight and that of a unit weight. And the observation of their weight is then only useful or interesting or valuable when further comparisons are made. Again, as Robertson (1838, p. 57) put it in his diatribe against the limited aims of the Statistical Society: “Facts are valuable only in relation to what they prove – the evidence is important or otherwise according to the proposition to which it relates”. One can have evidence only “for or against something”. There is no such thing as evidence in an absolute sense. This also means that facts can change. They can change because the world they are describing changes: your weight will change if you go on a diet. But facts can also change if their definition changes. We might have come to expect both of these sorts of changes in social and

1

SETTING THE SCENE

7

official statistics—inflation indices, for example, both changing over time and with altered definitions. But definitional changes also occur at a much more fundamental level. Indeed, even at the most fundamental level. Up until 1799 the mass of the kilogram was defined as that of one litre of water. Difficulty in replicating this led to a change and, up until 20 May 2019, it was defined the mass of the International Prototype Kilogram, a particular platinum alloy cylinder stored in Paris. The trouble is that the measured weights of the prototype and its duplicates deviated very slightly over time—something changed. This prompted a new definition, this time based on a fundamental and immutable physical constant—the Planck constant. Not only is this the same for all Earth-bound researchers, but it is the same wherever you are in the Universe. While the notion of the objectivity of the low-level fact is persistent, the notion of objective statistics—that is of information derived from more basic facts—has often been subject to attack. You will be familiar with hoary comments such as “there are lies, damned lies, and statistics” and “you can prove anything with statistics”. We should first comment that one person’s statistic is another person’s data. Data are used to produce estimates of population size and literacy rate for each country, each number being a statistic. But those sizes and rates themselves become data when summarised in a correlation coefficient describing the relationship between population size and literacy rate across the world’s countries. The distinction between data and statistic is again context-dependent. Indeed, this is captured in the modern phrase “data ecosystem”. The extra suspicion arising with statistics has several sources. By definition the statistics are produced as aggregates of lower level “facts”. That means there is an intrinsic indirectness to the results. While we might accept the result of weighing someone as a generally objective and accurate representation of their weight, we might be less happy with the objectivity when these weights are aggregated to yield an average for a population. At the least this introduces further steps, not so direct and representational in character—the mapping from the thing we are measuring to the number we want is not so immediate. We need to decide which people to include in our sample, ensure that people are not missed, consider how their weights might change over time if we cannot measure them all simultaneously, determine how to calculate an average (do we mean arithmetic mean, geometric mean, median, or what?), and so on.

8

P. ALLIN AND D. J. HAND

The key point is that further decisions must be made, decisions which are not forced upon us by the thing we are measuring. These decisions tell us both what we are measuring (the population’s average weight) and how to measure it (the sampling procedure, how to combine the individual weights into an average, and so on). And if we change the procedure then we change the thing being measured—it is a so-called pragmatic measurement procedure, with the definition and process being two sides of the same coin: the measurement procedure is the definition. The choices that must be made in defining a measurement are entirely distinct from matters of precision, accuracy, reliability, or trustworthiness in measurements and their resulting values. These characteristics might well impact on the objectivity—as in a measurement which is consistently biased. It is no coincidence that the word “bias” has a statistical meaning (a difference between a “true” value and the average value from multiple measurements) and a related everyday meaning (prejudice). Another way of looking at facts is that they represent a projection of the objects being studied. We can determine someone’s age, height, IQ, or wealth, and these, while they may be facts about the person, do not fully determine that person. They are merely a selection of characteristics from an unlimited possible number which could be chosen. This means that, inevitably, objectivity is lost at a higher level—a choice has to be made as to what “facts” to report, and this choice could be influenced in various ways. Incidentally, this projection notion enables us to sidestep the realist/constructivist problem with complex social measures like inflation and GDP. They are different aspects or projections of an underlying concept, which is impossible to define completely. Determining facts describing the objects being studied requires the definition of categories and classes that individuals can be recognised as falling into. We classify people as male or female or other, and as having an age within a certain band. These categories and classes are choices which must be made, and where different choices might lead to different conclusions based on analysis of aggregate numbers falling within each class. The challenges are made apparent when attempts are made to align administrative systems (e.g. when the national statistical offices of different European countries produce statistics for the European Union) or when changes are made to the definitions of time series (e.g. of inflation, unemployment, or GDP).

1

1.3

SETTING THE SCENE

9

What Are Official Statistics?

While clearly the role of numbers to represent facts goes back beyond human history (some animals can count, at least up to small numbers), numbers have a central role in modern macro and aggregate descriptions of collections, especially of human populations and human transactions. A prime example of this is in double-entry bookkeeping: the recording of facts describing transactions, ownership, and trade. Double-entry bookkeeping can be traced to before 1494, when it was described by Luca Pacioli in a section of a chapter of his book Summa de arithmetica, geometria, proportioni et proportionalità. Such elaborate systems of recording data were a necessary facilitator to the growing scope of trade. They enforced an honesty, guaranteed accuracy, and were a representation of an underlying reality (X sold Y three sheep for such-and-such a sum). Of course, the failure of the records to be completely accurate manifests itself in errors, certainly, but also in dishonesty and fraud. The other early example of large-scale data collection arises in censuses—which have origins with a biblical timespan. Then, gradually, compilations of numerical data began to be increasingly widely used: by John Graunt, for example, examining local records of baptisms, marriages, and deaths and developing the first life table in the seventeenth century; by insurance companies and actuaries producing mortality tables in the eighteenth century; by Adolphe Quetelet, with his tables of human behaviour and his notion of the “average man”. (Quetelet had been deputed by the Belgian government to attend the third meeting of the British Association and was present at, and indeed instrumental in, the creation of the Statistical Society of London). This chapter is far from a complete history of official statistics. We recognise that mentioning a few people means that we have not drawn attention to many others, including those “hidden figures” (Shetterly 2016) who also made significant contributions to statistics, as in mathematics, science, and engineering. Nevertheless, a key point should be clear, that much of the earlier data was concerned with, on the one hand, public health, and on the other economics, including taxation, and social welfare. An example of an important figure in the area of public health was William Farr, a doctor employed by the General Register Office in the UK (which had responsibility for the census), who established a system

10

P. ALLIN AND D. J. HAND

for recording causes of deaths. This in particular allowed the mortality rates of different occupations and of different cities to be revealed. For example, the population growth of Liverpool between 1801 and 1831 had led people to suppose that it had a particularly healthy climate—until it was revealed that half of its children died before age six (Desrosières 1998, p. 168). By publishing average mortality rates of districts, Farr enabled attention to be focused on the top end of the distribution (instead of, as Desrosières puts, it Quetelet’s average). Florence Nightingale was another of the “passionate statisticians” of that time (Bostridge 2008, p. 171), believing that research and especially statistics could be used to help the work of the God of her faith. Lynn McDonald (McDonald 2006) remarks that a “Nightingale Method” can be discerned in her approach to public health, the first step of which is to “get the best information available in print, especially government reports and statistics”. One important figure from the area of economics and social welfare was Arthur Bowley, who produced studies of such matters as trade, poverty, wages, and occupations, and introduced the novelty of sampling methods, as well as describing standardised methods for collecting and recording such data. Bowley began a 1908 paper read to the Royal Statistical Society by apologising for the fact that it had no statistical tables, and explaining that its aim was to improve official statistics. He then did so by focusing on seven areas: precise definitions of the units being studied, the homogeneity of the population of units, the adequacy of the sampling process, nonstationarity, consistency of methods to allow comparisons, and appropriate commensurability of numbers involved in proportions, as well as overall accuracy. If the rigour of the data collection is central to the usefulness of these collations of data, so also is the use of statistical methods—that is, of ways of analysing, summarising, and probing the raw data so that useful information and insights can be gleaned from them. The algorithms which are used to transform data into statistics must be clearly defined, and narrowly constrained by rules which specify precisely how the statistics are to be constructed. While one might prefer other algorithms or ways of constructing the statistics, at least this means we know what we are talking about. Precision in the rules means we can sensibly discuss the relative merits of different definitions—of different ways of constructing the statistics. Again, should we, for example, use the geometric or arithmetic mean to combine prices at the elementary level when constructing a measure of price inflation?

1

SETTING THE SCENE

11

Implicit in all this is a point which is fundamental to this book. A statistical summary of a distribution is a real descriptor of that distribution. The statistical process measures and defines higher-level properties which do not exist for lower level entities. For example, the variance of a set of values is a measure of how different they are. The concept “how different they are” does not exist for a single point (even though one can plug the single value into the formula for variance, obtaining the value 0). “Difference” is a property which refers to one or more points, not single points. By producing statistical summaries of the mass of members of a society we are determining characteristics of that society, not of the individuals. Desrosières (1998, p. 236) puts it beautifully: “The aim of statistical work is to make a priori separate things hold together, thus lending reality and consistency to larger, more complex objects. Purged of the unlimited abundance of the tangible manifestations of individual cases, these objects can then find a place in other constructs”. While, in principle at least, both double-entry bookkeeping and censuses aim to collect data on every member of a population, this is often not necessary. In particular, certain basic statistical laws—the law of large numbers especially—mean that we can find accurate estimates of population characteristics from a mere sample of data from the population—as described by Arthur Bowley. This is provided, however, that the sample is properly drawn. For this reason, many official statistics are based on samples drawn by surveys. Increasingly, however, official statistics are based on so-called administrative data—data collected in the normal course of some operation, such as collecting taxes or through the education or health systems—and there, at least in principle, data are available on all of the members of the population (but see Hand 2018, for a discussion of the challenges associated with such data). A further statistical complication arises if and when we wish to combine noncommensurate characteristics. In the opening paragraphs of this chapter we noted that progress had many aspects. While each of them separately might have a relatively clear meaning and implications, in some situations a single aggregate measure is desirable. It is a question of combining apples and oranges. Sophisticated and deep tools have been developed to tackle this, as discussed in Allin and Hand (2014, pp. 23– 26), but we need to recognise that the diversity of aspects lies at the heart of the challenge of measuring progress. Broadly speaking, nowadays “official statistics” are those produced by governments and their agencies, describing some aspects of society or the

12

P. ALLIN AND D. J. HAND

economy. Many countries have a designated national statistical institute, which has the primary responsibility for producing official statistics, while others (such as the US) have a federated statistical system. In the UK, for example, while the Office for National Statistics (ONS) is the primary producer of statistics relating to the UK as a whole, other government departments (and other bodies) also produce many official statistics, typically for restricted domains (e.g. education, health, etc.). These all come under the general umbrella of the Government Statistical Service (GSS), a cross-government network led by the National Statistician. The advantage of this dispersed structure is that the generation of the descriptive statistics is carried out close to the originating operations, so that they can be rapidly adapted to changing needs. Of course, the complementary downside is a possible lack of coordination and coherence across government departments. The history of UK official statistics is a long-drawn-out one of gradual recognition of the importance of statistical descriptors, coupled with a gradual trend of unification (Jenkins 2016): things have come a long way from the time when such statistics referred mainly to the “vital statistics” of births, deaths, marriages, health, and disease. The first UK body set up with the main objective of producing official statistics, rather than doing so as a side-effect of or in addition to other activities, was the Statistical Department of the Board of Trade, established in 1832. Awareness of the power and importance of statistical understanding gradually grew, and more and more governmental bodies began to produce statistics. The result was a lack of coordination and an ad hoc flavour in terms of societal coverage and statistical methodology. The pressing need to understand national resources during the Second World War led to the creation of the Central Statistical Office in 1941, with the remit of producing national accounts (the word “state” and “statistics” have the same root). A further rationalisation occurred in 1970, when the General Register Office and the Government Social Survey Department were merged to produce the Office of Population Censuses and Surveys. A quarter of a century later, in 1996, the ONS was created by merging the Central Statistical Office and the Office of Population Censuses and Surveys. In 2000 the director of the ONS became known as the National Statistician. In 2008, the Statistics and Registration Service Act (SARSA) came into force, establishing the UK Statistics Authority (UKSA) as a body, independent from Government, with the ONS as its executive office and the Office for Statistics Regulation (OSR) as its regulatory arm. One of

1

SETTING THE SCENE

13

the tasks of the UKSA was to publish a “Code of Practice for Official Statistics”. This was recently revised, to base it on the three pillars of trustworthiness, quality, and value. The SARSA left the dispersed nature of the statistical system unchanged, with each department having its own head of statistical profession, but these having professional accountability to the National Statistician, serving as head of the GSS. The challenges arising from statistical production being dispersed across government departments are replicated at a higher international level by the fact that each country produces its own statistics. In its description of the background to its Fundamental Principles of Official Statistics , the United Nations Statistics Division (United Nations 2014) notes how in the 1980s it became “essential to ensure that national statistical systems … would be able to produce appropriate and reliable data that adhered to certain professional and scientific standards”. The preamble to the Principles also stresses the critical role of high quality information “in support of sustainable development, peace and security”, and the importance of public trust in the integrity of the statistics, and notes that the fundamental values and principles must be guaranteed by legal and institutional frameworks. Adherence to the Principles, and demonstrating this by showing compliance with associated codes of practice, is therefore now a key aspect of the operation of organisations which produce official statistics. There are altogether ten UN principles (United Nations 2013). Principle 1 begins “Official statistics provide an indispensable element in the information system of a democratic society, serving the Government, the economy and the public…” Whereas in the past, official statistics were disseminated in the form of lengthy printed documents and books, nowadays the web is used. This has hugely improved accessibility and has also enabled a much wider variety of statistics to be produced, with more timely publication, and the potential for further analysis by other organisations. It also makes more real the notion that official statistics are for everyone, not merely the government but also the governed.

1.4 What Is GDP and What Does It Seek to Measure? In our Preface we characterised GDP as both the hero and villain of official statistics. It is a hero because it is the single most important barometer of progress in a country. Fractions of a percentage change

14

P. ALLIN AND D. J. HAND

up or down are monitored closely, and taken as signs of blooming economic health or of recession. In short, it is the so-called headline indicator. On the other hand, it is also a villain, because it is focused tightly on economic wellbeing. The economic perspective is important but it is a narrow perspective, which misses many other aspects of society which would generally be regarded as important contributors to national welfare. Indeed, one might argue that economic wellbeing is really merely the means, and that the end is characterised by other kinds of wellbeing (health, contentment, sustainability, etc.). As Costanza et al. (2009, p. 28) put it “The first step is to agree on new global goals: goals that recognise that our economic system is a tool for improving wellbeing and not something to grow mindlessly for its own sake”. In fact, and as we note below, GDP is not without its critics purely as a measure of economic wellbeing. And, as if all that was not enough, it has also been criticised because it is meaningless to ordinary people. The news that the national GDP has gone up might be treated with at best disinterest and at worst scorn in terms of the recipient’s own economic situation. Although the idea of measuring national wealth can be traced back some hundreds of years, originally to enable rulers to determine if they could fund wars, the notion of “an economy” which could be measured is a fairly recent one. As we saw above, only in response to the 1930s economic depression and the Second World War were modern ideas for measuring an economy introduced, led by economists such as Colin Clark in the UK and Simon Kuznets in the US. Subsequently, the United Nations established the System of National Accounts (SNA), specifying international standards for the production of such measures. (Kuznets had a wider vision than mere economic activity, but his broader vision became swamped by economic drivers.) The System of National Accounts (United Nations 2009, p. 1) is designed to be comprehensive (“all designated activities and the consequences for all agents in an economy are covered”), consistent (“identical values are used to establish the consequences of a single action on all parties concerned using the same accounting rules”), and integrated (“all the consequences of a single action by one agent are necessarily reflected in the resulting accounts”). However, it is solely concerned with economics: “The SNA measures what takes place in the economy, between which agents, and for what purpose. At the heart of the SNA is the production of goods and services” (United Nations 2009, p. 2).

1

SETTING THE SCENE

15

GDP is a key aggregate within the SNA and “the most frequently quoted indicator of economic performance” (United Nations 2009, p. 1). The development of GDP as a measure took place in an era when mechanical analogies of economics and society were dominant. Indeed, in 1949 economist Bill Phillips constructed a hydraulic computer modelling the behaviour of the UK economy, with coloured water flowing along transparent plastic pipes representing the flow of money around the economy, from the Treasury and elsewhere. (At the time of writing one of these models is on display in the Winton Mathematics Gallery at the Science Museum in London). The appropriateness and correctness of these sort of models of the economy seemed to be demonstrated by the post-war economic boom, when increased government spending and low interest rates led to dramatic GDP growth: while the world’s population increased by a factor of 1.6 from 2.5 to 4 billion between 1945 and 1970, GDP increased by a factor of twice that (allowing for inflation). However, this simplistic observation glosses over the fact that intrinsic to the (expenditure) definition of GDP are measures of investment and government spending. Increase investment and government spending and you necessarily increase GDP, regardless of any economic mechanisms. This clearly reveals the pragmatic nature of GDP, in the technical sense noted above: that the measurement procedure is the definition. Others have also noted this nature of GDP. Economist Diane Coyle says “… this is not a question of measuring a natural phenomenon like land mass or average temperature to varying degrees of accuracy. GDP is a made-up entity”, (Coyle 2014, p. 4). Nobel Laureate Richard Stone, commenting on the details of its construction, said: “This treatment, whereby commercial products are valued at market price, government services are valued at cost and unpaid household activities are simply ignored, is not a matter of principle but of practical convenience”, (Stone 1984). And Paul Samuelson and William Nordhaus described “…the GDP and the rest of the national income accounts … [as] truly among the great inventions of the twentieth century”, (BEA 2000). Note, “invention”, not discovery. The point of stressing the pragmatic or constructivist nature of GDP measurement is to drive home the fact that arbitrary choices are being made in its definition. Other choices, which could give different results, could be made. Indeed, there are three standard approaches to measuring GDP, the so-called production (or value-added) approach, the income approach, and the expenditure (or final demand) approach. In principle,

16

P. ALLIN AND D. J. HAND

they should give the same result, but of course they do not—though too substantial a difference would be cause for alarm. The same requirements apply to measuring GDP as they do to any other statistical measure of a population, noted earlier: it is necessary to ensure that the components of GDP are carefully defined, that the raw data are carefully and accurately collected (a point stressed by John Maynard Keynes), and that the measurement procedures stick to the definitions. Any actual or perceived political or governmental interference in the measurement process will necessarily detract from the objectivity of the final values and must be avoided. We refer in Sect. 5.2 below to unfortunate consequences of interference in the case of Argentina’s consumer price index. Even given a definition, measuring GDP has its challenges. One has been measuring the size of the service sector. It’s one thing to measure material resources and amount spent producing physical objects, but it’s another to decide quite how to measure a service. It is far from clear that the productivity of a teacher, a nurse, or a park attendant should be measured by number of pupils taught (how well?), patients seen (how many recovered?), or amount of dropped litter prevented. A pertinent example is given by the reduction in physical retail outlets, replaced by online facilities costing far less to run. For example, the number of high street travel agents has dramatically decreased simply because people nowadays buy their tickets online, using software instead of human agents. While this may imply a reduction in contribution to GDP (online facilities must be developed and maintained, but our selfservice in using them is not counted in GDP, and there no shops to be built, maintained, serviced, or staffed) it does not mean that people take fewer holidays. What has happened is that the ticket purchasing activity has become more efficient in some sense. You might think that this surely ought to indicate an increased GDP (as a measure of productivity). Similar points apply to a great number of products and services delivered online or by streaming, such as music, films, and books. The unit price reduction (sometimes even to zero) associated with such changes means a GDP reduction, but it is far from clear that it should. At a higher level, there is also the challenge of public sector services. In the private sector you can at least use as a measure the amount paid for a service, but “amount paid” might not be a relevant measure in the public sector if the service is provided by the state. This is illustrated by the challenge of how to measure productivity improvement in the public

1

SETTING THE SCENE

17

sector if wages haven’t changed but a better and more efficient way has been found to do something. A second challenge to GDP arises from technical innovation and change. Laptop computers have become a classic illustration. The unit price has not changed much, but the power of the machines has improved dramatically. The simple use of unit prices would fail to reflect this improvement in productivity, underestimating GDP. More generally, failing to take account of changes in the quality of products would artificially boost apparent inflation rate. Indeed, the Boskin Commission in the US in 1996 suggested that the Consumer Price Index had been overstating inflation by about 1.3 percentage points per year because of this (Roth 1996, p. 1). Sophisticated methods have been developed to tackle these challenges and so-called “hedonic” methods have been introduced. These use regression models of price on characteristics of the products (e.g. memory size, battery life, etc., of laptops) so that price changes unrelated to quality changes can be determined. These feed into the inflation and productivity measures. A third challenge is the perhaps familiar one of contributions of “unmeasured” aspects of the economy. These include, on the one hand, those activities which occur outside of the law and on the other those activities which occur at home in the normal course of events—such as cooking or childcare. Regarding the former, international standards dating from the 1990s require that illegal activities based on mutual agreement between two parties should be included in measures of GDP. Despite this, considerable media attention was stimulated when illegal drug sales, prostitution, and smuggling were explicitly included in the measurement of EU economic activity in 2014 (Eurostat 2018). An example of the latter is that two households each looking after their own children contribute nothing to GDP, whereas if they were to pay each other to look after each other’s children they would contribute. A fourth challenge arises from how products and services are classified. In the past, for example, software was treated as an intermediate good in GDP calculations, but this was changed to treat it as an investment, albeit one which depreciated more rapidly than most physical plant. Student loans in the UK provide another example. In 2018 the size of these loans amounted to some 6% of the UK’s GDP, predicted to rise to 40% by 2040 (ONS 2018). Repayment of these loans is conditional on salary level, and it is thought that a significant proportion will eventually be written

18

P. ALLIN AND D. J. HAND

off. While this has little impact on current government expenditure, it is clearly likely to have a big impact in the future—to the extent that the Office for National Statistics changed the classification, recommending that the loans should be treated “part as financial assets (loans), since some portion will be repaid, and part as government expenditure (capital transfers), since some will not”. The classification of financial services raises particular questions. They are regarded as making a major contribution to the GDP of the UK, but a look at the figures suggests this is a little odd. This is illustrated by the fact that, as a percentage of the output of the UK economy for each of the years between 1990 and 2017, the financial year when they apparently made the greatest contribution was 2009, at 9%, even though this was at the height of the financial crisis (Rhodes 2018; Haldane 2010). In contrast to most other service industries, which are paid for the service they provide, the banking sector makes much of its profit from the difference between the interest rates charged on loans, and the interest paid for borrowing money. One consequence is that if the standard methods for calculating GDP contribution were applied, subtracting the goods and services used in production from the output, this would mean the banks made very little contribution (or, in fact, even a negative contribution). To many, this seems counterintuitive—surely such services must be contributing to the economy—so sophisticated alternative approaches have been developed. These include the 1993 UN FISIM system—Financial Intermediary Services Indirectly Measured—based on comparing lending and borrowing rates with a “risk-free” rate. Under this system, a part of the bank’s output arises as compensation for bearing the risk. This can explain the 2009 apparent contribution to the economy, but it is not clear that bearing risk is actually a productive activity. There is, of course, a more general point underlying this. This is the difficult of distinguishing between economically “productive” and “unproductive” activities. At an extreme, Adam Smith regarded all services as unproductive. GDP tries to sidestep the difficulty by defining the measurement in terms of what people pay for it, but one wonders if “payment”, “output”, and “product” are intrinsically misleading, even for economic measurement. Other complications include the fact that inflation needs to be allowed for (and this has its own measurement difficulties), that seasonal adjustments are needed, and also general questions of timeliness of estimates

1

SETTING THE SCENE

19

and updating them as more data become available (this can even retrospectively eliminate an economic slump, which is alarming if policy decisions have been based on that slump). GDP is a measure created for an economy different in kind from today’s, with different kinds of activities—certainly one which relied more on physical manufacturing than service industries and software. In essence, GDP counts products, not taking into account the variety of products of a given kind, or, by implication, of the choice this gives the consumer. We might expect this variety will go on increasing as society advances, ultimately leading to a situation where each consumer can choose a customised product. This was always the case with, for example, tailored clothes instead of off-the-peg purchases, but we are now seeing the web moving it down the chain to a much wider customer base. The advent of 3-D printing is likely to have a similar impact for a yet wider range of products. This freedom of choice does not figure in current GDP calculations, but this greater variety is an intrinsic part of economic progress.

1.5

What Lies Beyond GDP?

We have noted that Simon Kuznets was one of the creators of GDP. In fact, however, his original aim was to produce something rather different—a measure of national welfare. While the activities of illegal drug sales and prostitution recently included in UK GDP estimates might count as economically productive activities, some might argue that it is hard to see that they can be regarded as measures of national wellbeing. As Kuznets put it in 1937: “It would be of great value to have national income estimates that would remove from the total the elements which, from the standpoint of a more enlightened social philosophy than that of an acquisitive society, represent disservice rather than service”. And he went on to say “Such estimates would subtract from the present national income totals all expenses on armament, most of the outlays on advertising, a great many of the expenses involved in financial and speculative activities” (quoted in Mitra-Kahn 2011, p. 239). Kuznets’ words were echoed by Robert Kennedy, thirty years later when he said: Too much and too long, we seem to have surrendered community excellence and community values in the mere accumulation of material things.

20

P. ALLIN AND D. J. HAND

Our gross national product […] – if we should judge America by that – counts air pollution and cigarette advertising, and ambulances to clear our highways of carnage. It counts special locks for our doors and jails for those who break them. It counts the destruction of our redwoods and the loss of our natural wonder in chaotic sprawl. It counts napalm and the cost of a nuclear warhead, and armoured cars for police who fight riots in our streets. It counts Whitman’s rifle and Speck’s knife, and the television programs which glorify violence in order to sell toys to our children. (Kennedy 1968, 16:20 minutes)

The problem is reflected in the fact that the reconstruction following a disaster, be it precipitated by humans or nature, leads to a boost in GDP. Apart from the inclusion of socially damaging (if “productive”) activities, it is clear that GDP omits a great deal of what might legitimately be regarded as contributing to progress and development; such things as improved education, sanitation, and health, perhaps as measured by indicators such as child mortality and life expectancy. GDP also misses out on the critical long-term aspect of sustainability. This is a subtle but vitally important aspect. We cannot engage non-existent possible future people in an explicit reciprocal contract about our relative balance of resource consumption (be those resources positive, such as quantity of copper ore, or negative, such as extent of plastic pollution or atmospheric carbon dioxide concentration), and yet surely a measure of “progress” which doomed our descendants would be a poor measure. The above considerations have led to a dynamic “beyond GDP” agenda, aimed at producing a measure or suite of measures which more adequately reflect the notion of human wellbeing and flourishing. For example, as an alternative to blunt economic indicators, Amartya Sen (1992, p. 39) introduced the idea of measuring capabilities and functionings. Capabilities obviously include income, but also education and other factors. Functionings are “the various things a person may value being and doing”, such things as eating well, avoiding illness, self-respect, political freedom, and a sound social life. Of course, it will be immediately obvious that measuring these will pose more challenges than measuring GDP, where the latter is couched in financial terms (even if that has colossal difficulties of its own). But just because something is difficult to measure does not mean it cannot be measured or that we should not try. Indeed, the emphasis on narrow economic concepts via GDP, rather than arguably more important broader concepts, reminds us of the old adage “what gets

1

SETTING THE SCENE

21

measured gets done”, regardless of whether it is the most appropriate thing to do. Naturally, since the economic aspects intrinsic to GDP also play a role in wider measures of wellbeing, there is a correlation between wellbeing measures and GDP, and some have argued that the strength of this correlation means that GDP is a sufficient measure. However, since attention is typically paid to relatively small changes of GDP (and wellbeing) the correlation is not such that GDP can stand as an adequate measure of broader wellbeing. Within a span of small changes, the correlation is weak. Something more elaborate is needed. We have discussed these problems and how to overcome them in depth in Allin and Hand (2014), where an appendix gives an extensive list of alternative measures, aimed at capturing more than narrow economic progress.

References Allin, P., & Hand, D. J. (2014). The Wellbeing of Nations: Meaning, Motive, and Measurement. Chichester: Wiley. BEA. (2000). https://apps.bea.gov/scb/account_articles/general/0100od/mai ntext.htm. Accessed 8 March 2020. Bonar, J., & Macrosty, H. W. (1834). The Annals of the Royal Statistical Society, 1834–1934. London: Royal Statistical Society. Bostridge, M. (2008). Florence Nightingale: The Woman and her Legend. London: Viking. Bowley, A. L. (1908). The Improvement of Official Statistics. Journal of the Royal Statistical Society, 81, 459–479. Costanza, R., Hart, M., Posner, S., & Talberth, J. (2009). Beyond GDP: The Need for New Measures of Progress. The Pardee Centre, Boston University. https:// www.bu.edu/pardee/files/documents/PP-004-GDP.pdf. Accessed 22 August 2019. Coyle, D. (2014). GDP: A Brief But Affectionate History. Princeton: Princeton University Press. Desrosières, A. (1998). The Politics of Large Numbers: A History of Statistical Reasoning. Cambridge, MA: Harvard University Press. Eurostat. (2018). Handbook on the Compilation of Statistics on Illegal Economic Activities in National Accounts and Balance of Payments, 2018 edition. Publications Office of the European Union, Luxembourg. Guy, W. (1865). On the Original and Acquired Meaning of the Term “Statistics,” and on the Proper Functions of a Statistical Society: also on the Question

22

P. ALLIN AND D. J. HAND

Whether There be a Science of Statistics; and if so, What are its Nature and “Social Science”. Journal of the Statistical Society, 28, 478–493. Haldane, A. (2010). The Contribution of the Financial Sector—Miracle or Mirage. https://www.bankofengland.co.uk/-/media/boe/files/speech/ 2010/the-contribution-of-the-financial-sector-miracle-or-mirage-speech-byandrew-haldane.pdf?la=en&hash=1FC4780AE5F8869BD4A8D18889A3B7 80FFB32C8A. Accessed 23 June 2019. Hand, D. J. (2018). Statistical Challenges of Administrative and Transaction Data (With Discussion). Journal of the Royal Statistical Society Series A, 181, 555– 605. Hill, I. D. (1984). Statistical Society of London—Royal Statistical Society. The first 100 years: 1834–1934. Journal of the Royal Statistical Society Series A, 147, 130–139. Jenkins, M. (2016). Official Statistic-Making as a Social Practice: The UK ‘Measuring National Well-being’ Programme. PhD Thesis, University of Newcastle, UK. https://theses.ncl.ac.uk/jspui/bitstream/10443/3210/1/Jenkins%2C% 20M.F.%202016.pdf. Accessed 9 March 2020. Kennedy, R. F. (1968). University of Kansas Address. https://www.youtube. com/watch?v=_z7-G3PC_868. Accessed 21 May 2020. McDonald, L. (2006). Florence Nightingale and Public Health Policy: Theory, Activism and Public Administration. https://cwfn.uoguelph.ca/nursing-hea lth-care/fn-and-public-health-policy/. Accessed 7 May 2020. McDowell, I., & Newell, C. (1996). Measuring Health: A Guide to Rating Scales and Questionnaires. New York: Oxford University Press. Mitra-Kahn, B. H. (2011). Redefining the Economy: How the “Economy” was Invented in 1620 and has been Redefined Ever Since. Department of Economics, City University of London. ONS. (2018). https://www.ons.gov.uk/economy/governmentpublicsectoran dtaxes/publicsectorfinance/articles/newtreatmentofstudentloansinthepublicse ctorfinancesandnationalaccounts/2018-12-17. Accessed 22 June 2019. Poovey, M. (1998). A History of the Modern Fact: Problems of Knowledge in the Sciences of Wealth and Society. Chicago: University of Chicago Press. Robertson, G. (1838). Transactions of the Statistical Society of London, Vol. 1, Part I. London and Westminster Review, 45, 72. https://books.google.co. uk/books?id=LfwEAAAAQAAJ&pg=PA45&source=gbs_toc_r&cad=3#v=one page&q&f=false. Accessed 20 April 2020. Rhodes, C. (2018). Financial Services: Contribution to the UK Economy. https://researchbriefings.files.parliament.uk/documents/SN06193/SN0 6193.pdf. Accessed 23 June 2019. Roth, W. V. (1996). Final Report of the Advisory Commission to Study the Consumer Price Index. https://www.finance.senate.gov/imo/media/doc/Prt 104-72.pdf. Accessed 21 May 2020.

1

SETTING THE SCENE

23

RSS. (1934). The Annals of the Royal Statistical Society, 1834–1934. London: Royal Statistical Society. Sen, A. (1992). Inequality Reexamined. Oxford: Clarendon Press. Shetterly, M. L. (2016). Hidden Figures. New York City: William Morrow. Statistical Society of London. (1840). Sixth Annual Report of the Statistical Society of London. https://www.jstor.org/stable/2337961?seq=1#metadata_ info_tab_contents. Accessed 26 July 2020. Stone, R. (1984). The Accounts of Society. https://www.nobelprize.org/uploads/ 2018/06/stone-lecture.pdf. Accessed 23 June 2019. United Nations. (2009). System of National Accounts, 2008. https://unstats.un. org/unsd/nationalaccount/docs/SNA2008.pdf. Accessed 19 August 2019. United Nations. (2013). https://unstats.un.org/unsd/dnss/gp/FP-Rev2013-E. pdf. Accessed 8 August 2019. United Nations. (2014). https://unstats.un.org/unsd/dnss/gp/fundprinciples. aspx. Accessed 8 August 2019. Weisman, S. R. (Ed.). (2010). Daniel Patrick Moynihan: A Portrait in Letters of an American Visionary. New York: PublicAffairs.

CHAPTER 2

Using Statistics to Assess Progress

Abstract For statistics to be a useful way to assess progress, they must be trusted. Official statistics compete with other sources and often arrive at the consumer after various intermediaries have imposed their own angles. A key aspect is the question of who the statistics are for. The term “user” is not necessarily well-defined. Identifying users and indeed potential users may not be straightforward. But identifying the users is a pre-requisite for producing something useful. It is not enough to produce the statistics in the hope that, once they exist, they will become useful. The diversity of users has resulted in a lack of consistent process about what to include in a nation’s set of official statistics. We focus on an authoritative report that sought to align better the metrics measuring wellbeing with what actually contributed to quality of life. While the report attracted a great deal of attention, there is little evidence that the construction of alternative measures has had much practical impact. The UN’s Sustainable Development Goals are a further attempt to move in the same direction, although they are not without their critics, not least because of the sheer scale of the ambition implicit in them. At bottom, measures of social progress need to be anchored in the real practical world. Keywords Official statistics · Utility · User engagement · Trust · Public value · Social indicators

© The Author(s) 2021 P. Allin and D. J. Hand, From GDP to Sustainable Wellbeing, Wellbeing in Politics and Policy, https://doi.org/10.1007/978-3-030-53085-3_2

25

26

P. ALLIN AND D. J. HAND

2.1

Introduction

In this chapter, we explore the question of how is the country doing? In particular, what is the current level of wellbeing in the country? And then how is the level of wellbeing changing and the nation progressing? We are especially interested in how answers to these questions may, or not, be constructed in terms of official statistics. For answers to those questions to be believed, for them to be trusted, they have to merit trust and be trustworthy. That is, the source of the data has to be trusted and the methods through which the answers are derived from the data have to have credibility and trustworthiness. In essence, this means that hearing a fact should prompt the listener to reflect on how the fact was derived and who is declaring it as a fact. If those declaring the fact obtained it from sources other than the primary source of their own data collection, then the chain of acquisition should, at least in principle, be traceable back to its origin. Much has been written and discussed about the current information age: the vast amounts of data accumulating and the granularity of data describing our everyday lives so that all of us cast long data shadows. Asta Manninen, former director of City of Helsinki Urban Facts has put it dramatically, commenting that “[w]e live in information chaos” although adding that “official statistics brings clarity and continuity” (Condon 2017). That, at least, should be the aspiration. One of the challenges is that the data ecosystem is not only large and complex, but is itself changing rapidly. The growth of social media is a prime example, in which people obtain their information not from official sources but from other individuals—who may not have taken the trouble to establish the authenticity of the information they pass on, for example misrepresenting the risks of vaccination and downplaying its scientifically verifiable benefits. Roger Mosey, a former head of BBC News, observes that “styles of communication have changed, but it’s the onslaught of social media that is proving a thorny challenge to the traditional broadcasters. The BBC in particular needs to be distinctive and focussed on the truth – a place of sanity amid the torrents of digital stuff” (Mosey 2018, p. 22). One would hope that official statistics should be a particularly trustworthy source of information about a nation’s wellbeing. But even if the statistics that the national bodies produce are themselves trustworthy, this

2

USING STATISTICS TO ASSESS PROGRESS

27

does not mean that the version of them that people see is equally trustworthy. The problem is that most people see official statistics through the filter of the media. This means, firstly, that the statistics they see have gone through the further interpretive hands of the journalists, bloggers, or pundits presenting them, rather than directly from the national statistical office. And, secondly, that they are competing with a welter of other statistics, much of which may arise from less trustworthy sources. The difficulty is how the public can distinguish between the sources. After all, at some level the public have to take the figures on trust, and while wider education in notions of why and when to trust figures would certainly be desirable, one cannot expect everyone to be an expert on statistical analysis. A further complication is implicit in some of the discussion of Chapter 1. Many statistics are estimates, based on collecting data from a sample of the population being studied, with formal statistical inferences then being made to the whole population. While the notion of sampling was a major advance, it is inevitable that any inferences based on a sample are subject to a degree of uncertainty. Applying statistical theory and methods can help assess confidence in the statistical result, but they cannot remove this uncertainty. Worse still, there may be subtle biases and distortions in the data—of the kind discussed by Hand (2020). Failure to understand, or perhaps even record, the issues associated with estimation, and sampling in particular, can lead to two opposite kinds of misunderstandings. The first is that indicating the range of uncertainty can lead to suspicion and doubt about the numbers: the “so you mean you do not really know?” school of thought. A recent example of this arose in the UK, when the ONS announced revisions to its estimates of international migration, as part of a programme to improve the statistics by integrating a wider range of data sources. To reflect that the statistics were under development they were downgraded to “experimental statistics” rather than National Statistics (Humpherson 2019). We observed that, while some commentators applauded the ONS for seeking to improve its estimates, others castigated it, apparently not understanding the complexity of measuring time-varying data with conflicting definitions. The second kind of misunderstanding is that failure to note the uncertainty in estimates can be very misleading. Here is a magazine report on the Chancellor of the Exchequer’s Spring Statement in 2019: “Mr Hammond loves his numbers. MPs were therefore treated to the chancellor reeling off forecast after forecast, often with an absurd level

28

P. ALLIN AND D. J. HAND

of precision. By 2023, Mr Hammond claimed, the economy will have 600,000 more jobs. GDP growth that year will be 1.6% – no more, no less” (The Economist, March 16 2019). The leader writer of The Economist magazine also makes it all sound quite easy to deal with. Commenting on the economics profession, but in terms that are applicable more widely to evidence and the use of evidence, they declared that “Instead of moving cautiously, the economics profession should do what it is best at: recognise there is a problem, measure it objectively and find solutions” (The Economist, March 23 2019). All of this is an ideal. Unfortunately, one might suspect that much modern political discourse has moved beyond fidelity to the facts (but perhaps this is not new). For example, Peter Pomerantsev, a Kiev-born writer and TV producer living in London, suggests that political discourse in America—and we reckon elsewhere—is no longer “more or less” within the boundaries of reasoned argument: facts in the environment of political campaigning enhanced by social media and big data “become secondary. You are not, after all, trying to win an evidence-driven debate about ideological concepts in a public sphere of rational actors. Your aim as a propagandist is not deliberative democracy, but finding a discourse which seals in your audience … the only facts that matter are the ones that confirm existing biases … shows just how many American voters no longer felt invested in evidence-based, rational progress” (Pomerantsev 2019, p. 250). Perhaps this might change now that the US federal government is bound by the Foundations for Evidence-Based Policymaking Act (Jolin 2020). Official statistics providers are also competing with many other providers of statistics, some of whom (e.g. estate agents) collect their own data, and some of whom are themselves using official data. These latter include academia and think tanks, increasing citizen use of open data, broadcasts, and articles from commentators, and a plethora of opinions on social media. Some of this has been categorised as post truth or alternative facts.

2.2

The Role of Official Statistics

We introduced official statistics in Sect. 1.3 above, noting that they are underpinned by a set of fundamental principles. These principles are owned by the United Nations Statistical Commission, established in 1947. The Commission is

2

USING STATISTICS TO ASSESS PROGRESS

29

the highest body of the global statistical system. It brings together the Chief Statisticians from member states from around the world. It is the highest decision-making body for international statistical activities especially the setting of statistical standards, the development of concepts and methods and their implementation at the national and international level. (United Nations 2019a)

The remit of the Commission covers a huge sweep of topics and it seems hard to see it excluding statistics on anything. One might suppose that somewhere among the websites of the national statistics organisations, which are all accessible via the website of the UN’s Statistics Division, must surely be numbers which could serve as the facts about wellbeing and progress. In practice, of course, the content of these websites is determined by a variety of factors, including meeting the needs of stakeholders, compliance with international agreements, and the availability of resources. All of these reflect decisions by statisticians curating their set of national statistics. The role of official statistics in measuring the progress of nations was highlighted in the 2007 Istanbul Declaration. This identifies national statistical authorities as “key providers of relevant, reliable, timely and comparable data and the indicators required for national and international reporting” (OECD 2007, p. 1). While there have undoubtedly been advances in the measurement of progress (see Allin and Hand 2014) the fact that more remains to be done is apparent from a commitment in the United Nations agenda for sustainable development, agreed in 2015. Deep in the text, in paragraph 48, is a universal commitment to developing “broader measures of progress to complement GDP” (United Nations 2015a). The precise intent of this paragraph is not entirely clear since it is mostly about the many sustainable development indicators under development and starting to be published—to be discussed further below. Are these to be the facts about progress, broadly defined, or are there broader measures still to appear, in addition to the sustainable development indicators? The challenge in moving beyond GDP as the main, or even sole, measure of progress is illustrated by commentators such as Fioramonti (2017, p. 30), who calls GDP “the most powerful statistic ever invented. It is not just a number, but the ultimate objective of policy and a global benchmark for success”. Indeed, until the recent international commitments, only a few governments wanted measures of progress in terms of

30

P. ALLIN AND D. J. HAND

wellbeing beyond GDP: Bhutan is the country widely quoted as “having rejected the concept of GDP growth since the early 1970s” (Fioramonti 2017, p. 171) although this is a country that still calculates traditional GDP as well as national wellbeing measures. (When we looked at the website of the Bhutan National Statistics Bureau in early 2020, GDP per person was the first of the list of key indicators: http://www.nsb.gov.bt/ main/main.php#&slider1=4). There are many reasons for the lack of facts about progress more widely defined than by GDP growth. An obvious one is a lack of resources, especially where a statutory duty to provide GDP would steer priority-setting with limited budget. Moreover, while the UN Fundamental Principles may be anchored in the utility of official statistics (United Nations 2014, p. 2), in practice two things actually happen. First, the emphasis on national statistics offices as primarily the providers and publishers of trustworthy data tends to take precedence over engagement with users and potential users. But while the trustworthiness of the statistics is critical, it is pointless unless the statistics are answering the right questions— those the users want to be answered. And, second, in cases where there is engagement, responding to emerging needs tends to major on addressing the needs of government, rather than the wider public. Important as it is to meet the needs of government, this is only part of the public value of official statistics, as was confirmed by a UK parliamentary inquiry in 2019 into official statistics. Giving evidence to the inquiry, the regulator of UK official statistics spelled out how they serve the public good in two quite distinct ways, beyond “a traditional view that statistics serve the public good by providing policymakers with sound evidence on which to base their decisions”. The first of these is to meet the needs of decision-makers not just in government or the central bank, but also “a wide range of other actors: businesses, trade unions, trade associations, community groups and charities, who all make active decisions about the things that they focus on using statistics—civil society”. Second is “helping inform the perceptions that citizens have of the world and the society that they live in. These may not necessarily be the product of active study of a particular set of official statistics. They may be perceptions formed from hearing other people use statistics in the media or on social media, but they are very important, for example, in people’s understanding of the nature of the economy, living standards, levels of crime and so on” (PACAC 2019, p. 2).

2

USING STATISTICS TO ASSESS PROGRESS

31

Official statisticians in the UK and elsewhere are responding to a vision of statistics to serve government, wider civil society organisations, and citizens. The United Nations Economic Commission for Europe (UNECE 2018) has set out the value of official statistics to society and has recommended ways for national statistical institutes to promote, measure, and communicate the value of their statistics. It subsequently reported that the recommendations were being piloted in seven countries, before further methodological guidance is prepared, noting that the “few” attempts at calculating the monetary value of official statistics to society “have demonstrated that Official Statistics bring net benefits” (UNECE 2019, p. 1). We are not saying that national statistics offices fail to engage with users and potential users beyond government. There are formal processes and many informal occasions on which such engagement happens. To take an example, the system of national accounts (SNA), by which GDP is derived, is subject to strict version control, so that changes in methodology do not detract from its use to provide consistent statistics over time. When potential revisions are identified, the priority to be given to any revision is assessed against criteria including the urgency and importance of the topic “to ensure that the SNA continues to be relevant to the users” (United Nations 2009, p. 603). This phrasing, however, reduces many aspects of user engagement into a few words. How do national accountants know “the users”, how frequent and how deep is the interaction between producers and users, and how are potential users identified and contacted? That last point is particularly important: a national statistics office will know who its current users are, but what about other potential users who perhaps are unaware of the value that official statistics could have to them? We have elsewhere (Allin and Hand 2017) made the case that the development of a measure of national wellbeing and progress that is broader than GDP should build on the rigour of the SNA. Our proposal is to fit the measurement of economic performance, the focus of the SNA, within a broader assessment of national wellbeing and progress. There is already a proliferation of indicators and accounts, for example in the development of non-monetary measures of natural resources, on which to draw. Of course, there remain significant measurement challenges, and points of contention. Not the least of these is whether the “fact” that is GDP should be accompanied by (or replaced with, in the view of some) a single fact for wellbeing, say in the form of a single,

32

P. ALLIN AND D. J. HAND

overall measure or index of wellbeing. But the challenge of measurement, per se, is one thing: in our view, a more critical issue is whether the measures will actually be used. It seems to us that the starting point should be to identifying user requirements for wider measures more fully than hitherto. This would provide a firmer basis for national and crossnational developments in wellbeing accounting, rather than just national economic accounting. We envisage greater branding and marketing of national wellbeing concepts to promote measures, support their use, and stimulate further development. In Allin and Hand (2017) we called for outreach by producers, to stimulate a dialogue about the development and use of measures. In connection with this, and to demonstrate that others are aware of the challenge, we draw attention to the Missing Numbers blog curated by data scientist Anna Powell-Smith. This is described as: about the data that the government should collect and measure in the UK, but doesn’t. … areas where the government doesn’t gather data at all, whether out of wilful ignorance, or just because it doesn’t think it’s a policy priority.… These tend not to get written about much, for the obvious reason that it’s hard for journalists to report on numbers that don’t exist.… But if we don’t gather and report data, we can’t spot where services are failing; we can’t track improvement over time; and fundamentally, we can’t improve people’s lives. (Powell-Smith 2020)

That official statistics might need to be marketed may not be a popular view among official statisticians, who can see their main function as making available statistics to a high level of technical quality. While this latter is clearly critically important, we seek to encourage official statisticians to go further, not just promoting their available products but also understanding the need for them and how they fit in. Indeed, they might even shape what could be described as a “market” for information. In short, we suggest that official statisticians can provide better answers to specific questions by finding out what the questions are beforehand, and collecting pertinent rather than general data. One obvious objection to the market notion is that official statistics are not commercial products but are public goods, invariably published free of charge. We also recognise that users can sometimes make specific data requests and that these will be met free of charge if they involve marginal effort, or can otherwise be paid for. However, in our experience,

2

USING STATISTICS TO ASSESS PROGRESS

33

such requests are limited in scope and only considered by a sub-group of expert users. We advocate a more open approach, based on marketing principles (Collins 2010) suitable for developers of any product or service who might otherwise believe that the world will beat a path to their door to acquire what has been produced. In the field of restorative ecology, and elsewhere, this has been referred to as the Field of Dreams hypothesis: “if you build it, they will come”. While there is some support for this hypothesis, it is “generally assumed rather than tested” (Palmer et al. 1997, p. 295), an observation we echo in the case of official statistics. Taking more of a marketing approach in official statistics involves more interaction between producers and prospective users, to understand why statistics are needed, and how a prototype or experimental set of statistics might be used. Later there will be market research and advertising, to bring the statistic to the attention of people who might just be persuaded to be interested by it and to use it—tackling the point we made above, namely that today’s users might not be the only potential users. As we noted, this may sit a little uneasily with official statisticians who are more accustomed to deciding how to measure, and to some extent what gets measured, and then publishing the resulting statistics. In the approach that we advocate, official statisticians would need to engage with a wide range of users, certainly not merely those in government. That would conform to the apolitical role of official statisticians (and would also presumably have had some appeal to the founders of the Royal Statistical Society, producing statistics for others to interpret and to apply in decision-making, as we described in Chapter 1). But that is not the total of our recommendations. We would also expect official statisticians to be more engaged with all their users, inevitably taking statisticians out into an arena in which there is a diversity of political, commercial, societal and personal values and attitudes in play. It may well be challenging for official statisticians to steer between party-political positions while working in a way that is apolitical. We suggest that it is preferable to statisticians effectively playing a unique political role in deciding what gets included as measures of the state of the nation, its wellbeing and progress. There does not seem to be a consistent process leading to decisions about what is included in a nation’s set of official statistics, but there are sound historical reasons for this. One reason is that there are many interested parties. Of course, official statistics are not alone in lack of consistency arising for this reason. The International Classification of

34

P. ALLIN AND D. J. HAND

Diseases “defines the universe of diseases, disorders, injuries and other related health conditions, listed in a comprehensive, hierarchical fashion” and “The ICD is important because it provides a common language for reporting and monitoring diseases. This allows the world to compare and share data in a consistent and standard way – between hospitals, regions and countries and over periods of time. It facilitates the collection and storage of data for analysis and evidence-based decision-making”. (ICD11 2019) However, Bowker and Star (1999, p. 21), commenting on their investigation of the statistical principles underlying the ICD, said “… what we found was not a record of gradually increasing consensus, but a panoply of tangled and crisscrossing classification schemes held together by an increasingly harassed and sprawling international public health bureaucracy”. The reason for this is that “[t]he ICD has been as heterogeneous as possible to enable the different groups to find their own concerns reflected. Because different models of medicine hold, they embody different rules for classifying. This has resulted in the fact that, although the list is in appearance homogeneous, there are at least four classificatory principles involved” (Bowker and Star 1999, p. 150). Recognising the multiplicity of parties interested in and producing official statistics, the UK Office for Statistics Regulation has implemented a successful programme of systemic reviews, each of which looks at a particular statistical domain (e.g. health, social care, data linkage, etc.), inviting parties to work together to better integrate their statistics, to avoid duplication, overlap, and conflicting and confusing definitions (UKSA 2019). A second reason for lack of consistency about what has been included in official statistics is that many statistics are long-standing components of the information that government needs to govern, including population, housing, and health statistics, and that these requirements might be set at different levels: local government, national government, and the supranational level (for example in the European Union). We remarked earlier that the system of national accounts, in the broad form it has today, was initially used during the Second World War to help plan industrial production and the war effort. The key headline measure of GDP in the national accounts, has emerged, as we have noted, as the dominant measure for assessing the progress of a country. This suggests one of two things happened. First, the national accounts have been used to manage the national economy by agreed objectives and key results, and so were an early example of the management approach now called

2

USING STATISTICS TO ASSESS PROGRESS

35

“Measure What Matters” (Doerr 2018). Second, society has been guided into focusing on GDP growth because that is the dominant measure of national progress and people might well assume that what matters must that which is measured (recall again that “what gets measured gets done”). Fioramonti, for one, notes that “Our entire development model rests on the way in which we measure prosperity, development and ultimately success”. He continues, warning that a global order based on GDP growth is driving “a suicidal race to the cliff, which imposes stressful lifestyles, generates irrational desires and threatens to tear the world apart, while undermining the very social and natural foundations that make life possible” (Fioramonti 2017, p. 7). There have been many calls over recent years to go beyond GDP, not only in what we measure but more generally in how we live our lives. Not all of them appear to have started by asking what matters most, with the result that they come up with statistics describing the way that the organisation producing the new measure sees things, without at least confirming that these are the things that matter to others. When it comes to seeking, publishing, and using wider measures of progress than GDP, we have already welcomed (Allin and Hand 2014, p. 162) a practical guide to developing societal progress indicators prepared by the OECD Statistics Directorate (Trewin and Hall 2010). This has, as the first step, to determine what matters most to a society. For the remainder of this chapter we are particularly interested in how, or if, people, organisations and governments (outside of national statistical organisations) influence which statistics are collected. We will look at two developments that stand out as landmarks in the journey beyond GDP, and particularly how those developments tackle the choice of measures. First, the report in 2009 by the Commission on the Measurement of Economic Performance and Social Progress (often referred to by the names of the three lead authors, Stiglitz, Sen, and Fitoussi). Second, the UN’s 2030 Agenda, agreed in 2015 and containing the Sustainable Development Goals. In the following two chapters of the book we will explore how, or even if, official statistics influence people, organisations, and governments.

2.3

The Stiglitz, Sen, Fitoussi Report

The “Stiglitz Report”, the report of the Commission on the Measurement of Economic Performance and Social Progress, was commissioned by the then President of France, Nicolas Sarkozy in early 2008, and was

36

P. ALLIN AND D. J. HAND

presented in September 2009. It had the objective of aligning “better the metrics of wellbeing with what actually contributes to quality of life, and in doing so, to help all of us to direct efforts to those things that really matter” (Stiglitz et al. 2010, p. xvii). In the foreword to the report, Sarkozy wrote: … we will not change our behaviour unless we change the ways we measure our economic performance … If we do not want our future and the future of our children and grandchildren to be riddled with financial, economic, social, and environmental disasters, which are ultimately human disasters, we must change the way we live, consume, and produce. We must change the criteria governing our social organisations and our public policies.… Our statistics and our accounts reflect our aspirations, the values that we assign things.… Treating these as objective data, as if they are external to us, beyond question or dispute, is undoubtedly reassuring and comfortable, but it’s dangerous. It is dangerous because we get to the point where we stop asking ourselves about the purpose of what we are doing, what we are actually measuring, and what lessons we need to draw. (Stiglitz et al., 2010, p. vii)

Sarkozy noted the gap between the rosy economic statistics (just before the financial crash) and the hidden truth of pollution, climate change, and other adverse consequences of measuring progress by narrow measures such as GDP. And he commented that the world had changed, and that the measures of progress had not kept pace. Of course, the report was not intended merely for France. As Sarkozy said, France will “put debate on this report’s conclusions on the agenda of every international meeting …” and will “strive to get all the international organisations to modify their statistical systems in accordance with the commission’s recommendations” (p. x). In the preface to the report, its three authors, Joseph Stiglitz, Amartya Sen, and Jean-Paul Fitoussi stressed how we see the world through the statistics we use to measure it—with the obvious implications and consequences when the major such statistics are economic ones, and in particular GDP. They cited examples of countries with increasing GDP but decreasing life expectancy and declining personal income. Statisticians go to great pains to ensure that their summary statistics are not biased, that they validly and reliably reflect the underlying reality being measured, and that their (inevitable) random measurement error is

2

USING STATISTICS TO ASSESS PROGRESS

37

not too great. But none of this sophisticated technical expertise suffices if the statistics are fundamentally measuring the wrong thing. It was certainly true that the Stiglitz report had impact, and contributed to the growth of interest in wider measures throughout the world. In the UK “[t]he recommendations of the Stiglitz Commission therefore fell on fertile soil …” with the then Prime Minister David Cameron “fulfilling a long-cherished promise when he launched a large-scale initiative in this domain in November 2010” (Kroll 2011). The report made twelve recommendations, five concerned with classical GDP issues, five with broader quality of life, and two with sustainable development and the environment. It was ironic that, between the time at which the report was commissioned and the time it was presented the financial crisis occurred— described in the report as “one of the worst financial, economic and social crises in post-war history” (Stiglitz et al. 2010, p. 4). (This was before the coronavirus pandemic). The report said: The reforms in measurement recommended by the Commission would be highly desirable, even if we had not had the crisis. But some members of the Commission believe that the crisis provides heightened urgency to these reforms. They believe that one of the reasons why the crisis took many by surprise is that our measurement system failed us and/or market participants and government officials were not focusing on the right set of statistical indicators. (Stiglitz et al. 2010, p. 4)

However, it is now over ten years since the report appeared, and while the construction and implementation of measures of national progress and wellbeing alternative to GDP has continued apace, there is little evidence of them yet having an impact. This could be simply a matter of the time required to break down the entrenched perspectives. If so, it is not the only domain where change is slow: the philosopher Thomas Kuhn wrote “[c]onversions [of scientists to new theories] will occur a few at a time until, after the last holdouts have died, the whole profession will again be practicing under a single, but now a different paradigm” (Kuhn 1962, p. 152). On the other hand, it is possible that the very crisis itself has slowed down the adoption of alternative measures, while governments have struggled to extricate their countries from the crisis.

38

P. ALLIN AND D. J. HAND

2.4

The Sustainable Development Goals

The Sustainable Development Goals [SDGs] are the blueprint to achieve a better and more sustainable future for all. They address the global challenges we face, including those related to poverty, inequality, climate, environmental degradation, prosperity, and peace and justice. (United Nations, 2019b)

The SDGs were introduced during the United Nations Conference on Sustainable Development in Brazil in 2012. They replaced and built on the earlier Millennium Development Goals (MDGs) which focused on global poverty. They were adopted by the General Assembly of the UN on 25 September 2015 with the title “Transforming our world: the 2030 Agenda for Sustainable Development”. There are 17 Sustainable Development Goals (SDGs) and 169 targets. Each target has up to 3 indicators to assess progress towards the target (up from 8 goals and 60 indicators for the MDGs). The indicators are described as “integrated and indivisible” (United Nations 2015a, para 5). The goals are certainly wide-ranging, going far beyond GDP, and including things as ambitious as ending poverty in all its forms everywhere, building resilient infrastructure, and reducing inequality within and among countries. Wellbeing is explicitly mentioned in Goal 3: Ensure healthy lives and promote wellbeing for all at all ages. Of course, macroeconomic considerations do appear: Goal 8 is to promote sustained, inclusive, and sustainable economic growth, full and productive employment, and decent work for all. As one might expect, the goals are not without their critics. Apart from the scale of ambition (surely not all are achievable), some people see them as requiring difficult balances, possibly even having incoherencies (e.g. promoting economic growth and at the same time inequality reduction; increasing wages while reducing the cost of living and sustaining ecosystems). The obvious criticism that 169 targets were far too many has also been made. Certainly, a corporation which had that many “key performance indicators” would struggle to meet them all, and even to decide on a coherent direction in the face of such an abundance. They have also been criticised as ignoring local context—even though, as characterised by Felix Dodds (n.d., p. 1), “Unlike 2000 where the MDGs were

2

USING STATISTICS TO ASSESS PROGRESS

39

predominately brought forward by developed countries, the UN has set up a process to enable all countries and stakeholders to participate”. Latent beneath the use of these goals and their associated indicators is the assumption that individual countries have sufficiently effective statistical offices to measure them, collecting accurate and reliable data, analysing it, and publishing the results. In some cases, substantial capacity building is needed. Having said that, there does seem to be a strongly held view among statisticians working in international development that targets and indicators can have profound effects. For example, the Partnership in Statistics for Development in the twenty-first century (PARIS21) works to promote “the better use and production of statistics throughout the developing world. Since its establishment in 1999, PARIS21 has successfully developed a worldwide network of statisticians, policy makers, analysts, and development practitioners committed to evidence-based decision making” (PARIS21 2019, p. 3). Initiatives like this have as their main objective the achievement of national and international development goals. But it has to be said that it is not clear that these effects go beyond raising awareness, important though that is, and do actually help improve people’s lives. Moving from data to action, in the form of changed behaviour by individuals and businesses, is assumed to involve the consideration of evidence and decisions informed by evidence. But how this actually happens, and the extent to which it happens, is not understood. The UN’s 2030 agenda refers only in general terms to how the indicators are to be used, for example that “Quality, accessible, timely and reliable disaggregated data will be needed to help with the measurement of progress and to ensure that no one is left behind. Such data is key to decision-making” (United Nations 2015a, para 48). While there has been an extensive and inclusive process to develop the indicators to support the SDGs, primarily through an inter-agency expert group, that group only started meeting in 2015 (United Nations 2015b), just ahead of the launch of the UN Agenda. In narrating how the SDGs were negotiated, Dodds et al. (2017, p. 125) show that statisticians were intentionally unable to influence the choice of the targets and goals, which was a political process played out at national, regional, and international levels. They recall that in early 2015, “Now that heads of state have adopted the SDGs, one of the significant issues that needed to be finally addressed was the issue of what indicators there should be

40

P. ALLIN AND D. J. HAND

for the targets … in March 2015, an open and transparent process to look for the relevant indicators for the targets was set out”. Selection criteria and processes were soon determined. The challenge was in finding methods, new sources, and resources to define suitable indicators—and then to publish timely, sound and trusted data. Work on the indicators in the UK is being led by the ONS. It ran a public consultation in the summer of 2017 on its approach to reporting and how to fill the gaps. Despite having an established and extensive system of official statistics, the UK is currently (early 2020) still exploring data sources for almost 1 in 4 (23%) of the 244 indicators (ONS n.d.). Many other countries are finding the task even more difficult. We can see that there is a case for deciding political goals without worrying about the cost and feasibility of measuring progress towards them. On the other hand, there must be some scope for having statisticians involved in goal setting, helping inform decision-making and building a clearer understanding of how indicators are to be used. Without some practical input, the exercise could be markedly hampered. The indicator data that is available around the world is being collated and published on UN websites and in UN reports (e.g. United Nations 2019c). Individual country data is also available from national statistics offices and from organisations collating global data (e.g. https://sdg-tra cker.org/). As a stepping stone towards 2030, each country is expected to review progress towards the SDGs at least once, and to report this to the United Nations. The UK was one of 50 countries due to report in 2019— see Voluntary Review (2019). It notes that one of the three principles adopted in delivering the review, and presumably the plan, is that it is “underpinned by data” (p. 7). It also records that information and data was collected from a range of sources, not only the global indicators data from ONS, but also other government and non-government sources, as well as government plans, annual reports, and accounts. Following the global agreement on the SDGs, the UK government published a document (DfID 2017) to detail how it would support the delivery of the SDGs, both domestically and internationally. Two years later, the government reported that all of the SDGs are “now reflected throughout the UK government’s [overall] programme of work collectively delivering activity on social, economic, and environmental issues. Each UK government department has embedded the Goals in its Single Departmental Plan – an established process to focus government efforts

2

USING STATISTICS TO ASSESS PROGRESS

41

on important issues” (UK Voluntary Review 2019, p. 7). Thus, the ambitions that the UK government has for achieving the SDGs are now central to its work plan (UK Government 2019), which also embraces getting the best Brexit deal for Britain (the reader will be able to judge how that worked out), making the economy work for everyone (raising deep measurement issues), and five other points including tackling injustices, wherever they exist in our society, and keeping our families, communities, and country safe—all ambitious aspirations. Delivering the SDGs is not just for governments and policy-makers. It is recognised that “we [must] act now and act together” (e.g. United Nations 2019c, p. 3), where “we” includes businesses, civil society, national and international organisations, and individuals. This is something that we (the authors) will expand on in the following chapters. One implication of this collective responsibility is that the audience for the SDG indicators is potentially huge and diverse. Nevertheless, there are ways for statistical offices to reach out, especially through an increasing number of communities of interest in the SDGs. One such community within the UK, for example is found at https:// www.ukssd.co.uk/. This has a range of projects, undertakes reviews and encourages “partner” organisations to act in ways that should deliver the SDGs. The UKSSD network (2020) wrote to the UK Prime Minister, noting that there were now ten years left until the SDG deadline and calling on him to “Help the UK understand its performance on the SDGs by openly disclosing progress towards the targets”.

2.5

Putting Official Statistics to Use

Compiling and publishing statistics is all very well, but statistics are not an end in themselves. Statistics are intended to reflect underlying conditions, but then, hopefully, they are used to monitor and guide changes. So, we might ask, what consequences do official statistics have? For example, assuming that the 244 indicators for the SDGs are calculated for each country and published regularly, will that actually lead to sustainable development? Or is there a risk of meeting the abstract statistical SDGs, rather than establishing developments that are genuinely sustainable, in the Brundtland sense of respecting future generations? Moreover, the approach to using indicators that appears to be envisaged is primarily one of assessing progress towards the goals, but we must be aware of the

42

P. ALLIN AND D. J. HAND

temptation and opportunities for gaming and manipulation of statistics, subconscious though those might be. In addition, different degrees of detail might be needed for different uses, such as describing the current state of a country, measuring a change of that state (requiring that change be detectable and measurable), and actionability in policy terms, as well as enabling comparative communication between countries. We must also ask what is, or should be, in place to help ensure that the statistical indicators are accepted as an authoritative picture of the world. “Authoritative” must include notions of trustworthiness, as discussed above, as well as transparency, inclusivity, and, at bottom, utility; the ultimate measure of value. While GDP is part of a complex system of national accounts, it also functions very much as the single headline measure, and we should likewise ask whether progress and sustainable development can be (or should be) expressed as a single index number. It is possible that the two opposing views expressed on this—“cannot be expressed as a single number” and “should be expressed as a single number”—might best be addressed by some sort of compromise. In their report, Stiglitz et al. (2010, p. xxv) wrote: “There is no single indicator that can capture something as complex as our society”. While this is clearly an obvious truism, in that the multiple facets of a multivariate object cannot be captured by a single indicator, perhaps a single aggregate indicator might be derived which is useful for our purposes, albeit with acknowledged weaknesses. We discussed such matters in depth in the context of the multivariate nature of national wellbeing in Allin and Hand (2014), contrasting a dashboard of indicators with a single derived indicator. Since producing 244 indicators across the 196 countries in the UN is a non-trivial and costly process, such questions need to be asked—and answered—before full-scale production of new measures and indicators gets under way. It appears that this has been done only in high-level terms, leaving as yet unanswered questions about the full applicability and value of the indicators. That these questions have force is demonstrated by the example of social progress. Although social progress is a dimension of sustainable development, there is already a long-standing tradition of publishing robust social statistics and social monitors that should provide examples and pointers to help reflect on the development and use of indicators for the SDGs. This stretches back to the origins of formal statistical methods

2

USING STATISTICS TO ASSESS PROGRESS

43

with the work of Arthur Bowley, Adolphe Quetelet, William Farr, and others. The social indicator movement has been a consistent force for over 60 years now and has been fully embraced by the European Statistical System, comprising national statistical offices and the statistical office of the European Union. Cantillon (2017, p. 585) has commented on the significance of the developments in European social indicators “to understand and monitor social progress, put the indicators into perspective, and present innovative ways for their improvement and enrichment”. The indicators and the surveys providing data are sophisticated and accurate, but the considerable effort made in compiling social indicators across Europe has in fact not been rewarded with progress towards meeting poverty and social exclusion goals. Moreover, new aspects of poverty, such as an increase in rough sleeping, are reported in the media (and are apparent on walking through many towns and cities). The picture is complicated because of the disruption caused by the financial crisis. Nevertheless, Cantillon (2017, p. 593) concludes that little if any progress has been made in combating relative financial poverty and social exclusion, despite “remarkable improvements in European output governance, including measurement, goal setting, and monitoring” and increasing use of the social indicators by researchers and by European Commission policy-makers. There seems to be a gap that is hard to bridge, between using indicators to observe and using indicators actually to make a difference. Most work is done in observation mode, monitoring levels and distributions of poverty, and evaluating policy measures that have been put in place to tackle poverty. Perhaps we are expecting too much of indicators and the clue is in the name: are indicators simply to be indicative, helping to steer fuller investigations of affluence and poverty, and of social progress more generally? Clearly there is a literature of such scholarly studies (e.g. Bosco and Poggi 2020), although these are characterised by Cantillon (2017, p. 593) as “complicated and time-consuming micro-simulation modelling”, and the results need to be translated into material of use to policy-makers and for those designing and delivering programmes to reduce poverty. One weakness of social indicators is that they have tended to be about outcomes, such as the number of people in employment. Cantillon (2017, p. 594) concludes “This has meant that the link between ‘goals’ - as measured by the portfolio of social indicators - and ‘policies’ has remained vague and unarticulated while difficult trade-offs (e.g. between work and

44

P. ALLIN AND D. J. HAND

poverty reduction) have not been made explicit”. Reassuringly, there are signs that new, broader indicators are in development and are being crafted with a view to their use in policy. For example, an independent commission has come up with a new poverty measurement framework for the UK. It is seeking broad, longterm political support for the measurement framework and its use in policy-making. In its 2019 report, the Social Metrics Commission (2019, p. 4) welcomed an announcement from the government that “the Department for Work and Pensions would be developing experimental statistics based on the Commission’s measurement approach”. The Commission also recognised that “its work is only the start of what needs to happen” with further development of the framework, collection of new data based on improved survey and administrative data. The Commission is committed “to work with the widest range of stakeholders possible to ensure that, once fully developed, the Commission’s measurement framework can form the basis of a consensus view on poverty measurement across the Government, the ONS, policymakers and those researching and working with people in poverty” (p. 5). Without detracting from the value of reaching consensus on the measurement of poverty, we should recognise that measuring social conditions is complex, detailed, and demanding. As Hills et al. (2009, p. 358) concluded, for example, answering the question of whether the UK had become a more equal society since 1997 “depends on which inequalities are being examined, between whom, and over which time period”. They looked across a range of available official indicators, finding that “trends improved after 1997 compared with the period before for nearly half of them, but they deteriorated for a quarter”. While there were some significant policy initiatives from 1997 onwards, “the scale of the action was often small in relation to the underlying inequalities, and the momentum gained by the middle of the period had often been lost by the end of it. Problems were often harder to tackle than the government appears originally to have assumed, and less amenable to a one-off fix”. The conclusion we draw from this review of social indicators is that the development of any system of indicators needs to be based on a full understanding of what users and potential users need and how the indicators are to be used in a sustained way. This may well include a dialogue about how new indicators can and should be used, and about things that might be problematic in using them. We are back to our point about the

2

USING STATISTICS TO ASSESS PROGRESS

45

critical nature of extensive user engagement at an early stage of development. As part of this, more emphasis needs to be placed on targets and on reporting the gap between the measured value and the target. Of course, necessary statistical capacity needs to be in place, along with good communication skills, but that is not sufficient. There is also the issue of analytical skills among users, which needs to be assessed, perhaps enhanced, so that statistical outputs can be designed to meet a range of needs and capabilities. We have set out how this might work in terms of new measures of progress and development, proposing that “the system of national accounts should evolve into a process of national wellbeing accounting” (Allin and Hand 2017). Two final points about usage should be made. The first is that usage can be prescribed in legislation. The Wellbeing of Future Generations (Wales) Act (Wales 2019) is an example of this, which we will return to in Chapter 3. The second point is that a rich body of comparable and regularly produced statistics has intrinsic value. While naturally we are concerned with the usability and usefulness of the statistics for particular purposes and the purposes for which they are created, we should not lose sight of the wider picture they give of the world in the Twenty-First Century—see, for example, Our World in Data (https://ourworldindata. org/).

2.6

Conclusion

Measures of social progress and of sustainable wellbeing continue to appear, most recently driven by the imperative to meet the SDGs by 2030. We have drawn on the experience of the development and use of European social indicators over the last half century to enter a note of caution about the apparent assumption that introducing new measures is all that is needed. As Cantillon (2017, p. 594) concluded, “although in some countries the ‘numbers’ became part of the public debate, their impact in national politics and social dialogue remained limited”. It is also clear that there is no consensus on how to measure wellbeing and progress among official statisticians. We have briefly referred to some of the emerging guidelines and recommendations and we acknowledge that international organisations such as OECD are facilitating and promoting new standards. However, the take-away message from this chapter is that measures of progress need to be anchored in their use in the real world. Considerable effort is going into the construction of a

46

P. ALLIN AND D. J. HAND

large set of indicators and finding the numbers for each indicator in each country. But building such an edifice does not mean that people will use it, or even find it. We stress the importance of assessing the quality of indicators in a way that emphasises fitness for purpose as well as technical accuracy, trustworthiness, and reliability. Indicators need to be relevant and meaningful to individuals and businesses as well as to governments. They also need to be designed in a way that recognises the challenge that progress is about where we are headed to, not just where we are today. The key issue is how the statistics are to be used, and who the intended users are. We see the role of national statistics offices not just as providers of data, but also interpreters and promoters. This is what we will explore further in the next two chapters, looking first at the use of wider measures of wellbeing, progress, and sustainable development in public policy and then at their use by companies, individuals, and civil society.

References Allin, P., & Hand, D. J. (2014). The Wellbeing of Nations: Meaning, Motive, and Measurement. Chichester: Wiley. Allin, P., & Hand, D. J. (2017). From a System of National Accounts to a Process of National Wellbeing Accounting. International Statistical Review, 85, 355–370. Bosco, B., & Poggi, A. (2020). Middle Class, Government Effectiveness and Poverty in the EU: A Dynamic Multilevel Analysis. Review of Income and Wealth, 66(1), 94–125. Bowker, G. C., & Star, S. L. (1999). Sorting Things Out: Classification and Its Consequences. Cambridge, MA: MIT Press. Cantillon, B. (2017). Monitoring Social Inclusion in Europe. Review of Income and Wealth, 63(3), 585–596. Collins, M. (2010). Building a Better Mousetrap Isn’t Enough. https://salesa ndmarketing.com/article/building-better-mousetrap-isnt-enough. Accessed 4 March 2019. Condon, K. M. (2017). Interview with Asta Manninen. Statistical Journal of the IAOS, 33, 567–573. DfID. (2017). Agenda 2030: The UK Government’s approach to delivering the Global Goals for Sustainable Development —At Home and Around the World. https://assets.publishing.service.gov.uk/government/uploads/system/upl oads/attachment_data/file/603500/Agenda-2030-Report4.pdf. Accessed 26 August 2019.

2

USING STATISTICS TO ASSESS PROGRESS

47

Dodds, F. (n.d.) https://unep.ch/natcom/assets/publications/2015_develop ment_goals_landscap_felix.pdf. Accessed 26 August 2019. Dodds, F., Donoghue, D., & Leiva Roesch, J. (2017). Negotiating the Sustainable Development Goals. Abingdon: Routledge. Doerr, J. (2018). Measure What Matters. UK: Penguin Business. Fioramonti, L. (2017). The World After GDP. Cambridge: Polity Press. Hand, D. J. (2020). Dark Data: Why What You Don’t Know Matters. Princeton: Princeton University Press. Hills, J., Sefton, T., & Stewart, K. (2009). Towards a More Equal Society? Poverty, Inequality and Policy Since 1997 . Bristol: Policy Press. Humpherson, E. (2019). Response on Reclassification of Migration Statistics. https://www.statisticsauthority.gov.uk/correspondence/response-on-rec lassification-of-migration-statistics/. Accessed 21 May 2020. ICD11. (2019). https://www.who.int/classifications/icd/en/. Accessed 23 August 2019. Jolin, M. (2020). Yes, There’s Hope for Good Government in the U.S. https://www.alliance4usefulevidence.org/yes-theres-hope-for-good-gov ernment-in-the-u-s/. Accessed 18 April 2020. Kroll, C. (2011). Measuring Progress and Well-being: Achievements and Challenges of a New Global Movement. Friedrich-Ebert-Stiftung, International Policy Analysis, Berlin. https://library.fes.de/pdf-files/id/ipa/08509.pdf. Kuhn, T. (1962). The Structure of Scientific Revolutions (2nd ed.). Chicago: University of Chicago Press. Mosey, R. (2018). Off the Air. New Statesman 4–10 May 2018, 22. OECD. (2007). The Istanbul Declaration. https://www.oecd.org/site/worldf orum/49130123.pdf. Accessed 9 March 2020. ONS. (n.d.). https://sustainabledevelopment-uk.github.io/reporting-status/. Accessed 10 March 2020. PACAC. (2019). http://data.parliament.uk/writtenevidence/committeeevi dence.svc/evidencedocument/public-administration-and-constitutional-aff airs-committee/governance-of-official-statistics/oral/99222.pdf. Accessed 30 August 2019. Palmer, M. A., Ambrose, R. F., & Poff, N. L. (1997). Ecological Theory and Community Restoration Ecology. Restoration Ecology, 5(4), 291–300. PARIS21. (2019). Partner Report on Support to Statistics 2019. http://paris21. org/press2019. Accessed 9 March 2020. Pomerantsev, P. (2019). Normalnost. Granta, 146, 239–253. Powell-Smith, A. (2020). https://missingnumbers.org/about/. Accessed 23 May 2020. Social Metrics Commission. (2019). Measuring Poverty 2019. https://socialmet ricscommission.org.uk/wp-content/uploads/2019/07/SMC_measuring-pov erty-201908_full-report.pdf. Accessed 22 May 2020.

48

P. ALLIN AND D. J. HAND

Stiglitz, J. E., Sen, S., & Fitoussi, J.-P. (2010). Mismeasuring Our Lives: Why GDP doesn’t Add Up. New York: The New Press. Trewin, D., & Hall, J. (2010). Developing Societal Progress Indicators: A Practical Guide. OECD Statistics Working Papers, 2010/06. OECD Publishing, Paris. http://dx.doi.org/10.1787/5kghzxp6k7g0-en. UNECE. (2018). Recommendations for Promoting, Measuring and Communicating the Value of Official Statistics. United Nations, Geneva. http://www. unece.org/index.php?id=51139. Accessed 9 March 2020. UNECE. (2019). What are Statistics Worth? Determining the Value of Official Statistics. http://www.unece.org/?id=51381. Accessed 9 March 2020. United Nations. (2009). System of National Accounts 2008. https://unstats.un. org/unsd/nationalaccount/docs/SNA2008.pdf. Accessed 9 March 2020. United Nations. (2014). Fundamental Principles of Official Statistics. https:// unstats.un.org/unsd/dnss/gp/FP-New-E.pdf. Accessed 9 March 2020. United Nations. (2015a). Transforming Our World: The 2030 Agenda for Sustainable Development. https://sustainabledevelopment.un.org/post2015/ transformingourworld. Accessed 26 August 2019. United Nations. (2015b). First Meeting of the IAEG-SDGs. https://unstats.un. org/sdgs/meetings/iaeg-sdgs-meeting-01/. Accessed 9 March 2020. United Nations. (2019a). https://unstats.un.org/unsd/statcom. Accessed 21 August 2019. United Nations. (2019b). https://www.un.org/sustainabledevelopment/sustai nable-development-goals/. Accessed 23 August 2019. United Nations. (2019c). The Sustainable Development Goals Report 2019. https://unstats.un.org/sdgs/report/2019/the-sustainable-developmentgoals-report-2019.pdf. Accessed 10 March 2020. UK Government. (2019). https://www.gov.uk/government/collections/acountry-that-works-for-everyone-the-governments-plan. Accessed 26 August 2019. UKSA. (2019). https://www.statisticsauthority.gov.uk/osr/what-we-do/sys temic-reviews/. Accessed 23 August 2019. UKSSD. (2020). https://www.ukssd.co.uk/brief-to-pm-boris-johnson. Accessed 12 March 2020. UK Voluntary Review. (2019). https://www.gov.uk/government/publications/ uks-voluntary-national-review-of-the-sustainable-development-goals. Accessed 26 August 2019. Wales. (2019). https://gov.wales/well-being-future-generations-wales-act-2015guidance. Accessed 31 August 2019.

CHAPTER 3

Statistics and Public Policy

Abstract In this chapter we major on the role of official statistics in policy, looking especially at measures of wellbeing. Using the UK as an example, we discuss policy-making as a system involving officials and politicians. Official statistics are only part of the evidence base that might be used to inform policy-making. Evidence has to be pushed towards policy-makers and politicians, as well as pulled by them. We note the role of intermediaries such as What Works Centres and at ways of improving the use of evidence in government policy-making. On wellbeing policy in particular, we draw a distinction between a wellbeing measurement policy and policies that aim to improve wellbeing. We review the approach to wellbeing in UK government policy since the launch of the wellbeing measurement programme in 2010 and contrast this with the wellbeing economy approach adopted in devolved administrations within the UK (and elsewhere). The evidence about how wellbeing evidence is used in policy is patchy but we do identify a role for official statistics, albeit one that needs nurturing. We are well aware that, when it comes to achieving major policy outcomes, “governments can’t do it alone” and we will explore the implications of this for official statistics in the following chapters. Keywords Official statistics · Evidence-informed policy · Evidence translators · Social value · Wellbeing economy

© The Author(s) 2021 P. Allin and D. J. Hand, From GDP to Sustainable Wellbeing, Wellbeing in Politics and Policy, https://doi.org/10.1007/978-3-030-53085-3_3

49

50

P. ALLIN AND D. J. HAND

3.1

Introduction

Much has already been written about the politics and policy of wellbeing; more will undoubtedly appear. (This book is one of a series about wellbeing in politics and policy). Most, if not all, of what has been written about UK developments has discussed measurement issues to some extent. This has included reviewing the measuring national wellbeing programme and the now regular publications about national wellbeing from the Office for National Statistics, as well as developments in the devolved administrations of the UK. Our aim in this book is nevertheless to put the spotlight on the role of wellbeing measures. However, our focus is on the utility of wellbeing measures: what uses are they designed to meet and how are they being used? We are particularly interested in wellbeing measures as official statistics and seek to explore further what that means in terms of their role and usage. This is the essence of the issue, as we see it, in assessing the quality of official statistics: do they overall produce a recognisable, coherent, and usable picture of society, going beyond their value as a set of individual components judged only on their technical quality (the three components of quality, value, and trustworthiness defined by the UK’s Office for Statistics Regulation mentioned in Chapter 1)? Do they help people find out what they want to know about? Official statistics are prescribed in legislation in terms of the bodies and organisations that produce them. The definition of official statistics adopted in the UK code is that government statistics produced in compliance with the code are called “official statistics” (OSR 2018, p. 6). This allows for a wider range of data and statistics to be seen as official statistics, if the organisations providing them have voluntarily adopted the code. If this all seems a little bureaucratic, it might be better to take a step back and remember the words of the UN’s fundamental principles of official statistics, rather than how the principles are interpreted and operationalised in any one country. As we noted in Chapter 1, the UN principles call for high quality information “in support of sustainable development, peace and security”, as an “indispensable element in the information system of a democratic society, serving the Government, the economy and the public with data about the economic, demographic, social and environmental situation” (United Nations 2014). The UN principles put great weight on the practical utility of the official statistics that are to be produced. Understanding fitness for purpose

3

STATISTICS AND PUBLIC POLICY

51

is generally part of the assessment of the overall quality of any product or service. However, the general objective underlying many official statistics makes the test of fitness for purpose rather opaque. While statistics might be produced to describe a particular economic, social, or environmental topic this is often without a clear understanding of the questions that need to be addressed or the analyses into which the statistics will be incorporated. This is different from the provision of many goods and services, which are designed to meet specific needs (including sometimes those that potential users do not realise that they have). Official statisticians often know little about the uses and the users of the statistics they produce, apart from specific uses within the government that frequently in turn gave rise to the data from which the statistics were generated. The parliamentary inquiry in 2019 into UK official statistics mentioned in Chapter 2 concluded that they “are used for a variety of purposes by many different audiences” and quoted evidence from the Statistics User Forum of examples of the breadth of uses for statistics: “Whether it is management decisions in the health service, targeting crime prevention initiatives, identifying the most deprived and vulnerable communities, or in business investment, marketing, or other commercial decision-making, official statistics are used way beyond the world of policy” (PACAC 2019, para 15). However, the inquiry also discovered that “it is not well understood why people use statistics and how, and why some people do not use them”, due to a lack of any systematic analysis or understanding of usage and users (PACAC 2019, para 20). Welcome steps are now being taken to address this in the UK. However, it is such a fundamental concern, and one that no doubt applies in many countries, that we see this as a key point of this book: to raise the profile of official statistics and to help promote their wider and wiser use. One complication here is a mis-match between the set of official statistics, as officially defined, and what we might call public statistics, or statistics for the public good. We observe that there are public statistics that are perceived to be official statistics, whether or not they strictly are, simply because they are quoted or used widely to describe key aspects of the economy or society. In the UK, for example, many of the services once provided by public utilities, such as power and water supplies and public transport, are now delivered by private companies, yet statistics on their performance are still likely to be considered as official statistics, though they are not. Any lack of understanding about the nature of official statistics may well be greater among users and potential users of

52

P. ALLIN AND D. J. HAND

official statistics outside of government departments, who we will consider more fully in the next chapter, than among policy-makers. For example, what is someone interested in buying a new car supposed to make of an advert from a well-known car manufacturer presenting what it calls “official fuel consumption figures” for a new range of vehicles? Are these figures produced by the manufacturer, albeit to “official” tests, or are they produced by an independent body? If so, is that organisation designated as a producer of official statistics? Judging the quality and trustworthiness of such figures is not easy for anyone other than those with a detailed knowledge of this aspect of consumer protection. It is our understanding that policy officials routinely receive relevant statistics, say on the level and distribution of unemployment, but it is our impression that there are many officials who do not otherwise reach for statistics. Awareness of relevant statistics, especially those produced elsewhere in government, will need refreshing from time to time, to take account of staff turnover. We have no firm evidence, but it seems likely that there was a delay before UK policy-makers started to become aware that there are official statistics on personal wellbeing produced by the Office for National Statistics, a non-ministerial government department reporting to Parliament via the UK Statistics Authority. This all suggests the value of building statistical awareness skills in policy-makers and ensuring that there are teams within policy departments responsible for sourcing appropriate evidence. In this chapter we look at the use of wellbeing measures by policymakers and politicians. We will look at the use of wellbeing measures by businesses, the public, and civil society in the next chapter, not as a secondary issue but because it seems to us a given that policy is necessary (but not sufficient) to bring about changes in how we live our lives and in how we look after the planet, its resources, and its climate. This is how things happen in a modern democracy. Politics and policy provide impetus and frameworks, but change only comes about if enough people and organisations recognise the need to change and do indeed follow the subsequent policy direction. Policy is invariably delivered top-down. While policy might respond to public concern and media pressure, policy formulation is undertaken within government, albeit with some consultation with others. Apart from policy relating to the delivery of public services, the implementation of policy is effectively handed down to businesses, the public, and others, who actually do (or do not do) something as a result of the policy.

3

3.2

STATISTICS AND PUBLIC POLICY

53

Policy-Making in General

In academic literature, policy-making is often described as a process, with its overarching policy goals, various ways to seek to reach these goals, and mechanisms for choosing the preferred option (e.g. Hall 1993, p. 278). This can lead to the notion that public policy-making takes place in one black box according to a single plan. Practitioners know it is far from that. Instead, they recognise that policy-making is often highly specific and decentralised. Policy-making takes place in a complex system made up of many teams in different departments, working more or less together through a number of formal and less formal networks. Some of these teams and networks process and circulate statistics and other evidence. Impetus for policy change, however, invariably arises from other drivers and prompts, such as political will, election manifestos, international treaties, public and business concerns, and media commentary. Policy outputs might be new or amended legislation, strategies, further programmes of work, or a decision to do nothing new and so continue with the existing situation. Part of policy work as adopted in the UK government is usually summarised as ROAMEF, shorthand for a sequence of consecutive analytical stages: Rationale, Objectives, Appraisal of options, Monitoring of implementation, Evaluation, and Feedback (HM Treasury 2018, p. 9). The starting point is to state clearly why change from the current “business as usual” is necessary and that some policy intervention is needed to address this. The Treasury guidance notes that “The rationale for intervention can be based on strategic objectives, improvements to existing policy, market failure or distributional objectives that the government wishes to meet” (HM Treasury 2018, p. 13). Clear objectives then need to be established, such as, for example, the changes in outcomes the policy is designed to produce. One of the five key attributes of objectives is that they should be measurable. The next stage is to draw up a list of possible options that can achieve the objectives and to select the option by whittling down the list, including by appraising the costs, benefits and effectiveness of each short-listed option. This analytical work is embedded in other dimensions to policy-making, such as opening-up debate, say by publishing a Green Paper to set out tentative proposals and invoke discussion. This may be followed by firmer proposals, draft legislation, and stages of parliamentary scrutiny.

54

P. ALLIN AND D. J. HAND

There is a clear distinction between the role of policy officials, who come up with options, and government ministers who make decisions on the evidence and the options. These are political choices, which may be based on ideology, commitment to a manifesto, expediency, public opinion, and a view on practicality. When choices have been made, the policy cycle runs on to implementing the preferred option, then monitoring implementation against the plan and ensure that the success of the policy intervention is assessed. This monitoring is often linked with, and overlaps with, the evaluation phase of the policy cycle. Moreover, monitoring and evaluation “should be part of the development and planning of an intervention” (HM Treasury 2018, p. 51), woven into the way in which policy can be made to work and be made to be shown to be working. (The Magenta Book provides guidance on what to consider when designing an evaluation, HM Treasury 2011). Policy-making is cyclical because there is a feedback stage, to connect with the rationale for the policy initiative in the first place. Has everything envisaged been achieved? Have circumstances changed so that further policy development is required, for example? The policy cycle is advocated as a logical and rational progression, albeit with some components, especially on monitoring and evaluation, worked on throughout the process. Formulating policy in this way also involved an expectation that policy proposals will be based on evidence. The Treasury guidance refers many times to “evidence-based decision making at all stages of the policy development cycle” (HM Treasury 2018, p. 9). Part of the evidence base can be, and often is, formed from official statistics and data, again throughout the policy process. This is most obviously the case in producing the rationale, determining costs and benefits, and in monitoring and evaluation. However, ultimate decisions about policy are informed by evidence, not precisely based on evidence. The reality of policy decision-making is that the timing of when political judgements need to be made cannot always wait on full availability of evidence, for example. We return to this below. Relating to the rationale for policy, a glance at media reports will be sufficient to see how often statistics are quoted as part of the case for government to do something. Recent examples include knife crime, the state of rivers and streams, and the ecological impact of life-style choices. However, in other areas, especially as new concerns arise, such as the time spent by young people on social media, there is often a lack of official statistics. These things were not explicitly measured or, if they were

3

STATISTICS AND PUBLIC POLICY

55

covered for example in a time-use survey, then the coding of activities did not recognise the new requirement. Demands for new statistics will arise as circumstances change, so that the creation of the new statistics will typically lag behind the issue which led to them being needed in policy formulation, even where existing sources of data can be adopted. Under political pressure to make a policy change and introduce legislation, for example to respond to advances in technology, policy-makers might well be unable to wait for new official statistics to appear. This confirms that the production of official statistics for policy use is far from a static process. Official statisticians need to be aware of potential new sources of data as well as of emerging policy issues and wider themes, so that data collection can be in hand where possible. Elsewhere in the policy cycle, determining the costs and benefits of policy options may include modelling based on the available statistics. Beyond that, however, the monitoring stage is explicitly to include “the collection of data, both during and after policy implementation” (HM Treasury 2018, p. 7). Statistics mentioned in the guidance include those on public sector finance, energy use, average earnings, and wellbeing. Supporting the guidance is a raft of detailed good practice material, training and networking across the policy profession, some of which focuses on the use of statistical and other evidence. The policy profession skills framework (Policy Profession 2019, Sect. 1) sets out the role of evidence and of statistics in policy work. In our experience, the extent to which statisticians are involved in policy still depends to a large extent on the strength of working relationships and on the degree to which statistics feature in the culture of a department. The Treasury guidance reflects best practice with the intention of delivering policy-making that is grounded, efficient, and effective. This is similar to the adoption of methodologies for project management, which aim to control as much of the uncertainty as possible. However, when the Institute for Government (Hallsworth et al. 2011) reported on its research into policy-making in the real world, it presented a picture of a gap between theory and practice: “civil servants often know what they should be doing, but experience difficulties putting it into practice … Virtually every interviewee dismissed policy cycles like ROAMEF as being divorced from reality” (p. 5). This research was qualitative, designed to be illustrative of the challenges faced in policy-making and the Institute for Government recognised signs that the policy profession is starting to address some of these problems. We are reassured that improvements

56

P. ALLIN AND D. J. HAND

have continued, such as the introduction of qualifications that are more geared to data and analysis, although more recent resource and political pressures may have added fresh challenges. An especially welcome development has been the formation of What Works Centres, acting as bridges between knowledge and action for decision-makers. They are independent of government, for example with no ministerial oversight, although they may rely at least in part on government funding. The What Works Centre for Wellbeing is tasked with providing policy-makers and others with robust, accessible and useful evidence that governments, businesses, communities and people use to improve wellbeing across the UK. We will return to this later in this chapter. Before we leave this overview of policy-making, we should emphasise that politics is an integral part of public policy-making in a democracy. Returning to the point made above, this is where aiming for evidencebased policy misses the point. Rather, the goal is evidence-informed policy, where a broad range of evidence and data are taken into account in pragmatic decision-making. UN sessions on the climate crisis have been widely reported in the media, making it clear that science-based policy proposals are squared with political ambitions and constraints in order to reach agreed plans, commitments, and targets. This is evidence informing a process that is not simply about adopting the evidence. But neither is this necessarily or inevitably a rejection of evidence, rather it is the balancing of some evidence with other aspects of political decision-making, such as the practicality of implementing a policy at a given point in time. Another example is the UN’s agenda for 2030, which was informed by evidence, including on the extent to which earlier development goals had or had not been met. However, this is a case where the indicators (official statistics coordinated by the UN Statistical Commission) needed to judge progress towards the 2030 Sustainable Development Goals were left until the goals themselves were agreed. The goals were decided through an extensive and multi-layered series of political negotiations between member states. It was clearly stated that “any indicators proposed would not redefine any of the targets” (Dodds et al. 2017, p. 124). This left official statisticians with a huge task of finding methods, new sources, and resources to define the indicators and start publishing the figures. As we noted in Chapter 2, eventually 244 indicators were defined and, not surprisingly, publishing indicators on a regular basis for every country is seen as “a tremendous challenge to all countries” (United Nations, 2016). That would seem the right way to set goals through

3

STATISTICS AND PUBLIC POLICY

57

informed dialogue and debate, rather than be constrained by the available statistics. However, we repeat that it does not mean that statisticians should be excluded from the room and presented with a task for which they have neither preparation nor understanding of why the indicators are needed and how they are to be used. In a thoughtful investigation into why democracies need science, Collins and Evans “argue that science should be seen as a moral enterprise and that the values that inform scientific work should be celebrated” (Collins and Evans 2017, p. vii). This leads them to conclude that “Above all, politicians must keep things separate. Politicians must clearly and transparently accept any policies that seem to arise from the scientific consensus or, equally clearly and transparently, over-turn them and make their own policies” (p. 93). This seems to somewhat over-simplify the distinction between evidence and policy decisions. It is not about politicians just accepting or rejecting evidence, as discussed above. Also, what these authors do not spell out is that there are ways of protecting the role of science in public policy. For example, government decisions may be subject to judicial review—the judge over your shoulder, as civil servants are taught about—although such reviews are mainly concerned with how decisions are made, rather than to test the evidence. In terms of UK statistics, the Office for Statistics Regulation has the protection of the role of statistics as one of its functions. OSR intervenes for example when “significant or persistent issues with how statistics on a particular issue are being used” are identified (OSR 2019, p. 8).

3.3

The Use of Evidence in Policy

To many people, the case for evidence-based policy-making is accepted as the only way of making and delivering policy and action. During an interview, scientist Richard Dawkins (2019) summarised the best piece of advice he had ever received as “Follow the evidence, follow logic and reason” (and we read the final word as equally valid as a noun or as an imperative verb). The foundation with which Dawkins is associated “strives to foster a secular society based on reason, science, freedom of inquiry, and humanist values” (https://centerforinquiry.org/). Despite such endorsements, the case for evidence even in evidenceinformed policy making does have to be made. This includes recognising that choosing which facts to collect, and how to derive them, can be

58

P. ALLIN AND D. J. HAND

shaped by ideology, theory, or politics. Then there is a danger that apparently well-established facts can be overridden by ideology or by other, opaque political considerations. A placard raised during a protest against science spending cuts, featuring in the media as we write this book, gives a clue: it said “REAL FACTS MATTER”. A slogan in climate-change protests is “TELL THE TRUTH”. We see these and many other examples as demonstrating that the contemporary information space has huge strengths and serious weaknesses. On the one hand, it is extensive, navigable, and able to produce instant answers seemingly to any question posed of it. On the other hand, this vast pool of information and data is often poorly curated and frequently shorn of details that would help assess its quality. This is compounded because consumers of information can appear to be unwilling to question its provenance, accuracy, or relevance. What are “real facts” and “the truth”? How do we know? As a newspaper columnist wrote at the beginning of 2019: “Can we teach our children to spot fake news?”. This can be addressed from two angles, in what might be described as the supply and the demand sides for information. There are things that suppliers, especially the intermediaries who bring us information, can do. In September 2018 the European Commission announced that representatives of online platforms, leading social networks, advertisers, and advertising industry had agreed on a self-regulatory Code of Practice to address the spread of online disinformation and fake news. The signatories agreed with the Commission that “the exposure of citizens to large scale Disinformation, including misleading or outright false information, is a major challenge for Europe. Our open democratic societies depend on public debates that allow well-informed citizens to express their will through free and fair political processes” (European Commission 2018, p. 1). There are many specific commitments in the code, which is accompanied by good practice examples. The emphasis here is on encouraging truth among suppliers and mediators when stating facts. In simple terms it is about persuading people “to say where their data and facts come from”, so that users could, if they wished, “check it out”. That is where the demand side for information has a role: to press for details of the provenance of facts, not to accept facts from anonymous sources, and to ask questions about the validity of the analysis and conclusions. The aim is “to promote a more honest and decent world” (Hand 2018, p. 9).

3

STATISTICS AND PUBLIC POLICY

59

Derek Taylor (2018, p. 352) suggests that courses in “news reliability” should be added to the standard curriculum. Such a course would show students: 1. How widespread and harmful fake news has become. 2. The role of responsible news media in a democracy, to hold governments to account. 3. Which internet sites or posts—such as those of TV channels with a legal obligation to be impartial, or reputable fact-checker organisations, or leading newspapers or news agencies—have a better reputation for accuracy or fairness. 4. Examples of fake news online, and what to notice about the author’s claims and writing style which should make us wary. 5. How to recognise the difference between factual reporting and opinion, and to be suspicious of sites which muddle the two. So that we do not lose sight of the role of official statistics, recall the fundamental principles that characterise official statistics as providing a trusted and trustworthy source of information. As Winston Churchill is famously remembered (among statisticians) as saying, when he commissioned the UK’s Central Statistical Office (CSO) in 1940, it was to stop Ministers arguing about the figures. This was in a minute he sent to the Cabinet Secretary, though the instruction was captured more diplomatically later as “to prevent the confusion arising from more than one set of figures … maintaining common definitions and standards so that a consistent system” of statistics could be constructed (Rudoe 1997, p. 148). The world of information has changed hugely over the 80 years since the formation of the CSO but the issue of trustworthiness still arises, both in terms of specific statistics and through having the authority to describe society overall. The purpose of the What Works Centres can also be seen as curtailing arguments about the figures, with cross-departmental, shared evidence bases and evaluations as envisaged in earlier civil service reform plans (Policy Profession 2013, p. 7). More generally, there is, of course, no guarantee that official statistics have been produced in accordance with the UN fundamental principles. While the UK Office for National Statistics, as the executive arm of the UK Statistics Authority (UKSA), is funded as a public sector body, the UKSA itself is independent of and does not report to the government.

60

P. ALLIN AND D. J. HAND

Not all national statistical offices around the world have the advantage of such independence. Likewise, not all countries have something equivalent to the other arm of the UKSA, the Office for Statistics Regulation, tasked with providing independent regulation of all official statistics produced in the UK and alert for statistical failings of quality, value, or trustworthiness. Indeed, there have been historical and perhaps not so historical examples of governments interfering with the statistical collection process. If the point of official statistics is to give an accurate picture of the economic and social health of a country, then distorted figures carry obvious risks. The evidence that specific policy-making has some form of evidence base, including official statistics, could well be extensive but it is also diffuse and difficult to assimilate. Bauler (2012) has collated and reviewed a substantial literature around the use of indicators, especially sustainable development indicators, in policy. He starts his report with the stark observation that “Evidence on the role of information and of knowledge for policy making shows that governmental and non-governmental policy actors seldom use information as a direct input to their decisions” (Bauler 2012, p. 38). His sources span 40 years, from 1979 to 2009, suggesting an entrenched tendency within policy-making towards not using indicators. There is, of course, no counter-factual, to show how decisions might be different if informed by indicators. Boulanger (2007, p. 15) also notes the lack of evidence of successful use of sustainable development indicators in policy, but also seeks (as we do) to encourage the wider use of such indicators in “discourses, public controversies and debates on the definition and legitimisation of social problems, the underlying causes and assigned responsibilities in their emergence, their possible solutions, etc.”. In short, contributing to the rationale for policy change. Selwood (2019, p. 191) does give an example of a government agency with a clear sense of the use to which its data collection activities are to be put, but with the worrying feature that the wider uses of official statistics are being downgraded. She reflects on “three visions of English cultural sector data … Considered consecutively these suggest a trajectory shaped by increasingly myopic, institutional concerns”. Also apparent from the literature is the simple but crucial observation that indicators, indeed most sources of evidence, are constructed outside of the policy arena. Given the different timescales for public policy-making and for the compilation of evidence, this is inevitable. Specific proposals for the potential policy use of indicators and other

3

STATISTICS AND PUBLIC POLICY

61

statistical evidence abound in the literature, ranging from books (e.g. Clarke et al. 2018), reports (e.g. O’Donnell et al. 2014), and journal articles (e.g. Sethia 2016). Tools such as systematic reviews can help with timely and relevant evidence. Donnelly et al. (2018, p. 1) make a persuasive case for rewarding “the creation of analyses for policymakers that are inclusive, rigorous, transparent and accessible” through such reviews. For evidence to be used in policy it has both to be pushed towards policy-makers and pulled by them into their considerations—but not, of course, merely the selection of evidence to support a pre-determined policy direction. So-called pathways for evidence to be used effectively in evidence-informed policy can be created or highlighted by intermediaries such as the What Works Centres and by bodies such as the Alliance for Useful Evidence (https://www.alliance4usefulevidence.org/). These all undertake “advocacy, organising events, sharing ideas, and delivering training and support” (in the words of the Alliance’s website). The EUfunded BRAINPOoL Project (Whitby 2014) was specifically aimed at bringing alternative indicators, other than GDP, into policy. Many recommendations were made, including calling for a strong “beyond GDP” narrative, more integrated policy-making and strategies for overcoming resistance within policy-making institutions. We were particularly struck that a key lesson of the project was “the need for indicator specialists to become more outward-facing and engage strategically with influential actors” (Whitby 2014, p. 40). This is very much our view as well, and that of many others operating in, or observing, the information space. Fortunately, the message is now starting to be accepted by producers of official statistics, though there is more to be done. Whatever the source of evidence, Bauler’s (2012, p. 40) helpful conclusion is that: “In order to become consistently influential, indicators need to be perceived simultaneously – consensually – by a group of policy actors as being legitimate, credible and salient”. It seems to us that official statistics are well placed to deliver legitimate, credible, and salient evidence for policy. There will be many examples in which official statistics are marshalled in support of the need to change policy, say on knife crime. We are less clear on the extent to which Bauler’s approach is adopted in practice, that is there is a process of sharing and agreeing on the evidence base. This is largely because examples of policy development are rarely documented, especially in the detail needed to scrutinise the process and the evidence together.

62

P. ALLIN AND D. J. HAND

There have been calls for government to get better at showing how it uses evidence in policy. Since 2015, the Institute for Government, Sense about Science, and the Alliance for Useful Evidence have been demonstrating the need for government to be more transparent in how it uses evidence (to “show your workings”) to improve policy-making and to increase trust in government. A practical and easily applied framework for testing evidence transparency has been devised and the organisations are working with government departments. In a recent spot-check of 12 UK domestic policy departments, the conclusion was a mixed picture: “most departments can attain high standards of transparency about evidence at times, and in diverse policymaking situations … But there were also large discrepancies and some frustrating cases where an otherwise transparent policy lost points on one section of the framework” (Sense About Science 2018, p. 17). Evidence is expected to be found in all four areas of the framework—diagnosis, proposal, implementation, testing, and evaluation. Overall, the picture in this spot-check was that testing and evaluation drew the lowest scores for the transparency of evidence, suggesting that the areas of greatest concern were in difficulties in answering such questions as: “How we will know if the policy has worked or, in the case of consultations and further investigations, how the information gathered will be used” (p. 8). One of the problems is that finding “the workings” is not always easy, even for someone who knows how the government works. Note that the arrival of What Works Centres, along with innovative approaches to policy implementation, will be improving the appraisal of government initiatives. A UK parliamentary committee recently reported on challenges in using data across government (House of Commons 2019). In its evidence to the committee, the Royal Statistical Society noted that statistics are one of an increasing number of ways in which such data are used. Processes and procedures are already in place to ensure that data are used for statistics in ways that people can trust, and hence meet the first objective set in a draft National Data Strategy proposed by the government. However, these processes may need to be reviewed to reinforce the message of the value of official statistics, and that they do not disclose individual details, as the public becomes more aware that data are increasingly being collected, shared and used for public services, government operations, and across the economy (RSS 2019). Zuboff (2019, p. 23) has drawn a distinction between business use of behavioural data collected “with permission and solely as a means to product or service

3

STATISTICS AND PUBLIC POLICY

63

improvement”, which she characterises as “committing capitalism”, and “surveillance capitalism”, which “unilaterally claims human experience as free raw material for translation into behavioural data” (p. 8). The committee’s conclusions and recommendations were phrased around a data strategy and the infrastructure and conditions needed for improved use of data across government. All of these impinge on the more effective use of data for statistics, including that “Government officials’ concerns about protecting data can stand in the way of using it to coordinate services” (House of Commons 2019, p. 6). The report is referring here to legislation that is particularly relevant to the statistical services of government, introduced as part of the Digital Economy Act 2017. It was hoped this would revolutionise access to data sets to facilitate the production of official statistics. The legislation also provides for the sharing of de-identified administrative data by public authorities with other persons for the purpose of research (Allin 2019, p. 7). However, it is still proving sometimes to be a long and slow process for official statisticians to make use of administrative data, as we consider further in Chapter 5. There are calls for new statistical measures, often as part of wider policy proposals. One recent example is that England’s Chief Medical Officer, Dame Sally Davies, in her 2018 report, identified among other things the “need to track progress in improving health and health outcomes, to and beyond 2040 with a new composite Health Index that reflects the multi-faceted determinants of the population’s health and equity in support of ensuring health is recognised and treated as one of our nation’s primary assets. This index should be considered by Government alongside GDP and the Measuring National Well-being programme” (Chief Medical Officer 2018, Chapter 1, p. 2). Dame Sally also made recommendations on several aspects of the construction of the new “index” (actually a health of the nation dashboard), including with an eye on how it would be used. This example also illustrates a general point, that more dialogue is needed between potential users and producers of new measures over how the measures would be used: for example, how often, how timely, and how much geographical detail is wanted? There may be trade-offs to be made between things like these. The practicality of proposals also needs to be tested. In this case, the proposer notes that “We regularly collect most of the datasets that have the individual measures that could be combined” (Chief Medical Officer 2018, Chapter 1, p. 2), but this does of course mean that a complete dashboard would not available until

64

P. ALLIN AND D. J. HAND

new data are found. We mentioned another example in Chapter 2, where people outside of government proposed a new set of measures: social metrics as new, wider measures of poverty. A common feature of many proposals for new measures is that they call for metrics that matter, rather than say relying on traditional proxies for success and progress, such as GDP.

3.4

Wellbeing Policy and Measures

It seems reasonable to start with a broad position, that wellbeing should underpin everything that governments do in democracies, and that progress includes some increase in wellbeing. As many people have noted (e.g. Easton 2012, pp. 77–78), the question as to “exactly what social progress looks like … is following a well-worn path”, which can be traced back over more than two thousand years, without a consistent definition. Moreover, as we discussed in Sect. 1.1, the measurement of social progress presents deep conceptual challenges of its own, deeply intertwined with how the concept is defined. Harari (2014, p. 264) puts things into perspective, writing that “Until the Scientific Revolution most human cultures did not believe in progress. They thought the golden age was in the past, and that the world was stagnant”; and there is clearly no universal agreement on what progress is nowadays. In modern times, however, there is a general political and public acceptance that one of the roles of government is to protect and enhance the wellbeing of the people, though how this is carried out, and indeed the relative importance of that role, varies over time and between countries. Taking a broad definition of wellbeing—as a holistic, sustainable view of the economic, demographic, social, and environmental situation of a country—fits closely with the intended scope of official statistics, as set out in the UN fundamental principles referred to earlier in this chapter. However, that does not guarantee that national wellbeing will be well described in official statistics, let alone that there will be specific measures of personal or of subjective wellbeing (which is a personal assessment of one’s own wellbeing). There are practical, resource, and organisational constraints. In the UK, for example, outputs specifically tagged as about aspects of wellbeing form a small proportion of the total of some 3570 statistical releases of official statistics each year. The fact is that official statistics have traditionally been a curated set of outputs, where the choice of what to include, or not to include, is determined by a complex

3

STATISTICS AND PUBLIC POLICY

65

of decisions. These decisions include international conventions, such as the worldwide System of National Accounts, which defines GDP and many other measures, and legislation, including European regulations, for example, requiring member states to produce and publish harmonised indices of consumer prices. Continuity of statistical production is often also part of the ethos of official statistics, maintaining statistics that are consistent over time. That suggests that there are two broad ways forward in using official statistics for national wellbeing measures: either design a new set of outputs specifically aimed at describing national wellbeing, or ensure that official statistics provide the material from which national wellbeing assessments can be made by others. The UK now seems to have both. In a recent interview, Sir Mark Sedwill, the UK Cabinet Secretary and head of the civil service, noted that “The Wellbeing Index introduced under the Cameron government” is an example of a national wellbeing framework that covers more than economic performance. Sir Mark went on to say: but we need to look more broadly. The Strategic Framework is really saying, we have a series of economic goals, set by government. Then we have goals around the wellbeing of the individual citizen. These relate to the inclusiveness of communities, the safety of individuals, whether they can go about their daily lives in the way that they want and get the education they need, is crime being tackled in their areas, and so on. You have a whole series of issues around security and safety, and around sustainability and the environment, and then a set of issues around the country’s role and influence in the world. And what we’re saying is, we need to judge how we’re doing as a government and as a country, not on purely economic criteria but against those broader measures. (Holder 2020)

This leads to our key point on official statistics and wellbeing, that measurement is necessary but not sufficient to change anything. We have especially in mind the then Prime Minister, saying at the launch of the ONS’s Measuring National Well-being programme, that we are now “measuring our progress as a country, not just by how our economy is growing, but by how our lives are improving; not just by our standard of living, but by our quality of life” (Cameron 2010). The point is a general one. As Mayer (2013, p. 2) has warned, and as we have observed above, measuring something “does not in itself automatically translate into policy” to tackle or improve the situation described. He made these comments in the context of accounting for

66

P. ALLIN AND D. J. HAND

natural capital, that is the natural environment and ecosystems. The popular author Bill Bryson (2015, p. 428) put it more acerbically when observing that a State of Nature report had found that about two thirds of all species in Britain are in decline: “Britain, it turns out, is outstanding at counting what it has, but not so good at holding on to it”. One can only imagine Bryson’s frustration as a further State of Nature report has since appeared, recording for example that “Since 1970, the indicator of abundance for 214 priority species has declined by a statistically significant 60%” (Hayhow et al. 2019, p. 11). This is described as the most comprehensive analysis to date, which may reinforce Bryson’s point about measuring something rather than doing anything about it, if it were not for the actions that the many partner organisations supporting the State of Nature reports are themselves undertaking, albeit so far with limited success. The notion of a disjoint between measurement and action is not new. In the 1910 novel Howards End, EM Forster wrote of the human weakness to “mistake the sign-posts for the destination” (Forster 1965, p. 114), a useful warning to those who see having specific official statistics on wellbeing as equating with doing something about the wellbeing of the nation. Similarly, statisticians, economists and scientists are cautioned about mistaking their models for the reality. What is needed are practical and specific policy initiatives, woven into the programmes of government, rather than a blanket assumption that wellbeing will somehow be taken care of. Hicks et al. (2013) discuss how the ONS measures of national wellbeing were constructed with a view to their use in policy. For the subjective wellbeing measures, they summarise (p. 81) how these “could relate to different purposes of public policy. The most general measures could be used for overall monitoring. More detailed domain and affect questions could be used for policy formulation. Finally, policy appraisal is likely to use the most detailed measures which are specific to particular activities and services”. Similarly, a report from the Legatum Institute (O’Donnell et al. 2014, p. 14) set out “how governments and individuals can take account of wellbeing and use it for everyday decisions” (our emphases). Of course, “could” and “can” are not the same as “will”. An official UK document (Cabinet Office 2013, p. 1) was more detailed in its “short update on relevant wellbeing work across Whitehall by presenting a few emerging examples/cases, a list of policy areas in which departments are specifically considering wellbeing, and some

3

STATISTICS AND PUBLIC POLICY

67

current analysis plans”. This may be a short report (20 pages) but it does cover 12 case studies, list 49 wellbeing policies or programmes, and summarises the wellbeing analysis and data collection plans in 16 government departments and two devolved administrations. The document emphasised that “this is a long-term programme. The current indicators are experimental statistics and still in development, and as such we should not expect to have examples of major decisions that have been heavily influenced by wellbeing at this stage. However, the foundations are in place and departments are clearly beginning to use the data where it is both relevant and adds value to their work” (p. 1). The official document was also included by the government (House of Commons 2014, Ev 68) in its evidence to a 2013–2014 Parliamentary committee inquiry into wellbeing measurement and policy. The committee suggested no changes to the ONS wellbeing measures (and concluded that the time was not ripe for an overall single measure of wellbeing, an issue which we will consider further in our conclusions, in Chapter 6). Rather, the committee (House of Commons 2014, p. 31) anticipated a time when “a measurement track-record has been built up on the component measures, they have achieved a reasonable level of public familiarity, and a general consensus has been reached on their value and usefulness”. Nevertheless, the committee also recommended that “The Government should immediately start to use the already available data to ‘wellbeing-proof’ existing policy proposals, with the Cabinet Office encouraging take-up through its oversight and scrutiny of other departments’ business plans”, including being alert to when “the data becomes sufficiently robust” for policy purposes. Hey (2018, p. 1) lists some of the subsequent analysis, to underpin her suggested “seven actions organisations can take to improve quality of life for people and their communities as they experience it”. Bache and Reardon (2016, p. 7) reported that UK developments in measurement and policy were “among the most advanced” although policy developments “generally amount to small-scale changes”. They nevertheless captured that the zeitgeist is that wellbeing measurement is “an idea whose time had come” (p. 158) and that there is interest in policy circles in what works for wellbeing, that is an evidence-based assessment. However, they concluded this will be through a “gradual accumulation of knowledge and experience relating to the wellbeingpolicy relationship” in which “evidence will play an important role” (p. 159). They also recognised an “on-going challenge for wellbeing

68

P. ALLIN AND D. J. HAND

advocates to sell the idea to a wider range of politicians, officials, interest groups, the media and the public at large” (p. 158). Now, a decade on from the launch of the UK’s Measuring National Wellbeing Programme, it seems to us that the wellbeing measures, which continue to be regularly updated, are now functioning as an adjoint to the policy process. The launch and continued publication of these official statistics helps set the scene and raise awareness of wellbeing, both in terms of personal wellbeing and in the broader context of national wellbeing and the “beyond GDP” agenda. We acknowledge concerns (e.g. Jenkins 2018, p. 282) around the “politics of measurement” and that “‘Well-being’ is not a well formed concept”, which might act as a brake on further developments. On the other hand, there was a national debate as part of the measuring wellbeing programme (Allin and Hand 2014, pp. 222–223) and, as noted above, a range of wellbeing measurement and policy work across UK government departments and devolved administrations had been initiated. What about the ongoing, direct policy use of official statistics on wellbeing? Bache (2019, p. 3) has analysed “the role of evidence in taking wellbeing from an issue that has government attention to one that leads to significant policy change”. His analysis of what policy-makers want from evidence providers, such as information that is simple and short, is delivered in practice through a web of organisations, including What Works Centres, rather than just directly between producer and user. Bache notes that one specific aspect of this is to “Create a demand for wellbeing information, evidence, knowledge, science and statistics” (p. 66). Bache and Reardon (2016) earlier concluded that wellbeing had “risen up the agenda” in the UK over recent years and that this resulted from the multiple streams in play: policy, politics, and problems. These streams have their own actors, professional groups, and cultures, all of which can (but do not necessarily) draw on relevant official statistics. In rather turbulent years of British politics since 2016—the year in which the UK voted in a referendum to leave the EU—wellbeing is acknowledged by many departments and agencies and included in policy documents, strategies, and Ministerial statements. It appears that wellbeing is now woven into much public policy in the UK. For example, in a speech setting out the scope of a forthcoming spending review, the Chief Secretary to the Treasury (Truss 2019) said: “we will look at the major projects we are investing in, and asking whether they are really working for us – whether they are having positive effects on

3

STATISTICS AND PUBLIC POLICY

69

growth and the wealth and wellbeing of individual people”. This suggests that wellbeing measures might be used as part of the evaluation and, by juxtaposing wellbeing and wealth, echoes the commitment by former Prime Minister Cameron to look not just at a growing economy and rising standard of living (i.e. wealth and its accumulation) but also at wellbeing. Interestingly, in his memoirs, Cameron (2019, p. 204) mentions issues that are captured in ONS’s statistics on national wellbeing, such as “a fairer, more equal society; a greener environment” but does not refer explicitly to national or personal wellbeing, with which he was once closely associated (and when the media referred to the national wellbeing measures as Mr. Cameron’s Happiness Index). A cogent case for using wellbeing measures in policy continues to be made. Layard et al. (2020) provide a highly topical example, publishing a wellbeing framework for analysing costs and benefits in choosing when to end a pandemic lockdown. They list a number of expected positive and negative benefits of releasing a lockdown. To assess how these add up “requires a common metric. We propose as a metric the number of Wellbeing Years (WELLBYs). This metric is analogous to the QALY metric which has been successfully used in the NHS for 20 years. It should now be extended to all fields of public policy” (p. 2). This is a framework, not an algorithm. Using it needs the selection of all the factors to be taken into account. The reliability of the estimates of the value of each benefit also needs to be considered, as does the possible variability of choice of benefits and value to different population sub-groups. The Green and Magenta Books are guidance issued by HM Treasury on how UK government departments are to appraise policies, programmes, and projects. They also provide guidance on the design and use of monitoring and evaluation before, during, and after implementation. The position on wellbeing measures in UK domestic policy is confirmed in the current version of the Green Book, starting with the principle that “Economic appraisal is based on the principles of welfare economics – that is, how the government can improve social welfare or wellbeing”, which is also referred to in the Green Book as “social value” (HM Treasury 2018, p. 5). The guidance notes that “Individual and society’s wellbeing is influenced by a number of interrelated factors” and that wellbeing evidence should be evaluated where appropriate (p. 16). Subjective (personal) wellbeing approaches are also summarised (p. 42). Another political factor impacting on wellbeing is the UN’s 2030 agenda, with its sustainable development goals, as discussed above. This

70

P. ALLIN AND D. J. HAND

is now a major setting for the development, publication, and use of measures that in effect go beyond GDP and addresses many dimensions of personal and national wellbeing, though without constant references to wellbeing. The text includes a commitment “to developing broader measures of progress to complement gross domestic product (GDP)” (United Nations 2015, para 48) though there is no further detail on what that means. The sentence comes at the end of a paragraph noting that indicators are being developed to assist in the follow-up and review of progress towards the goals. Since, as we have noted, over 240 indicators have now been defined, our working assumption is that this is what beyond GDP will look like. It is widely acknowledged that to compile the full set of SDG indicators for each country presents a huge challenge to many national systems of official statistics. The UK has a relatively established system. However, the third annual update by the UK national statistical office, the latest at the time of writing, reveals that its work to publish indicators for the UK was not yet completed. It shows “reporting at least headline data for 182 (75%) of the 244 indicators” and that the office was exploring data sources for the remaining 25% (ONS 2019, Sect. 1). The earlier, explicit, beyond GDP agenda also had international backing, including through the European Commission and the OECD. However, the UN agenda for sustainable development appears to have stronger traction on policymakers and on national statistical offices, which might lead to further improvements in measurement. As one illustration of the reach of the UN agenda, when the UK government’s Chief Medical Officer called for a new Health Index (as referred to above), she noted that the investigation of indicators for her goals for health “should involve the Office for National Statistics, which has experience in index development and should link to their work measuring the United Kingdom’s progress on delivering the United Nations’ agreed Sustainable Development Goals” (Chief Medical Officer 2018, Chapter 1, p. 2). We have focused so far on wellbeing measures and policy in the UK government. There are developments in other countries that also help understand the role and position of official statistics in wellbeing policy. In another volume in this series, Jennifer Wallace describes how, over the past decade, “the three devolved legislatures of the UK (Scotland, Wales and Northern Ireland) have embarked on substantial changes in how they understand, measure and contribute to social progress. This reframing of

3

STATISTICS AND PUBLIC POLICY

71

the role of government can broadly be described as a wellbeing approach, consisting of a measurement framework and a set of public policy reforms aiming to improve wellbeing” (Wallace 2019, p. 2). Scotland (p. 55) and Wales (p. 83) each have a set of national indicators and there is a set of wellbeing indicators in Northern Ireland (p. 111). Official statisticians are responsible for compiling and publishing the indicators. There are several general questions to consider arising from how official statistics feature in the respective implementations of the wellbeing approach adopted by each devolved legislature. Our first question is our central one of how are the indicators to be used? In broad terms, the indicators are the primary way of assessing progress towards particular goals or outcomes. Indicators therefore need to be seen as an integral part of a process involving each devolved government and their wider public sectors. The Welsh legislation, for example, requires Welsh Ministers to “publish indicators (‘national indicators’) that must be applied for the purpose of measuring progress towards the achievement of the well-being goals” and to lay a copy of the national indicators before the National Assembly, as well as setting milestones and publishing an “annual well-being report” on the “progress made towards the achievement of the well-being goals by reference to the national indicators and milestones” (National Assembly for Wales 2015, Sect. 10). The legislation also specifies other parts of government which play a role, including a Commissioner and the Auditor General for Wales. In Scotland the work is badged as a national performance framework, to show “how well Scotland is performing”: an inclusive phrase, but one that also melds the performance of government with that of everyone else, if we take it to cover the democratic processes that led to this choice of approach. The process into which national wellbeing indicators are absorbed follows a familiar policy circle: government setting goals, taking action to meet those goals, monitoring progress towards the goals using milestones and indicators, and presumably taking further or corrective action in light of each annual stock-take of progress. The indicators and annual reports are publicly available, so that progress can be monitored more widely, including at local level, and the government could be challenged, including in the legislatures and in the media. What we have not picked up from descriptions about the indicators and the supporting legislation is more precisely how indicators are meant to be used, or much sense that organisations and individuals outside government and its public bodies are invited to use the indicators to change their own behaviour.

72

P. ALLIN AND D. J. HAND

Our second question is therefore, are the indicators measuring what matters to the people of each country? Wallace (2019, p. 52) reports that a number of non-government organisations had been “advocating for a rethinking of the measurement of social progress”, including Oxfam Scotland’s first Humankind Index for Scotland in 2012. This index was based on consultation with people across Scotland “in order to establish what aspects of life make a difference to them”. There is evidence of further (one-off) consultation by the devolved administrations in the development of their national indicator sets, along with seeking ongoing feedback. The consultations tended to major on the topics that matter, leaving the statisticians to come up with specific indicators. The third question that comes to mind is how the national indicators fit with the indicators being developed on a global basis for measuring progress towards the UN’s sustainable development goals. It seems to us that countries should not be bound just to use the UN indicators. Conditions, priorities, and resources vary. However, the SDGs are a universal commitment, the goals are to be assessed in global as well as national terms, and measurement resources are always under pressure. The case for at least integrating national and international indicators appears to be strong. The Scottish Government’s website (see below) makes the connection explicit, stating that “The National Performance Framework and the Goals share the same aims. The National Performance Framework is Scotland’s way to localise and implement the SDGs”. The government is also seeking to work in partnership with “over 300 people and organisations across Scotland to assist with the development of a Scotland-wide response to the challenge set by the SDGs”. In Wales, the position is perhaps a little more reserved, or at least keen to keep the focus on the wellbeing of future generations in Wales. Their website (see below) states that “Many national indicators will help tell a story of progress in Wales against more than one of the Well-being Goals or United Nations Sustainable Development Goals” and there is an online tool to show the indicators that map to each of the Wellbeing Goals and each SDG. However, it also warns that “These mappings should not be considered as inferring a direct or indirect technical or legal link between the indicators, well-being goals and the United Nations Sustainable Development Goals”. The New Zealand government has also adopted a wellbeing economy approach. It started with the measures included in the OECD’s Better Life Index (see below) and added the areas of cultural identity and four

3

STATISTICS AND PUBLIC POLICY

73

measures of resources to support future wellbeing (natural capital, human capital, social capital, and financial/physical capital). Following consultation, 60 indicators were defined and presented in a dashboard. As Fisher (2019, p. 1) notes, “Budgeting is where New Zealand steps out in front. Measuring is good, it enables one to see at least. But reframing all Government budgeting around it is quite something else, requiring rewriting of financial law and substantial changes to budgeting processes. Every spending proposal by Government must demonstrate delivery of wellbeing outcomes. The Government is currently developing its cost benefit tools to facilitate these new computations”. This is a positive step forward, addressing how new spending is decided on. It updates Coyle’s (2014, p. 118) conclusion that “Unfortunately, though, there is no evidence yet of dashboards [based e.g. on the Better Life Index] displacing the prime status of GDP growth in political debate”. We agree with Coyle that “The Better Life Index is not a tool that could be used for macroeconomic policy, but it does present the trade-offs in a very accessible way” (ibid), also reminding us all that trade-offs may be needed rather than more comfortable win-win solutions. And as economists often remind us, some common currency (such as life satisfaction or quality adjusted life expectation) is needed to quantify such trade-offs once we move beyond GDP. For further details of the wellbeing indicators and how they are used in the UK, Northern Ireland, Scotland, Wales, and other wellbeing economies, please consult the respective websites: • Measuring national wellbeing UK dashboard https://www.ons.gov. uk/peoplepopulationandcommunity/wellbeing/articles/measureso fnationalwellbeingdashboard/2018-09-26 • Northern Ireland https://www.nisra.gov.uk/statistics/ni-summarystatistics/uk-national-wellbeing-measures-northern-ireland-data • Scotland https://nationalperformance.gov.scot/ • Wales https://gov.wales/national-well-being-indicators • Iceland https://www.government.is/lisalib/getfile.aspx?itemid=fc9 81010-da09-11e9-944d-005056bc4d74 • New Zealand https://treasury.govt.nz/sites/default/files/201811/lsf-introducing-dashboard-dec18.pdf • OECD Better Life Index http://www.oecdbetterlifeindex.org/ • The City of Santa Monica’s Wellbeing Project https://use.metrop olis.org/case-studies/the-wellbeing-project#casestudydetail

74

P. ALLIN AND D. J. HAND

3.5 Conclusion: Making Better Use of Statistics Around Wellbeing Policy To sum up how official statistics are used in policy-making, even in wellbeing policy circles, is an impossible task. In our experience, and in our research for this chapter, we have found a large divergence in the extent to which official statistics feature. Nevertheless, we retain an expectation that official statistics can contribute to evidence-informed policy-making and we are reassured that there is evidence that they are doing so. There are also positive signs that official statisticians are engaging more with policy. Examples from the ONS include the creation of five centres in public policy (Bell 2019) and producing publications “designed to provide the evidence policy makers need to improve people’s lives over the next decade” (ONS et al. 2020, p. 1). Interestingly, the latter (p. 55) concludes not with a call for more data but that “the evidence points policy makers towards reflecting on whether the country has a compelling enough narrative for society, and for improving people’s lives and their wellbeing in the 2020s”. We expect high standards for evidence and have found that this is being strived for in the wellbeing measurement frameworks and processes discussed in this chapter. The touchstones are that national wellbeing indicators should be legitimate, credible, and relevant to decision-making and action. However, these qualities also need constant care and attention. Policy needs evolve and often call for detailed indicators for specific places and groups in society. Other, non-official sources of data and other measures of wellbeing and progress are being devised, often drawing on official statistics. There is more to be done, even in the relatively small number of places so far adopting a wellbeing approach. Raising awareness and encouraging uptake of national wellbeing indicators among policy-makers and politicians needs to be ongoing. What goes with this is that official statisticians may have to become more adept at responding to new requests, and perhaps even at foreseeing future new requests. The development cycle for new statistics can be quite out of sync with the policy cycle. New tools can help make indicators more widely available, for example the increasing use of dashboards. These have been long promoted but their effectiveness still needs to be improved. Bartlett and Tkacz’s (2017, pp. 19–21) recommendations for successful dashboard implementation include “identifying the purpose of the dashboard from the beginning …

3

STATISTICS AND PUBLIC POLICY

75

designing for its users … establish user buy in … training”. These are dimensions of user consultation and engagement. User engagement by official statisticians more generally needs to be strengthened and good practice shared more widely. It is also clear that statistics, and research based on statistical data, often need translating for policy-makers to use. We welcome that “Over the past 5 years, the UK’s What Works initiative has been transforming the way we do government. We have been providing policy-makers and practitioners with evidence-based guidance on cost-effective practice in policy areas from policing to education, building capability in the Civil Service to use evidence intelligently, and agitating for stronger evidence-use and impactevaluation across government” (What Works Network 2018). The What Works Centre for Wellbeing advocates the use of the national wellbeing measures alongside taking wellbeing research into the policy world. The outputs of policy processes range from regulation and requirements, through strategies to softer ways of influencing behaviour. The outputs may be targeted at businesses, households, local government, or other national governments (for example through alliances). How taxes are raised, and how public funds are distributed, also mirror policy aims. Politicians and governments can only go so far, even with the increasing use of “nudge” (e.g. Wells 2010). Halpern et al. (2004, p. 3) concluded that “The achievement of major policy outcomes requires greater engagement and participation from citizens – ‘governments can’t do it alone’ – than traditional ways of delivering public services. Higher levels of spending and better-run public services can achieve improved outcomes. But in the long-run improvements depend as much on changes in personal behaviour: for example, in health on better diet and more exercise, and in education on children’s willingness to learn and parents’ willingness to help … better for governments to empower citizens as much as possible rather than making decisions on their behalf”. Such personal changes are about evolving ethical and cultural values and attitudes, not forgetting how we all respond, or do not respond, to what is available in the global marketplace for goods and services. Empowering citizens includes giving them information, including statistics. It can be hard to engage an audience unfamiliar with statistics. How information is presented can be relevant to changing behaviour. For example, showing life expectancy in terms of years lost due to persisting in smoking has been shown to be more effective than referring to years gained by quitting (Halpern et al. 2004, p. 39).

76

P. ALLIN AND D. J. HAND

However, all public policies, even a decision to nudge, are essentially top-down. There may well be a prior stage, of joining up across parts of government (horizontal connections) within a strategic framework. But government initiates and people, communities, and businesses may or not follow. It is difficult to dissent from the view, expressed in a recent New Statesman (2020, p. 5) editorial, that “Only governments can impose the regulation and provide the investment necessary to forge a green economy”. However, there is also a view, to which we subscribe, that to make real change, top-down needs to meet with bottom-up willingness to address the same issues, informed by the same understanding and description of wellbeing and sustainable development. It has been argued, for example, that to change behaviour in how we treat the natural environment, small, individual actions may not add up to enough. This suggests the perfectly sensible additional role for intermediaries, especially nongovernmental organisations, to help transmit public policy to people and businesses and to amplify public and business concerns to government, including by drawing on official statistics as part of the common evidence base. This is far from ignoring the role of individual people and individual businesses, whose actions informed by evidence can send signals to the market and to governments. One such non-governmental organisation, specialising in sustainable behaviour change, is Global Action Plan. Their report (2012) summarises the evidence available from its own programmes. We were intrigued, but not surprised, to note that they recommended drawing attention to periods of environmental stress in making a call for action, rather than refer to statistics about the climate and the environment. There may be a case for using crisis for advocacy purposes, but it seems more fruitful to anticipate rather than react, using evidence to inspire sustainable wellbeing. Although this chapter has been focused on statistics and public policy, we should be careful not to separate policy use from wider use. Our aim is to make greater use of official statistics throughout society, not least because that may feed back into how issues arise and are dealt with in public policy. Why for example is there only a “small community of specialists who pay attention to US road safety statistics”, who picked up the first signs of what turned out to be a marked upward trend in the number of pedestrians killed on American roads, after decades of falling numbers (Baker 2019)? We explore the role for official statistics beyond the production of public policy next, in Chapter 4.

3

STATISTICS AND PUBLIC POLICY

77

References Allin, P. (2019). Opportunities and Challenges for Official Statistics in a Digital Society. Contemporary Social Science. https://doi.org/10.1080/21582041. 2019.1687931. Accessed 19 October 2019. Allin, P., & Hand, D. J. (2014). The Wellbeing of Nations: Meaning, Motive, and Measurement. Chichester: Wiley. Bache, I. (2019). Evidence, Policy and Wellbeing. Cham, Switzerland: Palgrave Macmillan. https://www.palgrave.com/gp/book/9783030213756. Bache, I., & Reardon, L. (2016). The Politics and Policy of Wellbeing. Cheltenham: Edward Elgar. Baker, P. C. (2019). Collision Course: Why Are Cars Killing More and More Pedestrians? https://www.theguardian.com/technology/2019/oct/03/collisioncourse-pedestrian-deaths-rising-driverless-cars. Accessed 25 January 2020. Bartlett, J., & Tkacz, N. (2017). Governance By Dashboard: A Policy Paper. London: Demos. https://www.demos.co.uk/wp-content/uploads/ 2017/04/Demos-Governance-by-Dashboard.pdf. Accessed 23 January 2020. Bauler, T. (2012). An Analytical Framework to Discuss the Usability of (Environmental) Indicators for Policy. Ecological Indicators, 17, 38–45. Bell, I. (2019). How ONS Plans to Provide New Analysis on the Core Issues Facing Our Society. https://blog.ons.gov.uk/2019/07/10/how-ons-plansto-provide-new-analysis-on-the-core-issues-facing-our-society/. Accessed 17 March 2020. Bryson, B. (2015). The Road to Little Dribbling: More Notes from a Small Island. London: Transworld Publishers. Boulanger, P.-M. (2007). Political Uses of Social Indicators: Overview and Application to Sustainable Development Indicators. International Journal of Sustainable Development, 10(1/2), 14–32. Cabinet Office. (2013). Wellbeing: Policy and Analysis. https://www.gov.uk/gov ernment/publications/wellbeing-policy-and-analysis. Accessed 23 October 2019. Cameron, D. (2010). PM Speech On Wellbeing. https://www.gov.uk/govern ment/speeches/pm-speech-on-wellbeing. Accessed 23 October 2019. Cameron, D. (2019). For the Record. London: William Collins. Chief Medical Officer. (2018). Annual Report of the Chief Medical Officer, 2018. https://www.gov.uk/government/publications/chief-medical-officerannual-report-2018-better-health-within-reach. Accessed 20 October 2019. Clarke, A. E., Flèche, S., Layard, R., Powdthavee, N., & Ward, G. (2018). The Origins of Happiness. Princeton: Princeton University Press. Collins, H., & Evans, R. (2017). Why Democracies Need Science. Cambridge, UK: Polity Press. Coyle, D. (2014). GDP: A Brief but Affectionate History. Princeton: Princeton University Press.

78

P. ALLIN AND D. J. HAND

Dawkins, R. (2019). The NS Q&A. New Statesman 11–17 October, 62. Dodds, F., Donoghue, D., & Leiva Roesch, J. (2017). Negotiating the Sustainable Development Goals. Abingdon: Routledge. Donnelly, C. A., Boyd, I., Campbell, P., Craig, C., Vallance, P., Walport, M., Whitty, C., Woods, E., & Wormald, C. (2018). Four Principles to Make Evidence Synthesis More Useful for Policy. Nature, 20 (June). https://www. nature.com/articles/d41586-018-05414-4. Accessed 17 April 2020. Easton, M. (2012). Britain etc.: The Way We Live and How We Got There. London: Simon & Schuster. European Commission. (2018). Code of Practice on Disinformation. https:// ec.europa.eu/digital-single-market/en/news/code-practice-disinformation. Accessed 13 October 2019. Fisher, D. (2019). Wellbeing Worldbeaters: New Zealand, Scotland and Iceland. Institute of Welsh Affairs, Cardiff. https://www.iwa.wales/click/2019/ 10/wellbeing-worldbeaters-new-zealand-and-scotland/. Accessed 23 January 2020. Forster, E. M. (1965). Howards End. Harmondsworth: Penguin Books. Global Action Plan. (2012). Changing Environmental Behaviour: A Review of Evidence. https://issuu.com/gap_international/docs/gap_behaviour_cha nge_report/18. Accessed 25 January 2020. Hall, P. (1993). Policy Paradigms, Social Learning, and the State: The Case of Economic Policymaking in Britain. Comparative Politics, 25(3), 275–296. Hallsworth, M., Parker, S., & Rutter, J. (2011). Policy Making in the Real World: Evidence and Analysis. London: Institute for Government. https://www.ins tituteforgovernment.org.uk/publications/policy-making-real-world. Accessed 12 October 2019. Halpern, D., Bates, C., Mulgan, G., Aldridge, S., Beales, G., & Heathfield, A. (2004). Personal Responsibility and Changing Behaviour: The State of Knowledge and Its Implications for Public Policy. Prime Minister’s Strategy Unit discussion paper. http://webarchive.nationalarchives.gov.uk/+/http:/www. cabinetoffice.gov.uk/media/cabinetoffice/strategy/assets/pr2.pdf. Accessed 9 April 2020. Hand, D. J. (2018). Who Told You That? Significance (August 2018), 8–9. Harari, Y. N. (2014). Sapiens: A Brief History of Humankind. London: Harvill Secker. Hayhow, D. B., Burns, F., Eaton, M. A., Al Fulaij, N., August, T. A., Babey, L., & Gregory, R. D. (2019). The State of Nature 2019. The State of Nature partnership. https://nbn.org.uk/wp-content/uploads/2019/09/ State-of-Nature-2019-UK-full-report.pdf. Accessed 23 October 2019. Hey, N. (2018). Improve Wellbeing With Data Infrastructure: Seven Actions For Organisations. https://whatworkswellbeing.org/blog/improvewellbeing-with-data-infrastructure-seven-actions-for-organisations/. Accessed 18 April 2020.

3

STATISTICS AND PUBLIC POLICY

79

Hicks, S., Tinkler, L., & Allin, P. (2013). Measuring Subjective Well-Being and its Potential Role in Policy: Perspectives from the UK Office for National Statistics. Social Indicators Research, 114, 73–86. https://doi.org/10.1007/ s11205-013-0384-x. HM Treasury. (2011). The Magenta Book. https://www.gov.uk/government/ publications/the-magenta-book. Accessed 15 April 2020. HM Treasury. (2018). The Green Book: Central Government Guidance on Appraisal and Evaluation. https://assets.publishing.service.gov.uk/govern ment/uploads/system/uploads/attachment_data/file/685903/The_Green_ Book.pdf. Accessed 10 October 2019. Holder, S. (2020). The CSQ Interview: Sir Mark Sedwill, Cabinet Secretary and Head of the Civil Service. Civil Service Quarterly. https://quarterly. blog.gov.uk/2020/02/13/the-csq-interview-sir-mark-sedwill-cabinet-secret ary-and-head-of-the-civil-service/. Accessed 18 April 2020. House of Commons. (2014). Environmental Audit Committee, Well-being Report Volume I (HC 59). https://publications.parliament.uk/pa/cm2 01314/cmselect/cmenvaud/59/59.pdf. Accessed 19 March 2020. House of Commons. (2019). Committee of Public Accounts Report: Challenges in Using Data Across Government (HC 2492). https://publications.parlia ment.uk/pa/cm201719/cmselect/cmpubacc/2492/2492.pdf. Accessed 19 October 2019. Jenkins, M. (2018). The Politics of the Official Statistic: The UK ‘Measuring National Well-being’ Programme. In I. Bache & K. Scott (Eds.), The Politics of Wellbeing; Theory, Policy and Practice (pp. 279–299). London: Palgrave MacMillan. Layard, R., Clark, A. E., De Neve, J. -E., Krekel, C., Fancourt, D., Hey, N., & O’Donnell, G. (2020). When to Release the Lockdown: A Wellbeing Framework for Analysing Costs and Benefits. London: Centre for Economic Performance. http://cep.lse.ac.uk/pubs/download/occasional/ op049.pdf. Accessed 24 April 2020. Mayer, C. (2013). Unnatural Capital Accounting. https://assets.publishing. service.gov.uk/government/uploads/system/uploads/attachment_data/file/ 516947/ncc-discussion-paper-unnatural-capital-accounting.pdf. Accessed 20 October 2019. National Assembly for Wales. (2015). Well-being of Future Generations (Wales) Act 2015. http://www.legislation.gov.uk/anaw/2015/2/contents. Accessed 23 January 2020. New Statesman. (2020, January 24). Editorial: The Limits of Green Capitalism. New Statesman, 5. O’Donnell, G., Deaton, A., Durand, M., Halpern, D., & Layard, R. (2014). Wellbeing and Policy. London: Legatum Institute.

80

P. ALLIN AND D. J. HAND

ONS. (2019, November). Sustainable Development Goals in the UK: Progress On Monitoring and Reporting Data. https://www.ons.gov.uk/economy/enviro nmentalaccounts/articles/sustainabledevelopmentgoalstakingstockprogressan dpossibilities/november2019. Accessed 18 January 2020. ONS, Understanding Society, NatCen. (2020). Unresolved Public Policy Challenges. http://natcen.ac.uk/our-expertise/unresolved-public-policy-cha llenges/unresolved-public-policy-challenges/. Accessed 7 March 2020. OSR. (2018). Code of Practice for Official Statistics, Edition 2.0. Office for Statistics Regulation, UK Statistics Authority, London. https://www.statisticsautho rity.gov.uk/code-of-practice/. Accessed 10 October 2019. OSR. (2019). Statistics that Serve the Public Good: OSR’s Vision: What We Do and Why. Office for Statistics Regulation, UK Statistics Authority, London. https://www.statisticsauthority.gov.uk/osr/our-vision/. Accessed 12 October 2019. PACAC. (2019). Governance of Official Statistics: Redefining the Dual Role of the UK Statistics Authority; and Re-Evaluating the Statistics and Registration Service Act 2007 (HC 1820). London: House of Commons. https://publications.parliament.uk/pa/cm201719/cmselect/ cmpubadm/1820/1820.pdf. Accessed 10 October 2019. Policy Profession. (2013). Twelve Actions to Professionalise Policy Making: A report by the Policy Profession Board. https://assets.publishing.service.gov.uk/ government/uploads/system/uploads/attachment_data/file/805991/12_ actions_report_web_accessible.pdf. Accessed 17 April 2020. Policy Profession. (2019). Policy Profession Standards: A Framework for Professional Development. https://assets.publishing.service.gov.uk/government/upl oads/system/uploads/attachment_data/file/851078/Policy_Profession_Stan dards_AUG19.pdf. Accessed 17 April 2020. RSS. (2019). Written Evidence Submitted by Royal Statistical Society. http:// data.parliament.uk/WrittenEvidence/CommitteeEvidence.svc/EvidenceD ocument/Public%20Accounts/Challenges%20in%20using%20data%20across% 20Government/Written/103658.html. Accessed 19 October 2019. Rudoe, W. (1997). Obituary: Sir Harry Campion, 1905–1996. Journal of the Royal Statistical Society Series A (Statistics in Society), 160(1), 148–151. http://www.jstor.org/stable/2983385. Selwood, S. (2019). A Possible Teleology of Cultural Sector Data in England. Cultural Trends, 28(2–3), 177–197. https://doi.org/10.1080/09548963. 2019.1617940. Sense About Science. (2018). Transparency of Evidence: A Spot Check of Government Policy Proposals July 2016 to July 2017 . https://senseaboutsc ience.org/wp-content/uploads/2018/01/Transparency-of-evidence-spotch eck.pdf. Accessed 13 October 2019.

3

STATISTICS AND PUBLIC POLICY

81

Sethia, D. (2016). Regional Accounts of India: Methods, New Estimates and their Uses. Review of Income and Wealth, 62, 92–119. Taylor, D. J. (2018). Fayke News: The Media vs the Mighty from Henry VIII to Donald Trump. Stroud: The History Press. Truss, E. (2019). What Should the Spending Review Focus On? Speech by the Chief Secretary to the Treasury. https://www.gov.uk/government/speeches/ what-should-the-spending-review-focus-on-speech-by-the-chief-secretary-tothe-treasury. Accessed 18 January 2020. United Nations. (2014). Fundamental Principles of Official Statistics. https:// unstats.un.org/unsd/dnss/gp/FP-New-E.pdf. Accessed 9 March 2020. United Nations. (2015). Transforming Our World: The 2030 Agenda for Sustainable Development. https://sustainabledevelopment.un.org/post2015/ transformingourworld. Accessed 18 January 2020. United Nations. (2016). The Sustainable Development Goals Report 2016. http://unstats.un.org/sdgs/report/2016/The%20Sustainable%20Develop ment%20Goals%20Report%202016.pdf. Accessed 12 October 2019. Wallace, J. (2019). Wellbeing and Devolution: Reframing the Role of Government in Scotland. Wales and Northern Ireland: Palgrave Macmillan, Cham, Switzerland. https://doi.org/10.1007/978-3-030-02230-3. Wells, P. (2010). Review Article––A Nudge One Way, A Nudge the Other: Libertarian Paternalism as Political Strategy. People, Place and Policy, 4 (3):111–118. http://extra.shu.ac.uk/ppp-online/review-article-a-nudge-oneway-a-nudge-the-other-libertarian-paternalism-as-political-strategy/. Accessed 25 January 2020. What Works Network. (2018). The What Works Network Five Years On. https:// assets.publishing.service.gov.uk/government/uploads/system/uploads/att achment_data/file/677478/6.4154_What_works_report_Final.pdf. Accessed 23 January 2020. Whitby, A. (2014). BRAINPOoL Project Final Report: Beyond GDP––From Measurement to Politics and Policy. World Future Council. https://www.wor ldfuturecouncil.org/wp-content/uploads/2016/01/BRAINPOoL_2014_B eyond-GDP_From_Measurement_to_Politics_and_Policy.pdf. Accessed 13 October 2019. Zuboff, S. (2019). The Age of Surveillance Capitalism. London: Profile Books.

CHAPTER 4

Wider Audiences for New Measures of Progress

Abstract This chapter explores the relationship between the production and the use of statistics to yield beneficial social change. It seems that, although strides are being made, there is insufficient engagement with users and potential users, as well as with the multiple intermediaries between producers and users, to discover exactly what they need. This hinders attempts to determine the public value of official statistics. In addition, there is often a lack of awareness of what statistics are available, and of how to access them, as well as of the efforts made to ensure the trustworthiness and quality of official statistics. This can result in misunderstanding the basic aspects of society. We suggest four broad ways that statistics can be used to yield beneficial change: improving national and local debates, helping to build a national debate, exploring what it means if something is not included in a nation’s set of official statistics, helping to drive behaviour change in enacted policies. Keywords Users of statistics · Public value · Trustworthiness

4.1

Introduction

Understanding of the state of homeless people in the UK is only gradually being acquired. At 9.30am on 1st October 2019, the UK Office for National Statistics (ONS) released updated statistics of the number © The Author(s) 2021 P. Allin and D. J. Hand, From GDP to Sustainable Wellbeing, Wellbeing in Politics and Policy, https://doi.org/10.1007/978-3-030-53085-3_4

83

84

P. ALLIN AND D. J. HAND

of deaths of homeless people in England and Wales (ONS 2019a). These were experimental statistics, derived using statistical modelling to estimate the number of deaths of homeless people, whether or not it was stated that they were homeless on their death registration. One previous set of such statistics had been published, a year previously, when the ONS reported on deaths in 2013–2017. The updated statistics showed an estimated 726 deaths of homeless people in England and Wales registered in 2018, an annual increase of 22% and the highest year-to-year increase in the figures over the six-year period. This was widely and prominently reported in the media and circulated on social media. The newspapers for the following day carried reports based on the figures, along with opinion columns calling for changes in legislation, for care services to be improved and new, preventative approaches to be adopted. It was also reported that “Two dozen homeless protestors have stormed a council-owned building in Chester, north-west England, and barricaded themselves inside … The protest began as official figures revealed that a record number of homeless people died last year” (WolfeRobinson 2019). Even if this was just a reported coincidence, and whatever the rights and wrongs of such direct action, it does prompt us to wonder if statistics really do lead people to take action to change their situation themselves. We have frequently made the point that developing measures is not enough, but those measures have to be used: an example of what Marx argued in 1845, that “Philosophers have hitherto only interpreted the world in various ways; the point is to change it” (Thesis 11 in Smith and Cuckson 2002). It is our thesis in this book that statisticians as well as philosophers are needed to help us all interpret and change the world. It is not just down to policy-makers to use statistics (the subject of the previous chapter). In President Sarkozy’s comment welcoming the Stiglitz report that we quoted in Chapter 2, we (the authors) take it that he was being expansive in his remarks, not just directing his remarks to other politicians. This is not the only example by far of the use of such inclusive language. Germaine Greer noted that a film celebrity commented on the Australian bush fires thus: “The tragedy unfolding in Australia is climate changebased. We need to act based on science, move our global workforce to renewable energy and respect our planet for the unique and amazing place it is”. This prompted Greer to ask “Who, one wonders, is the ‘we’ he admonishes? What global director has the power to move the world’s workforce?” (Greer 2020, p. 21).

4

WIDER AUDIENCES FOR NEW MEASURES OF PROGRESS

85

In this chapter we explore how official statistics can be used throughout processes of social change, looking to the use of official statistics by civil society, businesses, and others. We anchor our exploration to the UN’s fundamental principles of official statistics introduced in Chapter 1, particularly that official statistics should be useful to the Government, the economy, and the public, and that they should earn their place as an indispensable element in the information system of a democratic society.

4.2

The Public Value of Official Statistics

A concise way of summarising the role of official statistics as set out in the UN’s fundamental principles is simply to say that official statistics should serve the public good. That is how the UK’s Office for Statistics Regulation (OSR 2019) describes its vision for official statistics, explaining “It means far more than the traditional notion that statistics provide the evidence base for policy decisions by Ministers and Parliaments, important though this is. Statistics should meet the needs of a much wider range of users and this is the essence of how they serve the public good” (OSR 2019, p. 2). Listing the breadth of the potential user base beyond policy is one way of illustrating the public good of official statistics: the OSR mentions organisations beyond governments and parliaments “including charities, researchers, trade unions, businesses and community groups” as well as citizens (OSR 2019, p. 2). The kinds of uses identified by OSR are listed in Table 4.1 and to which we have added some other uses that we are aware of. Education is included because schools were a major user of a previous version of a set of sustainable development indicators, for example. The OSR is formally part of the UK Statistics Authority, under which the ONS also sits, so we can take this typology as broadly representing how UK official statisticians understand their outputs to be used. A similar view of the role of official statistics has been captured for the European statistical system, through which Eurostat, the statistical office of the European Union, collects and collates statistics from member states for the EU as a whole. Vichi et al. (2015) constructed a classification of users that we summarise in Table 4.2 and a taxonomy that allocates users “according to their frequency of statistical usage and proficiency”, which we show in Table 4.3.

86

P. ALLIN AND D. J. HAND

Table 4.1 Kinds of uses of official statistics (OSR 2019, pp. 2–3, with additions from authors shown in italics ) How used Provide the evidence base for policy decisions by Ministers and Parliament Help inform decisions by organisations in civil society Help civil society organisations hold government to account Influence choices made by citizens Inform many public and political debates, raise awareness, advocate positions, and proposals Help construct a sense of place, society and democracy

In education and training

Comments Also used by devolved governments and administrations, local government Including charities, researchers, trade unions, businesses, and community groups Including how they vote, where they live and a wide range of other decisions For example, about health, education, the economy, crime, the environment, and many other topics Through the pervasive uses above and in education. Also, the process of developing new statistics could offer ways for citizens to be fully engaged and to be co-creators of social advance Including statistical literacy and data handling skills, as well as supporting the curriculum

Table 4.2 Users of European official statistics (Vichi et al. 2015, p. 3 and Appendix 1) Institutional users Non-institutional users

10 named institutions, headed by the European Parliament and the European Council 1. Users with a general interest (e.g., economic growth): • Journalists and media • Citizens • Students (by level of education, or age) and Teachers (by level of teaching education) 2. Users with a specific subject/domain interest (e.g., health): • Other decision-makers • Policy analysts • Marketing analysts • Experts in a specific field 3. Users with a research interest (e.g., innovation in enterprises) • Scientific community—academics and researchers at universities and research institutions • Consultants and researchers in Governmental Agencies and private sector

4

WIDER AUDIENCES FOR NEW MEASURES OF PROGRESS

87

Table 4.3 A taxonomy of individual users according to their frequency of statistical usage and proficiency (Vichi et al. 2015, p. 4) Heavy users

—of which, very heavy users

Light (occasional) users

Non-users who might be Potential users

Researcher, specialist, politically or civically engaged citizen, and others that use statistics on a daily basis. Typically, this is the person who knows where to find data and how to interpret it Researchers who would be routinely engaged in using disaggregated and micro data in their research and who could contribute to the improvement of data quality by engaging with data producers User who from time to time checks some figures. He/she would know the National Official Statistics and Eurostat websites but would find some difficulty in getting the data he/she needs and would not be looking for metadata All people who do not go looking for data believing it is something hard to understand and not being aware of data’s relevance and richness

Classifications and typologies like these are generally, and by definition, reductive. They are helpful in putting some structure into a plethora of individual users (and non-users) and can reinforce the point that “the needs of both political authorities and the common public are equalised” (Sæbø and Holmberg 2019, p. 173). However, we should be aware of the dangers of relying over-heavily on typologies of users. The categories may well reflect the experience and background of their compilers, as well as the existing suite of official statistics and dissemination channels. These may all be valuable but incomplete. For example, a list of user segments published by the UNECE (2018, p. 8) looks very similar to the segments in Table 4.2, with the addition of “Users with a reuse and reproduction interest”. That includes other producers of official statistics, and private or government organizations providing information services and products, including App builders. It conveys a fuller picture of official statistics in the contemporary information market, but we must still ask if it is complete.

88

P. ALLIN AND D. J. HAND

Classifications such as these tell us nothing about the relative size of the user base in each row of the table, or about the overall value of official statistics to each type of user category. Drawing up tables like these is therefore the starting point for a fuller understanding of the public value of official statistics; these tables should not be read as an assessment of public value. Work on the wider interpretation by official statisticians is under way, following a recent publication by the UN Economic Commission for Europe (UNECE 2018) that “showcases the value of official statistics and provides recommendations for statistical offices on ways to promote, measure and communicate this value”. There were few such valuations ahead of the recommendations and we are writing too soon after the publication of the recommendations for new figures to appear. Early examples tend to be for specific sets of statistics, rather than for official statistics as a whole. For example, Bakker (2014, p. 6) valued the New Zealand census of population, presenting a range of estimates of its net present value with a central conclusion of “close to $1 billion for the benefits to New Zealand gained through the use of census and population statistics information over the next 25 years. In other words, every dollar invested in the census generates a net benefit of five dollars in the economy. This value estimate though is not at the level of rigour applicable to assets recorded on an organisation’s balance sheet. It does not include many of the uses discussed but not quantified”. It is quite likely that big numbers will emerge from exercises like this, even if they are billed as lower bounds of the public value of official statistics. These numbers may well get even larger with greater use of the underlying data, anonymised for research, in fields such as commercial pharmaceutical research. Helm (2019) has reported that several US companies “have paid the Department of Health and Social Care, which holds data derived from GPs’ surgeries, for licences costing up to £330,000 each in return for anonymised data to be used for research” and that Britain’s 55 million health records are “estimated to have a total value of £10bn a year”. Official statistics based on data derived from general practitioners’ (GPs) surgeries include, for example, local area migration statistics, for which registrations with GPs are one of a number of sources. The ONS announced in September 2019 that it was beginning work to lead a UN project “to measure the true value of the world’s official statistics” because “until now, little work has been done to assess the monetary value” of these important datasets, which include measures of GDP,

4

WIDER AUDIENCES FOR NEW MEASURES OF PROGRESS

89

migration, and international trade (ONS 2019b). As this perhaps hints, research users are increasingly looking to access anonymised datasets within secure environments, as well as the statistics compiled from the data. This adds to the difficulty of measuring the value of statistics and in communicating this value to the public, who are ultimately the source of much of the data used for official statistics. Nevertheless, there is a logic to knowing the monetary value of official statistics. The ONS news release quotes Fiona Willis-Núñez at the UNECE: “We official statisticians like to repeat the adage that only when something is measured does it become truly visible and understandable. We know our products are important, but being able to prove this to decision makers relies on having quantifiable evidence of our efficiency and value for money” (ONS 2019b). The greater benefit of the exercise, though, would surely be the increased detailed knowledge that should result, about who uses official statistics, how they are used, and why they are not used. The UNECE recommendations acknowledge that “Statistics need to be developed with users in mind. User needs differ depending on circumstances … It may be useful to identify user segments so that it becomes possible to develop products and services that meet specific user needs better” (UNECE 2018, p. 8). Frankly, coming up with the total value of official statistics is likely to be a distraction. The real goal is to increase the use and usefulness of official statistics, which would of course be achieved if each statistical output became more widely used. We mentioned an example of the potential for this at the end of Chapter 3, to do with pedestrian deaths in the US. It was reported that the “small community of specialists who pay attention to US road safety statistics” saw in 2010 an upturn in the number of pedestrians killed on American roads, which was the first of a series of increases that has continued almost every year since (Baker 2019). This led to some debate about the design of streets as well as the more technological advances in car design. But, we suggest, would it not have been better if the small community of users of these statistics had been larger from the outset and included all sorts of interest groups, including in urban planning, vehicle design, and road safety? Very little seems to have been studied and published about how official statistics are used, or why they are not used. Bartlett and Tkacz (2017, p. 22) recognise “a well-established field of inquiry which has studied the role of numbers, facts and data in organisational life”, including for example finding that while numbers “can appear clear, objective and

90

P. ALLIN AND D. J. HAND

‘transparent’, this appearance can be misleading”. The literature tends to treat numbers and data in a generalised way, rather than focusing in on the different sources of numbers and data, especially official statistics. There are of course many papers and articles reporting on analyses of particular official statistics or data, invariably with some mention of potential policy applications, but these shed little light on how statistics are used within specific policy or other decision-making. There are also papers looking at how national and regional statistical systems operate in practice. For example, writing from Statistics Norway, Sæbø and Holmberg (2019, p. 173) discuss challenges in meeting the differing needs of a variety of users and note that the European code of practice requires that “procedures to consult users must be in place and that relevance and user satisfaction is monitored. Statistics must change according to user needs, at the same time as comparability over time must be considered”. However, while relevance of the statistics (i.e. meeting user needs) might be the most important attribute of statistics produced by a national statistical office, it must be supported by other requirements such as quality, cooperation, resources, and cost effectiveness on one side of a set of scales, and, on the other side, data confidentiality, professional independence, and impartiality (p. 172). Bache (2019) has studied “the role of evidence in shaping the prospects for wellbeing in UK public policy” (p. 1) but places official statistics explicitly (as “Government surveys [e.g. ONS]”) as just one entry in his list of 24 evidence sources reported by his interviewees (Table 4.1, p. 57). The list suggests to us that official statistics are not a primary source, but may well be included in other types of sources, such as academic papers, government reports, seminars, and the internet (but of course we do not know this for certain). This supposition is supported by a quote reported by Bache: “obviously statistics hold quite a large sway … but they need the stories behind it, so they need the qualitative work either to help them understand what the statistic is saying or for them to be able to translate that into a real-world environment for their decision-makers” (p. 108). These findings fit with our understanding of how statistics are generally used. Similarly, Bache and Reardon (2016, p. 132) make the point that “data should be used that is ‘good enough’ for the purposes at hand” and that there is “an emerging consensus on a mix of subjective and objective indicators”, as delivered by the ONS’s measuring national wellbeing programme. However, that only takes us to the supply of relevant data on

4

WIDER AUDIENCES FOR NEW MEASURES OF PROGRESS

91

wellbeing, not to its use, or even to wide enough awareness of its existence and how to access it. Hornik and Cherian (1993, p. 229) have argued that, “Literature about data-use has tended to overstate the importance of data-providers … and to understate the importance of data-users”. They were addressing their observations particularly to the field of marketing, where data is provided by market researchers and used by product and brand managers, but again the point probably holds generally. Models for data use, which appear applicable to official statistics, have long existed. Miller and Mork (2013, p. 57) take the well-established concept of a value-added chain and propose a data value chain as a framework to manage big data “holistically from capture to decision making and to support a variety of stakeholders and their technologies”. Value chains such as this are often presented as a linear pathway, from collecting data to decision-making. Miller and Mork’s first link in the chain involves “creating an inventory of available data sources and the metadata that describes the quality of those sources in terms of completeness, validity, consistency, timeliness, and accuracy” (2013, p. 57). This is a familiar starting point in official statistics, often taken with the aim of meeting a range of user requirements that are specified only in general terms. However, the value accumulated through a chain of activities should be enhanced if there is explicit feedback of user requirements from the start. Part of the final, decision-making stage should ideally be to join the ends of the chain together, with recommendations for changes or additions to future data collection. Kandogan et al. (2013) also see the potential of big data to answer open-ended questions. Their interest is particularly in business users of data but their approach is applicable to all users. They reported that “the tools of data science are still oriented toward the skills of a technical staff person rather than a business user. The new challenge is to enable business users with the powers of data science”. These authors, from IBM Research, propose “rapid-fire conversations in which data is automatically found, cleaned, transformed, and visualized so users can go through a number of questions very quickly in an iterative manner. Central to our system is social and intelligent conversations with data, where analytic work is placed in a social context with a user experience that matches most social networking applications, and in which our system participates in the conversation like an intelligent partner, recommending datasets, visualizations, and people with whom to collaborate” (2013, p. 427). Central to this approach is the theme we noted earlier, of drawing on

92

P. ALLIN AND D. J. HAND

multiple sources, here recognising that some datasets might be recommended over others. We will discuss data science and official statistics in more depth in the next chapter. We also urge readers to accept that big data is not immune from mistaken analyses and conclusions. Anything that seeks to increase the use and usefulness of official statistics will invariably have challenges to meet and barriers to overcome. Using any form of evidence may not be easy and may require skills, expertise, and experience to use it effectively. These skills also need to be mirrored by producers, to anticipate how their data and statistics can be applied. There are things that the producers of evidence can do to make their products more useful and more widely available. Support comes from initiatives and networks like the Alliance for Useful Evidence (https://www.alliance4usefulevidence.org/), which offer practical advice and act as evangelists for evidence, especially for useful evidence. This chimes well with the key characteristic of official statistics, to have utility. There is an important role to be played by intermediaries who add value to statistics and data beyond producer organisations. This is no better exemplified than by the late Hans Rosling, whose innovative presentations based on official data are available, along with new material, at the Gapminder foundation (https://www.gapminder.org/). The foundation “fights devastating misconceptions about global development … produces free teaching resources making the world understandable based on reliable statistics … promotes a fact-based worldview everyone can understand”. We will return later in this chapter to the role of influencers and intermediaries in the context of measures of wellbeing and progress. To conclude this section on the public value of official statistics, we again make the point that official statistics exist as part of a complex ecosystem made up of multiple sources of information and multiple transmission routes. It seems to us that the journey of any official statistic from its publication to any one of its uses might be considered against the steps shown in Table 4.4. This simplistic representation still suggests that there might be eight ways in which the statistic is used, only one of which is where it is the right statistic for the intended use, is quoted correctly, and was accessed directly from the national statistics office or other official source website or publication. Table 4.4 should be read only from left to right. Starting on the right-hand side with a question about statistics (e.g., what is the current level of net migration in the UK?) will involve the stages in Table 4.4 but also other choices, including the medium used to post the question. Using an Internet search engine, for example, can

4

WIDER AUDIENCES FOR NEW MEASURES OF PROGRESS

93

Table 4.4 The journeys that an official statistic can take Statistic in official source

→

User accesses official source OR

User accesses another source that quotes this statistic

→

Statistic is appropriate to intended use OR Statistic is not appropriate

→

Statistic is quoted correctly OR

Statistic is used

Statistic is mis-quoted

lead directly to official sources, but also to international collections of data, research agencies, think tanks, and media reports of the published data. Using social media opens up even more diverse sources.

4.3

Can Official Statistics Change Anything?

One reading of the Fundamental Principles of Official Statistics might be to conclude that it is sufficient that official statistics are used to reflect the social, economic, and environmental situation, acting as mirror held up to society. This is a passive role, albeit a form of infrastructure essential to a democratic society and available for all to use. It might just move political discourse onto a firmer footing, for example, but only if politicians used it. Interviewed in 2012, cultural theorist Stuart Hall observed that “Politicians always think they know what people feel. It’s a fallacy, because there is no such thing as ‘the people’. It is a discursive device for summoning the people that you want. You’re constructing the people, you’re not reflecting the people” (Williams 2012, para 14). We now have in the UK, and in other countries, official measures based on how individuals feel, in terms of their personal wellbeing, as well as measures of health, welfare, and social and economic position. However, having these measures available does not mean that everyone is aware of them or can quote them. Ipsos has been running studies on public awareness of what the statistics say for key social realities in their country (https://perils.ipsos.com/). Participants in these studies are asked to say what they think is the average happiness score, along with other statistics such as the level of immigration, rates of crime, teenage pregnancy, and obesity. In an overall assessment of the extent of misperceptions in 13 countries studied since 2014, Italy scored the worst.

94

P. ALLIN AND D. J. HAND

Sweden showed the least extent of misperception, around half of that for Italy, with Britain having a score around the middle of the range (Duffy 2018, p. 218). In short, “many of us get a lot of basic social and political facts very wrong” (p. 18). Of course, this might not be too severe a criticism, since it might be better to appreciate that facts are widely available and know where to look things up, rather than hold more than a handful of memorable statistics in one’s head, especially since the facts change over time. There are no shortages of examples where referring to official statistics could help understanding of what is going on, and even demonstrate that we are living in, or on the brink of, a golden age for statistics. But this does not always happen. One example is that official statistics were tracking, at least in some countries, increasing inequality “even though GDP per capita was going up” (instanced by the US, where “most individuals saw a decline in income, adjusted for inflation” over the same period: Stiglitz et al. 2010, p. xix). The prevailing political narrative was around increasing prosperity for the nation as a whole, apparently oblivious to the personal experiences of the majority of the population. There are major issues around how information is used and abused. Misperceptions and missed opportunities arise for all sorts of reasons, not least because people are not aware of where to find the official statistics or the analyses in which they are used. In terms of more manageable aspects to this, we might speculate that one, perhaps minor, contributory factor might be the extent to which particular official statistics enter the public domain. A number of official statistics might be judged to have gained the attention of the media at least. For example, GDP, unemployment, and inflation statistics are high-profile and have regular and frequent publication to pre-announced release dates, so they can be factored into news planning. Other official statistics occasionally gain wide attention, especially if they are novel, as was the case with the statistics on the deaths of homeless people quoted at the beginning of this chapter. We recall the launch of the ONS’s national wellbeing measures in 2010 by the then Prime Minister. These gained brief media attention as Mr. Cameron’s Happiness Index before fading from the headlines and feature pages. Away from the headlines, many specific sets of statistics are known to, and anticipated by, relatively small groups of engaged users (for example, maternity statistics). That leaves a very large number of official statistical releases for which, at best, their use and user base is unknown. This was noted by the UK Parliamentary committee looking into the

4

WIDER AUDIENCES FOR NEW MEASURES OF PROGRESS

95

governance of official statistics mentioned in Chapters 2 and 3. The committee began its conclusions thus: “We agree with the evidence we received that those producing official statistics do not understand all of today’s users and potential users of statistics and how statistics are used. It is surprising that [UK official statisticians] seem not to have carried out research into users … not delivering public good as required under the legislation” (PACAC 2019, p. 54). The committee recommends that cross-government research should take place, “to build an evidence base of how statistics are used in practice, taking into account the full breadth of stakeholders (not just users) and to establish where data gaps persist”. The committee recommended a structured approach, with “sector by sector reviews, to understand what stakeholders need or want, and to make statistics more relevant”. The UK National Statistician accepted these recommendations and this market research is now (i.e. May 2020) getting under way. When the results will appear is unclear, especially now that the coronavirus pandemic has curtailed activities. So, can official statistics change anything? Our experience of UK and European official statistics does leave us optimistic that official statistics are being used, and can be used even more. In part this rests on the concept of having a set of official statistics available for a country, along with processes for the system to evolve. We suggest four broad ways that statistics can help change things, as follows. First, to improve the quality and effectiveness of national or local debates and conversations about the country we live in. One uncontested sign of a modern democracy might be that it encourages a variety of opinions to be heard within society and within public, civil and businesses organisations and institutions. Statistics can help structure the discourse by providing a common set of facts, by keeping people informed and, indeed, by keeping issues alive. This can sound both idealistic and simplistic, but seems worth pursuing so that, for example, the statistics used in policy-making are also those available to assess the performance of government. It assumes that a distinction can be made, and recognised, between opinion and fact. As the masthead in the Guardian newspaper puts it, “Comment is free … but facts are sacred”. Problems arise when facts get caught up in social conventions which restrict the range of statistics available or, if they are available, how they are presented, consulted, and used. A particularly challenging aspect of official statistics is to provide statistics that can be used by all sides of a debate, including by those who

96

P. ALLIN AND D. J. HAND

want to speak truth to power, especially the power manifest in government, on issues such as poverty, homelessness, and government spending. Scott has studied how sustainability indicators are used in decision-making at all levels, including the production of many community quality of life indicators. She observes that “The debate and learning that occurred in the process of developing indicators became as important as the product itself” (Scott 2012, p. 46). Second, and this may only happen relatively infrequently, to help build a national debate. We have in mind particularly the wide-ranging national debate on measuring national wellbeing, hosted by the ONS, with the strap line “What matters to you?” (Allin and Hand 2014, pp. 222–223). Other national statistical offices have undertaken similar exercises. This is not without challenges. During the development of an earlier set of sustainable development indicators, Custance and Hillier (1998, p. 287) wanted as one role of the indicators “to encourage individuals and businesses to recognize that their behaviour and choices have an effect, and we hope by highlighting those effects to influence them to behave in a more sustainable way”. But in presenting indicators “in ways which people can relate to their own experience”, not surprisingly, some compromise in scientific soundness had to be made in favour of indicators that resonated with people. Third, a lack of statistics can signal that an issue is not being addressed. It can be instructive to consider what it means if something is not included in a nation’s set of official statistics. It seems to be a management consultancy mantra that what counts should be counted, but too often it is what is counted that counts. For example, a report in 1980 noted that although “a sizeable bank of official data on which to draw” in researching equality between women and men exists, it was the case that “the assumptions underlying the statistical portrayal of women are, at points, so divorced from reality as to be dangerously misleading”, such as that the chief economic supporter of a household is invariably male (Equal Opportunities Commission 1980, p. 1). Official statistics have developed in many ways over the last 40 years but there are still significant gaps, for example in measuring household work as a contribution to the performance of the economy. Moreover, as Caroline Criado Perez (2019, p. 1) demonstrates, there are many cases where the data used for policy, for resource allocation and other decision-making, reflects “men as the human default”.

4

WIDER AUDIENCES FOR NEW MEASURES OF PROGRESS

97

These three categories suggest how statistics play (including by their absence) into the politics and problem streams of influence on change relating to wellbeing, as discussed by Bache and Reardon (2016, p. 148). Our fourth category is more practical, it is that official statistics can help drive the behaviour change envisaged in enacted policies. The popular view is that change needs to be led by and stimulated by government, that it is the role of government to improve wellbeing and sustainability in response to issues highlighted by experts, politicians and pressure groups, and specific events or other problems (Bache and Reardon 2016, p. 148). The work of official statistics is not finished when governments acknowledge the need for more sustainable development and act to encourage and reward sustainable business and household activity, for example redesigning transport networks or shifting to renewable energy. The full range of official statistics, including GDP but no longer led by it, needs to be presented, to allow trade-offs to be assessed and acted on by businesses and households. Changing how we live may not just be about stopping some things but also involve shifting the balance and changing priorities. This is a huge challenge to official statistics since, in our view, one of the reasons that the vision of UN fundamental principles of official statistics may be hard to realise is that statistical agencies do not have access to unlimited resources. We concluded Chapter 3, on statistics in policy, with a brief mention of how policy is meant to lead to behaviour change. That is often seen as the way of changing society and of meeting goals such as the SDGs, through top-down approaches such as legislation, incentives, and nudges. On the other hand, it is at least an open question as to whether bottom-up activism, with individual people and businesses deciding for themselves, is also effective, for example in tackling the climate emergency. We have huge admiration for the insistence of Greta Thunberg (2019) that no one is too small to make a difference. Elsewhere there are many other practical solutions offered to individuals and households: one we spotted urged “persuade your neighbours to do the same”. We can see that official statistics can form part of the argument and rationale for change. It is less clear how they might be used after that. The Economist (2017) observed that “Social change often starts with a grassroots movement. It can promote new ways of thinking or reveal injustices that had long been ignored. New behavioural rules may follow. But if these emerging norms are not embraced by big parts of the population, they will not become

98

P. ALLIN AND D. J. HAND

entrenched. And if transgressions are seen to go unremarked or unpunished, they will continue”. It seems all too easy for many of us to carry on living in the way we are used to, despite some acknowledgement that things could be changed for the better. What role for official statistics here? What can companies do? Beyond GDP can be traced back in part to a reaction to neoliberalism ideology, especially aspects such as that publicly listed companies should direct their efforts purely to the benefit of their shareholders. This was said to lead to a drive for economic growth with no regard for wider impacts, be they benefits or dis-benefits. In response to that, and long pre-dating the SDGs, more businesses began giving attention to their social and environmental impacts. There is an increasing number of corporate, non-financial, or sustainability reporting approaches, such as guides published by www.accountingforsustainabil ity.org, so that a business’s scorecard not only covers financial aspects but also “doing the right thing” in terms of sustainability. Orts and Spigonardo (2020, p. 10) talks of the need to reconcile profits with “sustainable well-being”. Companies, non-governmental organisations, individuals, and the Wellbeing Economy Governments (see Chapter 3) are working through the Wellbeing Economy Alliance (2020, p. 3) to “transform the economic system into one that delivers human and ecological wellbeing”. The Alliance has noted the need for “A strong and coherent knowledge and evidence base”, not only on the theory underpinning the wellbeing economy, but also that “the evidence base of what works in practice needs to be galvanised and proactively disseminated. There is also a need to explore and demonstrate the effectiveness of wellbeing economy approaches on a large scale” (Wellbeing Economy Alliance 2020, p. 5). Some businesses and educational institutions have engaged with the SDGs relevant to their activities, supported by a number of national and international initiatives. For example, the Global Reporting Initiative, a recognised sustainability reporting approach, and the United Nations Global Compact are collaborating to “accelerate corporate reporting on the Global Goals” (GRI n.d.). Alignment and standardisation across wider accounting frameworks are also being explored. All of these initiatives and approaches start from a corporate decision to account to stakeholders—including their investors, employees, communities, regulators, and governments—in non-financial terms as well as standard financial reports. Where these wider reports are aligned with

4

WIDER AUDIENCES FOR NEW MEASURES OF PROGRESS

99

SDGs, then they illustrate how the goals are being tackled and should help governments report on progress towards the goals. UNCTAD (2019, p. 18) has prepared guidance on this, based on “a limited number of core SDG indicators” and sees (Fig. 1) these indicators as assisting “Governments to assess the private sector contribution to the SDG implementation” and “entities to provide baseline data on sustainability issues in a consistent and comparable manner that would meet common needs of many different stakeholders of the SDG agenda”. Widespread non-financial reporting in a consistent and timely way offers the prospect of aggregating costs and benefits to reveal national analyses of economic growth, social progress, and environmental impact. However, we have as yet no evidence of organisations referring specifically to published, national sustainable development indicators or linking their own performance metrics to these indicators, beyond the proposal in the UNCTAD guidance. In our view, this points to a clear role for national statistical offices to engage more fully with the world of non-financial reporting, including by recognising that businesses are increasingly linking their research and development to the SDGs relevant to them.

4.4

The Role of Influencers and Intermediaries

We take the optimistic view that there appear to be multiple opportunities for official statistics on wellbeing and sustainable development to be used in building the evidence base for decision-making, action and evaluation, at all levels of society. But our thesis is that having statistics available is not enough, we need to see them as tools for social change and use them as such. However, at the risk of labouring this point, official statistics are only part of the contemporary information space: the mirror held up by official statisticians does not stand alone and may not stand out. As novelist Olga Tokarczuk (2019, Part 2) put it in her Nobel Lecture: “The world is a fabric we weave daily on the great looms of information, discussions, films, books, gossip, little anecdotes. Today the purview of these looms is enormous - thanks to the internet, almost everyone can take place in the process, taking responsibility and not, lovingly and hatefully, for better and for worse. When this story changes, so does the world. In this sense, the world is made of words”. (Well, words and numbers perhaps, or including words about numbers).

100

P. ALLIN AND D. J. HAND

But we again need to ask who is the “we” who are doing the weaving of multiple data and information threads? There is likely to be a spectrum of engagement, rather like the taxonomy of statistical users in Table 4.3 above. We have also observed that the interrelated topics of wellbeing, progress, and sustainable development have gained a number of intermediaries and change-influencers. There are many storytellers, compilers of statistical compendia painting fuller pictures of societal development. They are curators, drawing from the available sets of statistics. For example: Prescott-Allen (2001, p. 4) compiled an index of quality of life and the environment for the 180 countries for which data could be obtained on at least half the indicators in his model; Briscoe (2005, p. ix) prepared a volume of the “essential statistics” for Britain because “despite seeing more and more figures being published every year, people are often no clearer about the important trends in our society”; and Ahsan and Tweed (2015) produced a statistical snapshot of Canada to coincide with national day. As one-off publications, these have a relatively short “shelflife” as topical accounts though they remain as useful archival sources of how the compiler saw the wellbeing of the nation at a point in time. This is perhaps being a little unfair: Pinker (2018) for example generated considerable debate over the extent to which his choice of metrics showed progress over 250 years. There are also intermediaries who appear to seek to influence with a more overt policy agenda. These may be thought of as examples of the policy entrepreneurs who “promote pet proposals, [and] frame problems in particular ways” in looking at wellbeing (Bache and Reardon 2016, p. 148). These are invariably ongoing projects, updated by drawing on the latest official statistics and other surveys. They undoubtedly bring official data to wider usage, where they include it, and help drive debate about wellbeing and progress. Examples include: • the annual Legatum Prosperity IndexTM , with the ambition that “leaders around the world use it to help set their agendas for growth and development, and also to enable others to hold them to account” (Legatum Institute 2019, p. 4); • the Social Progress Index (https://www.socialprogress.org/), a “new way to define the success of our societies. It is a comprehensive measure of real quality of life, independent of economic indicators … designed to complement, rather than replace, economic measures such as GDP”;

4

WIDER AUDIENCES FOR NEW MEASURES OF PROGRESS

101

• “How’s Life?” is a statistical report, released by the OECD every two years or so (e.g. the fifth edition OECD 2020). It seeks to chart “whether life is getting better for people” in 41 countries and is based on a multi-dimensional framework covering 11 dimensions of current wellbeing and four different resources for future wellbeing. The reports are complemented by online material including the interactive Better Life Index (http://www.oecdbetterlifeindex. org/); • the City Health Dashboard evaluates 36 key measures of health for 500 US cities. The purpose of this dashboard, built and maintained by academics with practitioner and foundation support, is “to be a tool for those on the ground to make better informed decisions about the things that most affect their local population”. The dashboard itself has then been mediated for patients, doctors, and healthcare specialists by other organisations, for example by Chicago Health (Mueller 2018); • RefinitivTM is one commercial agency assessing the performance of companies for their clients, in this case using over 400 metrics covering environmental, social, and governance dimensions (https://www.refinitiv.com/en/financial-data/company-data/ esg-research-data). All of these examples define wellbeing and progress in their own terms, albeit drawing on earlier research and engaging with stakeholders. This contrasts with the approach of some national statistical offices, including the UK Office for National Statistics in its What Matters to You? exercise, to engage with the public to define national wellbeing. All of this is radically different from the UN’s 2030 agenda discussed earlier, in which sustainable development goals and supporting checkpoints were agreed through a political process before metrics were put in place. This still requires bringing the goals and the indicators to as wide an audience as possible, for which one key intermediary is the UN’s own Project everyone (https://www.globalgoals.org/project-everyone) supported by marketing and communication initiatives. However, in terms of moving the focus in policy, business, and everyday life onto wellbeing, the approach generally is to change the metrics before changing things for real (e.g. Bache and Reardon 2016, pp. 67–70). The established sequence seems to be: design a new measure; seek to gain consensus and acceptance of the measure; then work to solve

102

P. ALLIN AND D. J. HAND

the social problem that was initially identified as requiring action (e.g. see Barnard 2018, on poverty in the UK).

4.5

Trust in Official Statistics

It is far from easy to build an understanding of how or why official statistics are used, either as products in their own right or as one of the inputs to more complex decision-making. One difficulty is that the term “official statistics” is not always used when official statistics are being referred to. For example, Jess Phillips (2019, e-book location 1444), a member of the UK Parliament, gave just one mention to using “statistics” when “trying to tell the story so that people can see it and easily understand it” while speaking truth to power. However, she also refers to census data, which are official statistics, several times (e.g. location 702) and also recalls that much of her work as an equalities campaigner involved identifying the data that businesses and government organisations “are not collecting by finding the gaps. Very often injustices are hidden in things that weren’t recorded properly” (location 732). We may be reading too much into examples like this, but they do perhaps point at least to some acceptance that official statistics are to be trusted and used. Do official statistics have a competitive advantage in the contemporary information space? As O’Neill (2002, p. 68) notes, “the very technologies that spread information so easily and efficiently are every bit as good at spreading misinformation and disinformation”. The UNECE (2018, p. 2) takes what might be called the producer view: the advantage is that “Official statistics are produced in a professionally independent way based on scientific methods, rigorous quality criteria, including relevance, and the Fundamental Principles of Official Statistics. Upholding these principles is essential to any country seeking to understand itself and respect the rights of its people”. Here relevance and utility are melded into the assessment of the quality of the statistics, as would be done in thinking about the fitness for purpose of any product. In practice, as we discussed in Chapter 2, this often leads to the “field of dreams” approach, of building statistics and believing users will find them and trust them. However, if we come at the question of competitive advantage in another way, we find more that official statisticians can do to hone their competitive advantage and extend their outreach. The mandate for official statistics, as set out in the UN fundamental principles, can be considered delivered only if the statistics hold up against three tests (OSR 2019, p. 2):

4

WIDER AUDIENCES FOR NEW MEASURES OF PROGRESS

103

• Trustworthiness: Confidence in the people and organisations producing statistics; • Quality: Data and methods that produce assured statistics; and • Value: Statistics that support society’s needs for information. O’Neill (2002, p. 64) explains why trustworthiness is important: “Reasonably placed trust [i.e. of users] requires not only information about the proposals or undertakings that others put forward, but also information about those who put them forward”. There were a number of national surveys on public trust in official statistics, from which the OECD developed a model questionnaire for measuring trust in official statistics. The UK Statistics Authority commissioned an independent social research organisation to undertake surveys in, to date, 2014, 2016, and 2018, and there was a survey with a somewhat different set of questions in 2009. Morgan and Cant (2019, p. 13) report that “Trust in ONS as an institution remains high – 88% of those who expressed an opinion either trusted it a great deal or tended to trust it. However, nearly a quarter (24%) did not express any opinion – stating that they did not know whether or not they trusted ONS”. Taking these two proportions together suggest that two thirds (67%) of the British public would say that they trust the ONS. The surveys on public trust also measure awareness and self-reported use of official statistics. For example, “Public awareness of ONS remains relatively high, at 69% - a similar level to that seen in 2016 and 2014” and “Just under a quarter of the British public (24%) report that they have used statistics produced by ONS”. Usage breaks down as 4% of the public are frequent users, 14% are occasional, and 5% last used the statistics more than 5 years previously (Morgan and Cant 2019, pp. 9–11). In a Eurobarometer survey in 2015, undertaken on behalf of the European Commission, citizens in all the then 28 European Union countries were asked about their knowledge and understanding of economic statistics and whether they trust them. The survey was framed around three key economic indicators, the growth rate, the inflation rate, and the unemployment rate. (We earlier mentioned these three areas as likely to have high media profile). As with other exercises to test knowledge about facts about society, respondents were generally not very accurate in the numbers they volunteered for these particular statistics (TNS 2015, pp. 4– 13). The headline result for trust was that half of Europeans do not trust these official statistics, with the proportions tending to trust them varying

104

P. ALLIN AND D. J. HAND

between countries, from 73% in Sweden down to 27% in Spain, and 44% in the UK (pp. 19–20). Surveys of knowledge, use, and trust of official statistics can be mined for many insights about the relationship between official statistics and the public. We leave that for others and draw attention to just two points. First, only a small proportion of the public describe themselves as regular users. Second, trust levels vary greatly, between countries, between people of different ages, and across socio-professional categories. There seems to be more for statisticians to do to raise the profile and the trustworthiness of official statistics.

4.6 Conclusion: Better Engagement Between Producers and Users of Official Statistics Official statistics should be put to work more. Although the concept of the public value of official statistics is still under development (and actual valuations are rare), it is clear that there are many issues to address in the role of official statistics beyond public policy, in public debate, and in business. Many people appear unaware of official statistics in general, or at least of how to access the available statistics. We accept that there may be a degree of cynicism, reflected in the old line that statistics should be ranked third, after lies and damned lies. Statistical literacy and confidence with handling statistics are being addressed, if only slowly, and the misuse and abuse of statistics by the media and by politicians can be tackled. But if we are to improve wellbeing and meet the sustainable development goals, we cannot leave it all up to governments and policy-makers. As George Monbiot (2018) forcibly puts it, “The task of all citizens is to understand what we are seeing. The world as portrayed is not the world as it is. The personification of complex issues confuses and misdirects us, ensuring that we struggle to comprehend and respond to our predicaments. This, it seems, is often the point”. There is no grand theory of change with official statistics embedded in the process: we are simply reflecting the approach adopted in the fundamental principles of official statistics. There is considerable potential for statistics to give a wider picture of wellbeing and sustainable development, to be used as a common resource by policy-makers, civil society, and businesses. We propose that better engagement between the producers of official statistics and users (and potential users) should be seen as the driving

4

WIDER AUDIENCES FOR NEW MEASURES OF PROGRESS

105

force to deliver the potential of official statistics. As we reported above, current engagement on UK statistics is at best patchy. We are not in doubt that producers work hard at making their statistical releases as clear and unambiguous as possible when they publish them, and that the statistics are soundly based. But we suspect that, in doing so, producers are demonstrating “what linguists call ‘transmitter orientation’ – that is, it is considered the responsibility of the speaker to communicate ideas clearly and unambiguously” (Gladwell 2009, p. 216). Users, at least those tunedinto receive the statistics, will also tend to expect this. However, it seems as if statistical producers can then sometimes flip into receiver orientation and leave it up to the user “to make sense of what is being said”, rather than treating engagement with users as a dialogue. Moreover, this is a model of broadcasting to those who are aware, with no outreach to those who are not currently using the statistics. Doing better may involve fairly straightforward actions, such as maintaining a calendar of past and planned statistics releases, and promoting the release calendar. It seems a little complacent to plan the orderly release of official statistics and claim transparency because there is a release calendar, but leave people to find it and not, as a matter of course, draw it to the attention of people and organisations who should find it useful. There is little evidence that official statisticians use approaches like the marketing funnel (e.g. Brooks 2019), defined (on Wikipedia) as “a consumer-focused marketing model which illustrates the theoretical customer journey towards the purchase of a product or service”. Although official statistics are invariably free at the point of delivery, this should not preclude the use of models like this to help raise their profile and increase their use. All of this of course applies to all official statistics. In Chapter 6 we will discuss how to increase the public value of official statistics, with a special eye to two sets of statistics relating to wellbeing. One set comprises the sustainable development indicators that we discussed in Chapter 2 and that each country is committed to compiling, so that progress towards the global, sustainable development goals for the year 2030 is measured. The second set of indicators, found in an increasing number of countries, comprises its measures of national wellbeing. This set also allows for measuring progress by taking account of more than GDP, which meets another commitment captured in the UN’s 2030 Agenda. Before then, we need in Chapter 5 to fill in a couple of stages in the process of delivering official statistics. These are the need to use new forms

106

P. ALLIN AND D. J. HAND

and sources of data, with associated challenges as well as benefits, and the role of the media in disseminating official statistics.

References Ahsan, S., & Tweed, D. (2015). The Atlas of Us. Canada: National Post, June 27, WP 6–7. Allin, P., & Hand, D. J. (2014). The Wellbeing of Nations: Meaning, Motive and Measurement. Chichester: Wiley. Bache, I. (2019). Evidence, Policy and Wellbeing. Cham, Switzerland: Palgrave Macmillan. Bache, I., & Reardon, L. (2016). The Politics and Policy of Wellbeing. Cheltenham: Edward Elgar. Baker, P. C. (2019). Collision Course: Why Are Cars Killing More and More Pedestrians? https://www.theguardian.com/technology/2019/oct/03/collisioncourse-pedestrian-deaths-rising-driverless-cars. Accessed 10 December 2019. Bakker, C. (2014). Valuing the Census. http://archive.stats.govt.nz/methods/ research-papers/topss/valuing-census.aspx. Accessed 4 December 2019. Barnard, H. (2018). We Have a New Way to Measure Poverty––Now Let’s Act to Solve it. https://www.jrf.org.uk/blog/we-have-new-way-measure-povertynow-act-solve-it. Accessed 11 December 2019. Bartlett, J., & Tkacz, N. (2017). Governance by Dashboard: A Policy Paper. London: Demos. https://www.demos.co.uk/wp-content/uploads/ 2017/04/Demos-Governance-by-Dashboard.pdf. Accessed 14 December 2019. Briscoe, S. (2005). Britain in Numbers: The Essential Statistics. London: Politico’s Publishing. Brooks, A. (2019). Marketing Funnel Strategies: 5 Steps to Increase Sales in 2019. https://www.ventureharbour.com/5-strategies-to-build-a-mar keting-funnel-that-converts/. Accessed 15 December 2019. Criado Perez, C. (2019). Invisible Women: Data Bias in a World Designed for Men. New York: Abrams. Custance, J., & Hillier, H. (1998). Statistical Issues in Developing Indicators of Sustainable Development. Journal of the Royal Statistical Society Series A, 161, 281–290. Duffy, B. (2018). The Perils of Perception: Why We’re Wrong About Nearly Everything. London: Atlantic Books. Equal Opportunities Commission. (1980). Women and Government Statistics. EOC Research Bulletin, 4. Manchester: EOC. Gladwell, M. (2009). Outliers: The Story of Success. London: Penguin. Greer, G. (2020, January 10). The Diary Column. New Statesman, 21.

4

WIDER AUDIENCES FOR NEW MEASURES OF PROGRESS

107

GRI. (n.d.). Business Reporting on the SDGs. https://www.globalreporting.org/ information/SDGs/Pages/Reporting-on-the-SDGs.aspx. Accessed 26 April 2020. Helm, T. (2019). Patient Data from GP Surgeries Sold to US Companies. https://www.theguardian.com/politics/2019/dec/07/nhs-medical-datasales-american-pharma-lack-transparency. Accessed 8 December 2019. Hornik, J., & Cherian, J. (1993). Data-Use in Marketing: A Normative Analysis From an Artificial Science Perspective. Journal of Business Research, 27, 229– 238. Kandogan, E., Roth, M., Kieliszewski, C., Özcan, F., Schloss, B., & Schmidt, M. T. (2013). Data For All: A Systems Approach to Accelerate the Path from Data to Insight. In 2013 IEEE International Congress on Big Data. https:// ieeexplore.ieee.org/document/6597173. Accessed 15 December 2019. Legatum Institute. (2019). The Legatum Prosperity Index TM 2019. https:// www.prosperity.com/. Accessed 4 March 2020. Miller, H. G., & Mork, P. (2013). From Data to Decisions: A Value Chain for Big Data. IT Professional, 15, 57–59. https://ieeexplore.ieee.org/document/ 6449385. Accessed 15 December 2019. Monbiot, G. (2018). Our Cult of Personality is Leaving Real Life in the SHADE. https://www.theguardian.com/commentisfree/2018/oct/03/cult-person ality-politics-boris-trump-corbyn-george-monbiot. Accessed 11 December 2019. Morgan, H., & Cant, J. (2019). Public Confidence in Official Statistics–– 2018. NatCen Social Research, London. http://natcen.ac.uk/our-research/ research/public-confidence-in-official-statistics/. Accessed 4 March 2020. Mueller, L. (2018). 5 Things We Learned About Chicago’s Health From the City Health Dashboard. https://chicagohealthonline.com/5-things-welearned-about-chicagos-health-from-the-city-health-dashboard/. Accessed 4 March 2020. O’Neill, O. (2002). A Question of Trust: The BBC Reith Lectures. Cambridge: Cambridge University Press. OECD. (2020). How’s Life? 2020: Measuring Well-Being. http://www.oecd. org/statistics/how-s-life-23089679.htm. Accessed 28 March 2020. ONS. (2019a). Deaths of Homeless People in England and Wales: 2018. https:// www.ons.gov.uk/peoplepopulationandcommunity/birthsdeathsandmarria ges/deaths/bulletins/deathsofhomelesspeopleinenglandandwales/2018. Accessed 4 December 2019. ONS. (2019b). UK to Lead UN Project Defining the Value of Global Statistics. https://www.ons.gov.uk/news/news/uktoleadunprojectdefiningth evalueofglobalstatistics. Accessed 4 December 2019. Orts, E., & Spigonardo, J. (2020). No Tine to Waste: Achieving the UN’s Sustainability Goals. https://knowledge.wharton.upenn.edu/special-report/ no-time-waste-achieving-uns-sustainability-goals/. Accessed 26 April 2020.

108

P. ALLIN AND D. J. HAND

OSR. (2019). Statistics That Serve the Public Good. OSR’s Vision: What We Do and Why. https://www.statisticsauthority.gov.uk/publication/osr-vision/. Accessed 4 December 2019. PACAC. (2019). Governance of Official Statistics: Redefining the Dual Role of the UK Statistics Authority; and re-evaluating the Statistics and Registration Service Act 2007. Report HC 1820, House of Commons, London. https://publications.parliament.uk/pa/cm201719/cms elect/cmpubadm/1820/1820.pdf. Accessed 4 March 2019. Phillips, J. (2019). Truth to Power: 7 Ways to Call Time on B.S. London: Monoray. Pinker, S. (2018). Enlightenment Now: The Case for Reason, Science, Humanism, and Progress. London: Allen Lane. Prescott-Allen, R. (2001). The Wellbeing of Nations: A Country-by-Country Index of Quality of Life and the Environment. Washington, DC: Island Press. Sæbø, H. V., & Holmberg, A. (2019). Beyond Code of Practice: New Quality Challenges in Official Statistics. Statistical Journal of the IAOS, 35, 171–178. https://content.iospress.com/download/statistical-journal-of-theiaos/sji180463?id=statistical-journal-of-the-iaos%2Fsji180463. Accessed 14 December 2019. Scott, K. (2012). Measuring Wellbeing: Towards sustainability?. Abingdon: Routledge. Smith, C., & Cuckson, D. (2002). Karl Marx, 1845, Theses on Feuerbach. https://www.marxists.org/archive/marx/works/1845/theses/. Accessed 10 December 2019. Stiglitz, J. E., Sen, A., & Fitoussi, J.-P. (2010). Mismeasuring Our Lives: Why GDP Doesn’t Add Up. New York: The New Press. The Economist. (2017). This Year has Seen an Explosion of Rage About Sexual Harassment: Will it Lead to Lasting Change? https://www.economist.com/ international/2017/12/19/this-year-has-seen-an-explosion-of-rage-aboutsexual-harassment. Accessed 8 March 2020. Thunberg, G. (2019). No One is Too Small to Make a Difference. London: Penguin. TNS Opinion & Social. (2015). Europeans and Economic Statistics: Standard Eurobarometer 83. https://ec.europa.eu/commfrontoffice/publicopi nion/archives/eb/eb83/eb83_stat_en.pdf. Accessed 10 December 2019. Tokarczuk, O. (2019). Nobel Lecture: The Tender Narrator. https://www. nobelprize.org/prizes/literature/2018/tokarczuk/104871-lecture-english/. Accessed 14 December 2019. UNCTAD. (2019). Guidance on Core Indicators for Entity Reporting on Contribution Towards Implementation of the Sustainable Development Goals. https:// unctad.org/en/PublicationsLibrary/diae2019d1_en.pdf. Accessed 26 April 2020.

4

WIDER AUDIENCES FOR NEW MEASURES OF PROGRESS

109

UNECE. (2018). Recommendations for Promoting, Measuring and Communicating the Value of Official Statistics. Geneva: United Nations. http://www. unece.org/index.php?id=51139. Accessed 8 March 2020. Vichi, M., Valente Rosa, M. J., & Ruane, F. (2015). The Users of Statistics and Their Role in the European Society. European Statistical Advisory Committee document 2015/1175. https://ec.europa.eu/eurostat/web/european-statis tical-advisory-committee-esac/other-documents. Accessed 8 March 2020. Wellbeing Economy Alliance. (2020). Our Vision. https://wellbeingeconomy. org/wp-content/uploads/2020/01/WEAll-brochure_Jan20_S.pdf. Accessed 14 March 2020. Williams, Z. (2012). The Saturday Interview: Stuart Hall. https://www.the guardian.com/theguardian/2012/feb/11/saturday-interview-stuart-hall. Accessed 30 November 2019. Wolfe-Robinson, M. (2019). Homeless Protesters Storm Council Building in Chester. https://www.theguardian.com/uk-news/2019/oct/02/homelessprotesters-storm-council-building-in-chester. Accessed 30 November 2019.

CHAPTER 5

Inputs and Outputs: Data Science and the Role of Media

Abstract In order to be able to make use of statistics in measuring, promoting, and guiding wellbeing policies, people must know how to access and interpret statistics, especially official statistics. “Interpretation” includes being able to assess their quality and trustworthiness. Data accessibility and use is not uniform around the world, and this impacts UN plans for sustainability. However, new sources and new kinds of data, such as social media data, administrative data, telecomms data, and unstructured data hold considerable promise—though they come with challenges of their own. A basic level of statistical literacy is necessary to be able to assess the reliability of statistics and to interpret them appropriately. Keywords Data · Trustworthiness · Quality · Reliability · Data science

5.1

Introduction

This book examines how effectively politicians, policy-makers, and others understand and use statistics in measuring and promoting sustainable wellbeing. To meet that broad aim we need to ask more nuanced questions, including to examine how measures are used and what measures should be used. Beyond that, lies the question of what can be done to facilitate the use of appropriate measures. © The Author(s) 2021 P. Allin and D. J. Hand, From GDP to Sustainable Wellbeing, Wellbeing in Politics and Policy, https://doi.org/10.1007/978-3-030-53085-3_5

111

112

P. ALLIN AND D. J. HAND

An obvious starting point is that, to use official statistics as evidence to support change, people need to know that the numbers are there and be able to access them. A case in point is the website of the UK’s Office for National Statistics which, some years ago, came under criticism as being difficult to navigate, so that even experts could not find the data and statistics they needed. Responding to the criticism, extensive redesign work has vastly improved it, so that facts can be more readily found to steer policy and other decisions. However, knowing that the numbers are there is but the first step. Numbers can only be expected to guide policy decisions (and to call people to account) if the intrinsic truth in them is acknowledged, and such an acknowledgement should only be expected if the numbers are accurate, valid, timely, and relevant. People might decline to use numbers in the form of official statistics because they deny the validity or relevance of the numbers (arguing that they are incorrect or irrelevant in some way) or, perhaps worse, because they simply ignore the numbers. While lack of correctness or relevance can be remedied, probably in discussion with those who point out the shortcomings, it is more difficult to cope with someone who simply turns a blind eye. (We are reminded of the apparent growth of acceptance of the notion that the Earth is flat, https://thefla tearthsociety.org/home/index.php). The United Nations has long been aware of the need for accurate data and the opportunities provided by the data revolution for supporting sustainable development. UN Global Pulse (Global Pulse 2020) is the UN Secretary General’s initiative on big data and artificial intelligence for development, humanitarian action, and peace. Established in 2010, Global Pulse is a network of laboratories partnering with the private sector and academia to access new data sources and explore how to use them for measuring wellbeing. Global Pulse works in three interconnected areas: discovery, policy, and scale. Discovery explores the application of big data and AI for sustainable development, humanitarian action, and peace. Policy contributes to global efforts to establish trusted frameworks for ethical and privacy-protective data practices and for digital cooperation. Scale provides the UN System and public sector organisations with tools and technical assistance needed for mainstream adoption of data and AI practices. A World That Counts, a report published in November 2014 by the United Nations Independent Expert Advisory Group on a Data Revolution for Sustainable Development (Data Revolution Group 2014,

5

INPUTS AND OUTPUTS: DATA SCIENCE AND THE ROLE OF MEDIA

113

pp. 2–3), says “Never again should it be possible to say ‘we didn’t know’”. This report, made five key recommendations to the UN Secretary General: 1. Develop a global consensus on principles and standards, bringing together public, private, and civil society data providers. 2. Share data technology and innovation for the common good. 3. New resources for capacity development: provide new resources for supporting the data revolution for sustainable development. 4. Leadership for coordination and mobilisation: a global partnership to mobilise and coordinate organisations to make the data revolution serve sustainable development. 5. Exploit some quick wins on SDG data: establishing an SDG data lab. The report commented: “Data needs improving. Despite considerable progress in recent years, whole groups of people are not being counted and important aspects of people’s lives and environmental conditions are still not measured. For people, this can lead to the denial of basic rights, and for the planet, to continued environmental degradation. Too often, existing data remain unused because they are released too late or not at all, not well documented and harmonized, or not available at the level of detail needed for decision-making”. That was 2014, and while progress has been made, much remains undone. The UN website on Big Data for Sustainable Development (UN Big Data 2020), containing useful links to UN reports in this area, comments “Critical data for global, regional and national development policymaking is still lacking. Many governments still do not have access to adequate data on their entire populations. This is particularly true for the poorest and most marginalized, the very people that leaders will need to focus on if they are to achieve zero extreme poverty and zero emissions by 2030, and to ‘leave no one behind’ in the process”. It is clear that there is still a long journey ahead. From within the data science community, one of the exciting changes is that new types of data, discussed below, are arising all the time: new ways of capturing data, data with new kinds of features, entirely new (unstructured) kinds of data, and so on. From outside, however, these exciting new sources come with associated challenges, not least their short-term nature. Classical official statistics often have time series stretching back

114

P. ALLIN AND D. J. HAND

years, or even decades, whereas (for example) social media data might stretch back only a short time (and is vulnerable to changing data capture methods—not least companies going out of business). On the other hand, the different characteristics might be complementary. This has certainly proved to be the case in population studies, where, as we describe below, conventional survey sampling complements modern administrative data, so that the combination yields a powerful synergy. Other organisations (for example, the Commonwealth Partnership for Technology Management [CPTM 2019] and the Asia Global Institute [Spence 2019]) are also exploring how to take advantage of modern data technology to move beyond conventional economic indicators and develop sound evidence-based (i.e. data-based) tools for assessing wellbeing and sustainable progress.

5.2

How Can Data Science Help?

Data science is the discipline of extracting value from data. As you might expect from that potted definition, data science is primarily statistics, with significant aspects of computer science (e.g. for manipulating and organising data) along with features which are domain-specific (e.g. different tools might be emphasised in healthcare than in finance, though there is very substantial overlap). “Value” in the definition might be in terms of enhanced understanding of something or improved ability to make decisions or predictions in some domain. For us in this book, primary interest lies in the power of data science for evaluating and assessing the effectiveness of public policies, especially societal progress where the current reported position is that “well-being has, in some respects, improved … Yet progress has been slow, or has even deteriorated in other areas” (OECD 2020, p. 19). As readers of this book will almost certainly be aware, publicity about the potential of data science has reached new heights in recent years. Couple this with concomitant publicity about associated areas (which overlap to a very great extent with data science, and might legitimately be regarded as subsumed within it), such as machine learning, data mining, and artificial intelligence, and an obvious question is to what extent can these developments assist in monitoring and enhancing progress in sustainable wellbeing. In this section we explore these questions, looking at some of the new kinds of data which have arisen and exploring their properties, and the extent to which they can facilitate our aims.

5

INPUTS AND OUTPUTS: DATA SCIENCE AND THE ROLE OF MEDIA

115

Discussions of modern data science often focus on the sizes of the data sets and the speed with which new data are acquired. Data sets consisting of billions of points are now common, and data often accumulate in a non-stop stream (hence “streaming data”). Examples are data arising from transactions, such as plastic card financial transactions, mobile phone calls, internet accesses, and so on. These examples illustrate a novel feature which we believe is even more important than the size or speed of acquisition of the data sets. This is the fact that so much data acquisition is automatic. Whereas surveys require an effort on behalf of those being surveyed, credit card, phone call, and web search data are an automatic spin-off of the transactions themselves. No extra effort is required to capture the data. This is what underlies the sizes of many modern data sets—the cost of acquisition is minimal. Data generated as an intrinsic and necessary part of a transaction (e.g. the amount spent in a credit card purchase, or the destination address of an email) have been supplemented by data deliberately captured by automatic measuring devices. Examples are wrist fitness monitors and odometers in vehicles. Telemetry in general is a rich new source of data—for example, monitoring the ongoing state of an engine, or the way a vehicle is being driven (which can then lead to reduced insurance premiums). But, again, the immediate operational use of such data (is the engine about to break down? Is someone driving erratically?), while perhaps the initial motivation for the measurement, is then supplemented by storing the data in large databases, where retrospective analysis can be undertaken to great effect (e.g. is there a common pattern to engine failures? What type of person is the safest driver?). In general, assurance must be provided that personal data will be used only for certain purposes—like measuring wellbeing or the effectiveness of policies. Although Article 5, Clause 1(b) of the General Data Protection Regulation says that personal data shall be “collected for specified, explicit and legitimate purposes and not further processed in a manner that is incompatible with those purposes” it then goes on to say “further processing for archiving purposes in the public interest, scientific or historical research purposes or statistical purposes shall … not be considered to be incompatible with the initial purpose”. (Our italics) Likewise, in the UK “The Statistics and Registration Service Act 2007, as amended by the Digital Economy Act 2017, creates a legal framework providing the Authority with access to data held by Crown bodies, other public authorities and private undertakings (including charities) to

116

P. ALLIN AND D. J. HAND

support the Authority’s statistical functions. As amended, the 2007 Act requires these data suppliers to consult the Authority before changes to data collection are made in order to protect the continuity of data supply, as well as the accuracy and reliability of statistics and statistical research derived from these data sources” (Digital Economy Act 2018, para. 2.2). Data held by “private undertakings” is potentially a very useful supplement to data captured by official organisations, provided suitable agreements can be reached. So-called “data trusts” are one way in which this might be done. Precursors to data trusts have long been used in the retail financial services industry, where data are pooled to yield improved models for default prediction, with the models being fed back to the diverse organisations providing the data, and without detailed data being divulged to competitor organisations. The result is that the industry as a whole, along with its customers and shareholders, benefits from the more accurate and reliable predictions. Such strategies are also used in fraud detection, in recognition of the fact that fraud is an industrywide challenge, and that everyone benefits if fraud is minimised (except, presumably, the fraudsters). Structured data are still the most common form of data. “Structured” describes data which come in a clearly specified format—with, for example, each record containing well-defined slots for data entries (which does not mean they are all necessarily filled). Increasingly, however, “unstructured” or “heterogeneous” data are being analysed. These include things like speech signals, images, video clips, free text, and so on. It will be obvious that this has the potential to dramatically increase the sources from which data can be captured, and the perspectives on wellbeing that they imply. Examples which spring to mind are voice assistants such as Alexa and Siri, surveillance cameras, footfall monitoring in retail outlets, as well as web scraping, social media, data from drones and airships, and so on. The tremendous richness of the modern data ecosystem will be apparent from the examples above, so a natural first reaction must be that these innovations will help the sustainable wellbeing and alternative to GDP initiative. However, it is important to be aware that new kinds of data have novel kinds of properties and are not without challenges of their own. Typically, these challenges are different from, and often complementary to, the challenges of more conventional data. To illustrate, we shall compare conventional survey data with modern administrative data.

5

INPUTS AND OUTPUTS: DATA SCIENCE AND THE ROLE OF MEDIA

117

The importance of administrative data is indicated by the efforts made by the UK Office for National Statistics to move towards a census based on administrative data after 2021 (ONS 2020). Administrative data, data initially collected for some, typically operational, purpose, and then stored in a database, appear to have various extremely valuable characteristics. For example (and see Hand 2018a, for a detailed discussion): 1. Minimal additional expense is incurred in collecting them; all that is required is to store them in the database. 2. In many situations we might assume that all the data are available: a credit card company will have records of all the transactions undertaken using its card, and a mobile phone company will have records of all calls made. Such data must be generated if the companies are to function. 3. Survival of an organisation might depend on high quality data: incorrect customer charges will not encourage extra custom, and will lead to mistaken company revenue. 4. In many cases the data are as up-to-the-minute as they can be. As soon as a transaction or call is made, it can be recorded in the database. 5. Administrative data tell us what people actually do, reflecting social reality better than survey data saying what people claim to do. However, closer examination shows that these ideals might not be met. Using the list above: 1. The data generated during an operation might not be in a format suitable for storage and later large-scale analysis, so cleaning and polishing will be required. This will incur a cost. If, further, the data are to be linked to other data from different sources before value can be realised, costs might be incurred in obtaining the additional data. 2. Even if all of an organisation’s transactions are stored in a database, the aim might be to make a statement about a wider population. For example, in the credit card example, we might want to make a statement about all retail purchasers, not merely those who use plastic cards or one particular company’s plastic card, and while tax

118

P. ALLIN AND D. J. HAND

records show the details of those who pay, or might pay, tax, they do not reveal details of the black economy. 3. While sampling variability, which is an intrinsic characteristic of survey data, will not apply, administrative data will have other sources of error. Often these induce a systematic bias, which is more difficult to cope with than random variability. An additional complication is that the underlying definitions will (necessarily) be those required for operational purposes (e.g. to run a company, tax office, hospital, school, etc.) which may not match those required for the purposes of building a wellbeing indicator. This can induce distortion. 4. While the organisation collecting the data might have instant access to them, other organisations, such as a government department measuring wellbeing, might have to wait. Unless there are wellestablished and smoothly running procedures for transferring data, the wait can be a long one. 5. Discovering what people do rather than what they say they do is all very well, but sometimes it is the latter that is needed. This can clearly apply in a wellbeing context, where subjective feelings are relevant. This example of administrative data serves to illustrate that new data sources (often generally referred to as “alternative data” in some contexts), often have properties complementary to existing sources. That means that using two sources together can yield a very useful synergy. A simple example is the combined use of administrative data, collected on everyone in a population, and survey data, collected on just a small subsample, to infer characteristics of small segments of the population which contain only a relatively few survey cases: tools like regression estimation and Bayesian methods can “steal power” from neighbouring or related segments. Likewise, estimates from entirely different data sources can be used to triangulate and check values: if very different estimates are obtained it suggests that something is wrong somewhere—perhaps in data collection (selection biases creeping in?), or in recording (inaccuracies?), or in calculations (coding errors?), and so on. A now classic example of the power of alternative data source was the “Billion Prices Project” (Cavallo 2013; Cavallo and Rigobon 2016). This collected the online prices of goods displayed on the web and used them to construct a daily price index—an inflation index—for Brazil, Chile,

5

INPUTS AND OUTPUTS: DATA SCIENCE AND THE ROLE OF MEDIA

119

Colombia, Venezuela, and Argentina. The effort involved in collecting such data is trivial compared with the effort involved in traditional inflation indexes data collection methods. The well-known result was that while the online price indexes for the first four countries roughly followed the official estimates, that for Argentina was nearly three times the official estimate: the official estimate was misrepresenting the true state of the economy. Since then, official statistics offices around the world have begun to use data from other sources to estimate inflation and other economic and social indicators (see, for example, Data Science Campus 2019). Something else which will be apparent from the outline of alternative data sources above is that often data acquisition is more than a merely passive exercise. Often data will be collected dynamically, with its characteristics changing over time or in response to data which have already been collected. At the least this allows effort to focus where it is most needed. Ricciato et al. (2019) describe this sort of change in detail, describing how the term “trusted smart statistics” has been coined to describe the “shift of focus from data sources to data systems and a change of perspective about innovation in official statistics from incremental augmentation towards a systemic paradigm change. The concept of data system is meant to signify an augmentation of the capabilities and role of data source beyond the mere generation of raw input data” (their italics). The concept of trusted smart statistics has been further examined in Vichi and Hand (2019). Complications can arise when international comparisons are concerned, since different countries have different approaches to different kinds of data, different attitudes to privacy and confidentiality, and different data capture capabilities. We also commented above about challenges arising from the short-term nature of many alternative data streams. Worse still, although a data stream might superficially appear to be internally consistent over an extended period, often this is an illusion. Whereas traditional data capture methods work to strict definitions and collection strategies, many modern data streams, especially those collected as a side-effect of some other activity (like administrative data) are subject to relative arbitrary definitional changes. Consider web scraping, for example. As discussed in Hand (2020, p. 301), Google’s search engine undergoes regular updates to make it more effective. According to Moz (2020, p. 1), “Each year, Google makes hundreds of changes to search. In 2018, they reported an incredible 3234 updates— an average of almost 9 per day, and more than 8 times the number

120

P. ALLIN AND D. J. HAND

of updates in 2009. While most of these changes are minor, Google occasionally rolls out a major algorithmic update (such as Panda and Penguin) that affects search results in significant ways”. The challenge of constructing a consistent time series using data collected from such changing methods will be apparent, especially given that the details of what changes have been made may not be known. In summary, exciting new ways of collecting data are leading to new kinds of data with greater granularity, comprehensiveness, timeliness, relevance, and bulk, with the potential to benefit the sustainable wellbeing initiative. However, these merits come with caveats: new types of data have new kinds of problems. It is still necessary to keep alert for distortions and misrepresentations in the data. Worse, these distortions might be of a kind we are relatively unfamiliar with. Decades of statistical theory means we can comfortably handle sampling variability, but systematic biases induced by hitherto unsuspected shortcomings in the way the data are collected are not so easy to spot. The context is that traditional official statistics have a long history of rigorous development: usually they can be relied upon. However, there have been dramatic examples where this has gone wrong. We mentioned the case of measuring Argentina’s inflation above. Daniel and Lanata Briones have examined higher-level debate about Argentina’s consumer price index, the INDEC CPI, as played out over 2007–2015, concluding that “The strong criticism of the official index derived from the fact that a matter regarded as purely technical appeared polluted by political interests. This ‘contamination’ undermined the INDEC CPI’s attributed objectivity. The tensions around this index were also translated into the public sphere as a moral problem related to the lack of honesty and transparency from those in charge of elaborating the index” (Daniel and Lanata Briones 2019, p. 136). They also remark that “Inflation can be thought, expressed, defined and quantified in multiple ways. Differences between these forms are not simple, technical details, but have a historical, political and sociological significance. In Argentina, the battle over the CPI was a missed opportunity to discuss the techniques of measuring inflation, as well as its purpose, beyond the walls of a technical agency. Unfortunately, it was also a missed opportunity to deliberate openly about the ways of conceiving, understanding and managing inflation in a country that still suffers from this problem” (p. 145).

5

INPUTS AND OUTPUTS: DATA SCIENCE AND THE ROLE OF MEDIA

121

The opportunities missed in Argentina are especially pertinent to our study here of new measures of sustainable wellbeing. Without full attention being given to the key characteristics of official statistics—that is to trustworthiness, value, and quality, as described in the UK Statistics Authority’s Code of Practice (UKSA 2018)—there is an increased likelihood that the world will continue to suffer as SDGs set for 2030 will fail to be achieved.

5.3

The Role of the Media

The unique selling point of national statistical institutes (NSIs) is (or at least should be) the quality, reliability, and trustworthiness of their output, created using the best available raw data. However, this output is of little value if it cannot be accessed by those who need it—by governments, commercial organisations, and citizens. In particular for us, of course, measures of wellbeing and sustainability are no use if people cannot see them or if they cannot understand them. Communication is a key step in whether or not such numbers are useful. In the past, this communication was achieved by (paper) publications from the NSIs, which were then picked up and propagated by various news media such as newspapers, radio, and television. Nowadays, however, we have in addition the web, and all the different channels which ride on that. And therein lies a risk, because not all these channels are equally trustworthy. Indeed, they sometimes serve as a breeding ground for conspiracy theories and misinformation epidemics. That last phrase seems to have been recently coined in the context of anxiety about the coronavirus pandemic, COVID-19. Bizarre, misleading, and potentially harmful stories about the source of the virus and how to protect oneself have spread, including crackpot notions that the virus was a deliberately engineered bioweapon, that one should avoid spicy food, that consuming large amounts of vitamin C is protective, that blowing a hairdryer up one’s nose will kill the virus, that it is caused by 5G telephone masts, and that one should avoid cold food and drinks such as ice cream (see, for example, BBC 2020). There are many other examples of misinformation spread across the web, including false claims about the dangers of vaccinations, conspiracy theories surrounding the 9/11 terrorist atrocity, and misleading information during election campaigns. It goes without saying that there are

122

P. ALLIN AND D. J. HAND

many potential adverse consequences from such misinformation, not least on policy-makers, who might be misled into adopting harmful policies or misunderstanding the real consequences of policies. The fundamental problem is that anyone can publish statistics, on the web or elsewhere. The web, certainly, has no overall quality control or veracity control. This leads us to the question of how to decide who to trust. It is encouraging that, as we reported in Chapter 4, national statistical institutes (NSIs) can gain broad trust. However, an earlier, cross-Europe study showed large variability in the trust in specific economic indicators between countries. Given the effort made to ensure that the statistics produced by an NSI are trustworthy, valid, and accurate, this trust seems well placed. It also suggests that there would be fewer misunderstandings and misrepresentations if more people were to go directly to their NSI for the information that they need. However, as already mentioned, the truth is that they typically go to other sources, which may involve partially digested statistics (even if originally from the NSI), or even deliberately distorted material. Some outlets—some newspapers, television channels, and blogs—are well-known for strategic choice of material to promote a position, rather than attempting to give a properly balanced perspective. Others have the same effect accidentally. Hand (2020, pp. 217–219) discusses these distortions in detail. A step towards a solution to the problem might be to require that assertions are accompanied by a source (Hand, 2018b). This is a standard practice in scientific publications, but there is no reason why it should not be more widely adopted, and it is simplicity itself on the web. Providing a source citation means that people can follow it up to check things if they want to, not that they must. The possibility that an assertion might be checked should serve as a moderator on the more absurd claims. Of course, such a practice alone is not a complete solution—that is probably impossible—but it will help. There are also higher-level issues. Numbers produced by an NSI, even with the best of intentions, might be in error. They might be subtly misleading because they use a definition which is slightly different from the one you are using. There can be no guarantee that they are right, but we can expect them to be produced by a trustworthy procedure, so that we might legitimately have confidence in them. We have said little about the deliberate promulgation of false ideas for nefarious purposes—fraud, as one might call it. Such things do sometimes

5

INPUTS AND OUTPUTS: DATA SCIENCE AND THE ROLE OF MEDIA

123

happen. The case of the official Argentinian price index introduced above would appear to have elements of this, and other cases are described in Oreskes and Conway (2010).

5.4

Statistical Literacy

The question of trust in statistical conclusions, and how this arises from the trustworthiness of the raw input data (whether it is subject to sampling distortions, for example) and the trustworthiness of the statistical tools used to analyse the data (whether they are appropriate to answer the question of interest, for example) is the central one. It has become the focus of considerable interest within the statistical community in the context of the so-called “reproducibility crisis” in science mentioned in Chapter 1. Of course, as far as science is concerned, we must remember that science is necessarily contingent: a scientific assertion may be falsified by further data which contravene it. That is the fundamental nature of science. Lack of statistical literacy is one of the primary enablers of the misunderstandings consequent on misinformation. An ability to critically assess numbers would reduce the prevalence of such mistakes. However, it is absurdly unrealistic to expect everyone to acquire a sufficient level of statistical expertise to be able to do this. Moreover, that assumes a rationality that is not supported by our increasing understanding of how human cognitive processes operate, particularly when seeking and responding to evidence. Endorsing the seminal work of Kahneman (2011) in this field, Wolf (2019, p. 13) has pointed out situations in which “We do not notice that a number simply cannot be right. And we certainly do not notice (or care) that we are getting only part of the story”. The best that can be hoped for is to raise awareness of the issues, to encourage people to undertake sense checks and reality checks, and to ask about the provenance of the numbers and the statistics they are presented with, as in the call in Hand (2018b) described above. The case for enhanced education in numeracy is clear. This should explicitly cover official statistics. As Nagaraja (2019, p. 6) has argued, “the public need a greater understanding of how these [official] statistics are put together, what they count, and why”. An abundance of measures, nominally of the same thing, but in fact using subtly different definitions and underlying ways of being calculated,

124

P. ALLIN AND D. J. HAND

can also cause confusion—and can lead to disagreements as different protagonists adopt different statistics. Tackling this sort of issue inevitably requires digging down into complex technical matters. Again, since this is often beyond the expertise of those engaged in the discussion, it poses a real challenge. (The issue is not one unique to measuring wellbeing. The 2008 financial crash was in large part due to a lack of understanding of the limitations of the inevitably complex statistical models underlying risk estimation). The subtitle of this book is “changing statistics or changing lives?” Statistics are indicators of current states, and change in their values indicates progress, or regress. But, of course, this assumes that the right statistics are being used. There are clear analogies elsewhere. Measurements of height would be of little use in a study of weight-loss diets. Self-reported measurements of calories consumed would be a little better, but would still miss the essential idea. (And there is evidence that that particular measure lacks quality, and is unreliable, [Harper and Hallsworth 2016]). Measurements of BMI or fat fold thickness would be nearer to the point. In the same way, and as we have repeatedly emphasised, if one is interested in sustainable wellbeing, while measures of GDP tap into part of what is needed, they gloss over other key parts and so have the potential to be seriously misleading. In short, lives will only be changed if those who should make use of the statistics do in fact make use of them, which requires the conditions discussed above to be satisfied: the numbers are accepted as representing the state of society and people are aware that the numbers exist.

References BBC. (2020). https://www.bbc.co.uk/news/blogs-trending-51271037. Accessed 12 March 2020. Cavallo, A. (2013). Online and Official Price Indexes: Measuring Argentina’s Inflation. Journal of Monetary Economics, 60, 152–165. Cavallo, A., & Rigobon, R. (2016). The Billion Prices Project: Using Online Prices for Measurement and Research. Journal of Economic Perspectives, 30, 151–178. CPTM. (2019). http://cptm.org/documents/Draft%20Outline%20Agenda.pdf. Accessed 11 March 2020. Daniel, C. J., & Lanata Briones, C. T. (2019). Battles Over Numbers: The Case of the Argentine Consumer Price Index (2007–2015). Economy and Society, 48(1), 127–151. https://doi.org/10.1080/03085147.2019.1579438.

5

INPUTS AND OUTPUTS: DATA SCIENCE AND THE ROLE OF MEDIA

125

Data Revolution Group. (2014). A World That Counts. https://www.undata revolution.org/wp-content/uploads/2014/11/A-World-That-Counts.pdf. Accessed 12 March 2020. Data Science Campus. (2019). https://datasciencecampus.ons.gov.uk/faster-ind icators-of-uk-economic-activity/. Accessed 9 April 2020. Digital Economy Act. (2018). https://www.gov.uk/government/consultat ions/digital-economy-act-part-5-data-sharing-codes-and-regulations/statis tics-statement-of-principles-and-code-of-practice-on-changes-to-data-systems. Accessed 9 April 2020. Global Pulse. (2020). https://www.unglobalpulse.org/what-we-do/. Accessed 11 March 2020. Hand, D. J. (2018a). Statistical Challenges of Administrative and Transaction Data. Journal of the Royal Statistical Society Series A, 181, 555–605. Hand, D. J. (2018b, August). Who Told You That?: Data Provenance, False Facts, and Separating the Liars From the Truth-Tellers. Significance, 8–9. Hand, D. J. (2020). Dark Data: Why What You Don’t Know Matters. Princeton: Princeton University Press. Harper, H., & Hallsworth, M. (2016). Counting Calories: How Under-Reporting Can Explain the Apparent Fall in Calorie Intake. https://www.bi.team/wpcontent/uploads/2016/08/16-07-12-Counting-Calories-Final.pdf. Accessed 12 March 2020. Kahneman, D. (2011). Thinking, Fast and Slow. London: Allen Lane. Moz. (2020). https://moz.com/google-algorithm-change. Accessed 12 March 2020. Nagaraja, C. H. (2019, December). Measuring society. Significance, 6–7. OECD. (2020). How’s Life? 2020: Measuring Well-Being. OECD Publishing, Paris. https://doi.org/10.1787/9870c393-en Accessed 28 March 2020. ONS. (2020). https://www.ons.gov.uk/census/censustransformationprog ramme/administrativedatacensusproject. Accessed 12 March 2020. Oreskes, N., & Conway, E. M. (2010). Merchants of Doubt: How a Handful of Scientists Obscured the Truth on Issues from Tobacco Smoke to Global Warming. New York: Bloomsbury Press. Ricciato, F., Wirthmann, A., Giannakouris, K., Fernando, R., & Skaliotis, M. (2019). Trusted Smart Statistics: Motivations and Principles. Statistical Journal of the IAOS, 35, 589–603. Spence, M. (2019). The “Digital Revolution” of Wellbeing. https://www.pro ject-syndicate.org/commentary/digital-revolution-impact-on-wellbeing-bymichael-spence-2019-06. Accessed 12 March 2020. UKSA. (2018). Code of Practice. https://www.statisticsauthority.gov.uk/wpcontent/uploads/2018/02/Code-of-Practice-for-Statistics.pdf. Accessed 17 February 2020.

126

P. ALLIN AND D. J. HAND

UN Big Data. (2020). https://www.un.org/en/sections/issues-depth/big-datasustainable-development/index.html. Accessed 12 March 2020. Vichi, M., & Hand, D. J. (2019). Trusted Smart Statistics: The Challenge of Extracting Usable Aggregate Information From New Data Sources. Statistical Journal of the IAOS, 35, 605–613. Wolf, A. (2019). Science and Statistical Understanding in the Media. In A. M. Herzberg (Ed.), Statistics, Science and Public Policy XXI, Environment, Education and the Global Economy (pp. 9–14). Kingston: Queen’s University.

CHAPTER 6

Conclusion

Abstract In this final chapter we review how the producers of official statistics are responding to calls to go beyond GDP, and how statistics are intended to support the attainment of the Sustainable Development Goals. We have concentrated on the actual and potential uses of official statistics, taking a teleological approach in our assessment. Usage and users form one of three pillars underpinning official statistics, along with methodological quality and trustworthiness. We reach five broad conclusions and we propose that the kinds of statistics covered by the Fundamental Principles of Official Statistics are considered as public statistics, to be produced not only by the national statistical offices but also by other organisations which, crucially, share and can demonstrate commitment to the principles. We finish with seven specific recommendations, including that national statistical offices need a better understanding of how public statistics are used. Keywords Public statistics · User engagement

© The Author(s) 2021 P. Allin and D. J. Hand, From GDP to Sustainable Wellbeing, Wellbeing in Politics and Policy, https://doi.org/10.1007/978-3-030-53085-3_6

127

128

P. ALLIN AND D. J. HAND

6.1

Overview

We were prompted to write this book by the increase in the number of calls to go beyond GDP (gross domestic product) in driving, assessing, and evaluating the progress of nations. These concerns were amplified by the financial crisis of 2008–2009 and subsequently by increasing recognition of environmental threats to us all. The 2015 commitment of every member country in the UN to transforming our world, through a 15-year programme for sustainable development, led us to our main title—From GDP to Sustainable Wellbeing. The agenda includes goals for wellbeing and for sustainable economic growth, and there is an explicit commitment to developing broader measures of progress to complement GDP. GDP and the sustainable development indicators are official statistics (along with perhaps more familiar statistics such as consumer price indices and the unemployment figures). This triggered the question in our subtitle: Changing statistics or changing lives? Put that another way, are the new statistics just to be used to track how governments are doing in reaching the sustainable development goals, with any possible consequent changes in how we are living our lives, or are beyond GDP statistics also intended to help businesses, civil society, and the public work out how to change their lives? And in short, our conclusion is that we feel that wellbeing and sustainable development measures are being undervalued, with a concentration of use on recording progress, rather than helping change things, especially now we are experiencing the COVID-19 pandemic and are beginning to imagine new norms for economies, societies, and the planet. Deconstructing the full title gave four points of departure for our essay, which we set out in Chapter 1. First is the notion of social progress. This is a complex phenomenon with multiple aspects and probably best defined through measures of the things that matter, such as better education, improved health, poverty reduction, increased wealth, and safeguarding the natural environment. The second building block comprises the systems of official statistics that are charged with serving governments, the economy, and the public with data about the economic, demographic, social, and environmental situation. We observe that national statistical offices (NSOs) are far from the only compilers and publishers of data in support of sustainable development, peace, and security. The expectation remains that NSOs are an indispensable element in the information system

6

CONCLUSION

129

of a democratic society, dispensing high quality, useful and trusted information, as the UN’s fundamental principles of official statistics require (United Nations 2014). The third departure point was to note the concept of GDP, with all its strengths and limitations, and that it is an official statistic. National and international statistics offices are the joint keepers of the concept of GDP and are responsible for measuring it. In our brief exploration of the beyond GDP agenda, the fourth of our starting points, we found that this embraces the definition and measurement of current wellbeing together with an assessment of the sustainability of current activities for the wellbeing of future generations. In subsequent chapters, we examined how these four strands are working in the context of measuring progress, wellbeing, and sustainable development in broader ways, not just in terms of GDP. We often came up with the same, largely-unanswered questions concerning precisely how new measures and indicators are intended to be used. Instead, there are mainly general statements about using indicators to assist in the work of implementing the sustainable development goals, or to view policy through a wellbeing lens. It is fine to assert that quality, accessible, timely, and reliable disaggregated data will be needed to help with the measurement of progress and to ensure that no one is left behind. Such data may well be a key to decision-making, but evidence specifically on how beyond GDP indicators are being used is elusive. This seems to follow a well-established pattern. Frederiksen and Gudmundsson (2013, p. 1) report that “Indicators conceived and developed to monitor the societal change in various sectors have been produced in huge amounts during the recent decades, including for environmental and sustainability policies” and that “The discourse on indicators mostly expresses an expectation that indicators are instrumental to policy decisions – that they are conceived, developed, produced, reported or otherwise handed over, and ultimately serving as evidence for decision-making”. However, “decades of research on the role of evidence for decision-making have argued that this direct, instrumental use of evidence for policy is seldom taking place, whereas evidence may inform and influence policy processes in several other ways, playing multiple roles at different stages of policy-making”. This disjoint, between aspiration and practice in the use of indicators, was again found in the EU-funded Policy Influence of Indicators (POINT) research project. For example,

130

P. ALLIN AND D. J. HAND

that “In most of the studies, the civil servants are main users, while politicians and the public are intended but less likely users” (Frederiksen et al. n.d., p. 6). This and other findings are amply illustrated in the set of concluding papers from that project (Frederiksen 2013). Where requirements for indicators are apparent, they can, not surprisingly, be varied. There is still demand for GDP to be measured, albeit with improvements to reflect the modern economy. UKPLC, for example, is campaigning that “Business, political, charity and not for profit decision makers need more and better data in a world that is becoming less reliable, more technologically fast paced and policy volatile … Primarily, we attribute the source of error in current methods to be a failure to record the economic contribution of the so-called gig economy, which is predominantly internet driven/enabled and thus able to develop faster than start-up businesses were ever able to in the past” (https://www. ukplc.uk.com/). While this campaign recognises that developments to UK economic statistics are under way, it is frustrated with the pace of improvements (Dent 2020). Requirements like this sit legitimately alongside the demand for wider measures of progress. The range of official statistics and methodologies used to compile them are evolving to meet new needs and to exploit new data sources. Preparing Chapters 1 and 2 reminded us of the strengths of the official statistics system, with its many developments in data collection techniques, analytical methods and professional codes and standards. Some suggestions for readers wanting further details of methodological developments are given in Table 6.1. In Chapter 5 we drew attention to what Deborah Ashby, the current President of the Royal Statistical Society, recently described as an explosion in computational power that has led, in turn, to an explosion in data. There is no shortage of data by which to address societal problems through the collection and classification of facts, as those who gathered in the 1830s and formed the Royal Statistical Society envisaged.

6.2

Five Broad Conclusions

Despite all the innovation in official statistics and in new sources of data, we conclude that the vision of “A world with data at the heart of understanding society and decision-making” (www.rss.org.uk/manifesto) is not yet fully realised. There is more to be done to maximise the public value of official statistics, especially in ensuring sustainable wellbeing. We draw

6

CONCLUSION

131

Table 6.1 Suggestions for readers wanting further details of methodological developments National statistical offices’ websites Journals, including: Journal of Official Statistics

The Review of Income and Wealth Social Indicators Research Ecological Indicators UN Sustainable Development Goals Knowledge Platform UK data for Sustainable Development Goal indicators—website based on the open source Open SDG platform PARIS21 (Partnership in Statistics for Development in the twenty-first century) especially promoting and supporting National Strategies for the Development of Statistics (NSDS)

https://unstats.un.org/home/nso_sites/ https://www.scb.se/en/documentation/ statististical-methods/journal-of-official-sta tistics-jos/ https://onlinelibrary.wiley.com/journal/ 14754991 https://www.springer.com/journal/ 11205 https://www.journals.elsevier.com/ecolog ical-indicators https://sustainabledevelopment.un.org/? menu=1300 https://sustainabledevelopment-uk.git hub.io/ https://open-sdg.readthedocs.io/en/lat est/ https://paris21.org/national-strategy-dev elopment-statistics-nsds

five broad conclusions from what we have explored, considered, and summarised in this book. First, technical and methodological standards are crucial in delivering official statistics. They are embedded in official statistics and supported by protocols, procedures, skills, and capability, whether in the ongoing delivery of statistical outputs or in the development of new statistics and new data sources. In the terminology of project and programme management, technical assurance is generally given due attention in official statistics. Second, while technical quality is necessary it is not sufficient. We conclude that there is considerable scope for official statisticians to devote more attention to those aspects of quality that address the fitness for purpose of their products and outputs. User assurance protocols, procedures, skills, and capability generally need honing and spreading more widely across the official statistics system. There are exemplars of good practice but, overall, official statisticians should give considerably greater

132

P. ALLIN AND D. J. HAND

and more sustained attention to users, potential users, and to the utility of the statistics they produce. Third, more trust has to be earned by the producers of official statistics. A simple formulation is that the trustworthiness of official statistics is based on their technical quality and on their usefulness. It is more complicated than that, given that the public can judge trustworthiness also on how politicians and others use, or even misuse, official statistics. However, it seems clear to us that improving use and usefulness of official statistics should be a major determinant in increasing trust in official statistics. As the UNECE (2018, p. 10) discovered, “Users who trust official statistics most also seem to value them most highly” and “Business users and decision makers tend to value and trust official statistics less than government users”. Fourth, official statistics were born and grew to maturity before the internet age. There were far fewer sources of information competing with official government sources. Information was mediated by fewer organisations, among whom there were clear authoritative channels, such as public service broadcasters and newspapers aiming to provide the record of the day. The internet has enabled more people to reach official statistics more easily. The web is indeed worldwide, connecting, providing inter-connections unrestrained by geography and, with some notable exceptions, unfettered by differences in language. But there are also many more sources, offering in some cases disparate readings of the statistics (sometimes even fake news), making access to reliable official statistics more challenging. This has also happened alongside a shift in many political economies, with government functions generally being reduced and private companies providing what citizens still see as public services. Our fifth broad conclusion is more about the way in which official statistics are treated, and perhaps have always been treated. Novelist David Lodge (2002, p. 266) has one of his characters display “a rather oldfashioned Enlightenment faith in the perfectibility of society through the application of science”. We agree this would be rather a fine way of proceeding, with statistics part of science. However, it is not like that. Even David Hume, a titan in the Scottish Enlightenment, displayed “a stunning diminution of the role that reason plays in human life … matched by a great expansion of the roles played by custom, habit, passion and the imagination”, according to one biographer (Rasmussen 2019, p. 23). The latter roles are found in politicians and members of the public alike.

6

CONCLUSION

133

Taking these all together convinces us that democracies need statistics, to paint a picture of society and then, also, perhaps ideally, to enable society to progress. However, it is far from clear precisely how official statistics do this in the contemporary information space. Kelman (1985, p. 379) presented evidence from the development of official statistics in the US that argues “that government became deeply involved in information-gathering for reasons that had little to do with any assistance that such information could provide private individuals”. Rather, the main reasons were as an aid to legislation, as a source of patriotic pride for all citizens, as a signal from society of recognition to individual groups, and as a statement by society about the special value of knowledge (p. 365). These suggest a significant role for official statistics, bound up in the way in which society operates; they are part of the culture. Official statistics do provide a language to summarise the state of society, the economy, the natural environment and how these are changing, but this language is also moderated, for example as to which groups in society are recognised, and which are not, and over what we mean by progress. Much as we are fans of statistics, we should at this point say that we also recognise other ways of drawing attention to the state of the world, which can appeal more to custom, habit, passion, and the imagination. For example, Canadian photographer and filmmaker Edward Burtynsky “has been travelling the planet making astonishing images of landscapes. In Burtynsky’s landscapes we see the Earth we live on right now: a place humans have hacked up, carved up, blown up, spilled on and recycled. ‘I want to use my images,’ Burtynsky said in his 2005 TED Prize acceptance speech, ‘to persuade millions of people to join in the global conversation on sustainability’” (Doerr 2018, p. 166). Susan Sontag is also convinced that photographic images have made profound changes to our way of looking at the world, creating a feeling of reality. The caveat is that the photographer can no longer be “thought to be an acute but noninterfering observer” taking impersonal, objective images. Rather, she presents as a fact that “photographs are evidence not only of what’s there but of what an individual sees, not just a record but an evaluation of the world” (Sontag 2008, p. 88). There is a read-across here to the compilation of statistics, how they are curated and presented as both a record and an evaluation of the world. But do people read and respond to statistics, even when presented graphically, in the way in which they see images? It is far from clear how official statistics are used and what their role will be from here on. As we anticipated from the outset, although we remain

134

P. ALLIN AND D. J. HAND

convinced of what the broad role should be, as set out in the Fundamental Principles, we have not come to a definitive answer how this will be played out. We have touched on a number of challenges facing official statistics, some of which are implicit in the opportunities for NSOs, such as big data and data science, discussed in Chapter 5. These challenges are known and well described elsewhere (e.g. Evans et al. 2019, p. 4). We conclude by stressing the importance and the urgency of taking forward debate and discussion in order to reach a consensus on the role of official statistics. It is reassuring to see that there are already places starting to appear (online of course) in which this can be progressed. In particular, we encourage everyone to participate in the discussions hosted on Official Statistics (https://officialstatistics.com/), a platform for individuals and institutions in the field of statistics in all parts of the world, supported by the International Association for Official Statistics. It would be tempting to build up a theory-of-change model for how the SDGs are to be achieved and how official statistics play a wider role in enhancing wellbeing. However, we are not convinced either of the value or of the practicality of doing this. There are many different players and agents: national and local government, inter-governmental and non-governmental organisations, corporations, financial institutions and individuals. Each may have their own aims, within which the aim of promoting the wellbeing of the population and of the natural environment may be at best subsidiary. The examples we have found that might identify where official statistics might be most effectively used in moving beyond GDP are all high-level and abstract. Fioramonti (2017, p. 37) proposes a political economy approach in which “post-GDP indicators connect top-down trends”, such as UN reforms and climate change regulations, with “bottom-up pressures” such as civil society and the wellbeing economies we referred to in Chapter 3, and enabled by “new technologies”. Radermacher (2019, p. 534) analyses the relationships between “data, facts, policy/politics” and identifies the key actors and activities found in the relationships between each of these, again in broad terms.

6.3

Our Recommendations

Turning to what might be done, we envisage that the official statistics system will be most effective if it is no longer configured only in terms of government-funded organisations. Rather, we propose that the kinds of statistics covered by the Fundamental Principles are considered as

6

CONCLUSION

135

public statistics, to be produced not only by existing NSOs but also by other organisations which, crucially, share and can demonstrate commitment to the principles. As we have repeatedly stressed, this especially has to deliver high levels of user and potential user engagement. An example of bringing other organisations into the scope of official statistics is found in the UK, where the Office for Statistics Regulation encourages other producers of data, statistics, and analysis to adopt the Code of Practice for Statistics on a voluntary basis and to publish a statement of compliance (https://www.statisticsauthority.gov.uk/code-of-practice/vol untary-application-of-the-code/). The case for public rather than official statistics can be made on grounds of principle, around improving public confidence in the numbers that underpin the progress of society. The point is that it is the numbers that are essential for society to function, not that the providers of this public service are themselves necessarily part of the public sector. This is particularly relevant in the complex web flow of information, where it is more important to be able to assess whether or not to trust figures as you come across them, rather than having to go to a limited number of outlets to get the figures. Think of “the great looms of information” described by Olga Tokarczuk, quoted in Chapter 4. There is also a pragmatic imperative. As for example MacFeely and Nastava (2019, p. 311) have pointed out, “The far reaching ambition of the [UN’s] 2030 Agenda has led to development targets that are well ahead of available official statistics and statistical concepts. In many cases, appropriate statistical methodologies do not yet exist from which to generate indicators”. Their proposal is that “official statistics switch from a purely production or manufacturing based model to a mixed business model: one combining the manufacture of official statistics with the franchising of production under license”, with accreditation based on the Fundamental Principles. The point of that is to help identify reliable and trustworthy sources of relevant information, to meet user needs, as we discussed in Chapters 2, 3, and 5. We finish with seven specific recommendations: 1. NSOs need a better understanding of how public statistics are used. At the beginning of this book we slipped in a mention of teleology, signalling that we would be seeking understanding of the uses and purposes of official statistics. We have found this lacking and so we encourage NSOs to take the lead in addressing

136

P. ALLIN AND D. J. HAND

this. It is not about building a full, detailed, overarching description of how official statistics are, or could be, used but it does need to be fit for the greater degree of user engagement that we recommend below, including recognising different segments within the potential user base. One promising way forward on data to support the SDGs is the initial phase of research carried out by the ONS (n.d.) about users of its SDG website, identifying four main types of user: concerned citizen; connected influencer; fact gatherer; involved analyst. There is a relevant literature within organisational and management studies on the importance of measurement in business decision-making, much building on the work of Frederick Taylor (1967). This will also apply to policy-making and the delivery of public services, where government statistics are used within an organisation, but less so to the more general use of official statistics across society. The development of non-financial business reporting is a reminder of the need to look at the use of statistics in specific ways. Where statistics are used in information aimed at empowering citizens, those preparing the information need to aware that how information is presented can be relevant to changing behaviour. For example, showing life expectancy in terms of years lost due to persisting in smoking is more effective than referring to years gained by quitting (Halpern et al. 2004, p. 39). 2. NSOs need to engage more, and more continually, with users and prospective users of data and statistics, including over the design and implementation of tools to access data and statistics. We have addressed this in the context of national wellbeing elsewhere (Allin and Hand 2017, pp. 364–366), as has the Statistics User Forum (2019, Annex 1, para 16) more generally in its evidence to a Parliamentary committee, but we are not being prescriptive on the detail of engagement, as long as effective engagement is put in place. Although we recognise that social media is pervasive (see recommendation 5 below), it gives rise to a rather superficial sense of engagement if NSOs give too much weight to the counts of shares and likes for the material that they post. 3. All proposals for new measures should be explicit on how they are to be used. A policy to measure something, even to bring together the latest research and data to give a clear picture, is unlikely to be adequate in tackling social and environmental issues

6

CONCLUSION

137

and inequalities. Intended use can then be shared around policy and research communities (e.g. VanderWeele et al. 2020). 4. NSOs should join with statistical societies, educators, and media organisations in raising the level of statistical literacy in the population. Life skills and media-handling skills should include greater understanding of statistics and the ability and confidence to ask questions about statistics presented in the media. We spotted recently that a UK government minister was reported as imploring the public to “take some simple steps before sharing information online, such as always reading beyond the headline and scrutinising the source” (Proctor 2020). There is research and advice available on how to engage as an active citizen online (e.g. Oremus 2020). Many people will still need help in scrutinising the source of statistics and there is a great opportunity to build and raise the “public statistics brand” as something that people can trust. 5. NSOs must use social media alongside other ways of publishing, publicising, and engaging with users. Work on user engagement is likely to confirm that social media are important but should not crowd out other channels of communication. 6. NSOs should work with others to turn indicators and statistics into messages and accurate narratives that can be understood, trusted, and used by people and businesses. In terms of wellbeing, this is likely often to be along the lines of summarising the likely impact on wellbeing now and in the future by doing X rather than Y. 7. NSOs already publishing indicators of national wellbeing should take the lead in introducing a single, summary measure of national wellbeing. A Parliamentary committee considered the pros and cons of this for UK statistics in 2013/2014, coming to the recommendation that “The Government and ONS should not at this stage attempt to define a headline measure for overall well-being, or for overall subjective well-being, and should not contemplate such a move until a measurement track-record has been built up on the component measures, they have achieved a reasonable level of public familiarity, and a general consensus has been reached on their value and usefulness” (Environmental Audit Committee 2014, Conclusions para 11). Value and usefulness have been subsequently endorsed, especially through the UN’s 2030 Agenda, and a measurement track-record has certainly been established, not only by the UK. As to public familiarity, we endorse

138

P. ALLIN AND D. J. HAND

the case made by Hall and Rickard (2013, p. 22), that wellbeing measures “need to be disseminated (and sometimes combined into a single composite index) in such a way as to help move debate and discussion beyond the traditional focus on GDP”. This applies even more now. The lessons learned from doing that will then need to be applied in assessing overall progress towards the 2030 SDGs, where there could be up to 240 indicators to summarise. We borrow the words of MacFeely and Nastava (2019, p. 324) about their proposal to apply to ours: this “is not a panacea. Myriad problems will remain, new and unforeseen ones will arise. But it may unleash the untapped productivity and creativity of a wider data ecosystem”. Like them, we are particularly concerned that the SDGs should be met by 2030. With only ten years to go, at the time of writing this, the OECD has warned that progress towards the achievement SDGs is uneven, both across and within countries. Tackling COVID-19 is now putting enormous pressures on governments, people, and households and is, of course, the primary focus for the present. This is nevertheless highlighting the importance of having relevant, trustworthy, and timely statistics. All countries, not just lower income countries, appear to be facing challenges in delivering statistics, which need to be urgently addressed. But having public statistics is not enough. There needs to be more debate, informed by the statistics, on the major changes that may well be necessary in our societies and our economies. We trust that the issues we have raised will be discussed, considered and applied, ensuring that we all have—and use—the reliable information we need, drawing on the ever-changing data landscape.

References Allin, P., & Hand, D. J. (2017). From a System of National Accounts to a Process of National Wellbeing Accounting. International Statistical Review, 85(2), 355–370. https://doi.org/10.1111/insr.12215. Dent, T. (2020). https://www.ukplc.uk.com/why-is-measuring-the-digital-eco nomy-so-difficult-when-everything-is-stored-as-data/. Accessed 29 March 2020. Doerr, A. (2018). Introduction to Renderings by Edward Burtynsky. Granta, 143, 166–192.

6

CONCLUSION

139

Environmental Audit Committee. (2014). Fifteenth Report: Well-Being. https:// publications.parliament.uk/pa/cm201314/cmselect/cmenvaud/59/5902. htm. Evans, J., Ruane, S., & Southall, H. (2019). Data in Society. Bristol: Policy Press. Fioramonti, L. (2017). The World After GDP. Cambridge: Polity Press. Frederiksen, P. (Ed.). (2013). Policy Use and Influence of Indicators. Ecological Indicators, 35, 1–62. Frederiksen, P., Barankova, Z., & Bauler, T., et al. (n.d.) Policy Influence of Indicators––POINT . https://ec.europa.eu/eurostat/cros/system/ files/S20P1.pdf. Accessed 30 March 2020. Frederiksen, P., & Gudmundsson, H. (2013). Policy Use and Influence of Indicators. Ecological Indicators, 35, 1–2. https://doi-org.iclibezp1.cc.ic.ac.uk/ 10.1016/j.ecolind.2013.05.016. Accessed 30 March 2020. Hall, J., & Rickard, L. (2013). People, Progress and Participation: How Initiatives Measuring Social Progress Yield Benefits Beyond Better Metrics. Global Choices 1. Bertelsmann Stiftung, Gütersloh. Halpern, D., Bates, C., Mulgan, G., & Aldridge, S. with Greg Beales and Adam Heathfield, Prime Minister’s Strategy Unit, Cabinet Office. (2004). Personal Responsibility and Changing Behaviour: The State of Knowledge and Its Implications for Public Policy. Prime Minister’s Strategy Unit discussion paper. http://webarchive.nationalarchives.gov.uk/+/http:/www.cabine toffice.gov.uk/media/cabinetoffice/strategy/assets/pr2.pdf. Accessed 9 April 2020. Kelman, S. (1985). Why Should Government Gather Statistics, Anyway? Journal of Official Statistics, 1(4), 361–379. Lodge, D. (2002). Thinks …. London: Penguin Books. MacFeely, S., & Nastava, B. (2019). “You Say You Want a [Data] Revolution”: A Proposal to Use Unofficial Statistics for the SDG Global Indicator Framework. Statistical Journal of the IAOS, 35, 309–327. https://doi.org/10.3233/SJI180486. ONS. (n.d.). User Personas. https://sustainabledevelopment-uk.github.io/userpersonas/. Accessed 23 March 2020. Oremus, W. (2020). The Simplest Way to Spot Coronavirus Misinformation on Social Media. https://onezero.medium.com/the-simplest-way-to-spot-cor onavirus-misinformation-on-social-media-4b7995448071. Accessed 24 March 2020. Proctor, K. (2020). UK Anti-Fake News Unit Dealing with Up To 10 False Coronavirus Articles a Day. https://www.theguardian.com/world/2020/mar/ 30/uk-anti-fake-news-unit-coronavirus. Accessed 31 March 2020. Radermacher, W. J. (2019). Governing-by-the-Numbers/Statistical Governance: Reflections on the Future of Official Statistics in a Digital and Globalised

140

P. ALLIN AND D. J. HAND

Society. Statistical Journal of the IAOS, 35, 519–537. https://doi.org/10. 3233/SJI-190562. Rasmussen, D. C. (2019). The Infidel and the Professor. Woodstock, UK: Princeton University Press. Sontag, S. (2008). On Photography. London: Penguin Modern Classics. Statistics User Forum. (2019). Written Evidence from the Statistics User Forum (GOS 24). http://data.parliament.uk/writtenevidence/committeeevidence. svc/evidencedocument/public-administration-and-constitutional-affairs-com mittee/governance-of-official-statistics/written/96403.html. Accessed 10 March 2020. Taylor, F. W. (1967). The Principles of Scientific Management. New York: W.W. Norton. UNECE. (2018). Recommendations for Promoting, Measuring and Communicating the Value of Official Statistics. United Nations, Geneva. http://www. unece.org/index.php?id=51139. Accessed 9 March 2020. United Nations. (2014). Fundamental Principles of Official Statistics. https:// unstats.un.org/unsd/dnss/gp/FP-New-E.pdf. Accessed 9 March 2020. VanderWeele, T. J., Trudel-Fitzgerald, C., Allin. P., Farrelly, C., Fletcher, G., Frederick, D. E., … Kubzansky, L. D. (2020). Current Recommendations On the Selection of Measures for Well-Being. Preventive Medicine, 133, 106004. https://doi.org/10.1016/j.ypmed.2020.106004.

Index

A Argentina, 16, 119–121

B Behaviour change, 76, 97 Better Life Index, 72, 73, 101 Beyond GDP, 19, 20, 29, 35, 38, 61, 68, 70, 73, 98, 128, 129, 134 Bhutan, 30 Big data. See Data, big Billion Prices Project, 118 Boskin Commission, 17 BRAINPOoL Project, 61 Businesses, 30, 39, 41, 46, 52, 56, 75, 76, 85, 86, 95–99, 102, 104, 128

C Cameron, David, 37, 65, 69 Capabilities, 20, 45 Census, 9, 11, 88, 102, 117

Citizens, 30, 39, 41, 46, 52, 56, 75, 76, 85, 86, 95–99, 102, 104, 128 Clark, Colin, 14 Climate change, 2, 36, 134 Commission on the Measurement of Economic Performance and Social Progress. See Stiglitz, Sen, Fitoussi Report Communication, 26, 42, 45, 101, 121, 137 COVID-19, 121, 128, 138

D Dashboard, 42, 63, 73, 74, 101 Data administrative, 11, 44, 63, 114, 116–119 alternative, 116, 118, 119 behavioural, 62, 63 big, 28, 91, 92, 112, 134 ecosystem, 7, 26, 116, 138

© The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 P. Allin and D. J. Hand, From GDP to Sustainable Wellbeing, Wellbeing in Politics and Policy, https://doi.org/10.1007/978-3-030-53085-3

141

142

INDEX

protection, 115 quality, 39, 58, 87, 90, 91, 103, 117, 121, 129 revolution, 112, 113 science, 9, 91, 92, 113–115, 123, 134 value chain, 91 Data science, 91, 114 Digital Economy Act, 63, 115, 116 Disinformation, 58, 102 Distribution, 10, 11, 43, 52 Double-entry bookkeeping, 9, 11

E Equality, 96 Estimation, 27, 118, 124 European Statistical System, 43, 85 Evidence, 3, 6, 28, 30, 37, 39, 51–57, 59–62, 67–69, 72, 74–76, 85, 86, 89, 90, 92, 95, 98, 99, 112, 123, 124, 129, 133, 136 Evidence-informed policy, 57, 61 Evidence translators, 50

F Facts, 3–10, 13–15, 18, 26, 28–31, 34, 43, 57, 58, 89, 94, 95, 103, 112, 115, 116, 130, 133, 134, 136 Fake news, 58, 59, 132 Field of Dreams, 33, 102 Functionings, 20, 68 Fundamental Principles of Official Statistics, 13, 50, 85, 93, 97, 102, 104, 129

G Gapminder, 92 GDP (Gross Domestic Product), 3–10, 13–15, 18, 26, 28–31, 34,

43, 57, 58, 89, 94, 95, 103, 112, 115, 116, 130, 133, 134, 136 critique, 13–16 definition, 8, 15, 16 Gender statistics, 96 General Data Protection Regulation, 115 Global Action Plan, 76 Google, 119, 120 Green Book (UK Treasury guidance), 69 GSS (Government Statistical Service), 11, 13 H Health index, 63, 70 Homeless people, 83, 84, 94 How’s Life?, 101 Humankind index, 72 I ICD (International Classification of Diseases), 34 IMR (Infant Mortality Rate), 3 Indicators development, 20, 29, 31, 39, 42–45, 56, 70, 72, 129 quality of life, 96, 100 social, 2, 3, 43–45, 73, 119 sustainable development, 29, 41, 42, 60, 70, 72, 85, 96, 99–101, 105, 128, 129 use, 3, 39, 42, 44–46, 60, 71, 72, 129 Information age, 26 Institute for Government, 55, 62 Intermediaries, 58, 61, 76, 92, 99–101 International Association for Official Statistics, 134 Istanbul Declaration, 29

INDEX

K Kennedy, Robert, 19, 20 Kuznets, Simon, 14, 19

L Legatum Prosperity Index, 100

M Magenta Book (UK Treasury guidance), 54, 69 Measurement, 4, 8, 15–18, 29, 31, 32, 36, 37, 39, 41, 43, 44, 50, 64–68, 70–72, 74, 115, 124, 129, 136, 137 Measuring National Wellbeing Programme (UK), 50, 68 Media, role of, 106, 121 Metadata, 87, 91 Misinformation, 102, 121–123 Missing Numbers blog, 32

N New Zealand, 72, 73, 88 Nightingale, Florence, 10 Non-financial reporting, 99 Northern Ireland, 70, 71, 73 NSOs (National statistical offices), 8, 27, 43, 60, 70, 90, 96, 99, 101, 128, 131, 134–137 Numbers, 5, 45

O Office for National Statistics (UK), 40, 59, 65–67, 69, 70, 74, 83, 101, 103, 117 Official statistics, 3, 7, 9–13, 26–35, 40, 41, 50–52, 54–56, 59–61, 63–66, 68, 70, 71, 74, 76, 85,

143

87–90, 92–97, 99, 100, 102, 103, 133, 135 code of practice, 13, 90 history, 9, 12, 120 marketing, 33, 51, 105 monetary value, 31, 88, 89 non-users, 87 role, 29, 33, 50, 59, 70, 76, 85, 98, 104, 133, 134 trust in, 13, 102, 103, 132 users, 31–33, 51, 75, 85, 87–89, 94, 95, 104, 131, 132, 135, 136 utility, 30, 50, 92, 132 value, 31, 50, 60, 62, 88, 89, 91, 104, 105, 121, 132, 133 OSR (UK Office for Statistics Regulation), 12, 34, 57, 60, 70, 74, 85, 88, 103

P PACAC, 30, 51, 95 PARIS21 (Partnership in Statistics for Development in the 21st Century), 39, 131 POINT (Policy Influence of Indicators), 129 Policy profession skills framework, 55 Poverty measurement, 44 Progress, 20 Public awareness of statistics, 93 Public good, 30, 32, 51, 85, 95 Public statistics, 51, 135, 138 Public value, 30, 85, 88, 92, 130

Q QALY (Quality-adjusted life year), 69 Quality, 13, 17, 32, 36, 37, 46, 50–52, 60, 65, 67, 73, 95, 102, 121, 124, 131, 132

144

INDEX

R Reliability, 8, 46, 69, 116, 121 Road safety, 76, 89 ROAMEF, 53, 55 Royal Statistical Society, 4, 10, 33, 62, 130 S Sarkozy, Nicolas, 35, 36, 84 Science, 4–6, 57, 58, 62, 68, 84, 123, 132 Scotland, 70–73 SDGs (UN Sustainable Development Goals), 38–42, 45, 72, 97–99, 121, 134, 136, 138 SNA (System of National Accounts), 14, 15, 31, 34, 42, 45, 65 Social media, 26, 28, 30, 54, 84, 93, 114, 116, 136, 137 Social Metrics Commission, 44 Social progress, 1, 3, 42, 45, 64, 70, 72, 99, 128 Social Progress Index, 100 Social value, 69 Standards, 13–15, 17, 18, 29, 34, 45, 59, 62, 65, 69, 74, 98, 130 Statistical Commission (UN), 28, 56 Statistical literacy, 104, 123, 137 Statistical Society of London, 4, 6, 9 Statistics User Forum, 51, 136 Stiglitz, Sen, Fitoussi Report, 35 Sustainability reporting, 98 Sustainable development, 13, 29, 37, 38, 46, 50, 76, 97, 99, 104, 112, 113, 128 T Teleology, 135 Transforming Our World (UN 2030 Action Plan), 38

Trust, 26, 62, 102, 103 Trustworthiness, 8, 13, 26, 30, 42, 46, 50, 52, 59, 60, 103, 104, 121, 123, 132

U UKSSD, 41 UNECE (The United Nations Economic Commission for Europe), 31, 87–89, 102, 132 UN Global Pulse, 112 User engagement, 31, 75, 135, 137 Users of statistics, 83 Utility, 42, 102

V Variance, 11 Voluntary Review, 40

W Wales, 45, 70–73, 84 Web scraping, 119 Wellbeing definition, 118, 123, 129 economies, 14, 21, 31, 32, 65, 69, 72, 73, 98, 134 framework, 65, 69, 71, 74, 76, 101 index, 32, 65 national, 13, 19, 26, 30, 31, 37, 42, 45, 50, 64–66, 68–71, 74, 75, 90, 94, 96, 100, 101, 105, 134, 136, 137 Wellbeing economy, 98 WELLBY (Wellbeing year), 69 Welling of Future Generations Act, 45 Welling policy, 52 What Works Centres, 56, 59, 61, 62, 68, 75