Artificial Intelligence and Inclusive Education: Speculative Futures and Emerging Practices [1st ed.] 978-981-13-8160-7;978-981-13-8161-4

Table of contents :
Front Matter ....Pages i-xiii
Introduction: AI, Inclusion, and ‘Everyone Learning Everything’ (Jeremy Knox, Yuchen Wang, Michael Gallagher)....Pages 1-13
Front Matter ....Pages 15-15
Towards Inclusive Education in the Age of Artificial Intelligence: Perspectives, Challenges, and Opportunities (Phaedra S. Mohammed, Eleanor ‘Nell’ Watson)....Pages 17-37
Accountability in Human and Artificial Intelligence Decision-Making as the Basis for Diversity and Educational Inclusion (Kaśka Porayska-Pomsta, Gnanathusharan Rajendran)....Pages 39-59
Artificial Intelligence in Education Meets Inclusive Educational Technology—The Technical State-of-the-Art and Possible Directions (Gunay Kazimzade, Yasmin Patzer, Niels Pinkwart)....Pages 61-73
Front Matter ....Pages 75-75
Interactions with an Empathic Robot Tutor in Education: Students’ Perceptions Three Years Later (Sofia Serholt)....Pages 77-99
A Communication Model of Human–Robot Trust Development for Inclusive Education (Seungcheol Austin Lee, Yuhua (Jake) Liang)....Pages 101-115
An Evaluation of the Effectiveness of Using Pedagogical Agents for Teaching in Inclusive Ways (Maggi Savin-Baden, Roy Bhakta, Victoria Mason-Robbie, David Burden)....Pages 117-134
Inclusive Education for Students with Chronic Illness—Technological Challenges and Opportunities (Anna Wood)....Pages 135-148
Front Matter ....Pages 149-149
Shaping Our Algorithms Before They Shape Us (Michael Rowe)....Pages 151-163
Considering AI in Education: Erziehung but Never Bildung (Alex Guilherme)....Pages 165-178
Artificial Intelligence and the Mobilities of Inclusion: The Accumulated Advantages of 5G Networks and Surfacing Outliers (Michael Gallagher)....Pages 179-194
Artificial Intelligence, Human Evolution, and the Speed of Learning (Michael A. Peters, Petar Jandrić)....Pages 195-206

Citation preview

Perspectives on Rethinking and Reforming Education

Jeremy Knox Yuchen Wang Michael Gallagher Editors

Artificial Intelligence and Inclusive Education Speculative Futures and Emerging Practices

Perspectives on Rethinking and Reforming Education Series Editors Zhongying Shi, Faculty of Education, Beijing Normal University, Beijing, China Shengquan Yu, Faculty of Education, Beijing Normal University, Beijing, China Xudong Zhu, Faculty of Education, Beijing Normal University, Beijing, China Mang Li, Faculty of Education, Beijing Normal University, Beijing, China

This book series brings together the latest insights and work regarding the future of education from a group of highly regarded scholars around the world. It is the first collection of interpretations from around the globe and contributes to the interdisciplinary and international discussions on possible future demands on our education system. It serves as a global forum for scholarly and professional debate on all aspects of future education. The book series proposes a total rethinking of how the whole education process can be reformed and restructured, including the main drivers and principles for reinventing schools in the global knowledge economy, models for designing smart learning environments at the institutional level, a new pedagogy and related curriculums for the 21st century, the transition to digital and situated learning resources, open educational resources and MOOCs, new approaches to cognition and neuroscience as well as the disruption of education sectors. The series provides an opportunity to publish reviews, issues of general significance to theory development, empirical data-intensive research and critical analysis innovation in educational practice. It provides a global perspective on the strengths and weaknesses inherent in the implementation of certain approaches to the future of education. It not only publishes empirical studies but also stimulates theoretical discussions and addresses practical implications. The volumes in this series are interdisciplinary in orientation, and provide a multiplicity of theoretical and practical perspectives. Each volume is dedicated to a specific theme in education and innovation, examining areas that are at the cutting edge of the field and are groundbreaking in nature. Written in an accessible style, this book series will appeal to researchers, policy-makers, scholars, professionals and practitioners working in the field of education.

More information about this series at http://www.springer.com/series/14177

Jeremy Knox Yuchen Wang Michael Gallagher •

•

Editors

Artificial Intelligence and Inclusive Education Speculative Futures and Emerging Practices

123

Editors Jeremy Knox Advanced Innovation Center for Future Education Beijing Normal University Beijing, China

Yuchen Wang Advanced Innovation Center for Future Education Beijing Normal University Beijing, China

Michael Gallagher Advanced Innovation Center for Future Education Beijing Normal University Beijing, China

ISSN 2366-1658 ISSN 2366-1666 (electronic) Perspectives on Rethinking and Reforming Education ISBN 978-981-13-8160-7 ISBN 978-981-13-8161-4 (eBook) https://doi.org/10.1007/978-981-13-8161-4 © Springer Nature Singapore Pte Ltd. 2019 This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. This Springer imprint is published by the registered company Springer Nature Singapore Pte Ltd. The registered company address is: 152 Beach Road, #21-01/04 Gateway East, Singapore 189721, Singapore

Foreword

In a world where AI is becoming a feature of everyday life mediated by digital technologies, there are important questions about the role it can and should play within facilitating a good education for all. Yet, at the time of writing this foreword, there are significant disconnects between the motivations and practices of the various academic, commercial, and practical stakeholders engaged in this space. This means that there is somewhat of a disconnect between the development and deployment of AI and the motivations and aims of researchers and practitioners in inclusive education. As a result, much of the discourse and commercial development towards building future education systems are focused on optimisation, where education is viewed in ‘commonsense’ terms of acquiring individual skills in the most efficient manner, and a belief that complex educational problems can be addressed in a similar way to any other business challenge. This has resulted in the focus on the themes present in this important collection, from a logic of replacement as opposed to enhancement, an emphasis on personalisation despite the complete lack of clarity around the term, a narrowing of curriculum where what we should learn becomes what we can measure, and a focus on learning as opposed to a discussion of the purposes of education. In doing so, despite the best of intentions, there are real risks of reinforcing and indeed perhaps exacerbating existing inequalities and injustice in our society. This state of affairs has emerged as governments have looked for a ‘technical fix’ (Robins and Webster 1989) to educational policy aims, with only a few experts able and willing to offer such as answer. These are typically experts with significant expertise in AI but no expertise in education. It is therefore an important moment for meaningful dialogue to occur across all stakeholders engaged in creating and visioning the future of inclusive education. Meaningful dialogue is not easy, as it requires each group to learn and understand one another better. As part of this genuine engagement, is to determine a collective way forward with the principles of accountability, ethics, and transparency at its core. This is necessary to develop a way forward that promotes trust, agency, and equality which are underpinnings of inclusive education.

v

vi

Foreword

It is not, as the contributions here show, sufficient to simply set up a good/bad dichotomy, with ‘traditional’ education on one side and AI on the other, nor create deterministic arguments of either utopian or dystopian kinds. AI can often be the scapegoat for wider fears we have for educational futures. As this book shows, AI is a complex material, social, and cultural artefact; AI is good at some things—but not great at others—and change is not inevitable. It is for us to determine what our educational futures could look like. Through engaging with the methodological, theoretical, practical, and political intersections of AI and inclusive education, this thought-provoking and much-needed book provides a welcome move in this direction, encouraging us all to take responsibility for shaping what comes next. Oxford, UK

Rebecca Eynon Associate Professor and Senior Research Fellow University of Oxford

Reference Robins, K., & Webster, F. (1989). The technical fix: Education, computers and industry. NY: Macmillan Education.

Preface

This book developed from a number of conversations at the Moray House School of Education at the University of Edinburgh in 2017, related to the links between inclusive education and emerging digital technologies. As part of our work with the Centre for Research in Digital Education, we were particularly interested in developing critical responses to the use of data-driven technology and to understand how such systems embody particular ideas about the role of teachers and learners. It was felt that the kind of conversations happening in the inclusive education space were too often overlooked in discussions around the development of ‘intelligent’ technologies for teaching and learning. When Jeremy and Yuchen visited the Advanced Innovation Centre for Future Education (AICFE) in 2018, they were pleased to be able to continue these conversations with colleagues at Beijing Normal University and to form the idea for this book in response to the ‘Future School 2030’ project. Artificial Intelligence and Inclusive Education: Speculative Futures and Emerging Practices has drawn together a broad range of international researchers and practitioners, united by an interest in the challenges and possibilities presented by artificial intelligence (AI). We are immensely grateful to the authors who have shared their exciting work and contributed to this collective exploration of the intersections of AI and inclusion. This volume has been organised into three themes: Artificial Intelligence and Inclusion—Opening a Dialogue; Emerging Practices; and Critical Perspectives and Speculative Futures. We hope that these parts provide readers with a structured way of engaging with the diverse work in this area, offering an overview of the key issues surfaced by AI technologies and inclusive practices, examples of technical development, and productive speculation on possible educational futures. We hope that this volume is relevant and interesting to a wide range of educationalists, in particular teachers and practitioners, and especially those who may not be fully aware of the implications of AI technologies, now and in the near future. Beijing, China

Jeremy Knox Yuchen Wang Michael Gallagher vii

Contents

Introduction: AI, Inclusion, and ‘Everyone Learning Everything’ . . . . . Jeremy Knox, Yuchen Wang and Michael Gallagher Part I

1

Artificial Intelligence and Inclusion—Opening a Dialogue

Towards Inclusive Education in the Age of Artificial Intelligence: Perspectives, Challenges, and Opportunities . . . . . . . . . . . . . . . . . . . . . . Phaedra S. Mohammed and Eleanor ‘Nell’ Watson

17

Accountability in Human and Artificial Intelligence Decision-Making as the Basis for Diversity and Educational Inclusion . . . . . . . . . . . . . . . Kaśka Porayska-Pomsta and Gnanathusharan Rajendran

39

Artificial Intelligence in Education Meets Inclusive Educational Technology—The Technical State-of-the-Art and Possible Directions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Gunay Kazimzade, Yasmin Patzer and Niels Pinkwart Part II

61

Emerging Practices

Interactions with an Empathic Robot Tutor in Education: Students’ Perceptions Three Years Later . . . . . . . . . . . . . . . . . . . . . . . . Sofia Serholt

77

A Communication Model of Human–Robot Trust Development for Inclusive Education . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 Seungcheol Austin Lee and Yuhua (Jake) Liang An Evaluation of the Effectiveness of Using Pedagogical Agents for Teaching in Inclusive Ways . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117 Maggi Savin-Baden, Roy Bhakta, Victoria Mason-Robbie and David Burden

ix

x

Contents

Inclusive Education for Students with Chronic Illness—Technological Challenges and Opportunities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135 Anna Wood Part III

Critical Perspectives and Speculative Futures

Shaping Our Algorithms Before They Shape Us . . . . . . . . . . . . . . . . . . 151 Michael Rowe Considering AI in Education: Erziehung but Never Bildung . . . . . . . . . 165 Alex Guilherme Artificial Intelligence and the Mobilities of Inclusion: The Accumulated Advantages of 5G Networks and Surfacing Outliers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179 Michael Gallagher Artificial Intelligence, Human Evolution, and the Speed of Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195 Michael A. Peters and Petar Jandrić

Editors and Contributors

About the Editors Jeremy Knox is Co-Director of the Centre for Research in Digital Education at the University of Edinburgh. He is Lecturer in Digital Education and, at the time of writing, also a Global Academies and Edinburgh Futures Institute Fellow. His current research is focused on AI, machine learning, algorithms, and data in higher education, with a specific interest in issues related to socio-technical systems, accountability, ethics, justice, and citizenship. He has also worked with critical posthumanism and new materialism and published critical perspectives on Open Educational Resources (OER) and Massive Open Online Courses (MOOCs). He is Associate Editor for new journal Postdigital Science and Education with Springer and serves on the editorial board of Teaching in Higher Education. He has also served as Guest Editor for Learning, Media and Technology and E-Learning and Digital Media. He also co-convenes the Society for Research in Higher Education (SRHE) Digital University network. Yuchen Wang is Research Associate at the Moray House School of Education, The University of Edinburgh. Her research interests include student voice, inclusive practice, disability rights, technology, and international development. Her Ph.D. research explored experiences of children with disabilities and teachers’ practices in Chinese mainstream schools, following which she was awarded the UK Economic and Social Research Council Global Challenges Research Fund Postdoctoral Fellowship to build capacity of disability communities, practitioners, and policymakers in the country. She is currently involved in projects that critically examine the relationships between technology and educational inclusion. She also provides consultancy support for NGOs to promote inclusive and quality provision for children with disabilities in China. Michael Gallagher is Lecturer in Digital Education at the Centre for Research in Digital Education at The University of Edinburgh. At Edinburgh, his projects

xi

xii

Editors and Contributors

include the Near Future Teaching project; a project exploring formal partnerships with edX around new redesigned master's programmes; and projects working with universities in Nepal, Nigeria, Tanzania, and Uganda, on digital education in developing contexts. His research focuses on educational futures, educational mobility, mobile technology, and the impact on local knowledge practices and communities. He is also Co-Founder and Director of Panoply Digital, an ICT4D consultancy that specialises in educational design for inclusion, particularly in low resource environments and particularly with areas where the gender digital divide is the most pronounced. His projects in this space include ongoing work with the World Bank, USAID, GSMA, UN-Habitat, and more.

Contributors Roy Bhakta Capp & Co. Ltd., Birmingham, UK David Burden Daden Limited, Birmingham, UK Michael Gallagher Centre for Research in Digital Education, University of Edinburgh, Edinburgh, UK Alex Guilherme Pontifícia Universidade Católica do Rio Grande do Sul, Porto Alegre, Brazil Petar Jandrić Zagreb University of Applied Sciences, Zagreb, Croatia Gunay Kazimzade Weizenbaum Institute for the Networked Society, Technical University of Berlin, Berlin, Germany Jeremy Knox Centre for Research in Digital Education, University of Edinburgh, Edinburgh, UK Seungcheol Austin Lee Chapman University, Orange, USA Yuhua (Jake) Liang (in memoriam), Chapman University, Orange, USA Victoria Mason-Robbie University of Worcester, Worcester, UK Phaedra S. Mohammed Department of Computing and Information Technology, The University of the West Indies, St. Augustine, Trinidad and Tobago Eleanor ‘Nell’ Watson AI & Robotics Faculty, Singularity University, Mountain View, USA; Dean of Cognitive Science, Exosphere Academy, Palhoça, Brazil Yasmin Patzer Humboldt-University Berlin, Berlin, Germany Michael A. Peters Beijing Normal University, Beijing, China Niels Pinkwart Humboldt-University Berlin, Berlin, Germany

Editors and Contributors

xiii

Kaśka Porayska-Pomsta UCL Knowledge Lab, University College London, UCL Institute of Education, London, UK Gnanathusharan Rajendran Edinburgh Centre for Robotics, Department of Psychology, Heriot-Watt University, Edinburgh, UK Michael Rowe University of the Western Cape, Cape Town, South Africa Maggi Savin-Baden University of Worcester, Worcester, UK Sofia Serholt The University of Gothenburg, Gothenburg, Sweden Yuchen Wang Centre for Research in Digital Education, University of Edinburgh, Edinburgh, UK Anna Wood University of Edinburgh, Edinburgh, UK

Introduction: AI, Inclusion, and ‘Everyone Learning Everything’ Jeremy Knox, Yuchen Wang and Michael Gallagher

Abstract This chapter provides an introduction to the book—Artificial Intelligence and Inclusive Education: speculative futures and emerging practices. It examines the potential intersections, correspondences, divergences, and contestations between the discourses that typically accompany, on the one hand, calls for artificial intelligence technology to disrupt and enhance educational practice and, on the other, appeals for greater inclusion in teaching and learning. Both these areas of discourse are shown to envision a future of ‘education for all’: artificial intelligence in education (AIEd) tends to promote the idea of an automated, and personalised, one-to-one tutor for every learner, while inclusive education often appears concerned with methods of involving marginalised and excluded individuals and organising the communal dimensions of education. However, these approaches are also shown to imply important distinctions: between the attempts at collective educational work through inclusive pedagogies and the drive for personalised learning through AIEd. This chapter presents a critical view of the quest for personalisation found in AIEd, suggesting a problematic grounding in the myth of the one-to-one tutor and questionable associations with simplistic views of ‘learner-centred’ education. In contrast, inclusive pedagogy is suggested to be more concerned with developing a ‘common ground’ for educational activity, rather than developing a one-on-one relationship between the teacher and the student. Inclusive education is therefore portrayed as political, involving the promotion of active, collective, and democratic forms of citizen participation. The chapter concludes with an outline of the subsequent contributions to the book. Keywords Personalisation · Individualism · One-to-one tutoring · Special education · Community J. Knox (B) · Y. Wang · M. Gallagher Centre for Research in Digital Education, University of Edinburgh, Edinburgh, UK e-mail: [email protected] Y. Wang e-mail: [email protected] M. Gallagher e-mail: [email protected] © Springer Nature Singapore Pte Ltd. 2019 J. Knox et al. (eds.), Artificial Intelligence and Inclusive Education, Perspectives on Rethinking and Reforming Education, https://doi.org/10.1007/978-981-13-8161-4_1

1

2

J. Knox et al.

1 Introduction On 16th October 2018, the Parliament Education Committee in the UK hosted its first ‘non-human’ witness. A humanoid robot, developed by Softbank robotics and known as ‘Pepper’, was presented to the select committee by researchers from Middlesex University, with the intention of answering questions about ‘the Fourth Industrial Revolution and the implications for education of developments in artificial intelligence’ (UK Parliament 2018). While this highly publicised event, in which the robot’s answers were pre-prepared, seemed to be more of a public relations exercise than a serious Parliamentary interrogation of intelligent machines, the session marked something of a significant achievement for raising the profile of AI in Education (see Luckin et al. 2016). Panel members also included (human) academics, students, and representatives from industry and charitable foundations, who supplied MPs with their views on the need for education to adapt to an era of increasing automation from AI. Indeed, the message appeared to be rather far-reaching, with all panel members supposedly agreeing that ‘the current educational system had to change drastically to accommodate the pace of technological change’ (Wakefield 2018). Significantly, the specific projects highlighted during the event related to ‘helping children with special needs improve their numeracy’ and ‘caring for older people’ (ibid.), positioning Pepper the robot in an assistive role, concerned with widening access and accommodating diverse groups of learners. The cause of Inclusive Education has been promoted for some time, recently through the UN’s Sustainable Development Goal 4: to ‘ensure inclusive and equitable quality education and promote lifelong learning opportunities for all’. In the same year, UNESCO launched the Incheon Declaration, which provided a framework through which this goal might be realised by 2030. Participants at the accompanying World Education Forum included delegates from 160 countries, comprised of government ministers, representatives from various educational organisations, youth groups, and the private sector. The declaration extends the legacy of the worldwide movement ‘Education for All’, established through similar forums in Jomtien in 1990 and in Dakar in 2000, positioning the inclusive education agenda firmly within the realm of global politics. Two high-profile visions for the future of education appear to be at work here, both seemingly concerned with what Friesen discusses, via the work of Johann Amos Comenius and Christoph Wulf, as the educational ‘dream’ of ‘everyone learning everything’ (forthcoming 2019: 2). The first vision promises with AI technologies that provide new kinds of scientific precision in the analysis of educational activity—known as AIEd; ‘giving us deeper, and more fine-grained understandings of how learning actually happens’ (Luckin et al. 2016: 18). Importantly, this vision is often grounded in, and oriented towards, the idea of providing ‘an intelligent, personal tutor for every learner’ (Luckin et al. 2016, p. 24). The second, predominantly concerned with universal educational provision, presents ‘a new vision’ for education that is ‘comprehensive, holistic, ambitious, aspirational and universal, and inspired

Introduction: AI, Inclusion, and ‘Everyone Learning Everything’

3

by a vision of education that transforms the lives of individuals, communities and societies, leaving no one behind’ (UNESCO 2015: 24). This book is motivated by an interest in the potential intersections, correspondences, divergences, and contestations between these future visions of ‘quality education for everyone’. While there is a long-established field of research and development, not only in AIEd (see Luckin et al. 2016), but also in supportive and assistive technology (e.g. Edyburn et al. 2005), both the rise in popularity and ubiquity of techniques such as machine learning, often understood as ‘the new AI’ (Alpaydin 2016), and the growing awareness of the distinction of inclusive rather than special education, have suggested a very particular space of enquiry for which this book hopefully provides an engaging opening. Nevertheless, the apparent shared interests of AIEd and the inclusive education movement have not been lost on others concerned with educational futures. Houser (2017) suggests the former as a solution for the ‘crisis’ in education identified by the latter, where ‘[d]igital teachers wouldn’t need days off and would never be late for work’, and further, ‘administrators wouldn’t need to worry about paying digital teachers’. Given that UNESCO suggests a need to ‘recruit 68.8 million teachers’ (UIS 2016) in order to achieve the broad aims of inclusive and equitable education, one might therefore view AIEd, and its promise of personalised, one-to-one tuition, as a rather neat technical and cost-efficient fix, especially where resources seem to be increasingly limited for education systems under current global economic and political circumstances. However, while both AIEd and inclusive education could be understood as sharing a vision of ‘education for all’, one might discern important differences, both in the means of achieving such a feat, as well as in the character of the education that is supposedly realised. Firstly, it might be pointed out that the UN and UNESCO’s work are concerned with the overarching governance of educational development at a global scale, while AIEd research and development tends to be more focused on context-specific pedagogical interventions and practices. Nevertheless, both these areas assume and convey a ‘worldview’ about education that is worth surfacing in this introduction. The definition of inclusive education tends to be considered in terms of educational practices that might be understood as expansive and embracing and concerned with the organisational and communal dimensions of education. In contrast, AIEd is often grounded in a much more individualised view of the learning experience. This key distinction, between the attempts at collective educational work through inclusive pedagogies and the drive for personalised learning through AIEd, offers an important way of distinguishing these future visions of ‘everyone learning everything’. However, personalisation and inclusion are certainly not mutually exclusive conditions, as the various chapters in this volume will demonstrate. To help set the scene, the following section of this introduction will elaborate on one possible way of articulating this distinction, as a way of identifying some of the productive territory for critical research in this volume.

4

J. Knox et al.

2 Personalised Learning Versus Teaching for a ‘Common Ground’ A Nuffield Foundation report published this year on Ethical and societal implications of algorithms, data, and artificial intelligence identified ‘personalisation versus solidarity’ as a key ‘tension between values’ (Whittlestone et al. 2019: 20). This tension is apparent in the development of AI for inclusive education goals. A narrative of ‘personalisation’ often appears to drive AI research and development in education, in which ‘intelligent’, data-driven applications are employed to tailor educational content, or automate tutor feedback, for an individual student. There is certainly high-profile interest in this area, with figures such as Bill Gates investing significant funds for research and development (Newton 2016), and Facebook developing ‘Personalized Learning Plan’ software, initially with a school network in California, but also with national ambitions (Herold 2016). Predictions for the success of such systems and claims about the need to restructure the role of the (human) teacher are rife. Notable here is the common interest in perceiving AI not just as replicating the role of teacher or tutor, but also as producing a kind of ultimate pedagogue that provides an advanced and elite form of education. For example, Sir Anthony Seldon’s recently predictions for educational AI included ‘the possibility of an Eton or Wellington education for all’ in which ‘everyone can have the very best teacher’ enabled through ‘adaptive machines that adapt to individuals’ (von Radowitz 2017). This notion also appears to have a significant influence at the educational publisher Pearson, where the director of artificial intelligence Milena Marinova portrays an idealised world in which ‘every student would have that Aristotle tutor, that one-onone, and every teacher would know everything there is to know about every subject’ (see Olson 2018). Media reporting seems particularly keen to publicise cases of AI tuition. ‘Jill Watson’ is one such example, developed by Professor Ashok Goel, about which media reports consistently emphasised the idea that it was indistinguishable from a human teacher (Hill 2016; Leopold 2016). While such efforts to envision and develop a supreme ‘AI tutor’ may represent far more serious endeavours than Pepper the robot visiting the UK Parliament, there appear to be a number of questionable assumptions underpinning the quest for machine-driven personalisation. As Friesen (forthcoming 2019) demonstrates, there is a long history, and orthodoxy, to the idea of the personalised tutor, which, through the promise of AIEd, manifests as part of the contemporary ‘technological imaginary’: The vision for the future that these technologies promise to fulfil, moreover, could not be any more total: Their global availability to every man, woman and child, and for any topic that they might wish to learn. (ibid.: 2)

While such ambitions might sound rather appealing to those interested in realising the goal of an inclusive educational system, Friesen (ibid.) highlights the underlying ‘mythology’ of this imagined future, involving the establishment of the primal value of the one-to-one pedagogical relationship, as well as the idea that computers are able to simulate it. Rather than innovation, this constitutes a ‘a kind of repetitive

Introduction: AI, Inclusion, and ‘Everyone Learning Everything’

5

continuity that educational innovators generally see themselves as leaving behind’ (ibid.: 4). Importantly, what Friesen suggests is that the ideal of one-to-one tuition is much better understood through the parables and allegories offered by the likes of Socrates (through Plato and Xenophon), Comenius, and Rousseau, rather than through the scientific precision of the Enlightenment, and the engineering disciplines of computer science that followed it. For Friesen, ‘[d]ialogue, in short, is a ubiquitous yet irreducible experience’ (ibid.: 12). Here, then, the reproduction of educational activity within rational, technical systems becomes something much more akin to a regime of control than a culture of inclusion. As Friesen suggests: for education or any other aspect of social activity to fall so completely under the dominance of a total vision of social and technical engineering would be “totalitarian” in and of itself. (ibid.: 13)

Speculation about the role of the teacher in the era of artificial intelligence features throughout this volume. Michael Rowe’s chapter discusses the increasing power of machine learning techniques employed for teaching purposes and reflects on the extent to which teacherly activity can be replaced by ‘brute force computation’ (p. 142). Alex Guilherme’s chapter also engages directly with the questions around teacher replacement, developing the concept of Bildung as a way of understanding the differences in human and machine capabilities in education. This volume also includes research of ‘pedagogical agents’, examined in the chapter from Maggi Savin-Baden, Roy Bhakta, Victoria Mason-Robbie and David Burden. Here, the effectiveness of adaptive tutoring software is measured against aspects of ‘human’ teaching. Emphasising this idea of AI in a supporting role, Ka´ska Porayska-Pomsta and Gnanathusharan Rajendran also examine the use of ‘AI agents’ to augment specific teaching scenarios. While authors in this volume consistently argue against the notion of AI straightforwardly replacing (human) teachers, we are certainly reminded of the importance of practitioners’ active participation in the decision-making processes of AI development, and the necessity of establishing standards that might promote inclusion, perhaps in a very similar way to the training of human teachers and regulation of professional teaching conduct. The utopian technological vision of personalisation, while perhaps grounded in the myth of the authentic educational dialogue, also appears to align rather seamlessly with contemporary views of ‘learner-centred’ education. AI systems that purportedly support one-to-one tutoring from human teachers, such as the Carnegie Learning1 or Third Space Learning2 platforms, function through analysing learner behaviours and providing real-time feedback from student activities. AI, here in a supporting role, provides value through its ability to observe, and ‘know’, students’ learning behaviour in forensic detail. Whether through adaptive software, or overt AI tutoring systems, work in AIEd is often premised upon a tacit assumption that education is at its best when it is developed around, and in response to, an individual student, who is understood to already possess particular abilities, proclivities, or desires in relation 1 See 2 See

https://www.carnegielearning.com/. https://thirdspacelearning.com/how-it-works/.

6

J. Knox et al.

to the learning process. In other words, AI-driven ‘personalisation’ not only views the individual as the key source of information on how to structure and organise educational activity, but also positions education as ultimately subservient to a notion of innate human characteristics, rendered discoverable through ever-expansive data capture practices. This orientation tends not only to view student behaviour as the decisive factor in determining the action of the teacher, but also to restructure educational activity around the idea of the personal. Thus, the ‘learnification’ of education (see Biesta 2005, 2006, 2012), as part of a broad societal shift towards the individual, achieves a particular intensity through data-driven educational technologies (Knox et al. forthcoming 2019). Indeed, as Knox et al. (ibid.) discuss, through the training of machine learning systems and the use of nudging techniques in educational software, the domain of learning appears to offer less opportunities for participation and agency, while, somewhat incongruously, maintaining a core rationale of ‘student-centred’ design and learner empowerment. In contrast to the tendencies towards ‘personalisation’ in AI, inclusive education offers some potentially different directions for thinking about educational activity. Inclusive education is, on the one hand, driven by the idea of an individuals’ rights to education and development and, on the other hand, also presented as a response to the challenges faced by many educators in different parts of the world, where learning communities have become increasingly diverse and less homogeneous. However, the definition of inclusive education can be highly contested and ambiguous (Slee 2011). The term is often closely associated with the idea of removing barriers to learning for groups who are vulnerable to marginalisation and exclusion and concerns attempts to ensure all learners’ participation regardless of their individual differences (Florian 2008). Importantly, the process of achieving inclusive education is never straightforward and can involve endless negotiations among stakeholders and continual pedagogical decisions within specific educational environments. It is also productive, therefore, to view inclusion as political. In critiquing the broad shifts towards personalised forms of education, Ginsburg questions ‘to what extent attempts to promote active, collective, and democratic forms of citizen participation are possible within the discursive framework of personalisation’ (Ginsburg 2012: p. x). While at a glance, providing individualised support might seem to offer a kind of inclusion. Indeed, special support for those perceived to be different has been the rationale for special education—a more traditional disciplinary response to learning differences. However, recent research on inclusive pedagogy has stressed the limitation of special educational thinking, while arguing for the need to extend what is generally available to all learners (Florian 2008). Interestingly, when we consider the context of (human) teaching and learning in mainstream educational settings, inclusion is approached by developing a ‘common ground’ within educational activities. Rather than solely developing a one-on-one relationship between the teacher and the student, inclusive education attempts to generate the many-to-many kind of communal dialogue that authentically fosters equality within a group of learners. Inclusion, in this sense, is about staying ‘within the trouble’, and perhaps the ‘messiness’, of difference and diversity in educational settings, and viewing such a balancing act as a pedagogical ideal, rather than as a practice of excess and superfluity to be excised

Introduction: AI, Inclusion, and ‘Everyone Learning Everything’

7

through efficiency measures. At the heart of inclusive pedagogy, therefore, is a notion of what is valued by the community, rather than a focus on individual achievement. For UNESCO, while an economic rationale for education is nevertheless apparent, inclusive practice ‘goes beyond a utilitarian approach to education and integrates the multiple dimensions of human existence’ (UNESCO 2015: 26). This broad and idealistic view of inclusive education has tended to eschew specific engagement with questions around the use of technology. However, research and development in assistive technology has long sought to provide specialist support, and AIEd itself is often suggested to be not just more personalised, but also more inclusive and engaging. For example, they can provide additional help for learners with special educational needs, motivate learners who cannot attend school, and support disadvantaged populations. (Luckin et al. 2016: 30)

As Anna Wood’s chapter demonstrates with respect to chronic illnesses, this approach has been productive and helpful for many in terms of assisting with everyday working practices. However, limitations are also encountered, as reflected by the child participants in Sophia Serholt’s chapter, when the presence of technology is perceived as an all-encompassing solution without considering its dynamic, interactive, and agential role within educational environments. As pointed out by several authors in this volume, to make AI technologies work for inclusive education, it is essential to re-examine what data are being gathered for the training of such systems: Are the data inclusive enough to represent all groups of learners? However, one should also view the challenge of inclusive education in terms of curricula, not just the application of technology, which may only amplify learning material that is outdated or biased. Further, AI technologies are put to work within wider, and inevitably exclusionary educational systems driven by an agenda of performance (Wang 2016), and such conditions would only be intensified by data-driven technologies. In other words, the challenge of inclusion lies within society and cannot necessarily be ‘fixed’ with a technical solution. The usage of AI may instead aggravate other forms of injustice. For example, wealthy learners might take advantage of the ability to purchase AI tutoring services from private developers, and regions with more advanced digital infrastructure, or learners already equipped with better digital skills, might also benefit more from the availability of such technologies. Michael Gallagher’s chapter discusses this ‘Matthew effect’ in the context of the mobile technologies increasingly involved in AI. We suggest that one of the key purposes of this book is to engage researchers and teachers who tend to consider their work with inclusion to be exclusively about human relationships, within physical classrooms. We hope that the following chapters offer critical views of AI that counter the assumption that one either engages with technology, or not, or that technology is simply an ‘add-on’ to core humanistic pedagogies. As readers will encounter in the following work, the issues surfaced by ever more pervasive AI systems in education have profound consequences for all kinds of educational endeavours and particularly for those concerned with understanding how commonality, inclusion, and exclusion manifest through the structural arrangements of pedagogy. We invite you to join this very important conversation

8

J. Knox et al.

regarding the future of education and continue to discuss the remaining unspoken issues beyond the scope of this volume.

3 Artificial Intelligence and Inclusive Education—Speculative Futures and Emerging Practices The various contributions in this book offer a rich view, not only of research and development that attempts to engage with the intricate intersection of ideas across ‘AI’ and ‘inclusion’, but also of the terms themselves. As editors, we chose not to prescribe the use of these terms too rigidly, with the idea that authors might surface a range of perspectives and understandings, linked to the specific contexts of their research and practice. This final part of the introduction will outline the various ways ‘AI’ and ‘inclusion’ have been interpreted throughout the book, as a way of summarising the contributions made by each chapter. Firstly, this book began with the understanding that the term ‘AI’ serves as a fairly loose umbrella term for a wide range of concepts, practices, and technologies. This has been particularly productive, as the authors were able to bring together and demonstrate a broad range of educational work with AI and offer informed speculation on critical issues and future developments. The chapters from Phaedra S. Mohammed and Nell Watson, and Gunay Kazimzade, Yasmin Patzer, and Niels Pinkwart provide useful general overviews of a range of AI for education approaches and technologies. The former focuses on ‘intelligent learning environments’ (ILEs), while the latter links more established work in assistive technology with emerging AI developments. Maggi Savin-Baden, Roy Bhakta, Victoria Mason-Robbie, and David Burden offer a more focused study, examining and measuring the effectiveness of ‘pedagogical agents’ capable of assisting with mathematics tuition. Ka´ska Porayska-Pomsta and Gnanathusharan Rajendran describe ‘AI agents’, in one case specifically developed to teach children with autism spectrum disorders, in which a human teacher’s ordinary teaching method might be considered less effective. Anna Wood’s chapter reflects on technology related to assisting those with chronic illness and outlines a number of real and speculative AI technologies capable of responding to a range of needs. AI is also often associated with hardware, and the chapters from Sofia Serholt and Seungcheol Austin Lee and Yuhua (Jake) Liang offer their perspectives on that most familiar of AI manifestations, the robot. Theoretical discussions of AI are also represented. Michael Rowe focuses on ‘algorithms’, which, as an important technical component of machine learning AI approaches, offers an insightful way of understanding the socio-technical relations that connect educational activity to technological production. Alex Guilherme considers ‘intelligent tutoring systems’ and relates this educational concern with broader debates around so-called weak and strong AI. Rather than discussing specific AI technology, Michael Gallagher analyses the ‘mobile ecosystems’ in which machine learning increasingly functions and through which issues of accumulated advantage

Introduction: AI, Inclusion, and ‘Everyone Learning Everything’

9

take place, offering important speculation on future inclusions and exclusions in education. The final chapter from Michael A. Peters and Petar Jandri´c presents further speculation on the future of AI, and its relationship with ideas of human evolution, and the development of ‘algorithmic non-carbon-based ‘living’ systems’. Secondly, this book began with a broad view of inclusion—a notion of quality education for everyone regardless of individual differences—in order to encourage authors’ creative and critical engagement with these ideas. In this final section of the introduction, we see value in clarifying, from our editorial perspective, how each chapter in turn has interpreted and engaged with the idea of inclusion in the context of AI technologies. Phaedra S. Mohammed’s and Nell Watson’s chapter, Towards Inclusive Education in the Age of Artificial Intelligence: Perspectives, Challenges, and Opportunities, makes an important case for considering the cultural inclusions and exclusions related to various AI systems and technologies. This chapter calls for an interdisciplinary approach, incorporating insights from cultural anthropology, sociocultural linguistics, and educational psychology, to broaden the understanding of the specific contexts into which AI is applied. Ka´ska Porayska-Pomsta and Gnanathusharan Rajendran’s chapter—Accountability in human and artificial decision-making as the basis for diversity and educational inclusion—adopts AI as a conceptual framework to rethink learning and inclusion. They provide examples of AI software for contexts in which human teaching might be viewed as challenging, if not a hindrance and a limitation: the teaching of children with autism spectrum disorders. Here, the replacement of the human teacher might be seen as a necessity, rather than an efficiency measure. However, Porayska-Pomsta’s and Rajendran’s chapter also highlights the issue of accountability in AI, which frames inclusion in terms of the accessibility of the technology and calls for more human agency in developing socially just and inclusive technologies. This is a key aspect of the intersection we attempted to explore in this volume and is also discussed in Michael Rowe’s chapter (see below). Gunay Kazimzade, Yasmin Patzer, and Niels Pinkwart’s chapter—Artificial Intelligence in Education meets inclusive educational technology: the technical state-ofart and possible directions—includes a wide focus on various inclusion issues, such as the way specific AI systems exclude those with physical disabilities and examples of cultural exclusions. They ask critical questions about what kind of data models underpin AI systems, for instance, data from users with impairments that might appear to be ‘irregular’ and thus likely to be dismissed. This highlights the extent to which inclusion issues can be addressed through additions and refinements in the development of AI itself. Sofia Serholt’s work with robots, discussed in her chapter, Interactions with an Empathic Robot Tutor in Education: Students’ Perceptions Three Years Later, offers the interesting possibility of moving away from the more typical uses of AI for performative assessments of individual students. This chapter provides a glimpse of AI technology utilised for the social and relational aspects of pedagogy that are the hallmarks of inclusion. One might usefully read Michael Rowe’s and Alex Guilherme’s

10

J. Knox et al.

chapters alongside this work to explore possible limitations to these ideas, however. Particularly valuable in Serholt’s chapter are the insights from children related to the ways robots are perceived by those on the receiving end and how special assistance through technology may generate experiences of isolation if used without the consideration of group dynamics in a classroom. Seungcheol Austin Lee and Yuhua (Jake) Liang make useful connections between inclusive practice and the notion of ‘trust’. This chapter, entitled A Communication Model of Human-Robot Trust Development for Inclusive Education, offers interesting ways for AI to develop towards classroom presence. It considers the reality of interactions that take place in classroom settings and offers ideas about how one could possibly make better use of AI in daily practice for educational ends. In addition, the chapter also points out how certain groups of learners might be disadvantaged in the era of AI, resulting from apprehension towards technology development. In their chapter, An evaluation of the effectiveness of using pedagogical agents for teaching in inclusive ways, Maggi Savin-Baden, Roy Bhakta, Victoria MasonRobbie, and David Burden examine the ways specific AI technology can support diverse learners, focusing on the idea of personalised pedagogy. This chapter makes a case for the accuracy of technology in teaching scenarios and calls into question the assumption that a human teacher is always superior. Importantly, inclusion issues are identified in the training of AI systems through the use of specific datasets, highlighting the need to take teacher education seriously, whether involving humans or machines. This chapter also emphasises the benefits resulting from learners reflecting their own learning strategies through AI software and the potential advantages for increased engagement with education as an outcome. Anna Wood’s chapter, Inclusive Education for Students with Chronic Illness—Technological Challenges and Opportunities, focuses on one group of atypical learners as a key example of the potential liberating, but also limiting, capacities of AI technologies. This specific view of the particular needs of those with chronic illnesses highlights the complex barriers to participating in everyday work-based activities, often involving individual’s physical bodies. Importantly, this chapter connects specific inclusion concerns to a broader political sphere, through which access to resources is governed. Michael Rowe’s chapter, Shaping our algorithms before they shape us, foregrounds an important call for the development of AI technology that is inclusive, in particular, to involve teachers as key agents in the decision-making process. This is a valuable critical interpretation of inclusion that offers an alternative to the dominance of the tech industry in the production of educational technology. For Rowe, human teachers provide a crucial means of contesting the deterministic outcomes of algorithmic decision-making. Moreover, the critical discussion of the instrumentality of algorithms in this chapter also highlights the increasing datafication and performativity of the sector, within which professional teachers, and students, are both reduced to ‘cheerful robots’ (Giroux 2011: 3). Ultimately, Rowe’s framing of AI in education as a social and pedagogical problem, rather than simply a technical one, should resonate with educators concerned with instilling inclusive practices and resisting neoliberal models of the institution.

Introduction: AI, Inclusion, and ‘Everyone Learning Everything’

11

Alex Guilherme’s chapter, Considering AI in Education: Erziehung but never Bildung, focuses on the relationships between people in educational activity, and this aligns with key practical aspects of the inclusive education agenda. Rather than assuming technology as a quick fix for educational dilemmas, this chapter asks important questions about future directions for inclusive education development—What will happen when technology potentially disrupts our (human) relationships within an educational community? It is thus very ironic, given that relationships are something very important in education, that the impact of the technologisation of education and its potential depersonalisation of the classroom is not discussed in more detail and philosophically questioned. (p. 154)

Guilherme’s call for ‘real dialogue’ (p. 157) with students, but also an education that moves beyond the acquisition of skills, surfaces important social and political dimensions of inclusive practice, and highlights important aspects of learning to live collectively through education. Michael Gallagher’s chapter, Artificial intelligence and the mobilities of inclusion: the accumulated advantages of 5G networks, machine learning, and surfacing outliers, highlights the accessibility of technology, surfacing important questions about those with and without the skills to make use of it effectively. For Gallagher, such issues must be addressed before the benefits of deployed AI can be realised. This chapter also questions reductionist views of education engineered through AI, ‘as curricula [are] being aligned to largely derivative computational models of learning’ (p. 171). The work of inclusion instead requires us to grapple with the complexity of educational process when diverse learners learn together. Embracing the speculative approach suggested by this book, Michael A. Peters and Petar Jandri´c present some useful alternatives to established notions of inclusion. The chapter, Artificial Intelligence, Human Evolution, and the Speed of Learning, bypasses the humanism that governs most understanding of inclusive education, and poses intriguing questions about how we might categorise both humans and machines in the future. As Peters and Jandri´c discuss, where AI development and human genetic research are increasingly interconnected, the boundaries become increasingly blurred, and the concept of educational equality may need considerable reinvention.

References Alpaydin, E. (2016). Machine learning: the new AI. Cambridge: MIT Press. Biesta, G. (2005). Against learning. Reclaiming a language for education in an age of Learning. Nordisk Pedagogik, 25(1), 54–66. Biesta, G. (2006). Beyond learning. Democratic education for a human future. Boulder, CO: Paradigm Publishers. Biesta, G. (2012). Giving teaching back to education: Responding to the disappearance of the teacher. Phenomenology & Practice, 6(2), 35–49. Edyburn, D., Higgins, K., & Boone, R. (2005). Handbook of special education technology research and practice. Oviedo: Knowledge By Design, Inc.

12

J. Knox et al.

Florian, L. (2008). Special or inclusive education: Future trends. British Journal of Special Education, 35(4), 202–208. Friesen, N. (forthcoming 2019). The technological imaginary in education, or: Myth and enlightenment in “Personalized Learning”. In M. Stocchetti (Ed.), The digital age and its discontents. University of Helsinki Press. Available: https://www.academia.edu/37960891/The_Technological_ Imaginary_in_Education_or_Myth_and_Enlightenment_in_Personalized_Learning_. Ginsburg, M. (2012). Personalisation is political, but what kind of politics? In M. E. Mincu (Ed.), Personalisation of education in contexts: Policy critique and theories of personal improvement, foreword. Giroux, H. (2011). On critical pedagogy. Continuum. London: The Continuum International Publishing Group Ltd. Herold, B. (2016). Facebook’s Zuckerberg to bet big on personalized learning. Education Week. Available https://www.edweek.org/ew/articles/2016/03/07/facebooks-zuckerberg-to-bet-big-onpersonalized.html. Hill, D. (2016). AI teaching assistant helped students online—and no one knew the difference. Singularity Hub. Available https://singularityhub.com/2016/05/11/ai-teaching-assistant-helpedstudents-online-and-no-one-knew-the-difference/#sm.0001x3wextuewdw0112hpwla2e8bh. Houser, K. (2017).The solution to our education crisis might be AI. Futurism.com. Available: https:// futurism.com/ai-teachers-education-crisis/ Knox, J., Williamson, B., & Bayne, S. (forthcoming 2019) Machine behaviourism: future visions of ‘learnification’ and ‘datafication’ across humans and digital technologies. Learning, media and technology, special issue: Education and technology into the 2020s. Leopold, T. (2016). A secret ops AI aims to save education. Wired. Available https://www.wired. com/2016/12/a-secret-ops-ai-aims-to-save-education/. Luckin, R., Holmes, W., Griffiths, M., & Forcier, L. B. (2016). Intelligence unleashed an argument for AI in education. Pearson Report. Available https://static.googleusercontent.com/media/edu. google.com/en//pdfs/Intelligence-Unleashed-Publication.pdf. Newton, C. (2016). Can AI fix education? We asked Bill Gates. The Verge. https://www.theverge. com/2016/4/25/11492102/bill-gates-interview-education-software-artificial-intelligence. Olson, P. (2018). Building brains: How pearson plans to automate education with AI. Forbes. https:// www.forbes.com/sites/parmyolson/2018/08/29/pearson-education-ai/#47c32cf41833. Slee, R. (2011). The irregular school: Exclusion, schooling, and inclusive education. London: Routledge. UIS. (2016). The world needs almost 69 million new teachers to reach the 2030 education goals UNESCO Institute for Statistics. Available http://uis.unesco.org/sites/default/files/documents/ fs39-the-world-needs-almost-69-million-new-teachers-to-reach-the-2030-education-goals2016-en.pdf. UK Parliament. (2018). October 12th pepper the robot appears before education committee. https://www.parliament.uk/business/committees/committees-a-z/commons-select/educationcommittee/news-parliament-2017/fourth-industrial-revolution-pepper-robot-evidence-17-19/. UNESCO. (2015). Education 2030: Incheon declaration and framework for action for the implementation of sustainable development goal 4: Ensure inclusive and equitable quality education and promote lifelong learning. Available https://unesdoc.unesco.org/ark:/48223/pf0000245656. von Radowitz, J. (2017). Intelligent machines will replace teachers within 10 years, leading public school headteacher predicts. The Independent. Available https://www.independent.co.uk/news/ education/education-news/intelligent-machines-replace-teachers-classroom-10-years-ai-robotssir-anthony-sheldon-wellington-a7939931.html. Wakefield, J. (2018). Robot ‘talks’ to MPs about future of AI in the classroom. BBC News, Technology section. https://www.bbc.co.uk/news/technology-45879961. Wang, Y. (2016). Imagining inclusive schooling: An ethnographic inquiry into disabled children’s learning and participation in regular schools in Shanghai (Ph.d. thesis). University of Edinburgh, Edinburgh.

Introduction: AI, Inclusion, and ‘Everyone Learning Everything’

13

Whittlestone, J., Nyrup, R., Alexandrova, A., Dihal, K., & Cave, S. (2019). Ethical and societal implications of algorithms, data, and artificial intelligence: a roadmap for research. Nuffield report. Available http://www.nuffieldfoundation.org/sites/default/files/files/Ethical-andSocietal-Implications-of-Data-and-AI-report-Nuffield-Foundat.pdf.

Jeremy Knox is Co-Director of the Centre for Research in Digital Education, where he leads the Data Society research theme. He is Lecturer in Digital Education at The University of Edinburgh and, at the time of writing, also a Global Academies and Edinburgh Futures Institute Fellow. His current research is focused on AI, machine learning, algorithms, and data in higher education, with a specific interest in issues related to socio-technical systems, accountability, ethics, justice, and citizenship. He has also worked with critical posthumanism and new materialism and published critical perspectives on Open Educational Resources (OER) and Massive Open Online Courses (MOOCs). He is Associate Editor for new journal Postdigital Science and Education with Springer and serves on the editorial board of Teaching in Higher Education. He has also served as Guest Editor for Learning, Media and Technology and E-Learning and Digital Media. He also co-convenes the Society for Research in Higher Education (SRHE) Digital University network. Yuchen Wang is Research Associate at the Moray House School of Education, The University of Edinburgh. Her research interests include student voice, inclusive practice, disability rights, technology and international development. Her Ph.D. research explored experiences of children with disabilities and teachers’ practices in Chinese mainstream schools, following which she was awarded the UK Economic and Social Research Council Global Challenges Research Fund Postdoctoral Fellowship to build capacity of disability communities, practitioners, and policymakers in the country. She is currently involved in projects that critically examine the relationships between technology and educational inclusion. She also provides consultancy support for NGOs to promote inclusive and quality provision for children with disabilities in China. Michael Gallagher is Lecturer in Digital Education at the Centre for Research in Digital Education at The University of Edinburgh. At Edinburgh, his projects include the Near Future Teaching project; a project exploring formal partnerships with edX around new redesigned master’s programmes; and projects working with universities in Nepal, Nigeria, Tanzania, and Uganda on digital education in developing contexts. His research focuses on educational futures, educational mobility, mobile technology, and the impact on local knowledge practices and communities. He is also Co-Founder and Director of Panoply Digital, an ICT4D consultancy that specialises in educational design for inclusion, particularly in low resource environments and particularly with areas where the gender digital divide is the most pronounced. His projects in this space include ongoing work with the World Bank, USAID, GSMA, UN Habitat, and more.

Part I

Artificial Intelligence and Inclusion— Opening a Dialogue

Towards Inclusive Education in the Age of Artificial Intelligence: Perspectives, Challenges, and Opportunities Phaedra S. Mohammed and Eleanor ‘Nell’ Watson

Abstract In the West and other parts of the world, the ideal of an individualised, personalised education system has become ever more influential in recent times. Over the last 30 years, research has shown how effective, individually tailored approaches can be achieved using artificial intelligence techniques and intelligent learning environments (ILE). As new audiences of learners are exposed daily to ILEs through mobile devices and ubiquitous Internet access, significantly different challenges to the original goal of personalised instruction are presented. In particular, learners have cultural backgrounds and preferences that may not align with most mainstream educational systems. When faced with practical cultural issues, the transfer of successful research and ILEs to underserved contexts has been naturally quite low. This chapter first takes a step back and analyses perspectives on how intelligent learning environments have transitioned from focusing on instructional rigour to focusing more deeply on the learner. Next, it examines some major challenges faced when ILEs aim to integrate culturally sensitive design features. The chapter then discusses several opportunities for dealing with these challenges from novel perspectives such as teacher modelling, the use of educational robots and empathic systems and highlights important concerns such as machine ethics. Keywords Intelligent learning environments · Intelligent tutoring systems · Culturally aware technology-enhanced learning · Enculturated conversational agents · Contextualisation

P. S. Mohammed (B) Department of Computing and Information Technology, The University of the West Indies, St. Augustine, Trinidad and Tobago e-mail: [email protected] E. ‘Nell’ Watson AI & Robotics Faculty, Singularity University, Mountain View, USA e-mail: [email protected] Dean of Cognitive Science, Exosphere Academy, Palhoça, Brazil © Springer Nature Singapore Pte Ltd. 2019 J. Knox et al. (eds.), Artificial Intelligence and Inclusive Education, Perspectives on Rethinking and Reforming Education, https://doi.org/10.1007/978-981-13-8161-4_2

17

18

P. S. Mohammed and E. ‘Nell’ Watson

1 Introduction Advanced personalised learning was defined 10 years ago as one of the 14 engineering grand challenges (Grand Challenges—Advanced Personalised Learning 2018). Personalised learning is essentially learner-centred instruction that has been tailored and paced according to the specific interests, educational goals and personal preferences of an individual learner (Bray and McClaskey 2016). The grand challenges were meant to “raise public awareness of the biggest global issues of our time” (The US National Academy of Engineering 2018). If solved, the outcomes would result in radical improvements in the way that we live, and in the case of advanced personalised learning, how students learn and how they are taught. The interest in this challenge is motivated by the issues faced in traditional classroom settings. In many educational institutions, teachers are typically required to deliver instruction, feedback and attention to a large number of students in a limited time with the goal of students passing assessments with good grades. This is often difficult to achieve because students have varied learning styles, different prior knowledge levels and cognitive skills, diverse emotional dispositions and most importantly distinct learning needs. The fields of artificial intelligence in education (AIED) and intelligent tutoring systems (ITS) have been directly positioned to take on the challenge of advanced personalised learning since their formal inception in the late 1980s. Both research areas aim to provide an intelligent software tutor for every learner. By using interactive software, adaptive learning delivers paced, customised instruction with real-time feedback that allows faster student progression, encourages effective skill development and promotes greater learner engagement with educational content (Six key benefits of adaptive learning 2013). For over the past 20 years, there has been a growing body of positive outcomes reported in the literature where intelligent cognitive tutors perform as well as human tutors (VanLehn 2011) when producing learning gains in students. A variety of applied artificial intelligence (AI) techniques have been commonly employed in achieving these results. The role of AI in shaping education is therefore critical since it has the potential to solve many of the previously mentioned problems (Woodie 2018). Advances in hardware, such as faster graphical processing units (GPUs) and widespread access to machine learning software libraries, have further spurred the use of AI, particularly in deep learning research and the use of data analytics (Janakiram 2018). A larger transformation over the coming years is expected as robots and other new technologies enter the field at a rapid pace where AI is expected to grow by as much as 47.5% by 2021 (Borgadus Cortez 2017). One of the main problems being faced, however, is that of transfer. Many of the intelligent systems and software tutors do not perform as well or as expected in learning environments that differ culturally from the original context of use (Ogan et al. 2015b). In 2009, multicultural understanding was listed as a critical capability for dealing with the grand challenges in order to ensure successful uptake of solutions in intended environments (The US National Academy of Engineering 2018). Smaller, more powerful portable devices combined with ubiquitous Internet access have top-

Towards Inclusive Education in the Age of Artificial …

19

pled the ratio of users from the developing world to almost double those from the developed world in just under 10 years (ICT Facts and Figures 2017). In 2008, the ratio of developed world users to developing world users was approximately 4.2. In 2017, that ratio was 2.0. Moreover, 70% of the world’s youth (aged 15–24) are online and they make up the largest group of Internet users (ibid.). Consider the implications of these statistics when faced with the challenge of building an adaptive, culturally inclusive educational system. Firstly, a lot of data is being generated daily and this will continue to increase. Secondly, not only has the sheer volume of users increased, the cultural backgrounds of these users are being quickly diversified. Thirdly, as the human sources of this data change, so does the quality of the data, and more importantly the cultural bias. The risk of using such data in these ways is that the models are trained to detect patterns that are dictated by the data. For instance, in Donnelly et al. (2016), classroom audio data was used to detect features of teaching events, such as question and answers, that may be beneficial to student learning. A particular style of teaching may or may not be employed in certain cultural environments, and differences may occur even within such environments with temporal factors (Uchidiuno et al. 2018). When data comes from a particular source, naturally those patterns will be skewed towards the environmental conditions of the source environment, and actors and models may or may not work well in alternate cultural conditions (Rudovic et al. 2018). While there is scope for error in these systems, caution is still required when using AI for important tasks especially when dealing with more complex or nuanced interpretations which are positioned in a cultural context that differs from the original developmental setting. The rest of the chapter is structured as follows: the next section gives an overview of the different types of technology-enhanced learning systems in use nowadays that support and have the potential to support inclusive education. Sections follow on perspectives, key developments, breakthroughs and challenges faced in cultural modelling as well as opportunities for leveraging the success of AIED and ITS research.

2 Intelligent Learning Environments Technology-enhanced learning (TEL) helps teachers to do their job more effectively by liberating them from increasing levels of administration and bureaucracy, in the form of marking, planning their lessons, and less important paperwork exercises. By leveraging AI techniques, machines may be able to take over many of these functions, thereby allowing the teacher to focus on their main purpose: to teach and mentor (Ferster 2014). However, the quality and the extent of the adaptive customisations of a TEL environment vary across different implementations. For example, intelligent learning environments (ILEs) are specialised TEL systems that aim to produce interactive and adaptive learning experiences that are customised for a learner using various AI techniques (Brusilovsky 1994). These systems can vary from serious games such as the Tactical Language Training System (John-

20

P. S. Mohammed and E. ‘Nell’ Watson

son et al. 2004) to intelligent tutoring systems such as the ASSISTments platform (Heffernan and Heffernan 2014). Learning management systems (LMSs) on the other hand, simply organise and deliver content and help with course administration (VanLehn 2011). These systems are the most widely used in formal educational institutions. They offer customisable features such as grade books, assignments, quizzes, blogs, wikis, email and forums for an instructor to manage courses and interact with students (Rhoades 2015). Popular open-source examples include Moodle, Sakai, LRN—which has an enterprise-level focus, Schoology—which focuses on kindergarten to K-12 learners, and Docebo—which offers e-commerce features for selling content and focuses on corporate training. Out of any TEL environment, Massive Open Online Courses (MOOCs) reach the largest volume of learners through freely available courses from top-tier universities with large amounts of online content (Rhoades 2015). As much as 81 million learners have signed up for MOOCs (Shah 2018). Successful examples include Udacity, Coursera and edX which all interestingly have a strong AI influence either by way of the content featured or the founders’ research backgrounds.

3 Perspectives Several themes of artificial intelligence (AI) are at the forefront of discussions about the future of education as highlighted by the TEL environments in the previous section. This section takes a step back and examines how the definition of the term AI has evolved in relation to the successes and breakthroughs in the fields of AIED and ITS. This is meant to give an understanding and awareness of how intelligent learning environments have transitioned over years from focusing on instructional rigour to focusing more deeply on the learner.

3.1 Defining Artificial Intelligence A core definition of artificial intelligence is to skilfully imitate human behaviour. Over many years, various definitions have evolved to include more details, specifically related to technology and its applications as shown in Table 1. It can be argued that AI software now exceeds many of the capabilities of a human being due to the sheer volume and variety of computations that can be performed, the incredible speed at which complex decisions can be made and the derivation of new knowledge and detection of trends from vast amounts of data. Despite this, the essential nature of AI’s definition has not changed, and the realisation of the goal of skilfully imitating human behaviour has not been perfectly met.

Towards Inclusive Education in the Age of Artificial …

21

Table 1 Evolution of the term “Artificial Intelligence” Source and year

Definition of artificial intelligence

Dictionary of English language (1979)

None

Merriam-Webster dictionary (1983)

Subheading under artificial: the capability of a machine to imitate intelligent human behaviour

The Concise Oxford Dictionary of Current English (Allen 1991)

Subheading under artificial: the application of computers to areas normally regarded as requiring human intelligence

Cambridge advanced learner’s dictionary (2003)

Full entry: the study of how to produce machines that have some of the qualities that the human mind has, such as the ability to understand language, recognise pictures, solve problems and learn

Online search—English Oxford Living Dictionaries (Aug 2018)

Full entry: the theory and development of computer systems able to perform tasks normally requiring human intelligence, such as visual perception, speech recognition, decision-making and translation between languages

3.2 AIED and ITS Research Just as the definition of AI has evolved, so too have the fields of AIED and ITS. The field of AIED was started in the 1970s (Woolf 2009). The goal was to build intelligent tutors that cared about the learner and met critical emotional, cognitive and psychological aspects of the learning process (Kay and McCalla 2003). Naturally, it was grounded at the intersection of research in computer science, psychology and education (Woolf 2009). ITS research on the other hand focused more on building ILEs that functioned as effectively and efficiently as a human tutor (VanLehn 2011). Significant transitions in the research were observed especially when disruptive technologies were introduced. For instance, in the early years, expert systems and inference rules were commonly used in rigid step-based tutors (VanLehn 2011). Models of domain (expert) knowledge, student mastery of a learning domain and tutoring strategies were (and still are) core components of the ILEs being developed (Dillenbourg 2016). The focus then was on modelling correct instruction, giving immediate feedback and following a particular curriculum. Over time, automatic courseware generation and adaptive hypermedia became the norm especially when the Internet started to be used as a delivery medium through web-based systems. In addition, computer-supported collaborative learning (CSCL) and dialogue systems were actively pursued particularly in feedback systems using intelligent pedagogical agents. Knowledge representation and semantic reasoning were commonplace. A gradual subtle shift away from declarative AI models towards probabilistic models and techniques was observed as uncertainty and flexibility in the

22

P. S. Mohammed and E. ‘Nell’ Watson

learning process were tackled (Dillenbourg 2016). As more learners interacted with ILEs through mobile devices and made use of ubiquitous, cheaper Internet access, data analytics and the global impact of big data started to draw attention. By then, many in-house ILEs were already producing effective learning gains and interest shifted towards testing and deployment in other settings and educational contexts. The advent of MOOCs caused further excitement in being able to reach students from diverse socioeconomic backgrounds and offering large amounts of learning data (Rosé and Ferschke 2016). New technologies such as touch screens, haptic devices and sensors changed the simple data input and output modes of keyboards, mouse clicks and screens towards facilitating richer interactions and engagement. This opened up new avenues for interactivity in classrooms. Nowadays, machine learning and various types of supervised and unsupervised learning techniques are commonly employed in the research reported in the literature. Now, there is a growing body of research that aims to deploy existing ILEs in multiple cultural settings and observe the effects such as in Mavrikis et al. (2018) and Ogan et al. (2015a) or build customised ones capable of dynamically adapting to cultural environments using AI techniques and methods such as in Mohammed (2017).

3.3 CulTEL Research: Strengths and Successes Cultural awareness, when applied to an ILE, refers to the use of culturally relevant data and information to shape the overall appearance, behaviour and presentation of the learning environment (Blanchard and Ogan 2010). The field of culturally aware technology-enhanced learning (culTEL) is relatively young (Rehm 2018), but there have been several important development trends that have contributed towards culturally inclusive educational systems. Culture has been studied extensively from sociological, psychological, anthropological and ethnographic perspectives. Based on these areas of study, formalised, abstract representations of knowledge and concepts that are central to a culture are critical for any system that aims to be culturally aware. Ontologies have been a natural fit for these representations owing to their machine-readable nature and their potential for deriving meaningful conclusions through reasoning (Hitzler et al. 2010). These representations are especially well suited for the knowledge representation and inferencing needs of ILEs (Mizoguchi and Bourdeaux 2016). The more advanced upper ontology of culture (MAUOC) (Blanchard and Mizoguchi 2014) is one of the first heavyweight ontologies that provide a neutral, computational backbone for structuring detailed computational descriptions of culture for use by ILEs. Savard and Mizoguchi (2016) outline another upper-level ontology that defines cultural variables specific to instructional design and pedagogy. Closer to the domain level, Mohammed (2018) describes a trio of ontologies that are derived using MAUOC concepts. These ontologies can be merged to relate conceptual cultural knowledge to sociolinguistic terms that map to the cultural influences that contribute to the background of a student. Lastly, Thakker et al. (2017) present

Towards Inclusive Education in the Age of Artificial …

23

an ontology for representing the cultural variations that occur in interpersonal communication in user-generated content. These efforts touch on diverse areas that allow machine-readable cultural representations to be explicitly accessed and shared across ILEs. Another important success story has been the development of enculturated conversational agents (ECAs). For example, Cassell (2009) studied children’s acceptance, usage and recognition of African American Vernacular English (AAVE) using ECAs. Endrass et al. (2011) describe virtual characters with physical appearances adapted to suit particular cultural backgrounds. In Aylett et al. (2009), the ORIENT system features characters, modelled using agent technology, that exhibit culturally enriched behaviour directed by emotive events. Lugrin et al. (2018) explain that two approaches have been used to build ECAs thus far: data-driven approaches and theory-driven approaches. Theory-driven approaches have classically been used to predict expected human behaviour in particular contexts based on established cultural theories (Rehm 2018). Theoretical models of culture commonly referenced in culTEL research include the works of Hofstede (2001), Hall (1966), Trompenaars and Hampden-Turner (1997), Bennett (1986) and Brown and Levinson (1987). Datadriven approaches are becoming more popular with the availability of machine learning (ML) tools. These approaches tend to rely on large amounts of sample data such as video recordings, audio samples or images for training models that can uncover patterns in new data sets for generalising predictions in a domain of interest such as agent behaviour. A final key development has been the use of weighted approaches for approximating cultural values. Weights have been used to prioritise values assigned to particular cultures based on theoretical models (De Jong and Warmelink 2017; Mascarenas et al. 2016). Mohammed (2017) also used a weighted, theory-driven approach to model the sociocultural and demographic influences that contribute to a student’s cultural affinity or ignorance of linguistic terms native to a local language. Nouri et al. (2017) show that these approaches are useful when there is no data on which to base the weights and demonstrate that the weights can be learnt from data using ML techniques resulting in more accurate models of cultural decision-making. Rather than assigning blanket cultural values to a student, weights allow a more nuanced approach towards the modelling of subcultures. This is essential in ILEs especially when students have multiple cultural identities.

4 Challenges Blanchard (2015) observes that research in the ITS and AIED fields has been strongly biased towards Western, educated, industrialised, rich and democratic (WEIRD) nations (Henrich et al. 2010). He reports that 82–95% of the research published over 10 years (2002–2013) has been dominated by researchers from WEIRD nations with clear cultural imbalances with respect to the data sets used in models and systems, and sociocultural group representations. Furthermore, the kinds of research being

24

P. S. Mohammed and E. ‘Nell’ Watson

conducted do not tend to focus on cultural contexts and practical ILE issues of application and deployment (Mohammed and Mohan 2013; Nye 2015). It makes sense, therefore, that much of the successes of ILEs reported for a WEIRD context are not often transferable to other cultural contexts (Ogan et al. 2015b). Despite these observations, there is growing awareness of the need for culturally inclusive research within the AIED and ITS communities (Nye 2015; Pinkwart 2016; Roll and Wylie 2016). There are many challenges that need to be overcome before culturally inclusivity becomes a mainstream consideration in TEL software systems and this section examines some of these key issues.

4.1 Cultural Granularity Many TEL projects use Hofstede’s (2001) national indices as a measurement of the cultural influences on students in order to adapt system appearance and behaviour, and select and modify educational content. This is a common starting point, and it has yielded positive results in some areas of research particularly those with enculturated agents (Endrass et al. 2011; Mascarenas et al. 2016; Rehm 2018). New challenges are being observed with country-level categorisations, however, such as the broadness of the scope and generic categories for typifying students who happen to have some interaction with a particular country. For example, mismatches have been documented with the expectations of student preferences based on Hofstede’s values for a country and the actual preferences of the students from that country (Chandramouli et al. 2008). This means that the level of cultural granularity provided by broad cultural models is not typically designed with computational applications in mind (see Blanchard et al. 2013 for an overview) and may be too high level to meet the requirements for culTEL environments. Finer-grained measurements sensitive to the subtle but critical differences across student cultural backgrounds caused by their differing degrees of membership to cultural groups are required for educational applications (Mohammed 2017). Furthermore, the reliance on the values assigned at a countrywide or national level can introduce potential flaws if the values were collected from small or biased samples. Related to this issue is the granularity of cultural data that can be collected from users. Studies have shown that various important pieces of information are required from users (Mohammed and Mohan 2013) but cannot be collected uniformly across countries (Blanchard 2012). The multi-layered identity of an individual presents a significant challenge when attempting to model the cultural contextual backgrounds of students. Globalisation, the redefinition of national identities (Sharifian 2003) and the multiplicity of cultural influences (Rehm 2010) add even more complexity to the problem. Country-level categorisations are commonly used to profile students as evidenced by the heavy use of Hofstede’s (2001) national indices but these are not at a deep enough level of granularity to differentiate between the layers of a student’s cultural identity. In the absence of ontological models of culture, it is, therefore, difficult to modify

Towards Inclusive Education in the Age of Artificial …

25

cultural features in an ILE without requiring extensive recoding and modification of the underlying system as is the case for many existing ILEs.

4.2 Cultural Bias The most common challenge is guarding against cultural bias (Blanchard 2015; Mohammed and Mohan 2013; Rehm 2010). Cultural bias stems from subjective, personal descriptions and perceptions, also called folk approaches. When these narrow interpretations are used to determine cultural features, it introduces problems from conceptual and developmental perspectives. From a conceptual standpoint, folk approaches lack neutral abstractions that can be generalised and reused in different features of culTEL. This is a critical flaw because it works against the goal of “coherent global views of the cultural domain” identified by Blanchard and Mizoguchi (2014) and prevents the interoperability and standardisation of cultural representations. From a developmental standpoint, folk approaches make developer bias more difficult to deal with. This kind of bias occurs when developers knowingly or unknowingly skew the design and software architecture of TEL environments towards their own preferences and instincts which would be most certainly influenced by their cultural backgrounds. For situations where the developer is native to the target culture, the consequences may not be so severe. However, when there is a serious mismatch, a culTEL environment can be seriously affected as discussed in Rehm (2010), to the point of being irrelevant and even offensive. Cultural bias can also occur in the data sets that are used to train models. For example, Rudovic et al. (2018) reported that the training sets used to train models that detected affect in autistic children were biased towards physiological features linked to cultural expressions of engagement. Using country-level generalisations for detection of affect and engagement, therefore, presented challenges in detecting a particular emotion cross-culturally which required retraining of the models.

4.3 Cultural Realism In the past, shallow cosmetic applications of cultural symbols have been criticised as tokenistic and stereotypical in TEL. Early attempts focused on digital images or languages used in web-based systems. Simple approaches were common because of the lack of computational models that could facilitate deep cultural modelling. Simple, culturally grounded features can, however, have a significant impact on user acceptance and attitudes. McLoughlin and Oliver (2000) describe an online environment for Australian indigenous learners with the intention of a culturally responsive design. The system used cultural dimensions in the design of authentic learning activities which were supported by different online tools and offline groups. In Robbins (2006), a more detailed online system is presented which targets students from

26

P. S. Mohammed and E. ‘Nell’ Watson

the South Pacific. Dialogic contextualisation was used which framed educational multimedia content as personified interactions or static conversational items. The system featured semiotic contextual elements heavily and stressed the importance of conversations in establishing rapport with students. Since then, the cultural realism of TEL systems has advanced greatly particularly when ITSs or serious games are used. On the lower end of the cultural realism scale, examples such as the ActiveMath system (Melis et al. 2011), the Assistment plugin described in Vartak et al. (2008), the CAWAS platform (Chandramouli et al. 2008) and the AdaptWeb system (Gasparini et al. 2011) incorporate simple cultural elements into system content and behaviour. On the higher end of the cultural realism scale, examples such as the Tactical Language Training System (TLTS) (Johnson et al. 2004) and the CRITS system (Mohammed and Mohan 2015) adapt and change system responses to suit user language preferences. Both the TLTS and the AAVE tutors (Finkelstein et al. 2013) present virtual agents that resemble users in cultural appearance. Cultural realism is also particularly observable when virtual agents are used as in the CUBE-G Project (Rehm et al. 2007). ILE appearance changes are more prevalent in systems with virtual characters and enculturated conversational agents (ECAs). Endrass et al. (2011) describe virtual characters with physical appearances adapted to suit particular cultural backgrounds. In the ORIENT system (Aylett et al. 2009), cultural characters are modelled using agent technology and demonstrate culturally enriched behaviour directed by emotional events. An interesting observation in the literature is that learners opt to design pedagogical avatars that look like themselves (Allessio et al. 2018). Exploration of how learners respond to pedagogical agents that resemble their physical appearance was conducted in another study (Wang et al. 2018). Early results indicate that while this has no effect on learning, it is the first study of its kind to test this type of hypothesis since the technology did not exist prior. Either way, more research is needed regarding how best to design pedagogical agents that have character appearances within an acceptable range of cultural realism and how to define appropriate behaviours that will promote learning.

5 Opportunities Many of the challenges described in the previous section are difficult to address because they relate directly to cultural modelling. This section describes several promising avenues that have the potential to break down the complexity of these challenges by tackling contributing factors. Areas such as teacher modelling and multimodal interaction, empathic systems and the use of educational robots offer opportunities for untapped research when dealing with cultural diversity. A discussion of ethical concerns rounds off this section since social interactions and research that involve young learners and technology need to be prioritised along the lines of good conduct and moral decisions that benefit the learner.

Towards Inclusive Education in the Age of Artificial …

27

5.1 Teacher Modelling We must recognise that it is unlikely and probably undesirable that intelligent systems, robots and machines might entirely replace teachers in the future. Teachers and students together form part of a broader learning culture. AI machines can improve the learning process (and may do so significantly for many students), yet they are no substitute for a shared culture of learning where an inspirational teacher and a class of engaged pupils work together to explore and learn. AI, however, can be used to help teachers reflect on and improve the effectiveness of their instructional activities in classrooms. For example, Donnelly et al. (2016) used Naïve Bayes classifiers to automatically detect five instructional activities used by teachers simply using audio data of the teachers’ speech collected in live classrooms. This type of research essentially confirms the advent of teacher modelling where successful strategies can be identified, encoded and used to develop cognitive tutors patterned by best practices from teachers in actual classrooms. Another example described in Holstein et al. (2018), used real-time analytics and live feeds of student activities involving the use of an ILE for learning math. Feedback on student performance was synchronously delivered to a teacher through a mixed-reality glasses connected to the ILE. The experiment reported significant increases in learner gains when the teacher used the glasses to assist the students compared to when the students simply used the ILE without the analytics-led interventions. In these classroom contexts, the assessment of a student’s learning process was done instantly, giving the teacher access to rapid results on the student’s progress. This means that the teacher does not have to wait for the test results before making decisions on a learning plan. Adjustments can be immediate and rapidly iterative. In both cases, the focus is clearly on assisting the teacher. In order to have an authentic representation of what works, why it works and why things are the way they are in classrooms, we need to model not only students but teachers as well. Most times when a student uses an e-learning system, we get only one side of the equation: the digital traces of how the student uses the system. A deeper question could be: Why does the student use a system this way? Personal choices and learning preferences account for part of the answer. The other part is that students have been trained to learn a particular way by the teachers they encounter in their classrooms and schools. The way that these teachers instruct largely shapes how students learn, and the ways of instruction are heavily culturally dependent (Savard and Mizoguchi 2016). Therefore, we need to have some insight into the cultural factors engrained in teachers as well as students and whether they match, differ or complement each other. This then needs to be factored into the learning systems that are being built and used.

28

P. S. Mohammed and E. ‘Nell’ Watson

5.2 Multimodal Interactions The embodiment of an ILE is extremely important. Tims (2016) explains that ILEs have been using hardware and platform technologies that were designed for business settings rather than for education. Sensing technology, ambient classroom tools and educational robots introduce interesting dynamics into the learning environment by increasing the levels of interactivity, engagement and feedback about the learning process for both students and teachers. These modes offer untapped potential for reaching students in ways that classic ILEs have not been able to. For instance, Alavi and Dillenbourg (2012) describe the Lantern project where an ambient light placed in a classroom setting was used to signal help-seeking behaviour details to teaching assistants such as whether a group required assistance, when they last requested help and which problems they were working on. Experiments showed that the light encouraged more frequent problem-solving discussions amongst students in collaborative groups especially while waiting for help. ILEs complemented with sensing technology currently produce as good learning gains as the expertise of traditional intelligent systems and in some cases exceed the benchmarks. Gaze tracking technology was shown to provide feedback to a learner that was a good as having a pedagogical conversational agent (Hayashi 2018). Realtime analytics using mixed-reality tools such as the Lumilo glasses project (Holstein et al. 2018) augment and extend the reach of what a teacher can glean from sitting behind a desktop computer and interacting with an ILE console. Being able to assist struggling students in real-time essentially personifies what happens with expert teachers, and the technology is now catching up to promoting that ideal. A common problem with MOOCs identified in the literature is the issue of low engagement (Rhoads 2015). Pham and Wang (2018) describe an interesting approach in the AttentiveLearner2 system that uses the back camera of a mobile phone to detect physiological changes in the blood levels of a learner’s fingertip (holding the phone) and the front camera to detect affective engagement based on facial expressions. They show that these simple sources of data were useful for detecting six emotional states with a high level of accuracy and offer more informative data than the click-stream analytics currently available in MOOCs.

5.3 Educational Robots and Empathic Systems Empathy, in its straightforward dictionary definition, is usually described as: “the ability to understand and share the feelings of another” (Empathy 2018). However, empathy is more complex than this straightforward understanding. Dial (2018) points out, that we are mainly social animals and, therefore, empathy is considered to be a fundamental or desirable trait in human beings. It facilitates positive human interaction and can be considered as a cornerstone of our intelligence alongside logic, cognition and emotion (Dial 2018).

Towards Inclusive Education in the Age of Artificial …

29

Robots can sometimes be described as “empathic”, despite obviously not being able to accurately model human emotions, nor possessing any internal affective qualia or felt senses. Making a machine appear to be empathic through encoding is a complex task. Yet when we consider the potential value of such robots to the development of young children, as well as adults, it is vital that machines of learning are empathic as they can also encourage children to adopt positive behavioural characteristics. It is a good match, therefore, that one of the main goals of AIED is to build systems that care about a learner (DuBoulay et al. 2010; Kay and McCalla 2003). Empathic ILEs are currently able to interact with learners, infer and recognise (through speech, facial and gesture recognition) the emotions and feelings of the learner, draw on data models of appropriate interventions and then react accordingly. The AttentiveLearner2 system (Pham and Wang 2018) described in the previous section is a key example of this. Affective technology has been developed extensively in software systems in AIED and ITS research but is at a relatively early stage of development in robotic and embodied systems. Many of the intended users of educational robots tend to be young children and K-12 students. For example, a trial in operation in Finland uses robots to teach language and math by encouraging students to code the robots (Finland Schools 2018). In particular, children are the target users because they respond positively to these types of systems due to anthropomorphised characteristics of “feelings” and “eyes”, as well as the ability to express emotions (Druga et al. 2017; Johnson and Lester 2016; Tims 2016). Another key finding is that children tend to respond to devices that appear and behave human-like (Druga 2018). In Druga et al. (2017), the voice and tone of various devices such as Alexa (a voice-controlled, virtual assistant), Google Home (a speaker and voice assistant), Cozmo (a robot toy) and Julie (a chatbot) affected the extent to which children wanted to interact with the machines. For instance, Julie the chatbot was seen as friendly and having “feelings” compared with Google Home’s emphasis on knowledge. Two boys described Cozmo in a positive light due to its humanised characteristics (physical features, such as eyes in particular), as well as its ability to express apparent affect such as happiness or anger. When describing the other devices, the children remarked: “they didn’t have eyes, they didn’t have arms, they didn’t have a head, it was just like a flat cylinder” (ibid). Work by Yadollahi et al. (2018) also confirms that children treated a reading robot with empathy. The children were required to correct the robot when they detected reading mistakes, and they did so with explicit care and concern for the robot’s feelings even though they understood that the robot was inanimate. Robots, in particular, due to their embodied form, offer the potential to interact with children who have special needs. For instance, young people with autism will often find it difficult to interact with other human beings. In Luxemburg, LuxAI1 has developed a social robot named QTrobot for research purposes, but also for children with autism. The robots tend to make the children feel less anxious than they might around other humans, and it was noted that children flapped their hands (a common manifestation of autism) less frequently in the company of the robot. The robot operated alongside a human therapist to interact with the child in a relaxed environment. 1 http://luxai.com/.

30

P. S. Mohammed and E. ‘Nell’ Watson

It essentially allowed the therapist to form a more rewarding relationship with the autistic child. The research also made a further discovery: the embodied robot was more effective than an app or a tablet in helping the child to learn (Biggs 2018). Empathic robots may, therefore, be more effective when it comes to bonding with the child through “features such as shared gaze, synchronization of gestures and sensitivity to certain movements on the side of the human” (Castellano et al. 2013). These experiments have now proved that embodied empathic robots are not only contributing to the individualisation of the learning process, but also have the potential to take the fear that some children may feel during that process, turning it into engagement and enjoyment (Finland Schools 2018). The implications of these interactions, however, require careful design since certain gestures can be interpreted with different emotional outcomes in certain cultures (Rehm 2018). It is also important to explore how student behaviours can be influenced by particular interventions using empathic, educational robots and systems.

5.4 Ethical Issues Unlike adults, who tend to show a higher degree of caution when interacting with robots, young children are more susceptible to their influences. They tend to regard robots as psychologically sound, moral beings that can offer friendship, trust and comfort (Kahn et al. 2012). Research has confirmed the results of earlier work that children between the ages of 4–10 see robots as trustworthy (Williams et al. 2018). Further research was conducted to find out whether the “moral judgements” and “conformity behaviours” of children from this age group might be directly influenced by AI machine toys. A talking doll was discovered to wield influence and could persuade children to alter their moral judgements, but it was not able to make the children disobey an instruction (Williams et al. 2018). Johnson and Lester (2016) further confirm that after 25 years of research experience that pedagogical agents are especially more effective at promoting learning for K-12 students rather than post-secondary ones. The clear ethical danger here is that empathic robots could be set up in order to benefit the manufacturer or other interested parties more than the personal development of minors. The potential for emotional manipulation or “nudging” that could infringe the personal liberty of an oblivious child is substantial. In a similar manner to other forms of media, it is likely that many commercial robots will try to sell certain brands to the child or even exercise the development of political and social outlooks through subtle propaganda. There are also potential privacy and child safety issues. The AI machines may draw on a range of intimate details about the child’s cognitive, physical and emotional state through a sophisticated system of audio, touch and biometric sensors. Robots may also gather details about the child’s home environment. If this data is well protected and controlled this may be palatable; however, many of these devices may have unsecured online links, specifying the locations and personal activities of

Towards Inclusive Education in the Age of Artificial …

31

children. This also raises the issue of transparency when it comes to the data that a given robot may gather in its interactions with the child. It may be that the robot is set up to share information with third parties about those interactions (The IEEE Global Initiative on Ethics of Autonomous and Intelligent Systems 2018; Wang 2018). Governments need to either prompt or take on board examinations of the ethics around empathic robots and their potential inculcation of children. This is already being done in some cases, such as the European Parliament (Rise of the Robots 2017). It is likely that recommendations and regulations will be introduced to protect both adults and vulnerable children from manipulation. For example, the IEEE Global Initiative points to several measures including the ability to differentiate between various nudges, including those aimed at more social goals, for example, healthy lifestyle choices or manipulation to promote the sale of products (The IEEE Global Initiative on Ethics of Autonomous and Intelligent Systems 2018). They propose other measures, such as “an opt-in system policy with explicit consent” or where children are unable to give their consent other safeguards may be necessary. The domain of machine ethics (i.e. the actual process of loading values into machines) is building momentum. Practical efforts at collecting information about the social values of communities such as MIT’s Moral Machine2 and the EthicNet3 data sets of pro-social behaviour, promise to provide practical solutions for instructing machines on the nature of social interactions which best suit the cultural and individual preferences for given communities and their members.

6 Conclusion In conclusion, it is clear that the development of AI in education is rapidly changing conventional thinking about teaching and learning. Traditional models of schools and classrooms are likely to see dramatic changes over the coming years and decades as technological advances filter down into educational institutions. For many years, AIED and ITS systems have been shown to live up to the potential of tailoring learning to the specific needs of individual students along cognitive, emotional and instructional dimensions (Baker 2016). Technology-enhanced learning environments such as content management systems, though not as sophisticated as the systems currently under development in research, can also liberate teachers from bureaucracy, so that they can concentrate on working with machines to facilitate the progress of each child. MOOCs should be considered not simply as courses but as vessels for interactive textbooks (Rosé and Ferschke 2016) where text-based adaptations are driven by the cultural factors important to students, and dictated by students (Mohammed 2017). The effectiveness of educational robots will be improved over time, although it is important that regulation is introduced to ensure that they properly serve their purpose from ethical perspectives. Irrespective of the type of technology-enhanced 2 http://moralmachine.mit.edu/. 3 https://www.ethicsnet.com/.

32

P. S. Mohammed and E. ‘Nell’ Watson

learning environment being used, the cultural factors that govern the presentation, delivery and customisation of content and system behaviour need to be formalised using neutral, well-designed AI techniques. Furthermore, the issues in education that occur in the developed world manifest differently in the developing world. Key problems should be approached with the limitations of sparse and incomplete data sets, low resource settings and lack of technical skills at the forefront of any AI technique or approach for successful uptake in developing world contexts (De-Arteaga et al. 2018). When novel, innovative technologies converge with new paradigms of instruction, pivotal points occur that drive radical changes in intelligent educational research (Dillenbourg 2016). The pivotal point at this time is the need for culturally appropriate inclusive education, and the age of AI is here to facilitate it.

References Alavi, H. S., & Dillenbourg, P. (2012). An ambient awareness tool for supporting supervised collaborative problem solving. IEEE Transactions on Learning Technologies, 5(3), 264–274. Allen, R. E. (1991). The Concise Oxford Dictionary of Current English. Oxford: Clarendon Press. Allessio, D., Woolf, B., Wixon, N., et al. (2018). Ella me ayudó (she helped me): Supporting Hispanic and English language learners in a Math ITS. In C. Penstein Rosé, et al. (Eds.), Lecture Notes in Computer Science: Vol. 10948, Artificial intelligence in education. AIED 2018. Cham: Springer. English Oxford Living Dictionaries (2018). Retrieved from https://en.oxforddictionaries.com/ definition/artificial_intelligence. Aylett, R., Paiva, A., Vannini, N., et al. (2009). But that was in another country: Agents and intercultural empathy. In: K. S. Decker, J. S. Sichman, C. Sierra, et al. (Eds.), Proceedings of 8th International Conference on Autonomous Agents and Multi Agent Systems, Budapest, Hungary, 10–15 May 2009 (pp. 329–336). Richland, SC: International Foundation for Autonomous Agents and Multiagent Systems. Baker, R. S. (2016). Stupid tutoring systems, intelligent humans. International Journal of Artificial Intelligence in Education, 26(2), 600–614. Bennett, M. J. (1986). A development approach to training for intercultural sensitivity. International Journal of Intercultural Relations, 10, 179–196. Biggs, J. (2018). This happy robot helps kids with autism, Techcrunch. https://techcrunch.com/2018/ 08/13/this-happy-robot-helps-kids-with-autism/. Accessed October 22, 2018. Blanchard, E. G. (2012). Is it adequate to model the socio-cultural dimension of e-learners by informing a fixed set of personal criteria? In 12th IEEE International Conference on Advanced Learning Technologies (ICALT), USA (pp. 388–392). Blanchard, E. G. (2015). Socio-cultural imbalances in AIED research: Investigations, implications and opportunities. International Journal of Artificial Intelligence in Education, 25(2), 204–228. Blanchard, E. G., & Ogan, A. (2010). Infusing cultural awareness into intelligent tutoring systems for a globalized world. In Advances in intelligent tutoring systems, SCI 308 (pp. 485–505). Berlin Heidelberg: Springer. Blanchard, E. G., & Mizoguchi, R. (2014). Designing culturally-aware tutoring systems with MAUOC, the more advanced upper ontology of culture. Research and Practice in Technology Enhanced Learning, 9(1), 41–69. Blanchard, E. G., Karanasios, S., & Dimitrova, V. (2013). A conceptual model of intercultural communication: Challenges, development method and achievements. In E. G. Blanchard & I. Gas-

Towards Inclusive Education in the Age of Artificial …

33

parini (Eds.), 4th International Workshop on Culturally-Aware Tutoring Systems (CATS)—AIED 2013 Workshops Proceedings (Vol. 5, pp. 1–10). Borgadus Cortez, M. (2017). AI in education will grow exponentially by 2021. Edtech, Focus on K-12. https://edtechmagazine.com/k12/article/2017/07/ai-education-will-grow-exponentially2021. Accessed October 22, 2018. Bray, B. A., & McClaskey, K. A. (2016). How to personalize learning: A practical guide for getting started and going deeper. Thousand Oaks, CA: Corwin. Brown, P., & Levinson, S. C. (1987). Politeness: Some universals in language usage. New York: Cambridge University Press. Brusilovsky, P. (1994). Student model centered architecture for intelligent learning environments. In Proceeding of 4th International Conference on User Modeling, Hyannis, MA, USA, August 15–19, 1994 (pp. 31–36). http://www.pitt.edu/~peterb/papers/UM94.html. Cambridge advanced learner’s dictionary. (2003). Cambridge, UK: Cambridge University Press. Cassell, J. (2009). Social practice: Becoming enculturated in human-computer interaction. In C. Stephanidis (Ed.), Lecture Notes in Computer Science: Vol. 5616, Proceeding of 5th International Conference on Universal Access in HCI, San Diego, CA, USA, July 19–24, 2009 (pp. 303–313). Berlin, Heidelberg: Springer. Castellano, G., Paiva, A., Kappas, A., et al. (2013). Towards empathic virtual and robotic tutors. In H. C. Lane, K. Yacef, J. Mostow, et al. (Eds.), Lecture Notes in Computer Science: Vol. 7926, Artificial Intelligence in Education. AIED 2013. Berlin, Heidelberg: Springer. http://www.macs. hw.ac.uk/~ruth/Papers/planning/AIED13-EMOTE-sub.pdf. Chandramouli, K., Stewart, C., Brailsford, T., et al. (2008). CAE-L: An ontology modelling cultural behaviour in adaptive education. In P. Mylonas, M. Wallace, & M. Angelides (Eds.), 3rd International Workshop on Semantic Media Adaptation and Personalization (pp. 183–188). Washington, USA: IEEE Computer Society. De-Arteaga, M., Herlands, W., Neill, D. B., et al. (2018). Machine learning for the developing world. ACM Transactions on Management Information Systems, 9(2), Article 9. De Jong, M., & Warmelink, H. (2017). OASISTAN: An intercultural role-playing simulation game to recognize cultural dimensions. Simulation & Gaming, 48(2), 178–198. Dial, M. (2018). Heartificial empathy: Putting heart into business and artificial intelligence. Dillenbourg, P. (2016). The evolution of research on digital education. International Journal of Artificial Intelligence in Education, 26, 544–560. Donnelly, P. J., Blanchard, N., Samei, B., et al. (2016). Automatic teacher modeling from live classroom audio. In Proceedings of 24th ACM International Conference on User Modeling, Adaptation, and Personalization (UMAP) (pp. 45–53). New York: ACM. Druga, S. (2018). The story of Cognimates. In Proceedings of 17th ACM Conference on Interaction Design and Children (IDC), Trondheim, Norway, June 19–22, 2018. New York, USA: ACM. Druga, S., Williams, R., Breazeal, C., et al. (2017). Hey Google is it OK if I eat you? Initial explorations in child-agent interaction. In 2017 Conference on Interaction Design and Children, Stanford, California, USA, June 27–30, 2017 (pp. 595–600). ACM. http://robotic.media.mit.edu/ wp-content/uploads/sites/14/2017/06/idcwp0180-drugaACR.pdf. DuBoulay, B., Avramides, K. A., Luckin, R., et al. (2010). Towards systems that care: A conceptual framework based on motivation, metacognition and affect. International Journal of Artificial Intelligence in Education, 20(3), 197–229. Empathy. (2018). https://en.oxforddictionaries.com/definition/empathy. Accessed October 22, 2018. Endrass, B., André, E., & Rehm, M. (2011). Towards culturally-aware virtual agent systems. In E. G. Blanchard & D. Allard (Eds.), The handbook of research on culturally-aware information technology: Perspectives and models (pp. 412–428). Hershey, PA: IGI Global. Ferster, B. (2014). Learning from the intersection of education and technology. Baltimore: Johns Hopkins University Press. Finkelstein, S., Yarzebinski, E., Vaughn, C., Ogan, A., & Cassell, J. (2013). The effects of culturally congruent educational technologies on student achievement. In H. C. Lane, K. Yacef, J. Mostow,

34

P. S. Mohammed and E. ‘Nell’ Watson

et al. (Eds.), Proceedings of 16th International Conference on Artificial Intelligence in Education, AIED, Memphis (pp. 493–502). Berlin, Heidelberg: Springer. Finland schools are testing out robot teachers. (2018). https://nypost.com/2018/03/27/finlandschools-are-testing-out-robot-teachers/. Accessed October 22, 2018. Gasparini, I., Pimenta, M. S., & Moreira de Oliveira, J. P. (2011). How to apply context awareness in an adaptive e-learning environment to improve personalization capabilities? In G. Acuña, J. A. Baier (Eds.), Proceedings of 30th International Conference of the Chilean Computer Society, SCCC, Curico, Chile, November 9–11, 2011 (pp. 161–170). Washington, DC: IEEE Computer Society. Grand Challenges—Advanced Personalised Learning. (2018). http://www.engineeringchallenges. org/challenges/learning.aspx. Accessed October 22, 2018. Hall, E. T. (1966). The hidden dimension. Garden City, NY: Doubleday. Hayashi, Y. (2018). Gaze feedback and pedagogical suggestions in collaborative learning investigation of explanation performance on self’s concept in a knowledge integration task. In C. Rosé, R. Martínez-Maldonado, H. U. Hoppe, et al. (Eds.), Lecture Notes in Artificial Intelligence: Vol. 10947, Proceedings of 19th International Conference on Artificial Intelligence in Education (AIED) (pp. 78–87). Berlin: Springer. Heffernan, N., & Heffernan, C. (2014). The ASSISTments ecosystem: Building a platform that brings scientists and teachers together for minimally invasive research on human learning and teaching. International Journal of Artificial Intelligence in Education, 24(4), 470–497. Henrich, J., Heine, S. J., & Norenzayan, A. (2010). The weirdest people in the world? Behavioural and Brain Sciences, 33, 61–83. Hitzler, P., Krötzsch, M., & Rudolph, S. (2010). Foundations of semantic web technologies. Boca Raton, FL: Taylor and Francis Group. Hofstede, G. (2001). Culture’s consequences: Comparing values, behaviours, institutions and organizations across nations (2nd ed.). Thousand Oaks, CA: Sage Publications. Holstein, K., McLaren, B., & Aleven, V. (2018). Student learning benefits of a mixed-reality teacher awareness tool in AI-enhanced classrooms. In C. Rosé, R. Martínez-Maldonado, H. U. Hoppe, et al. (Eds.), Lecture Notes in Artificial Intelligence: Vol. 10947, Proceedings of 19th International Conference on Artificial Intelligence in Education (AIED) (pp. 154–168). Berlin: Springer. ICT Facts and Figures. (2017). https://www.itu.int/en/ITU-D/Statistics/Documents/facts/ ICTFactsFigures2017.pdf. Accessed October 22, 2018. Janakiram, M. S. V. (2018). https://www.forbes.com/sites/janakirammsv/2018/04/02/nvidiapushes-its-gpu-technology-to-the-front-and-center-of-artificial-intelligence/#173c7a32abc9. Accessed October 22, 2018. Johnson, S. (1979). A dictionary of the english language (Vol. 1). London: Times Books. Johnson, W. L., & Lester, J. C. (2016). Face-to-Face interaction with pedagogical agents, twenty years later. International Journal of Artificial Intelligence in Education, 26, 25–36. Johnson, W. L., Beal, C., Fowles-Winkler, A., et al. (2004). Tactical language training system: An interim report. In Lecture Notes in Computer Science: Vol. 3220, 7th International conference on Intelligent Tutoring Systems, Maceió, Alagoas, Brazil, August 30–September 3, 2004 (pp. 336–345). Springer, Berlin, Heidelberg. Kahn, P. H. Jr., Kanda, T., Ishiguro, H., et al. (2012). Robovie, you’ll have to go into the closet now: Children’s social and moral relationships with a humanoid robot. Developmental Psychology, 48(2), 303. https://depts.washington.edu/hints/publications/Robovie_Closet_ Study_Developmental_Psych_2012.pdf. Accessed October 22, 2018. Kay, J., & McCalla, G. (2003). The careful double vision of self. International Journal of Artificial Intelligence in Education, 13, 11–18. Lugrin, B., Frommel, J., & Andre, E. (2018). Combining a data-driven and theory-based approach to generate culture-dependent behaviours for virtual characters. In C. Faucher (Ed.), Intelligent Systems Reference Library: Vol. 134, Advances in culturally-aware intelligent systems and in cross-cultural psychological studies (pp. 111–142). Cham: Springer.

Towards Inclusive Education in the Age of Artificial …

35

Mascarenas, S., Degens, N., Paiva, A., et al. (2016). Modeling culture in intelligent virtual agents from theory to implementation. Autonomous Agent Multi-Agent Systems, 30, 931–962. Mavrikis, M., Holme, W., Zhang, J., et al. (2018). Fractions lab goes east: Learning and interaction with an exploratory learning environment in China. In C. Rosé, R. Martínez-Maldonado, H. U. Hoppe, et al. (Eds.), Lecture Notes in Artificial Intelligence: Vol. 10947, Proceedings of 19th International Conference on Artificial Intelligence in Education (AIED) (pp. 209–214). Berlin: Springer. McLoughlin, C., & Oliver, R. (2000). Designing learning environments for cultural inclusivity: A case study of indigenous online learning at tertiary level. Australian Journal of Educational Technology, 16(1), 58–72. Melis, E., Goguadze, G., Libbrecht, P., et al. (2011). Culturally-aware mathematics education technology. In E. G. Blanchard & D. Allard (Eds.), The handbook of research on culturally-aware information technology: Perspectives and models (pp. 543–557). Hershey, PA: IGI Global. Merriam-Webster Inc. (1983). Webster’s ninth new collegiate dictionary. MA: Springfield. Mizoguchi, R., & Bourdeaux, J. (2016). Using ontological engineering to overcome AI-ED problems: Contribution, impact and perspectives. International Journal of Artificial Intelligence in Education, 26, 91–106. Mohammed, P. S. (2017). Dynamic cultural contextualisation of intelligent learning environments (Ph.D. thesis). The University of the West Indies, St. Augustine. Mohammed, P. S. (2018). Ontological support for the cultural contextualisation of intelligent learning environments for inclusive education. Paper presented at the international workshop on context and culture in intelligent tutoring systems at intelligent tutoring systems, Montreal, Canada, June 12, 2018. Mohammed, P., & Mohan, P. (2013). Contextualised student modelling for enculturated systems. In E. G. Blanchard, I. Gasparini (Eds.), 4th International Workshop on Culturally-Aware Tutoring Systems (CATS)—AIED 2013 Workshops Proceedings (Vol. 5, pp. 20–29). Mohammed, P., & Mohan, P. (2015). Dynamic cultural contextualisation of educational content in intelligent learning environments using ICON. International Journal of Artificial Intelligence in Education, 25(2), 249–270. Nouri, E., Georgila, K., & Traum, D. (2017). Culture-specific models of negotiation for virtual characters: Multi-attribute decision-making based on culture-specific values. Culture AI & Society, 32(1), 51–63. Nye, B. D. (2015). Intelligent tutoring systems by and for the developing world: A review of trends and approaches for educational technology in a global context. International Journal of Artificial Intelligence in Education, 25(2), 177–203. Ogan, A., Walker, E., Baker, R., et al. (2015a). Towards understanding how to assess help-seeking behaviour across cultures. International Journal of Artificial Intelligence in Education, 25, 229–248. Ogan, A., Yarzebinski, E., Fernández, P., et al. (2015b). Cognitive tutor use in Chile: Understanding classroom and lab culture. In C. Conati, et al. (Eds.), Lecture Notes in Artificial Intelligence: Vol. 9112, Proceedings of 17th International Conference on Artificial Intelligence in Education (AIED) (pp. 318–327). Switzerland: Springer International Publishing. Pham, P., & Wang, J. (2018). Predicting learners’ emotions in mobile MOOC learning via a multimodal intelligent tutor. In C. Rosé, R. Martínez-Maldonado, H. U. Hoppe, et al. (Eds.), Lecture Notes in Artificial Intelligence: Vol. 10947, Proceedings of 19th International Conference on Artificial Intelligence in Education (AIED) (pp. 150–159). Berlin: Springer. Pinkwart, N. (2016). Another 25 years of AIED? Challenges and opportunities for intelligent educational technologies of the future. International Journal of Artificial Intelligence in Education, 26, 771–783. Rehm, M. (2010). Developing enculturated agents: Pitfalls and strategies. In E. G. Blanchard & D. Allard (Eds.), The handbook of research on culturally-aware information technology: Perspectives and models (pp. 362–386). Hershey, PA: IGI Global.

36

P. S. Mohammed and E. ‘Nell’ Watson

Rehm, M. (2018). Affective body movements (for robots) across cultures. In C. Faucher (Ed.), Intelligent Systems Reference Library: Vol. 134, Advances in culturally-aware intelligent systems and in cross-cultural psychological studies (pp 165–188). Cham: Springer. Rehm, M., Andre, E., Nakano, Y., et al. (2007). The CUBE-G approach—Coaching culture-specific nonverbal behavior by virtual agents. In: I. Mayer & H. Mastik (Eds.), ISAGA 2007: Organizing and learning through gaming and simulation. Rhoads, R. A. (2015). MOOCS, high technology and higher learning. Baltimore, MD: John Hopkins University Press. Rise of the robots: Mady Delvaux on why their use should be regulated, EU Parliament. (2017). http://www.europarl.europa.eu/news/en/headlines/economy/20170109STO57505/ rise-of-the-robots-mady-delvaux-on-why-their-use-should-be-regulated. Accessed October 22, 2018. Robbins, C. (2006). Providing cultural context with educational multimedia in the South Pacific. Educational Technology & Society, 9(1), 202–212. Roll, I., & Wylie, R. (2016). Evolution and revolution in artificial intelligence in education. International Journal of Artificial Intelligence in Education, 26, 582–599. Rosé, C. P., & Ferschke, O. (2016). Technology support for discussion based learning: From computer supported collaborative learning to the future of massive open online courses. International Journal of Artificial Intelligence in Education, 26, 660–678. Rudovic, O., Utsumi, Y., Lee, J., et al. (2018). CultureNet: A deep learning approach for engagement intensity estimation from face images of children with autism. In IEEE/RSJ International Conference on Intelligent Robots and Systems, Madrid, Spain, October 1–5, 2018. IEEE Xplore. Savard, I., & Mizoguchi, R. (2016). Ontology of culture: A procedural approach for cultural adaptation in ITSs. In W. Chen, et al. (Eds.), Proceedings of 24th International Conference on Computers in Education. India: Asia Pacific Society for Computers in Education. Shah, D. (2018). By the numbers: MOOCS in 2017. https://www.class-central.com/report/moocstats-2017/. Accessed October 22, 2018. Sharifian, F. (2003). On cultural conceptualisations. Journal of Cognition and Culture, 3(3), 187–207. Six key benefits of adaptive learning. (2013). http://www.dreambox.com/blog/six-benefits-ofadaptive-learning. Accessed October 22, 2018. Thakker, D., Karanasios, S., Blanchard, E., et al. (2017). Ontology for cultural variations in interpersonal communication: Building on theoretical models and crowdsourced knowledge. Journal of the Association for Information Science and Technology, 68(6), 1411–1428. The IEEE global initiative on ethics of autonomous and intelligent systems. Affective computing. (2018). https://standards.ieee.org/content/dam/ieee-standards/standards/web/documents/ other/ead_affective_computing_v2.pdf. Accessed October 22, 2018. The US National Academy of Engineering. (2018). https://www.scientia.global/the-usnationalacademy-of-engineering/. Accessed October 22, 2018. Tims, M. J. (2016). Letting artificial intelligence in education out of the box: Educational cobots and smart classrooms. International Journal of Artificial Intelligence in Education, 26, 701–712. Trompenaars, F., & Hampden-Turner, C. (1997). Riding the waves of culture. London: Nicholas Brealey Publishing. Uchidiuno, J., Yarzebinski, E., Madaio, M., et al. (2018). Designing appropriate learning technologies for school versus home settings in Tanzanian rural villages. In Proceedings of 1st ACM SIGCAS Conference on Computing and Sustainable Societies, Menlo Park and San Jose, CA, USA, June 20–22, 2018. New York: ACM. VanLehn, K. (2011). The relative effectiveness of human tutoring, intelligent tutoring systems, and other tutoring systems. Educational Psychologist, 46(4), 197–221. Vartak, M. P., Almeida, S. F., & Heffernan, N. T. (2008). Extending ITS authoring tools to be culturally-aware. In E. G. Blanchard & D. Allard (Eds.), Proceedings of 1st International Workshop on Culturally-Aware Tutoring Systems (CATS 2008)—ITS 2008 Workshops Proceeding, Montreal, Canada, June 23–27, 2008 (pp. 101–105).

Towards Inclusive Education in the Age of Artificial …

37

Wang, S. (2018). AI robots are transforming parenting in China, CNN. https://edition.cnn.com/ 2018/09/28/health/china-ai-early-education/index.html. Accessed 22 Oct 2018. Wang, N., Shapiro, A., Feng, A., et al. (2018). Learning by explaining to a digital doppelganger. In R. Nkambou, R. Azevedo, & J. Vassileva (Eds.), Lecture Notes in Computer Science: Vol. 10858, Proceedings of 14th International Conference on Intelligent Tutoring Systems (pp. 256–264). Cham: Springer. Williams, R., Vazquez, C., Druga, S., et al. (2018). My doll says it’s OK: Voice-enabled toy influences children’s moral decisions. In Proceedings of 17th ACM Conference on Interaction Design and Children, Trondheim, Norway, June 19–22, 2018 (pp. 625–631). New York: USA, ACM. Woodie, A. (2018). https://www.datanami.com/2018/03/13/jeff-dean-thinks-ai-can-solve-grandchallenges-heres-how/. Accessed October 22, 2018. Woolf, B. P. (2009). Building intelligent interactive tutors: Student-centered strategies for revolutionizing e-learning. Burlington, MA: Morgan Kaufmann. Yadollahi, E., Johal, W., Paiva, A., et al. (2018). When deictic gestures in a robot can harm childrobot collaboration. In Proceedings of 17th ACM Conference on Interaction Design and Children, Trondheim, Norway, June 19–22, 2018 (pp. 195–206). New York: USA, ACM.

Phaedra S. Mohammed is a lecturer at the Department of Computing and Information Technology, at the University of the West Indies, St. Augustine. She teaches undergraduate and graduate courses in Information Technology and Computer Science. Her Ph.D. thesis focused on developing computational models of culture and techniques for linguistic and semantic adaptations of educational content in intelligent learning environments using sociocultural influences that shape a student’s background. Phaedra’s research interests include semantic web technologies, natural language processing, cultural modeling and intelligent learning environments. Eleanor ‘Nell’ Watson (FBCS, FIAP, FRSA, FLS, FRSS, CITP) is an engineer, educator and tech philosopher who grew up in Northern Ireland. Nell founded Poikos (now QuantaCorp). This original, patented technology enables fast and simple 3D body measurement from only two planes (front and side), using a simple cellphone camera, by applying sophisticated deep learning technologies. This service enables fast and accurate personalization services in telemedicine, mass customization and retail. Today, Nell educates others in how to implement such technologies, for example, by creating video coursebooks for O’Reilly. She is also Co-Founder of EthicsNet.org, an NGO, building a movement of people who are committed to help machines understand humans better. This community acts as role models and guardians to raise kind AI, by providing virtual experiences, and collecting examples of pro-social practices. In her spare time, Nell enjoys coding games, such as her start-up life simulator, Founder Life which attempts to teach the mindfulness and psychological habits necessary for entrepreneurs to consistently execute under pressure. Nell lectures globally on machine intelligence, AI philosophy, human–machine relations, and the future of human society, serving on the faculty of AI and robotics at Singularity University.

Accountability in Human and Artificial Intelligence Decision-Making as the Basis for Diversity and Educational Inclusion Ka´ska Porayska-Pomsta and Gnanathusharan Rajendran

Abstract Accountability is an important dimension of decision-making in human and artificial intelligence (AI). We argue that it is of fundamental importance to inclusion, diversity and fairness of both the AI-based and human-controlled interactions and any human-facing interventions aiming to change human development, behaviour and learning. Less debated, however, is the nature and role of biases that emerge from theoretical or empirical models that underpin AI algorithms and the interventions driven by such algorithms. Biases emerging from the theoretical and empirical models also affect human-controlled educational systems and interventions. However, the key mitigating difference between AI and human decisionmaking is that human decisions involve individual flexibility, context-relevant judgements, empathy, as well as complex moral judgements, missing from AI. In this chapter, we argue that our fascination with AI, which predates the current craze by centuries, resides in its ability to act as a ‘mirror’ reflecting our current understandings of human intelligence. Such understandings also inevitably encapsulate biases emerging from our intellectual and empirical limitations. We make a case for the need for diversity to mitigate against biases becoming inbuilt into human and machine systems, and with reference to specific examples, we outline one compelling future for inclusive and accountable AI and educational research and practice. Keywords Accountability · AI agents · Autism spectrum · Bias · Decision-making · Neurodiversity

K. Porayska-Pomsta (B) UCL Knowledge Lab, University College London, UCL Institute of Education, London, UK e-mail: [email protected] G. Rajendran Edinburgh Centre for Robotics, Department of Psychology, Heriot-Watt University, Edinburgh, UK e-mail: [email protected] © Springer Nature Singapore Pte Ltd. 2019 J. Knox et al. (eds.), Artificial Intelligence and Inclusive Education, Perspectives on Rethinking and Reforming Education, https://doi.org/10.1007/978-981-13-8161-4_3

39

40

K. Porayska-Pomsta and G. Rajendran

1 Introduction Artificial intelligence (AI) presently receives a lot of press, both for its potential to tackle challenges, from policing, to health care, to education, and for the perceived threat that it poses to our (human) identity, autonomy and future functioning. There is a tension in the current perception of AI between its utopian and dystopian overtones, which Stiglitz has synthesised recently as one between AI as an human replacing machine (AI) versus as an human assisting machine (IA).1 He placed this distinction at the heart of the questions about the implications of AI for society, and for the future of human self-determination, wellbeing and welfare. As Reisman et al. (2018) point out, the recent transition of AI from a purely scientific domain to real-world applications has placed AI at the centre of our decision-making without our having had a chance to develop a good understanding of the nature of those implications, or to define appropriate accountability measures to monitor and safeguard against any harms. Crawford2 refers to this situation as an inflection point at which we are starting to comprehend how AI can reinforce a whole plethora of sociocultural biases that are inherent in our existing social systems, and where there is a pressing need for us to question and to hold to account the AI solutions and the decisions that are based on them. Thus, in the present context where AI technologies and their impact are still largely unknown, questions (and actions) related to social and educational inclusion, and education more broadly, are critical to how we develop and utilise AI in the future, and how we develop a system of accountability that is able to guide us in doing so in socially responsible, and empowering ways. There is a growing awareness that current AI systems tend to expose and amplify social inequalities and injustice, rather than address them (Crawford and Calo 2016; Curry and Reiser 2018). There are two known reasons for this. First, the sociocultural biases that are inherent in the data consumed by the AI models make those models also socially skewed. Such biases may originate from (i) our historic and current sociocultural prejudices (be it related to race, gender, ethnicity, etc., e.g., police records that are skewed towards particular social groups such as young black males as more likely to commit crimes), (ii) lack of data that is representative of the society as a whole (Crawford and Calo 2016), or (iii) they may be an artefact of the specific classification algorithms used and the ways their success is being measured (e.g. Lipton and Steinhardt 2018). Although many AI ‘solutions’ are well intentioned, given the current state of both the AI technologies and our own limited understanding of the ways in which they impact human decision-making, their deployment in real high-stake contexts, such as arrest decisions, seems premature (Reisman et al. 2018). The fact that many AI solutions—especially machine learning—are seldom open to being inspected, or contested by humans represents the second reason why AI is thought to reinforce social biases. Specifically, the so-called black-box AI often prevents humans (engineers and users) from even knowing that biases are present, or from fully understanding how they arise (e.g. from data or from classification 1 https://www.youtube.com/watch?v=aemkMMrZWgM. 2 https://royalsociety.org/science-events-and-lectures/2018/07/you-and-ai-equality/.

Accountability in Human and Artificial Intelligence Decision-Making …

41

algorithms, or both—see, e.g., Brinkrolf and Hammer 2018). This is known interchangeably as the explainability or interpretability problem. Addressing this problem is seen increasingly by the AI community as a remedy to data and AI interpretation bias, speaking to questions of accountability and trustworthiness of the AI-driven decisions (Lipton and Steinhardt 2018; Conati et al. 2018). The AI community is also beginning to recognise that to be genuinely beneficial, AI-driven decisions must be contestable and open to being changed by the users (Brinkrolf and Hammer 2018; Bull and Kay 2016). In this chapter, our treatise is that in order to conjure, a positive future of educational inclusion involving AI requires us first to appreciate that our own human systems (educational, clinical, social justice, etc.) and models of inclusion do not represent absolute truths. Instead, those systems are inherently biased representations of the world, which are limited by our current knowledge and social structures, which determine who and how may influence those representations. Second, in order to genuinely understand the potential of AI in the context of human learning, development and functioning, and to safeguard against misuse, there is a need for an informed differentiation between human and artificial intelligence. Such differentiation is needed to make us stop ‘worshiping at the altar of technology’3 and to admit diverse stakeholders, who are not AI experts, into partaking actively in the design of AI technologies and their use for education. The rest of the chapter is structured as follows. In Sects. 2 and 3, respectively, we briefly introduce the concepts of accountability and inclusion. We outline the definitional challenges, highlighting how the two concepts relate to one another, to scientific and technological innovation and to dominant approaches to inclusion. In Sect. 4, we define AI in order to provide an informed basis for considering its true potential in the context of educational inclusion. In particular, we introduce AI not solely as a solution to some ‘curable’ problem, but as a conceptual framework for formulating pertinent questions about learning, development and inclusion, and as a method for addressing those questions. In this section, we also outline the key differences between human intelligence (HI) and artificial intelligence (AI). As will be argued in Sect. 5, acknowledging and understanding this difference explicitly allows us to appreciate how AI can be designed and used to assist learners and educators through (i) providing relative safety zones for learners to accommodate and even reduce any pronounced differences or difficulties, e.g. social anxiety in autism (AI as a stepping stone), (ii) acting as a mirror in self-exploration and development of self-regulation competencies (AI as a mirror) and (iii) offering a medium for understanding and sharing of individual perspectives and subjective experiences, as the basis for nurturing tolerance, compassion and for developing appropriately tailored educational support (AI as a medium). We employ examples from our own research, which are of relevance to social and educational inclusion in which we used AI: one involving a genetically determined case of autism and the other—a socio-economic one of youth unemployment. Section 6 will conclude the chapter by summarising the interdependency of the key concepts considered (inclusion, accountability and AI) 3 https://royalsociety.org/science-events-and-lectures/2018/07/you-and-ai-equality/.

42

K. Porayska-Pomsta and G. Rajendran

and will outline the steps that are needed to achieve our vision of AI as a technology for social good.

2 Accountability Accountability is a key dimension of decision-making in human and artificial intelligence, and it is crucial to any democratic, tolerant and inclusive society. This is because accountability is fundamentally about giving people the autonomy of action through knowledge. However, although accountability has become de facto a cultural term, it is not always clear what it actually means in practice. To date, two main ontological perspectives on accountability have been adopted in law and policy (Dubnick 2014). The first perspective relates to the post-factum accountability, involving a blameable agent whose attempts to manipulate another agent’s actions according to their wishes require them to be held responsible for those actions and for the consequences thereof, e.g. the blameable agent after the 2008 financial crisis was the financial sector. The second dominant perspective is the normative one, representing some preferential solutions to a range of aspirational problems such as justice, democracy, racial discrimination, where societal, political or institutional organisations are the decision-makers. This is referred to as the prefactum type of accountability, involving an a priori blameworthy agent or agents. Here, it is assumed that the aim of accountability measures is to reach a societal change or mass acquiescence in anticipation of some possibly blameworthy actions or events. For example, after the 2008 financial crisis, a set of accountability measures were imposed by the UK’s Financial Conduct Authority (FCA) on the financial sector to prevent similar crises in the future, and to offer transparency in the sector’s decision-making and actions. Recently, these two dominant stances have been critiqued as being too rigid to allow for an operationalisation of accountability as an ongoing social process that it is (Dubnick 2014). The main issue here is that although the pre- and post-factum definitions provide a moral and legal framing of accountability, they do not specify how accountability can be actioned in an agile way in diverse and often changeable contexts, involving different stakeholders, and given our perpetually changing understanding of the world. For example, by exposing the existing biases in our pre-AI representations of the world (e.g. in policing), AI has also demonstrated a substantial gap between the aspirational social rhetoric of inclusion, tolerance and welfare and the reality on the ground. Specifically, it showed not only that our systems are still based on historical and socially skewed data, but also that our predominant accountability measures and laws struggle to catch up with our social aspirations and changing norms, and with our developing scientific and practical knowledge related to inclusion. A more flexible approach is offered by an ethics-based theory of accountability (Dubnick 2014), where accountability is defined as a social setting and a social negotiation. Here, the rules and the moral codes which define how it is operationalised

Accountability in Human and Artificial Intelligence Decision-Making …

43

in practice can be adjusted according to the changes and needs occurring within the individual stakeholder groups in tandem with and in response to the developments in our scientific, economic and social circumstances and understandings. In this view, accountability is of relational nature, involving ‘multiple, diverse and often conflicting expectations (MDCE)’, priorities and investments of different stakeholders, along with temporal fashions that determine who is accountable for decisions and actions to whom, with respect to what, and when (ibid.: 4). In this account, the who, the whom, the for-what and the when represent context-dependent variables that are instantiated based on the salience assigned to the specific MDCEs, with accountability becoming an exchange and an ethically regulated, tractable and auditable compromise between different competing interests and gains of the decision-makers. This relational approach is of particular relevance both in the context of AI and educational inclusion. In particular, this interpretation acknowledges that there is no one-size-fits-all, best way to make the decisions of others auditable and that, fundamentally, the judgements related to the blameability or blameworthiness of decisions are based on the relative needs and goals of the stakeholders affected. This means that if the system is designed in such a way that it hinders or by definition excludes some groups of stakeholders from being able to inspect and influence it, for example by obstructing their participation in making decisions in matters that affect them, or by preventing them from acquiring appropriate skills to engage in such auditing, then social inclusion, equity and fairness are compromised. In contrast, the relational definition of accountability: (i) allows us to appreciate it as a social construct that assumes different and often conflicting interests and prioritisations thereof that affect people’s decisions; (ii) presupposes the existence of stakeholders who are empowered intellectually, financially, etc., to generate and respond to the different expectations, and invest in enhancing their salience, therefore also highlighting accountability practices themselves as being neither perfect or neutral; (iii) it can be used directly to examine the role of AI in first exposing this lack of neutrality (as already discussed in the introduction), and second, in highlighting the continuing need to empower different potential stakeholders to invest in generating and lobbying for their priorities. Thus, the uniqueness of AI in this context lies not only in its ability to act as a moral mirror and a magnifying glass for examining our pre-existing conceptions of social inclusion and social justice, tolerance and welfare. It also lies in its ability to provide concrete, tractable, interactive and scalable means for genuinely democratising accountability mechanisms, including those related to the explainability and contestability of educational interventions and assessments. As will be elaborated and exemplified further in Sect. 5, this latter affordance of AI represents one of the most exciting avenues for AI in educational inclusion.

3 Inclusion So far, we discussed the potential of the relational framing of accountability in the context of inclusion in allowing us to devise accountability policies in ways that

44

K. Porayska-Pomsta and G. Rajendran

respond to our developing knowledge and changing social norms. We also highlighted the link between accountability and AI and the latter’s ability to expose preexisting biases. Although the relational view of accountability may describe how the accountability processes play out, its present operationalisations rely predominantly on the pre- and post-factum framings. This is problematic from the point of view of inclusion in two ways. First, the two non-relational stances on accountability deemphasise the need for empowering all potential stakeholders to influence decisions that affect them, instead surrendering the responsibility for enforcing accountability to those who are endowed with appropriate governing powers, but may lack the experiential, contextual and intellectual basis for their decisions. Second, they are prescriptive top-down approaches which reinforce existing definitions of inclusion, rather than assuming a priori that those definitions are likely to evolve with the changing scientific knowledge, social norms and aspirations. Historically, inclusion tends to be defined in terms of specific pronounced differences from what may be currently considered the ‘norm’; i.e., the definitions of inclusion tend to be exclusive by default. For example, the OED defines inclusivity as: The practice or policy of including people who might otherwise be excluded or marginalized, such as those who have physical or mental disabilities and members of minority groups. ‘you will need a thorough understanding of inclusivity and the needs of special education pupils’4

This definition explicitly uses special education highlighting as its illustration those who otherwise would be marginalised. This is also indicative of the definitional problems with inclusion that arise at the systemic level, where inclusion is treated as a solution to integrating and assimilating those who are considered at the margins, rather than as a process through which differences can be used to extend our understanding of ‘normality’ or ‘typicality’, and where societies can expect to be influenced by and to benefit from diversity. The definitional limitations of this framing of inclusion also both reflect and are reflected in many educational and clinical intervention approaches. In particular, the history of psychiatry and psychology is one of exclusion and marginalisation, and it is based explicitly on notions of abnormality. Although laudable in its aims to understand conditions through aetiology by taxonomy, the resultant diagnoses, classifications and practices have been historically decided by the majority and imposed on the minority. To illustrate this, the Diagnostic and Statistical Manual for the American Psychiatric Association was first published in 1952 (DSM-I). This manual has framed our clinical and scientific understanding of abnormality, and it has remained disorderfocused. For example, in the case of autism, the very name in DSM-V (2013) autism spectrum disorder (ASD) reflects this perspective. In 1952, DSM-I listed 106 disorders. In 2013, DSM-V listed 300 disorders (Baron-Cohen 2017). Homosexuality was classified as a disorder in DSM-I and DSM-II and was only removed from DSM-III in 1980. With respect to autism, O’Neil (2008) argues that in much the same way as homosexuality was no longer considered a disorder; the classification of autism should also be reconsidered. 4 https://en.oxforddictionaries.com/definition/inclusivity.

Accountability in Human and Artificial Intelligence Decision-Making …

45

One criticism of framing developmental differences as disorders rather than as conditions is that it misses the point of functionality that reflects an evolutionary purpose. For example, conditions like Tourette syndrome, ASD and ADHD include behavioural phenotypes of executive control, such as behavioural inhibition or inability to start or stop oneself from engaging in certain behaviours. An evolutionary perspective asks about the function of poor inhibitory control and about the purpose of keeping certain traits and behaviours in the gene pool, i.e. leading to neurodiversity in the population. This is in contrast to viewing abnormality as an aberration either from a social ideal of ‘normal’, or from a statistically derived norm, like intellectual disability.5 Some animal behaviour researchers (e.g. Dobson and Brent 2013) postulate a mechanism by which neurodiversity might be functional and beneficial; i.e. variations in the genome can help animals be adaptive and that such differences are part of natural selection and fitness, rather than abnormalities to be eradicated. Thus, the neurodiversity perspective takes a different position from the pathological, emphasising the role of environments in accommodating diversity. In the context of autism, increasingly researchers have been investigating how social and educational environments can be co-created with stakeholders to represent and empower, rather than segregate neuro-diverse learners both in traditional practices (e.g. Baron-Cohen 2017; Rajendran 2013; Remington 2018), and those involving the application of AI (Porayska-Pomsta et al. 2018). The take-home message here is that abnormality is socially rather than biologically constructed, and thus, any developments related to educational and social inclusion, including AI use in education, must take this into account explicitly. Importantly, a closer look at the history of social and educational practices in this context reveals that the questions of bias and discrimination predate the emergence of AI, by a long way. This highlights that the questions of accountability, inclusion and the role of AI in shaping our collective understanding of ourselves are intricately intertwined, and that for AI to serve educational inclusion and best educational practices, they need to be considered together.

4 Artificial Intelligence In order to appreciate how AI technologies may interplay with the constructs of accountability and inclusion and to help us understand how AI can be used to deliver more inclusive education, it is important to consider the original conception of AI as: 1. An applied philosophy allowing us to formulate key questions about different aspects of (human) intelligence (Davis et al. 1993; Davis 1996; Russell and Norvig 2003; Woolf 2008); 5 An

IQ of less than 70 is considered intellectually disabled because it falls two standard deviations from the population mean of 100. The assumption is that IQ is normally distributed and abnormality can be statistically determined in an objective was.

46

K. Porayska-Pomsta and G. Rajendran

2. A method for testing our different theories about intelligence by operationalising them in computational models which produce observable and measurable behaviours without our having to take real action (e.g. Davis et al. 1993; PorayskaPomsta 2016); 3. A solution to specific real-world challenges (like policing or medical diagnosis), but which are nevertheless artefacts of our questioning and experimentation, based on the current, and hence by definition incomplete, state of our knowledge and understanding. With AI having now crossed over from a purely scientific domain to practical mainstream applications, AI as a solution has taken the centre stage. However, we believe that this single lens limits our view on the actual strengths and weaknesses of AI in the context of socially embedded practices such as in education, and more broadly as a tool for scientific enquiry into what makes us human. It obscures the need for our asking what society do we want, instead permitting technological advances (and the few tech specialists behind those) to dictate what society we end up with. In contrast, the broader three-lens view of AI makes us appreciate that both the questions and the answers formulated with the help of AI are relative to the current state of our knowledge. Importantly, this definition helps us further in approaching inclusion and education not merely as some fixed state for which there is a set of equally fixed solutions that can be administered like medicine, but as a social process and a state of mind, which requires our own investment, enquiry and willingness to change. Seen in this way, the necessary prerequisites of a socially inclusive AI, that caters for and involves the human in its decision-making, become readily apparent. As will be illustrated in Sect. 5, AI can uniquely provide both the intellectual and physical means that are manipulable and scalable, allowing for an exploration, speculation and rigorous experimentation (e.g. through simulated scenarios) about what it means to be inclusive along with the mechanisms that may be conducive and effective to fostering inclusion through education and educational practices. To appreciate this point, it is necessary also to understand the ways in which AI differs from human intelligence by considering how it operates at a lower level of description. Two broad schools of thought define how AI has been implemented to date: (i) the so-called good old-fashioned AI (or GOF AI) and (ii) machine learning. The GOF AI requires explicit representation of knowledge, which reflects an ontological conceptualisation of the world and actions that are possible therein, along with some well-defined measures of success in terms of concrete goals and goal satisfaction constraints. For example, in the context of maths tutoring, the ontological representations will relate to the specific sub-domains of maths, say misconceptions in column subtraction, and rules that define the possible operations on this subdomain. The goal satisfaction in this case may be in terms of student’s correct or incorrect answers. As such, GOF AI is by definition limited, with the concepts and rules being hard-coded into the systems, often during laborious and time-consuming design stages (Porayska-Pomsta and Bernardini 2013). Such rules are typically elicited through questioning of human experts in a given domain, by observing their expertise in real contexts or by hand-annotating data (video

Accountability in Human and Artificial Intelligence Decision-Making …

47

recordings, interaction logs, etc.) of humans engaging in specific tasks. From the point of view of human learning and accountability of AI decision-making, the key advantage of such knowledge-based systems is that they require a detailed understanding of the domain, in order for knowledge ontologies to be constructed, thus also potentially leading to a greater understanding of the domains represented, and the fact that the resulting ontologies are transparent, inspectable, and often understandable by humans (Davis et al. 1993; Russell and Norvig 2003). By contrast, machine learning (ML) learns solutions from first principles by applying statistical classification methods to large data sets. ML is largely inspired by our current knowledge of how the brain works and by cognitive psychology theories, such as reinforcement learning (Russell and Norvig 2003; Sutton and Barto 2000). ML carries a substantial promise both in terms of reducing the effort required to specify knowledge ontologies and in being able to go beyond the knowledge we have ourselves, and in so doing—in driving more accurate decision-making than our own capabilities allow for. Thus, one of the most exciting aspects of ML is that it can discover new associations in the world and predict future outcomes based on prior data in complex domains which may be hard for the human to grasp and analyse efficiently. One of the recent prominent examples of this ability of ML is the success of the AlphaGo programme by Google Deep Mind (henceforth AlphaGo-DM)6 (Silver et al. 2017). The game of Go represents a highly complex, albeit constrained, problem space where the solutions require more than simply knowing the game’s rules. It is an ancient game which takes a lifetime to master and is considered one of the most challenging games ever invented. In 2017, AlphaGo-DM beat the human Go world champion, by presenting strategies that were not known to him. Interestingly, despite his defeat, the master, expressed his excitement at being able to learn new game strategies from a machine. In this, he made an explicit link between AI and its potential for human learning and creativity. However, it is important to appreciate that ML’s ability to come up with novel solutions is not a sign of its humanity or creativity, but rather of a different and in many ways far more advanced, computational prowess and efficiency than afforded by the human brain in similar tasks. In this sense, the ML employed in AlphaGoDM demonstrated its ability to engage in interpolation, i.e. averaging information based on voluminous data, and extrapolation, i.e. finding new information within a given data set (e.g. Sutton and Barto 2000). However, what ML and AI more broadly cannot do, and what differentiates it further from human learning and intelligence, is to invent new things (e.g. to invent a new game), to imagine things, to entertain fantastical scenarios, to employ counterfactual or critical thinking beyond the gain/loss measures and crucially to entertain moral judgement. More generally, the fundamental difference between AI and HI is that although AI aims to emulate our own behaviours, on the whole and for pragmatic reasons of tractability, it does not require fidelity to human cognition and functioning (Russell and Norvig 2003). This 6 https://deepmind.com/research/alphago/.

48

K. Porayska-Pomsta and G. Rajendran

difference is central to the present debates about AI safety, ethics and its implications for society and it explains why AI’s ability to surpass (or more accurately—to bypass) some of our own talents, may lead to our sense of disempowerment and impending doom for our welfare and wellbeing,7,8 and even our status as a species.9 However, what is far less audible in the current debates is the fact that these same characteristics that frighten us, make AI precisely the tool that might be needed to enhance our abilities, to make us reflect on who we are and who we want to be, and to use it as an educational instrument of social change (Hernandez-Orallo and Vold, 2019). In the next section, we use concrete examples from our own research to elaborate on how AI can act in this positive way. The key question to bear in mind here is the extent to which we want to surrender our autonomy and learning to the AI versus to use AI to enhance our learning and decision-making capabilities (see also Stiglitz’s AI vs. IA introduced in Sect. 1).

5 AI and Educational Inclusion: Beyond the Bias The future of AI and educational inclusion is not necessarily a dystopian one. As discussed throughout this chapter, the current issues of AI bias actually provide a strong pair of glasses onto how we create systems, and on the extent and nature of our own inherent biases. Our aim here is not to rail against systems, which often allow for patterns to be seen. We argue that as a precise philosophical and methodological tool, AI can help us first to understand, regulate and accept ourselves, and second, to understand, and be able to access other people’s experiences and points of view. According to a large body of cross-disciplinary research (e.g. Flavell 1979; Paul and Binkler 1990; Moshman 2011; Terricone 2011; Prizant et al. 2003, 2006; Lai 2011), such understanding and access represent two foundational prerequisites to inclusion regardless of whether AI is present. In this section, using examples from our own research, we demonstrate how AI, with its ability to shine a bright light onto our own behaviours and conceptions of the world, can help us gain a better understanding of ourselves and of others and pave the way for a more inclusive education and society. We have identified three affordances of AI in this context, which we see as key research investment areas of the future.

7 https://www.theguardian.com/commentisfree/2018/feb/01/robots-take-our-jobs-amazon-go-

seattle. 8 https://www.independent.co.uk/life-style/gadgets-and-tech/news/stephen-hawking-artificial-

intelligence-fears-ai-will-replace-humans-virus-life-a8034341.html. 9 http://unesdoc.unesco.org/images/0026/002615/261563E.pdf.

Accountability in Human and Artificial Intelligence Decision-Making …

49

5.1 AI as a Stepping Stone AI-driven environments are very good at providing situated, repeatable experiences to their users, offering an element of predictability and a sense of safety, while creating an impression of credible social interactions, e.g. through adaptive feedback. This is important, in contexts where the users may experience social anxiety, or when they lack self-efficacy and self-confidence. For example, the ECHOES project (Porayska-Pomsta et al. 2018; Bernardini et al. 2014) created an AI environment for young children with autism spectrum disorders (ASD), through which they learned, practiced and explored social interaction skills. Autism is a neurodevelopmental condition which involves difficulties in social communication and interaction, and restricted and repetitive behaviours and often includes feelings of social anxiety. The aim of autism interventions is to reduce those difficulties. One issue increasingly highlighted by interdisciplinary research (e.g. Prizant et al. 2003, 2006) is that many interventions focus on correcting the deficits in a bid to adapt the children to the environment, rather than on correcting the environments to alleviate children’s difficulties. By focusing on correction rather than accommodation of differences, such interventions often fail to access children’s needs, and their interpretation of the world, leading to missed opportunities for understanding and learning about each other’s perspectives by both the learners and practitioners (Rajendran 2013). ECHOES was developed for use in schools. It utilised an AI agent as a social partner in a variety of semi-fantastical scenarios involving both exploratory, openended activities and well-defined closed tasks, e.g. picking flowers, or throwing a bouncy ball through a virtual cloud to change the ball’s colour. Most activities were a collaboration between children and the agent and could include a human social partner (a teacher or researcher accompanying the child), if the child wanted to involve them. As our target users were children at the lower end of the autism spectrum who were classified as non-verbal, ECHOES employed a large multitouch screen through which they acted on the environment (see Fig. 1). The agent acted in a positive and structured way through initiating interactions with the children and enthusiastically responding to any bids for interaction from them. Since initiating and responding to bids for interaction is an area of particular difficulty in autism, these skills were the focus in ECHOES. The agent’s actions were aided by a GOF AI planning architecture, which determined the agent’s: (i) choice of actions in real time given its appraisal of children’s behaviours, and (ii) longer-term action plans related to helping children become more used to initiating and responding to bids for interaction. The planner also catered for the emotional predispositions of the agent, e.g. its propensity for happiness and positivity (Dias and Paiva 2005). The agent was endowed with an ability to display a wide range of complex emotions for the child to explore (see Fig. 2). However, given that the agent was quite obviously not a real child (it was a cartoon character able to support social interaction contingently) coupled with children having control over the type, number and sequence of activities, provided a needed safety zone for them to

50

K. Porayska-Pomsta and G. Rajendran

Fig. 1 A child playing with the ECHOES agent through the multitouch screen interface (left). The agent points to a flower that it wants a child to pick and put in the basket in a bid for attention and interaction with the child (right)

engage in social interactions without having to endure the typical drill-and-practice training. It allowed them to explore the causes and effects of their actions repeatedly and without the anxiety of real-world consequences, giving them the time to get used to particular forms of interactions, to rehearse them with the agent and to decide if and when they were ready to interact with a human. This level of control and quality of interaction practice are rarely possible in classrooms or during contrived clinical environments, where children may feel inhibited to engage in communication at all. Importantly, in adopting this approach, in line with best autism practices and contrary to the corrective approaches to inclusion, ECHOES centred around the child’s needs, allowing them to reveal their abilities and strengths at their own pace and gradually. A rigorous evaluation of ECHOES revealed that the frequency of children’s responses and initiations has increased over time, with a significant increase in responses to human partners during ECHOES use. Additionally, teachers’ reports suggest transfer of some critical social behaviours from ECHOES to classroom contexts, such as children’s initiating and responding to greetings, transitioning between activities, and even initiating and responding verbally, which in many cases was revelatory to teachers who thought those children to be non-verbal (see Porayska-Pomsta et al. 2018). Unlike some AI environments and contrary to some teachers’ fears of AI being set to replace them, in ECHOES we recognised the strength of AI residing in its imperfect, but nonetheless a credible approximation of human social abilities. These imperfections were explicit and critical to boosting children’s confidence and their own sense of social competence. The role of the human partner (a teacher) was then to build on the strengths demonstrated by the children and to reinforce the sense of confidence acquired with the AI agent in typical classroom and playground contexts. Here, the fact that AI was not the same as a human, but that it was able to approximate plausibly some human behaviours in a just-in-time socially congruent manner was key, because it allowed children to get used to the different social scenarios, with the agent providing consistent, but not fully predictable (owing to its autonomous decision-making facilitated by the AI planning architecture) interaction

Accountability in Human and Artificial Intelligence Decision-Making …

51

Fig. 2 A tool design tool demonstrating the complex emotional displays of the AI agent in the ECHOES project. The sliders to the left represent individual emotions such as anger, happiness, fright. These can be blended to display ambiguous or nuanced emotions depending on the instructions from the planner as to what emotions the agent is ‘experiencing’ given its interpretation of the child’s actions and its own goals

partnership. The recognition by children of the difference between the AI agent and a human is critical for their engagement, for lessening of their social anxiety, and for increasing their sense of autonomy and control over the interaction, all of which are rarely afforded in real social situations. AI allows to regulate carefully this sense of autonomy and self-efficacy in preparation for the real-world situations.

5.2 AI as a Mirror AI operates on precise data and this means that it is also able to offer us precision of judgement and recall of events. With respect to inclusion, provided that there is a possibility of comeback from the human, this can be very valuable, even if in all its precision, AI does not necessarily offer us the truth. Systems that employ the socalled open learner models (OLMs) show how users’ self-awareness, self-regulation and ultimately self-efficacy can be supported by allowing them to access, interrogate

52

K. Porayska-Pomsta and G. Rajendran

and even change (through negotiation with the AI system) the data generated of them (Bull and Kay 2016; Conati et al. 2018). For example, the TARDIS project (Porayska-Pomsta et al. 2014; Porayska-Pomsta and Chryssafidou 2018) successfully used the OLM approach to provide young people at risk of exclusion from education, employment or training (NEETs) with insight into their social interaction skills in job interview settings and with strategies for improving those skills. Here, data about the young people’s observable behaviours is first gathered and interpreted during interactions with AI agents acting as job recruiters. This data, which relates to the quality of users’ specific verbal and non-verbal behaviours (e.g. length of answer to specific interview questions, facial expressions, quality of gestures, posture, and voice respectively), is then used as the basis for detailed inspection by the learner, aided by a human coach. Such inspection is intended to provide a platform for the learners to explore their specific strengths and weaknesses in their job interview performances, and for developing a set of strategies for self-monitoring and self-regulation during further interviews. In TARDIS, the learning interaction was facilitated through an off-the-shelve Microsoft Kinect and a high-quality microphone (Fig. 3). These collected data such as specific gestures performed by the user, voice quality and speech duration. These data provide the necessary input to the system, which allows it to create a user profile (a model) and to assess the users’ performance in terms of the quality of their verbal and non-verbal behaviours. The assessments are stored in learner models and are used by other modules responsible for managing the interaction scenarios, to select appropriate questions during interviews and to drive the behaviours of the AI agents acting as job recruiters (Jones et al. 2014). As in ECHOES, the agents were furnished with a wide range of behaviours, underpinned with an emotion-driven planning architecture. Of particular importance to the relationship between AI and data bias, accountability and social inclusion considered in this chapter, is the fact that the TARDIS learner models are open for user inspection after the interviews with the AI agents and that such access offers a level of control and agency to the users that is seldom available in traditional contexts. These models display data gathered about the users’ behaviours during interview simulations, along with the system’s interpretation of this data (see Fig. 4). Through the TARDIS open learner model (OLM), the users have access to interactive timelines of their interview simulations, including precise information on all the actions that they and the agents performed moment by moment. The replay of these actions is synchronised with video recordings of the learners and the agents during interview simulations (top left of Fig. 4). The learners can also inspect the AI’s interpretation of the quality of their individual behaviours (top and bottom right-hand side of Fig. 4), e.g. the energy in their voice, expansiveness of their gestures, together with a commentary on whether these are appropriate at any given point during the interview and what might need to be corrected in the future. A controlled study compared TARDIS and a traditional online self-improvement programme. It revealed significant improvements for the TARDIS users in terms the quality of their interview answers, verbal and nonverbal behaviours, and self-reported measures related to their levels of anxiety, self-efficacy and quality of their answers

Accountability in Human and Artificial Intelligence Decision-Making …

53

Fig. 3 Interaction with TARDIS was facilitated through an off-the-shelf Microsoft Kinect, which was used to detect users’ gestures and posture as well as facial expressions, and high-quality microphone to detect voice

Fig. 4 TARDIS scrutable OLM showing synchronised recordings of the learners interacting with the AI agents along with the interpretation of the learner’s low-level social signals such as gaze patterns, gestures, voice activation in terms of higher-level judgements about the quality of those behaviours, e.g. energy in voice

54

K. Porayska-Pomsta and G. Rajendran

(Porayska-Pomsta and Chryssafidou 2018). As well as providing a situated experience of job interviews to the young people many of whom have never experienced a job interview before, through its OLM, TARDIS offered the learners an invaluable insight into their own behaviours, triggering self-awareness, self-reflection, explanation, planning and self-monitoring in future interactions, including during humanto-human job interviews. Here, the goal was very explicitly to provide the learners with an objective mirror that they could look into, through which they could question themselves either privately, with peers, or with practitioners, and which they could use as the basis for developing informed self-understanding and agency. Thus, in this context, inclusion is not merely about having access to information; rather through having access to information and appropriate scaffolding as to how to act on this information, it is about giving the learners the necessary means to guide their own behaviours and to self-determine.

5.3 AI as a Medium Just as AI systems, such as those based on OLMs, can support the development of selfunderstanding and self-regulation, they can also provide unique and unprecedented insight into educational practitioners about their pupils. Gaining such insight can be game-changing in inclusion practices and individual support interventions, because it can reveal learners’ behaviours and abilities that might be hard to observe or foster in traditional environments. For example, in ECHOES some children who were thought to be uncommunicative, became motivated to communicate, revealing their previously hidden potential and changing the way in which teachers supported them beyond ECHOES. Indeed, in several cases, the practitioners used ECHOES as conduit through which to engage children in social interaction, with ECHOES then providing a focus of such interactions. The potential of AI as a medium is not merely in the data and its classification, but also in the way that it provokes human reflection, interaction and adaptation of the existing points of view and practices; i.e., it aids self-accountability, which is of crucial importance to learning. This affordance was particularly manifest in the context of TARDIS, where the OLMs facilitated close inspection and reflection not only by the learners, but also by the practitioners. This allowed for access to the learners’ experiences with AI interpretations of the job interview performances giving an objective prop (with data of users’ actions and behaviours readily accessible) for the practitioners to pump the learners for explanations, for identifying the strengths and weaknesses in their performances and for devising plans for how to build on the former while addressing the latter. One striking observation from the TARDIS studies was the change in the quality of feedback and conversations using TARDIS OLM as opposed to relying on the learners’ and practitioners’ imperfect recall of the situations. As such the tool allowed to alleviate learners’ sense of being judged, putting them in control over the interpretations of their own experiences and over the directions they wanted to take their debriefing conversations with the

Accountability in Human and Artificial Intelligence Decision-Making …

55

practitioners. Here, TARDIS’ OLM became a medium through which practitioners could access subjective experiences of the learners and use it the basis for discussing those experiences through data rather than by pronouncing their judgements about the learners’ performance. As such, the OLM opened a possibility for the learners to assess themselves and to share both the possible reasons for their performance and plans on how to improve it in the future. TARDIS also provided a platform for discussions amongst the practitioners about their own practices and interpretations of the young people’s job interview performances, offering them invaluable means for continuous professional development—an affordance which has been taken forward by the practitioners participating in TARDIS in their practices beyond the life of the project (Porayska-Pomsta 2016).

6 Discussion and Conclusions In 1941, Fromm argued that the rise of the Nazis was helped by the human tendency to not want to have too many choices, preferring to surrender the responsibilities for making decisions to the few, thus leaving humans open to authoritarianism and ultimately fascism (Fromm 1941). Throughout history, the consequences of such a surrender were profound for inclusion, tolerance, democracy and human life. Presently, with the rise of the ‘intelligent machine’ our social biases already ingrained in our systems have been acutely exposed. Feeding on the pre-existing data, AI has exposed our shockingly exclusive systems. As such, it has also been shown to reinforce those biases and even as a tool to fuel social and political divide (Crawford 2018). The application of AI as such a tool is aided, it seems, by the same ease as described by Fromm, with which we delegate our decision-making and choices to others. This surrender of choice is not necessarily premeditated. Instead, we seem predisposed by nature to making decisions based on what we already know, rather than to processing new information. We are predisposed to choosing simpler strategies over those that require more effort to implement; i.e., we are by our very design lazy (Gavalas 2014; Satpathy 2012). According to Houdé (2013), we seem to lack cognitive inhibition in strategy selection between perceptual to logical brain, which on the whole requires us to make a heroic effort to engage in logical thinking and often leads us to making decisions based on first impressions, to jumping to conclusions and to acting parsimoniously (Epstein 1984). If reinforced, our resistance to changing and to anything that opposes our beliefs and knowledge (Strebel 1996; Gavalas 2014) is bad news for inclusion, for our learning and development, and for our AIenhanced future. Given this view, the hazards of AI for society do not reside in AI per se. Instead, they are located in our propensity for parsimony in complex decisionmaking that seems amplified by the AI’s unwavering ability to find optimal, rather than simplest, strategies in complex domains, releasing us from having to make an effort. With this in mind, accountability presents itself as a key prerequisite of inclusion in human and AI-enhanced contexts, rendering the process of making oneself or itself an accountable mechanism for overcoming our parsimonious tendencies.

56

K. Porayska-Pomsta and G. Rajendran

This is also where AI brings new and exciting opportunities in helping us challenge and question ourselves concretely, as a matter of habit, and also across time (since AI can make predictions about future events based on past occurrences). Such questioning has been shown to require advanced meta-cognitive competencies which are particularly beneficial to learning (Aleven and Koedinger 2002; Richardson et al. 2012). As we discussed in this chapter, such competencies are also fundamental to inclusion, to our development of ethically balanced moral judgement and to our selfdetermination (Moshman 2011; Paul and Binkler 1990), with positive implications for the excluding and the excluded. In Sect. 5, we offered concrete examples from our own research, showing how the application of particular forms of AI (AI humanoid agent technology and open learner models) can act as a catalyst in our understanding of ourselves and of others, and how they can provide a much-needed mirror onto our systems, established ways of thinking, prejudices and ultimately ignorance. The purpose of such a mirror is not to shame us, but to support us in becoming more informed about ourselves, more confident at recognising when our systems fail to cater for our needs, and in taking cognisant steps to change. Sometimes, all that is needed is a safe space in which to rehearse situations that make us anxious or to provide such safe spaces to others in which we can witness their full potential. Sometimes we need a stepping stone or a medium to help us achieve this, something that can act as an unthreatening trigger for us to try out our strengths. AI, with its ability to emulate our own behaviours, while clearly being different from us, can give us just this, provided that we acknowledge that change has to come from us and not from AI’s application alone. The outcomes can be revelatory to all concerned and may lead to changes in attitudes and support practices, as was the case in ECHOES. At other times, like in TARDIS, guided self-inspection is needed to empower learners to become self-efficacious, to self-reflect and to shed their inhibitions to share their reflections with others, while also offering the others a chance to see and to understand different perspectives and interpretations of the world. AI represents an increasingly powerful tool in this respect, through precise data and uncompromising, but nevertheless devoid of personal criticism (it’s a machine after all!) interpretation thereof that aids concrete inspection and questioning of ourselves, and a platform for planning and rehearsing next steps. To be such a tool, however, AI must be designed in ways that allow its decisions to be explainable and interpretable by humans. Furthermore, to be educationally efficacious, it also needs to allow for an appropriate adaptive management of human versus artificial autonomy, with humans being given the possibility to challenge and to edit AI’s interpretations of their data (Conati et al. 2018). It is important to appreciate that the success of AI in the context of educational inclusion, as in the examples offered in this chapter, depends critically on an understanding that AI does not offer a solution per se. It is not a magic bullet to cure our ills, but rather, and more usefully, it offers a very strong lens through which we can study the extent to which our ideas of ourselves as an inclusive society match the reality on the ground, and a tool for simulating and rehearsing different states of the world and behaviours therein. In this, AI both facilitates our accountability and requires to be accountable itself to be truly an empowering learning tool for all.

Accountability in Human and Artificial Intelligence Decision-Making …

57

As we have discussed extensively, accountability and inclusivity are prima facie frequently used concepts and have clear dictionary definitions, but delved deeper, truly workable definitions are not only hard to find, they also are de facto socially exclusive. By those definitions, the ways that we view and implement accountability and inclusion at the front line, are inflexible and slow to reflect our changing scientific knowledge, social understandings and aspirations. AI shows us that accountability and inclusivity are processes rather than ‘set states’, challenging our knowledge orthodoxies and putting to question our ‘ground truths’ (e.g. abnormality as a social construct vs. objective transparent criteria). It also offers ways in which inclusion as a social process can be democratised through empowering all stakeholders to own their data and influence how it is interpreted and shared. Viewed from this perspective, AI and educational inclusion share a potentially compelling, mutually informing future worth investing in. However, in order for this future to become a reality AI cannot be a purely engineering solution. Instead it needs to be co-created by multiple stakeholders in a human-centred, socially contextualised way, whereby accountability of human and AI decision-making is built-in explicitly not only into AI, but also into the educational and social systems within which AI is being applied.

References Aleven, V. A., & Koedinger, K. R. (2002). An effective metacognitive strategy: Learning by doing and explaining with a computer-based cognitive tutor. Cognitive Science, 26(2), 147–179. Baron-Cohen, S. (2017). Editorial perspective: Neurodiversity—A revolutionary concept for autism and psychiatry. Journal of Child Psychology and Psychiatry, 58(6), 744–747. https://doi.org/10. 1111/jcpp.12703. Bernardini, S., Porayska-Pomsta, K., & Smith, T. J. (2014). ECHOES: An intelligent serious game for fostering social communication in children with autism. Information Sciences, 264, 41–60. Brinkrolf, J., & Hammer B. (2018). Interpretable machine learning with reject option. De Gruyter Oldenbourg at—Automatisierungstechnik, 66(4), 283–290. Bull, S., & Kay, J. (2016). SMILI: A framework for inter-faces to learning data in open learner models, learning analytics and related fields. International Journal of Artificial Intelligence in Education, 26(1), 293–331. ISSN 1560-4306. https://doi.org/10.1007/s40593-015-0090-8, https://doi. org/10.1007/s40593-015-0090-8. Conati, C., Porayska-Pomsta, K., & Mavrkis, M. (2018). AI in education needs interpretable machine learning: Lessons from open learner modelling. CML Workshop on Human Interpretability in Machine Learning (WHI 2018), Stockholm, Sweden. Crawford, K., & Calo, R. (2016). There is a blind spot in AI. Nature Comment, 538(7625). Crawford (2018). https://royalsociety.org/science-events-and-lectures/2018/07/you-and-aiequality/. Curry, A. C., & Reiser, V. (2018). #MeToo Alexa: How conversational systems respond to sexual harassment. In Proceedings of the Second ACL Workshop on Ethics in Natural Language Processing (pp. 7–14). New Orleans, Louisiana, June 5, 2018. Davis, R. J. (1996). What are intelligence? And why? 1996 AAAI presidential address. The American Association for Artificial Intelligence. Davis, R., Shrobe, H., & Szolovits, P. (1993). What is knowledge representation? AI Magazine, 14(1), 17–33.

58

K. Porayska-Pomsta and G. Rajendran

Dias, J., & Paiva, A. (2005). Feeling and reasoning: A computational model for emotional characters. In Lecture Notes in Computer Science, Vol. 3808. Progress in artificial intelligence (pp. 127–140). Berlin, Heidelberg: Springer. Dobson, S. D., & Brent, L. J. (2013). On the evolution of the serotonin transporter linked polymorphic region (5-HTTLPR) in primates. Frontiers in Human Neuroscience, 7, 588. Dubnick, M. J. (2014). Toward an ethical theory of accountable governance. International Political Science Association meeting, July 19–24, Montreal. Epstein, R. (1984). The principle of parsimony and some applications in psychology. The Journal of Mind and Behavior, 5(2), 119–130. Flavell, J. H. (1979). Metacognition and cognitive monitoring: A new area of cognitivedevelopmental inquiry. American Psychologist, 34, 906–911. Fromm, E. (1941). Escape from freedom. New York: Reinhart. Gavalas, A. (2014). Brain parsimony and its effects on decision making. OKS Review, 3(1), EN, 1–14. Hernandez-Orallo, J., & Vold, K. (2019). AI extenders: The ethical and societal implications of human cognitively extended by AI, AAAI 2019. Houdé, O. (2013). The psychology of a child. Thessaloniki: Vesta Editions. Jones, H., Sabouret, N., Damian, I., Baur, T., André, E., Porayska-Pomsta, K., et al. (2014). Interpreting social cues to generate credible affective reactions of virtual job interviewers. IDGEI 2014, ACM. arXiv preprint arXiv:1402.5039. Lai, E. R. (2011). Metacognition: A literature review (Research Report, Pearson). https://images. pearsonassessments.com/images/tmrs/Metacognition_Literature_Review_Final.pdf. Lipton, Z., & Steinhardt, J. (2018). Troubling trends in machine learning scholarship. In ICML 2018: The Debates. arXiv:1807.03341. Moshman, D. (2011). Adolescent rationality and development. Routledge. O’Neil, S. (2008). The meaning of autism: Beyond disorder. Disability & Society, 23(7), 787–799. https://doi.org/10.1080/09687590802469289. Paul, R. W., & Binkler, J. A. (1990). Critical thinking: What every person needs to survive in a rapidly changing world. Rohnert Park, CA: Center for Critical Thinking and Moral Critique. Porayska-Pomsta, K. (2016). AI as a methodology for supporting educational praxis and teacher metacognition. International Journal of Artificial Intelligence in Education, 26(2), 679–700. Porayska-Pomsta, K., & Bernardini, S. (2013). In Sage handbook of digital technology research. http://dx.doi.org/10.4135/9781446282229.n30. Porayska-Pomsta, K., & Chryssafidou, E. (2018). Adolescents’ self-regulation during job interviews through an AI coaching environment. In International Conference on Artificial Intelligence in Education (pp. 281–285). Cham: Springer. Porayska-Pomsta, K., Rizzo, P., Damian, I., Baur, T., André, E., Sabouret, N., et al. (2014), Who’s afraid of job interviews? Definitely a question for user modelling. In International Conference on User Modelling, Adaptation and Personalization (pp. 411–422). Cham: Springer. Porayska-Pomsta, K., Alcorn, A. M., Avramides, K., Beale, S., Bernardini, S., Foster, M.-E., et al. (2018). Blending human and artificial intelligence to support autistic children’s social communication skills. ACM Transactions on Human-Computer Interaction (TOCHI) TOCHI, 25(6), December 2018, Article No. 35, New York, NY, USA: ACM. Prizant, B. M., Wetherby, A. M., Rubin, E., & Laurent, A. C. (2003). The SCERTS model: A transactional, family-centered approach to enhancing communication and socioemotional ability in children with autism spectrum disorder. Infants and Young Children, 16(4), 296–316. Prizant, B. M., Wetherby, A. M., Rubin, E., Laurent, A. C., & Rydell, P. J. (2006). The SCERTS ® model: A comprehensive educational approach for children with autism spectrum disorders. Brookes. Rajendran, G. (2013). Virtual environments and autism: A developmental psychopathological approach. Journal of Computer Assisted learning, 29(4), 334–347. https://doi.org/10.1111/jcal. 12006.

Accountability in Human and Artificial Intelligence Decision-Making …

59

Reisman, D., Schultz, J., Crawford, K., & Whittacker, M. (2018). Algorithmic impact assessments: A practical framework for public agency accountability. AI Now Institute Report, April 2018. Remington, A. (2018, July). Autism can bring extra abilities and now we’re finding out why. New Scientist. https://www.newscientist.com/article/mg23931860-200-autism-can-bring-extra-abilitiesand-now-were-finding-out-why/. Richardson, M., Abraham, C., & Bond, R. (2012). Psychological correlates of university students’ academic performance: A systematic review and meta-analysis. Psychological Bulletin, 138(2), 353. Russell, S. J., & Norvig, P. (2003). Artificial intelligence: A modern approach (2nd ed.). Prentice Hall. Satpathy, J. (2012). Issues in neuro-management decision making. Opinion: International Journal of Business management, 2(2), 23–36. Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., et al. (2017). Mastering the game of go without human knowledge. Nature, 550, 354–359. Strebel, P. (1996). Why do employees resist change? Harvard Business Review on Change (pp. 139–157), USA. Sutton, R. S., & Barto, A. G. (2000). Reinforcement learning: An introduction. The MIT Press. Terricone, P. (2011). The taxonomy of metacognition. Psychology Press. Weizenbaum, J. (1976). Computer power and human reason: From judgment to calculation. W. H. Freeman. Woolf, B. (2008). Building intelligent tutoring systems. Morgan Kaufman.

Ka´ska Porayska-Pomsta is Reader in Adaptive Technologies for Learning. She holds a M.A. Joint Honours in Linguistics and Artificial Intelligence (1997) and a Ph.D. in Artificial Intelligence (2003), both from the University of Edinburgh, School of Informatics. Prior to joining the UCL IOE in 2006 as Research Councils UK Academic Fellow, she worked as Research Fellow at the University of Edinburgh, Informatics. She is departmental Head of Research at the UCL Institute of Education and Member of the management committee for the Centre for Educational Neuroscience and the steering committee for the Institute of Digital Health. Gnanathusharan Rajendran is Reader in Psychology at Heriot-Watt University. He is interested in how new technologies might influence the social and cognitive development of children, especially children with developmental disorders like autism. He graduated with undergraduate and master’s degrees in Psychology from the University of Birmingham, followed by a Ph.D. in Developmental Psychology at the University of Nottingham. He was ESRC Research Fellow at the University of Nottingham, before becoming Lecturer at the University of Edinburgh, and then Lecturer and Senior Lecturer at the University of Strathclyde. He joined Heriot-Watt University in 2012 as Reader.

Artificial Intelligence in Education Meets Inclusive Educational Technology—The Technical State-of-the-Art and Possible Directions Gunay Kazimzade, Yasmin Patzer and Niels Pinkwart

Abstract Adaptive educational technologies as well as inclusion are two research fields that have a huge impact on current educational questions. Nevertheless, they are seldom seen together, which explains why there are not too many results in the intersection of the fields yet. This contribution discusses possible directions for combining Artificial Intelligence in Education (AIED) and inclusive educational technologies and shows some emerging practices. The introduction presents a state of the art on the history of adaptive learning technologies and on assistive technology (AT). A section that highlights the impairment/disability dimension of inclusion follows. Furthermore, emerging practices that combine accessibility/inclusion with Artificial Intelligence (AI) are discussed. The next section focuses on cultural dimensions of inclusion and the impact on AI in learning technologies. We then discuss the origins of cultural biases in technology and how to address this issue. Gender and ethnicity are connected to this cultural dimension and therefore are considered in this discussion as well. The conclusion describes the requirements for combining AI and inclusive learning technologies in the future. There is a need for more awareness of possible biases when creating learning systems and training algorithms with suitable data sets. Keywords Assistive technology · Bias · Cultural inclusion · Disability · Gender

G. Kazimzade (B) Weizenbaum Institute for the Networked Society, Technical University of Berlin, Berlin, Germany e-mail: [email protected] Y. Patzer · N. Pinkwart Humboldt-University Berlin, Berlin, Germany e-mail: [email protected] N. Pinkwart e-mail: [email protected] © Springer Nature Singapore Pte Ltd. 2019 J. Knox et al. (eds.), Artificial Intelligence and Inclusive Education, Perspectives on Rethinking and Reforming Education, https://doi.org/10.1007/978-981-13-8161-4_4

61

62

G. Kazimzade et al.

1 Introduction Today, Artificial Intelligence shapes not only areas such as finance, security, human resources, medicine but also education by providing different solutions for personalization, customization and adaptivity and creating student-oriented learning environments. Adaptive educational technologies are not a new line of research or development. They have a history of more than 40 years, dating back at least to the work of Jaime Carbonell on the SCHOLAR system (Carbonell 1970), an adaptive a computeraided instruction system which both asks and answers questions in dialogue with a student. Core developments in the field of adaptive educational technologies that have been made in the last decades include Intelligent Tutoring Systems, Educational Recommender Systems (Manouselis et al. 2011) or Open Learner Models (Bull and Kay 2010). With recent trends towards more data-driven approaches such as Educational Data Mining or Learning Analytics (Baker and Inventado 2014; Greller and Drachsler 2012), the field is currently flourishing. Examples of current research on adaptive educational technologies include multimodal learning analytics—approaches that are able to track and process human’s body postures, gestures, mimicry, body and eye movements or even physiological data in addition to “classical” log data from learning tools (Ochoa 2017). While some of these research approaches may still need some time until they can hit the consumer market, many leaning apps that can currently be found on the market are already adaptive in the sense that they can be configured to individual preferences, keep a user model and often combine this with personalized tasks. In order to allow for such system functionality, modern AI-driven educational approaches require data about the learners and their background. This may include information about the learner’s personality, preferences, abilities/disabilities, cultural, demographic, ethnic background as well as information with respect to the environment, interactions and feedback provided during the learning process. As computers are increasingly being used for educational purposes, personalization and adaptivity are not the only types of system functionality to gain relevance. To access digital content, some users need the support of assistive technology (AT). There exists a variety of AT (software and hardware) for different target groups. Screen readers, braille displays, screen magnifiers, speech input, adapted keyboards or mouses, screen keyboards and eye-tracking systems are just a few prominent examples—there exists much more assistive technology that is used by persons with disabilities and impairments. It attempts to compensate for non-existing or just partly functioning physical abilities to the extent possible. Most ATs have different adaptation options to allow for individual solutions for each user’s needs. Screen reader users, for instance, can choose between speech or braille output, language, the speed of speech output, voices for the output, shortcuts and other adjustments. Especially, blind people rely on screen readers as they cannot perceive content visually. But AT alone does not always resolve barriers. When developing websites, software or other digital offers, technical and design guidelines should be followed to ensure that compatible interfaces, structures and designs are implemented. The Web Content Accessibility Guidelines (WCAG) (World Wide Web Consortium 2008) as well

Artificial Intelligence in Education Meets Inclusive …

63

as the Universal Design for Learning (UDL) (CAST 2018a) framework supports developers in creating accessible solutions. With the importance of inclusion and an increasingly ageing population, the responsibility of software developers to contribute to accessibility can no longer be denied. User-centred design approaches that are commonly used to build products that fit their user’s needs need more awareness for heterogeneous user groups as well. Accessible user-centred systems combined with AT can be a huge step towards inclusion. There are already numerous approaches and solutions for special target groups that can also be useful within digitally supported learning processes: accessible PDFs (Darvishy 2018), universal accessible EPUBs (Schwarz et al. 2018), automatic adaptation of subtitle size according to screen size for better readability (Kushalnagar and Kushalnagar 2018), an app to teach the Arabic and French braille alphabet to blind children (Bouraoui and Soufi 2018), an accessible interaction model for data visualization in statistics (Godfrey et al. 2018) or an accessible pictograph interface for users with intellectual disabilities (Sevens et al. 2017). While both adaptivity/personalization and inclusion/accessibility are important facets of modern educational technologies and lines of technology development, research at the intersection of these two fields seems to be surprisingly scarce. In this chapter, we discuss the existing work and outline possible future lines of development, adopting both disability impairment perspectives and cultural perspectives on inclusion together with their respective implications for adaptive educational technologies.

2 The Disability Impairment Perspective This perspective partly follows the idea of the social model of disability as described by Shakespeare (2006) for instance. In this model, disability is seen as a problem that is caused by society and the barriers it causes for persons with impairments or disabilities. A resulting demand is that society needs to take the needs and requirements of all its members into account and make sure that it is accessible throughout. The social perspective thereby differs from the medical perspective that views persons with impairments or disabilities as physically or psychologically not fully functional individuals. The medical perspective thus approaches barriers on the individual level by medical treatments or technological support in the form of assistive technology for instance. Both perspectives can help reduce barriers for persons with disabilities and impairments and do not necessarily contradict each other. In the technology field, the individual perspective is addressed via various forms of assistive technology. The societal approach is supported by guidelines like the WCAG, but still very often under-represented in website or software development. One reason is that in many countries like Germany, there is still not enough legislation regarding accessible technology. When talking about learning with digital media, eLearning platforms are usually one core element. Though their usage widened extensively in the last years, they often

64

G. Kazimzade et al.

do not consider heterogeneous learner groups. This causes problems for different users like persons with disabilities or impairments, elderly learners or learners with little technical experience. There are solutions and partial solutions that fit special target groups like blind or visually impaired people (Worsley et al. 2018), people with impaired motor skills (Worsley et al. 2018), those who are deaf or hard of hearing (Volpato et al. 2018), those with intellectual disabilities (Ferreras et al. 2017) and others. Although these solutions are accessible to the targeted learner groups, they are not inclusive as their aim is not to address a truly heterogeneous audience but rather to compensate for a specific problem. Solutions that follow an inclusive approach to address as many heterogeneous learners as possible are rare. Most well-known eLearning platforms like Moodle or Blackboard have been made accessible in a post hoc manner—i.e. accessibility has been added after the core platforms were created. This process is costly and timeconsuming. Making an existing system accessible or inclusive can mean a complete redesign and reconstruction and therefore has its limits. For newly developed systems, accessibility/inclusivity needs to be considered right from the beginning. Existing guidelines on technical and design aspects like the WCAG and UDL should be followed. The WCAG are on a more technical level and aim at accessible website implementations. They differentiate between three levels of accessibility that can be achieved. UDL, on the other hand, “[…] is a framework to improve and optimize teaching and learning for all people based on scientific insights into how humans learn” (CAST 2018b). It addresses heterogeneity by asking to always provide and allow different approaches and ways of “engagement”, content “representation” and “action and expression” for learners (CAST 2018a). At the Humboldt-Universität zu Berlin, we currently develop the inclusive eLearning system LAYA (Learn As You Are) (Patzer and Pinkwart 2017). Our attempt is to follow the above-mentioned guidelines from the beginning and also involve potential users with different requirements. The platform has a modular structure that allows learners to adapt it to their needs. Content is represented as text, audio or video and can be perceived on a normal or simple difficulty level. The video player includes subtitles, captions, audio description and an additional sign language video and therefore allows for different ways of perceiving content. Users can choose their preferred way of content representation via their profile or directly above the content. The idea is to create a learning platform that enables joint learning for people with and without disabilities and impairments. In the long term, adaptivity is a feature that LAYA should fulfil. Another approach for inclusive eLearning is ATutor (Gay et al. 2009), a learning management system that was developed at the University of Toronto. It was the first eLearning system that was accessible to blind users. Like LAYA, the platform aims at accessibility. Systems like ATutor or LAYA can contribute to realizing inclusion in learning processes. They can support instructors or teachers to provide suitable learning material via one platform that is adaptable to different needs. Learners, on the other hand, can choose between different forms of representation and complexity. This way, similar content or learning material can be consumed, processed, exercised and expressed fitting the individual learner.

Artificial Intelligence in Education Meets Inclusive …

65

There are remarkably scarce research results (or concrete tools) at the intersection of adaptive learning with the related area of inclusive or assistive learning technologies. If one looks at well-known journals like the International Journal for Artificial Intelligence in Education (IJAIED) and the articles published in this venue within the last five years (2014–2018), there is no mention of the words inclusion or accessibility in any of the paper titles. There is only one article within these five years that can be related to accessibility as it looks specifically at blind students (D’Mello 2016). Likewise, looking at the AT journal “Technology and Disability”, there is not a single AI-related paper in the volumes between 2014 and 2018. This illustrates that there is only little exchange between these two research fields so far. These findings are surprising, as one strength of adaptive systems is that they allow for addressing the needs of diverse learner groups by adapting the user interface or different system components. Learning barriers like operating problems, different learning styles, cognition levels, or cultural backgrounds, or the need for additional information or different visualization could thus be resolved via AI-driven adaptivity. While these opportunities exist in theory, most of today’s (adaptive) educational systems are rarely designed for diversity. As an effect, users with disabilities or impairments might appear “irregular” in the data models underlying these systems—which is what results if, for example, machine learning-based feedback systems were trained only on data of “normal” users. This way, some chances for using adaptivity for inclusion are not used. One reason for this problem might be a lacking awareness of rather homogeneous developer teams for existing problems and barriers for persons with impairments or disabilities. Consequently, these are not considered when collecting data and therefore are either under-represented or completely absent in the resulting data sets. Human bias or ignorance is transferred into technical systems this way. More heterogeneous data sets that include minority groups like people with different impairments or disabilities are one necessity to create more inclusive learning systems. Involving these minority groups into development processes by working in heterogeneous teams would be a huge step in the right direction and help to involve heterogeneous needs naturally. More interdisciplinarity within development teams as well as in research endeavours would likely contribute to solve the existing lack of knowledge as well. In the long term, it would also lead to more awareness on all sides. But it is not always the data collection that excludes certain users via not being part of training data sets. Some systems explicitly exclude certain user types by design via the interaction forms they make use of (e.g. eye tracking will not work for blind users, speech interaction for the deaf, etc.). If eye-tracking AI systems for instance became mainstream educational tools, that would cause barriers for different learner groups. Blind people’s eye movement would surely not fit into the models developed for seeing people’s eye movement. And what about dyslexic people, whose eye movement can deviate from “normal” eye movement while interacting with text (Bellocchi et al. 2013)? Learners with attentional deficits might also have enormous problems to stay as focused as needed for eye tracking. Thus, if eye-tracking systems became mainstream educational tools, the exclusion for the mentioned learner groups would increase tremendously, simply because their eye movement patterns do not fit

66

G. Kazimzade et al.

the models of “normal” eye movement. This systematic discrimination and exclusion caused by AI could lead to a whole new dimension and range of exclusion than we have experienced so far in education. Besides this example, there are of course more system components that will simply not be usable if not carefully designed for inclusion: e.g. open user models tend to be complex and require skills to parse—what about cognitive impairments? There are indeed projects and initiatives that go beyond a single learning platform that is adaptable or adapts to the learner. Worsley et al. for instance describe two “[…] prototypes that utilize artificial intelligence and multimodal interfaces to advance more equitable participation among people with disabilities” (2018, p. 389). MultiCAD is a 2D/3D multi-modal interface based on AI that allows users with tremors to work with design software. Tangicraft allows blind users and those with low vision to play and participate equally in the game Minecraft by using “tangible blocks and haptic feedback” (Worsley et al. 2018, p. 390). Another project is “The Global Public Inclusive Infrastructure” (GPII), which aims to reach accessibility of the internet and all its information for everyone. One of their solutions to solve this issue is a “[…] cloud-based auto-personalization of digital interfaces based on user needs and preferences […]” (GPII.net 2018). These interfaces are supposed to be accessible form every digital device, including cash points or ticket machines for instance. Based on innovative navigation and interaction concepts like the GPII approach, user interfaces could transform into something that suits every single learner. Especially, elderly learners, technical novices and learners with disabilities would benefit from systems which recognize their needs and adapt the interface accordingly. This also includes the use of any form of assistive technology and compatibility with these, including multimodal forms of in- and output. An interesting issue would be an automated (and educational theory based) adaptation of learning content and exercises. For instance, this would allow for individualized differentiation even for large heterogeneous learning groups, including performance-based immediate adaptations. For “non-inclusive” settings, corresponding approaches exist—yet, the transfer to inclusive settings has to be demonstrated. Concepts like Universal Design for Learning should be considered for such a transfer. Our own work on inclusive gamification in learning systems shows an example of using badges, trophies and simple performance-based recommendations in an inclusive learning system (Patzer et al. 2018). The combination with AI allows thinking about more complex gamification elements and concepts. This gives the possibility to create gamification experiences that adapt to each user and therefore allow comparable gamification experiences for users with completely different needs. These interfaces should be available on different devices, which can help users to feel more comfortable in using different environments. This way, they do not have to learn to find their way again and again on every new device. Especially for those people who currently encounter barriers when using digital devices, this will be a huge improvement. If one thinks about adaptive learning platforms, analytical interfaces are common features to support learners’ self-reflection. Presented analytics can show various facets of user interaction related to platform usage, learning process, interaction with other learners and many more. They are typically presented as graphs and statis-

Artificial Intelligence in Education Meets Inclusive …

67

tics which presume visual abilities as well as cognitive abilities to understand these abstract data representations. Learners who are blind or visually impaired and those who have cognitive limitations often have very little chance to understand these statistics. Alternative text can be one alternative representation for persons with visual limitations (Godfrey et al. 2018). There are other approaches that work with vibrations and sound to visualize graphs for blind people (Brewster 2002; Cohen et al. 2005). AI needs to be trained with heterogeneous data sets that reproduce the heterogeneity of our societies if we do not want to systematically exclude people whose needs deviate from what is defined as “normal”. Models that are built need to be based on this heterogeneous data to enable accessible and inclusive systems.

3 Cultural Inclusion Perspective If we are investigating the inclusiveness of adaptive educational technologies, the disability impairment perspective is a common analytical lens. However, with the development of Artificial Intelligence systems, the future perspectives become wider and we should also focus on possible challenges with respect to cultural inclusiveness in AI-enabled educational technologies. Inclusiveness should be investigated not only with the consideration of students/users with special needs but also with respect to different student groups whom we aim to provide accessible, equally qualitative education. In that occasion, it is important to discuss cultural bias and cultural inclusion and to emphasize the possible challenges and opportunities which might occur as educational technologies further transform with the involvement of AI. Integrating culture into computer-based education systems and adaptive educational technologies has become demanding in the era of digitalization. It is crucial to address inclusiveness with respect to possible cultural biases in technology-driven education (Blanchard 2015). Collis discusses the importance of considering the cultural backgrounds of learners when designing computer-based learning because culture shapes learner’s values, perceptions and goals and determines how they respond to computer-based learning (Collis 2002). Thus, with the involvement of data-driven educational technologies, the cultural patterns have a value to discuss from the inclusiveness perspective. Exclusion and bias had become a hidden danger in Artificial Intelligence systems and it is a hidden danger for inclusive societies (Stone et al. 2016). One of the possible examples of bias in predictive and decision-making systems could be the training data sets which may contain historical or “bad” data with the intentional systematic/political discrimination and unintentional exclusion or the data which might be collected from a group of people that do not represent everyone. Also, data used to train a system can contain implicit racial, gender or ideological mappings which might lead to biases. As a result, biases are reflecting the systems we design which is used to make decisions by many, from governments to businesses as well as educational institutions and policymakers of adaptive educational technologies.

68

G. Kazimzade et al.

Scheel and Branch state that culture shapes and is shaped by language, ethnicity, religion, class, power, history, geography, ideology, aesthetics, gender, lifestyle, values, beliefs, traditions and ways of thinking and doing (Scheel and Branch 1993). Cultural bias and incompatible cultural behaviours, often without the people concerned realizing or recognizing their existence, contribute to what can amount to disconnection in the educational community (Dillon et al. 2007; Blanchard 2012). Developing a student-centred learning environment might lead to the exclusion of specific groups of students in AI-enabled education. As follows, adaptive educational technologies in some cases might risk segregating a user with respect to student’s gender, culture, race, ethnicity and overall cultural specifications. Facial recognition (emotion, eye tracking), speech recognition (feedback, emotion, assessment, etc.), eye tracking are several examples of data collection for training in AI-enabled educational technologies to develop a user-oriented approach for creating adaptive learning environments. However, studies show that most of the facial recognition systems are sensitive and discriminative towards people of colour (Buolamwini 2017). Also, a language barrier might be the obstacle for the speech recognition software to recognize and adapt to student’s needs. Moreover, the nonadaptive language of the AI-enabled educational systems and resources itself might reflect on the accessibility of the materials and can lead to exclusion of the specific groups (e.g. immigrants, refugees) from the quality education. In addition to that, a gender-specific approach in the automated grading, personalization and gamification of the learning is a possible reason for a huge gender gap in a specific subject. It is important to implement cultural inclusion in adaptive educational technologies in order to include every possible student groups in education process, to allow participation and to provide quality education to all possible user groups, prevent generation of knowledge making associations that leads to stereotypes in gender or ethnicity, to break undesirable, unfair associations, as well as to allow participating of heterogeneous groups in education. The discriminatory or injurious implications of symbolic power such as gender dominance and racism (Bourdieu 1982) becomes an ultimate risk in education, as the culture is reflected by values, ideas, knowledge and other sources we use to interact with the world around (Ginman and von Ungern-Sternberg 2003) and sociopolitical decision-making behaviour are reflected in intelligent systems and in data collection which are further used in adaptive educational technologies. Thus, AIassistive educational technologies based on the data and logic generated from this context could lead to the biased knowledge generation or decision-making. Further, with the use of smart robots, education became more accessible and assistive (Miller et al. 2008; Pop et al. 2013; Reich-Stiebert and Eyssel 2015). Several types of research have been made until today towards developing educational robots with an embedded cultural component. For instance, Bruno and colleagues propose a robot that is able to autonomously reconfigure its way of acting and speaking when offering a service to match the culture, customs and etiquette of the person it is assisting (Bruno et al. 2017). Correspondingly, in an educational domain, it is

Artificial Intelligence in Education Meets Inclusive …

69

proposed to discuss the cultural component of robots that enable adaptivity during the assisted learning process. From a data analytics perspective, in a learning scenario, the number of stakeholders may increase significantly when aiming at avoiding cultural bias. Besides students and teachers, different educational institutions, organizations, companies and governmental agencies might use the collected data for different purposes, research and policymaking. During and after the data collection, not taking into consideration the diverse groups could lead to biases in data and algorithms which are used for the systems developed for educational purposes. Thus, “[…] principles from a sociocritical perspective to guide institutions to address ethical issues in learning analytics” are suggested by some authors (Pardo and Siemens 2014). The first direction towards approaching bias in adaptive educational technologies should be the investigation of data sets used for training the system. Intelligence based on data that is too homogenous leads to biases in AI. It should be carefully examined whether the training data includes the right samples in the database and if it was tested with the students/users who were not part of the training sample. Also, there should be a strong focus on the development team and its diversity as well as the team’s sensitivity to recognizing biases. Furthermore, there is huge potential but also danger behind the use of AI-enhanced education technologies by children. Wrong implementation and design with a lack of emphasis on ethics might lead to destructive consequences in adaptive educational technologies. During the interaction between a student and a machine, a huge amount of biased data (offensive language, socio-political representations) might be generated if systems are designed without much care. Such effects can result from the users who operate racist or sexist language to train a system to use offensive language. Hence, there should be a robust spotlight on checks to identify a bad intent towards the system and what it learns from the users in case of real-time interaction for adaptivity/customization. Additionally, technologies built on adaptiveness in education should be carefully investigated with respect to their pattern matching. It would be critical, if unknown individual patterns were wrongly considered as a selection or distribution variable of existing popular patterns. This might lead to a lack of diversity in AI-enabled educational technologies due to misjudged learner behaviour. Thus, main investigations should direct towards algorithms and their reinforcement in only popular preferences. AI systems should be able to evolve dynamically as the users (students, teachers) change over time. Besides they should encourage the users to develop a more diverse and inclusive view of the world.

4 Conclusion There already is a lot of research on (partially AI-driven) adaptive learning technologies, as well as on AT and specialized solutions for people with disabilities or impairments. For the future, a deeper collaboration between these fields seems

70

G. Kazimzade et al.

desirable, to ensure that there are solutions that allow people with different needs and requirements to take part in digital learning equally. Currently, data sets do not contain the heterogeneity of learners with disabilities and impairments, which results in systems that simply can not identify certain patterns or classify them incorrect. Heterogeneous developer teams could be one step in the right direction, as they help to include accessibility and inclusion naturally. Existing guidelines like the WCAG and UDL can help developing suitable systems and mechanisms. Solutions like Multi-CAD, Tangicraft and the GPII project are a huge step towards adaptive and inclusive eLearning systems, as they can apply individual solutions for each user. Still, disability is not the only important dimension of inclusion. With increasing importance of data-driven educational technologies, also the cultural dimension of inclusion becomes more prominent. Reasons for a cultural exclusion in computerbased educational technologies are oftentimes hidden behind the mapping of a human bias into AI-enhanced education. Some system designers may simply overlook their own biases—as such, the built systems might be culturally biased unintentionally since they have been built on practices that are only applicable in certain countries or simply use the wrong metaphors. Other system designers may be aware of their biases but unable to take appropriate countermeasures, resulting in consequences such as discrimination, inaccessibility and wrong purpose built into AI-enabled educational technologies. To approach this problem, decision-makers, school leaders, researchers, developers, administrators, designers, students and parents must work together to verify interdisciplinarity and engage diverse student groups and teachers in agile product development. Moreover, an intercultural approach should be involved in educational technology policymaking, product development and testing to amplify diversity in teams. One of the possible solutions might be considering different ways of perception as a domain for the different ways of representation in education. Furthermore, educating teachers and students about diversity and inclusiveness is a fundamental approach towards addressing biases in adaptive educational technologies. As this paper argues, some noteworthy approaches are underway in both research and practice to address inclusiveness of AI-supported educational technologies—yet, the field has certainly not solved the existing problems. In conclusion, one could say that data sets need to be more heterogeneous to reproduce the diversity of learners. This not only holds for disability/impairment and cultural perspectives but for all possible dimensions of diversity. An exchange between research fields on AI and the different dimensions of inclusion would be beneficial for all stakeholders and especially learners. An international exchange of data sets would help addressing some of the above-mentioned issues. Opening up data sets with student data is of course tightly coupled to privacy issues and needs further elaboration. Data sets that are exchanged need accompanying metadata that helps understanding and interpreting the collected data. Yet, even with good metadata, having access to data might not be enough for truly addressing inclusive adaptive educational technologies. Initiatives that support and advance intercultural exchange and provide a framework might be necessary and could contribute valuable support to the whole topic of inclusive AI learning tools.

Artificial Intelligence in Education Meets Inclusive …

71

References Baker, R. S., & Inventado, P. S. (2014). Educational data mining and learning analytics. In Learning analytics (pp. 61–75). New York, NY: Springer. Bellocchi, S., Muneaux, M., Bastien-Toniazzo, M., & Ducrot, S. (2013). I can read it in your eyes: What eye movements tell us about visuo-attentional processes in developmental dyslexia. Research in Developmental Disabilities, 34(1), 452–460. Blanchard, E. G. (2012). On the WEIRD nature of ITS/AIED conferences: A 10 year longitudinal study analyzing potential cultural biases. In 11th International Conference on Intelligent Tutoring Systems (ITS2012) (pp. 280–285). Chania, Greece: Springer LNCS. Blanchard, E. G. (2015). Socio-cultural imbalances in AIED research: Investigations, implications and opportunities. International Journal of Artificial Intelligence in Education, 25(2), 204–228. Bouraoui, A., & Soufi, M. (2018). Br’Eye: An Android mobile application to teach Arabic and French Braille alphabets to blind children in Tunisia. In International Conference on Computers Helping People with Special Needs (pp. 357–364). Cham: Springer. Bourdieu, P. (1982). Language and symbolic power. Brewster, S. (2002). Visualization tools for blind people using multiple modalities. Disability and Rehabilitation, 24(11–12), 613–621. Bruno, B., Chong, N. Y., Kamide, H., Kanoria, S., Lee, J., Lim, Y., et al. (2017). Paving the way for culturally competent robots: A position paper. In 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), Lisbon (pp. 553–560). Bull, S., & Kay, J. (2010). Open learner models. In Advances in intelligent tutoring systems (pp. 301–322). Berlin, Heidelberg: Springer. Buolamwini, J. (2017). Gender shades: Intersectional phenotypic and demographic evaluation of face datasets and gender classifiers (MIT master’s thesis). Carbonell, J. R. (1970). AI in CAI: An artificial-intelligence approach to computer-assisted instruction. IEEE Transactions on Man-Machine Systems, 11(4), 190–202. CAST. (2018a). Universal design for learning guidelines version 2.2. URL: http://udlguidelines. cast.org/?utm_medium=web&utm_campaign=none&utm_source=cast-about-udl. Accessed February 11, 2019. CAST. (2018b). About universal design for learning. URL: http://www.cast.org/our-work/aboutudl.html#.W443qugzbb0. Accessed September 04, 2018. Cohen, R. F., Yu, R., Meacham, A., & Skaff, J. (2005). PLUMB: displaying graphs to the blind using an active auditory interface. In Proceedings of the 7th International ACM SIGACCESS Conference on Computers and Accessibility (pp. 182–183). ACM. Collis, B. (2002). Designing for Differences: Cultural Issues in the Design of WWW-Based CourseSupport Sites. British Journal of Educational Technology, 30, 201–215. https://doi.org/10.1111/ 1467-8535.00110. Darvishy, A. (2018). PDF accessibility: Tools and challenges. In International Conference on Computers Helping People with Special Needs (pp. 113–116). Cham: Springer. Dillon, P., Wang, R., & Tearle, P. (2007). Cultural disconnection in virtual education. Pedagogy. Culture & Society, 153–174. https://doi.org/10.1080/14681360701403565. D’Mello, S. K. (2016). Giving eyesight to the blind: Towards attention-aware AIED. International Journal of Artificial Intelligence in Education, 26(2), 645–659. Ferreras, A., Poveda, R., Quílez, M., & Poll, N. (2017). Improving the quality of life of persons with intellectual disabilities through ICTs. Studies in Health Technology and Informatics, 242, 257–264. Gay, G., Mirri, S., Roccetti, M., & Salomoni, P. (2009). Adapting learning environments with AccessForAll. In Proceedings of the 2009 International Cross-Disciplinary Conference on Web Accessibility (W4A) (pp. 90–91). ACM. Ginman, M., & von Ungern-Sternberg, S. (2003). Cartoons as information. Journal of Information Science, 29(1), 69–77. https://doi.org/10.1177/016555150302900109.

72

G. Kazimzade et al.

Godfrey, A. J. R., Murrell, P., & Sorge, V. (2018). An accessible interaction model for data visualisation in statistics. In International Conference on Computers Helping People with Special Needs (pp. 590–597). Cham: Springer. GPII.net. (2018). About the global public inclusive infrastructure (GPII). URL: https://gpii.net/ about. Accessed September 04, 2018. Greller, W., & Drachsler, H. (2012). Translating learning into numbers. Journal of Educational Technology & Society, 15(3), 42–57. Kushalnagar, R., & Kushalnagar, K. (2018). SubtitleFormatter: Making subtitles easier to read for deaf and hard of hearing viewers on personal devices. In: International Conference on Computers Helping People with Special Needs (pp. 211–219). Cham: Springer. Manouselis, N., Drachsler, H., Vuorikari, R., Hummel, H., & Koper, R. (2011). Recommender systems in technology enhanced learning. In Recommender systems handbook (pp. 387–415). Boston, MA: Springer. Miller, D. P., Nourbakhsh, I. R., & Siegwart, R. (2008). Robots for education. In Springer handbook of robotics (pp. 1283–1301). Berlin, Heidelberg: Springer. Ochoa, X. (2017). Multimodal learning analytics. In C. Lang, G. Siemens, A. Wise, & D. Gaševi´c (Eds.), Handbook of learning analytics (1st edn., pp. 129–141). Pardo, A., & Siemens, G. (2014). Ethical and privacy principles. British Journal of Educational Technology, IEEE, 45, 438–450. Patzer, Y., & Pinkwart, N. (2017). Inclusive E-learning—Towards an integrated system design. Studies in Health Technology and Informatics, 242, 878–885. Patzer, Y., Russler, N., & Pinkwart, N. (2018). Gamification in inclusive eLearning. In International Conference on Computers Helping People with Special Needs (pp. 154–158). Cham: Springer. Pop, C. A., Simut, R. E., Pintea, S., Saldien, J., Rusu, A. S., Vanderfaeillie, J., et al. (2013). Social robots vs. computer display: Does the way social stories are delivered make a difference for their effectiveness on ASD children? Journal of Educational Computing Research, 49(3), 381–401. https://doi.org/10.2190/EC.49.3.f. Reich-Stiebert, N., & Eyssel, F. (2015). Learning with educational companion robots? Toward attitudes on education robots, predictors of attitudes, and application potentials for education robots. International Journal of Social Robotics, 7(5), 875–888. Scheel, N. P., & Branch, R. C. (1993). The role of conversation and culture in the systematic design of instruction. Educational Technology, 33(8), 7–18. Schwarz, T., Rajgopal, S., & Stiefelhagen, R. (2018). Accessible EPUB: Making EPUB 3 documents universal accessible. In International Conference on Computers Helping People with Special Needs (pp. 85–92). Cham: Springer. Sevens, L., Daems, J., De, A. V., Schuurman, I., Vandeghinste, V., & Van, F. E. (2017). Building an accessible pictograph interface for users with intellectual disabilities. Studies in Health Technology and Informatics, 242, 870–877. Shakespeare, T. (2006). The social model of disability. In The disability studies reader (Vol. 2, pp. 197–204). Stefik, M. (1985). Intelligent tutoring systems. Artificial Intelligence, 26(2), 238–245 Stone, P., Brooks, R., Brynjolfsson, E., Calo, R., Etzioni, O., Hager, G., et al. (2016). Artificial intelligence and life in 2030: One hundred year study on artificial intelligence. Report of the 2015–2016 study panel. Stanford, CA: Stanford University. Doc: http://ai100.stanford.edu/2016report. Accessed September 06, 2016. Volpato, L., Hilzensauer, M., Krammer, K., & Chan, M. (2018). Teaching the national written language to deaf students: A new approach. In International Conference on Computers Helping People with Special Needs (pp. 163–171). Cham: Springer. World Wide Web Consortium. (2008). Web content accessibility guidelines (WCAG) 2.0. URL: https://www.w3.org/TR/WCAG20/. Accessed February 11, 2019. Worsley, M., Barel, D., Davison, L., Large, T., & Mwiti, T. (2018). Multimodal interfaces for inclusive learning. In International Conference on Artificial Intelligence in Education (pp. 389–393). Cham: Springer.

Artificial Intelligence in Education Meets Inclusive …

73

Gunay Kazimzade is working at the Weizenbaum Institute for the Networked Society which is run by Technical University Berlin in cooperation with Free University of Berlin, Humboldt University of Berlin, University of the Arts, Berlin, as well as University of Potsdam and Fraunhofer Institute for Open Communication Systems. She is also a Ph.D. student in Computer Science at TU Berlin under supervision of Prof. Dr.-Ing. Ina Schieferdecker. After applied mathematics and computer science degrees, she was involved in the computer science (CS) education and managed two social projects focused on women and children CS education. She trained over 3000 women and 300 children as well as students with the special needs. She is currently working on Research Group “Criticality of Artificial Intelligence (AI) - based systems” which focuses on raising the awareness of citizens for the capabilities and limitations of AI technologies, and breaking new ground for scientific research through social dialogue and inclusion. Her main research directions are AI in Education, biases in AI, AI in inclusive societies. Yasmin Patzer holds a Master of Education in special needs’ education and computer science education. Since spring 2016, she works as Research Assistant in computer science at HumboldtUniversity Berlin. There she is associated with the research group “Computer Science Education/Computer Science and Society” lead by Niels Pinkwart. She is significantly involved in the development of the LAYA system and is doing her Ph.D. on the implementation and usage of inclusive eLearning systems. Her main research interests are inclusive eLearning and inclusive computer science education. She is currently involved in an interdisciplinary research project on inclusive teacher education at Humboldt-University, as well as in a project that sensitizes employers to hire employees with special needs. Both projects are connected to the LAYA system. Niels Pinkwart studied Computer Science and Mathematics at the University of Duisburg from 1995 to 1999. As part of the COLLIDE research team at the University of Duisburg-Essen, he completed his Ph.D. studies in 2005 with a dissertation on collaborative educational modeling systems. After a postdoctoral position at the Human-Computer Interaction Institute of Carnegie Mellon University (2005/2006), he accepted offers for Assistant Professor and Associate Professor positions at Clausthal University of Technology. In 2013, he moved to the Humboldt-Universität zu Berlin where he heads the research group “Computer Science Education/Computer Science and Society”, the ProMINT Kolleg and the center of technology-enhanced learning located in the Professional School of Education of HU Berlin. In addition to his activities at HU Berlin, he acts as Principal Investigator at the Einstein Center Digital Future and at the Weizenbaum Institute for the networked society (German Internet Institute). Within the German Computer Science Association, he is currently Co-chair of the working group on learning analytics and member of the steering committee of the section on educational technologies.

Part II

Emerging Practices

Interactions with an Empathic Robot Tutor in Education: Students’ Perceptions Three Years Later Sofia Serholt

Abstract In 2015, and three years prior to the writing of this chapter, a three-month field study of a humanoid empathic robot tutor was conducted at a primary school in Sweden with children in grades 4–6. At that time, video analysis of the children’s interactions with the robot revealed that the children responded socially to the robot, but also that breakdowns often occurred during the interactions. Studies of robots in education are typically considered complete when the trial ends, which means that lasting or long-term implications of the child–robot relationship are seldom explored. The aim of this chapter is to explore children’s retrospective perceptions of the child–robot relationship in an educational setting. In a follow-up study at the school in question, the children responded to a survey and participated in discussion groups in which they were asked about their relationships with the robot, their recollections of breakdowns and how this has affected their normative perspectives of robots, as well as their views on robots in relation to the notion of inclusive education. A key finding in this study is that, when compared to their peers without robot experience, the students had become more critical towards the idea of emotion recognition in robots. Keywords Child–Robot Interaction · Emotion recognition · Robots · Education · Special needs · Students’ perceptions

1 Introduction to This Chapter Robots that tutor children in education can be understood as physical representations of a particular form of social AI—the kind of AI that is designed to emulate teacher behaviour. In their recent review of social robots for education, Belpaeme et al. (2018) conclude that ‘an increasing number of studies confirm the promise of social robots for education and tutoring’ (p. 6). Such robots extend the idea of standardized and S. Serholt (B) The University of Gothenburg, Gothenburg, Sweden e-mail: [email protected] © Springer Nature Singapore Pte Ltd. 2019 J. Knox et al. (eds.), Artificial Intelligence and Inclusive Education, Perspectives on Rethinking and Reforming Education, https://doi.org/10.1007/978-981-13-8161-4_5

77

78

S. Serholt

personalized learning content made possible with the use of computers to encompass also the social realm of what it means to undergo education and attend school: i.e. where children form social relationships and develop as human beings. Given the move towards the digitalization of schools, and a seemingly steady technological progress, robots may seem to make a lot of sense; not only are robots expected to be used to teach content in new and engaging ways, they are also expected to interact with children much like teachers do. In theory, robots could therefore facilitate inclusive education by moving away from using technology primarily for performative assessments of individual students (i.e. the traditional view), towards paving way for more social and relational practices in education that are landmarks of inclusion (Haug 2016). Despite these possibilities, introducing robots in schools on a wider scale would entail a significant transformation of what it means for children to go through the educational system, making it wise to proceed with caution until thorough investigations of their effects have been performed. In studies on child–robot interaction (CRI) in educational settings, along with human–robot interaction (HRI) studies, in general, the need for long-term and naturalistic studies have long been emphasized (Šabanovi´c 2010; Severinson-Eklundh et al. 2003). However, implementing social robots in educational settings for several years is not only impractical given their current technical restrictions (Belpaeme et al. 2018), but it also introduces a wide range of ethical issues (Serholt et al. 2017; Sharkey 2016). Thus, little is known about the long-term effects of robots in education. Currently, research is conflicted when it comes to the effects that robots have on children’s learning as this will depend on a variety of things such as the learning content, the robot embodiment, the setting and the children. Nevertheless, education is about more than just learning. If robots can contribute to a more inclusive educational environment, then that might be considered an added value in itself. And, this is much more a question of subjective experience than it is about measurable effects. Yet, little is known about children’s perceptions of robots beyond the novelty effect. Indeed, most such studies are carried out either in conjunction with a particular intervention (e.g. Liles and Beer 2015; Obaid et al. 2018; Westlund et al. 2018), or without any intervention at all (e.g. Lin et al. 2009; Serholt et al. 2016). Taking into account teachers’ concerns on the matter, children may not fully realize the extent to which robots could have an effect on them (Serholt et al. 2017), making it pressing that children are given some time and experience to reflect upon what it might really mean to implement social robots in education on a wider scale. The aim of this chapter is to propose and execute a particular research approach that, to my knowledge, has not yet been explored in the CRI field. That is, to conduct an extended follow-up study of an earlier educational intervention with a robot tutor. The advantage of this approach is twofold: (1) the novelty effect has likely worn off in most cases, and (2) research instruments can be tailored to what actually happened during the intervention. Of course, there will be drawbacks and limitations to this approach, but these will be discussed later on. Thus, in the study that will make up this chapter, I returned to a school where a robot tutor had been implemented and studied three years earlier to explore the students’ perceptions of the experience, but

Interactions with an Empathic Robot Tutor in Education …

79

also to see how they, given this experience, reason about robot capabilities, especially in regard to the theme of the current book, i.e. inclusive education.

2 Background The intervention, carried out in 2015, was a three-month field study of a humanoid empathic robot tutor at a primary school in Sweden. The students who took part in the study attended grades 4–6, and the robot tutor was developed within the three-year European project EMOTE.1 To briefly describe the set-up that was developed within the project, it suffices to say that it consisted of a NAO T14 robot torso from Aldebaran Robotics,2 which was attached to an interactive table surface where students could play either an individual map-reading game, or a collaborative sustainable development game in pairs. In the first game, the robot was designed to guide the student through the task much like a tutor, while in the second game, the robot acted as a co-player and mentor who tried to facilitate active discussions between the students on topics of sustainability. Common for both games was that the robot was supposed to behave empathically by inferring students’ emotional states and to respond accordingly. For neither game could the robot understand spoken language, although it could speak itself. More detailed description of the set-up and games can be found in Obaid et al. (2018) and Serholt (2017). At the time of the field study, video analyses of the students’ interaction sessions with the robot were carried out, resulting in two separate publications: the first explored their social engagement with the robot (Serholt and Barendregt 2016), and the second focused on interaction breakdowns (Serholt 2018). In the first paper, the results showed that the students expressed social engagement with the robot using the different communicative channels analysed (gaze, gestures, verbal communication, and facial expressions). Over time, social engagement declined somewhat, but this seemed to be related to their eagerness to begin the task, hence less of a focus on the robot itself, rather than a loss in general engagement. In the second paper, occurrences of interaction breakdowns for six of the students (or 12 children if considering also the partner in the sustainability game) were analysed across all of their sessions with the robot, where a total of 41 breakdowns were discovered and analysed through interaction analysis. The nature of these breakdowns varied, but were explained to be related to the robot’s inability to evoke initial engagement and identify misunderstandings, confusing scaffolding, lack of consistency and fairness, or controller problems, i.e. problems of a more technical nature. Since three years had gone by since the field study, I was doubtful that the children would remember much about the study, which makes this study quite explorative. Nevertheless, the following research questions guide this work: What do the children remember from their interaction experiences with the robot tutor? Do they recall 1 www.emote-project.eu. 2 The

company has since the project took place been purchased by Softbank Robotics.

80

S. Serholt

breakdowns occurring, what they were, and have an opinion on whose fault it was? What are their perceptions of the robot tutor and the experience? Given their experiences, do their normative perspectives on robots differ from those of children who do not have experiences interacting with the robot, i.e. the students who enrolled in the school only after the field study? My final question centres on how the children consider robots to factor into the idea of inclusive education; for this, I have adopted a broad definition which describes inclusion as an arena, wherein all students are able to participate regardless of capability. As inclusive education is a highly contested notion already (Haug 2016), my hope is that this broad definition can open up speculative discussions where the students can consider a variety of ways to realize an inclusive educational environment in the future.

3 Methodology The study took place in the form of a mixed-method study at the primary school for two consecutive days. Research instruments and empirical data have for the sake of this chapter been translated from Swedish to English.

3.1 Procedure and Participants On the first day of the study, I introduced the aim of the study after which the students filled out an individual questionnaire. On the second day, the students were introduced to the robot Pepper from Softbank Robotics, which is a 120-cm-tall humanoid robot capable of some social interaction through its autonomous life feature. They were then allowed to interact with the robot for 5–10 min before engaging in group discussions. In line with the provisions of the CODEX Rules and Guidelines for Research by the Swedish Research Council, each participant provided informed consent, and legal guardians provided assent for participants under the age of 15. A total of 34 students who were at the time of the current study in grades 7–9, and aged between 10 and 16 (M = 14.39; SD = 1.223), consented to participate in the study (21 boys and 13 girls). Fifteen of the students confirmed that they had taken part in the initial field study, whereas 19 had not participated since they were new at the school.

3.2 Questionnaires The questionnaire consisted of the following four parts: (1) demographic information (grade, age, and gender), (2) basic information regarding their participation in the

Interactions with an Empathic Robot Tutor in Education …

81

earlier field study (what of the two activities they had done with the robot, and their enjoyment of each3 ), (3) retrospective questions on incidences of breakdowns in the interactions and their views of the same, and (4) a set of questions on normative perspectives of robots in education and society adopted from (Serholt et al. 2016), yet slightly modified to address earlier limitations of the instrument.

3.3 Group Discussions During the group discussions, the students formed groups of 3–4 students in each (eight groups in total), decided by the class teacher, and audio recorders were used to capture the discussions. In the 7th grade class, a teacher and an assistant teacher oversaw the study and intermittently moderated the discussions. The discussions centred around six questions that were handed out on note cards, two4 of which are considered in the current study: 1. What do you remember from using Nao? 2. If you envision that school should be inclusive: a school for everyone where all children are able to participate regardless of capability, how do you think that the use of robots such as Pepper and Nao can help or hinder this?

4 Results and Discussion In this section, the results of the study are presented and discussed, beginning with the findings from the questionnaire, followed by the group discussions.

4.1 Questionnaire In the following, the results of the questionnaire are divided into three subsections, namely: breakdowns, perceptions of the earlier study, and finally, normative perspectives on robots. In cases where students decided to check multiple responses (e.g. ‘yes’ + ‘no’) or add a free written response (e.g. ‘maybe’), these were recoded as ‘don’t know’.

3 These

questions were also posed to the children in the earlier study, and were part of a longer questionnaire on perceived learning, previously published in Deliverable 7.2 of the EMOTE project. 4 The remaining questions were used to elicit design considerations for an educational robot under development in the research project Student Tutor and Robot Tutee (START).

82

S. Serholt

4.1.1

Breakdowns

Indicators In Serholt (2018), three indicators were used to identify when breakdowns occurred in the interaction between the child and the robot, i.e. adverse emotional reactions, sustained inactivity or off-task activity, as well as requests for researcher assistance. In the current study, four Likert scale items were used to gauge to what extent the students remembered such situations. Figure 1 shows the frequency of students who responded to each option on the four questions, showing that 47% of the students recalled becoming irritated, frustrated or confused during the interaction, but no students recalled (or were willing to admit) that they became angry or sad. The inability to continue on in the task, and hence, becoming inactive, was more common for students to affirm (73%), as was the need for researcher assistance (80%).

Breakdown Situations When it comes to the different explanations for breakdowns, these were illustrated in the form of situations for the students to check if applicable. Thirteen out of the 15 students checked that they had experienced at least one of the situations, where students on average marked their affirmation for 3.87 situations (SD = 2.875). Frequencies for individual students are presented in Fig. 2 showing that there was a wide spread in the number of different types of breakdown situations encountered for individual students, ranging from no breakdowns at all, to nine different breakdown types. Breakdown indicators

16 14

Not at all

12 10

Once or twice

8 6

Sometimes

4 Pretty often

2 0 When I played with Nao, I became irritated, frustrated or confused

When I played with Nao, I got angry or sad

When I played When I played with Nao, it with Nao, Sofia became so messy had to come in or difficult that I and help me got "stuck" and could not continue on with the task

All the time Don't know/don't want to answer or missing response

Fig. 1 Frequency of student responses to each Likert scale option (N = 15)

Interactions with an Empathic Robot Tutor in Education …

83

Number of breakdown situations remembered

10 9 8 7 6 5 4 3 2 1 0 S1

S2

S3

S4

S5

S6

S7

S8

S9

S10

S11

S12

S13

S14

S15

Fig. 2 Number of unique breakdown situations recalled by individual students (N = 15), ordered from least to most

The number of students to check individual breakdown situations is shown in Fig. 3. Given the variety in students’ responses to individual items, it seems that they were able to recall details about the experience that I had not anticipated, where almost half of the students responded that they were unable to hear the robot’s verbal communication, that there were contradictions between what the robot was talking about and its deictic gestures, and also that the interactive surface was unresponsive to touch. A third of the students experienced that the robot stopped working completely, and four of the students responded that they tried to verbally ask the robot for help without success. Taken together, these results show that almost all students encountered breakdowns during the intervention, yet, it should be noted that it is not established how often each of the situations occurred. Moreover, when asked if there were additional breakdown situations that occurred during the interaction not mentioned in the questionnaire, only one student added: ‘The compass was in English. It would have been easier if it was in Swedish’. This provides slight indication that the questionnaire covered most of the breakdown types that occurred during the study, despite only six students (12 when including the collaborative game) being included in that particular analysis. Yet, three of the students also wrote down that they did not remember enough of the study to be certain.

Ascription of Blame When asked who they found to be at fault for the breakdowns occurring, half of the response frequencies indicated that no blame should be ascribed. However, 53% of the students considered either Nao or themselves to be at fault (see Fig. 4). The fact that some students blamed the robot for the breakdowns suggests that they ascribed

84

S. Serholt Different breakdown situations remembered

Problems with the robot or touch-interface (including false signifiers)

Lack of consistency and fairness

Confusing scaffolding

The robot's inability to evoke initial engagement and identify misunderstandings

0

1

2

3

4

5

6

7

Nao explained things in an unclear or bad way so that I/we did not know how or what to do. It was impossible to hear what Nao said. I/we tried to ask for help, but Nao didn't understand. I/we tried to show Nao things in the game by gesturing (e.g., pointing), but Nao didn't understand. I/we tried to express emotions toward Nao, but Nao didn't understand. Nao asked a question, but didn't seem to understand my/our response. Nao helped me/us in the wrong way so that the games didn't go well. Nao interrupted or talked all the time so that I/we lost focus on the task. Nao said things that felt dumb or strange in the moment, e.g., that I/we should "take it nice and easy". Nao was talking about something, but seemed to point in the wrong direction. During the map reading game, Nao said that I should read all 3 clues, but there was only 1. Nao built something in EnerCities so that the oil ran out and we lost. Sometimes we had to skip Nao's turn during EnerCities since he usually made us lose. Nao didn't do what we (my classmate and I) wanted when we played EnerCities. The games reacted strangely or not at all when I/we tried to tap on something on the touchtable. Sometimes the games progressed to the next step/player without me/us having selected something. In EnerCities, we accidentally tapped a button which made the game end even though we didn't want to. Nao stopped working completely. Nao said I was wrong, but it turned out that I was right. Nao said that I was right when I actually pressed the wrong symbol. Number of students who responded affirmatively to each item

Fig. 3 Number of students who responded affirmatively to individual breakdown situations (N = 15)

Interactions with an Empathic Robot Tutor in Education …

85

Ascription of fault or blame

7 6 5 4 3 2 1 0 My fault

Nao's fault Classmate's Researcher's No one's fault fault fault

The I never technology experienced simply did any not work breakdowns

Fig. 4 Students’ perceptions of fault for encountered breakdowns, where some students selected more than one option (20 responses in total)

some form of agency to the robot, wherein it was (or should have been) capable of intentional action. It is alarming that some students blamed themselves for the breakdowns, as the situations were deliberately framed in a way to not suggest that the students were at fault. That the researcher should be blamed for the breakdowns is obvious as I took part in designing the robot and held the overall responsibility for the study. In this sense, it was positive to see that at least one student was willing to select this option despite the risk of displeasing the researcher, which is a common challenge when developing research instruments for children (de Leeuw 2011). Perhaps conducting extended follow-up studies when children are a bit older is also a way to circumvent issues with social desirability.

4.1.2

Perceptions of the Intervention

Enjoyment of the Games For the items on the questionnaire that were also posed to the students in the earlier study regarding their enjoyment of the two games, a Wilcoxon signedrank test showed that there were no significant changes for individual students in their perceived enjoyment over the three years for either the map-reading game (n = 10, Z = −0.707, p = 0.480) or the collaborative sustainability game (n = 13, Z = −0.544, p = 0.586). Figure 5 shows the frequencies of selected response options on a Likert scale for each game, indicating that there was a slight decrease in enjoyment overall.

86

S. Serholt

Enjoyment of the two learning applications 12 10 8 6 4 2 0 Map-reading game (old) Strongly disagree

Map-reading game (new) Disagree

Sustainability game (old)

Neutral

Agree

Sustainability game (new)

Strongly agree

Missing

Fig. 5 Frequency with which the students selected different response options to the questions: I enjoyed working with Nao and (game), from 2015 and 2018, respectively

Ontological labels used to describe Nao Free response labels Predefined labels

0

1

2

3

4

5

6

7

8

Human Toy Computer Animal A robot A teacher or friend Difficult to know (what Nao is) Unknown creature dangerous for humankind

Fig. 6 Frequency of labels used to describe the Nao-robot, including free response labels

Ontological Status of the Robot In terms of the students’ perceptions of the robot’s ontological status, Fig. 6 presents the frequency of selected labels. The graph illustrates that the majority of students considered Nao to be best described as a computer. Most students selected multiple labels with some combination of computer and other descriptors; one student expressed fear of the robot as demonstrated by the last free written label. Only one student described the robot solely as human.

Interactions with an Empathic Robot Tutor in Education … 10

87

Student ratings on the experience and perceived liking Strongly negative

9

Fairly negative

8 7

Neutral

6

Fairly positive

5

Strongly positive

4

Nao is not capable of liking someone

3 2 1 0

Would you have How was your Did Nao meet the Did it feel as Was it important liked to keep Nao experience as a expectations you though Nao liked to you that Nao at your school? whole having Nao had before or liked you? you? at your school? during the study?

Fig. 7 Student response frequencies to affinity items, where the answer option ‘Nao is not capable of liking someone’ was only available for the final two items (N = 15)

Affinity Towards the Robot In terms of overall enjoyment of having the robot at their school, whether the experience corresponded to their expectations of robots, as well as their perceptions regarding the robot’s affinity towards them, five Likert scale questions were asked in the questionnaire. The results shown in Fig. 7 indicates that the majority were positive about keeping the robot at their school and the experience as a whole; in comparison, they were slightly more neutral about whether or not the robot met their expectations. Regarding their perceptions of the robot’s affinity towards them, and whether such affinity was important to them, the students were more negative. The final question on the first part of the questionnaire was an open-ended question about whether the students wanted to add anything else that had not been asked already. Four students each added a short comment: It was fun, something different. (Male, 9th grade). He had no legs, but it was fun anyway, and the games were fun. (Female, 8th grade). Thought that the robot really didn’t do anything, would have worked equally well with just a voice. (Male, 9th grade). I thought it was fun and exciting, but since it was such a long time ago, it’s difficult to remember details and if it malfunctioned. (Female, 9th grade).

88

S. Serholt

Normative perspectives on robots Students talking to robots (Q21) Robots helping with schoolwork (Q22) Humanlike robots in schools (Q19) Students talking to robots with peers (Q24) Students trusting in robots (Q26) Students having frienships with robots (Q23) Robots expressing emotions (Q20) Robots recognizing emotions (Q30) Robots being responsible (Q31) Robots recording interactions (Q29) Robots grading schoolwork (Q25) Robots teaching young children (Q27) Robots replacing teachers (Q32) Robots deciding things in society (Q28) 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Yes

I don't know

No

Fig. 8 Normative perspectives on robots for all students (N = 34), where questionnaire items have been rephrased to fit into the graph

4.1.3

Normative Perspectives on Robots

The final section of the questionnaire comprising questions on the students’ normative perspectives on robots and their capabilities in educational contexts and society in general, were given to all students. The items were phrased as yes/no-questions (e.g. ‘Would you like a robot to grade your schoolwork?’ or ‘Could you trust a robot?’). Figure 8 presents the results, sorted from the top according to the level of positive responses.

Study Comparison Comparing these results against a different study using the same instrument (Serholt et al. 2016), responses at the current school showed both agreement and disagreement on the different items. In both studies, a majority of students were positive towards having humanlike robots in schools, speaking with robots by themselves or in front of their peers, as well as asking robots for help with their schoolwork. Moreover in both studies, a majority of students were negative towards robots grading schoolwork, teaching young children, replacing teachers and deciding things in society. In both studies, the samples were evenly split regarding the idea of developing friendships with robots.

Interactions with an Empathic Robot Tutor in Education …

89

Given the various discrepancies between the earlier and the current study (i.e. differences in time, methodology and sample size), these comparisons should be interpreted with care. Nevertheless, it suggests that normative perspectives on robots for students in general are consistent with what we found a few years ago with different students across three European countries. However, while a majority of students in the earlier study opposed the idea of robots recording interactions (69%), this was not as clear in the current study where only 41% were negative and quite a few were positive or uncertain. Further, a slight majority in the current study were positive towards trusting robots, which the students in the earlier study were more uncertain about. In addition, a majority of students in the current study were negative towards robots taking responsibility for their mistakes, compared to a slight majority in the earlier study. This sheds light on the finding presented earlier where only five students assigned blame to the Nao-robot for the breakdowns that occurred during their interaction sessions in the sense that assigning blame to robots may not have been seen as a reasonable option. Interestingly, while a majority of the students in the earlier study were positive about robots recognizing and expressing emotions, only half of the students in the current study agreed with this sentiment. Why this might be the case is explored next.

Divergent Findings for Students with Robot Experience In order to explore whether there were any differences in normative perspectives between the students who had been part of the earlier intervention with the robot tutor (G1), and the new students at the school (G2), the response frequencies for the two groups were compared. These results showed that there were statistically significant differences for only one of the items, namely that the students in G1 were significantly more negative than the students in G2 towards the idea of robots analysing their emotional states based on, e.g. facial expressions (Mann–Whitney U = 64.000, nG1 = 15, nG2 = 19, p = 0.006), as shown in Fig. 9.

Fig. 9 Comparison of response frequencies for G1 and G2

16 14 12 10 8 6 4 2 0

Comparison of Q30 between groups

G1

G2

Would you like a robot to be able to analyze your emotions based on, e.g., your facial expressions and pulse? (Q30)

Yes

I don't know

No

90

S. Serholt

Given the small sample of students, and only one item addressing this construct, this finding should be interpreted with care. Nevertheless, it is noteworthy that students who have experienced interacting with an ‘empathic robot’ became more critical towards the idea of robots analysing emotions. One explanation for this could be that the experienced students considered the technical limitations of robots, making them sceptical about the capability of robots to recognize emotions, but this seems unlikely giving the framing of the question. Another possibility could be that they did not reflect upon Nao as that of an empathic robot, i.e. that the emotion perception capabilities of the robot simply went unnoticed during the intervention. Yet this seems unlikely given that this particular trait was often emphasized during the study: for instance, when the intervention began, it was explained to the students that the robot would not be able to understand their speech, but that it would try to understand them based on their feelings through facial expressions among other ways. The students also wore skin conductance bracelets for their interaction sessions, which were explained to be a device that could measure if they became stressed. After the study, the students also completed questionnaires in which the robot’s empathic traits were evaluated. Hence, the students were likely aware that the robot was supposed to perceive and respond to their emotions. Thus, the plausible explanation then becomes that the students became critical towards this particular capability of robots because of the intervention. It could be the case that the students did not feel that the empathic traits of the robot added value to the learning tasks, or perhaps that they found the idea frightening. Whatever the explanation, it is not possible to discern whether such correlations can be generalized or if it is simply an effect of this particular intervention. However, it illustrates that the CRI field can benefit from extended follow-up studies to evaluate the possible success of similar interventions when the novelty effect is no longer an issue.

4.2 Discussion Themes The transcriptions of the audio-recorded group discussions were analysed inductively by means of thematic analysis (Braun and Clarke 2006). As the two discussion questions posed to the students aimed at addressing two different research questions in this study, the results of each are presented in individual subsections. The main themes are indicated through unnumbered headings, whereas subthemes are presented in italic font in each section. Pseudonyms are used for student quotes, and asterisks denote students who took part in the intervention, while group affiliation is illustrated using [grade: group number].

4.2.1

Memories of Nao at the School

At the beginning of the group discussions, the students talked about their memories of having Nao at the school. In all of the groups (N = 8), there were at least

Interactions with an Empathic Robot Tutor in Education …

91

one student who had participated in the earlier intervention. Two main themes with accompanying subthemes were developed based on their discussions.

Factual Memories of the Experience In their attempts to remember the study and share these memories with each other and the group members who had not participated in the intervention, all groups discussed factual memories of the experience. First off, references were made to the appearance of Nao and the technical set-up, such as the small size of Nao, in relation to the much larger Pepper-robot present. They also mentioned the colours on Nao, i.e. white and orange, as well as the fact that the robot was just a torso that lacked legs and was mounted to a stand. In reference to the chest-mounted tablet on Pepper, the students explained that Nao did not have a tablet but instead sat next to a tablet the size of their desks. To interact with Nao, students explained that they entered a small room, a group room, which was sometimes very warm. They also discussed the two educational applications, but most students had trouble remembering and/or explaining more about the applications than the fact that the first game was about map-reading and geography, and the second was about building a sustainable city together with a classmate. A few students went further and mentioned that the map-reading game was about them improving their learning in geography, navigation and that clues were used. In some groups where only one student from the earlier study was included, the student could complain about not remembering much, whereas in groups where there were more than one, the students tried to trigger each other’s memories. In this excerpt from Group 9: 5 comprising three students who were part of the earlier study, two students were discussing the games which the third student had trouble remembering: Jason*: “Do you remember now? Anything?” Levi*: “Yeah… Yeah, now I remember some kind of map?” Jason*: “And then there was that other game.” Levi*: “Yeah, but there was some kind of map-game where you traveled to different places or something.” —Group 9: 5

Additionally, students discussed robot abilities, e.g. that Nao spoke Swedish. Against ideas conveyed by the students who had not been part of the intervention that robots can think and reflect autonomously, one student was seemingly annoyed by his group members’ overconfidence in what Nao could accomplish, stating that robots are simply computers in robot bodies, and he did not care for them. In two of the groups, breakdown situations as described in the previous questionnaire were addressed explicitly, as shown in the following excerpts: Josef*: “Did you guys ever run into like problems ever?” Walter*: “He shut down once.” Josef*: (laughing)

92

S. Serholt Caitlin*: “I don’t remember that.” Josef*: “I was like this the last time, so it got all strange. It was like, ‘ye ye ye, I’m going to kill you now!’ ” Caitlin*: “I remember the clues, that there was some problem with the clues.” —Group 9: 2

Here, Josef tried to elicit information regarding the breakdown experiences of his group members. Walter then confirmed that he indeed encountered technical problems with the robot, i.e. that the robot shut down. Caitlin, in contrast, explained that she had a different experience, namely that she experienced a ‘lack of consistency and fairness’, with emphasis on consistency, when the clues needed to solve the mapreading task were missing (indeed, this was a common glitch in the application that required the researcher’s assistance). Josef, however, described a different situation that was not mentioned in the questionnaire that he had trouble describing. A similar situation can be seen in this second illustration of breakdown talk: Jason*: “It was a little chaotic. But then there was that game where you were supposed to build like a city.” Luke*: “Yes, but I remember this one time when I was supposed to go this way, and it pointed in the other direction.” Jason*: “Yeah, I think that happened for me too. There was chaos for me a few times. Luke*: “And then it froze.” Jason*: “Mm.” —Group 9: 5

In this excerpt, Jason referred to situations that he interpreted as ‘chaotic’ during the interactions, but, like Josef in the earlier excerpt, he was unable to specify exactly what happened. Moreover, he confirmed that the concrete situation of ‘confusing scaffolding’ described by Luke also happened during his interactions. Luke went on to mention that he also encountered technical problems with the robot when it ‘froze’. As these two excerpts illustrate, there may have occurred certain breakdown situations during the interactions that did not surface in my earlier analysis, which the students found difficult to describe in words. An obvious limitation here is that the questionnaire may have biased the students to think about and remember specific breakdowns occurring. Indeed, four of the breakdowns discussed denoted situations that had been mentioned in the questionnaire (i.e. that the robot ‘shut down’ or ‘froze’ could be in reference to ‘Nao stopped working completely’, the ‘problem with the clues’ was indeed mentioned in the questionnaire, and that ‘it pointed in the other direction’ was one of the most frequently selected breakdown situations in the questionnaire). However, it is possible that this simply provided the students with the appropriate language to be able to discuss the situations, especially in light of the other attempts to describe breakdown situations (‘chaos’ and that Nao started behaving strangely [‘I’m going to kill you now!’]). Regardless, these experiences could have affected their perceptions of robots in general, especially if, as suggested by Josef’s statement, some students found the experience unpleasant or threatening.

Interactions with an Empathic Robot Tutor in Education …

93

Feelings Associated with the Experience How students felt about taking part in the study, whether it be about the interaction experience with Nao or the games, were mentioned in most of the groups (n = 7). In four of the groups, students expressed positive feelings towards the games; e.g. most said that the games were ‘fun’. Whereas one student said that the map-reading game had educational value, another student stated that he wanted to play the games all the time, and yet another agreed that it was nice to get out of ordinary lessons. In three groups, students mentioned that the map-reading game was difficult, one student stating that she was not very good at it, and another that he failed at the game. Both positive and negative feelings towards the robot were expressed in four of the groups, where, on the one hand, positive feelings included perceptions of the overall experience to be ‘cool’ or ‘amazing’. On the other hand, negative feelings were expressed by three of the students, where two students stated that it was ‘scary at first’, and the other said that Nao was ‘ugly and disgusting’.

4.2.2

Robots and Inclusive Education

When the groups were asked to consider how robots such as Nao or Pepper could help or hinder educational inclusion, the students discussed a variety of ways that robots could potentially be used, along with envisioned issues thereof.

Robots that Support Children with Special Needs In considering ways to make teaching and learning accessible for children in need of special support, students envisioned that robots could feature as extra support for particular children either within or outside their ordinary classroom lessons. Half of the groups envisioned this to be in the form of a student assistant in the classroom, e.g. personal for children in need of extra support in mathematics: ‘I think it might feel better for the student in need of a little extra support to have something special’ (Group 9: 4). However, the potential unfairness of this approach was also raised in one group: Daisy*: “If we imagine that I have Pepper, and only I have it because I need a lot of help…” Emily: “Then that becomes pretty unfair.” Matilda*: “Yes, exactly. And that’s the negative thing with the robot.” —Group 8: 1

One group extended this argument by challenging the notion that personal robot assistants would lead to an inclusive environment: Lee: “Yes, but will it though? It might become more segregating because that child’s the only one who has a robot. But if the robot would go around to everyone, it would be easier. Then it’s better, because then that child won’t be alone with having a robot.”

94

S. Serholt Clara: “Yes, exactly. That’s what I think.” Lee: “Then it [the child] becomes more integrated that way.” —Group 9: 4

Robots as Teaching Assistants for Classrooms Drawing on their personal experiences of having assistant teachers in the classroom, the students considered several ways in which robots could add value to the classroom context by adopting a similar role, perhaps in effect, facilitating education for them. For example, the robot could assist during mathematics lessons, play games with them, answer questions or remind them about various aspects of everyday education. In one group, a student postulated that the Pepper-robot is more ‘mature’ than the Nao-robot, since it would be able to move around the classroom and help individual students; in another group, students reasoned that with a robot present, they would not have to wait as long for help as they do currently from their teacher. Some of the groups considered that the possibility of using robots as adaptive learning tools could lead to a more inclusive educational environment, provided that it was possible to programme such robots to behave in a personalized manner. Here, they emphasized that robots should be able to adapt to individual students, a variety of social situations, as well as to use language suitable for different generations. In considering the importance that such a robot could adapt its explanations, rather than always repeat the same explanations, one group anticipated that the robot would nevertheless just repeat the same stock phrases over and over again. Robots were also discussed in terms of featuring as substitutes if a particular school lacked teachers. At the same time, it was emphasized that a robot could never replace a teacher. One student emphasized the potential novelty effect, and said that they would find the robot interesting in the beginning, but grow tired of it after some time.

Robots as Disruptive for Classrooms Regardless of the role of the robot in the classroom, a majority of the groups discussed anticipated disruptions for classrooms. Some students envisioned that robots could become distractions in themselves, whereas one group speculated that with only one robot in a class, all students would try to get the robot’s attention, creating ‘chaos’ for the lessons, as well as for the robot’s perception. Against the belief that robots would be connected to the Internet and be able to answer any and all questions, students discussed the possibility of robots encouraging bad behavior in students. Especially in the 9th grade groups, students envisioned a robot helping them cheat on tests, but also that students would likely not take the robot seriously; instead, they could try to mock the robot or get it to answer random and irrelevant questions during lessons.

Interactions with an Empathic Robot Tutor in Education …

95

Social Relationships with Robots Reflecting their perceptions about loneliness, and that some children do not have friends at school, one group discussed how robots could help them through social support or friendship. However, one student exhibited scepticism towards the moral acceptability of this approach, arguing that children might become asocial since they would grow accustomed to and subsequently prefer to interact with robots over humans, whereby the group conceded that children probably should not be supplied with their own personal robot. In two of the 9th grade groups, the students considered a robot in terms of a learning tool by drawing parallels to calculators and textbooks. Hence, the idea of a social relationship as per their own definitions of such, seemed a far-fetched fantasy. Notwithstanding, the 9th graders looked towards future developments in this field with cautious optimism, recognizing that social robots are still in their infancy and that they ‘may become more useful in the future, when they [researchers/developers] have had a few more years to develop them and make them more human’ (Group 9: 4). Even so, some students nevertheless raised the possibility that breakdowns may be unavoidable: Luke*: “Technology… Anything can happen, can break down or something.” Jason*: “I guess we’ll see… Interesting to see how these developments unfold.” —Group 9: 5

5 Conclusion In this chapter, I have presented an extended follow-up study with students at a school where an empathic robot tutor was implemented three years earlier for a duration of three months. The aim of the current study has been to explore a set of research questions through the use of a questionnaire and student group discussions at the school in question. In this study, both students who had interacted with the robot (n = 15), as well as students who were new at the school and had never interacted with the robot (n = 19) took part. Against the results that have been presented and discussed in the previous section, I will in this conclusion answer each research question in turn, followed by final remarks.

5.1 Research Questions What do the children remember from their interaction experiences with the robot tutor? Despite that three years had passed since the initial field study, students were able to recount a set of details regarding the experience, such as the appearance of the robot

96

S. Serholt

and the set-up and the different games and breakdowns. This suggests that extended follow-up studies are feasible approaches for conducting post-evaluations of robot field studies with children; perhaps, in part, because the experience was emotionally arousing and therefore memorable, but also because the questionnaire targeted very specific events that might have acted as triggers for their memory. This is further supported by the finding that some students were explicit about not remembering much about the study, but they were all still able to recount specific details about the experience when probed by the questionnaire items or group discussions. Do they recall breakdowns occurring, what they were, and have an opinion on whose fault it was? It seems that most students had recollections of breakdowns occurring during the intervention. In fact, only two students responded that they had never experienced any of the breakdown situations in the questionnaire. Yet, in my earlier analysis of breakdowns in interaction (Serholt 2018), video analysis revealed that both of these particular students had in fact experienced at least one of the breakdown situations described. This suggests that breakdowns did indeed occur for almost all students at some point during the intervention. In terms of fault, students either blamed Nao for the breakdowns, or considered that no one was at fault, or that they saw it simply as a technology problem. One student responded that it was the researcher’s fault, whereas yet another three considered it to be their own fault. Only one student blamed their classmate for the breakdowns (presumably for something that occurred during the game that was played in pairs), while also responding that the technology simply did not work correctly. What are their perceptions of the robot tutor and the experience? Overall, most students responded that they had enjoyed the experience despite the breakdowns, although they were not quite as optimistic about this as they were shortly after the intervention had taken place. However, some students were very negative about the experience. Looking back, the majority of students considered the robot to be best described as a computer, where some considered it to be a combination of different descriptors. Perhaps, this is why most students did not blame Nao for the breakdowns, nor find it important that it liked them, as they may not consider themselves having a two-way relationship with their computers. This is further supported by the finding from some of the group discussions, where the idea of a social relationship with a robot was viewed as unlikely. Given their experiences, do their normative perspectives on robots differ from those of students who do not have experiences interacting with the robot? Overall, the students’ normative perspectives towards robots were similar in distribution for several items in a study using the same instrument, but with a larger sample (Serholt et al. 2016). However, the results showed that the views of students with robot experience stood out significantly on one aspect of robots when compared to their peers without earlier experience interacting with robots, i.e. the experienced

Interactions with an Empathic Robot Tutor in Education …

97

students were significantly more negative towards the idea of robots perceiving and analysing their emotional states. Anecdotally, listening to the group discussions, it was evident that the students who had experience working with the robot were typically more knowledgeable about robots in general. Several instances where an experienced student explained to the other group members how robots work on a technical level, helping to demystify some of the behaviour, were observed. This suggests that they had acquired a level of knowledge about the overall workings of robots. Finally, how do they consider robots to factor into the idea of inclusive education? Taken together, the students saw a number of possible application areas for robots that could contribute to a more inclusive educational environment. Featuring as extra support for children with special needs was one application mentioned, with the possible drawback that this approach might increase segregation, or be perceived as unfair for the other students. Adaptive learning tools in the form of assistant teachers were also considered a possibility, where such robots could help all students in a personalized way, yet, not as replacements for teachers. The notion of friendship with robots was brought up as well, although some foresaw dangers therein, and some of the older students saw robots more as educational tools than relational actors. The findings illustrate that the students consider inclusion in both what Haug (2016) describes as a narrow and a broad sense. In the narrow sense, inclusion is focused only on special education, and how teaching and learning can be changed (i.e. robots as extra support), while the broad definition takes into account all students who are at risk of being segregated (e.g. lonely children).

5.2 Final Remarks and Outlooks Social robots have found increasing applications in the areas of education in recent years. Coupled with AI, there is yet no trend to suggest that the use of robots will regress. This chapter has examined one such application of a social and empathic robot tutor. Admittedly, robots can be considered moving targets in the sense that once one robot has been evaluated, there is another upgraded version waiting around the corner. Yet, insofar as we can consider the robots available today, the greatest challenge seems to be to create artificial social interaction (Belpaeme et al. 2018). Not that children do not already interact with robots in a social manner as there is much empirical evidence to suggest that they do (Breazeal et al. 2016; Kanda et al. 2004; Okita et al. 2011; Serholt and Barendregt 2016), but rather because we are nowhere near creating robots with the kind of intelligence necessary to create and uphold a dynamic dialogue with children and avoid breakdowns (Ros et al. 2011; Serholt 2018). This impedes, rather than facilitates, an inclusive agenda as conceptualized by the students in the current study. In an earlier study where ethical tensions of classroom robots were discussed with pre-service and practicing teachers in three European countries, teachers considered

98

S. Serholt

the possibility of social robots collecting affective data about children. Here, one teacher raised concerns about the emotional harm that could follow as children grow older and realize that their school holds emotional profiles on them (Serholt et al. 2017). Contrary to the reasoning that stakeholders are sceptical about robots because they lack experience and are therefore highly influenced by depictions of robots in science fiction and the media, this does not seem to be the case here. Indeed, as has been demonstrated in the study presented in this chapter, students who had interacted with a robot became more critical than their peers towards functions that are often argued to hold most potential in the field, i.e. the possibility of developing empathic relationships between robots and children based on affective data gathered about individual children. Taking these concerns seriously, it may be wise to proceed with caution in the endeavor to introduce emotion perceptive social robots in education on a wider scale. Acknowledgements I thank all students for participating in this study, as well as their teachers for making the study possible. I also thank my colleagues in the START project for helping me develop the questionnaire and discussion questions, and for assisting in transporting Pepper to the school. Thanks also to Dennis Küster for providing valuable feedback on statistical methods. This work was partially supported by the Marcus and Amalia Wallenberg Foundation and was funded in part by the project START (Student Tutor and Robot Tutee). The author is solely responsible for the content of this publication. It does not represent the opinion of the Wallenberg Foundation, and the Wallenberg Foundation is not responsible for any use that might be made of data appearing therein.

References Belpaeme, T., Kennedy, J., Ramachandran, A., Scassellati, B., & Tanaka, F. (2018). Social robots for education: A review. Science Robotics, 3(21). https://doi.org/10.1126/scirobotics.aat5954. Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101. https://doi.org/10.1191/1478088706qp063oa. Breazeal, C., Harris, P. L., DeSteno, D., Kory Westlund, J. M., Dickens, L., & Jeong, S. (2016). Young children treat robots as informants. Topics in Cognitive Science, 8(2), 481–491. https:// doi.org/10.1111/tops.12192. de Leeuw, E. D. (2011). Improving data quality when surveying children and adolescents: Cognitive and social development and its role in questionnaire construction and pretesting. Department of Methodology and Statistics, Utrecht University. Retrieved from http://www.aka. fi/globalassets/awanhat/documents/tiedostot/lapset/presentations-of-the-annual-seminar-10-12may-2011/surveying-children-and-adolescents_de-leeuw.pdf. Haug, P. (2016). Understanding inclusive education: Ideals and reality. Scandinavian Journal of Disability Research, 19(3), 206–217. https://doi.org/10.1080/15017419.2016.1224778. Kanda, T., Hirano, T., Eaton, D., & Ishiguro, H. (2004). Interactive robots as social partners and peer tutors for children: A field trial. Human-Computer Interaction, 19, 61–84. https://doi.org/ 10.1207/s15327051hci1901&2_4. Liles, K. R., & Beer, J. M. (2015). Rural minority students’ perceptions of Ms. An, The Robot Teaching Assistant, as a social teaching tool. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 59(1), 372–376. https://doi.org/10.1177/1541931215591077. Lin, Y.-C., Liu, T.-C., Chang, M., & Yeh, S.-P. (2009). Exploring children’s perceptions of the robots. In M. Chang, R. Kuo, G.-D. Chen, & M. Hirose (Eds.), Learning by playing. Game-based

Interactions with an Empathic Robot Tutor in Education …

99

education system design and development (Vol. 5670, pp. 512–517). Berlin, Heidelberg: Springer. https://doi.org/10.1007/978-3-642-03364-3_63. Obaid, M., Aylett, R., Barendregt, W., Basedow, C., Corrigan, L. J., Hall, L., et al. (2018). Endowing a robotic tutor with empathic qualities: Design and pilot evaluation. International Journal of Humanoid Robotics, 15(6). https://doi.org/10.1142/s0219843618500251. Okita, S. Y., Ng-Thow-Hing, V., & Sarvadevabhatla, R. K. (2011). Multimodal approach to affective human-robot interaction design with children. ACM Transactions on Interactive Intelligent Systems, 1(1), 5:1–5:29. https://doi.org/10.1145/2030365.2030370. Ros, R., Nalin, M., Wood, R., Baxter, P., Looije, R., Demiris, Y., et al. (2011). Child-robot interaction in the wild: Advice to the aspiring experimenter. In Proceedings of the 13th International Conference on Multimodal Interfaces, Alicante, Spain (pp. 335–342). https://doi.org/10.1145/ 2070481.2070545. Šabanovi´c, S. (2010). Robots in society, society in robots. International Journal of Social Robotics, 2(4), 439–450. https://doi.org/10.1007/s12369-010-0066-7. Serholt, S. (2017). Child–robot interaction in education (Doctoral thesis). University of Gothenburg, Gothenburg. Retrieved from http://hdl.handle.net/2077/52564. Serholt, S. (2018). Breakdowns in children’s interactions with a robotic tutor: A longitudinal study. Computers in Human Behavior, 81, 250–264. https://doi.org/10.1016/j.chb.2017.12.030. Serholt, S., & Barendregt, W. (2016). Robots tutoring children: Longitudinal evaluation of social engagement in child-robot interaction. In Proceedings of the 9th Nordic Conference on HumanComputer Interaction (NordiCHI’16), Gothenburg, Sweden. https://doi.org/10.1145/2971485. 2971536. Serholt, S., Barendregt, W., Küster, D., Jones, A., Alves-Oliveira, P., & Paiva, A. (2016). Students’ normative perspectives on classroom robots. In J. Seibt, M. Nørskov & S. Schack Andersen (Eds.), What social robots can and should do: Proceedings of robophilosophy 2016/TRANSOR 2016 (pp. 240–251). IOS Press. https://doi.org/10.3233/978-1-61499-708-5-240. Serholt, S., Barendregt, W., Vasalou, A., Alves-Oliveira, P., Jones, A., Petisca, S., et al. (2017). The case of classroom robots: Teachers’ deliberations on the ethical tensions. AI & SOCIETY, 32(4), 613–631. https://doi.org/10.1007/s00146-016-0667-2. Severinson-Eklundh, K., Green, A., & Hüttenrauch, H. (2003). Social and collaborative aspects of interaction with a service robot. Robotics and Autonomous Systems, 42(3–4), 223–234. https:// doi.org/10.1016/S0921-8890(02)00377-9. Sharkey, A. (2016). Should we welcome robot teachers? Ethics and Information Technology, 18(4), 283–297. https://doi.org/10.1007/s10676-016-9387-z. Westlund, J. M. K., Park, H. W., Williams, R., & Breazeal, C. (2018). Measuring young children’s long-term relationships with social robots. Paper presented at the Proceedings of the 17th ACM Conference on Interaction Design and Children, Trondheim, Norway.

Sofia Serholt is a Senior Lecturer at the Department of Applied IT at the University of Gothenburg, Sweden. Her research concerns the use of ICT in educational settings, including schools and public libraries. In her dissertation titled Child–Robot Interaction in Education, she explored children’s interactions with an empathic robot tutor in a classroom setting, focusing on instruction, social interaction, and breakdowns, as well as perceptions and normative perspectives of teachers and students on the use of educational robots. She is an active member of Applied Robotics in Gothenburg: a research group focused on exploring human-centered approaches to robot design, as well as social and ethical implications of robots in society. She regularly engages in public speaking at conferences and events, has participated in several research projects on social robots in education, and her research has received extensive coverage in Swedish and international press. She holds a PhD in Applied IT with specialization in Educational Sciences, and a Master of Education. She has worked as research assistant in Child–Robot Interaction at University West, and recently carried out a postdoc in Interaction Design at Chalmers University of Technology. Her research on educational robots was appointed one of the top advancements in research and technology in 2017 by the Royal Swedish Academy of Engineering Sciences.

A Communication Model of Human–Robot Trust Development for Inclusive Education Seungcheol Austin Lee and Yuhua (Jake) Liang

Abstract Integrating robots into the educational environment offers tremendous opportunities to support and augment learning. However, building trust between human users and robots can be a challenge for inclusive education, as females, minorities, and the less privileged individuals tend to report higher levels of the anticipated fear and distrust toward robots. In this chapter, we examine how communication affects human–robot trust in light of the verbal messages that humans and robots exchange. The chapter overviews the four guiding foci of human–robot trust: (1) human–robot trust is a communication-driven process; (2) human–robot trust develops over time; (3) trust optimization requires calibration to the particular situation and circumstance; and (4) trust is based on multidimensional perceptions of trustee’s trustworthiness. The chapter outlines systematic research to examine how trust is developed, calibrated, and affected by communication messages across different temporal stages in the inclusive learning environment: pre-interaction stage, entry stage, and relationship stage. Keywords Human–robot trust · Communication · Partnerships · Co-roboting environment

1 Introduction As robots are becoming more technically and communicatively capable, they will soon be integrated into the educational environment. For example, a humanoid robot named RoboThespian taught a group of elementary school students, explaining scientific concepts, running hands-on experiments, and administering a quiz (Polishuk and Verner 2018). Another autonomous humanoid named Bina48 taught a philosYuhua (Jake) Liang: Deceased. S. A. Lee (B) · Y. (Jake) Liang (Deceased) Chapman University, Orange, USA e-mail: [email protected] © Springer Nature Singapore Pte Ltd. 2019 J. Knox et al. (eds.), Artificial Intelligence and Inclusive Education, Perspectives on Rethinking and Reforming Education, https://doi.org/10.1007/978-981-13-8161-4_6

101

102

S. A. Lee and Y. (Jake) Liang

ophy course at West Point military, delivering the lecture and answering questions (Atkinson 2018). While these attempts are still at the experimental stage, we predict that robot instructors will become more capable, available, and accessible in the near future (for a review, see Belpaeme et al. 2018). Although robots may not replace human instructors completely, such robots may assist human instructors and students to enhance learning experiences and to achieve learning outcomes. While robotic instructors and teaching assistants offer tremendous opportunities to support and augment learning, the application of intelligent robots for inclusive education will require successful coupling between human users and robots or co-robots (National Science Foundation 2016). Human trust in robots and their autonomy is crucial to the co-robot partnership and successful outcomes (Hancock et al. 2011; Wagner 2015). Especially in educational contexts, trust is a fundamental component of learning. In his seminal work in interpersonal trust, Rotter (1967) asserted, “much of the formal and informal learning that human beings acquire is based on the verbal and written statements of others, and what they learn must be significantly affected by the degree to which they believe their informants without independent evidence” (p. 651). The same principle applies to the relationship between robot instructors and human learners (Westlund et al. 2017). Building trust, however, can be challenging, especially for a diverse student population. Our study with nationally representative data (Liang and Lee 2017) revealed that about 26% of the US population reported a moderate or severe level of fear toward robots and artificial intelligence. Among participants, females, ethnic minorities, and people with low socioeconomic status and low education levels were more likely to report a heightened level of fear and distrust. Overcoming such fear and distrust will be an important issue for inclusive education in the future. Communication serves as the basis of trust and enables both humans and robots to create initial impressions, to establish each partner’s role and ultimately to achieve their goals. In this chapter, we conceptualize the human–robot communication and trust dynamic as a dyadic process where human users and robots both simultaneously serve as senders and receivers in a co-robot educational environment. Furthermore, we examine human factors related to their trust toward robots. The extent to which robots rely on human users is primarily based on how the robots are engineered. Instead of focusing on robots’ technical features, we focus on how to engineer the human aspect of their trust toward robots through the messages they exchange. As trustors, human users need to rely on their robot partners to take the correct action and continually reciprocate and iterate their progression toward goal achievement (Lee and Liang 2016). For example, in an educational setting, human learners should trust the expertise of the robot and follow its instruction to allow optimal efficiency in the classroom and to achieve desired learning outcomes (Edwards et al. 2016). In this chapter, we examine how communication affects human–robot trust in light of the verbal messages that humans and robots exchange. Understanding the effect of verbal messages in human–robot interaction is an important yet challenging research problem (Lee and Liang 2018). Despite the plethora of research on humanbased trust, developing and maintaining trust between people are still an effortful and sometimes challenging endeavor (Mayer et al. 1995). If trust between people is not

A Communication Model of Human–Robot Trust Development …

103

even a universal quality in interpersonal relationships, human–robot relationships certainly carry new theoretical and practical challenges. The resemblance and discrepancy between robots and people have demarcating challenges for the theoretical development of human–robot trust. Clearly, robots are not people, despite some that are designed with human resemblance and characteristics. Although there are established conceptual frameworks showing that people tend to treat technology similarly as other people (e.g., Nass and Moon 2000), there are also clear boundaries to these effects. In some cases, the embodiment or human-like cues of robots and artificial intelligence have almost no influence on how technology affects users (Cho et al. 2016; Lee and Liang 2015, 2016). In this regard, human learners may need to establish mental schemas or frameworks to approach a new, trusting partnership with robots. In this chapter, we discuss the four guiding foci of human–robot trust: (1) humanrobot trust is a communication-driven process; (2) human–robot trust develops over time; (3) trust optimization requires calibration to the particular situation and circumstance; and (4) trust is based on multidimensional perceptions of trustee’s trustworthiness. Given the importance of the foci above, we discuss the effect of messages on trust across different temporal stages in the inclusive learning environment: preinteraction stage, entry stage, and relationship stage.

2 Human–Robot Trust Is a Communication-Driven Process As robots permeate industrial, military, commercial, domestic, medical, and especially educational settings, their applications will require a successful partnership between human users and robots (i.e., co-roboting; National Science Foundation 2016). Co-robots are robots working alongside human users to achieve a common goal. Co-robots are becoming more technically and communicative capable. In this trend, their scope of goals to accomplish with their human partner both ranges and expands greatly, from education robots teaching students in the classroom to military robots working alongside human personnel in reconnaissance missions. Achieving co-robot goals requires human users and robots to trust each other. Trust is defined as “the willingness of a party to be vulnerable to the actions of another party based on the expectation that the other will perform a particular action important to the trustor, irrespective of the ability to monitor or control that other party” (Mayer et al. 1995: 712). This view of trust focuses on the users’ vulnerability to the robot actions. Given this consideration, human–robot trust is linked to the willingness for individuals to allow the robot partner to take on the important action without excessive monitoring or intervention. Trust carries broad importance across different co-robot task types and various levels of risks (Yagoda and Gillian 2012). Trust is certainly crucial when the goal involves high-risk outcomes (e.g., robots directing disaster victims to evaluation during an emergency; Wagner 2015). Trust is also important in lower risk situations such as education (e.g., learners have to believe the expertise of instructor robots in

104

S. A. Lee and Y. (Jake) Liang

Fig. 1 Transactional model of human–robot communication. Adopted from Lee and Liang (2018)

order to learn effectively). In essence, human trust toward robots, or human–robot trust (HRT), enables both humans and robots to optimally achieve the intended goal. Communication serves as the basis of human–robot trust. The traditional model of human–robot interaction was limited to a “master–slave” relationship, where human operators issue commands and robots simply perform these commands (Rosen 1985; Sheridan 1984). Communication was unidirectional, where humans “speak” and robots “listen.” In this control schema, the issue of trust was limited largely to the reliability of the robots’ performance. However, in a co-roboting environment where humans and robots collaborate as a team, both parties need to (1) leverage each other’s strengths, (2) coordinate their actions, and (3) assist their partner with problem solving (Lee and Liang 2018). To enable collaboration, robots should be able to ask questions, request human assistance, and even assign tasks to their human partners (Bruemmer et al. 2002). Robots should also be able to convince their human partners to trust the information and decisions that they made (Marble et al. 2004). In this interactive and transactive relationship, robots are not merely the passive, receiving end of communication. Instead, both human and robots can serve as senders and receivers simultaneously and exchange verbal and nonverbal messages, as depicted in the transactional model of human–robot communication (Fig. 1). When applied to educational contexts, the transactional model of human–robot communication (Lee and Liang 2018) assumes a dynamic and dyadic interaction occurring in a social (e.g., student–teacher relationship) and physical (e.g., classroom, home) environment, where human learners and robot instructors collaborate

A Communication Model of Human–Robot Trust Development …

105

Fig. 2 Model of trust. Adopted from Mayer et al. (1995: 715)

to achieve learning objectives (i.e., co-roboting). Humans and robots can be both the senders and receivers of communication. Using verbal commands or other input devices (i.e., channel), human learners may ask robots to explain new concepts, summarize key information, or provide examples. Robot instructors may respond to such requests using a synthesized voice or text/images displayed on a screen (i.e., channel). Robots may also initiate communication by giving instructions or asking questions, while human learners listen and answer. Humans and robots may frequently alternate between these roles of sender and receiver. Finally, each message exchanged in this dynamic communication process creates meaning that implicates the next message exchange through feedback. Both verbal (e.g., asking questions for clarification) and nonverbal (e.g. nodding head) can be used for this feedback (Serholt and Barendregt 2016). This transactional process (Fig. 1) overlays to the trust development process (Mayer et al. 1995; Fig. 2), which will be discussed later. Trust is based on the expectation of how the other party performs (Mayer et al. 1995), and communication plays a vital role in setting this expectation by allowing humans and robots to leverage each other’s strengths and coordinate their actions accordingly. User’s expectations of the robots depend on the manner to which robots disclose their technical capabilities and possibly prepare users for failures (Yagoda and Gillian 2012). Such expectations, in relation to how robots perform, are the crucial factor of trust development. According to Hancock et al.’s (2011) metaanalysis, the strongest predictor of trust was the robot’s actual performance (i.e., the robot’s capability to carry out a designated task). By comparison, the design of the robot or the characteristics of the users had smaller or insignificant impacts. This finding may appear to discount the importance of the human-related factor. However, it should be noted that this performance expectation is shaped and modified through communication (Serholt 2018). We found that communication shapes the performance expectations even before people meet robots for the first time (Liang and Lee 2016). Human users can receive messages such that they shape the users’ expectations regarding the upcoming inter-

106

S. A. Lee and Y. (Jake) Liang

action. They can anticipate either a positive or negative experience with an upcoming interaction with the robots. Prior research shows that people predict the outcome of an impending interaction and that this prediction affects their initial interaction (Sunnafrank 1986). When robots do meet and interact with people, they communicate directly with human partners and continue to modify the partner’s expectation of the robots’ performance. Put differently, both the messages people receive about robots and the messages robot generate afford the ability to alter how people respond to the robot’s performance, and this affects human–robot trust.

3 Human–Robot Trust Develops Over Time Trust between people is developed longitudinally and over experiences of behaviors (i.e., as the partners following through with the promised action) (Mayer et al. 1995), and these experiences set the expectations. Communication also plays a vital role in setting such expectations. In a co-robot situation, human and robot partners overview what is expected of each other and provide a benchmark of performance expectations. For example, human learners and robot instructors may set the expectations by communicating what they know, what they can, and what they plan to achieve. Trust ensues given the relationship between the expectations and actual performance (i.e., outcomes in Fig. 2). As trust develops, the communication also changes. People utilize technology and strategically adapt communication to adapt to that technology (Liang and Walther 2015). As a result, human–robot communication creates specific challenges in how humans and robots exchange messages to understand each other, especially in light of their expectations in the trust development cycle. We discuss how to develop trust in the three stages of the trust development: pre-interaction stage, entry stage, and relationship stage.

3.1 Pre-interaction Stage We propose that the development of trust begins even before the human users meet the co-robots. People develop expectations about others before they meet for the first time, based on the information they have (Burgoon 1993; Burgoon and Walther 1990). As Jones (1986) asserted, “it is all but impossible to conceive of a participant approaching social interaction without some set of expectancies, hypotheses, or predictions about how the other participant is likely to behave under various circumstances” (p. 43). In educational settings, for example, students’ expectations about their prospective instructors influence the instructors’ perceived trustworthiness, and ultimately, the students’ cognitive and behavioral learning (Edwards et al. 2009; Liang 2015; Liang et al. 2015).

A Communication Model of Human–Robot Trust Development …

107

A challenge for employing robots as instructors is that human learners may have negative expectations toward them. In our survey of fear toward autonomous robots and artificial intelligence (Liang and Lee 2017), more than a quarter of participants reported a heightened level of fear and distrust, although most of them had no direct experience with such technology. In other words, their fear was not actual but anticipated. This anticipated fear was influenced by their fear of new technology and their exposure to science fiction. This challenge is especially pertinent to inclusive education, as females, minorities, and the less privileged individuals tend to hold higher levels of the anticipated fear. To reduce the anticipated fear and to elevate trust in the pre-interaction stage, information can be provided to human learners who anticipate their interaction with robot instructors. This strategy is more effective when the information source is their peers, as word-of-mouth communication is often perceived as a genuine, unbiased, and accurate source of learning (Banerjee and Fudenberg 2004; Ellison and Fudenberg 1995). Our study (Liang and Lee 2016) showed that when participants received wordof-mouth information about their robot partner before interaction, the effects of human–robot trust on interaction outcomes were augmented. Specifically, enhanced trust elicited positive mood and more favorable evaluations of the robot. This finding suggests that messages can be strategically deployed to enhance trust and interaction outcomes. Such messages can be applied to the learning environment to foster positive expectation, especially among the diverse student population.

3.2 Entry Stage The entry stage includes the initial encounter between human learners and robot instructors and their first few interactions. High levels of uncertainty and anxiety characterize this entry stage. People experience anxiety when interacting with strangers because they have less certainty in predicting and explaining others’ attitudes, beliefs, and behaviors (Berger and Calabrese 1975). People experience higher anxiety when they communicate with strangers of different groups than when they communicate with strangers of their own group (Gudykunst 2005). The level of uncertainty is even more heightened when people expect to interact with a robot as opposed to another human because people do not hold mental representations that guide their comprehension of and interaction with robots (Spence et al. 2014). As a consequence, the high level of uncertainty in human–robot interaction produces increased anxiety. The high levels of uncertainty and anxiety may lead to negative attitudes toward robots and even fear. Nomura et al. (2006) suggested that communication apprehension and anxiety toward technological products together can create negative attitudes toward robots. The feelings of uncertainty and anxiety can also lead to avoidance in communication, which can be detrimental to the co-robot experience. Building trust is essential in reducing uncertainty. Turner (1988) argues that to be motivated to interact with strangers, people need to trust others. People need

108

S. A. Lee and Y. (Jake) Liang

Fig. 3 Trust matrix. Adopted from Manchala (1998)

to feel that their partners are, to some degree, reliable and predictable. Yamagishi and Yamagishi (1994) also asserted that trust provides a solution to the problems caused by uncertainty. In this regard, trust can function as an uncertainty reducer. For example, Colquitt et al. (2012) reported strong negative correlations between uncertainty and cognition-based trust as well as uncertainty and affect-based trust. Conversely, reducing uncertainty is also crucial to building trust. By reducing uncertainty, people can better predict and explain the other’s behaviors, which is crucial to the development of any relationship involving trust (Berger and Calabrese 1975). The uncertainty–trust matrix (Fig. 3; Manchala 1998) indicates that as the level of uncertainty increases, the “trust zone” becomes narrower. When the uncertainty is extremely high, the trust zone becomes almost nonexistent. In a co-robot context, if users believe that the collaborative outcome is unpredictable and controllable, they may either take over the control of the robot or avoid interaction with the robot (Serholt 2018). The trust zone can be successfully expanded by reducing uncertainty. Information seeking and reciprocity can serve as tools for reducing uncertainty and enhancing trust. Berger and Calabrese (1975) expected that the amount of information seeking and reciprocity is the highest during the entry stage of interaction, given the high level of uncertainty present. The way to reduce mutual uncertainty is to ask for and give the same kinds of information at the same rate of exchange. In the context of human–robot interaction, robots may use messages strategically to maintain high levels of information seeking and reciprocity over an extended period, until both parties reach the level of trust for collaboration. As discussed in the transactional model of human–robot communication, robot instructors may actively engage in information seeking, instead of passively waiting for human learners to initiate conversation. Robots should take initiatives in maintaining a continu-

A Communication Model of Human–Robot Trust Development …

109

ous exchange of information, as people are often unmotivated to seek information (Kellerman and Reynolds 1990; Kramer 1999). To effectively reduce uncertainty, the messages of robots need to be designed strategically. As trust develops gradually over the course of interaction, a robot may begin with relatively shallow, non-intimate questions and then move to deeper, more intimate ones, as proposed in the social penetration theory (Altman and Taylor 1973). Instead of asking questions abruptly, the robot may engage in verbal self-disclosure first (e.g., “I was manufactured by Aldebaran Robotics. Could you tell me where you are from?”), and then ask related questions (e.g., “I sometimes fear that my backup battery dies and I lose all my memory. What is your biggest fear?”) that are more personal and intimate (Bailenson et al. 2006; Mumm and Mutlu 2011). When this message strategy is employed, most human learners will reciprocate the robot’s information seeking by disclosing themselves. The norm of reciprocity (Gouldner 1960) compels people to reciprocate when the robot reveals information about itself first. Our research documented the robust effect of reciprocity in both human–computer interaction (Lee and Liang 2015; Liang et al. 2013) and humanrobot interaction (Lee and Liang 2016). Reduced uncertainty and enhanced trust using this approach are expected to influence other interaction outcomes such as intimacy, liking, and communication satisfaction (Neuliep and Grohskopf 2000).

3.3 Relationship Stage The relationship stage refers to the long-term interaction and maintenance of the relationship between robot instructors and human learners until the relationship dissolves (e.g., graduation, completion, or discontinuation of the course). The uncertainty reduction theory (Berger and Calabrese 1975) posits that as uncertainty decreases, the need for information seeking diminishes. Similarly, the need for symmetric exchanges of information declines at a rapid rate. Thus, the frequency of information exchange may decrease in this stage. Instead, communication is more focused on the subject matter and the content of learning. In the earlier stages, the challenge was to establish a basic level of trust. In the relationship stage, the challenge becomes to calibrate and optimize the level of trust.

4 Trust Optimization Requires Calibration Trust requires calibration to the particular situation and circumstance (Lee and Moray 1992). The calibrated trust will be optimized to produce the desired task outcomes. Simply aiming to maximize trust can negatively impact co-robots. An over-heightened level of trust can reduce situational awareness (e.g., lack of attending to robots when needed) and create potential risks (e.g., safety problems, performance issues). By contrast, an underdeveloped level of trust can reduce desired

110

S. A. Lee and Y. (Jake) Liang

Fig. 4 Stages of trust development through communication

reliance (e.g., unnecessary monitoring or intervention, failure to comply with robot instructions) and limit the potential benefit of robotic systems. Ideally, the co-robot team should reach the optimal level of trust where human learners and robot instructors leverage each other’s strengths, coordinate their actions, and assist each other accordingly. Failure to do so results in overreliance or underuse of robots and thus produces suboptimal collaborative outcomes. The calibration of trust requires the development of new strategies for robots to assess trust through the messages they receive from humans. Although robots can utilize messages to increase trust, they also need to adapt to the human learners’ state of trust. As discussed previously, a communication view of trust requires a dynamic process where robots utilize messages to increase trust when deemed appropriate. The appropriateness and context require the formulation of an approach for robots to gauge and assess human trust toward them. Simultaneously, robots may use strategic messages to adjust the human learners’ level of expectations. For example, Wang et al. (2016) found that when a robot provided hand-crafted explanations about their decision-making process, human users were able to understand its mechanism more accurately, which led to enhanced transparency, trust, and performance outcomes. Figure 4 summarizes the three stages of trust development that we discussed so far and the required communication strategies.

A Communication Model of Human–Robot Trust Development …

111

5 Trust Is Based on Multidimensional Perceptions of Trustworthiness Trust depends on the trustor’s perceptions of the trustee’s trustworthiness (Fig. 2). The extent to which users view robots as trustworthy should affect their actual trust behavior. The three dimensions of trustworthiness outlined by Mayer et al. (1995) include ability, benevolence, and integrity. Ability refers to “that group of skills, competencies, and characteristics that enable a party to have influence within some specific domain” (Mayer et al. 1995: 717). In the co-robot context, robots typically bring a specific skill set that they have been designed to carry out, making robots essentially experts in specified domains. For instance, an industrial robot may have the ability to assemble parts more precisely than human workers. A robot instructor may have more knowledge in the subject matter than human learners. Trust in robot ability means that the human users allow robots to complete tasks according to designed capabilities with minimal interferences or surveillance. However, users also need to engage in the appropriate level of monitoring. Therefore, successful co-robots in the ability dimension involve strategies that robots can utilize to provide indicators of successful task performance and expertise. Benevolence “is the extent to which a (robot) is believed to want to do good to the (human user), aside from an egocentric profit motive” (Mayer et al. 1995: 718). Although benevolence is typically attributed to the human characteristics of goodwill, people also attribute robot motives based on benevolence. A large body of literature documents that people often treat technology in a similar way as they treat other people (Liang et al. 2013; Nass and Moon 2000; Reeves and Nass 1996). The same applies to human–robot communication: People tend to follow social rules, even when they interact with robots. When people interact with a benevolent robot, they are more likely to evaluate the robot favorably, to perceive the robot trustworthy, and to be persuaded by the robot. In our study (Lee and Liang 2016), for example, we found that when a robot helped human partners with a 5-min trivia quiz, participants were more likely to believe that the robot is competent, likable, and trustworthy. In addition, 60% of participants returned the robot’s favor by helping the robot for 15 min. This study clearly shows that robots can build credibility by engaging in benevolent behavior and gain compliance from human partners, following the norm of reciprocity. Integrity refers to the belief that the robot will follow a set of principles that the users will find acceptable (Mayer et al. 1995). Typically, this refers to morality, standards, or ethical behavior. Certainly, robots are programmed to complete the task as a primary goal. However, it is important to acknowledge that the human users may not trust their integrity entirely based on the robot’s task performance alone, but based on a possible anthropomorphizing belief that the robots adhere to moral standards similar to other people (Kahn et al. 2012). Although integrity appears to be less of a concern for less involving tasks, high-involving tasks require users to rely on or even risk their own safety or outcome on the belief that the robot will

112

S. A. Lee and Y. (Jake) Liang

perform accordingly (e.g., emergency evacuations or military contexts). Messages can strategically also target robot integrity, such as to disclose its morality or standards to maximize human users’ understanding. The more users understand how robots are designed to be ethical, the more likely they will attribute the robots with integrity.

6 Conclusion In this chapter, we speculated on the future of education, where robot instructors are employed to assist and enhance learning. While robots carry a tremendous potential for education, it is important to develop an interface that allows human learners to interact with robots in a manner that is intuitive, efficient, enjoyable, and trustworthy (Breazeal 2004). Understanding the principles of communication is crucial to developing such interaction, especially for a diverse student population. The messagebased model of human–robot trust discussed in this chapter will allow both human learners and robot instructors to assess, address, and calibrate trust in a given context to enhance learning experiences and to achieve learning outcomes.

References Altman, I., & Taylor, D. A. (1973). Social penetration: The development of interpersonal relationships. New York, NY: Holt, Rinehart & Winston. Atkinson, K. (2018, October). This robot co-taught a course at West Point. Axios. Retrieved from https://www.axios.com/robot-ai-teaching-college-course-at-west-point-98ce5888-873b-4b728de5-0f7c592d66b0.html. Bailenson, J. N., Yee, N., Merget, D., & Schroeder, R. (2006). The effect of behavioral realism and form realism of real-time avatar faces on verbal disclosure, non-verbal disclosure, emotion recognition, and co-presence in dyadic interaction. Presence: Teleoperators and Virtual Environments, 15, 359–372. Banerjee, A., & Fudenberg, D. (2004). Word-of-mouth learning. Games and Economic Behavior, 46, 1–22. Belpaeme, T., Kennedy, J., Ramachandran, A., Scassellati, B., & Tanaka, F. (2018). Social robots for education: A review. Science Robotics, 3(21), 1–9. Berger, C. R., & Calabrese, R. J. (1975). Some explorations in initial interaction and beyond: Toward a developmental theory of interpersonal communication. Human Communication Research, 1, 99–112. Breazeal, C. L. (2004). Designing sociable robots. Cambridge, MA: MIT press. Bruemmer, D. J., Marble, J. L., & Dudenhoeffer, D. D. (2002). Mutual initiative in humanmachine teams. In Proceedings of the IEEE 7th Conference on Human Factors and Power Plants (pp. 22–30). IEEE. Burgoon, J. K. (1993). Interpersonal expectations, expectancy violations, and emotional communication. Journal of Language and Social Psychology, 12, 30–48. Burgoon, J. K., & Walther, J. B. (1990). Nonverbal expectancies and the evaluative consequences of violations. Human Communication Research, 17, 232–265.

A Communication Model of Human–Robot Trust Development …

113

Cho, S., Lee, S. A., & Liang, Y. (2016). Using anthropomorphic agents for persuasion. Paper presented at the 66th Annual Convention of the International Communication Association, Fukuoka, Japan. Colquitt, J. A., LePine, J. A., Piccolo, R. F., Zapata, C. P., & Rich, B. L. (2012). Explaining the justice–performance relationship: Trust as exchange deepener or trust as uncertainty reducer? Journal of Applied Psychology, 97, 1–15. Edwards, A., Edwards, C., Shaver, C., & Oaks, M. (2009). Computer-mediated word-of-mouth communication on ratemyprofessors.com: Expectancy effects on student cognitive and behavioral learning. Journal of Computer-Mediated Communication, 14, 368–392. Edwards, A., Edwards, C., Spence, P. R., Harris, C., & Gambino, A. (2016). Robots in the classroom: Differences in students’ perceptions of credibility and learning between “teacher as robot” and “robot as teacher”. Computers in Human Behavior, 65, 627–634. Ellison, G., & Fudenberg, D. (1995). Word-of-mouth communication and social learning. The Quarterly Journal of Economics, 110, 93–125. Gouldner, A. W. (1960). The norm of reciprocity: A preliminary statement. American Sociological Review, 25, 161–178. Gudykunst, W. B. (2005). An anxiety/uncertainty management (AUM) theory of effective communication: Making the net of the mesh finer. In W. B. Gudykunst (Ed.), Theorizing about intercultural communication (pp. 281–322). Thousand Oaks, CA: Sage. Hancock, P. A., Billings, D. R., Schaefer, K. E., Chen, J. Y., De Visser, E. J., & Parasuraman, R. (2011). A meta-analysis of factors affecting trust in human-robot interaction. Human Factors: The Journal of the Human Factors and Ergonomics Society, 53, 517–527. Jones, E. E. (1986). Interpreting interpersonal behavior: The effects of expectancies. Science, 234, 41–46. Kahn, P. H., Kanda, T., Ishiguro, H., Gill, B. T., Ruckert, J. H., Shen, S., et al. (2012). Do people hold a humanoid robot morally accountable for the harm it causes? In Proceedings of the 7th Annual ACM/IEEE International Conference on Human-Robot Interaction (pp. 33–40). ACM. Kellerman, K., & Reynolds, R. (1990). When ignorance is bliss: The role of motivation to reduce uncertainty in uncertainty reduction theory. Human Communication Research, 17, 5–75. Kramer, M. W. (1999). Motivation to reduce uncertainty. Management Communication Quarterly, 13, 305–316. Lee, S. A., & Liang, Y. (2015). Reciprocity in computer-human interaction: Source-based, normbased, and affect-based explanations. Cyberpsychology, Behavior, and Social Networking, 18, 234–240. Lee, S. A., & Liang, Y. (2016). The role of reciprocity in verbally persuasive robots. Cyberpsychology, Behavior, and Social Networking, 19, 524–527. Lee, S. A., & Liang, Y. (2018). Theorizing message strategies for persuasive robots. In A. L. Guzman (Ed.), Human-machine communication: Rethinking communication, technology, and ourselves (pp. 119–143). New York, NY: Peter Lang. Lee, J., & Moray, N. (1992). Trust, control strategies and allocation of function in human-machine systems. Ergonomics, 35, 1243–1270. Liang, Y. (2015). Responses to negative student evaluations on RateMyProfessors.com: The effect of instructor statement of credibility on student lower-level cognitive learning and state motivation to learn. Communication Education, 64, 455–471. Liang, Y., & Lee, S. A. (2016). Advancing the strategic messages affecting robot trust effect: The dynamic of user- and robot-generated content on human-robot trust and interaction outcomes. Cyberpsychology, Behavior, and Social Networking, 19, 538–544. Liang, Y., & Lee, S. A. (2017). Fear of autonomous robots: Evidence from national representative data with probability sampling. International Journal of Social Robotics, 9, 379–384. Liang, Y, & Walther, J. B. (2015). Computer-mediated communication. In International encyclopedia of social and behavioral sciences (2nd ed., pp. 504–509). Waltham, MA: Elsevier. Liang, Y., Lee, S. A., & Jang, J. (2013). Mindlessness and gaining compliance in computer-human interaction. Computers in Human Behavior, 29, 1572–1579.

114

S. A. Lee and Y. (Jake) Liang

Liang, Y., Bejerano, A. R., McPherson, M., Kearney, P., & Plax, T. (2015). The effects of peer and online sources of information on student course selection and impressions towards prospective teacher. Western Journal of Communication, 79, 435–455. Manchala, D. W. (1998). Trust metrics, models and protocols for electronic commerce transactions. In Proceedings of the 18th International Conference on Distributed Computing Systems (pp. 312–321). IEEE. Marble, J. L., Bruemmer, D. J., Few, D. A., & Dudenhoeffer, D. D. (2004). Evaluation of supervisory vs. peer-peer interaction with human-robot teams. In Proceedings of the 37th Hawaii International Conference on System Sciences. IEEE. Mayer, R. C., Davis, J. H., & Schoorman, F. D. (1995). An integrative model of organizational trust. Academy of Management Review, 20, 709–734. Mumm, J., & Mutlu, B. (2011). Human-robot proxemics: Physical and psychological distancing in human-robot interaction. In Proceedings of the 6th International Conference on Human-Robot Interaction (pp. 331–338). ACM. Nass, C., & Moon, Y. (2000). Machines and mindlessness: Social responses to computers. Journal of Social Issues, 56, 81–103. National Science Foundation. (2016). National robotics initiative (NRI): The realization of co-robots acting in direct support of individuals and groups (Program Solicitation: 16-517). Retrieved from http://www.nsf.gov/pubs/2016/nsf16517/nsf16517.htm. Neuliep, J. W., & Grohskopf, E. L. (2000). Uncertainty reduction and communication satisfaction during initial interaction: An initial test and replication of a new axiom. Communication Reports, 13, 67–77. Nomura, T., Kanda, T., & Suzuki, T. (2006). Experimental investigation into influence of negative attitudes toward robots on human–robot interaction. AI & Society, 20, 138–150. Polishuk, A., & Verner, I. (2018). An elementary science class with a robot teacher. In W. Lepuschitz, M. Merdan, G. Koppensteiner, R. Balogh, & D. Obdržálek (Eds.), Robotics in education: Latest results and development (pp. 263–273). Cham, Switzerland: Springer. Reeves, B., & Nass, C. (1996). The media equation: How people treat computers, televisions, and new media like real people and places. New York, NY: Cambridge University Press. Rosen, C. A. (1985). Robots and machine intelligence. In S. Y. Nof (Ed.), Handbook of industrial robotics (pp. 21–28). New York, NY: Wiley. Rotter, J. B. (1967). A new scale for the measurement of interpersonal trust. Journal of Personality, 35, 651–665. Serholt, S. (2018). Breakdowns in children’s interactions with a robotic tutor: A longitudinal study. Computers in Human Behavior, 81, 250–264. Serholt, S., & Barendregt, W. (2016). Robots tutoring children: Longitudinal evaluation of social engagement in child-robot interaction. In Proceedings of the 9th Nordic Conference on HumanComputer Interaction (NordiCHI ‘16), Gothenburg, Sweden. Sheridan, T. B. (1984). Supervisory control of remote manipulators, vehicles and dynamic processes: Experiments in command and display aiding. In W. B. Rouse (Ed.), Advances in man-machine systems research (Vol. 1, pp. 49–173). Greenwich, CT: JAI Press. Spence, P. R., Westerman, D., Edwards, C., & Edwards, A. (2014). Welcoming our robot overlords: Initial expectations about interaction with a robot. Communication Research Reports, 31, 272–280. Sunnafrank, M. (1986). Predicted outcome value during initial interactions: A reformulation of uncertainty reduction theory. Human Communication Research, 13, 3–33. Turner, J. H. (1988). A theory of social interaction. Stanford, CA: Stanford University Press. Wagner, A. R. (2015). Exploring human-robot trust: Insights from the first 1000 subjects. In Proceedings of the Collaboration Technologies and Systems Annual Conference (pp. 485–486). IEEE. Wang, N., Pynadath, D. V., & Hill, S. G. (2016). Trust calibration within a human-robot team: Comparing automatically generated explanations. In Proceedings of the 11th ACM/IEEE International Conference on Human Robot Interaction (pp. 109–116). IEEE.

A Communication Model of Human–Robot Trust Development …

115

Westlund, J. M. K., Dickens, L., Jeong, S., Harris, P. L., DeSteno, D., & Breazeal, C. L. (2017). Children use non-verbal cues to learn new words from robots as well as people. International Journal of Child-Computer Interaction, 13, 1–9. Yagoda, R. E., & Gillian, D. J. (2012). You want me to trust a robot? The development of a humanrobot interaction trust scale. International Journal of Social Robotics, 4, 235–248. Yamagishi, T., & Yamagishi, M. (1994). Trust and commitment in the United States and Japan. Motivation and Emotion, 18, 129–166.

Seungcheol Austin Lee is Associate Professor in the School of Communication at Chapman University. He studies how emerging information communication technology influences human attitudes and behaviors. His research integrates communication and human–machine interaction to advance a theory of persuasive technology that enables robots and agents to build credibility, gain trust, and, ultimately, function more effectively. Yuhua (Jake) Liang (in memoriam) was Assistant Professor in the School of Communication at Chapman University. His work examined the theoretical connections between emerging technologies (e.g., participatory systems, agents, and robots) and their practical persuasive effects. He endeavored to utilize technology to be more communicatively influential and develop theories that help us understand the fundamental process of communication.

An Evaluation of the Effectiveness of Using Pedagogical Agents for Teaching in Inclusive Ways Maggi Savin-Baden, Roy Bhakta, Victoria Mason-Robbie and David Burden

Abstract This chapter presents research on the use of pedagogical agents as a tool to support the learning of skills related to the transposition of formulae. Participants from diverse backgrounds were recruited from those being taught on a compulsory mathematics course and allocated to one of three conditions. Each undertook a one-hour training session on mathematical transposition appropriate to their group allocation. The Approaches and Study Skills Inventory for Students (ASSIST) questionnaire and Technology Acceptance using a questionnaire based on the Technology Acceptance Model Framework (TAM) were administered. Interviews and focus groups were undertaken to explore their experiences. The pedagogical agent provided a positive learning experience that enabled learners to achieve the same levels of attainment as those who undertook human teaching. There is a need to improve techniques for designing and encoding the database of responses to natural language inputs and to make more use of automated strategies for acquiring and constructing databases. However, it is evident that this model of learning can be used to increase access to mathematics learning across sectors and devices. Such agents can be used with diverse learners, enabling them to personalise their learning and thereby improve the possibility for teaching in inclusive ways. Keywords Autonomous agent · Conversational agents · Pedagogical agents · Virtual human interaction · Voice font M. Savin-Baden (B) · V. Mason-Robbie University of Worcester, Worcester, UK e-mail: [email protected] V. Mason-Robbie e-mail: [email protected] R. Bhakta Capp & Co. Ltd., Birmingham, UK e-mail: [email protected] D. Burden Daden Limited, Birmingham, UK e-mail: [email protected] © Springer Nature Singapore Pte Ltd. 2019 J. Knox et al. (eds.), Artificial Intelligence and Inclusive Education, Perspectives on Rethinking and Reforming Education, https://doi.org/10.1007/978-981-13-8161-4_7

117

118

M. Savin-Baden et al.

1 Introduction The use of autonomous agents has been increasing year on year as an easier means to interact with technology, databases and knowledge repositories. The literature presents numerous studies that suggest the importance of exploring the use of these agents within a learning context. Autonomous agents used within a learning context are generally referred to as pedagogical agents. These agents aim to support learners through the learning process and to date have largely been used in higher education. The use of pedagogical agents ranges from supplementing existing instruction to being a replacement for human teachers. What is required is the use of such agents in places of widening access, increasing diversity and spaces that work against standardised models of learning. Ongoing advances in technology and computing devices have seen an increased demand for improved ways to make this technology more accessible and easier to use for consumers. Pressure from end-users for easier to use and more intuitive interfaces (e.g. Harrison et al. 2013) has pushed research and development to identify evermore innovative and cost-effective methods for improving the quality of human-computer interactions. One recent development is the autonomous agent which can be defined as pieces of software that allow the user to interact with them using natural language through spoken or types of questions or statements (Rickel et al. 2002: 32). These Agents include online ‘virtual assistants’ that aim to help users with their purchasing requirements. Examples include ‘Anna’ on the Ikea website or the KiK service (KiK Interactive Inc. 2016). Similarly, many current smartphone and computer interfaces are utilising similar interfaces (e.g. ‘Siri’, Alexa, ‘Cortana’ or ‘Google Now’) to improve the ability of users to perform everyday tasks such as booking train tickets, making entries in calendars or searching for information. Pedagogical agents have been found to improve motivation and to some extent accessibility to the knowledge base being taught (e.g. Bowman 2012; Schroeder and Adesope 2014). A systematic review of the literature by Schroeder and Adesope (2014) examined 15 studies and suggests that the use of pedagogical agents can be beneficial for learning, attributing several potential benefits including both improved motivation and reduced cognitive load amongst the learners. Furthermore, Schroeder and Adesope’s review suggests that the use of pedagogical agents can result in improvements in attainment with d = 0.2 (small effect size), which is in line with Mayer (2005, 2014). It should be noted that Schroeder and Adesope (2014) suggest that the studies to date vary greatly in experimental design and also perceived effectiveness of the pedagogical agent, with pedagogical agents in some cases being perceived as less trustworthy and having less value as a teacher in comparison with a human. Pedagogical agents have been used across a wide range of subjects to support teaching. One area of use is that of mathematics, whereby the learners need to learn facts and develop their problem-solving skills and strategies. An example of pedagogical agents that have been used to support learning includes AutoTutor (Graesser et al. 2014), which aimed to provide students with a virtual tutor that could respond

An Evaluation of the Effectiveness of Using Pedagogical Agents …

119

intelligently to their questions and also simulates teacher like responses that could help students improve their skills by providing immediate feedback. While research has examined the effectiveness of pedagogical agents, most studies examine their role as an information giver rather than a guide or coach. Inclusion in this chapter is defined as policy and practice that recognises the differences in ways digital technologies can support or enhance learning for different groups of users, irrespective of age, gender or ability. Research into the use of pedagogical agents for raising and engaging with issues of inclusion and disability is few, and much of the research to date has been in the field of virtual worlds. For example, in 2010 Carr and colleagues reported on the reaction of the Deaf community when Second Life added speech, which resulted in a conflict between deaf protesters objecting to voice functionality, and non-disabled users, who viewed the protesters as ‘martyrs’ requiring ‘special measures’ to cope in Second Life (Carr et al. 2010). Furthermore, in a study by Davis and Chansiri (2018) who have been exploring virtual worlds and disability communities for over seven years, show the importance of choice in online representation of avatars in creating work and online social engagement. Their study illustrates the continuing presence of visual bias in the workplace and shows the ways in which emerging virtual reality technologies can provide social interactions for people with disabilities. However, the paucity of studies of pedagogical agents examining the coaching aspect underlines the importance of the present research. Given the advances in other technologies such as artificial intelligence, improved understanding of the role of personalisation of the pedagogical agent, and its potential for tutoring provision would be worthwhile. This chapter presents a comparative study of the use of pedagogical agents as a tool to support the learning of skills related to the transposition of formulae. Participants were from diverse backgrounds and were recruited from those being taught on a compulsory mathematics course. Participants were allocated to one of three conditions (pedagogical agent, Distance Learning, face-to-face teaching).

2 Study Rationale The rationale for his study was to move beyond the current work carried out within educational and commercial sectors, which largely focuses on agent appearance and content coverage, and instead focus on creating a relationship between the pedagogical agent and user. Studies in the US Education System have explored the social interactions of autonomous agents and students, considering the human-like qualities of agent-student interaction and in particular the realism of the agent and agent appearance. The focus has thus been on the development of agents that are human-like and can complete tasks in an efficient manner, in effect issues of design. According to Kim and Baylor (2015), successful learning and engagement seem to be related to the extent to which there is a perceived relationship or association between the pedagogical agent and the user. These features can be summarised as follows:

120

M. Savin-Baden et al.

• Appearance: Learners tend to be more influenced by a pedagogical agent of the same gender and ethnicity as them, similar to human–human interactions where humans are more persuaded by members of their in-group. Yet, both the learning/motivational context, age of learners and topic play a significant role, which builds on earlier studies that indicated that participants did not identify physical characteristics but rather the emotions that these characteristics invoke. Feelings of approachability, friendliness and professionalism seemed to be particularly important, along with ensuring a non-threatening approach. The physical appearance of the pedagogical agent thus helped to shape participants’ feelings of immersion in the engagement and the sense of social presence they experienced (Savin-Baden et al. 2013). • Attitude: Using the pedagogical agent as a motivator that demonstrates positive attitudes towards the task, and the desired levels of performance seem to be helpful for learners to cope in situations where they feel themselves to be novices. • Interaction Model: Using the pedagogical agent as a guide or friend seems to be more effective than using one that is perceived to be a high-level expert. Where a pedagogical agent acts as a guide, users indicate significantly enhanced motivational outcomes as compared to the mentor who is an expert. • Perceived Competence: pedagogical agents with similar competency to learners were more influential than highly competent agents in enhancing student selfefficacy beliefs. The research reported in this chapter explores the effectiveness of the pedagogical agent by adopting a mixed-methods approach using quantitative measures and qualitative interviews to focus on: • How the pedagogical agent operates technically and pedagogically. • The value of the pedagogical agent in different contexts, on different tasks, and in different settings. • The perceived advantages and disadvantages for those using the pedagogical agent. • The ways in which the intellectual tasks, academic capabilities, and experiences of learners are enhanced through the pedagogical agent. • What is most valued and most valuable as well as the most significant features.

3 Methodology This study employed a mixed-methods design and used a maximum variation sampling technique (Patton 1990). Qualitative data was collected through semi-structured interviews, and quantitative data through both objective (maths attainment) and subjective [learning approaches (ASSIST) and technology acceptance (TAM)] selfreport measures.

An Evaluation of the Effectiveness of Using Pedagogical Agents …

121

Table 1 Participant details (n = 127) Group 1 (pedagogical agent, n = 52)

Group 2 (online, n = 36)

Group 3 (human, n = 39)

Number of males

47 (90.4%)

34 (94.4%)

34 (87.2%)

Age (mean)

20.3

22.1

19.6

Mathematics level (number with grade GCSE grade C)

13 (7 unknown)

11 (11 unknown)

13 (8 unknown)

Ethnicity (British)

39 (75.0%)

26 (72.2%)

32 (82.1%)

3.1 Participants Participants were recruited from an organisation where new entrants are required to undertake a period of compulsory training that includes mathematics (at approximately GCSE level) during the initial stages. The pedagogical agent was text-based rather than voice activated, and the output was text and audio. Those who volunteered to participate did so with the knowledge that they were involved in a study to assess the effectiveness of pedagogical agents for the use of learning mathematics. A power analysis suggested that a minimum of 108 participants would need to be recruited in order to observe medium to large effects (see Table 1 for participant characteristics). Based on previous cohorts who have taken this initial training, it was expected that the majority of participants would have a GCSE mathematics qualification or lower. Information from the organisation suggests that in rare instances an individual may enter with a numerate degree.

3.2 Design In order to assess the utility of the pedagogical agent for improving achievement and knowledge retention a 3 × 2 mixed design was used with a parallel pre-test posttest experimental design rather than a cross-over design. A cross-over design was considered where each participant would participate in all the experiential conditions; however, this was deemed to be problematic (Matthews 1988; Yang and Stufken 2008), due to carry-over effects, learning effects and order effects.

3.2.1

Quantitative Measures

The effectiveness of the pedagogical agent was examined using quantitative measures that focused on participants’ assessment of the perceived usefulness of the agent, their ability to access the knowledge and a measure of the learning (achievement and knowledge retention) that had taken place:

122

M. Savin-Baden et al.

• Attainment Questionnaire—knowledge and understanding relating to the topic of transposition within mathematics was measured using an attainment measure. This was the key metric for this study and was used to assess the utility of the pedagogical agent. The items for this inventory were generated in conjunction with the current learning materials and topic content currently used to teach learners on the course. This was administered immediately prior to and following the learning intervention. • Technology Acceptance Model Form (TAM)—items used in this study were based on the Technology Acceptance Model Framework, and the items proposed in the literature (Davis 1993; Venkatesh and Bala 2008; Lorenzo et al. 2013) to assess usability and perceived usefulness of the pedagogical agent. The items and subscales used aimed to explore how individuals felt about using the technology by exploring anxiety, ease of use, and how in control of the technology an individual perceives themselves to be. • Learning Approaches using the ASSIST Questionnaire Form—the Approaches to learning questionnaire (Tait et al. 1998) was used to evaluate participants’ learning approach in order to understand whether those with a particular approach engaged with the agent more effectively.

3.2.2

Qualitative Data

Semi-structured interviews (n = 5) and focus groups (n = 3) were used to examine which features of the pedagogical agent could be improved and the user’s perceptions and experiences of the agent. The purpose of the focus groups was to collect views of users of the pedagogical agent in order to synthesise perspectives. Each of the focus groups consisted of between five and eight participants and lasted up to an hour. Interviews and focus groups were audio-recorded and transcribed verbatim, and recordings were destroyed after completion of the study.

3.3 Procedure Group 1 undertook a period of learning with the pedagogical agent tutor. Group 2 undertook the training by accessing the same content online in a distance learning fashion, but without the support of the pedagogical agent or Human tutor. Participants in Group 3 undertook the training using traditional face-to-face teaching methods with a human tutor only. The effectiveness of the pedagogical agent was measured by a comparison of Group 1 to Groups 2 and 3. For each group, data was collected at two points during the study using the online questionnaires described above in addition to the qualitative interviews and focus groups. All participants completed a test of their knowledge prior to and within 24 h of the training period. Learning Approaches (ASSIST) were assessed within 24 h of the intervention for all participants. For those

An Evaluation of the Effectiveness of Using Pedagogical Agents …

123

in Groups 1 (pedagogical agent) and 2 (Online), the Technology Acceptance Model (TAM) questionnaire was completed following the intervention.

3.4 Ethics Ethical clearance was gained from the University ethics committee. Given that different kinds of instruction were provided in the three conditions, it was necessary to ensure all participants had access to the different instruction methods after the completion of the study.

4 Findings Quantitative data analysis focused on a comparison of the changes in attainment over time (baseline and post-intervention) across the three experimental groups, using an Analysis of Covariance (ANCOVA). This analysis aimed to highlight differences between using the pedagogical agent and current teaching methods in place at the organisation. Qualitative Data Analysis involved a thematic analysis (Braun and Clarke 2006). Analysis began by identifying codes and their meaning in the context of the data. These codes were then clustered into semantically related categories and subsequently developed into themes. Close attention was paid to interpreting the experience and the associated meaning to participants.

4.1 Quantitative Findings Upon completion of the study, the data (summarised in Table 2) suggested that the three groups (pedagogical agent, Online, Human) were comparable in terms of their retention of the skills associated with transposition attainment, although the human group received a slightly longer duration of intervention, which may partly explain the modest increase in attainment observed. Learners across the three groups reported favouring a deep, followed by strategic and then surface learning approach. Overall, learning approach scores were highest for the Online and Human groups and lowest for the pedagogical agent group. For the pedagogical agent groups, scores on the measure of technology assessment, the TAM, were highest for computer playfulness and lowest for computer anxiety. For the Online group, scores were highest for the perception of external control and also lowest for computer anxiety. • PU—Perceived Usefulness—How useful does the individual feel the software/technology being used is in relation to their own needs;

124

M. Savin-Baden et al.

Table 2 Mean test scores on the attainment, ASSIST and TAM measures Group 1 (pedagogical agent, n = 52)

Group 2 (online, n = 36)

Group 3 (human, n = 39)

Attainment—pre

4.0

4.6

4.8

Attainment—post (1 day)

4.0

4.6

5.4

Assist—post (1 day)

Deep 26.7 Strategic 26.1 Surface 24.8

Deep 40.7 Strategic 38.6 Surface 35.3

Deep 37.9 Strategic 37.3 Surface 34.8

TAM—post (1 day)

PU 2.9 PEU 3.4 CSE 3.2 PEC 3.4 CP 4.2 CA 2.2 PE 2.8

PU 4.2 PEU 4.8 CSE 4.6 PEC 4.9 CP 4.6 CA 2.2 PE 4.5

Not applicable

• PEU—Perceived Ease of Use—Does the individual feel that the software/technology is easy or difficult to use; • CSE—Computer Self-Efficacy—A self-evaluation of the individual’s ability to use a computer/technology/software. Do they feel confident; • PEC—Perceptions of External Control—How confident is an individual in being able to control the way a computer works. For example, do they feel they are skilled/able enough to change the way it behaves or have ‘control’ over its workings? Or does the individual feel they are not very adept and can do little to change how a computer behaves; • CP—Computer Playfulness—How willing or able is an individual to try new approaches or be creative in their use of computers/software/technology; • CA—Computer Anxiety—Does the individual have a general fear or apprehension when using a computer; and • PE—Perceived Enjoyment—How much fun does the individual feel they have when interacting with the computer/software/technology.

4.2 Qualitative Findings Three broad themes were identified from the data: first, self-efficacy which includes aspects of self, level of understanding of the topic of learning, confidence in ability and pre-conceived ideas about using a pedagogical agent, second, virtual human interaction in terms of the attributes and functionality of the pedagogical agent and third, learning preferences including views about the wider learning context in which studying with the help of a pedagogical agent takes place, participant views about its usefulness, and suggestions for improving its functionality.

An Evaluation of the Effectiveness of Using Pedagogical Agents …

4.2.1

125

Theme 1: Self-Efficacy

Bandura (1986: 391) defined self-efficacy as ‘People’s judgments of their capabilities to organize and execute courses of action required to attain designated types of performances’. Thus, self-efficacy is a belief about one’s capability and which may not match actual capability in a specific domain. However, perhaps what is more important for this study is that as Bandura argued, individuals use efficacy judgements in order to attain a goal. Early studies in the development of pedagogical agents suggest that it is important that they are enhancing a learner’s self-efficacy beliefs; their belief that they are able to accomplish tasks (Baylor 2009). Participants talked about aspects of themselves including their knowledge of the subject, their confidence and their personal views. Individuals who knew what they were studying can be contrasted with those who do not: [It is] helping me with transposition and maths. We were working on just maths. I can’t even remember what the main subject was. Well obviously, maths, but I’ve completely forgot[ten].

Such contrasting examples illustrate that there were different levels of knowledge and confidence prior to engaging with the pedagogical agent. Those with higher knowledge or confidence may find it easier than those who do not, as these participants suggest: If you didn’t really have an understanding of transposition before, I don’t think it would help too much in learning it from the beginning. Maybe some of the students are a bit … You’ve got a bit more ability about it [you] and a bit more independence. They don’t really need the help from the teacher, they could put … Obviously use it on the laptop, plug headphones in and use it on the laptop. I see it being very effective that way.

Knowledge of the subject being taught, confidence and beliefs about learning was central aspects of this theme and relates to aspects of self and self-efficacy. These perspectives were related to attitudes towards, and beliefs about, the pedagogical agent which is the subject of the next theme. To date there are few studies that have explored the impact of virtual humans or pedagogical agents for motivating learning, and most of these have been undertaken in secondary schools. For example, a study by van der Meij et al. (2015) sought to understand if students’ motivation and knowledge changed over time and whether gender affected such changes but found that the knowledge gains and motivation were the same for boys and girls. The authors conclude that in designing a motivational agent that can influence student motivation a strong focus should be on the design of the agent.

4.2.2

Theme 2: Virtual Human Interaction

The application of pedagogical agents to learning settings introduces questions about how these technologies in new spaces alter how pedagogy might be seen. Perhaps

126

M. Savin-Baden et al.

what is being seen is what Thrift has termed ‘augmented existence’, in which it is not just tagging and integration of new technologies that affect our lives and practices but the recognition that the meta-systems themselves become a new form of categorisation (Thrift 2006). In the present study, participants discussed their opinions of the pedagogical agent and its interactive functionality. For example, the information was trusted, and the interface was of a high standard and allowed the individual to work at their own pace. Because once it gets to its finishing stages then I think that would be an absolutely crucial bit of kit. Because I think they could shrink training time down because everyone will be able to learn at their own paces in a much more interactive and sort of non-pressured way. Because if you’ve got a teacher leaning over you, then you’re a bit, oh God. But if it’s just you and the computer, then I think it’s a bit more relaxed.

Regarding some of the more critical views expressed, the voice font was mentioned as a problem for some, as was the difficulty phrasing questions (through textual input) in a way that would elicit an appropriate response. I think the voice. Yes, the voice recording was quite annoying. It does get on your nerves a bit, especially after using it for quite a while. And then obviously the program; typing all the questions in and that. Then it not recognising … But that’s about it.

However, not just the robotic sound of the voice per se, but the nature of the speech and script and the absence of interactivity was also highlighted. I think the voice. It’s like … Even just itself. Although the sort of script that it is run on is limited in itself but the actual the sound of the voice, because it was very sort of robotic. It’s not very human-like or quite engaging. It’s sort of just talking at you. So, I think if the voice was changed to a more modern sounding voice. So similar to a conversation like this, then I think it would be a bit more helpful, rather than just, yes, a robot talking at you sort of thing. Because it does sound quite robotic.

Conversely, the voice was also seen as a positive aspect of its functionality because it allowed the learner to hear as well as read the information. This relates to metacognition because it shows that the learner is aware of the way that they learn best. Yes, because some like … I prefer people speaking to me to learn it, not having to read it quite a lot because I get more when it’s spoken.

As with the participant above, the following participant found it difficult to phrase questions in a way that the pedagogical agent would provide an answer that helped learning. Like with the virtual learning thing, you have to ask questions a certain way and it’s like quite hard to like word the questions how they want you to word them. But with a teacher, they will understand if you word it a bit different.

Overall, there were some contrasting views about aspects of the pedagogical agent’s functionality including the voice and the difficulty in phrasing questions to elicit a response that aided learning. Evidence has shown that many users are not only comfortable interacting with high-quality pedagogical agents, but that an emotional

An Evaluation of the Effectiveness of Using Pedagogical Agents …

127

connection can be developed between users and pedagogical agents, resulting in a more positive engagement experience. For example, Hasler et al. (2013) found, in a comparison of human interviewees with virtual world chatbots (pedagogical agents in non-learning situations) that chatbots and human interviewees were equally successful in collecting information about their participants’ real live backgrounds. In the next theme, preferences and recommendations made by participants are explored.

4.2.3

Theme 3: Learning Preferences

There is increasing focus in the twenty-first century on what and how students learn and on ways of creating learning environments to ensure that they learn effectively—although much of this remains contested ground. Veletsianos and Russell (2013) argue that the social discourse that occurs between agents and learners are generally overlooked in the education literature and suggest it is vital to examine explicit and concealed meanings, in order to gain in-depth understandings of agent-learner interactions and relationships. The authors argue that well-designed agents can ask guiding questions, prompt reflection, provide feedback and summarise information. The findings in this chapter also suggest that more research is needed to examine the extent to which deep, surface and strategic approaches to learning affect student engagement with a pedagogical agent. This theme illustrates the way in which participants’ learning approaches and preferences affected their engagement with the agent and indicates the ways in which such preferences could inform future development. It also relates to the preferences individuals have with regard to what constitutes effective learning, including ideas around meta-cognition, i.e. understanding of how they and their juniors learn. It includes the perceived advantage of being able to learn at your own pace with the virtual tutor, the preference for a ‘real’ teacher, and the desire for a more interactive interface. Preferences and views about the pedagogical agent’s usefulness were identified in the context of learning. In the focus group, it was suggested that: It might be quite beneficial to have more than one kind of method of working out or explaining it at least.

Thus, providing further examples and alternative explanations for concepts being learnt would be advantageous to prospective users. A number of useful suggestions were made to improve learning when interacting with the pedagogical agent, for example, options to modify the level of difficulty. Many comments related to the apparent preferences for a human tutor because they are able to interact and have a conversation to enable them to understand something, rather than having to try to rephrase a question to elicit the answer they need. In the area of education, people are likely to make comparisons with a human tutor because that represents the majority of their experience and is therefore their reference point. This may lead to less favourable comparisons because the human tutor is held as the ‘gold standard’ of teaching.

128

M. Savin-Baden et al.

Me personally, I like a tutor with obviously … Not a program. Just because on the program when you write in, you have to write it really specifically if you have a question. Obviously in person you can explain it and if they don’t understand, you can explain it more or whatever. But if you do it on the PC then you just have to make it really straightforward and straight to it. And if it’s wrote incorrectly it just won’t identify it. So that was a bit of a problem. But besides that, it was pretty good.

Participants also made some recommendations, in particular, the benefits of using the pedagogical agent for revision and self-paced study and in conjunction with a human tutor as these participants explain. Those who are slower, again, they can work at their own pace. So, in that respect, yes, it would be quite a good benefit and also for revision piece. If you don’t have access to your tutor, you can do it in your room. You can go through a lesson, like I said, in your room and not only … You don’t have to do it by yourself as well. If you really want to, you can get a friend as well, which … Yes, I think that would be the best benefit of it, very much so.

In the quote below, there is a sense of the advantage of having both a teacher and a pedagogical agent for effective learning. Having a teacher in front of you or having the virtual one on the screen, because it means that you’ve got the option if you’re in a class. You do that and then you can go out of class and if you’re doing revision, you can sort of have the teacher there with you but it’s not actually there with you. If you know what I mean?

Overall, learning preferences considers the preferences of individuals, their beliefs about the improvements that can be made, and their recommendations for how they can see the pedagogical agent being used in future.

5 Discussion The demographic data from Table 1 highlight that the intake of recruits entering into the training program was predominantly white and male and was not likely to have achieved a grade C or better in GCSE mathematics (at least 56% had not achieved this). The data suggests a high bias towards these groups within the participant sample. To some extent, this was observed in the larger pool of those who did not participate in the research at the research site. While females and those from minority groups were represented in the samples and across the observed cohorts, these accounted for a small proportion of all individuals. Regarding the qualitative findings, there was significant variation in the views of participants from those with an overwhelmingly positive view of the pedagogical agent, to those with a more critical perspective. In general, positive views were expressed by those who felt quite confident in their maths ability. Conversely, less confident individuals were critical but keen to acknowledge that others might benefit. This point reflects the importance of the learning approach adopted and their approach to using technology, both of which were measured in the quantitative part of the study. Some individuals showed evidence of meta-cognition, i.e. awareness of their own approach to learning and how

An Evaluation of the Effectiveness of Using Pedagogical Agents …

129

the ability to both listen and see the information through interacting with the pedagogical agent helped. Many views were expressed about aspects of the pedagogical agent that were liked and those that were disliked. In particular, the ability to learn at your own pace was viewed positively, and this could be viewed as supporting and promoting inclusivity. Conversely, the voice font was problematic. Participants also offered suggestions about how the pedagogical agent could be used in future. Such user feedback is crucial to the development of the technology itself and promotes acceptable and inclusive use in the context of learning. A number of theories can be drawn upon to make sense of the findings. Selfefficacy has long been linked with educational performance in educational settings (Bandura 1977; Pajares 1996; Pajares and Miller 1994). In the present context individuals who expressed confidence in their own ability also felt more confident in using the pedagogical agent. Thus, individuals with less knowledge or lower confidence levels may be more reluctant to engage if they take confidence from interacting with a human tutor. Indeed, this view is expressed by a number of participants. Although not tested, aspects of personality such as openness to experience may also impact on willingness to engage in a new experience such as learning through interacting with a pedagogical agent. Furthermore, how aware individuals are about their own learning, or meta-cognition (Flavell 1976; Hacker et al. 2009) may also be important as those who know how they learn best may be reluctant to engage with other methods that do not fit with their preferred modality. To use this technology in inclusive ways, learners need to be supported in understanding their own learning strategies and to build on individual self-efficacy to promote more effective engagement. The findings from the pedagogical agent interviews and focus groups show that self-efficacy, experience and interaction with the pedagogical agent, and learning preferences, impact on the attitudes towards the pedagogical agent. Particularly for subjects where factual content is high, having a strong belief in one’s own ability is important for willingness to engage with a pedagogical agent. Attitude towards aspects of the pedagogical agent’s interface varied according to individual preferences, but the most frequently mentioned topics were dislike of the voice and frustration when the system did not recognise the words being typed by the users. Such technical features would be an important target for adjustment to improve functionality and improve the user experience. Learners varied in their attitude towards using technology, but in some instances, a preference emerged for human tutors, which provides an opportunity for a pedagogical agent to be used to complement the learning achieved in the classroom. Understanding the way that learning takes place within a particular context may be essential to finding a role for pedagogical agents. However, higher education it seems is a liminal space, subject to government demands for accountability characterised (in the UK at least) by the Research Excellence Framework, the Teaching Excellence and Student Outcomes Framework, and Knowledge Exchange Framework, as well as the emergence of more covert forms such as ‘transferable open educational resources’ and ‘lecture-capture’. As Hall (2016: 1012) has noted:

130

M. Savin-Baden et al.

The enclosure of the University under neoliberalism by transnational activist networks is one critical form of dispossession. This has tended to overwhelm academic autonomy, in order to facilitate the accumulation of value.

He suggests that there needs to be: … the recognition that co-operative organising principles, both for the University and for the curriculum, might offer a set of alternatives to the prevailing political economy of higher education.

The issue here is that the adoption of inclusive strategies through pedagogical agents is likely to be hampered through a neoliberal restraint on academic freedom. Learning today is a mashup of home-school-work-media-peer-collaboration. For education to be inclusive it would be important to see learning as unbundled, as something that no longer largely takes place within educational institutions, but instead includes some of the following practices (based on Savin-Baden 2015): • Mentorship: using mobile devices to keep in touch with parents or other significant adults in order to get advice, feel supported or use as a sounding board through Whatsapp or Facebook messaging. • Gaming: alone and together to share, teach, learn, offer advice, negotiate and give and receive hints, tips and solutions. • Co-operative online learning: supporting and guiding each other about homework, assignments and exam revision. • Teaching technology: sharing and teaching each other about apps, new devices and helpful sites. • Emotional learning: using digital media for peer-to-peer support to manage personal challenges and difficulties, and to receive advice. • Playful learning: trying things out and fiddling around, in order to experiment and discover. • Co-production: creating presentations together, making and sharing cybercreations, creating posters, mashups and vidding. Therefore, in terms of artificial intelligence and inclusion, this study suggests that as Treviranus (2017) has highlighted, there are potential dangers in just relying on machine learning approaches for applications, and it is important that apps and agent are inclusive and work for as many people as possible. However, at the same time, as Trewin (2018) has pointed out, the issues of fairness are complex. She suggests that fairness for people with disabilities is different to fairness for other protected attributes such as age, gender or race, and argues for different ways of approaching fairness for people with disabilities in AI applications. We suggest then that for education, and technology guided education in particular to be inclusive, there is a need for staff to have the capacity and autonomy to improvise, enquire and take intellectual risks in order to ensure higher education (and agents used within them), remains a place of creativity and experimentation.

An Evaluation of the Effectiveness of Using Pedagogical Agents …

131

6 Conclusion This study has demonstrated that there is a role for pedagogical agents in the learning of mathematics and that this could be adapted to deliver education, ‘regardless of any perceived difference, disability or other social, emotional, cultural or linguistic difference’ (Florian 2008: 202). Following further development and resolution of technical concerns, a number of suggestions for how the technology can be utilised can be suggested: • The pedagogical agent provides a useful complement to human tutors to cement the learning that takes place in the classroom. • It provides an opportunity to learn factual information at a person’s own pace. • It provides an opportunity to learn where people are geographically dispersed. • It would have considerable utility for pre-learning, refresher courses and for revision prior to assessment. Given the variation in response to questions about the pedagogical agent, it might be that a virtual tutor is useful for certain individuals (those with greater knowledge and confidence in their own ability) and in certain learning contexts (revision, refreshing knowledge). Beyond the issues with functionality, this research raises questions about what the barriers might be to a positive learning experience with a pedagogic agent. Preference for a human tutor, reluctance to engage with technology, aspects of functionality and desire for greater interactivity might all be important predictors of engagement.

7 Future Directions Daden, KSharp, the University of Worcester, and the University of Warwick1 have been working to create a virtual persona, which is a digital representation of some of the memories, knowledge, experiences, personality and opinions of a specific living physical human. The project and persona are very much seen in the context of knowledge management and intended to explore how a virtual persona could aid in knowledge capture, retention, distribution, access and use. There are also other high-profile android/chatbot projects which should be approached with caution. For example, in Autumn 2017 Sophia, a humanoid robot gave a ‘speech’ at the United Nations (2017) and has even been granted citizenship of Saudi Arabia (Griffin 2017). While this may encourage recognition of the fact that there needs to be more debate, as well as legislation in this area, the level of technical accomplishment actually shown is debated (Ghosh 2018).

1 Sponsored

by Defence Science and Technology Laboratory (Dstl)—in collaboration with Dstl Technical Partners.

132

M. Savin-Baden et al.

References Bandura, A. (1977). Self-efficacy: Toward a unifying theory of behavioural change. Psychological Review, 84,191–215. Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory. Englewood Cliffs: Prentice Hall. Baylor, A. (2009). Promoting motivation with virtual agent and avatars: Role of visual presence and appearance. Philosophical Transactions of the Royal Society B, 364, 3559–3565. Bowman, C. D. D. (2012). Student use of animated pedagogical agents in a middle school science inquiry program. British Journal of Educational Technology, 43(3), 359–375. Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101. Carr, D., Oliver, M., & Burn, A. (2010). Learning, teaching and ambiguity in virtual worlds. In A. Peachey, J. Gillen, D. Livingstone & S. Smith-Robbins (Eds.), Researching learning in virtual worlds. UK: Springer. Davis, F. D. (1993). User acceptance of information technology: System characteristics, user perceptions and behavioral impacts. International Journal of Man-Machine Studies, 38(3), 475–487. Davis, D. Z., & Chansiri, K. (2018). Digital identities—Overcoming visual bias through virtual embodiment. Information, Communication & Society, 22(4), 491–505. https://doi.org/10.1080/ 1369118X.2018.1548631. Flavell, J. H. (1976). Metacognitive aspects of problem solving. In L. B. Resnick (Ed.), The nature of intelligence (pp. 231–236). Hillsdale, NJ: Erlbaum. Florian, L. (2008). Inclusion: Special or inclusive education: Future trends. British Journal of Special Education, 35(4), 202–208. Ghosh, S. (2018). Facebook’s AI boss described Sophia the robot as ‘complete b—–t’ and ‘Wizardof-Oz AI’. Business Insider UK. Retrieved from http://uk.businessinsider.com/facebook-ai-yannlecun-sophia-robot-bullshit-2018-1. Graesser, A. C., Li, H., & Forsyth, C. (2014). Learning by communicating in natural language with conversational agents. Current Directions in Psychological Science, 23(5), 374–380. https://doi. org/10.1177/0963721414540680. Griffin, A. (2017). Saudi Arabia grants citizenship to a robot for the first time ever. The Independent. Retrieved from https://www.independent.co.uk/life-style/gadgets-and-tech/news/saudiarabia-robot-sophia-citizenship-android-riyadh-citizen-passport-future-a8021601.html. Hacker, D. J., Dunlosky, J., & Graesser, A. C. (Eds.). (2009). Handbook of metacognition in education. Hall, R. (2016). Technology-enhanced learning and co-operative practice against the neoliberal university. Interactive Learning Environments, 24(5), 1004–1015. Harrison, R., Flood, D., & Duce, D. (2013). Usability of mobile application: Literature review and rationale for a new usability model. Journal of Interaction Science, 1(1). Hasler, B. S., Tuchman, P., & Friedman, D. (2013). Virtual research assistants: Replacing human interviewers by automated avatars in virtual worlds. Computers in Human Behavior, 29, 1608–1616. KiK Interactive Inc. (2016). KiK Bot shop—Lifestyle. https://bots.kik.com/#/category/lifestyle (viewed August 1, 2016). Kim, Y., & Baylor, A. L. (2015). Research-based design of pedagogical agent roles: A review, progress, and recommendations. International Journal of Artificial Intelligence in Education, 26(1), 160–169. Lorenzo, C. M., Lezcano, L., & Alonso, S. S. (2013). Language learning in educational virtual worlds—A TAM based assessment. Journal of UCS, 19(11), 1615–1637. Matthews, J. N. S. (1988). Recent developments in crossover designs. International Statistical Review/Revue Internationale de Statistique, 56(2), 117–127.

An Evaluation of the Effectiveness of Using Pedagogical Agents …

133

Mayer, R. E. (2005). Principles of multimedia learning based on social cues: Personalization, voice, and image principles. In R. Mayer (Ed.), The Cambridge handbook of multimedia learning (pp. 201–212). New York, NY: Cambridge University Press. Mayer, R. E. (2014). Principles based on social cues in multimedia learning: Personalization, voice, image, and embodiment principles. In R. Mayer (Ed.), The Cambridge handbook of multimedia learning (2nd ed., pp. 345–368). New York, NY: Cambridge University Press. Pajares, F. (1996). Self-efficacy beliefs in academic settings. Review of Educational Research, 66, 543–578. Pajares, F., & Miller, M. D. (1994). Role of self-efficacy and self-concept beliefs in mathematical problem solving: A path analysis. Journal of Educational Psychology, 86(2), 193–203. Patton, M. Q. (1990). Qualitative evaluation and research methods (2nd ed.). Thousand Oaks, CA: Sage. Rickel, J., Marsella, S., Gratch, J., Hill, R., Traum, D., & Swartout, W. (2002). Toward a new generation of virtual humans for interactive experiences. IEEE Intelligent Systems, 17(4), 32–38. Savin-Baden, M. (2015). Rethinking learning in an age of digital fluency is being digitally tethered a new learning nexus?. London: Routledge. Savin-Baden, M., Tombs, G., Burden, D., & Wood, C. (2013). It’s almost like talking to a person: Student disclosure to pedagogical agents in sensitive settings. International Journal of Mobile and Blended Learning, 5(2), 78–93. Schroeder, N. L., & Adesope, O. O. (2014). A systematic review of pedagogical agents’ persona, motivation, and cognitive load implications for learners. Journal of Research on Technology in Education, 46(3), 229–251. Tait, H., Entwistle, N. J., & McCune, V. (1998). ASSIST: A re-conceptualisation of the approaches to studying inventory. In C. Rust (Ed.), Improving students as learners (pp. 262–271). Oxford: Oxford Brookes University, Centre for Staff and Learning Development. Thrift, N. (2006). Space: Special issue on problematizing global knowledge. Theory, Culture & Society, 23, 139–146. Treviranus, J. (2017). AI’s problem with disability and diversity. Retrieved January 7, 2019 from https://www.cbc.ca/radio/spark/362-machine-learning-outliers-smart-device-ownership-andmore-1.4279433/ai-s-problem-with-disability-and-diversity-1.4279444. Trewin, S. (2018). AI fairness for people with disabilities: Point of view. Retrieved January 7, 2019, from https://arxiv.org/abs/1811.10670v1. United Nations. (2017). At UN, robot Sophia joins meeting on artificial intelligence and sustainable development. UN News. Retrieved from https://news.un.org/en/story/2017/10/568292-un-robotsophia-joins-meeting-artificial-intelligence-and-sustainable. van der Meij, H., van der Meij, J., & Harmsen, R. (2015). Animated pedagogical agents effects on enhancing student motivation and learning in a science inquiry learning environment. Educational Technology Research and Development, 63(3), 381–403. Veletsianos, G., & Russell, G. (2013). What do learners and pedagogical agents discuss when given opportunities for open-ended dialogue? Journal of Educational Computing Research, 48(3), 381–401. Venkatesh, V., & Bala, H. (2008). Technology acceptance model 3 and a research agenda on interventions. Decision Sciences, 39(2), 273–315. Yang, M., & Stufken, J. (2008). Optimal and efficient crossover designs for comparing test treatments to a control treatment under various models. Journal of Statistical Planning and Inference, 138(1), 278–285.

Maggi Savin-Baden is Professor of Education at the University of Worcester and has researched and evaluated staff and student experience of learning for over 20 years and gained funding in this area (Leverhulme Trust, JISC, Higher Education Academy, MoD). She has a strong publication record of over 50 research publications and 16 books which reflect her research interests on

134

M. Savin-Baden et al.

the impact of innovative learning, digital fluency, cyber-influence and pedagogical agents on student engagement and learning. She has two new books forthcoming, Virtual Humans, with David Burden and Stories of Displacement with Lynn Butler-Kisber, and Kerry Keefe. Roy Bhakta, Capp & Co. Ltd. works as an Analytics Consultant at Capp and Co. Ltd. where he is focused on data visualisation and analytics to support the development of assessment tools and processes. His interests are focused around the use of technology to improve learning, psychology of achievement, engagement within higher education with a focus on STEM and the promotion of equality in attainment and selection in the workplace. In addition to research, he is involved in research student supervision, and teaching research methods and statistics. Previously he has worked at the University of Worcester and at Coventry University as a researcher and has taught students in a variety of contexts including schools, colleges and Higher Education Sectors. Victoria Mason-Robbie is an Associate Lecturer in the School of Psychology and a Senior Research Fellow in the School of Education, University of Worcester. She obtained her B.Sc. (Hons) in Psychology from the University of Warwick, followed by an M.Sc. (with distinction) and a Ph.D. in Health Psychology from the University of Bath. She has held research posts at the Universities of Bristol and Warwick. Victoria is a Chartered Psychologist with the British Psychological Society, a Chartered Scientist, and a Fellow of the Higher Education Academy. Her interests lie within Education and Health Psychology, and she has written book chapters on the psychological aspects of pain and has published over 25 research papers in journals from the fields of psychology, medicine and education. In addition to teaching a wide range of undergraduate and postgraduate courses from the disciplines of psychology, medicine, and sport, her current research focuses on the use of pedagogical agents in particular learning environments. David Burden is a Chartered and European Engineer who started his career in army communications managing a range of mobile and wireless systems in a variety of challenging and operational environments. After being ‘demobbed’ in 1990, David joined Aseriti, the £70 m turnover IT arm of Severn Trent plc where he ended up as Marketing Director. David has been involved in virtual worlds and immersive spaces since the mid-1990s. David also has a keen interest in artificial intelligence and has been a finalist in the BCS Machine Intelligence competition. David set up Daden Limited in 2004 to help businesses and organisations explore and exploit the social and commercial potential of using immersive 3D and VR environments and chatbots for learning and education. David and his team have delivered nearly 50 immerse learning and training projects for clients in the UK, USA, Middle East and Far East, with projects winning numerous prizes including two Times Higher Education Awards. David has led over a dozen collaborative research projects funded by Innovate UK and the MoD with a number of UK universities and written many articles and academic papers on immersive worlds and AI, and contributed to several books on these topics.

Inclusive Education for Students with Chronic Illness—Technological Challenges and Opportunities Anna Wood

Abstract Although the general issues related to disability inclusion have been examined in the education literature, there is still insufficient discussion of those specific challenges experienced by students with chronic illness. This chapter explores how artificial intelligence technologies can support the educational inclusion of people with chronic illness. Drawing on my own experiences of living and studying with ME (myalgic encephalomyelitis/chronic fatigue syndrome), I will discuss the issues faced by students with chronic illnesses such as energy impairment, fluctuations in symptoms and cognitive difficulties and the educational challenges that these issues cause. I then explore the examples of nascent, emergent and futuristic AI technologies, sourced from both personal experience and community knowledge, that could enable better inclusion of students with chronic illness in education. These include systems which could make it easier to search for text, equations and diagrams in digital documents; voice-controlled applications which can be used to create nontextual artefacts such as diagrams and graphs; improvements to the production of spoken language from textual documents to create more natural speech; and intelligent tutor systems which are able to produce adaptive, tailored and interactive teaching, enabling students with chronic illness to gain the best possible learning experiences. Keywords Chronic illness · ME/CFS · Intelligent tutors · Natural language processing · Energy impairment

1 Introduction The aim of this chapter is to explore artificial intelligence (AI) technologies which have the potential to support the education of people with chronic illness, with examples sourced from both personal experience and community knowledge. While there A. Wood (B) University of Edinburgh, Edinburgh, UK e-mail: [email protected] © Springer Nature Singapore Pte Ltd. 2019 J. Knox et al. (eds.), Artificial Intelligence and Inclusive Education, Perspectives on Rethinking and Reforming Education, https://doi.org/10.1007/978-981-13-8161-4_8

135

136

A. Wood

are approximately 15 million people in the UK with a long-term physical health condition (Department of Health 2012), my focus here is on a specific group of people who identify as having a chronic illness rather than those who have been given a particular medical diagnosis. The chronic illness community has recently been given agency and voice through the Chronic Illness Inclusion Project (CIIP) which argues that what sets people with chronic illness apart is not their diagnostic categories but their experience of impairment (Hale 2018). One common factor amongst this group is the experience of significantly debilitating fatigue and/or pain which is not alleviated by medical interventions. Chronic illness can have a significant impact on access to both compulsory and further/higher education and can affect both children and adults. For example, it has been shown that ME/CFS is the most common cause of long-term sickness absence in school children and therefore has serious implications for the education of those affected (Dowsett and Colby 1997). My own experience illustrates how varied the impact of chronic illness can be. I became ill with ME in 1999 halfway through my Ph.D. After an acute phase, my health improved enough that I was able to return to complete my thesis, albeit with plenty of rest and careful planning. However when my health deteriorated, nearly a decade later, I became housebound and studying in a face to face setting was no longer possible. Instead, I took advantage of the growing possibility for online learning, which enabled me to study, part-time, for an M.Sc. in E-learning. However, while issues of disability in education have been the focus of much scholarship, the experiences of those with chronic illness are rarely addressed in the educational inclusion literature. This chapter will begin by briefly discussing the difference between illness and disability and how this affects our understanding of how technology may support educational inclusion in this group. I then explore the key issues facing people with chronic illness and the educational challenges that these bring. The remainder of the chapter then focusses on exploring specific educational challenges in detail and proposing speculative AI technologies which have the potential to support students with each of these challenges.

2 Illness Versus Disability In disability studies, the “social model” of disability is prominent (and contrasted with the “medical model” of disability). The social model argues that disabled people are disadvantaged by the social environment and people’s attitudes, rather than by their physical attributes. As the disabled activist Stella Young wrote, “My disability comes not from the fact that I’m unable to walk but from the presence of stairs” (2014). This model has led to some improvements in accessibility, both in education and in wider society, and has resulted in disabled people being given a language, which enables them to make claims on both institutions and individuals (de Wolfe 2002). There is a lively debate within the chronic illness community (Fox 2018) about whether or to what extent the social model of disability applies to people with chronic

Inclusive Education for Students with Chronic Illness …

137

illnesses. Opinions range from those who fully identify as disabled and feel that it is the society which creates barriers to inclusion, to those who feel that there is no change to society that could take away their pain or in the case of people with severe ME/CFS, enable them to get out of bed. My experiences described above highlight how the severity of illness is closely tied to the potential impact that any material (and technological) change can have on improving inclusion. When mildly affected, I may well have agreed that it was the social environment which was most prominently disadvantaging me. In contrast, the severity of my current illness means that while much more intelligent use of technology could reduce some of the barriers that I face, I also recognise that this alone will not compensate for the effects of the illness, particularly that I am housebound and have severely reduced concentration. This discussion is important for prefacing the rest of this chapter which is predicated on the assumption that technology can play a role in reducing the barriers that people with chronic illness face, while acknowledging that for some people this will not be enough to enable them to return to education.

3 Key Issues, Inclusion and Educational Challenges The CIIP has found that the conditions affecting people who self-identify as having a chronic illness include ME/CFS (myalgic encephalomyelitis/chronic fatigue syndrome), lupus, fibromyalgia, chronic pain and Ehlos Danlos syndrome. Such chronic illnesses tend to be lifelong but not life-threatening and may affect people of all ages including children. While all chronic illnesses have different symptoms, there is also a commonality of experience. According to the CIIP chronic illness, “entails global or systemic impairment, often of a fluctuating nature … [which is] both physical and cognitive, cumulative and, to some extent, interchangeable” (Hale 2018). Chronic illnesses vary in severity, and this will have the most significant impact on how or even whether it is possible to participate in education. For example, someone with mild ME/CFS may be able to attend school full time so long as they take regular rest breaks and have limited activities outside the school. However, at the severe end of the spectrum, someone may be totally bed-bound and completely unable to speak, swallow or tolerate light for 24 h per day. Many people fall between these two and must either reduce or stop their involvement with conventional formal education altogether. For some of these sufferers, online learning or other technological solutions may assist in enabling students to continue their education in some forms (A’Bear 2014; Benigno et al. 2016; Børsting and Culén 2016; Zhu and Van Winkel 2016). In my own experience, online learning and technologies such as video conferencing, speech-to-text, Twitter and collaborative writing software have been invaluable in enabling me both to study and to continue as an academic (Wood 2017). For this reason, there have been calls for digital solutions for the provision of learning for school children with ME/CFS (Sheridan and Cross Party Group-Education of Children with ME 2013).

138

A. Wood

Inclusion in education is a contested term (Liasidou 2012; Robinson 2017), with one common conceptualisation focussed on enabling the participation of disabled students in mainstream educational settings. Here, I argue that inclusion for students with chronic illnesses should not be fixed, but instead something which is more varied and nuanced, taking into account the severity of the illness and the possibilities of technology. For example, a student who is mildly affected and able to attend school on some days but not on others may benefit from the AV1 (“No Isolation” 2018), a robot which is designed to be used in schools in the place of a pupil with a chronic illness so that they can still take part in their lessons. This enables them to continue their education while maintaining social contact with their classmates and their teachers. Here, inclusion is about maintaining previous levels of both social and educational engagement. In contrast, for someone who is completely housebound, the Nisai Virtual Academy (“The Nisai Group” 2018), which offers online education for pupils who are not able to take part in mainstream education may be more appropriate. As with my experience of online learning, this enables students to engage in education when face to face learning is impossible but, critically, in a way that is at an equal level to their fellow students. Inclusion in this instance involves access to participation in education in an environment which is necessarily different to mainstream school experiences, but this difference should not be seen as a deficit. There are a number of key aspects of chronic illness which are likely to have an impact on educational participation. While many of these stem from one of the commonest symptoms of chronic illness: fatigue, (Dailey 2010; Jackson 2013) the descriptions below are the ones that are used informally by the chronic illness community as a way of describing the experience of living and studying with chronic illness. For the purposes of this chapter, these are felt to be more useful than medical terms. • Energy Impairment: Energy impairment is the feeling that energy resources are limited, must be rationed and need to be used wisely and on carefully chosen activities. While everyone’s daily energy ration is different, (and indeed may vary considerably from day to day), this experience unites people with chronic illness. Aids, adaption’s and technology can help to reduce the overall energy expenditure of a given activity, for example, students may use speech-to-text software rather than typing. However, they do not increase the total amount of energy available. • Energy Management: Due to the need to cope with the reduced level of energy available, many people with chronic illness use a technique called pacing, whereby activity is broken into small, manageable tasks and interspersed with rest. How much can be achieved will vary for each individual. If a person with a chronic illness tries to keep going longer than is sustainable, this will inevitably lead to “payback”—the dramatic worsening of symptoms and to a reduction in the energy budget available, which can last for days, weeks or even months and years. The aim of pacing is to avoid payback. Pacing is likely to mean that a student needs regular rest breaks and cannot engage in learning for the same length of time as their peers.

Inclusive Education for Students with Chronic Illness …

139

• Fluctuating Nature: One of the most difficult and frustrating aspects of chronic illness is the fluctuating nature of the symptoms. These fluctuations may mean that what someone can do one day is impossible on another day. Fluctuations can follow any timescale, from hours to months or even years. Fluctuations rarely mean that a person with a chronic illness is feeling completely well, but might result in both a lessening and worsening of their symptoms. The consequence of this for education is that learning patterns may need to be flexible and variable and cannot be predicted in advance. • Cognitive Difficulties: Commonly called “brain fog”, people with chronic illness may have a range of cognitive difficulties including problems with concentration, short-term memory and word finding. They may be unable to tolerate loud noise. Cognitive difficulties will impact all the aspects of learning, including how quickly someone is able to process information and the ability to retain new ideas. Difficulty coping with noise may make group activities particularly problematic. • Pain: Many people with chronic illnesses will suffer from significant pain, which may take many forms, for example, muscle pain, joint pain and neurological pain. This may limit the movements that sufferers can make and the length of time that they can do a particular movement. For example, both typing on a computer and the ability to sit at a desk to use a computer may be limited. Medication for pain is also likely to exacerbate the symptoms described above.

4 AI: Current, Future and Speculative Technologies This section will explore a range of artificial intelligence technologies that could aid the educational inclusion of people with chronic illness. The selection of technologies was developed through both crowd sourcing from friends and colleagues with chronic illnesses and reflecting on my own experiences of living and studying with ME. The boxed quotes illustrate the range of ideas and experiences of the people who contributed to this work. While the list is not exhaustive, similar ideas were expressed by different people, resulting in some consensus for technologies that would be useful for people with chronic illnesses. While talking to different people about this chapter, it became clear that people do not generally distinguish between technologies that depend on AI and those that do not, and in fact many potentially useful non-AI technologies were suggested. Therefore, for the purposes of this chapter, I define AI broadly as “the ability of a digital computer or computer-controlled robot to perform tasks commonly associated with intelligent beings” (Encyclopaedia Britanica 2018). In this context, the focus is on machines which mimic the cognitive functions we associate with humans such as learning and problem solving (Russell and Norvig 2016) as well as speech recognition and production. The technologies in this chapter include those which currently exist in similar forms but which could be relatively easily adapted for the purposes described, those which have not yet been developed but are on the horizon, and those which are

140

A. Wood

more futuristic and speculative, but which may one day become reality. Four categories are considered here: understanding language and speech, producing speech, personalisation of learning and virtual reality.

4.1 Understanding Language and Speech Actually good speech recognition would help. The type with enough AI to understand even if I mumble or have to spell a word or mis-speak (like human listeners error correct). Needs context/knowledge to do that. (Ricky Buchanan, Person with ME)

4.1.1

Interactive Searching for Text Documents

The Problem It is common these days for students to need to navigate a multitude of digital media, from digital textbooks and lecture slides to digital notes and papers. Even finding the right document to begin the search can be a difficult and timeconsuming activity. The advantage of digital media is that it is easy to search if you know what you are looking for, but for people with a chronic illness which can result in memory problems, trying to search for a paragraph on a specialised subject when you can’t remember the technical name for it, is tricky. The Solution While systems like Amazon’s Alexa and Apple’s Siri can already respond to simple requests to turn on the lights or find the nearest Chinese takeaway, there is potential for improvement in this area. There are two challenges here—voice recognition which can cope with different types of voices, and systems with advanced natural language processing which have the ability to more effectively understand meanings in speech, particularly in relation to searching through digital media. A system that can understand different speech types is particularly important for people with chronic illnesses as fatigue can dramatically affect the voice, causing changes from day to day and during the course of the day, and in severe cases a person with ME/CFS may need to communicate through whispering. Most current techniques are targeted at understanding everyone regardless of voice type; however, a new approach under development by Cambridge Consultants (2018), which aims to determine the accent of the speaker and then to use machine learning tailored to that accent, may yield systems which can be used by those with difficulty speaking clearly. Such a system may also give information about the mood of the speaker and could be adapted to spot when a speaker is becoming fatigued and highlight that they need to take a break. Improvements in natural language processing could also lead to more intuitive and helpful ways to search through the textbooks and other digital media provided to

Inclusive Education for Students with Chronic Illness …

141

the students during their course. It could be possible for previous uses to be logged and time-stamped, so if a student wanted to find something that they had looked at recently, they only need to say something like: “show me the bit about Fresnel equations that I read about two weeks ago”. Similarly, it could go beyond picking out individual words to understanding the type of information that the book contains. A student could then ask: “show me all the bits related to a social constructivist approach” or “is there an example of something written in iambic pentameter?” So far, this discussion has applied to text—new generations of AI would need to be capable of interpreting figures, diagrams, graphs and equations through improved image recognition. Being able to search by asking, “show me the diagram of the venous system of frogs” or “find an example of an indefinite integral” (without relying on text labels) would be even more useful.

4.1.2

Creating Non-text Artefacts

The biggest challenge I have found with technology and learning is anything to do with maths/scientific notation. I have found most of the time no matter how bad I feel it is still easier to write out equations than it is to type them out or try and get dragon to do what I want. (Student with a chronic illness)

The Problem AI which can translate speech to text is now fairly well advanced, and more advances can be expected in the future. Many students have access to speechto-text software, either on their mobile devices or through commercial applications, some of which are provided by disability support services. These are invaluable for students with chronic illness, as speaking is often easier than typing, which involves a lot of small motor movements and which may be difficult due to pain and the energy required to move the same muscles over a significant period of time. Typing also involves a substantial degree of concentration, even if the student is proficient at touch-typing. However, not all assignments involve text. In the sciences, students may need to plot graphs, draw chemical structures, create and label diagrams. In architecture, scale drawings are essential. Even in the humanities, which are still dominated by writing essays, there has been a growing critical interest in the potential for multimodal assessment. Here, students may need to produce posters, create videos or record screencasts. However, software for the creation of non-textual artefacts almost always relies on or is easier to do with the use of a keyboard. The Solution Future developments in speech-to-text AI technology could lead to systems which easily allow the creation of equations, images, graphs, diagrams and videos. This will rely on improved natural language processing but will also require carefully designed systems that have a low barrier to use.

142

A. Wood

The difficulties involved in this are evident in the work to create accessible mathematical editors which can create mathematical notation from either speech or from handwriting. While much progress has been made over the last 25 years (Noble et al. 2018), challenges still remain. For example, one of the most recent systems, the Pearson Accessible Equation Editor (AEE 2018) requires the use of the phonetic alphabet (e.g. alpha for “a”, etc.) which makes it cumbersome to use. Other promising systems are g(Math) and its successor EquatIO (2018); however, no data has been collected on the error rates of these systems. The challenges to creating accessible mathematics editors which also apply to creating non-textual artefacts from speech include the large number of characters and notations which must be easy to access and the tendency for spoken language to be ambiguous. Further, as novices and experts tend to use language differently, it may also be necessary to have different systems for each group. Systems also need to be able to cope with subject-specific differences, such as where the mathematics looks the same but has a different function. Other promising approaches to creating images and graphs could involve hand gestures (perhaps using haptics or virtual reality) and 3D scanners, such as the HP Sprout technology. A system could be adaptive over time, learning from the user (as well as other students) which types of phrases are most common and what is required from a given phrase. There will always be a certain level of learning necessary for the user here, but if AI systems can be developed which can interpret the commands correctly without this becoming too arduous, there would be huge benefits to people with chronic illnesses who wish to create non-textual artefacts without the use of a keyboard.

4.2 Producing Speech: Text to Speech I’d like something to read out textbooks in a listenable voice (artificial speech curdles my brain. Too monotonous to concentrate). (Ricky Buchanan, Person with ME)

The Problem Reading text can be very demanding, particularly if cognitive energy is limited. This is often more difficult when reading on a computer or device screen. Difficulties reading on a screen can be due to a number of factors: the brightness of the screen, the flicker rate of the screen, the colour (screens tend to emit blue light, whereas paper reflects diffuse light and is redder) and the size of the screen if using a mobile phone. Other factors which may be particularly difficult for people with chronic illness include having to sit upright when using a computer or having to support the weight of the computer when lying in bed.

Inclusive Education for Students with Chronic Illness …

143

The Solution Intense research in the area of text to speech (TTS) has resulted in the creation of apps which are readily available on phones and tablets and are already used widely by people with chronic illnesses and disabilities. But despite the advances, there is the potential for improvement. One area which would help students with chronic illness is to generate more natural sounding speech as this would reduce the concentration required to listen to it. Some progress has been made towards this by Google’s DeepMind project which has produced WaveNet (WaveNet 2018)—a system using deep neural networks to create waveforms, which produce more human-like speech from text than previous technologies (van den Oord et al. 2016). Further improvements require systems to take care of prosody—the attributes of speech which give it more natural emphasis such as speaking rate, timing, pausing, articulatory quality, voice quality and pitch. This would also include the way in which TTS systems handle punctuations (such as brackets, hyphens and quotations marks), which are vital for correct interpretation of the meaning that the writer is trying to convey. However, this needs to be done in such a way as to convey the punctuation without taking away from the flow of the prose. A system developed by the Google AI team, called Tacotron 2 (2018) aims to do just this. Their approach does not use complex linguistic and acoustic features as input. Instead, it generates human-like speech from text using neural networks trained using only speech examples and corresponding text transcripts. The system however has problems pronouncing complex words and cannot yet generate audio in real time. The difficulty that TTS systems have with unusual words is particularly problematic for students as academic texts often use specialised or technical language. Hearing a badly pronounced word can result in the user needing to spend valuable cognitive energy on translating that word for themselves, possibly meaning that they lose the thread of the sentence or may mean they need to stop the speech and resort to reading the text. One possible approach would be to use crowd sourcing, perhaps using CAPTCHAs (checks that websites use to prove that the user is a human) to get people to speak words commonly used in their own discipline together with machine learning to work out which are the key parts of the sound pattern which make that particular word. The result would be an expanded dictionary which could be accessed by text-to-speech systems. Related to this are the challenges of converting mathematical notation to speech. As discussed above, progress has been made in systems which make mathematics more accessible (Noble et al. 2018) including improved systems for speaking mathematical notation. One challenge here is that the common forms of human speech tend to be ambiguous. Approaches to resolving this include changes to speed, pitch, prosody or adding in brackets.

144

A. Wood

4.3 Personalising Learning There are a lot of times where I do not feel like I can ask a question about a subject because it is something that should have been reinforced many times throughout high school and when I have asked I end up feeling stupid or like I have to explain why I don’t know that particular thing. (A student who missed a lot of school due to chronic illness)

4.3.1

Intelligent Tutoring Systems—Tailored Learning

The Problem In general, mainstream classes are designed to take students through the material at a set speed and in a set order, predetermined by the teacher. However, students with a chronic illness may not be able to keep up with this schedule—they may have to miss a lot of classes if they are too ill to attend in person, they may not have the cognitive energy to be able to concentrate for long and so be limited in how much learning they can do in a particular day. They may have memory problems and need some topics to be repeated in order to achieve deep learning. The Solution Intelligent tutor systems (ITS) use AI techniques to simulate oneto-one human tutoring, delivering learning activities best matched to a learner’s cognitive needs (Luckin et al. 2016). Their most important characteristic is that they are adaptive—they can change the learning and the teaching process in response to anything from students’ knowledge to behaviour, interests, learning style, cognitive style or goals (Grubiši´c et al. 2015). This makes ITS ideal for people with chronic illness whose needs may change even from minute to minute as their condition fluctuates. This could be achieved through the development of systems which are able to detect and respond to the students’ energy levels. The field of “affective computing”, where intelligent agents aim to detect human emotions through speech and video input (Poria et al. 2017), may provide a useful starting point for developing a system which could recognise fatigue. Similarly, a system could be developed which includes the integration of wearable technologies such as eye tracking or heart rate monitors. These biological data could then be correlated with how a student is feeling and how well they are doing on learning tasks, enabling the system to detect when a student is becoming exhausted. It could then either adjust the content and type of activities or recommend that they take a break. It could even potentially pick up signs that a student needs to rest before they are aware of it themselves. Such a system would enable students with chronic health conditions to efficiently manage both their learning and their available energy, though there would need to be robust systems for dealing with privacy and security issues related to such personal data.

Inclusive Education for Students with Chronic Illness …

4.3.2

145

Intelligent Tutoring Systems—Social Interactions

Some sort of interactive education that could be done when someone is feeling up to it rather than when the lesson is arranged with the tutor would be good. (Father of a child with a chronic health condition)

The Problem Students with chronic illness who suffer from cognitive problems such as brain fog are likely to find loud environments challenging. They may also struggle to process information and need longer to understand what is being said or to form a response to a question. This means that students with chronic illness are likely to struggle with learning which involves social interactions, such as group work, discussions with peers and with teachers. This is significant because from a sociocultural perspective, social interactions with others are critical for learning (Vygotsky 1978). The Solution Intelligent tutor systems (ITS) could interact with students to simulate interactive learning. This could be an ITS which is taking the role of a teacher; however, peer learning is also known to have important benefits (Boud 2001) as peers use the same type of language and express themselves in the same way as other students of a similar level, age and background. Intelligent tutors could therefore be developed which mimic another student, thus creating a peer-learning environment. The advantages of such a system would be that students would not feel the pressure to respond quickly, and they would easily be able to take a break and return to the conversation when they felt able. Such a tutor would be programmed to be more proficient than the student in some areas, while having gaps in its knowledge in other areas, so that the student has an opportunity to explain the concepts and to “teach” the system, as they would do in a real-life peer discussion situation. A system like this could be based on the “dialogue games” developed by Ravenscroft and his team. The first of these was an intelligent computer-based argumentation system called CoLLeGE (Computer-based Laboratory for Language Games in Education) (Ravenscroft and Pilkington 2000). Similar games developed by the group aim to support students’ development of explanatory models of science concepts through collaborative argumentation, in which the tutor system encourages students to reason about their explanations rather than simply giving them the correct answer. This has been shown to be important for science learning (Twigger et al. 1991) but could be equally beneficial in other subject areas.

4.3.3

Virtual Reality

The Problem Students with a chronic illness are often unable to physically attend classes. In many cases, online learning or recorded lectures can fill this gap, enabling

146

A. Wood

students to catch up with the missed lectures or classes. But for many subjects, there are specific practical skills or experiences which are not easily replicated online. These include laboratory sessions in biology, physics and chemistry, as well as field trips which may form part of a wide range of courses from geography and geology to archaeology and architecture. The Solution Virtual reality technologies such as Oculus Rift headset and Google Glass, which enable the user to experience a three-dimensional world through sound and vision, are now well established. More recently, systems which achieve immersion through a headset and mobile phone have been developed. While these systems are not yet regularly used in education, partly due to their cost, they can have a valuable role in learning (Psotka 1995), providing immersive experiences which would be otherwise impossible, such as enabling students to explore the surface of Mars, coral reefs under the sea or understanding the three-dimensional nature of chemical structures. Virtual reality systems do not necessarily involve AI, but by doing so they could be used to create immersive, adaptive and personalised learning environments. For example, by incorporating an intelligent tutor system, students could experience increasingly complex and tailored learning challenges. Such systems could be envisioned which mimic laboratory sessions and support the development of the practical skills required in the sciences, or which recreate field trips, or language learning from exchange programmes abroad. This would particularly benefit students with chronic illnesses who are unable to attend these types of experiences in person.

5 Conclusions Chronic illness can have a substantial impact on the lives of young people and adults and as a consequence is likely to severely impact their ability to engage in both compulsory and further education. Key difficulties faced by people with chronic illness include energy impairment (and consequently the need to manage available energy levels effectively) and cognitive difficulties, such as “brain fog”, concentration and memory problems. While it is important to note that many of the issues associated with chronic illness cannot be alleviated by technology, the examples discussed in this chapter show that there is also a huge potential for AI to facilitate the educational inclusion of people with chronic illness by providing support in a variety of areas. These include: systems with more advanced natural language processing which could make it easier to search for text, equations and diagrams in digital documents; voice-controlled applications which can be used to create non-textual artefacts such as diagrams and graphs; improvements to the production of spoken language from textual documents to create more natural speech and to more accurately render technical language; intelligent tutor systems which are able to produce adaptive, tailored and interactive teaching, enabling students with chronic illness to gain the best pos-

Inclusive Education for Students with Chronic Illness …

147

sible learning experiences in a way which works for them; and virtual reality systems to give students experiences that they might be otherwise too ill to access.

References A’Bear, D. (2014). Supporting the learning of children with chronic illness. The Canadian Journal of Action Research, 15(1), 22–39. AEE. (2018). Pearson accessible equation editor. Retrieved September 11, 2018, from http:// accessibility.pearson.com/aee/. Benigno, V., Epifania, O. M., Fante, C., Caruso, G., & Ravicchio, F. (2016). Which technological skills and teaching strategies for inclusive education: Syngergies and discordances. In Proceedings of the 9th International Conference of Education, Research and Innovation (Spain), Sevilla (pp. 987–996). Børsting, J., & Culén, A. L. (2016). A robot avatar: Easier access to education and reduction in isolation? Boud, D. (2001). Making the move to peer learning. In D. Boud, R. Cohen & J. Sampson (Eds.), Peer learning in higher education: Learning from and with each other (pp. 1–21). London. Cambridge Consultants. (2018). CES 2018—From voice recognition to voice understanding. Retrieved September 3, 2018, from www.insights/.com.ces-2018-voice-recognition-voiceunderstanding. Dailey, M. A. (2010). Needing to be normal: The lived experience of chronically ill nursing students. International Journal of Nursing Education Scholarship, 7(1). Department of Health. (2012). Long-term conditions compendium of Information: 3rd edition. Retrieved from https://www.gov.uk/government/publications/long-term-conditionscompendium-of-information-third-edition. de Wolfe, P. (2002). Private Tragedy in social context? Reflections on disability, illness and suffering. Disability and Society, 17(3), 255–267. https://doi.org/10.1080/09687590220139847. Dowsett, E. G., & Colby, J. (1997). Long-term sickness absence due to ME/CFS in UK schools: An epidemiological study with medical and educational implications. Journal of Chronic Fatigue Syndrome, 3(2), 29–42. Encyclopaedia Britanica. (2018). Artificial intelligence | Definition, examples, and applications. Retrieved September 7, 2018, from https://www.britannica.com/technology/artificialintelligence. EquatIO. (2018). EquatIO Math writing software. A digital math tool for teachers & students of all abilities | Texthelp. Retrieved September 11, 2018, from https://www.texthelp.com/en-gb/ products/equatio/. Fox, J. (2018). Ill versus disabled—Is there a distinction between the two? Retrieved July 13, 2018, from http://inclusionproject.org.uk/social-model/ill-versus-disabled-is-there-a-distinctionbetween-the-two/. Grubiši´c, A., Stankov, S., & Žitko, B. (2015). Adaptive courseware: A literature review. Journal of Universal Computer Science, 21(9), 1168–1209. Hale, C. (2018). Reclaiming “Chronic illness”—An introduction to the chronic illness inclusion project (p. 38). Centre for Welfare Reform. Retrieved from https://www.centreforwelfarereform. org/uploads/…/617/reclaiming-chronic-illness.pdf. Jackson, M. (2013). The special educational needs of adolescents living with chronic illness: A literature review. International Journal of Inclusive Education, 17(6), 543–554. Liasidou, A. (2012). Inclusive education, politics and policymaking. London: Continuum International Publishing Group. Luckin, R., Holmes, W., Griffiths, M., & Forcier, L. B. (2016). Intelligence unleashed: An argument for AI in education. London: Pearson.

148

A. Wood

No Isolation. (2018). Retrieved July 12, 2018, from https://www.noisolation.com/uk/. Noble, S., Soiffer, N., Dooley, S., Lozano, E., & Brown, D. (2018). Accessible Math: Best practices After 25 years of research and development. Journal on Technology & Persons with Disabilities, 30. Poria, S., Cambria, E., Bajpai, R., & Hussain, A. (2017). A review of affective computing: From unimodal analysis to multimodal fusion. Information Fusion, 37, 98–125. Psotka, J. (1995). Immersive training systems: Virtual reality and education and training. Instructional Science, 23(5–6), 405–431. Ravenscroft, A., & Pilkington, R. M. (2000). Investigation by design: Developing dialogue models to support reasoning and conceptual change. International Journal of Artificial Intelligence in Education, 11(1), 273–298. Robinson, D. (2017). Effective inclusive teacher education for special educational needs and disabilities: Some more thoughts on the way forward. Teaching and Teacher Education, 61, 164–178. Russell, S. J., & Norvig, P. (2016). Artificial intelligence: A modern approach. Pearson Education Limited. Sheridan, A. K., & Cross Party Group-Education of Children with ME. (2013). Exploring E-learning provision for Children with ME in Scotland. Other Education, 2(1), 78–80. Stella Young. (2014, March). Practicing pride in the face of exclusion [item]. Retrieved September 13, 2018, from http://www.abc.net.au/rampup/articles/2014/03/24/3968171.html. Tacotron. (2018). Tacotron 2: Generating human-like speech from text. Retrieved September 11, 2018, from http://ai.googleblog.com/2017/12/tacotron-2-generating-human-like-speech.html. The Nisai Group. (2018). Retrieved July 12, 2018, from http://www.nisai.com/. Twigger, D., Byard, M., Draper, S., Driver, R., Hartley, R., Hennessy, S., et al. (1991). The ‘conceptual change in science’ project. Journal of Computer Assisted Learning, 7(2), 144–155. van den Oord, A., Dieleman, S., Zen, H., Simonyan, K., Vinyals, O., Graves, A., et al. (2016). Wavenet: A generative model for raw audio. ArXiv Preprint arXiv:1609.0349. Vygotsky, L. (1978). Interaction between learning and development. Readings on the Development of Children, 34–41. WaveNet. (2018). WaveNet: A generative model for raw audio. Retrieved September 3, 2018, from https://deepmind.com/blog/wavenet-generative-model-raw-audio/. Wood, A. (2017, November). Being a housebound digital academic. Retrieved January 9, 2019, from https://www.thesociologicalreview.com/blog/being-a-housebound-digital-academic.html. Zhu, C., & Van Winkel, L. (2016). A virtual learning environment for the continuation of education and its relationship with the mental well-being of chronically ill adolescents. Educational Psychology, 36(8), 1429–1442.

Anna Wood has lived, worked and studied with a chronic health condition (ME/CFS) since 1999. With a background in Physics, she obtained her Ph.D. in 2000 and subsequently worked as a research fellow in institutions around the UK. A deterioration of her health in 2008, that has left her housebound, forced her to leave an RCUK fellowship in bio-nano-technology at the University of Strathclyde. Although she remains largely housebound, she has since gained an M.Sc. in Elearning (University of Edinburgh, 2013) and now works 5 h per week as an education researcher with a particular interest in the use of technology in large lectures, and how this affects student participation, learning and classroom dialogue. She regularly writes about her health condition and its impact on learning and working in higher education.

Part III

Critical Perspectives and Speculative Futures

Shaping Our Algorithms Before They Shape Us Michael Rowe

Abstract A common refrain among teachers is that they cannot be replaced by intelligent machines because of the essential human element that lies at the centre of teaching and learning. While it is true that there are some aspects of the teacher–student relationship that may ultimately present insurmountable obstacles to the complete automation of teaching, there are important gaps in practice where artificial intelligence (AI) will inevitably find room to move. Machine learning is the branch of AI research that uses algorithms to find statistical correlations between variables that may or may not be known to the researchers. The implications of this are profound and are leading to significant progress beign made in natural language processing, computer vision, navigation and planning. But machine learning is not all-powerful, and there are important technical limitations that will constrain the extent of its use and promotion in education, provided that teachers are aware of these limitations and are included in the process of shepherding the technology into practice. This has always been important but when a technology has the potential of AI we would do well to ensure that teachers are intentionally included in the design, development, implementation and evaluation of AI-based systems in education. Keywords Artificial intelligence · Machine learning · Educational technology

1 Introduction It has become commonplace to argue that the practice of teaching with care, ethics and justice cannot be replicated by machines and that the essence of a learner- or relationship-centred pedagogy relies on human values and the nature of the interactions between people. As is often the case, we are happy to acknowledge that technological disruption is inevitable elsewhere but are usually able to find good explanations for why our own professions are safe (Susskind and Susskind 2015). M. Rowe (B) University of the Western Cape, Cape Town, South Africa e-mail: [email protected] © Springer Nature Singapore Pte Ltd. 2019 J. Knox et al. (eds.), Artificial Intelligence and Inclusive Education, Perspectives on Rethinking and Reforming Education, https://doi.org/10.1007/978-981-13-8161-4_9

151

152

M. Rowe

The claim that smart algorithms and humanoid robots are not capable of the emotional connection that drives meaningful, socially constructed learning may well be true but will also not matter. Even though individual human teachers are unlikely to be replaced, teaching as a profession remains vulnerable to automation in the face of increasingly capable machines. In the context of teaching and learning, there are three assumptions that constrain our thinking and cloud our judgement when considering the impact of artificial intelligence (AI) on teaching and learning. The first assumption is that all teachers are able to work in a system where care, justice and human connection are present and prioritised. The second is that “teaching” is a single, monolithic practice instead of a collection of isolated and routine tasks. The third assumption is that these tasks—whether routine or not—nonetheless require a level of intelligence that is beyond the capability of computer algorithms. By interrogating these assumptions, we can disrupt the common sense understanding of teaching and learning and expose the gaps in practice into which AI-based systems will inevitably move. The first assumption is that only human teachers possess the emotional and personal connections necessary for the meaningful interactions that are central to socially constructed learning. And while this may be true (for now) the reality is that much of the education system remains untouched by concerns of care and justice. The infrastructure of education ensures that control and authority are vested in the teacher who is positioned, both physically and epistemologically, as the only legitimate source of knowledge in the classroom. Students are reminded that their words and personal experiences have no value in their own learning (Freire 2005), and this lack of power dulls their enthusiasm and cultivates an obedience to a system that generates attitudes of conformity (hooks 1994). In this paradigm, learning may be seen as a process of moving information from the notes of the teacher to the notes of the student via “communiques” that students receive, memorise and repeat (Freire 2005). This banking model of education turns the student into a container to be filled and the more passively they allow themselves to be filled, the “better” they are. Students and teachers are thus both reduced to “cheerful robots” through an instrumental rationality in which matters of justice, values, ethics and power are erased (Giroux 2011). In addition, assessment emphasises the failings of students in a process that seems preoccupied with ranking them rather than creating formative contexts in which learning is prioritised (Morris and Stommel 2017). The argument against the introduction of AI into educational contexts cannot be that algorithms are unable to offer an ethical and careful pedagogy in the context of human relationships because—for reasons beyond their control—many teachers may find themselves in the same position. AIbased systems in education will not need to replicate the best possible version of a human teacher. For economic reasons, they will only need to be a little bit cheaper than the average teacher (Brynjolfsson and McAfee 2014). The perception of costeffectiveness alone will be enough to drive the implementation of AI in education. In addition, there is evidence that the mere appearance of intelligence is enough to mediate our cognitive and emotional response to machines (Susskind and Susskind 2015). In other words, we may not need our teaching machines to care; it may be

Shaping Our Algorithms Before They Shape Us

153

that the appearance of caring is enough for students to form learning relationships with them, further denting the argument that emotional connection is a requirement for learning. The second assumption is that teaching as a profession is a single, discrete occupation rather than a collection of often unrelated tasks, many of which are physically or cognitively routine and thus vulnerable to algorithmic manipulation (Susskind and Susskind 2015). The non-routine cognitive tasks are those that are necessary for the development of empathy and human connection and are implicated in meaningful learning. A teacher in the twenty-first century cannot simply provide students with facts about the world and then rank them according to their test scores because these are the tasks that intelligent agents will soon perform at negligible cost. The ability of an algorithm to scale from 100 users to 100, 000 is an aspect of education that anyone in control of a budget will find appealing. If teaching is a series of tasks, some of which are routine and therefore vulnerable to automation, then we should ask which are the ones that require insight, care, creativity and connection, because these are less vulnerable to algorithmic control. Teachers who work collaboratively with students to provide them with the tools to engage critically and creatively with the world are more than conduits for specialised knowledge. This relationship-centred process is not amenable to computational solutions and would, in some sense, protect these tasks from the risk presented by AI-based systems. Teachers who, through systemic constraints and overly rigid regulatory requirements, are subject to what Freire called the banking model of education may be rendered obsolete as emerging technologies outperform them in the narrow tasks of information transfer and student ranking. The final assumption is that AI will simply not be able to take over even the most basic tasks that teachers must perform because their current performance is so severely constrained by high costs and narrowly defined working parameters. However, human beings have a tendency to focus our attention on very short time horizons at the expense of seeing what is possible over longer periods. While it is true that the performance of machine learning algorithms currently leaves a lot to be desired, it would be a mistake to discount both the current performance and improvement in AI research over the next decade. For example, image classification is both mundane and incredible. Mundane because computer vision has only now reached a point where it is at parity with human performance in a small subset of categories. But incredible because five years ago even this parity was impossible (Frankish and Ramsay 2017). This is a testament to advances in the data available for algorithm training, more powerful computation at much lower cost, and improved algorithm design (Brynjolfsson and McAfee 2014). Even if all technical progress on machine learning were to stop tomorrow (which seems unlikely given the amount of interest and funding available), the steady increase in computational power and access to data means that the functional intelligence of AI systems will keep getting better. At some point, this will bottom out, and in fact, there are already signs that the results produced from the current generation of machine learning systems will soon plateau (Jordan 2018). But even when that happens we will still see gains in performance speed and associated decreases in cost that will continue driving the introduction of AI-based systems into education.

154

M. Rowe

I believe that these three assumptions underpin many of the arguments against the implementation of AI in education and that when challenged, they open up the possibility that the introduction of AI into the educational context is not only possible but inevitable. If the primary role of a teacher is to provide access to specialised knowledge and rank student performance according to a set of standardised competencies, we may soon see the end of the profession. However, if the role of a teacher is to become with the student in a pedagogy of liberation in which both teachers and students engage in acts of cognition with the intent of transforming the world (Freire 2005), then we may see AI used to augment a creative, ethical, careful and socially just pedagogy. In this latter case, it seems unlikely that AI will replace teachers but rather that teachers who use AI appropriately may replace those who do not. The purpose of this chapter is to explore the implications of AI in education and to make suggestions for how teachers can be included in the development of AI-based systems in order to ensure that the human values of care and justice are embedded in intelligent teaching machines.

2 AI in the Context of Education The field of artificial intelligence research began in the 1950s with the design of algorithms that could solve well-defined problems in structured, clearly described environments. This meant manually coding all possible routes through a solution space which, even in tightly controlled artificial environments, would cause the algorithms to break down (Frankish and Ramsay 2017). These early AI systems were not resilient to small changes in even very simple scenarios and could not adapt to the much more complex interactions found in the real world. Because algorithm designers could not predict, and therefore solve, all of the problems arising in uncertain environments, it was soon understood that these brittle systems could not produce results that had much practical or commercial value (ibid.). Developers realised that they needed computational models that could more closely approximate reality and would need to “learn” and adapt in real time by using feedback from the environment. Machine learning (ML) is the branch of AI research that deals with this challenge and is responsible for much of the recent progress being made in AI. Machine learning works by using statistical techniques to identify relationships between variables in very large data sets with varying levels of confidence (Pearl and Mckenzie 2018). Basically, machine learning uses statistics to solve classification problems and predict missing information (Agrawal et al. 2018). In this context, the word “learn” is problematic because we may associate it with other aspects of human experience, including emotions and consciousness. This is confusing because of our tendency to anthropomorphise non-human objects, leading to an assumption that “learning” algorithms are conscious, moral and emotional and that they should also understand the outputs of their computation (Frankish and Ramsay 2017). When algorithms fail to display these additional characteristics, we may see this as an argument for why they cannot replace teachers. But no serious

Shaping Our Algorithms Before They Shape Us

155

AI researcher claims that a machine translation algorithm understands language or that a game-playing algorithm understands that it is playing a game (Searle 2011). An algorithm does not know what a game is. Our ability to understand the world and interact with others is a non-declarative type of knowledge that cannot be captured by any rule-based system (Frankish and Ramsay 2017). The statistical computation in machine learning is fundamentally different to the phenomenological process of human learning and the conflation of the two leads to misunderstanding and unrealistic expectations of ML algorithms. Indeed, ML systems need not have an understanding of the world, nor of emotion, consciousness or human relationships in order to outperform humans in a wide variety of the tasks that make up the job description of a teacher. For example, timetable and curriculum planning, lecture transcription, and content preparation and review are all tasks that take up significant portions of teachers’ time, are all vulnerable to automation, and do not require an emotional connection to anyone. Algorithms are not conscious, and they do not care about us but this says nothing about their ability to outperform us across an increasing variety of tasks. While high intelligence scores are correlated with an increased probability of success across many domains (Ritchie 2016), successful human interaction is often dependent on empathy, imagination, tolerance of ambiguity, and the use of metaphor, all of which are resistant to computational solutions (Frankish and Ramsay 2017). In addition, relevance, or the ability to distinguish the essential from the inessential, and to effortlessly draw on our experience and knowledge in accordance with the demands of the task, are also regarded as a major stumbling blocks for AI (ibid.). Another important aspect of human interaction is an innate understanding of causality; we can predict what is likely to happen next based on our understanding of what has happened in the past. Indeed, prediction is the aspect of AI that will become increasingly commoditised (i.e. it will soon be very cheap and ubiquitous), driving its integration across and within many sectors of society (Agrawal et al. 2018). However, while much has been made of the predictive utility of ML it cannot say anything about the direction of causality (Pearl and Mckenzie 2018). An algorithm is able to identify patterns between variables and identify when they are correlated but it cannot say that one variable was a necessary precursor to another. For example, an algorithm may determine that low grades and parental income are positively correlated, i.e. as parental income decreases there is a better than random chance that grades will also decrease. However, it is impossible for current AI systems to conclude that the cause of the low grades may be the low income. There are therefore a range of human capabilities that are necessary for the kinds of meaningful interactions that are central to learning and teaching, and which present significant technical challenges that make them resistant to computational solutions. Simply increasing the functional intelligence of an AI-based system is not enough for them to work around issues of human values and care in the context of learning and teaching. However, we should also be clear that practical AI research is not about trying to accurately model all the components of human cognition that are implicated in effective communication. We can already generate useful inferences and accurate predictions about the world using nothing more than brute force computation and,

156

M. Rowe

relative to the human brain, unsophisticated algorithms. The successful implementation of AI-based systems in education will not require consciousness, morality or an ability to generalise reasoning across contexts. Intelligent reasoning about interactions in the world does not require a complete understanding of it, or even an accurate representational model of human cognition (Frankish and Ramsey 2017). They will only need to complete very narrow, routine tasks that can be well-defined, at a lower cost than human beings. And they can already do this very well. Teachers need not worry about being replaced by humanoid robots but rather that software upgrades to existing systems will gradually take over tasks that were previously theirs to perform. As the success of real-world organisations and industries is increasingly premised on their ability to abstract workflows and production into software, we should begin planning for a similar disruption of education (Andreesson 2011). This disruption will happen when algorithms become the dominant decisionmakers in the system, first at the level of isolated apps (e.g. automated essay graders), then integrated across platforms (e.g. student risk assessment via learning management systems), institutions (e.g. admissions and assessment) and finally, at the level of the industry as a whole (e.g. managing students from entry into the system all the way through to employment). When AI is fully integrated into the education system, then every organisation in that system will have to become a software organisation. This is what the disruption of education will look like; less a robot army and more a gradual loss of autonomy in decision-making.

3 Shaping Our Algorithms We are fond of saying that technology must follow pedagogy and in the case of AI in education it is clear that this is precisely what has happened. The technology has followed the pedagogy. Education has misrepresented itself as objective, quantifiable and apolitical (Morris and Stommel 2017) and as a result, educational technology companies have positioned AI in education as objective, quantifiable and a political. They assert that technology is neutral and that they simply need to get the right data and analysis in order to find the ground truth that will allow us to “fix education” (Hendrick 2018). But every feature of a technology is the result of a series of human decisions that optimise it towards a fitness function and as long as people are involved the fitness function will not be objective or neutral. Before we can decide if a technology is fit for purpose we first have to know the purpose of the technology, and with that, the values that are encoded into it. Likewise, algorithms do not have purposes that are predicted by inviolable laws of nature. They are optimised towards a fitness function that is the result of human choices (Frankish and Ramsay 2017). The dominant discourse around AI in education is about saving time, reducing costs and increasing efficiency, reflecting a continuation of the neoliberal policies driving austerity and cuts to services that have emerged over the past two decades. But there is nothing inevitable about these values and it is reasonable to consider a different set of values against which the fitness function of algorithms could be optimised.

Shaping Our Algorithms Before They Shape Us

157

For example, instead of developing an algorithm that maximises cost-effectiveness, profit or attention, there is nothing preventing us from choosing to maximise human well-being instead. The sense of inevitability associated with technological progress is disempowering because it can make us believe that we cannot change its direction or destination. But the reality is that human decisions informed by human values are what drives technological progress and the same must be true of the decisions that inform the implementation of AI in education. As the potential of AI to affect society grows, the role of teachers needs to change in order to help students prepare for a critical engagement within that society. Any discussion around AI in education should therefore emphasise the need to understand and shape this increasingly ubiquitous technology, foregrounding the necessity of input from all stakeholders rather than only from those who are technologically literate (Jordan 2018). If the narrative around AI in education is being driven by venture capital firms and wealthy entrepreneurs (Williamson 2018), it may be because teachers have been distanced from decision-making and increasingly managed by a regime of performance targets that incites them to perform in narrowly measurable ways. Much like medical doctors are, in some sense, the final arbiters of what technology is allowed to operate in the clinical context, so should teachers, informed by values of care and justice, make the final decision about what technology should be allowed in schools. If education technology was shaped by regulation and policies developed by students and teachers, then edtech start-ups would not be able to “move fast and break things”. There is a narrative among these start-ups that education is broken, which gives them leave to bring their significant resources to bear on the “problem” (Hendrick 2018). It may therefore be up to teachers to challenge the assumptions of edtech companies by guiding the design, implementation and evaluation of AI-based systems within a contextual framework that includes them. When teachers are absent from the conversation around the use of technology in education, techno-evangelists will position the technology as a form of emancipation, freeing teachers from an outdated model that is not fit for purpose (Hendrick 2018) In order to shape the space in which AI operates, teachers must ensure that the values of a socially just pedagogy are integrated into the development of ML algorithms. Bostrom and Yudkowsky (2014) ask what human values must be integrated with the computational intelligence of smart machines and suggest that they might include responsibility, transparency, auditability, incorruptibility and predictability. They suggest that since these are some of the criteria that we apply to humans performing social functions we should therefore look for the same criteria in any algorithm intended to replace human judgement in those functions. There is no reason that the fitness function of ML algorithms could not be optimised towards developing careful and socially just pedagogies that privilege student learning and well-being. If we simply accept that AI-based systems incorporate our values, we may become passive or unable to respond when we eventually find that they do not. But if we begin by asking whose values are represented in code and what those values are, we may find that we disagree with them. Once we understand that there is nothing inevitable about the path that a technology takes to mature, nor what the final product should be, it becomes easier to see that we are not powerless to influence the design

158

M. Rowe

of these systems. The infrastructure and communication channels that are necessary for democratic participation in the design and implementation of AI in education are currently missing, making it difficult for teachers to be included. However, unless teachers are intentionally involved in establishing the guiding values of AI in education, we run the risk that our professional decision-making will not be informed by machine intelligence but rather that we are subject to it.

3.1 Data Collection Successful AI relies on finding patterns in large data sets. However, there are many reasons for why training algorithms with educational data may lead to inconsistent, inaccurate, unreliable or invalid conclusions. Education data is often poorly structured, inauthentic, lacking demonstrable validity and reliability and consists almost entirely of grades (Lynch 2018). These proxies for learning are used to train the algorithms embedded in AI systems, not because they are pedagogically meaningful but because they are easy to collect (ibid.). There are other reasons for why the outputs of algorithmic decision-making in education may be wrong. The knowledge base might be biased; the inferences drawn may be incorrect because of errors in the algorithm; the algorithm’s reasoning might not be able to adapt to unexpected contingencies; and the decision criteria and outputs may not be universally acceptable (Mittelstadt et al. 2016). In addition to the social biases encoded in training data, it should not be controversial to say that human decision-making is also influenced by subconscious biases and that these biases are so deep that we are blind to them (Kahneman 2011). Cleaning and transforming the data for algorithm training adds further uncertainty to the process as there are many subjective decisions that will need to be made, each of which creates further opportunities for introducing errors that will have an impact on the algorithm outputs. If these sources of bias remain unchecked algorithms may consolidate and deepen the already systemic inequalities in education and society all while making them harder to notice and challenge (Hart 2017). It is therefore incumbent on teachers to ensure that ML training data is diverse both in terms of the student voices present in the data, but also diverse in the range of proxies for learning that are gathered. Diversity in student populations means that we can be more confident that AI-based predictions are generalisable across different populations and contexts, regardless of what data it was trained on. Having diverse teams, including teachers, students and education researchers, will increase the likelihood that our biases are recognised and addressed, rather than becoming encoded within AI-based systems. Of course, the practical challenges of making these changes from within a system that has already disempowered teachers and students are significant. Teachers not only lack the support for gathering diverse examples of student learning, they lack the time to even think about it. However, without a broader understanding of the data we use to make judgements with respect to student learning, we risk constructing machine intelligence that reflects a narrow and relatively poverty-stricken vision of education.

Shaping Our Algorithms Before They Shape Us

159

3.2 Teaching Practice The computational intelligence of technology is not a substitute for relevant background knowledge in the practice of teaching, and the appropriate use of AI in the classroom will require that computer scientists, teachers and students work closely together in order to ensure that these systems are fit for purpose. The important question is, whose purpose? In order to be active and informed citizens, students will need a sound understanding of AI as well as a critical approach to assessing the implications of data collection at very large scales. Thus, teachers will need to use and evaluate AI systems in the classroom so that they are able to contribute to the conversation and play a role in setting the agenda for AI in education. In this way, teachers can shape the discourse around AI in education so that it is framed within an approach that prioritises care and human relationships in learning. To this end, teachers will need to engage with AI-based systems in similar ways to how they work with colleagues. They will need to use critical judgement to make decisions about the context in which algorithms produce outputs. Rather than seeing algorithmic decisions as fundamentally “right” or “wrong”, teachers will need to understand that algorithms provide probabilistic outputs based on imperfect information and are therefore inherently prone to making mistakes. Just as we will need to decide when those outputs can be trusted, we will need to make choices about when they should be ignored. Unfortunately, there is evidence that we struggle to objectively judge the decisions made by algorithms and that we will often simply follow the instructions we receive (Lyell and Coiera 2017). For example, teachers may champion the use of recommendation engines to identify personalised content for students, thinking that a more focused collection of information is helpful for learning. But these systems make inferences that result in increasingly deterministic recommendations, which tend to reinforce existing beliefs and practices (Polanski 2016). If we want students to be exposed to different ideas as part of their learning then recommendation systems that narrow the focus of information may effectively close down students’ options for diverse perspectives. Teachers who unquestioningly assume the correctness of the algorithmic output may therefore inadvertently reinforce stereotypes and systemic biases. If AI-based systems are left to operate solely in the rational domain of cognition and teachers ignore the emotion-laden interactions that drive meaningful learning they may, with the best intentions, lock students into a category of demographically classified content from where it is difficult to see anything else. The use of AI in education has implicit ethical, social, political and pedagogical choices, and it is essential that both students and teachers are included in order to develop guidelines and theoretical frameworks that can help minimise the risk of unintended consequences.

160

M. Rowe

3.3 Research Audrey Watters asked, “what will happen when robots grade students’ essays? What happens when testing is standardized, automated? What sorts of signals will we be sending students?” (Watters 2015). These are empirical questions that can be tested but when we decide the lines of inquiry we want to explore we should ask questions without having already decided the outcome. For example, we might also reasonably ask, What will happen if algorithmic assessment turns out to be more accurate, more consistent and more equitable than human assessment? What if students preferred it? What if algorithmic feedback and instruction improve students’ intrinsic motivation? Do students prefer a simple algorithm that is available 24/7 or a friendly teacher who is available on Tuesdays between 14:00 and 16:00? Rather than be guided by an emotional response to the thought of automated grading systems, teachers should recognise that they have an opportunity to be involved in the research that might help us better understand optimal strategies for enhancing student learning, which should remain a central priority. Right now, it seems as if the questions are being asked by edtech companies who frame educational problems as problems of efficiency rather than problems of care, relationship and power (Hendrick 2018). The technology underpinning much of the AI-based progress in education is far from perfect and perhaps more importantly, we do not yet have an agreed upon philosophical foundation upon which to build (Jordan 2018). Those who are currently developing educational AI may end up making inappropriate recommendations that actually hinder learning, and then attempt to generalise their findings across different contexts. But without a theoretical foundation for AI in education we may not have good reasons to reject the conclusions that are provided (ibid.). Much like the theory of Connectivism was developed in response to the emergence of networked learning environments (Siemens 2005), it is likely that we will need a theory of AI in education in response to the emerging challenge of trying to understand the relationship between smart machines and human beings in the context of learning and teaching. Teachers will need to be involved in studies that ask a wide variety of questions around the use of AI in education, none of which begin with assumptions about what works, or what is better. Research that aims to answer empirical questions about student learning should guide decision-making about what projects have merit, and how the outcomes of research should be used to inform education policy.

3.4 Policy We currently lack a language and set of social, professional, ethical and legal norms that will enable us to appropriately implement AI in education. In June 2018, the American Medical Association released a set of guidelines for the use of augmented intelligence systems in the context of clinical care, highlighting the potential impact of AI in health care. The education community needs a similar set of guidelines to

Shaping Our Algorithms Before They Shape Us

161

ensure that AI in education serves the needs of students, teachers and administrators and is fit for the purpose of enhancing student learning. If edtech start-ups are moving into the education space, we will need teachers to first configure that space to ensure that AI-based systems conform to regulatory frameworks and paradigms that prioritise pedagogies of care and justice. These frameworks might suggest that teachers help to set priorities for the use of AI in education, identify opportunities to integrate their own perspectives into the development of AI, and promote the development of thoughtful, pedagogically sound AI. All of this would help to develop the regulatory frameworks, discourse and set of norms within which AI-based systems would be required to operate. It might move us towards mandating that AI-based systems must first show that they will “do no harm” and demonstrate evidence that holistic student well-being and learning will not be compromised by the introduction of algorithms in the classroom. Education policymakers and teachers will need to have difficult conversations related to the current landscape of education, including the quality and range of data used to assess student learning, teaching and learning practices, challenges around the replication and generalisation of education research, and a host of other concerns that will emerge with the inevitable movement of AI into the classroom.

4 Conclusion The introduction of AI into various aspects of teaching and learning is inevitable and the more we rely on algorithms to make decisions, the more they will shape what is seen, read, discussed and learned. These systems will continue improving in several narrow domains of practice, eventually outperforming human beings in a wide variety of routine tasks. While these technologies are in their infancy and therefore unlikely to be implemented at scale in the near future, the arguments presented in the opening of the chapter should highlight the problem in ignoring or trivialising the use of AI in education. Machine learning has important technical limitations that will tend to prioritise grades and other easily measurable variables, rather than values like care and justice. It is also clear that teachers and students are limited in what they are practically able to do. Nonetheless, it is important for all stakeholders to develop strategies for understanding and working with AI-based systems in order to avoid the algorithmic determinism that will otherwise influence our decision-making. We must participate in the conversation around AI development so that the discourse is not framed entirely by software developers and technology entrepreneurs. We must ensure that the voices of students are included, not only in the algorithm training data but in the design, implementation and evaluation of AI-based systems in the classroom. We must refocus our attention on those aspects of teaching and learning that incorporate human values like care, emotional connection and relationshipbuilding. We should design and conduct education research using AI-based systems with the intention of developing and refining a theoretical framework for AI in education. The introduction of AI into education is not a technology problem; it is a

162

M. Rowe

human and social problem. To frame it as a technology problem with a technological solution is to hand the responsibility for stewarding these systems to those who may not have the same pedagogical values as teachers who value student learning. This closes down the opportunities that emerge when a diverse group of people from different disciplines and backgrounds work together on projects with their own unique perspectives. When we see AI as a human problem rather than a technical one, it becomes clear that it is incumbent on all of us—teachers, students and software engineers—to develop an equitable and humane pedagogy of AI in education. To shape our algorithms before they shape us.

References Agrawal, A., Gans, J., & Goldfarb, A. (2018). Prediction machines: The simple economics of artificial intelligence. Boston: Harvard Business Review Press. American Medical Association. (2018). AMA passes first policy recommendations on augmented intelligence. Available from https://www.ama-assn.org/ama-passes-first-policyrecommendations-augmented-intelligence. Andreesson, M. (2011). Why software is eating the world. Available at https://a16z.com/2016/08/ 20/why-software-is-eating-the-world/. Bostrom, N., & Yudkowsky, E. (2014). The ethics of artificial intelligence. In K. Frankish & W. M. Ramsay (Eds.), The Cambridge handbook of artificial intelligence (pp. 316–334). Brynjolfsson, E., & McAfee, A. (2014). The second machine-age: Work, progress, and prosperity in a time of brilliant technologies. New York: W.W. Norton & Company. Frankish, K., & Ramsay, W. M. (2017). The cambridge handbook of artificial intelligence. Cambridge: Cambridge University Press. Freire, P. (2005). Pedagogy of the oppressed. 30th anniversary edition. Continuum. London: The Continuum International Publishing Group Ltd. Giroux, H. (2011). On critical pedagogy. Continuum. London: The Continuum International Publishing Group Ltd. Hart, R. D. (2017). If you’re not a white male, artificial intelligence’s use in healthcare could be dangerous. Quartz. Available at https://qz.com/1023448/if-youre-not-a-white-male-artificialintelligences-use-in-healthcare-could-be-dangerous/. Hendrick, C. (2018). Challenging the ‘education is broken’ and Silicon Valley narratives. ResearchED. Available at https://researched.org.uk/challenging-the-education-is-broken-andsilicon-valley-narratives/. Hooks, B. (1994). Teaching to transgress. Education as the Practice of Freedom. New York: Routledge, Taylor & Francis Group. Jordan, M. (2018). Artificial intelligence—The revolution hasn’t happened yet. Medium. Available at https://medium.com/@mijordan3/artificial-intelligence-the-revolution-hasnt-happenedyet-5e1d5812e1e7. Kahneman, D. (2011). Thinking fast, and slow. Straus and Giroux: Farrar. Lyell, D., & Coiera, E. (2017). Automation bias and verification complexity: A systematic review. Journal of the American Medical Informatics Association, 24(2), 423–431. Lynch, J. (2018). How AI will destroy education. Medium. Available at https://buzzrobot.com/howai-will-destroy-education-20053b7b88a6. Mittelstadt, B. D., Allo., P., Taddeo, M., Wachter, S., & Floridi, L. (2016). The ethics of algorithms: Mapping the debate. Big Data & Society.

Shaping Our Algorithms Before They Shape Us

163

Morris, S. M., & Stommel, J. (2017). Open education as resistance: MOOCs as critical digital pedagogy. In E. Losh (Ed.), MOOCs and their afterlives: Experiments in scale and access in higher education. London: University of Chicago Press. Pearl, J., & Mckenzie, D. (2018). The book of why: The new science of cause and effect. New York: Basic Books. Polanski, V. (2016). Would you let an algorithm choose the next US president? World Economic Forum. Available at https://www.weforum.org/agenda/2016/11/would-you-let-analgorithm-choose-the-next-us-president/. Ritchie, S. (2016). Intelligence: All that matters. Teach yourself. United Kingdom: Hachette. Searle, J. (2011). Watson doesn’t know it won on ‘Jeopardy!’. Wall Street Journal. Retrieved on September 27, 2018 from https://www.wsj.com/articles/ SB10001424052748703407304576154313126987674. Siemens, G. (2005). Connectivism: A learning theory for the digital age. International Journal of Instructional Technology and Distance Learning, 2(1), 3–10. Susskind, R., & Susskind, D. (2015). The future of the professions: How technology will transform the work of human experts. Oxford: Oxford University Press. Watters, A. (2015). Teaching machines and turing machines: The history of the future of labor and learning. Available at http://hackeducation.com/2015/08/10/digpedlab. Williamson, B. (2018). The tech elite is making a power-grab for public education. Code Acts in Education. Available at https://codeactsineducation.wordpress.com/2018/09/14/newtech-power-elite-education/.

Michael Rowe is an Associate Professor in the Department of Physiotherapy at the University of the Western Cape (South Africa). He conducts research into the use of digital technologies in the classroom and their influence on teacher and student relationships as part of teaching and learning practice. His PhD evaluated the use of technology-mediated practices for clinical education, and led to the development of design principles for blended learning environments in the health professions. He is currently interested in the impact of artificial intelligence in clinical and educational contexts. Michael is the founder and Editor of OpenPhysio, an open access journal with a focus on physiotherapy education, and a South African National Research Foundation rated researcher.

Considering AI in Education: Erziehung but Never Bildung Alex Guilherme

Abstract A defining aspect of our modern age is our tenacious belief in technology in all walks of life, not least in education. It could be argued that this infatuation with technology or ‘techno-philia’ in education has had a deep impact in the classroom changing the relationship between teacher and student, as well as between students. Running parallel to this and perhaps exacerbating the problem is the so-called process of ‘learnification’, which understands that teachers are mere facilitators of the learning process, rather than someone with an expertise who has something to teach others. In this article, I first assess the current technologization of education and the impact it has had in relations within the classroom, leading to an understanding of education as Erziehung rather than Bildung; secondly, I investigate through a thought experiment if the development of AI could one day successfully replace human teachers in the classroom. Keywords AI · Gert biesta · Though experiment · Erziehung · Bildung

1 Introduction Biesta and Säfström (2011: 544) comment in the Manifesto for Education that ‘[m]odern education has been associated with the development of the modern welfare state. The early pragmatists of the North American melting pot already saw education as a springboard to a new and better society. Technology was to become the driving force whilst education was to repair the ground for such a new society … [T]he values and norms through which this brave new world would form itself were based on the power of technology to make human living smoother and more effective in achieving its aspirations’. These changes in modern education were meant to instigate the advent of critical and democratic citizens, forming better and more just societies—and this is in direct opposition to the authoritarian Fascists and Nazis A. Guilherme (B) Pontifícia Universidade Católica do Rio Grande do Sul, Porto Alegre, Brazil e-mail: [email protected] © Springer Nature Singapore Pte Ltd. 2019 J. Knox et al. (eds.), Artificial Intelligence and Inclusive Education, Perspectives on Rethinking and Reforming Education, https://doi.org/10.1007/978-981-13-8161-4_10

165

166

A. Guilherme

(and Communists) experiences of the past (cf. Säfström 2004). It is clear in Biesta and Säfström’s Manifesto that there is a longstanding connection between technology and education, which has become increasingly stronger in recent decades due to fast-paced technological changes experienced by humanity. This acceleration is happening in tandem with the process of globalization, meaning that new discoveries and technological developments spread throughout the world very rapidly. Further, it is important to emphasize that the link between technology and education is not confined to the issue of improving learning in schools and the quality of education that is being provided in schools; rather, it is also connected, at least originally, to the idea of citizenship and the improvement of societies, which would become increasingly more democratic and just. This understanding of education would be critical of and has repercussions to the current processes of technologization and learnification in education because they tend to overlook the importance of the right kind of relations between teacher and student, and between students in education. Moreover, the technologization process favours a diminished understanding of education as the mere learning of skills (i.e. Erziehung) and does not favour education as character formation (i.e. Bildung). This problem is compounded by the ‘learnification’ trend, a term coined by Gert Biesta, which fails to appreciate the importance of the role of the teacher and of teaching in the educational process. Given this situation, in this chapter, I provide a discussion on the technologization of education and its implication for education as Erziehung and Bildung; then, I engage with a thought experiment enquiring if the development of AI would one day be capable of fully replacing teachers in the classroom. I wish to emphasize that the position I am defending in this chapter is not that ‘we should not be using technology to aid teaching and learning in the classroom’; rather, the point I am making is that ‘we should not overlook the importance of relations between teacher and student, and between students in the classroom’.

2 Technologization of Education If we look closely, the connection between technology and education is very complex and multifaceted because of the political, economic, social and pedagogical implications that the use of technology has in education. Generally speaking, it is understood that given that we live in ‘technological societies’, we must use technology to help with teaching and learning tasks, and in addition to this, learning about and using technologies must be an important part of the curriculum. It is also understood that the technologization of education will support students who often feel disadvantaged by the traditional educational system, improving their performances through access to computers, special softwares and Internet (Laura and Chapman 2009: 289). For instance, the issue of ‘technological inclusion’ of individuals by means of education has deep social, political and economic effects, such as individuals being fit to join the labour market and contribute to the economic development of societies; likewise, ‘technological exclusion’ presents us with serious social, political and economic

Considering AI in Education: Erziehung but Never Bildung

167

problems, such as unemployment. In addition, the use of technology in education may change educational contexts, their geography, as well as the dynamics between individuals. In fact, Buchanan et al. (2015: 227) note that we have undergo a ‘digital turn’, and this is to say that digital technologies ‘are no longer simply something that students learn “about,” but are now something that they increasingly learn “with”’; digital competences are now imbedded in the curriculum, spanning across all levels of education, and embraced by countries worldwide. These substantial changes in education mean that research must also shift its focus to start ‘looking beyond learning’ (Selwyn 2010: 65; cf. Buchanan et al. 2015: 227), so that ‘wider political, social, and cultural contexts of the use of digital technologies’ are taken into account, identifying the implications of this technologization process for social justice and democracy (cf. Buchanan et al. 2015: 227)—and I would add and emphasize, as already mentioned in the introduction, the impact of this technologization of education on teacher–student and student-student relations. In addition to this, the rapid increase in the use of new technologies in education ‘is definitely not what one would call a slow movement’ (Apple 1988: 151). These changes in schools and in educational systems are associated to a notion of progress and this might lead us to ask questions such as: ‘Whose idea of progress? Progress for what? And fundamentally … who benefits [from this progress]’? (Apple 1988: 151). These are important questions connected to the political economy of education, but it is just as necessary to ask other questions directly connected to changes in the classroom and within the school setting. This hegemonic trend focusing on the importance of technology for education has had a direct impact on teachers and teacher education because they are expected to combine students’ and their own development of i. ‘basic skills’ and ii. ‘creativity and intellectual excellence within a globally technological and economically demanding society’ (Laura and Chapman 2009: 290). However, ‘skills-based training, combined with ever- growing technologies, have overshadowed personal creativity, humour, imagination, intellectual excellence, dialogue, collaborative learning, compassion and spiritual sensitivity, which, in turn, has diminished our educational purpose’ [as teachers] (ibid.). Thus, the tension between the development of basic skills and of personal excellence has not been resolved successfully within the current educational context, which makes us wonder about how successful it has been in encouraging citizenship and the development of more democratic and just societies, as originally envisaged (cf. Biesta and Säfström 2011). My argument here is that the technologization of education has had a deep impact on teachers and teaching because of its focus on education as Erziehung, or education as the learning of a skill or trade, to the detriment of education as Bildung, or education as character formation. It is arguable that this focus on education as Erziehung has serious implications for the social, political and ethical spheres of education because it interferes directly and negatively with the individual’s capacity to be someone who is concerned for others in the community, who engages with the various problematic issues of society, and who is aware of the impact of actions upon himself or herself, others and society as a whole—that is, the Bildung aspect of education.

168

A. Guilherme

Biesta provides us with a very useful discussion on the issue of Bildung.1 Biesta (2002: 344) says: A brief look at one possible history of Bildung shows that there is both an educational and a political dimension to it. On the one hand Bildung stands for an educational ideal that emerged in Greek society and that, through its adoption in Roman culture, humanism, neo-humanism and the Enlightenment became one of the central notions of the modern Western educational tradition. Central in this tradition is the question as to what constitutes an educated or cultivated human being. The answer to this question is not given in terms of discipline, socialisation or moralisation, i.e. as the adaptation to an existing ‘external’ order. Bildung refers to the cultivation of the inner life, i.e. the human mind or human soul….Since then Bildung has always also been self-Bildung.

This educational idea, Bildung, has implications for civil society, for the political arena, because if an individual is capable of thinking autonomously and pass judgement, then he is going to be critical of civil society. Bildung is directly connected to Citizenship (cf. Biesta 2002: 345). Further, the focus on education as Erziehung has happened alongside a pedagogical shift from teaching to learning, the so-called process of ‘learnification’, a term coined by Gert Biesta, which promotes the idea that teaching should be primarily concerned with the creation of rich learning environments that are very often supported by various technologies, such as the use of computer programs and internet connection, to aid scaffolding learning (e.g. a computer program to help with the learning of ‘Logic’ or ‘Ancient Greek’). In doing so, this process of ‘learnification’ has also attacked the idea that ‘teachers have something to teach and that students have something to learn from their teachers’ (Biesta 2010, 2013: 451). Biesta (2015: 76) writes: In the past decade I have written about a phenomenon which I have referred to as the ‘learnification’ of educational discourse and practice. ‘Learnification’ encompasses the impact of the rise of a ‘new language of learning’ on education. This is evident in a number of discursive shifts, such as the tendency to refer to pupils, students, children and even adults as ‘learners’; to redefine teaching as ‘facilitating learning’, ‘creating learning opportunities’, or ‘delivering learning experiences’ or to talk about the school as a ‘learning environment’ or ‘place for learning’. It is also visible in the way in which adult education has been transformed into lifelong learning in many countries. The influence of constructivist psychological theories and of thinkers like Vygotsky and Bruner in this paradigm shift is quite evident, but also misguided—it is a misreading of constructivism. In this connection, and as we have mentioned before, valid constructivism does not portray the teacher as a facilitator. This can be demonstrated convincingly by referring to the sociocultural theory of Vygotsky and Bruner, which considers that the individual learns in the condition of a social being. Through education, the subject receives models and cognitive supports that help him acquire certain knowledge. The teaching and learning process, viewed from this perspective, 1 Bildung and Erziehung are usually translated as ‘education’ in English. However, in other Germanic

languages, a distinction is present; for instance: Vorming in Dutch; Bildning in Swedish (cf. Biesta 2002: 344).

Considering AI in Education: Erziehung but Never Bildung

169

uses an instrument that plays a fundamental role: language. It has two functions, one communicative and the other cognitive. Communicative because, through language, those who teach and those who learn exchange their thoughts. Cognitive because it is the vehicle through which the child internalizes the concepts of their culture. This theoretical model is important insofar as it gives a central role to the social aspects of learning (Lacasa 1994). Following on from this, to understand the individual aspects of the construction of knowledge, it is fundamental to understand the social relations in which the individual develops. Vygotsky considers that the transition from the social to the individual implies a transformation. To explain this process of change, he elaborated the concepts of ‘Internalization’, ‘Zone of Proximal Development’ (ZPD) and mediation of the ‘More Knowledgeable Other (MKO)’ (Vygotsky 1995). Coll (1990), a prominent commentator on the constructivist school, helps us to understand the process of teaching and learning from the perspective of sociocultural theory. He states that the constructivist conception is organized around three fundamental ideas: 1. The student is ultimately responsible for his or her own learning process. It is the student who constructs knowledge and no one can replace him in this task. 2. Students internalize, construct or reconstruct objects of knowledge that had already been socially elaborated by his society. 3. This situation of pre-existing and accepted cultural knowledge has implications for the role of the teacher. The teacher’s function cannot be limited solely to creating the optimal conditions for the student to realize their own individual rich and diverse mental construction. The teacher, as the More Knowledgeable Other (MKO), must manage and guide the student activity so that their developing comprehension progressively understands the meanings and representations of the culture. This guidance occurs through the Zone of Proximal Development (ZPD), establishing what the student already knows, and what can be learned with the help of the More Knowledgeable Other (cf. Guilherme et al. 2017). However, this gives rise to a tension in what a teacher is and what teaching entails, because the teacher, by definition, is someone who has something to teach students, and not merely a facilitator of the learning process as the misguided understanding of constructivism would have (cf. Guilherme 2014: 252–253). In connection to this, Guilherme (2014: 256) argued whilst commenting on Martin Buber’s philosophy of education that: Buber suggests that there is something that is essential to education; that is, the act of teaching must fundamentally entail revealing something that was hidden from the student, the Other…It is important to note…that this revelation does not occur just at the Erziehung level, when the student in a ‘eureka’ moment grasps how to perform a task successfully (e.g. how to do additions), it also happens at the Bildung level, when the individual understands the importance, the ethical weight, of being a moral being (e.g. the serious consequences of lying).

Hence, I argue that as a consequence of the diminished understanding of education as Erziehung, because of both the modern technologization trend and of the failure to appreciate the importance of the role of the teacher and of teaching due to

170

A. Guilherme

the learnification process, there has been a significantly negative impact on the relations between teachers and students, and between students, in education. I believe this is something that is often overlooked by both educators and policy makers. In connection with this, several studies have established the importance of the quality of relationships between students and teachers for issues of personal self-esteem, motivation to learn and confidence in facing new challenges, all of which play a crucial role in overall academic achievement (Laura and Chapman 2009: 290). For instance, McDevitt et al. (2013: 15; 456) note that: [R]eciprocal relationships exist between children and their environments…[I]f parents and teachers develop mutually respectful relationships, they may exchange information and together reinforce their support for a child. If parent-teacher relationships are poor, they may blame each other for a struggling child´s limitations, with the result that no one take responsibility to teach the child needed skills….[Further], when caregivers are kind and responsive, children begin to trust them and gain confidence in their own abilities. We learn that good relationships help children express their emotions productively and blossom into healthy, one-of-a-kind personalities. Finally, we see that educators can contribute immensely to children´s healthy emotional development.

It is thus very ironic, given that relationships are something very important in education, that the impact of the technologization of education and its potential depersonalization of the classroom is not discussed in more detail and philosophically questioned. It seems that in some quarters, we have been too ready to accept the successes of technology in education because it is very much part of the hegemonic discourse without being critical about it, without questioning its possible hindrances, and this might be the case because it is possible that technology has become the very standard for measuring progress and success, and therefore, the appropriate way of resolving problems, including pedagogical ones (cf. Laura and Chapman 2009: 291). In connection to this criticism, Warschauer et al. (2004: 584–585) noted in a study on computer and Internet connections in American schools, particularly in low-SES schools, that: [T]here is no single digital divide in education but rather a host of complex factors that shape technology use in ways that serve to exacerbate existing educational inequalities. We found effective and less effective uses of information and communication technologies…in…schools. At the same time, we found no evidence to suggest that technology is serving to overcome or minimize educational inequalities within or across the…schools we examined. Rather, the evidence suggests the opposite: that the introduction of information and communication technologies in the…schools serves to amplify existing forms of [educational] inequalities.

3 Thought Experiment Thought experiments are powerful philosophical devices that use the imagination to investigate a whole range of theoretical problems. They are commonly used in philosophy, economics and the sciences in general. Kuhn (1977: 241; 261) commented that they are ‘potent tool[s] for increasing our understanding … Historically their role

Considering AI in Education: Erziehung but Never Bildung

171

is very close to the double played by actual laboratory experiments and observations. First, thought experiments can disclose … failure[s] to conform to a previously held set of expectations. Second, they can suggest particular ways in which both expectation and theory must henceforth be revised’. Thus, through resourcing to thought experiment, I wish to investigate if the development of AI could one day successfully replace human teachers in the classroom. AI research has taken generally speaking two interconnected approaches. The first approach, which is very ambitious, seeks to develop a computer program that successfully mimics human intelligence, and in so doing it seeks to find explanatory models for human cognition. The second approach is less bold and seeks to develop computer programs that deal with particular problems (e.g. drawing; chess game; learning a language) without referring to models of human cognition, but which nevertheless display highly intelligent behaviour (McCorduck 1988: 68; cf. also McCorduck 1979). The former aims to imbue computers with the virtue of intelligence with the objective that the computer might one day replace human beings, occupying bureaucratic positions in the armed forces or corporations; the latter envisages developing discreet computer programs that could serve to enhance human intelligence, assisting human beings to carry out certain tasks (cf. Mirowski 2003: 136). This means that AI can be understood in two ways: i. we can understand AI as a computer program that successfully mimics human cognition—this is that which I call a thick conception of AI. ii. we can conceive of AI as a computer program that deals with a particular aspect of knowledge in a highly intelligent way, aiding human beings to perform certain tasks—this is that which I call a thin conception of AI. The terminology thick and thin conception of AI is also referred to in the literature as Weak and Strong AI. Weak AI is usually characterized by a simulation of human intelligence whilst in the case of Strong AI, there is an expectation of genuine understanding and the instantiation of other cognitive states, so that a machine is capable of conscious thought (al-Rifaie and Bishop 2015: 44). This can be further explained in terms of creativity. In the case of Weak AI, the machine is capable of exploring a particular simulation of human creativity, and it is not required to be completely autonomous and provide genuine understanding; and in the instance of Strong AI, there is an expectation that the machine is fully autonomous, have a genuine understanding and is capable of other cognitive states (cf. al-Rifaie and Bishop 2015: 45). There is a long philosophical tradition discussing the possibility of AI, Weak and Strong. Alan Turing published ‘Computing Machinery and Intelligence’ in the prestigious Mind in 1950 (cf. Turing 1950) where he proposed that which became known as ‘The Turing Test’; that is to say: if a computer can pass for a human in an online chat, then we should understand that it displays intelligence. However, in 1980, John Searle challenged this notion, expanding on the discussion about AI, and arguing in his paper ‘Minds, Brains, and Programs’ (cf. Searle 1980) that it is impossible for computers to display intelligence, to understand a language and to think. And in 1999, he summarized his argument in a very elegant manner:

172

A. Guilherme

Imagine a native English speaker who knows no Chinese locked in a room full of boxes of Chinese symbols (a data base) together with a book of instructions for manipulating the symbols (the program). Imagine that people outside the room send in other Chinese symbols which, unknown to the person in the room, are questions in Chinese (the input). And imagine that by following the instructions in the program the man in the room is able to pass out Chinese symbols which are correct answers to the questions (the output). The program enables the person in the room to pass the Turing Test for understanding Chinese but he does not understand a word of Chinese (cf. Cole 2014).

Searle’s argument demonstrates the serious difficulties involved in trying to create a machine capable of Strong AI, and this was further emphasized by him in his paper ‘Why Dualism (and Materialism) Fail to Account for Consciousness’. Searle (2010: 17) says: I demonstrated years ago with the so-called Chinese Room Argument that the implementation of the computer program is not by itself sufficient for consciousness or intentionality (Searle 1980). Computation is defined purely formally or syntactically, whereas minds have actual mental or semantic contents, and we cannot get from syntactical to the semantic just by having the syntactical operations and nothing else. To put this point slightly more technically, the notion “same implemented program” defines an equivalence class that is specified independently of any specific physical realization. But such a specification necessarily leaves out the biologically specific powers of the brain to cause cognitive processes. A system, me, for example, would not acquire an understanding of Chinese just by going through the steps of a computer program that simulated the behaviour of a Chinese speaker. (cf. Cole 2014)

All that said, I note that the contrast between thick conception and thin conception of AI (i.e. Weak/Strong AI) has different implications for education and I shall deal with these in turn. Let me deal with the thin conception of AI first. The use of computer programs to help with teaching and learning is now quite ubiquitous in certain countries, especially in the Global North. These programs have been used to help with a whole range of teaching and learning activities, from aiding with the learning of a particular subject (e.g. ‘Logic’ or ‘Ancient Greek’), to exercise practices and drills (e.g. ‘Arithmetic’ or ‘Geometry’), to formative or summative tests. These are now used at all levels, from primary to postgraduate, and in a whole range of subjects, not just in the sciences but also in the arts and humanities. Many of these programs are AI in essence, fitting the thin conception, and working as instrumental tools that help students to learn their subjects (e.g. Arithmetic).2 One such early AI program aimed at dealing with a particular aspect of knowledge, which fits the thin conception of AI, is AARON, a computer program endowed with ideas about plants, size and shape of human beings, and balance and symmetry in art. This program does thousand of drawings, it knows what it has drawn and will not repeat it unless asked otherwise—thus, focusing on mimicking human creativity. AARON was created by the artist Harold Cohen (cf. McCorduck 1988: 65–66), and one could envisage it as being used pedagogically, teaching students about certain aspects of drawing, such as human and plant physiology in art and balance and 2 Christensen

(1997: 8) notes that the successful use of technology in the classroom is highly dependent on teachers’ attitudes towards computer as well as expertise and experience in the use of technology.

Considering AI in Education: Erziehung but Never Bildung

173

symmetry in composition. It is interesting to note that when questioned if AARON is just producing images or really creating a form of art, Harold Cohen responds that it is indeed art and comments that: ‘Within Western culture … we have always afforded the highest level of responsibility—and praise or blame—to the individual who works on the highest conceptual level. We may hear a hundred different performances of a Beethoven quartet without ever doubting that we were listening to Beethoven. We remember the names of architects, not those of the builders who made their buildings. And, particularly, we value those whose work leaves art in a different state to the state in which they found it” (McCorduck 1988: 81; cf. also McCorduck 1985). It is arguable that AARON can only create a particular form of image; that is, it can work only within a set paradigm. Unlike the human being, AARON cannot change its paradigm and develop a new innovative style of producing images; it cannot argue against or accept criticism against its work; it cannot provide a rationale for why it has chosen to produce a particular drawing, for what inspired it to do so, and this makes us question if it is really intelligent. Similarly, the above criticisms could be raised against AI computer programs currently being used to help with the learning of other subjects such as logic, languages, geometry and so on. However, it is important to note that there have been some new and very interesting developments. Baker et al. (2018: 224) note that: Intelligent Tutoring Systems (ITSs) are hypothesized to be a particularly good candidate from improvement by addressing the connections between affect, cognition, motivation, and learning…ITSs are a type of educational software that offer guided learning support to students engaged in problem-solving. Existing intelligent tutors tailor their support of students’ needs in a variety of ways, including identifying and correcting student errors…and promoting mastery learning through assessments of the probability that the student knows each skill relevant to the system.

Instances of these ITSs are (i) AutoTutor, a software focusing on Newtonian physics, (ii) The Incredible Machine (TIM), a simulation environment for logical puzzles; and (iii) Aplusix, an Algebra learning assistant (Baker et al. 2018: 232–233). Whilst it can be argued that these programs increase contact with the subject, help accessing topics, enable the possibility of doing exercises and drills, facilitate the identification of areas within the subject that require further work, and thus, ‘produce … learning gains … better than classroom teaching alone’ (Boulay and Luckin 2015: 6; cf. also Olney et al. 2012), they cannot, in the same way that AARON cannot, engage on a real dialogue with the student. That is, such programs cannot engage in a real debate over a point of contention, cannot argue against or accept criticism, cannot improvise and pursue a different (and interesting) avenue suggested by students, cannot change its working paradigm. This means that for the self-taught student using such AI computer programs, the educational experience will be confined to a poorer and thinner understanding of education. Education is not just about the learning of a skill (i.e. Erziehung) but also about character formation (i.e. Bildung). In the classroom, the use of such AI programs would only become problematic if the role of the teacher is undermined, if the teacher is seen as a mere ‘facilitator’ due to the process of ‘learnification’ and to the belief that the process of ‘technologization’ will eventually provide all the

174

A. Guilherme

answers. Certainly, some of those who understand that computers and Internet are the very expression of progressiveness in education might fail to see this problem because of their belief on the importance of ‘rich environments’ for students’ learning and that the teacher is a ‘facilitator’ of the process; however, as I have argued, this fails to understand the importance of relationships and human encounter for education. I maintain here that it is not the case that we should not be using technology to aid teaching and learning in the classroom, but at the same time, we should not overlook the importance of relations between teacher and students, and between students in the classroom. We must encourage and facilitate rich relations in education if the educational process is to be fully accomplished, not mere Erziehung and also developing into Bildung, so that teachers and students understand that their reflections and actions have an impact upon themselves, their societies and the world. This brings me to the thick conception of AI, a computer program that successfully mimics human cognition, and the thought experiment inquiring if AI could one day substitute teachers in the classroom. Attempts to create such a computer program have so far been unsuccessful but we can imagine the possible creation of a successful program in the (near) future. Sci-Fi literature and cinema can provide us with some useful examples of this kind of AI, and examples of this are I Robot (2004), A.I. Artificial Intelligence (2001), Bicentennial Man (1999) and Ex Machina (2015) films. The main characters in these films are robots who are clearly capable of intelligent behaviour and meaningful interaction with human beings, providing us with fertile ground for our discussion. In the case of I Robot, the robot character is bestowed with internal laws that prevent it ever harming human beings following from the three laws (i.e. (i) a robot may not injure a human being or, through inaction, allow a human being to come to harm; (ii) a robot must obey the orders given it by human beings, except where such orders would conflict with the First Law; (iii) a robot must protect its own existence as long as such protection does not conflict with the First or Second Laws (cf. Asimov 1950). However, the main robot character in I Robot is incapable of emotions, which makes us feel that it is intelligent but not human-like. Further, because of the internal laws in its programming, the robot is constrained in its capacity to choose otherwise, which contrasts with human beings as we can always choose otherwise, we can choose between A and B, and take responsibility for it, feeling good about our right choices and bad about our wrong ones. We could envisage a computer program, and let us call it T for teacher, which is endowed with the same kind of AI capabilities as that of the robot in the I Robot film. T would be capable of displaying perfect intelligent behaviour, of teaching skills extremely well, of interacting meaningfully with its students, but it would not be capable of feeling emotions (which is arguably a major hindrance in the classroom), of truly connecting with its students (which is very problematic in education, at least insofar as Bildung is concerned). This is to say, as T is not capable of feeling emotions, it would be incapable of truly empathizing with its students in the classroom (e.g. an event has happened and this has had an effect on students) and of reading the mood of the class when teaching and adapting its performance accordingly (e.g. the topic might be considered boring by students and a particular effort

Considering AI in Education: Erziehung but Never Bildung

175

to bring them on board might be necessary); these are all part of the ‘specialized tactics that human teachers apply effectively’ in the classroom, which are derived ‘from the conversational and social interactive skills used in everyday settings such as listening, eliciting, intriguing, motivating, cajoling, explaining, arguing and so on’ (Boulay and Luckin 2015: 4). Further, as T is incapable of truly connecting with its students, the kind of education it can provide will always be confined to the learning of a skill, Erziehung, and will never be capable of developing into character formation, Bildung.3 On a practical level, we can envisage T having problems controlling the class through the use of voice (e.g. raising the voice slightly to catch the groups attention), look (e.g. glancing at a particular group of distracted students), and presence (e.g. controlling behaviour and drawing attention through one’s own presence in the classroom). This is at the heart of the ‘impoverished repertoire of teaching tactics and strategies available to’ A.I. educational systems ‘compared with human expert teachers’ (Boulay and Luckin 2015: 1; cf. also Carroll and McKendree 1987; Ohlsson 1987; Ridgway 1988). However, it is conceivable that the AI program could eventually develop in the same way as the main characters in films such as Bicentennial Man (1999) and A.I. Artificial Intelligence (2001) and develop emotions and the capacity to engage in rich relations. In this case, it is conceivable that the objections raised above would not apply, but it raises serious questions and challenging problems for AI research; for instance: What is consciousness? What is it to be human? Flood (1951: 34; cited in Mirowski 2003: 137) notes: [N]obody really knows anything about consciousness. Now the purpose of Robotology is to take a hard problem such as this one of consciousness, or a relatively easy one like the learning problem - I can feel the psychologists shudder as I say this - so that a mixed team can be truly scientific in their work on them. Robotology, then, is a way of solving the communication problem in the sense that we don’t just let people talk philosophy, or methodology, or just plain hot air; they must talk in terms of something to be put into the design of an object.

The question about the nature of consciousness is very problematic because as we come to see in Ex Machina (2015), the main robot character is so human-like that we start to empathize with it, to believe that when we are faced with it, we are faced with someone like us, with an equal. However, this is just appearances with no substance to it as at the end of the film, we discover that the main robot character only cares for continuing to exist, lacking a moral compass, ethical behaviour and ‘humanity’. The crucial issue then is not to successfully ‘mimic human consciousness’ as it happens in Ex Machina, but to find a way of enabling the rise of something like human consciousness in a machine—and Searle’s Chinese Room Argument applies and would have to be successfully resolved. If this were indeed to happen, then the objections to an AI program substituting teachers permanently in the classroom would no longer 3 I note that some can argue that given that the concept of education as Bildung, as character formation, could be used politically and for ideological purposes, then the notion of education as Erziehung, the learning of skills is preferable because it would be perceived as being more natural and not political. However, as Paulo Freire taught us ‘all education is political’ and therefore, Bildung can be conceived as the formation of critical individuals and citizens, whilst Erziehung as a form of ‘banking education’ and ‘domestication of the masses’ (cf. Freire 1996).

176

A. Guilherme

apply as rich relations between teacher and students would become a real possibility. Perhaps, new developments in AI using strategies such as (i) the observation of human expert teachers, (ii) theoretical derivation from learning theories and (iii) empirical observation of human and simulated students, which are used by Artificial Intelligence Educational Programs such as GURU and INSPIRE (Boulay and Lurkin 2015: 2; 6; cf. also Olney et al. 2012; Lepper and Woolverton 2002) will lead us in this direction.

4 Final Thoughts In this chapter, I set out to assess the current technologization of education and the impact it has had in relations between teachers and students, as well as between students within the classroom. The position I defended was not that ‘we should not be using technology to aid teaching and learning in the classroom’ (otherwise we might still be using just oral skills or wax tablets and stylus); rather, I argued that ‘we should not overlook the importance of relations between teacher and students, and between students in the classroom’. There needs to be a balance between the technologization of education and the provision of the right conditions for rich relations to arise, which is something that educators and policy makers are not always aware. Postman (1995: 171; cited in Laura and Chapman 2009: 293) noted that the introduction of computers and technology in the classroom is an imperative, but when asked the question ‘“[w]hy should we do this?”’, answer that it is ‘“[t]o make learning more efficient and more interesting”. Such an answer is considered entirely adequate, since … efficiency and interest need no justification. It is, therefore, not usually noticed that this answer does not address the question “What is learning for?” “Efficiency and interest” is a technical answer, an answer about means, not ends; and it offers no pathway to a consideration of educational philosophy’. This is to say, that education is not solely for efficiency or market sake. These are pragmatic issues that must be considered but there is much more to education. Education is directly connected to the psychological, social and political facets of the human being, which can only be truly fulfilled by Bildung, and not merely Erziehung.

References al-Rifaie, M. M., & Bishop, M. (2015). Weak and strong computational creaitivity. In T. R. Besold, M. Schorlemmer & A. Smaill (Eds.), Computational creativity research: Towards creative machines. Amsterdam: Atlantis Press. Apple, M. (1988). Teachers and texts: A political economy of class and gender relations in education. London: Routledge. Asimov, I. (1950). I, Robot. New York: Gnome Press. Baker, R. S. Jd., Mello, S. K. D., Rodrigo, M. M. T., & Graesser, A. C. (2018). Better to be frustrated than bored: The incidence, persistence, and impact of learners’ cognitive-affective states during

Considering AI in Education: Erziehung but Never Bildung

177

interactions with three different computer-based learning environments. International Journal of Human-Computer Studies, 68(4), 223–241. Biesta, G. J. J. (2002). Bildung and modernity: The future of Bildung in a world of difference. In Studies in philosophy and education (Vol. 21, pp. 343–351). Biesta, G. J. J. (2010). Good education in an age of measurement: Ethics, politics and democracy. Boulder, CO: Paradigm Publishers. Biesta, G., and Säfström, C.A. (2011). Manifesto on Education. In Policy Futures in Education, 9(5), 540–547. Biesta, G. J. J. (2013). Receiving the gift of teaching: From ‘learning from’ to ‘being taught by. In Studies in philosophy of education (Vol. 32, pp. 449–461). Biesta, G. J. J. (2015). What is education For? On good education, Teacher judgement, and educational professionalism. European Journal of Education, 50(1), 75–87. Boulay, B., & Luckin, R. (2015). Modelling human teaching tactics and strategies for Tutorimh systems: 14 years on. International Journal of Artificial Intelligence Education (earlyview), 1–12). Buchanan, R., Holmes, K., Preston, G., & Shaw, K. (2015). The global and the local: Taking account of context in the push for technologization of education. In S. Bulfin, N. F. Johnson, & C. Bigum (Eds.), Critical perspectives on technology and education (pp. 227–244). New York: Palgrave Macmillan. Carroll, J., & McKendree, J. (1987). Interface design issues for advice-giving expert systems. Communications of the ACM, 30(1), 14–31. Christensen, R. (1997). Effect of technology integration education on the attitudes of teachers and their students (Ph.D. thesis). University of North Texas. Cole, D. (2014). The Chinese room argument. In Stanford encyclopedia of philosophy. Available on: https://plato.stanford.edu/entries/chinese-room/. Last accessed on 05/02/2019. Coll, C. (1990). Un marco de referencia psicológico para la educación escolar: la concepción constructivista del aprendizaje y de la enseñanza”. En C. Coll, J. Palacios, A. Marchesi (Comps.), Desarrollo psicológico y educación. II. Psicología de la Educación. Madrid. Alianza. Flood, M. (1951). Report on a seminar on organizational science. Santa Monica: CA: The RAND Corporation. Freire, P. (1996). Pedagogy or the oppressed. London: Penguin Books. Guilherme, A. (2014). Reflexions on Buber’s ‘Living-Centre’: Conceiving of the Teacher as ‘the builder’ and teaching as a ‘situational revelation’. Studies in Philosophy and Education, 34(3), 245–262. Guilherme, A., Santos, B. S., Spagnollo, C. (2017). Teachers’ lifelong learning: Emerging dialogues from gert biesta’s philosophical views. Policy Futures in Education, 861–873. Kuhn, T. (1977). The essential tension (pp. 240–265). Chicago: University of Chicago Press. Lacasa, P. (1994). Piaget and Vygotsky: A convergent approach to ‘consciousness’, ‘activity’, and ‘word’. In A. Rosa, & J. Valsiner (Eds.), Explorations in social-cultural studies , Historical and Theoretical Discourse, 2, Madrid: Fundacion Infacia y Aprendizaje. Laura, R. S., & Chapman, A. (2009). The technologisation of education: Philosophical reflections on being too plugged in. International Journal of Children’s Spirituality, 14, 3, 289–298. Lepper, M. R., & Woolverton, M. (2002). The wisdom of practice: Lessons learned from the study of highly effective tutors. In J. M. Aronson (Ed.), Improving academic achievement: Impact of psychological factors on education (pp. 135–158). New York: Academic. McCorduck, P. (1979). Machines who think: A personal inquiry into the history and prospect of artificial intelligence. San Francisco, CA: Freeman. McCorduck, P. (1985). The universal machine: Confessions of a technological optimist. New York: McGraw-Hill. McCorduck, P. (1988). Artificial intelligence: An Aperçu. Daedalus, 177(1), 65–83. McDevitt, T. M., Ormrod, J. E., Cupit, G., Chandler, M., & Aloa, V. (2013). Child development and education. Frenchs Forest, Australia: Pearson.

178

A. Guilherme

Mirowski, P. (2003). McCorduck’s machines who think after twenty-five years—Revisiting the origins of AI. AI Magazine, 135–138. Ohlsson, S. (1987). Some principles of intelligent tutoring. In R. W. Lawler & M. Yazdani (Eds.), Learning environments and tutoring systems (pp. 203–237). Alex: Norwood. Olney, A. M., D’Mello, S., Person, N., Cade, W., Hays, P., Williams, C., et al. (2012). Guru: A computer tutor that models expert human tutors intelligent tutoring systems. In Proceedings of the 11th International Conference, ITS 2012, Chania, Crete, Greece, 14th–18th June (pp 127–144). Dordrecht: Springer. Postman, N. (1995). The end of education: Redefining the value of school. New York: Knopf. Ridgway, J. (1988). Of course ICAI is impossible. Worse though, it might be seditious. In J. Self (Ed.), Aritificial intelligence and human learning. London: Chapman and Hall Computing. Säfström, C.A. (2004). Den pedagogiska psykologin, differentieringsfrågan och den liberaldemokratiska värlfärdsstaten [Educational psychology, the issue of differentiation and the liberal democratic welfare state]. In J. Bengtsson (Ed.), Utmaningar i filosofisk pedagogik [Challenges in philosophical education]. Lund: Studentlitteratur. Searle, J. (1980). Minds, brains and programs. Behavioral and Brain Sciences, 3, 417–457. Searle, J. (2010). Why dualism (and materialism) fail to account for consciousness. In R. E. Lee (Ed.), Questioning nineteenth century assumptions about knowledge, III: Dualism. NY: SUNY Press. Selwyn, N. (2010). Looking beyond learning: Notes towards the critical study of educational technology. Journal of Computer Assisted Learning, 26(1), 65–73. Turing, A. (1950). Computing machinery and intelligence. Mind, 59, 433–460. Vygotsky, L. S. (1995). El desarrollo de los procesos psicológicos superiores. Barcelona: Crítica. Warschauer, M., Knobel, M., & Stone, L. (2004). Technology and equity in schooling: Deconstructing the digital divide. Educational Policy, 18(4), 562–588.

Alex Guilherme is Adjunct Professor at Pontifícia Universidade Católica do Rio Grande do Sul. He received his Ph.D. in Philosophy from Durham University (2008), and postdoctoral degree from the Institute of Advanced Studies in Humanity, University of Edinburgh (2010). He worked at the universities of Edinburgh, Durham, and Liverpool Hope University. He has been Visiting Professor at the College of Education at the University of Cambridge and at other universities such as the University of Maastricht, the Bern University, the Université de Neuchatel, the University of Oslo, the University of Luxembourg, and the University of Haifa. He has often visited the Buber Archives at the National and University Library of Israel and the Yad Vashem (Holocaust Museum) in Jerusalem, Israel. He is currently Associate Professor of PPGEdu and PPGP of PUCRS, working mainly in the following subjects: education and violence and education and dialogue. His book Buber and Education: Dialogue the Conflict Resolution (London: Routledge 2014) was nominated for the American Jewish Book Award 2015, and as a result, it was invited to give a talk on the subject of education and conflict resolution at UNESCO headquarters in Paris. He is Project Coordinator for the Capes-PrInt program.

Artificial Intelligence and the Mobilities of Inclusion: The Accumulated Advantages of 5G Networks and Surfacing Outliers Michael Gallagher

Abstract The use of artificial intelligence in a learning increasingly mediated through mobile technology makes inclusion problematic. This is due to the ubiquity of mobile technology, the complexity of the machine learning regimens needed to function within increasingly sophisticated 5G cellular networks, and the legions of professionals needed to initiate and maintain these AI and mobile ecosystems. The promise of artificial intelligence in inclusion is curtailed due to the accumulated advantage (the Matthew effect) presented in such a technological sophistication: only those with the most sophisticated of educational systems will stand to benefit, a scenario that poses significant impact on inclusion strategies increasingly mediated through ICT. Inclusion operates as an outlier in these data-driven environments: as an equitable model in education, it is designed to counter prevailing societal biases, rather than conforming to them. As more and more education is engaged through mobile technology and more and more of that mobile education is driven by an artificial intelligence emerging from curricula of greater and greater sophistication, a situation emerges that poses great challenges for any sort of meaningful inclusion, particularly in the potential acceleration of entrenched advantage. This chapter explores the problematic intersections of AI, mobile technology, and inclusion. Keywords Accumulated advantage · Artificial intelligence · ICT4D · Digital divide · Mobile learning · 5G

1 Relationality and Mobility: Positioning AI in a Broader Sociocultural Context The futures of artificial intelligence and mobile technology are inexorably linked precisely due to the amounts of data generated by the latter being used to fuel the machine learning of the former. Mobile web traffic has overtaken desktop and laptopM. Gallagher (B) Centre for Research in Digital Education, University of Edinburgh, Edinburgh, UK e-mail: [email protected] © Springer Nature Singapore Pte Ltd. 2019 J. Knox et al. (eds.), Artificial Intelligence and Inclusive Education, Perspectives on Rethinking and Reforming Education, https://doi.org/10.1007/978-981-13-8161-4_11

179

180

M. Gallagher

based in many if not most markets with mobile data expected to increase tenfold between 2016 and 2022 growing at a rate of 45% in that span annually (Ericsson 2018), and the number of unique mobile subscribers will reach 5.9 billion by 2025, equivalent to 71% of the world’s population (GSMA 2018a). The big data required for machine learning to build intelligent models suggests a natural pivot to mobile technology; if the future of artificial intelligence needs increasingly more user data to build its intelligence, it must engage in mobile, precisely because data is increasingly generated this way and particularly in developing contexts. This increase in mobile data has profound implications for education and inclusion. Indeed, the burgeoning use of artificial intelligence in a learning increasingly mediated through mobile technology makes technological inclusion problematic. This is largely due to the sheer ubiquity of mobile technology globally, the complexity of the machine learning regimens needed to function within increasingly sophisticated 5G cellular networks, and the legions of professionals needed to initiate and maintain these AI and mobile ecosystems. The promise of artificial intelligence in education is curtailed precisely due to the accumulated advantage presented in such a technological sophistication: only those with the most sophisticated and agile of educational systems will stand to benefit, a scenario that poses significant impact on inclusion strategies increasingly mediated through ICT. The mobile context for the data that powers AI and the technological space in which much AI is deployed (via mobile technology) inherently involves issues of inclusion and exclusion at its onset. There are inherent differentiations of these issues surrounding technological access and use according to geographical, social, political, and economic. A failure to meaningfully address these differentiations will likely entrench existing relations of inclusions and exclusion.

1.1 Mobilities and Education Yet, the critique of such a system of accumulated advantage, and its potential impact on inclusion in education, requires a theoretical lens which mobility theory graciously provides. Mobilities approaches are typified by a structural typology consisting of five mobility types: mobility of objects, corporeal mobility, imaginative mobility, virtual mobility, and communicative mobility (Fortunati and Taipale 2017). What this chapter is primarily concerned with is virtual mobility, the mobility experienced online by Internet users; communicative mobility, person-to-person communication modalities connected to movement; and the effects of these on imaginative mobilities, the representation of mobility as elaborated and broadcasted by the media (ibid.). It is in the intersection of these three mobilities that the use of artificial intelligence in education has the greatest potential impact. More importantly, mobilities approaches are largely non-representational and concerned with the relationality of “bodies and objects and conjoined metabolisms of bodies and space, so that the pulses and rhythms between them are discernible in the shifting mobilities of urban life” (Lefebvre 2004). While this chapter is not concerned

Artificial Intelligence and the Mobilities of Inclusion …

181

with the urban of this description, it is concerned with the relationality of the movements of educational systems, AI, and people; as such mobilities approaches present utility in understanding the “dynamic intersections of people, objects and places, interfaces of the social and spatial” (Waterton and Watson 2013), that permeate this discussion of AI and inclusion. The use of mobilities approaches, beyond providing utilitarian value in unpacking the intersectionality that AI in education foregrounds, also inherently broadens the theoretical gaze away from a humanist assumption, emphasising the material relations that exist between humans and non-humans (Fenwick et al. 2011), the mobilities that course through these relations, and the new educational spaces created as a result. Traditional educational systems built from a territoriality emerging from the compulsory schooling legislation of many countries in the nineteenth and twentieth centuries engendered a particular expectation of social mobility. AI supplants aspects of this educational mobility with potentially dramatic consequences. A focus on mobility pushes away from territoriality with a host of attendant and both positive and desultory effects. These effects seep into educational systems as we are taken away “from such a focus on bounded regions and terrains (the nation, the city, the campus), toward a consideration of the new kinds of ‘mobilities and moorings’ (Hannam et al. 2006: 2) experienced in contemporary political, economic and social space” (Bayne et al. 2014). AI supplants actors in relational educational systems, reconfiguring both the mobilities made possible in this new educational space and introducing “extensive systems of immobility” (Sheller and Urry 2006b: 210) in its wake. Using this mobilities lens with artificial intelligence is necessary precisely because of the significance of the departure that AI poses for traditional educational systems, and consequently, the potentially radical redrawing of the spaces of education created as a result. Emergent technologies such as AI “introduce a significant break in the way individuals, groups and society as a whole conduct their everyday activities, as well as add new dimensions to our understanding of the social world”; these shifts have cascading “practical and epistemological implications for mobile methods” (Hesse-Biber 2011: 4). The immobilities posed by this new relationality are both offshoots of the “material inequalities in the distribution of communication technologies” (Chouliaraki 2012), discussed further in this chapter as an ownership and access inequality, and technological intent; beyond human ownership and access of ICT, AI largely assumes no human involvement at all. This poses a scenario ripe for accumulated advantage: those with significant access to ICT and sophisticated educational systems stand to accumulate advantage by this new relationality. This is positioned in the following sections through a discussion of the Matthew effect and its codification of immobility.

182

M. Gallagher

1.2 Matthew Effect as a Means of Investigating Mobilities: Education, Access, Hardware, Code, and AI as Boundary Objects The Matthew effect, or the Matthew effect of accumulated advantage, was originally designed to refer to the social processes through which various kinds of opportunities for scientific inquiry as well as “the subsequent symbolic and material rewards for the results of that inquiry tend to accumulate” (Merton 1988) in certain practitioners or organisations of science. Namely that those scientists or scientific organisations that have invested in their own development and have gleaned some success from those initial investments will accumulate advantage over a course of time. Broadly, the Matthew effect is used to describe “the general pattern of self-reinforcing inequality” that can be related to economic wealth, political power, influence, educational attainment, or any other desired commodity (Perc 2014). The gaps between those who accumulated advantage (the haves) and those without (the have-nots) “widen until dampened by countervailing processes” (Merton 1988), such as legislation, educational interventions, or shifts in public sentiment or social mores. The Matthew effect also contributes to a number of other concepts in the social sciences, education included, that may be broadly characterised as social spirals. These spirals exemplify positive feedback loops, in which processes feed upon themselves in such a way as to cause nonlinear patterns of growth (Perc 2014). The manifestation of the Matthew effect in education is well documented. Stanovich has documented this effect through the impact of early age reading on the learning of new skills subsequently, noting that falling behind in reading accumulates a disadvantage that proves notably difficult to overcome in later life (2008). Raizada and Kishiyama (2010) further this Matthew effect in education by demonstrating the impact of socioeconomic status on brain development and in enabling a lifelong self-reinforcing trend towards self-control and greater intellectual discovery. As we investigate the speculative futures of artificial intelligence in education towards inclusion, it is critical to consider these feedback loops being created in these educational systems and how they represent potentially an exponential acceleration of accumulated advantage. The Matthew effect provides utility to a mobilities interpretation of the use of AI in education by identifying the material of those mobilities and how that material is increasingly situated in a select few. Further, it assists in identifying the boundary objects of these mobilities. Boundary objects are “artifacts, documents, terms, concepts, and other forms of reification around which communities of practice can organize their interconnections” (Wenger 1998). Positioning AI itself as a boundary object both provides analytical utility (Star 1998) and remains true to this original position of boundary objects, despite the lessening of agency suggested in its use. With AI, boundary objects function as material by which a community can organise their interconnections; with AI, the shift from possibility (can organise) to requirement (is organised by) is implied. Star (1998) advanced boundary objects as a data structure for artificial intelligence as they are designed to be adaptable across multiple viewpoints yet maintain some continuity of identity.

Artificial Intelligence and the Mobilities of Inclusion …

183

Examples of how this Matthew effect is codified through boundary objects are numerous; Merton details the use of intellectual property restrictions as a means of consolidating accumulated advantage for scientists (1988); Bothner et al. (2010) position the accumulation of junior colleagues in academia in much the same way. Merton (1968) discusses admission to the French Academy and its artificial limitation of seats (40) as a means of consolidating advantage and status; Yang et al. (2015) find evidence of the Matthew effect in the uneven distribution of academicians of the Chinese Academy of Sciences amongst different regions and disciplines; Antonelli and Crespi (2013) discussed business in the context of discretionary allocation of public R&D subsidies. Largely what these measures of consolidation collectively represent is a means of manipulating mobility: “the aspiring crowd is likely to exhibit particular structural features (beyond large size) and associated behaviors that make it harder to cross the status boundary and enter the elite status grade—whether that grade is an honorific group of scientists or a band of corporate officers” (Piezunka et al. 2017). These measures of consolidation exist as opaque and largely inaccessible boundary objects, performing a bridging function to the larger community even if the individual engaging with them is not. Boundary objects allow different groups to work together without consensus (Star 2010) and inform those without advantage as to the contours of the community where the accumulated advantage continues to manifest itself (much as is the case with the French Academy example and the prestige afforded those with a seat). AI functions in much this same way: it allows potentially for greater sophistication in collaborative structure for those within and reveals the boundaries of the community to those without community access. Access to and use of AI signals further advantage in community membership. The Matthew effect extends to the Internet itself and the technology used to access it, and indeed, these technologies can and should be seen as boundary objects as they are used by communities to organise their interconnections and mobilities within these interconnections. As AI is built from these earlier technological infrastructures of desktop and mobile connectivity, and indeed still is largely dependent on them, their role as a boundary object remains critical to any speculative future of education. Taipale (2016) discusses this in the context of Internet access, noting the advantages of a mixed fixed/mobile Internet connection in stimulating an advantageous, and accumulated, mobility conferred in the Finnish context on a largely young, male, and urban population. The accumulated disadvantage of a lack of technological ownership and consistent use is felt disproportionately by certain segments of the larger global population, primarily women, children, and broadly those from the Global South: Africa, which has mobile penetration of 82% and Internet penetration of only 34%; Asia-Pacific with mobile penetration at near 100% and Internet penetration at 48% (We Are Social 2018). Globally, women are 12% less likely than men to use the Internet (ITU 2017). Barriers to Internet access and use include cost of devices and data, lack of awareness and understanding of the Internet, lack of education, low confidence, lack of digital skills, poor literacy, a feeling that the Internet is not relevant, concerns around safety and security, and lack of access to infrastructure, such as quality network cov-

184

M. Gallagher

erage and electricity, all of which are experienced more acutely than men (GSMA 2018b). Computer home ownership is rare throughout most of the world; broadband connectivity even less so. All of these limitations mitigate the capacities of these disadvantaged groups to organise their interconnections at a scale and an efficiency enjoyed by those without these limitations; as such, the technology itself functions as a boundary object. For most of the world, mobile technology is and will remain the ICT of first, and in some cases only, use, yet differentiated access within that mobile technological environment exacerbates and even accelerates the Matthew effect. Yet, this is not merely a technological access and use issue. Mobile technology carries with it significant capacity to shape sociocultural exchanges, as well as acting as the “material symbol of one’s relational ties” (Gergen 2003). It acts as a social object (Srivastava 2005) rather than merely as a technological one and is associated with social relationships both symbolically and functionally; further, it provides capacity for and structures the intimacy of these social relationships (Goggin 2012). As such, the Matthew effect as it applies to mobile technology is not merely the expression of a financial, educational, technological, or political deficit, but rather a sociocultural one: the lack of mobile technology, or the possession of a less advanced mobile technology, or the accompanying access issues that governs its use (cellular coverage, cost, literacy or educational capacity, gender dynamics, and more) mitigates the possibility of managing networks of social relationships optimally. The social self in this position enjoys less advantage, and a relative position within a larger sociocultural power dynamic suffers as a result. The performance of sociality within the conduit of mobile technology consequently differentiates dramatically: “the types of mobile phones, usage, and text messages can become key tools in practices of display and disguise, two strategies underpinning the performance of respect” (Pype 2018), tools that provide capacity for increasingly nuanced communication at greater price points. If we are to extend this sociocultural lens in mobile technology to the use of artificial intelligence, we see significant limitations in the types of data being generated that can feed into machine learning; indeed, if data is being generated at all (technological inclusion is particularly pronounced in certain regions—South Asia and sub-Saharan Africa—and amongst particular groups—the gender digital divide, in particular), it is limited to particular frames of activity and is limited, often, to particular actors within communities. Sinha (2018) and others have pointed to the risks that artificial intelligence poses for labour participation for women, but these risks extend beyond financial inclusion. The paucity of meaningful data available for machine learning for particular groups in particular regions will likely reinforce existing sociocultural barriers to inclusion by rendering particular social practices invisible. The accompanying management of social relationships through AI presents significant advantage—personalisation, recommendation services, and the general data-driven automation of one’s social position—yet the disadvantages are rendered largely invisible existing as they exist largely outside technological data. In the frame of artificial intelligence and inclusion, the Matthew effect as expressed in mobile technology is particularly problematic precisely due to the intimacy of the technology and its relationship and structuring of the social engagements encapsu-

Artificial Intelligence and the Mobilities of Inclusion …

185

lated therein. This includes the use of mobile technology in education and the articulation of the artificial intelligence used within this mobile technology. Those with greater technological capacity (emerging from financial and educational advantages) carry with them the possibility of greater capacity for making use of AI educationally: as a key tool in the performance of social relations for educational effect, in the organisation of interconnections and mobilities within these interconnections for educational communities, and even in the offloading of educational labour onto AI (in the execution of computational educational activities, the completion of administrative duties, in educational timetabling, and more). Due to its sheer ubiquity and the social intimacy that it structures, mobile technology represents the most seamless bridge between human education and artificial intelligence.

2 5G Networks, Complexity and Cost as Barriers to Entry Yet, this bridge is increasingly laden with barriers that mitigate inclusion. The increasing use of artificial intelligence more broadly further illustrates the point, precisely in the types of operating infrastructures it requires to prove beneficial. In the space of mobile technology, this is largely the purview of 5G networks. 5G networks are illustrative of the intersectionality of accumulated advantage and the myriad of ways in which this might express itself educationally. 5G cellular networks have built within them access and service provisioning mechanisms unavailable to past mobile networks, mechanisms that both increase the potential complexity of mobile networks and the scope of the benefits provided therein. 5G networks are also inexorably linked to the future of artificial intelligence for both machine learning content (user data, primarily) and the precision required in the execution of AI applications. Some AI applications, such as ones with augmented and virtual realities (AR/VR), require extremely high connection speeds; 5G networks offer multi-gigabit connections. Many other AI applications such as drone surveillance might require large amounts of data capacity and 5G brings this capacity. AI applications will likely require low latency and 5G offers sub-millisecond latency, which is more than 10 times quicker than 4G (Sangam 2018). Pragmatically, this potentially involves machine learning with unstructured data—a mixture of audio, video, text, and numbers that humans process routinely—towards the execution of particular tasks: personal assistants distinguishing commands from different voices in a household; AI stitching together 3D composites of images taken by drones and mobile phones of an emergency response site, to name but two examples. Technologically, the configurable parameters of 2G nodes were 500, 1000 in 3G networks, 1500 in 4G networks with 5G networks expected to surpass 2000 (Li et al. 2017). The sheer volume of configurations made possible with an ever-increasing range of parameters of nodes, along with the self-organising features of a 5G network (self-configuration, self-optimisation, and self-healing), presents a complexity that many without sufficient resources will not be able to enjoy. 5G networks adjust the bandwidth of the data transmission to the variable user density and the speed of

186

M. Gallagher

their movements, yet due to the limited storage and processing capacity of the mobile devices themselves (as opposed to the 5G mobile network as a whole), the processing and storage of Internet queries and activities take place in cloud computing systems (Khetselius et al. 2017), thereby adding an additional layer of complexity (exchanges between the device, the Internet, and the cloud storage and processing centres) and cost (bandwidth, cloud storage, and more). Artificial intelligence would “live” in the complexity of this cloud architecture, providing measures of intelligent automation, predictive analytics, and proactive interventions, ultimately moving towards autonomous systems that “understand, learn, predict, adopt and operate autonomously and give rise to a spectrum of intelligent implementation” (Khetselius et al. 2017: 31). The sheer complexity of such a system and the possibility, if not probability, of autonomous systems operating within it present considerable challenges to those operating on the deficit end of the Matthew effect. The economic barriers to entry to 5G networks are sufficient in and of themselves to prohibit mobility through them. The rollout of 5G networks in Europe alone is expected to account for e57 billion by 2020. This prohibitive cost of entry is merely the first of two time markers on a larger Matthew effect; the potential benefits (estimated in the EU at e113 billion by 2025) are thereby lost by this barrier to entry to the 5G environment (Mansell 2017), benefits that are likely funnelled into further research and development. Further are political considerations as the benefits of investments in 5G networks require concerted and elongated effort, likely across several political regimes; advantages exist in regional cohesion to offset costs and increase the saturation in 5G networks. The 5G Infrastructure Public Private Partnership (5GPPP 2014) is an example of just such a coordinated regional response: 5GPPP is a joint initiative between the European Commission and European ICT industry (ICT manufacturers, telecommunications operators, service providers, SMEs, and researcher Institutions) to research and accelerate the adoption of a 5G infrastructure. Such a coordinated, interdisciplinary, and regional response is unavailable to many. It is likely that in some sectors, the advent of 5G mobile networks will exacerbate the digital divide, despite technological innovations such as free-space optical links and solar-powered equipment in offsetting the cost and skills needed to deploy 5G networks (Lavery et al. 2018). The “material inequalities in the distribution of communication technologies” (Chouliaraki 2012) is a highly intersectional enterprise with potential immobilities presented at each layer of the intersection. It is through this intersectional 5G enterprise that much of the world will interact with or be interacted on by artificial intelligence and will do so in increasingly reduced timeframes: “it took less than 30 years to successfully transform cellular networks from pure telephony systems to networks that can transport rich multimedia content and have a profound impact on our daily life” (Li et al. 2017) and presumably less time for the same to happen with the AI that will increasingly impact our lives through 5G networks.

Artificial Intelligence and the Mobilities of Inclusion …

187

3 The Emerging Divide: AI and Educational Curricula Yet, the digital divide and its spillover effects on inclusion are never exclusively a digital enterprise; as many have posited, it extends well beyond mere technological ownership and extends well into the values and outcomes encoded into education and associated curricula. What the digital suggests in this instance, particularly in the use of artificial intelligence across mobile networks, is both the sheer ubiquity of such an enterprise (in respect of mobile ownership rates worldwide) and the acceleration of deficits generated in such ubiquity: the intersection of AI, 5G networks, and a supporting educational infrastructure to supply expertise to such an endeavour is available to a very select few. This very select few has entrenched this accumulated advantage presented by AI and its use in 5G networks by a structuring of the underlying data on which AI depends. The USA, the European Union, and China are aligning policy with other intersectional factors, such as education, infrastructure, and more, to develop databased economies of scale and in doing so have created three distinct data realms with different approaches to data governance (Aaronson and Leblond 2018). These data realms act as accelerants to the digital divide: those without the means for scale and differentiation (to act independently as a data regime) must spend resources in compliance towards these three regimes. These data regimes, along with the current negative uses of data that have the potential to accelerate with AI (surveillance, reinforcement of monopolies, loss of privacy, and algorithmic profiling, all of which are likely to disproportionately affect those without means), present a data injustice (Heeks and Renken 2018) in that there is a lack of fairness in the way people are made visible, represented, and treated as a result of their production of digital data (Taylor 2017). Artificial intelligence has the capacity to accelerate these injustices by (further) abstracting the relationship between private power and public accountability (Joh 2018). As AI emerged largely from the private sector and mobilises through largely private 5G networks operating on slices of the auctioned mobile spectrum, public accountability becomes a convoluted intersectional affair. The potential for injustice and the increase in the digital divide are probable. All this carries with it profound challenges to existing educational systems and measures taken to improve inclusion; indeed, these challenges are encoded in them. Accumulated advantage is not only a dichotomy between those with access to these sophisticated technologies and those that do not enjoy that same privilege. A further parallel is between the “curriculum” of artificial intelligence for learning in 5G networks as outlined in Li et al. (2017) and traditional and human-centred educational curricula that are being increasingly redrawn as a reductionist enterprise aligned with national and international quantitative metrics. AI has evolved to include multidisciplinary techniques such as machine learning, optimisation theory, game theory, control theory, and meta-heuristics, and various pedagogical applications of these theories in machine learning, unsupervised learning, and reinforcement learning (ibid.).

188

M. Gallagher

Unsupervised machine learning, in particular, bears resemblance to adaptive learning and self-efficacy learning programmes in humans. Unsupervised machine learning exists as a measure of learning consolidation, relying on the AI itself to find the embedded patterns in its own input, rather than through the direction of a secondary instructional agent (ibid.). It exists as a measure of formative assessment, whereby the learner (i.e., the AI) identifies hidden patterns in its input and identifies strategies for consolidating these patterns in future activities. Rather than using AI as a channel for formative assessment for human students in a host of potential instructional roles such as peers, team members, game players, co-workers, teachers (Graesser and McDaniel 2017), and through ongoing “stealth” assessment strategies—a game-based assessment framework which links observed behaviour with evolving competencies (Min et al. 2017)—AI uses unsupervised learning as formative assessment for itself. “In the field of AI, unsupervised learning is applied to estimate the hidden layer parameters in neural networks and plays an important role in deep learning methods” and it is the most widely used AI category in mobile (cellular) networks (Li et al. 2017). The AI that emerges from the curricula of 5G networks and winds into educational spaces will be one that has largely learnt from itself, without direct instruction from human agents. The impact of these techno-educational configurations and their development outside human environments on educational inclusion is potentially significant: curricula, pedagogy, assessments, evaluations, behavioural metrics, and an alignment of educational practice with linear (and predictable) activity. Traditional educational curricula designed for humans are increasingly influenced by third-party commercial enterprises designed to largely simplify the messiness of the educational experience. The commodification of learning, redrawn increasingly as a commercial enterprise, has impacted curriculum development, seeing it in much the same way as policy development: “a linear succession of events (formulation, implementation, evaluation) rather than as a complex, messy and iterative process” (Whitty 2017). This shifting of education towards predictable evaluation and evidence-based policy paradigms, largely attempts to tidy the messiness, have increased private-sector participation in education largely through the use of technology (Riep 2017a, b). This alignment of commercial and educational objectives, and its subsequent impact on educational curricula, is born most heavily on those labouring under an accumulated disadvantage. Education is largely being redrawn as a reductionist enterprise as curricula are being aligned to largely derivative computational models of learning (Azhar 2016). It is an education that is designed to provide skills associated with an unbundled labour market: task decomposition, task completion, moments of labour in small gaps in time (Teevan 2016), largely a predictable and granular approach to education. Education is increasingly attempting to recreate the computation practices at work in the technology sector and repurpose them into pedagogical employ (discussed in Gallagher 2019). Examples abound in this area. Bridge International Academies, particularly active and contentious in India, Uganda, Liberia, and Kenya, use technologies and a highly formalised curriculum “to construct mass markets for low-cost schooling, including GPS devices that map low-income communities, smartphones that automate admin-

Artificial Intelligence and the Mobilities of Inclusion …

189

istrative functions, and computer devices that perform the duties of a teacher” (Riep 2017a, b). As Riep details, Bridge International Academies take this turn towards computational thinking towards a new power dynamic through the use of “teachercomputers”, which are tablet e-readers that convey “ … step-by-step instructions explaining what teachers should do and say during any given moment of a class” (BIA 2016); as of 2018, the significant emphasis on tablets as “teacher-computers” has been replaced with tablets as “teacher guides” (BIA 2018), yet the flow from computer to teacher to student remains, what Riep refers to as a type of techagogy, a technology-directed form of pedagogy in which instruction is led by machines. While increasingly contentious and in some instances ending its operations (in Uganda, detailed in McVeigh and Lyons 2017), Bridge International Academies are merely representative of this drive towards techagogy; the political realignment of the teacher servicing the instructions of the computer is merely a further element of a larger intersectionality that potentially accelerates the Matthew effect. The curriculum associated with the machine learning of artificial learning is suffering from no such reductionism: as discussed, machine learning, optimisation theory, game theory, control theory, meta-heuristics, unsupervised learning, and reinforcement learning (Li et al. 2017) all are employed to develop AI in 5G networks. It is a curriculum growing in dynamic complexity to service an increasingly complex field; human-centred education in contrast is increasingly reduced to align with measures designed to increased predictable outcomes. Ultimately, the question that can conceivably be asked in such a scenario is whether the curriculum of artificial intelligence in 5G networks pedagogically surpasses that of traditional educational curricula, particularly in regions where education is increasingly mediated through third-party providers and computational curricula, such as initiatives like Bridge International Academies. As more and more education is engaged through a larger and increasingly commercial educational technology enterprise and more and more of that education is driven by an artificial intelligence emerging from curricula of greater and greater sophistication, a situation emerges that poses great challenges for educational inclusion, particularly for those that largely sit outside the advantageous intersections of education and technology.

4 Rethinking Equitable Futures of Inclusion Returning to the potential acceleration of the Matthew effect in the light of the technological sophistication inherent to artificial intelligence, 5G networks, and increasingly dynamic (for AI) and deterministic (for humans) educational curricula, we pause to consider offsets to this seemingly inevitable accumulation of advantage, what Piezunka et al. (2017) refer to as external judges which seek to limit the operation of the Matthew effect. These external judges might include social unrest or outage (Bebchuk 2009), policy and legislation, ethical frameworks, and educational curricula, all of which have spillover effects on inclusion.

190

M. Gallagher

As the subject of this chapter is the use of AI in education for inclusion, what follows will cleave to this subject, but it bears mentioning that the weight of noneducational external judges (policy, legislation, data protection, possible outrage over surveillance, and transparency) will more greatly impact the shape of the marriage of AI and inclusion than the curriculum used to engage it. However, a curricular focus is significant in that it is largely a codification of entrenched values and advantages (the curriculum as political barometer of what is) and an aspirational endeavour (as a measure of what it could be). As such, the use of AI in education requires potential offsets to the Matthew effect that will likely emerge as a result, particularly for those traditionally disadvantaged groups which inclusion has attempted to serve. These offsets are presented by Dignum (2018) as AI development principles: Accountability: an AI system needs to be able to justify its own decisions based on the algorithms and the data used by it. We have to equip AI systems with the moral values and societal norms that are used in the context in which these systems operate; Responsibility: although AI systems are autonomous, their decisions should be linked to all the stakeholders who contributed in developing them: manufacturers, developers, users and owners. All of them will be responsible for the system’s behaviour; Transparency: users need to be able to inspect and verify the algorithms and data used by the system to make and implement decisions (2018: 6).

Educational institutions that seek to employ AI are further stakeholders in this process, beholden to the same measures of accountability, responsibility, and transparency that Dignum presents here, measures that are increasingly at odds with the data that AI learns from. As with all data-driven technologies, the underlying data that the AI learns from is sensitive to discrimination and bias (Caliskan et al. 2017), a point of particular concern for inclusion. Decisions emerging from AI are assumed, incorrectly, to be emerging from fair and unbiased computations and are less likely to be questioned as biased than those from human agents (The AI Now Report 2016), a position that proves particularly problematic for educational organisations engaged in inclusion, ones that will largely lack the expertise to unpick the biases emerging from AI engaged in ongoing educational work. AI in these spaces will largely be emerging from commercial enterprises and will largely encode the biases and discrimination at work in these commercial spaces (Miller et al. 2018). Data used to train AI for employ in educational inclusion efforts will likely be drawn from broader sectors of society than just educational inclusion programmes. Broader datasets will likely reinforce the biases emerging from society as a whole, including biases that largely disadvantage students in inclusion programmes, an “unequal opportunity virus” (ibid.) coursing through the larger AI apparatus of machines, learning, and educational work. As with all data-driven technologies, the AI will learn from the mobile data that what counts is what is counted, and what has happened will structure what will happen, learning that runs counter to the ideas of equitable educational inclusion. This learning serves to potentially neglect underrepresented groups and reinforce the barriers that made them largely underrepresented in the first instance. It is conceivable that AI that equitably services those in inclusion programmes will need to learn from data and neural networks emerging from these inclusion programmes themselves.

Artificial Intelligence and the Mobilities of Inclusion …

191

This data and these neural networks operate on a different data-driven reality where inclusion is a stated objective of the educational enterprise and not a statistical outlier in a broader societal dataset. Fidelity to Dignam’s (2018) measures of accountability, responsibility, and transparency begins here in the selection of data that drives the learning of AI; too broad a data scope disadvantages those that would otherwise function computationally as outliers, students in inclusion programmes included. Transparency demands that much of this work is surfaced in both human educational curricula, as a critical data education, and AI curricula, as both surfacing the biases in the data-driven machine learning and the employ of an external judge to mitigate these biases, perhaps by way of coding for equitable outcomes or external review. These external judges need to be structural directives, embedded into the machine learning curricula itself, and not merely post-facto compliance mechanisms. The sheer accelerating volume of mobile-generated data and the primacy of its use in machine learning for future artificial intelligence make this transparency problematic, however. Whether this sort of transparency is a probable, or even possible, future for the use of AI in educational inclusion remains to be seen, yet the potential acceleration of the Matthew effect in this context is clear as is the increased interdependence of mobile technology and artificial intelligence.

References Aaronson, S. A., & Leblond, P. (2018). Another digital divide: The rise of data realms and its implications for the WTO. Journal of International Economic Law, 21(2), 245–272. Antonelli, C., & Crespi, F. (2013). The “Matthew effect” in R&D public subsidies: The Italian evidence. Technological Forecasting and Social Change, 80(8), 1523–1534. Azhar, A. (2016). Coding is not enough, we need smarter skills. Financial Times. https://www.ft. com/content/7babc12c-f662-11e5-96db-fc683b5e52db. Bayne, S., Gallagher, M. S., & Lamb, J. (2014). Being ‘at’ university: The social topologies of distance students. Higher Education, 67(5), 569–583. Bebchuk, L. A. (2009). Pay without performance: The unfulfilled promise of executive compensation. Cambridge, MA: Harvard University Press. Bothner, M. S., Haynes, R., Lee, W., & Smith, E. B. (2010). When do Matthew effects occur? Journal of Mathematical Sociology, 34, 80–114. Bridge International Academies (2016). Model. Accessed January 22, 2016. http://www. bridgeinternationalacademies.com/approach/model/. Bridge International Academies. (2018). Teaching tools. Accessed July 13, 2018. http://www. bridgeinternationalacademies.com/academics/tools/. Caliskan, A., Bryson, J. J., & Narayanan, A. (2017). Semantics derived automatically from language corpora contain human-like biases. Science, 356, 183–186. Chouliaraki, L. (2012). Cosmopolitanism as irony: a critique of post-humanitarianism. In After cosmopolitanism (pp. 87–106). London: Routledge. Dignum, V. (2018). Designing AI for human values. ITU Journal, 1(1). Available at: https://www. itu.int/en/journal/001/Pages/01.aspx. Ericsson. (2018). Future mobile data usage and traffic growth. Available at: https://www.ericsson. com/en/mobility-report/future-mobile-data-usage-and-traffic-growth. Fenwick, T., Edwards, R., & Sawchuk, P. (2011). Emerging approaches to educational research: Tracing the sociomaterial. London: Routledge.

192

M. Gallagher

Fortunati, L., & Taipale, S. (2017). Mobilities and the network of personal technologies: Refining the understanding of mobility structure. Telematics and Informatics, 34(2), 560–568. Gallagher, M. (2019 Forthcoming). Moving beyond microwork: Rebundling digital education and reterritorialising digital labour. In M. A. Peters, P. Jandri´c, & A. J. Means (Eds.), Education and technological unemployment. Berlin: Springer. Gergen, K. J. (2003). Self and community in the new floating worlds. In K. Nyiri (Ed.), Mobile democracy: Essays on society, self, and politics. Vienna: Passagen Verlag. Goggin, G. (2012). Cell phone culture: Mobile technology in everyday life. London: Routledge. GPPP. (2014). The 5G infrastructure public-private partnership. Available at: https://5g-ppp.eu/. Graesser, A., & McDaniel, B. (2017). Conversational agents can provide formative assessment, constructive learning, and adaptive instruction. In The future of assessment (pp. 85–112). London: Routledge. GSMA. (2018a). The mobile economy 2018. Available at: https://www.gsma.com/mobileeconomy/ wp-content/uploads/2018/02/The-Mobile-Economy-Global-2018.pdf. GSMA. (2018b). A toolkit for researching women’s internet access and use. Available at: https://www.gsma.com/mobilefordevelopment/wp-content/uploads/2018/05/GSMAWomen-and-Internet-Research-Toolkit_WEB.pdf. Hannam, K., Sheller, M., & Urry, J. (2006). Editorial: Mobilities, immobilities and moorings. Mobilities, 1(1), 1–22. Heeks, R., & Renken, J. (2018). Data justice for development: What would it mean? Information Development, 34(1), 90–102. Hesse-Biber, S. N. (Ed.). (2011). The handbook of emergent technologies in social research. Oxford: Oxford University Press. International Telecommunication Union (ITU). (2017). ICT facts and figures 2017. Available at: https://www.itu.int/en/ITU-D/Statistics/Documents/facts/ICTFactsFigures2017.pdf. Joh, E. E. (2018). Artificial intelligence and policing: First questions. Available at: https://papers. ssrn.com/sol3/papers.cfm?abstract_id=3168779. Khetselius, O. Y., Glushkov, A. V., Buyadzhi, V. V., & Bunyakova, Y. Y. (2017). New generalized chaos-dynamical and neural networks approach to nonlinear modeling of the chaotic dynamical systems. Photoelectronics, 26, 29–40. Lavery, M. P., Abadi, M. M., Bauer, R., Brambilla, G., Cheng, L., Cox, M. A., … & Marquardt, C. (2018). Tackling Africa’s digital divide. Nature Photonics, 12(5), 249–252. Lefebvre, H. (2004). Rythmanalaysis: space, time and everyday life. London: Continuum. Li, R., Zhao, Z., Zhou, X., Ding, G., Chen, Y., Wang, Z., et al. (2017). Intelligent 5G: When cellular networks meet artificial intelligence. IEEE Wireless Communications, 24(5), 175–183. Mansell, R. (2017). Are we losing control? Intermedia, 45(3), 4–7. McVeigh, K., & Lyons, K. (2017, May 5). ‘Beyond justification’: teachers decry UK backing for private schools in Africa. The Guardian. https://www.theguardian.com/global-development/ 2017/may/05/beyond-justification-teachers-decry-uk-backing-private-schools-africa-bridgeinternational-academies-kenya-lawsuit. Merton, R. K. (1968). The Matthew effect in science. Science, 159, 56–63. Merton, R. (1988). The Matthew effect in science, II: Cumulative advantage and the symbolism of intellectual property. Isis, 79(4) (1988), 606–623. Miller, F. A., Katz, J. H., & Gans, R. (2018). The OD imperative to add inclusion to the algorithms of artificial intelligence. OD PRACTITIONER, 50(1). Min, W., Frankosky, M. H., Mott, B. W., Wiebe, E. N., Boyer, K. E., & Lester, J. C. (2017, June). Inducing stealth assessors from game interaction data. In International Conference on Artificial Intelligence in Education (pp. 212–223). Springer, Cham. Perc, M. (2014). The Matthew effect in empirical data. Journal of the Royal Society, Interface, 11(98), 20140378. Piezunka, H., Lee, W., Haynes, R., & Bothner, M. S. (2017). The Matthew effect as an unjust competitive advantage: Implications for competition near status boundaries. Journal of Management Inquiry. https://doi.org/10.1177/1056492617737712.

Artificial Intelligence and the Mobilities of Inclusion …

193

Pype, JK. (2018). Mobile secrets. Youth, intimacy, and the politics of pretense in Mozambique by Julie-Soleil Archambault (review). African Studies Review, 61(1), 275–277. Raizada, R. D., & Kishiyama, M. M. (2010). Effects of socioeconomic status on brain development, and how cognitive neuroscience may contribute to levelling the playing field. Frontiers in Human Neuroscience, 4, 3. Riep, C. B. (2017a). Fixing contradictions of education commercialisation: Pearson plc and the construction of its efficacy brand. Critical Studies in Education, 1–19. Riep, C. B. (2017b). Making markets for low-cost schooling: The devices and investments behind Bridge International Academies. Globalisation, Societies and Education, 15(3), 352–366. Sangam, P. (2018). Living on the wireless edge with AI And 5G. Available: https://www.forbes. com/sites/forbescommunicationscouncil/2018/09/06/living-on-the-wireless-edge-with-ai-and5g/#71cc8de6b6b4. Sheller, M., & Urry, J. (2006). The new mobilities paradigm. Environment and Planning A, 38, 207–226. Sheller, M., & Urry, J. (2016). Mobilizing the new mobilities paradigm. Applied Mobilities, 1(1), 10–25. Sinha, S. (2018). Gender digital divide in India: Impacting women’s participation in the labour market. In Reflecting on India’s development (pp. 293–310). Singapore: Springer. Srivastava, L. (2005). Mobile phones and the evolution of social behavior. Behavior & Information Technology, 24(2005), 111–129. Stanovich, K. E. (2008). Matthew effects in reading: some consequences of individual differences in the acquisition of literacy. Journal of Education, 189, 23–55. Star, S. L. (1998). 13 Working together: Symbolic interactionism, activity theory, and information systems. Cognition and communication at work, 296. Star, S. L. (2010). This is not a boundary object: Reflections on the origin of a concept. Science, Technology and Human Values, 35(5), 601–617. Taipale, S. (2016). Do the mobile-rich get richer? Internet use, travelling and social differentiations in Finland. New Media & Society, 18(1), 44–61. Taylor, L. (2017). What is data justice? The case for connecting digital rights and freedoms globally. Big Data & Society, 4(2), 2053951717736335. Teevan, J. (2016). The future of microwork. XRDS: Crossroads, The ACM Magazine for Students, 23(2), 26–29. The AI Now Report. (2016, September 22). The social and economic implications of artificial intelligence technologies in the near-term. AI Now (Summary of public symposium). Available at: https:// artificialintelligencenow.com/media/documents/AINowSummaryReport_3_RpmwKHu.pdf. Waterton, E., & Watson, S. (2013). Framing theory: Towards a critical imagination in heritage studies. International Journal of Heritage Studies, 19(6), 546–561. We Are Social. (2018). Digital Report 2018. Available at: https://digitalreport.wearesocial.com/. Wenger, E. (1998). Communities of practice: Learning, meaning, and identity (learning in doing: social, cognitive and computational perspectives). Cambridge: Cambridge University Press. Whitty, G. (2017). The marketization of teacher education: Threat or opportunity? In A Companion to Research in Teacher Education (pp. 373–383). Singapore: Springer. Yang, X., Gu, X., Wang, Y., Hu, G., & Tang, L. (2015). The Matthew effect in China’s science: Evidence from academicians of Chinese Academy of Sciences. Scientometrics, 102(3), 2089–2105.

Michael Gallagher is a Lecturer in Digital Education at the Centre for Research in Digital Education at The University of Edinburgh. At Edinburgh, his projects include the Near Future Teaching project; a project exploring formal partnerships with edX for new educational provision around new redesigned master’s programmes; and projects working with universities in Nepal, Nigeria, Tanzania, and Uganda on digital education in developing contexts. His research focuses on educational futures, educational mobility, mobile technology, and the impact on local knowledge prac-

194

M. Gallagher

tices and communities. He is also Co-Founder and Director of Panoply Digital, an ICT4D consultancy that specialises in educational design for inclusion, particularly in low resource environments and particularly with areas where the gender digital divide is the most pronounced. His projects in this space include ongoing work with the World Bank, USAID, GSMA, UN Habitat, and more.

Artificial Intelligence, Human Evolution, and the Speed of Learning Michael A. Peters and Petar Jandri´c

Abstract Stephen Hawking suggests that a living system has the two parts: “a set of instructions that tell the system how to sustain and reproduce itself, and a mechanism to carry out the instructions” (genes and metabolism). On this definition, computer viruses count as living systems as do artificial intelligences. Hawking explains that human evolution has speeded up. While “there has been no detectable change in human DNA”, “the amount of knowledge handed on from generation to generation has grown enormously” (maybe a hundred thousand times as much as in DNA). This signals that we have entered a new stage of evolution—from natural selection based on the Darwinian model of internal transmission to cultural or self-designed evolution based on an accelerated external transmission of information. This paper presents a thought experiment about philosophical and educational consequences of the possible arrival of: (1) Hawking-inspired postdigital human beings created through self-designed evolution quicker than non-tampered (natural) evolution of human intelligence and (2) algorithmic non-carbon-based “living” systems. In our postdigital age, we are slowly but surely taking natural selection into our own hands, and we need to grapple with the pertinent responsibility. Keywords Stephen hawking · Artificial intelligence · Evolution · Education · Algorithm · Postdigital

1 Introduction Evolution is one of the most contested theories in the history of science. From the right-wing, often religious extremists who completely subscribe to one or another M. A. Peters Beijing Normal University, Beijing, China e-mail: [email protected] P. Jandri´c (B) Zagreb University of Applied Sciences, Zagreb, Croatia e-mail: [email protected] © Springer Nature Singapore Pte Ltd. 2019 J. Knox et al. (eds.), Artificial Intelligence and Inclusive Education, Perspectives on Rethinking and Reforming Education, https://doi.org/10.1007/978-981-13-8161-4_12

195

196

M. A. Peters and P. Jandri´c

creationist theory, to philosophers such as Steve Fuller who thinks “that science could turn out to be a fool’s errand unless we believe that we really have a chance of acquiring ‘God’s point-of-view’ in some literal sense” (Fuller and Jandri´c 2019: 204), theories of evolution have been rivalled by various intelligent design approaches. To further complicate things, the concept of evolution has many meanings and applications. Evolution of species is a fundamental theory in biology and the sciences, yet its application to various social issues is much more contested. In sociology, evolutionary thinking is characterized by two main approaches: “sociobiological explanations and coevolutionary accounts of the interaction of genes and culture” (Dietz et al. 1990). In “Life in the Universe”, Hawking subscribes to a specific coevolutionary account where the interaction between genes and culture becomes increasingly conscious and controlled by human beings. Whatever becomes of Hawking’s predictions, it is impossible to disagree that we are today witnessing an increasing “blurring of human, machine, and nature” (Onlife Initiative 2015: 7, see more in Floridi 2015; Peters and Jandri´c 2019). This blurring is analogue and digital, informational and biological—therefore, we call it postdigital (Jandri´c et al. 2018; Peters and Besley 2019). Furthermore, and Hawking is very aware of this fact, the relationship between information science and biology is inherently political. Peters describes this political relationship using the notion of bio-informational capitalism as “the emergent form of fourth or fifth generational capitalism based on investments and returns in these new bio-industries: after mercantile, industrial, and knowledge capitalisms”, which is “based on a self-organizing and self-replicating code that harnesses both the results of the information and new biology revolutions and brings them together in a powerful alliance that enhances and strengthens or reinforces each other” (Peters 2012: 105). Building on Peters’ work, we recently concluded that The biological challenge needs to be understood as part of the wider innovation of technocapitalism and can only really be understood in term of posthumanism though bio-digitalism – specifically how these two forces between them shape the future of human ontologies of what we can become. Bio-digitalism or bio-informationalism combines that two major technical forces, or force fields, that determine our social environment and it is the intersection of these two forces and our interaction with the bio-digital environment that determines open ontologies. (Peters and Jandri´c 2019).

In “Life in the Universe”, Hawking describes two different directions for further development of life forms. The first direction looks into human beings improved through self-designed evolution. The second direction looks into machines which “would be a new form of life, based on mechanical and electronic components, rather than macromolecules. They could eventually replace DNA based life, just as DNA may have replaced an earlier form of life” (Hawking 1996). At least since Isaac Asimov and his Three Laws of Robotics (Asimov 1950), much has been written about non-carbon-based Artificial Intelligence (AI), and there is a lot of disagreement whether it can be understood as a life form (see Fuller and Jandri´c 2019). In Hawking’s definition, however, life forms are simply defined as self-replicating entities with the ability to improve themselves regardless of their material base (carbon, silicon, etc.). At the current stage of scientific development, it is impossible to predict

Artificial Intelligence, Human Evolution, and the Speed …

197

what will arrive first: human beings improved through self-designed evolution, or non-carbon-based life forms. Furthermore, in the postdigital age characterized by a convergence between information, biology, and nature, these directions of development are dialectically intertwined. Human brain research informs development of AI systems, and AI systems are extensively used in genetic research (Williamson 2019). Therefore, these two radically different directions for further development of life forms require equal attention. This sets the scene for our thought experiment about philosophical and educational consequences of the possible arrival of: (1) Hawking-inspired postdigital human beings created through self-designed evolution which is quicker than non-tampered (natural) evolution of human intelligence, and (2) algorithmic non-carbon-based living systems. Our analysis focuses to education as a project of inclusion. If “we” are to be left behind in the intelligence game, what role or position is left for “us” in education, or the idea of the university? Who amongst us, crucially, gets to remain in the game, or is able to set the agenda? For the purpose, we offer some initial working definitions which arise from Hawking’s “Life in the Universe”. (1) We define life forms as self-replicating and self-improving entities made of any material. (2) We understand evolution in the broadest sense of postdigital coevolution between biology, information, culture, and politics. (3) Our understanding of inclusive education reaches beyond standard UNESCO’s definition of “a process of addressing and responding to the diversity of needs of all learners through increasing participation in learning, cultures and communities, and reducing exclusion from education and from within education” (UNESCO 1998), to include all life forms from definitions (1) and (2).

2 The Postdigital Humans Today we can treat more diseases than ever, but we are far from curing some of the biggest causes of premature death, such as cancer. We are starting to get a grasp on gene therapy, but “the technique remains risky and is still under study to make sure that it will be safe and effective” (US National Library of Medicine 2018a). Looking at neurotechnology, Ben Williamson writes: The possibilities opened up by neurotechnologies suggest the need for novel forms of analysis drawing on postdigital, biosocial, sociotechnical and posthumanist theory and methods that can unpack how human life is being made amenable to being scanned, scraped and sculpted, how new forms of hybrid posthuman, postdigital and plastic subjectivity are being envisaged and to trace how the plastic brain has become the focus of efforts to govern and enhance societies. (Williamson 2019: 83).

Whatever their success, these modifications will inevitably die out in the next generation—to speak of evolution, they should spread over generations. In this area, we are even less powerful. We can clone animals, and probably humans, but it is inconclusive whether these clones will be completely functional in the long run.

198

M. A. Peters and P. Jandri´c

For instance, sheep Dolly, who was the first cloned mammal, has had multiple diseases and has lived a much shorter life than her naturally conceived relatives—and researchers are far from sure whether her problems were related to cloning (Shiels et al. 1999). Scientists are also experimenting with germline gene therapy, which targets egg and sperm cells and “which would allow the inserted gene to be passed to future generations”. However, this kind of therapy is deeply controversial. “While it could spare future generations in a family from having a particular genetic disorder, it might affect the development of a fetus in unexpected ways or have long-term side effects that are not yet known”. Therefore, “the US Government does not allow federal funds to be used for research on germline gene therapy in people” (US National Library of Medicine 2018b). While we have no idea what happens in privately funded laboratories, it is reasonably safe to assume that contemporary science is still quite far from Hawking’s projection “that during the next century, people will discover how to modify both intelligence, and instincts like aggression” (Hawking 1996). Self-designed evolution which is quicker than non-tampered (natural) evolution is still a distant dream, yet even much milder modifications which are currently at hand provoke strong and predominantly negative social responses. Humans undergoing non-inheritable modifications, such as transgender people, are routinely “othered” (Hines 2007; Hester 2018; Préciado 2013), and we can only imagine possible responses to inheritable genetic alterations. During history, Charles Darwin’s theory of evolution has been used to justify all kinds of exclusions, discriminations, and atrocities. Pseudo-scientific theories such as scientific racism, and their more proactive siblings such as eugenics, have been closely associated with poisonous social systems and politics such as slavery, apartheid, and fascism. While we do not want to ask the philosophical question, When does a genetically engineered person cease to be human?—which, in the context of Hawking’s definition of life, becomes redundant—inclusion is indeed a key question for the artificially evolved human. Possible outcomes of such evolution are prominent themes in popular culture and science fiction—areas of human thinking, which are unable to provide scientific insights, but which speak a lot about our collective unconscious at the historical moment here and now (Jandri´c 2017: 133; see also 164). Thinking of technologically improved humans, one usually imagines Hollywood heroes with super skills such as the X-Men (Singer 2000) and Blade Runner (Scott 1982). While these characters always pay a price for their superiority (that’s what makes the plot so interesting!), their alterations are portrayed in a predominantly positive light. Then there are more negative films, such as The Fly (Cronenberg 1986). Here, a scientist unintentionally mixes his own genes during the process of teleportation with genes of the fly. Slowly but surely, in a Kafkaesque manner, he turns into a hybrid between a human and a fly. Hawking warns that complex human traits, “such as intelligence, are probably controlled by a large number of genes. It will be much more difficult to find them, and work out the relations between them” (Hawking 1996). As we have discussed, the US Government explicitly warns about unknown long-term side effects of germline gene treatment in people (U.S. National Library of Medicine 2018b). It seems that Hollywood’s collective unconscious has gotten into the heart of the matter: scenarios of non-natural evolution must take into account both success and failure.

Artificial Intelligence, Human Evolution, and the Speed …

199

Let us start with the happy scenario. If germline gene treatment or some other technology manages to enable people to gain superior intelligence for a small or nonexistent price, then we are facing a problem of dealing with students who are more intelligent than their teachers—and after they leave the school, with their inclusion into the society. At an individual level, educators have a lot of experience of dealing with gifted children. Hawking himself was probably more intelligent than all his teachers, and even his disability did not prevent him from living a fulfilled professional and personal life. If the percentage of such children suddenly went through the roof, it would surely be more difficult to provide them with suitable education. However, this is largely a problem of scale: for as long as improvements in human intelligence are reasonably gradual, education systems should have enough time to adapt to new circumstances. Furthermore, in “Life in the Universe”, Hawking seems to ignore that “intelligence is a complex trait that is influenced by both genetic and environmental factors” (US National Library of Medicine 2015). At present, we are far from complete understanding of relationships between intelligence and education, yet we do know that this relationship is extremely complex. People who score better on intelligence tests tend to stay longer in education, to gain higherlevel qualifications, and to perform better on assessments of academic achievement. Some of the correlations between intelligence scores at the end of primary school and academic results some years later are high, suggesting that it is not just a matter of education boosting intelligence. Also, educational attainment has a moderately high heritability, and a strong genetic correlation with intelligence. On the other hand, there is also evidence that education can provide a boost to scores on tests of complex thinking, and some of these increments last into old age. Therefore, there is probably a bidirectional causal association between intelligence and education. (Deary 2013: 675)

Furthermore, intelligence is closely correlated to other factors such as health. “People who score better on intelligence tests tend to make healthier lifestyle and dietary choices, to have better health, to be less likely to have chronic illnesses like cardiovascular disease, and to live longer” (Deary 2013: 675). Again, possible causes for this correlation are uncertain. And we could go on and on … Looking at various factors such as education, health, social position, parental background, and many others, Deary concludes: “results show that, when it comes to attained social position in maturity, intelligence, education and parental background all count to some extent. That is, there is some meritocracy and intelligence-driven social mobility, and there is also some social inertia” (ibid.). This discussion about relationships between intelligence and education is a special case of the ancient nature versus nurture debate. While the debate reaches far beyond the scope of this chapter, it is worthwhile mentioning that experts strongly disagree about the degree of influence of genetic and environmental factors to various human traits. However, even those authors who strongly downplay environmental factors, such as Robert Plomin, recognize that genetic factors are inextricably linked to environmental factors. In Plomin’s words: Throughout my career I have emphasized nature and nurture, not nature versus nurture, by which I mean that both genes and environment contribute to the psychological differences

200

M. A. Peters and P. Jandri´c

between people. Recognition that both genes and environment are important fosters research at the interplay between nature and nurture, a very productive area of study. (Plomin 2018: 13)

Plomin’s (2018) position in the nature vs. nurture debate sits well with Deary’s (2013) study of relationships between intelligence and education. Good education paired with good social conditions (supportive parents, reasonable well-off surroundings, etc.) can uplift children with moderate physical potentials, and bad education or no education at all, paired with bad social conditions (dysfunctional family, poverty, etc.) can suppress even the best physical potentials. Therefore, film scenarios which predict that education will suddenly get flooded by superhumans are not real. Instead, it is more reasonable to predict that, if superhumans are about to succeed us, education will have a very important role in their creation and upbringing. According to Hawking, the advent of superhumans would create a sharp divide between the “improved” and the “unimproved” humans. In this context, Hawking writes: Once such super humans appear, there are going to be major political problems, with the unimproved humans, who won’t be able to compete. Presumably, they will die out, or become unimportant. Instead, there will be a race of self-designing beings, who are improving themselves at an ever-increasing rate. (Hawking 1996)

In the context of our conclusions, the very notion of self-design is misleading—human intelligence cannot be developed in isolation, and biological improvement always goes in hand with educational improvement. Consequently, biological inequality cannot be viewed in separation from other forms of inequality such as educational inequality—and we have been researching these other forms for centuries. Therefore, instead of theorizing biological inequality from scratch, we need to build the challenge of biological inequality into the existing field of equality studies. At the flip side of improvement and privilege lies a disturbing question: What if genetic engineering does not turn into what it was cracked up to be? What if, for instance, we manage to genetically improve people’s intelligence only at the expense of other physical or psychological traits such as shorter lifespan or proneness to this or that disease? And what happens in a very likely scenario that we cannot predict these changes until they are irreversibly done? Turning into a fly is perhaps a radical example—visually catching, and therefore suitable for Hollywood’s obsession with shocking imaginary. However, the case of sheep Dolly (Deary 2013) and the US Government’s warnings (U.S. National Library of Medicine 2018b) clearly indicate that genetic engineering can bring about unpredictable and unwanted consequences. Our caution is well supported by history; today, we are dealing with unpredicted and unwanted consequences of yesterday’s technologies. We now have ozone holes, because scientists and engineers of the last century used chlorofluorocarbons and other ozone depleting substances as a cheap way of powering refrigerators and spray cans. We are now boiling the planet, because scientists and engineers developed carbon-based industries centuries before they grasped global warming. Today’s genetic engineering is in its infancy. There is a real prospect of creating a human equivalent of ozone holes or global warming, and large-scale tinkering with human

Artificial Intelligence, Human Evolution, and the Speed …

201

beings could end up in a humanitarian nightmare. This type of crisis would require a collective response from the whole society, and we can only hope that it would not lead to a fascism. The postdigital reality is marked by a “biologization of digital reason”, which “is a distinct phenomenon that is at an early emergent form that springs from the application of digital reason to biology and the biologization of digital processes” (Peters and Besley 2019). Hawking’s focus on biological aspects of human evolution, at the expense of various other factors including but not limited to education, follows the logic of “the biologization of our culture”, which Nicanor Ursúa describes as “biological fatalism, which, as an ideology of secularized counter-enlightenment, makes destiny instead of the will of the gods responsible for human hereditary matters: the gods are replaced by genes” (Ursúa 1996: 227). According to our analysis, however, the advent of artificially evolved humans does not seem to lead towards domination of biology, or education, or culture—at least not in the foreseeable future. Accelerated evolution cannot be achieved in isolation of bio-tech laboratories, because it requires a wide array of positive social influences including but far from limited to education. For all we know, questions such as who will be allowed to undergo accelerated evolution, who will be able to afford accelerated evolution, and what should be done with people who have undergone accelerated evolution, are not so far from questions such as who will be allowed to become an artist, who will be able to afford to become an artist, and what should be done with all these artists after they graduate. Yet, the postdigital nature of the genetically altered human being reshapes traditional relationships between biology, education, and culture, thus indeed making its protagonist “both a rupture in our existing theories and their continuation” (Jandri´c et al. 2018: 895). In our postdigital reality, we need to reinvent the concept of (educational) equality with a keen eye on the biological challenge.

3 The Living Machines In our society, carbon-based life forms have very different rights. We grow some life forms for food, we put other life forms in zoos, we adopt some life forms for pets, and when it comes to human beings, we allocate them different rights based on the lottery of place of birth and citizenship. For the sake of simplicity, let us assume that all human beings have equal human rights and equal rights to education. Then the question of rights of non-carbon-based self-improving entities, or the living machines, translates into the question: how far or close are the living machines from human beings? Responding to question to demarcate humans from non-humans, Steve Fuller recently wrote: ‘Human’ began – and I believe should remain – as a normative not a descriptive category. It’s really about which beings that the self-described, self-organised ‘humans’ decide to include. So we need to reach agreement about the performance standards that a putative ‘human’ should meet that a ‘non-human’ does not meet. The Turing Test serves to focus minds on this problem, as it suggests that any being that passes behavioural criteria that we require

202

M. A. Peters and P. Jandri´c

of humans counts as human, regardless of its material composition. While the Turing Test is normally presented as something that machines would need to pass, in fact it is merely a more abstract version of how non-white, non-male, non-elite members of Homo sapiens have come to be regarded as ‘human’ from a legal standpoint. So why not also say ‘non-carbon’ in the case of, say, silicon-based androids? (Fuller and Jandri´c 2019: 207).

Following Fuller’s argument, our question becomes: under which circumstances should we accept living machines as (equal to) human beings? It is extremely hard to quantify humanity, and it would take genius of (at least) Alan Turing’s calibre to give a satisfactory answer. Therefore, we finally settle at a more down to earth question: how should we treat living machines of the moment and their potentially more advanced successors in the future? In this area, we can find plenty of guidance in fields such as philosophy of technology, science and technology studies, psychology, and many others; significant contributions also arrive from popular culture and science fiction. Mapping the field of philosophy of technologies in education, Michael Peters reveals a lot of disagreement between “the humanities versus the engineering traditions” (Peters 2006: 112). He points towards the Heideggerian project of philosophy of technology, the related philosophies of Marcuse, Foucault, and Dreyfus, and then “Haraway’s socialist–feminism project and Feenberg’s sociological constructivism”, as the main philosophical influences.1 These philosophies have found a lot of resonance in educational research, theory, and practice. According to Sian Bayne, the critical posthumanist perspective to education has particularly benefitted from Andrew Feenberg’s philosophy (e.g. Feenberg 2002) about the intersections between the material and the social worlds, “where the human teacher’s agency comes up against the workings of data to conduct another, and different, kind of teaching which is neither human not machinic but some kind of gathering of the two” (Jandri´c 2017: 206). Furthermore, data are inextricably linked to human action, as “the complex systems of data production and representation co-constitute the very systems they purport to describe, and in this process, they often embed, replicate or reinforce pre-existing attitudes and prejudices” (Jones 2018: 49; see also Williamson 2016; Knox 2016). In this way, we arrive to a broader approach of sociomaterialism. The use of the blanket term sociomaterialism has been justified by the claim that all of the foundational theories for this approach (ANT, activity theory, post-humanism and complexity theory) conceptualise knowledge and capacities as being emergent from the webs of interconnections between heterogeneous entities, both human and nonhuman. ANT and more broadly sociomaterialist approaches offer the prospect of being able to integrate the material technologies and media found in networked learning into a framework that encompasses people and machines in a symmetrical way. (Jones 2018: 47)

Bayne and Jones do not use a “strong” concept of equality between human beings and living machines. Instead, they discuss a symmetry—which recognizes that the 1 For

the purpose of the argument, we put all these philosophers under the wide umbrella of “the most important influencers”. However, Peters is very aware of differences and discrepancies between their philosophies; he questions whether “it is correct to construe Marcuse, Foucault, and Dreyfus as Heidggerian” (Peters 2006: 112), and contrasts the Heideggerian programme with Haraway’s and Feenberg’s.

Artificial Intelligence, Human Evolution, and the Speed …

203

two life forms are radically different and imply one or another kind of respectful gathering between the two. However, there is a lot of disagreement whether the relationships between people and machines should be understood as symmetrical. In June 2002, Steve Fuller and Bruno Latour staged a popular public debate with the following motion: “A strong distinction between humans and non-humans is no longer required for research purposes” (Barron 2003: 78). The debate showed that Latour’s position that “all phenomena should be treated equally, whether it comes from something human, natural or artificial” leads to what Fuller calls the “abdication of responsibility” (Fuller and Jandri´c 2019: 212). While the debate “was never intended to offer solutions” (Barron 2003: 98), both sides seemed to agree that treating people and machines in a symmetrical way reaches all the way to questions of values and morality. Looking more pragmatically, Jones similarly “argues that all actors cannot be treated as completely symmetrical for research purposes because of the particular access that we have to accounts of experience from human actors” (Jones 2018: 51). Today’s philosophy does not offer conclusive answers to questions of equality and symmetry between human beings and living machines. At a more practical level, however, the critical posthumanist perspective to education practiced, amongst others, by the Edinburgh University’s Centre for Research in Digital Education (University of Edinburgh 2018) and the Networked Learning community (Jones et al. 2015) provides solid theoretical and practical foundations for (educational) treatment of living machines. In words of Sian Bayne: We should not be asking the question: In 50 years from now, will there be a human or a robot teaching? Rather, we should be asking the question: What kind of combination of human and artificial intelligence will we be able to draw on in the future to provide teaching of the very best quality? What do we actually want from artificial intelligence? We should not allow artificial intelligence in education to be driven entirely by corporations or economists or computing and data scientists – we should be thinking about how we take control as teachers. So the important questions to be asked are: How could we do our jobs better with artificial intelligences? What might that look like, and how might our students benefit? (in Jandri´c 2017: 207)

The critical posthumanist perspective takes into account debates and uncertainties pertaining to relationships between human beings and living machines yet refuses to be restricted by them. It firmly places humans in control of their own destiny yet allows living machines a lot of responsibility and agency in teaching. In this way, the critical posthumanist perspective offers sound guidance for our everyday practice and theoretical background for approaching deep philosophical questions of equality and symmetry between human beings and living machines.

4 Postdigital Humans and Living Machines Inspired by the late Stephen Hawking’s lecture “Life in the Universe”, in this article we tease out some speculations about education of self-replicating and self-

204

M. A. Peters and P. Jandri´c

improving entities of any material. Our thought experiment is imbued with the postdigital paradigm, which corresponds to the blurring of information, human biology, and nature, within the context of bio-informational capitalism. Looking at postdigital human beings created through self-designed evolution which is quicker than non-tampered (natural) evolution of human intelligence, we are reminded that human intelligence is both inherited and developed through one’s lifetime. Genetical engineering may provide people with more efficient brains, but their biological capabilities can never be realized without adequate upbringing and education. This conclusion alleviates us from stress that one lovely day, a bunch of superhumans will walk into our classrooms and wreak havoc, or from (probably even worse) scenario that they will stay away from our classrooms. Who will be allowed to become superhuman and how superhumans will deal with “regular” humans are questions which require us to reach beyond our existing thinking about educational equality focused to teaching and learning practices that work to avoid excluding particular groups of human beings (see UNESCO 1998), and to welcome new, perhaps radically different life forms into our current equality studies. The question of algorithmic non-carbon-based “living” systems, or living machines, is somewhat different. We openly admit that we have no idea what kind of living machines should be accepted as (equal to) human beings. Furthermore, our current concepts such as equality and symmetry might be fundamentally inadequate for representing these complex relationships. “Our postdigital age is one of cohabitation, blurring borders between social actors and scientific disciplines, mutual dependence, and inevitable compromise”. Therefore “we need to develop a new language of describing social relations” (Jandri´c and Hayes 2019), and we need to quickly develop a practical attitude towards treating living machines in our current practices. In this area, we find a lot of good guidance in the critical posthumanist perspective to education. Development of postdigital human beings created through quicker than natural self-designed evolution and algorithmic non-carbon-based living systems are true postdigital challenges consisting of “blurred and messy relationships between physics and biology, old and new media, humanism and posthumanism, knowledge capitalism and bio-informational capitalism” (Jandri´c et al. 2018: 896). These challenges reject totalizing discourses such as educationalization and biologization, reshape fundamental concepts such as equality, reframe political questions such as human rights, and require new ways of acting within the world (Freire 1972). Yet, the crucial role of education as a project of inclusion has become even more prominent. Throughout human history, education has built on top of biological human capabilities. These days, however, education has acquired an opportunity to actively shape biological capabilities, and that opportunity immediately translates into a different kind of responsibility for our collective future. Furthermore, education has always had a contested relationship with technologies, but the advent of living machines has brought an important posthumanist dimension into the fore. The challenge of improved human beings and the challenge of algorithmic non-carbon-based living systems should be understood as two equally important sides of the coin of technological development.

Artificial Intelligence, Human Evolution, and the Speed …

205

The ancient relationship between being and becoming has now acquired an important postdigital turn. While we speculate what kind of future world we will inhabit in coexistence with new forms of intelligent life, we should firmly focus on the questions what forms of intelligent life should be included in our collective decisions about the future and how we might raise them. Postdigital science and education has become an active contributor to the evolution of life. While we examine the development and inclusion of various carbon-based and non-carbon-based life forms, we are slowly but surely taking natural selection into our own hands—and these new powers bring about new responsibilities.

References Asimov, I. (1950). The robot series. New York City: Doubleday. Barron, C. (2003). A strong distinction between humans and non-humans is no longer required for research purposes: A debate between Bruno Latour and Steve Fuller. History of the Human Sciences, 16(2), 77–99. Cronenberg, D. (1986). The fly [Motion picture]. Los Angeles: Brooksfilms and SLM Production Group. Deary, I. J. (2013). Intelligence. Current Biology, 23(16), 673–676. https://doi.org/10.1016/j.cub. 2013.07.021. Dietz, T., Burns, T. R., & Buttel, F. H. (1990). Evolutionary theory in sociology: An examination of current thinking. Sociological Forum, 5(2), 155–171. https://doi.org/10.1007/BF01112590. Feenberg, A. (2002). Transforming technology: A critical theory revisited. New York: Oxford University Press. Floridi, L. (Ed.). (2015). The onlife manifesto: Being human in a hyperconnected era. Springer Science+Business Media. Freire, P. (1972). Pedagogy of the oppressed. Harmondsworth: Penguin Education Specials. Fuller, S. & Jandri´c, P. (2019). The postdigital human: Making the history of the future. Postdigital Science and Education, 1(1), 190–217. https://doi.org/10.1007/s42438-018-0003-x. Hawking, S. (1996). Life in the universe. http://www.hawking.org.uk/life-in-the-universe.html. Accessed October 1, 2018. Hester, H. (2018). Xenofeminism. London: Polity. Hines, S. (2007). Transforming gender: Transgender practices of identity, intimacy and care. Bristol: Policy. Jandri´c, P. (2017). Learning in the age of digital reason. Rotterdam: Sense. Jandri´c, P., & Hayes. S. (2019). The postdigital challenge of redefining education from the margins. Learning, Media and Technology. https://doi.org/10.1080/17439884.2019.1585874. Jandri´c, P., Knox, J., Besley, T., Ryberg, T., Suoranta, J., & Hayes, S. (2018). Postdigital Science and Education. Educational Philosophy and Theory, 50(10), 893–899. https://doi.org/10.1080/ 00131857.2018.1454000. Jones, C. (2018). Experience and networked learning. In N. Bonderup Dohn, S. Cranmer, J. A. Sime, M. de Laat, & T. Ryberg (Eds.), Networked learning: Reflections and challenges (pp. 39–56). Springer International. Jones, C., Ryberg, T., & de Laat, M. (2015). Networked Learning. In M. A. Peters (Ed.), Encyclopedia of educational philosophy and theory. Springer International. Knox, J. (2016). Posthumanism and the MOOC: Contaminating the subject of global education. Abingdon: Routledge. Onlife Initiative. (2015). The onlife manifesto. In L. Floridi (Ed.), The onlife manifesto: Being human in a hyperconnected era (pp. 7–13). Springer Science+Business Media.

206

M. A. Peters and P. Jandri´c

Peters, M. A. (2006). Towards a philosophy of technology in education: mapping the field. In J. Weiss, J. Nolan, J. Hunsinger, & P. Trifonas, (2006). The international handbook of virtual learning environments (Vol. I, pp. 95–116). Dordrecht: Springer. Peters, M. A. (2012). Bio-informational capitalism. Thesis Eleven, 110(1), 98–111. https://doi.org/ 10.1177/0725513612444562. Peters, M. A. & Besley, T. (2019). Critical philosophy of the postdigital. Postdigital Science and Education, 1(1), 29–42. https://doi.org/10.1007/s42438-018-0004-9. Peters, M. A. & Jandri´c, P. (2019). Posthumanism, open ontologies and bio-digital becoming. Educational Philosophy and Theory. https://doi.org/10.1080/00131857.2018.1551835. Plomin, R. (2018). Blueprint: How DNA makes us who we are. Cambridge, MA: MIT Press. Préciado, P. B. (2013). Testo Junkie: Sex, drugs, and biopolitics in the pharmacopornographic era. London: Routledge. Scott, R. (1982). Blade Runner [Motion picture]. Los Angeles: The Ladd Company, Shaw Brothers, Blade Runner Partnership. Shiels, P. G., Kind, A. J., Campbell, K. H. S., Wilmut, I., Waddington, D., Colman, A., & Schnieke, A. E. (1999). Analysis of telomere length in Dolly, a sheep derived by nuclear transfer. Cloning, 1(2). https://doi.org/10.1089/15204559950020003. Singer, B. (2000). X-men. [Motion picture]. Los Angeles: 20th Century Fox, Marvel Entertainment, and The Donners’ Company. UNESCO. (1998). Teachers and teaching in a changing world. Paris: UNESCO. University of Edinburgh. (2018). The Centre for Research in Digital Education. https://www.de.ed. ac.uk/. Accessed October 1, 2018. Ursúa, N. (1996). The biologization of our culture: The challenge of new technologies. In G. Munevar (Ed.), Spanish studies in the philosophy of science (pp. 227–242). Boston: Kluwer. U.S. National Library of Medicine. (2015). Is intelligence determined by genetics? https://ghr.nlm. nih.gov/primer/traits/intelligence. Accessed October 1, 2018. U.S. National Library of Medicine. (2018a). What is gene therapy? https://ghr.nlm.nih.gov/primer/ therapy/genetherapy. Accessed October 1, 2018. U.S. National Library of Medicine. (2018b). What are the ethical issues surrounding gene therapy? https://ghr.nlm.nih.gov/primer/therapy/ethics. Accessed October 1, 2018. Williamson, B. (2016). Digital methodologies of education governance. Pearson plc and the remediation of methods. European Educational Research Journal, 15(1), 34–53. https://doi.org/10. 1177/1474904115612485. Williamson, B. (2019). Brain data: Scanning, scraping and sculpting the plastic learning brain through neurotechnology. Postdigital Science and Education, 1(1), 65–86. https://doi.org/10. 1007/s42438-018-0008-5.