Frontiers And Advances In Positive Learning In The Age Of InformaTiOn (PLATO) 3030265773, 9783030265779, 3030265781, 9783030265786

Table of contents :
Preface......Page 5
Contents......Page 7
Chapter 2: Towards Quality Higher Education: Barriers and Enablers......Page 10
References......Page 14
Part I: Barriers and Facilitators of Learning in Higher Education......Page 15
Introduction and Background......Page 16
The ``Three Cs´´ of Quality Higher Education......Page 17
Barriers to the Achievement of Goals......Page 20
Conclusion......Page 25
References......Page 26
Introduction......Page 28
Role of Higher Education......Page 29
Modeling Truth-Seeking Behavior......Page 32
Conclusion......Page 42
References......Page 44
Why Teach Literature?......Page 47
Literature as Simulation......Page 49
Modeling Deeper Learning......Page 57
Conclusion......Page 59
References......Page 60
Chapter 5: Successful and Positive Learning Through Study Crafting: A Self-Control Perspective......Page 62
Self-Control Strength and Capacity for Self-Control......Page 63
Self-Control Strategies......Page 65
Environmental Self-Control: Study Crafting......Page 67
Improving Structures of Study Programs and Study Tasks: Autonomy......Page 69
Motivating Students´ Crafting Behaviors: Setting Increasingly Complex Goals......Page 70
Summary and Conclusions......Page 72
References......Page 74
Preliminary Remarks......Page 78
A View on the Causal Range......Page 81
Components of ``Education´´ as Sub-Functions of ``Critical Thinking´´......Page 82
Education and Critique......Page 83
Education and Knowledge......Page 85
Education and Morality......Page 86
``Education´´ and ``Critical Thinking´´ Today......Page 87
References......Page 90
Introduction: Three Levels of Critical Thinking......Page 93
Critical Thinking in the USA and in Europe: A Critical Approach......Page 95
Critical Thinking and Domain Specificity......Page 98
Some Basics for Critical Thinking and Epistemic Respect......Page 99
Critical Thinking in Connection with the 7 Social Dimensions......Page 100
Established Test Formats......Page 102
Back to the Three Levels of Critical Thinking: How to Undertake Questioning in Each of It......Page 103
The Basic Situation Stimulating Critical Thinking......Page 104
Critical Thinking and Medical and Psychological Stress Factors......Page 105
Critical Thinking and Professional Morality......Page 106
Critical Thinking and Political Decision-Making......Page 107
Conclusion......Page 108
References......Page 109
Part II: Learning with New Media and Technology......Page 111
Introduction......Page 112
Positive and Negative Media Effects on Learning......Page 113
Pilot Study......Page 114
Discussion: Limitations and Outline for a Broader Research Program......Page 119
References......Page 120
Objectives......Page 123
Media Conversion......Page 124
Two Types of Understanding: Subjective and Objective......Page 125
Sensing States......Page 126
Future Directions......Page 127
References......Page 128
Introduction......Page 129
Logic of Universal Truths......Page 130
Universal Truths as a Tool for Critical Thinking......Page 131
Aristotelian Definitions......Page 132
Aristotelian Definitions as a Pedagogical Tool......Page 134
Summary and Conclusion......Page 135
References......Page 136
Introduction and State of Research......Page 137
Project Aim and Research Questions......Page 138
Participants......Page 139
Study Design......Page 140
Results......Page 142
References......Page 143
Introduction......Page 145
Information Density Through Information Structure......Page 146
Reader Characteristics......Page 147
Reading Goals......Page 148
Types of Memory Traces......Page 149
Methods to Study Comprehension and Episodic Memory Encoding......Page 151
Open Questions and Implications for Student Learning......Page 152
References......Page 153
Idea Density......Page 155
Education and Gender......Page 156
Experimental Procedure......Page 157
Transcription of Speech Data......Page 158
Statistical Analysis......Page 159
Functional Imaging Data......Page 160
Functional Neuroimaging Data......Page 162
Limitations and Future Directions......Page 163
References......Page 164
Part III: Innovative Analytical Approaches for Modeling and Measuring of Learning......Page 166
Introduction......Page 167
Requirement Analysis......Page 170
An Architecture for Modeling the Context-Sensitive Assessment of the Complexity of Texts......Page 174
Module M1: Learning Theory......Page 175
Module M2: Discourse Theory......Page 176
Module M3: Feature Theory......Page 177
Module M4: Schema Learning Theory......Page 182
Module M5: Transfer Learning for Enabling Big Data Analysis of Educational Text Data......Page 184
A Systems Engineering View of Multimodal Cognition......Page 188
On the Way to Modeling Networks of Learners: A Methodical Discussion......Page 189
Conclusion......Page 191
References......Page 192
The Need for Semantic Structures......Page 196
The Cognitive Role of Semantic Structures......Page 197
Situation Modelling......Page 199
Positive and Negative Learning......Page 201
Outlook: Pointers to Multilingual Settings......Page 202
References......Page 203
Setting the Stage: Cross-Linguistic Variation and Cognitive Skills......Page 205
Information Structure: Japanese and Korean as Topic-Prominent Languages......Page 209
Clause Combining and Converbs......Page 212
Evidentials and Modality......Page 220
Potential Effects on Performance......Page 228
Conclusions......Page 230
References......Page 231
Introduction......Page 234
Conceptual Framework......Page 236
Test Instrument and Sample......Page 237
Latent Analysis and IRT Modeling of Student Learning Patterns with Macroeconomics Data......Page 238
Latent Analysis and IRT Modeling of Student Learning Patterns with Microeconomics Data......Page 243
Discussion and Conclusion......Page 246
References......Page 247
Introduction and Background......Page 249
Participants......Page 251
Materials and Procedure......Page 252
Results......Page 253
Discussion......Page 258
References......Page 261
Part IV: Perspectives......Page 262
Introduction and Background......Page 263
Activity Theory of Learning and ``Positive Learning in the Age of Information´´: Could Traditional Theories of Learning Still .........Page 265
Conclusion......Page 268
References......Page 269
PLATO´s Brief History......Page 271
PLATO´s Vision and Mission......Page 273
Toward a Possible Identity for PLATO......Page 275
Education Research into Practice......Page 276
Public Outreach......Page 277
PLATO´s Research......Page 278
Concluding Comment......Page 279
References......Page 280
Contexts and Developments in Knowledge Building in the Internet Age......Page 281
Theoretical Considerations on Learning in Higher Education......Page 285
Knowledge Development in Higher Education......Page 288
Online Reasoning and Critical Handling of Online Information......Page 290
Cognitive Interviews......Page 292
Summary......Page 294
Integration and Further Specification of Learning Models in an Interdisciplinary Research Framework......Page 295
Conclusion......Page 297
References......Page 298
Index......Page 304

Citation preview

Olga Zlatkin-Troitschanskaia Editor

Frontiers and Advances in Positive Learning in the Age of InformaTiOn (PLATO)

Frontiers and Advances in Positive Learning in the Age of InformaTiOn (PLATO)

Olga Zlatkin-Troitschanskaia Editor

Frontiers and Advances in Positive Learning in the Age of InformaTiOn (PLATO)

Editor Olga Zlatkin-Troitschanskaia Department of Business and Economics Education Johannes Gutenberg University Mainz Mainz, Germany

ISBN 978-3-030-26577-9 ISBN 978-3-030-26578-6 https://doi.org/10.1007/978-3-030-26578-6

(eBook)

© Springer Nature Switzerland AG 2019 This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors, and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, express or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. This Springer imprint is published by the registered company Springer Nature Switzerland AG The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland

Preface

The Internet poses not only great opportunities but also many challenges for university learning. Filtering and processing enormous amounts of information is a challenge for all students. Relying on the first few search hits and avoiding contradictory information, they may overlook or ignore facts and well-founded knowledge in the vast masses of information. To reduce complexity and increase time efficiency, students (perhaps subconsciously) may start outsourcing thinking processes and complex information processing to online tools (e.g., it is easier to Google a fact than to remember it). Knowledge acquired in this way is not only rather episodic, inert, not well interconnected, and difficult to retrieve, but can also impair a deeper understanding of subject content or contexts and can lead to misconceptions. This phenomenon of insufficient information processing is referred to as negative learning (NL). NL usually occurs unintentionally and subconsciously and is therefore difficult to avoid, particularly online, as it can easily be amplified through distorted or counterfactual information. Although current learning research can describe what is (not) learned on the Internet, it cannot explain the underlying processes or how learning can be effectively fostered. We do still not know which factors promote or mitigate NL in the interactions between human learning and dynamic, partly artificially intelligent (AI)-based online learning environments. In PLATO, the Internet is modeled as a learning space that comprises both the theoretically available online information resources and the resources students actually use in their course of study in higher education. On this basis, disciplinespecific and individual “information maps” are created and new integrative theories and models for the explanation and prediction of NL are developed, which are then tested in experimental and longitudinal studies. Using innovative approaches from the fields of psychology, neuroscience, computer science, pedagogy, media and communication sciences, linguistics, mathematics, and physics, PLATO models neural, cognitive, affective, and motivational states as well as behavioral patterns in learning on the Internet and examines the way they change during interaction and communication processes. These changes are investigated online and by means of narrative case studies, text and data mining, simulations, big data analyses, and other v

vi

Preface

technologies. The results are then incorporated into the development of practical application tools to prevent and transform NL. The PLATO research program, as well as this volume, would not have been possible without the excellent collaboration of several experts from various disciplines and research fields. This volume is based in particular on the national and international dialogue that enabled unique insights into the most recent, in part not yet published, research projects and developments. We would especially like to thank, among others, Patricia Alexander, Rainer Bromme, Jay McCelland, Onur Güntürkün, Jacqueline Leighton, Thomas Metzinger, Robert Mislevy, Roy Pea, Michael Posner, Manfred Prenzel, Christiane Spiel, and Sam Wineburg for their active contribution to the PLATO program. Of course, not all colleagues and experts were able to contribute to this volume. However, we are very grateful for the collaboration of the abovementioned colleagues as well as many more researchers, as they have made a significant contribution to PLATO. We also thank all the authors who have supported us with their excellent contributions. In particular, we thank our graduate students at the University of Mainz for participating in the PLATO program and for providing tremendous support in preparing this volume, especially Jennifer Fischer and Mirco Kunz. We also thank Katja Kirmizakis and Annika Weibell for editing and proofreading the articles. Finally, we would like to thank the university administration of the JGU Mainz for supporting the PLATO program. Mainz, Germany

Olga Zlatkin-Troitschanskaia

Contents

1

Positive Learning in the Internet Age: Developments and Perspectives in the PLATO Program . . . . . . . . . . . . . . . . . . . . Olga Zlatkin-Troitschanskaia, Walter Bisang, Alexander Mehler, Mita Banerjee, and Jochen Roeper

Part I

1

Barriers and Facilitators of Learning in Higher Education

2

Towards Quality Higher Education: Barriers and Enablers . . . . . . Howard E. Gardner and Wendy Fischman

3

The Role of Modeling for “Seeking Truth” in an Educational Policy Classroom . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . David C. Berliner

21

Literature, Simulation, and the Path Towards Deeper Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Mita Banerjee

41

Successful and Positive Learning Through Study Crafting: A Self-Control Perspective . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Christian Dormann and Christina Guthier

57

On the Relationship Between “Education” and “Critical Thinking” . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Klaus Beck

73

A Three-Level Model for Critical Thinking: Critical Alertness, Critical Reflection, and Critical Analysis . . . . . . . . . . . . . . . . . . . . . Fritz K. Oser and Horst Biedermann

89

4

5

6

7

9

vii

viii

Contents

Part II

Learning with New Media and Technology

8

Positive and Negative Media Effects on University Students’ Learning: Preliminary Findings and a Research Program . . . . . . . 109 Marcus Maurer, Christian Schemer, Olga Zlatkin-Troitschanskaia, and Judith Jitomirski

9

The Role of Media Conversion for Positive Learning . . . . . . . . . . . 121 Koichi Kise

10

Explicating the Logic of Biology to Support Critical Thinking . . . . 127 Vinay K. Chaudhri, Yan Gong, Craig Heller, and Shizuka Yamada

11

Picture Bias in Upper-division Physics Education . . . . . . . . . . . . . . 135 Pascal Klein, Stefan Küchemann, Paul van Kampen, Leanne Doughty, and Jochen Kuhn

12

What Can the Eyes and the Brain Tell Us About Learning? The Role of Information Density in the Comprehension and Retrieval of Complex Concepts . . . . . . . . . . . . . . . . . . . . . . . . 143 Daniela Czernochowski, John Gamboa, and Shanley E. M. Allen

13

The Neural Basis of Idea Density During Natural Spoken Language . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153 Arne Nagels, Svenja Lüll, Lisa Friederich, Benjamin Straube, Michael Grosvald, and Silvia Hansen-Schirra

Part III

Innovative Analytical Approaches for Modeling and Measuring of Learning

14

TextInContext: On the Way to a Framework for Measuring the Context-Sensitive Complexity of Educationally Relevant Texts—A Combined Cognitive and Computational Linguistic Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167 Alexander Mehler and Visvanathan Ramesh

15

From Cognitive Structures to Positive and Negative Learning in a Dialogue Semantics Perspective . . . . . . . . . . . . . . . . . 197 Andy Lücking

16

Performance in Knowledge Assessment Tests from the Perspective of Linguistic Typology . . . . . . . . . . . . . . . . . . . . . . 207 Walter Bisang and Patryk Czerwinski

17

IRT Modeling of Decomposed Student Learning Patterns in Higher Education Economics . . . . . . . . . . . . . . . . . . . . . . . . . . . 237 Susanne Schmidt, Olga Zlatkin-Troitschanskaia, and William W. Walstad

Contents

18

ix

Assessing Mathematics Knowledge and Skill: What College Students Actually Know and Can Do? . . . . . . . . . . . 253 Marta K. Mielicki, Mara V. Martinez, Louis V. DiBello, Alexa W. C. Lee-Hassan, and James W. Pellegrino

Part IV

Perspectives

19

On the Way of Developing a Holistic Explanatory Model of Positive Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269 Andrey Podolskiy

20

PLATO in Search of Identity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277 Richard J. Shavelson

21

What Can We Learn from Theoretical Considerations and Empirical Evidence on Learning in Higher Education? Implications for an Interdisciplinary Research Framework . . . . . . . 287 Olga Zlatkin-Troitschanskaia, Sebastian Brückner, Dimitri Molerov, and Walter Bisang

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 311

Chapter 1

Positive Learning in the Internet Age: Developments and Perspectives in the PLATO Program Olga Zlatkin-Troitschanskaia, Walter Bisang, Alexander Mehler, Mita Banerjee, and Jochen Roeper

The Internet has become the main informational entity, i.e., a public source of information. The Internet offers many new benefits and opportunities for human learning, teaching, and research. However, by providing a vast amount of information from innumerable sources, it also enables the manipulation of information; there are countless examples of disseminated misinformation and false data in mass and social media. Much of the information presented online is conflicting, preselected, or often colliding with fundamental human values and posing moral or ethical problems. When Internet users generate knowledge based on these types of information, negative learning (NL) occurs that can manifest, for instance, in the acquisition of domain-specific misconceptions or counter-factual knowledge (ZlatkinO. Zlatkin-Troitschanskaia (*) Department of Business and Economics Education, Johannes Gutenberg University Mainz, Mainz, Germany e-mail: [email protected] W. Bisang Department of English and Linguistics, Johannes Gutenberg University Mainz, Mainz, Germany e-mail: [email protected] A. Mehler Department of Computer Science and Mathematics, Goethe University Frankfurt am Main, Frankfurt am Main, Germany e-mail: [email protected] M. Banerjee Obama Institute of Transnational American Studies, Department of English and Linguistics, Johannes Gutenberg University of Mainz, Mainz, Germany e-mail: [email protected] J. Roeper Institute of Neurophysiology, Goethe University Frankfurt, Frankfurt, Germany e-mail: [email protected] © Springer Nature Switzerland AG 2019 O. Zlatkin-Troitschanskaia (ed.), Frontiers and Advances in Positive Learning in the Age of InformaTiOn (PLATO), https://doi.org/10.1007/978-3-030-26578-6_1

1

2

O. Zlatkin-Troitschanskaia et al.

Troitschanskaia et al. 2018). NL is difficult to avoid, as it often occurs unintentionally and unconsciously. As shown by, for instance, the Stanford History Education Group (SHEG), only a very small group of Internet users can deal with the complexity of online information. Studies reveal that most Internet users do not have the skills to autonomously select and consciously process (online) information and are vulnerable to being misled and unwittingly acquiring false information (e.g., Wineburg et al. 2018). In the Internet Age, individual beliefs or prejudices become more important than factual knowledge as most Internet users struggle to filter and process vast amounts of information and tend to rely on the first few hits presented in an online search. While concepts of new skills (e.g., information literacy) for autonomous selection and conscious processing of information from an ever-changing online media landscape are emerging in research, such skills alone are considered insufficient for addressing these challenges: “The myth that all young people have a kind of natural ‘digital intelligence’ because they can easily use digital devices is not true” (SHEG). To date, research on human learning has not kept up with the rapid development of information communication technologies (ICT) and little is known about the effects of ICT use on learning. Educational research can currently explain only a small portion of variance in learning (outcomes), both inside and outside of formal ITC-supported education. Current models of learning do not offer sophisticated explanations for the phenomena such as NL in the Internet Age, while higher education neither prevents NL nor guarantees successful learning. PLATO investigates the representation of information on the Internet and social media and its effects on students’ positive learning as a vital, scientifically, morally, and ethically oriented learning approach in academic education. Considering current research, in PLATO, we examine the influence of new ICT realities and upcoming technological developments and how they affect positive learning (PL). PL is defined as the acquisition of warranted (verified) knowledge that is in line with a reliable and scientifically substantiated knowledge base as well as discipline-specific and generic ethical norms and moral values. PLATO is developing an innovative, interdisciplinary approach to examining NL and how it can be avoided or transformed into PL. We focus on the fundamentals of learning: thought processes, language and other sign systems used to process information, properties of the media that transmit the information, beliefs which guide our attention, retention, and others, physiological–psychological states, environments, our social and communication situations during conscious or unconscious acquisition of new information and when we recall it, as well as regularly used ICTs. These aspects and their influences are examined in an integrative model aimed to provide us with a better understanding of how to effectively and autonomously acquire reliable knowledge in the Internet Age, how to design ICTs, and how to shape social and human–machine interactions for PL. Assuming an integrative approach to exploring PL and NL and harnessing new methods enabled by recent technological advances, PLATO has great potential for synergistic effects between the humanities and social sciences, establishing an innovative field of research that will be of significant importance over the next decades.

1 Positive Learning in the Internet Age: Developments and Perspectives

3

This volume is divided into four parts and presents contributions from manifold disciplines, methodologies, and research areas, ranging from the theoretical and conceptual foundations of academic learning in the Internet Age to novel modeling approaches and further perspectives. The first part, “Barriers and Facilitators of Learning in Higher Education,” consists of six articles, which, both theoretically and based on empirical studies, outline different perspectives on the quality of academic education. Howard E. Gardner and Wendy Fischman argue in their contribution that quality higher education requires attention to character, context, and curriculum, and describe key internal and external barriers on the way to positive learning in higher education. David C. Berliner discusses in his contribution how to deal successfully with the problem of the various sources of information, i.e., false knowledge, in higher education. Using practical examples, he emphasizes the importance of the role of modeling for “seeking truth” in an educational policy classroom to help university students learn to verify or refute messages and to discriminate between what is likely to be true, and what is not. Mita Banerjee expands this discussion and focuses on the role of teaching literature for deeper learning, since literary texts can be seen as experimental social action. The proposed model of literary simulations can constitute a path towards positive learning in higher education. Christian Dormann and Christina Guthier argue that using social media and other Internet-based sources could distract students from decent academic learning and lead to negative learning. To foster successful, self-regulated learning, self-control is required. The authors call for developing academic study environments in which self-control is reasonably demanded. Klaus Beck focuses on a key facet of positive learning, “critical thinking,” and discusses the core meaning of this concept and its relationship with “education” from different philosophical traditions. The contribution by Fritz K. Oser and Horst Biedermann continues the consideration of critical thinking. They argue that this construct significantly differs from logical, fluid, or general thinking and consists of three central dimensions: critical alertness, critical reflection, and critical analysis. The second part, “Learning with New Media and Technology,” encompasses six contributions. Marcus Maurer, Christian Schemer, Olga Zlatkin-Troitschanskaia, and Judith Jitomirski present the results of a study on which learning media and online resources students use in academic studies. The findings indicate both positive and negative effects of mass and social media on university students’ learning. Based on eye-tracking and typing analysis, Koichi Kise discusses the role of media conversion for positive learning and how the optimal mixture of different media for maximizing the level of students’ understanding could be achieved by multimedia learning. Vinay K. Chaudhri, Yan Gong, Fiorella Grandi, Craig Heller, and Shizuka Yamada describe how using an intelligent textbook, which is based on mathematical logic and philosophical ontology like the Aristotelian definition, can play an important role in explicitly identifying universally true statements in biology textbooks and improving biology education by teaching critical thinking. Based on eye-tracking studies, Pascal Klein, Stefan Küchemann, Paul van Kampen, Leanne Doughty, and Jochen Kuhn explore possible negative aspects of adding pictures to

4

O. Zlatkin-Troitschanskaia et al.

text, and study the impact of equation density in the context of upper-division physics education. The findings confirm the hypothesis that pictures can also have detrimental effects on students’ learning when surface features are conflicting with scientific concepts. Using online measures of cognitive processing like eye tracking and registering neuronal activity via EEG, Daniela Czernochowski, John Gamboa, and Shanley E. M. Allen investigate the role of text characteristics like information density in the comprehension and retrieval of complex academic concepts. Arne Nagels, Svenja Lüll, Lisa Friederich, Benjamin Straube, Michael Grosvald, and Silvia Hansen-Schirra focus on the neural basis and analyze processing correlates of idea density as an index of general linguistic ability during natural spoken language. Five articles in the third part present a multitude of “Innovative Analytical Approaches for Modeling and Measuring of Learning.” Alexander Mehler and Visvanathan Ramesh propose a combined cognitive and computational linguistic approach as an integrative framework for modeling the context-sensitive complexity of interpretation of educationally relevant texts. This model contributes significantly to the development of measurements for predicting learning outcomes in the form of positive or negative learning that depends on the underlying learning material and texts. Andy Lücking discusses in his paper how dialogue semantics provides representations for building cognitive structures, and expands this perspective to a dialogical model of learning that distinguishes between positive and negative learning. Walter Bisang and Patryk Czerwinski present a new analyzing modeling approach of performance in knowledge tests from the perspective of linguistic typology. The findings reveal that underlying grammatical categories are related to comprehension and reasoning, and that their presence or absence enhances or inhibits students’ ability of problem solving as reflected in their test performance. Susanne Schmidt, Olga Zlatkin-Troitschanskaia, and William W. Walstad present a new item response theory (IRT) modeling approach of decomposed student learning patterns as positive and negative learning in higher education to gather more and richer information about student learning and understanding over the course of studies. The analyses also reveal several test items that show remarkably high values for positive or negative learning. Marta K. Mielicki, Mara V. Martinez, Louis V. DiBello, Alexa W. C. Lee-Hassan, and James W. Pellegrino present a newly developed assessment of college students’ of mathematics knowledge and skills. The analyses reveal highly variable performance within three different content strands but no strong performance differentiation across strands. These findings suggest that particularly the requirement from mathematical knowledge and skills differentiates between high-difficulty and low-difficulty items. The fourth and last part of this volume consists of three contributions that illustrate further “Perspectives” on the investigation of positive and negative learning in higher education in the Internet Age. Andrey Podolskiy proposes a way of developing a holistic explanatory model of positive learning. He considers interdisciplinary collaboration and advanced computational approaches as the most important resources to reaching the goals of PLATO as a holistic, multilayered, multiscale, environment- and value-sensitive model of learning, and highlights the crucial role of a holistic explanatory model of positive learning based on the activity theory of

1 Positive Learning in the Internet Age: Developments and Perspectives

5

learning. Richard J. Shavelson expands this discussion on developing perspectives in PLATO and proposes a framework where research and development take place in “Pasteur’s Quadrant” and in using “inspired basic research.” Olga ZlatkinTroitschanskaia, Sebastian Brückner, Dimitri Molerov, and Walter Bisang consider and consolidate the current state of research on learning in higher education and on the Internet. Building on an overview of fundamental theoretical considerations and empirical evidence, they argue for developing a new interdisciplinary science of learning in the Internet Age. Overall, this volume describes how scholars from the humanities, social, life, and natural sciences as well as artificial intelligence (AI) join efforts to explore the phenomena of NL and PL from distinct but related angles and to identify effective ways to prevent NL and facilitate PL. This task is of immediate importance for academic education and modern societies. PLATO’s highly integrative approach is scientifically and technologically challenging, yet feasible. PLATO fosters this promising endeavor to transcend the limitations of single methods and disciplines over the next decade.

References Wineburg, S., Breakstone, J., McGrew, S., & Ortega, T. (2018). Why Google can’t save us: The challenges of our post-Gutenberg moment. In O. Zlatkin-Troitschanskaia, G. Wittum, & A. Dengel (Eds.), Positive learning in the age of information – A blessing or a curse? (pp. 221–228). Wiesbaden, Germany: Springer. Zlatkin-Troitschanskaia, O., Schmidt, S., Molerov, D., Shavelson, R. J., & Berliner, D. (2018). Conceptual fundamentals for a theoretical and empirical framework of positive learning. In O. Zlatkin-Troitschanskaia, G. Wittum, & A. Dengel (Eds.), Positive learning in the age of information – A blessing or a curse? (pp. 29–52). Wiesbaden, Germany: Springer.

Part I

Barriers and Facilitators of Learning in Higher Education

Chapter 2

Towards Quality Higher Education: Barriers and Enablers Howard E. Gardner and Wendy Fischman

Introduction and Background Over the years, there has been increasing recognition of the need and the importance of education beyond childhood. While an attainment of basic literacies once sufficed, it is now widely acknowledged that individuals need and benefit from a number of more demanding abilities and understandings: the ways of thinking associated with the major disciplines and beyond, the assumptions and values of other cultures as well as their own, and the nature of many changes that are occurring rapidly all over the world. Such an education should enable individuals to find a suitable occupation and to secure a decent life for themselves and their family. If we are to have citizens who are informed and who comport themselves in an ethical manner, higher education should also go well beyond vocational preparation. To understand the forms of higher education that are not purely vocational, we and our colleagues have recently completed an ambitious study of higher education in the United States. Often these forms have been called “liberal arts” or “liberal arts and sciences.” As this phrase is often misunderstood, or not understood at all, we embrace the more neutral phrase with which this essay is titled. Our research approach has been to conduct semi-structured interviews of about an hour with the various constituent groups in select institutions of higher education.1 The ten campuses were quite different from one another. At each institution, we have

1

The project has been generously funded by three anonymous funders, Jackie and Mike Bezos, Carnegie Corporation of New York, Paula and Jim Crown, The Endeavor Foundation, The Meyer and Raena Hammerman Foundation, Thomas H. Lee, The Lumina Foundation, The Andrew W. Mellon Foundation, The Spencer Foundation, The Teagle Foundation, and The Saul Zaentz Charitable Foundation. H. E. Gardner (*) · W. Fischman Harvard Graduate School of Education, Harvard University, Cambridge, MA, USA e-mail: [email protected]; wendy_fi[email protected] © Springer Nature Switzerland AG 2019 O. Zlatkin-Troitschanskaia (ed.), Frontiers and Advances in Positive Learning in the Age of InformaTiOn (PLATO), https://doi.org/10.1007/978-3-030-26578-6_2

9

10

H. E. Gardner and W. Fischman

spoken to approximately 50 incoming students, 50 students who were expected to receive their bachelor’s degree, and a smaller number of faculty, senior administrators, alumni/ae, parents, trustees, and job recruiters (approximately 15–25 in each group). These interviews are wide-ranging; they touch on academic topics, campus issues and resources, social and extra-curricular activities, the overall purposes of higher education, as well as the factors that contribute to or impede the achievement of these goals (see section “Barriers to the Achievement of Goals”). All interview subjects received the same basic set of questions; however, small adjustments were made so that each question was appropriate for each of the several constituencies. While participation in our study was voluntary, we endeavored to have a representative sample of each of these constituent groups. When we had smaller numbers than we would like from a given constituency (e.g., fewer athletes, fewer teachers of science), we usually succeeded in securing additional participants. All in all, we have conducted over 2000 interviews (approximately 200 per campus), recorded and transcribed these interviews, and are currently analyzing the accrued data in multiple ways.2 To provide an accurate representation of higher education, the campuses included in our study vary widely in terms of size, selectivity, location, and mission (Table 2.1). They are all located in the United States; most of them have a significant residential component, and all the campuses have a stated commitment to an education that is not purely vocational. We cannot assume, therefore, that our findings and recommendations will be applicable to institutions of higher education in Europe or in other regions of the world, nor are they necessarily applicable to institutions that are dedicated to vocational education. That said, we suspect that at least some of what we have learned and described in the following will prove applicable elsewhere. We hope that the picture of higher education that we are assembling will prove of use to the PLATO program, which, too, has adopted the ideal of academic education aimed at fostering holistic personal development (for a description of PLATO program, see Zlatkin-Troitschanskaia et al. 2018a).

The “Three Cs” of Quality Higher Education Were we the czars of higher education, we would focus on three issues (each described in some detail in an earlier essay for the PLATO project; Gardner 2018a): • Context: Every institution of higher education should have a stated mission. That mission should be expressed succinctly: widely available, widely known, and

Some of the preliminary findings are reported on our blog “Life Long Learning”—see howardgardner.com. Our methodology is explicated in detail in three blogs (https:// howardgardner.com/2018/08/13/the-method-in-our-madness-data-collection-and-analysis-for-ourstudy-of-higher-education-part-i/). Ultimately we expect to write further articles and books that document our findings and make concrete practical recommendations.

2

Size of undergraduate student body Medium

Small

Small

Small

Small

Medium

Large

Large

Medium

Large

University/ college 1

2

3

4

5

6

7

8

9

10

Public

Public

Public

Public

Private

Private

Private

Private

Private

Type of Institution Public

Table 2.1 Description of campuses in our study

Less selective

Selective

Selective

Less selective

Selective

Very selective

Very selective

Very selective

Very selective

Selectivity (based on SAT scores) Selective

Location in the USA Suburban (Northeast) Urban (Northeast) Suburban (South) Rural (Midwest) Suburban (Northeast) Urban (Midwest) Suburban (West) Urban (Midwest) Urban (Northeast) Urban (Northeast) Non-residential

Non-residential

Residential

Mix of residential and non-residential Non-residential

Residential

Residential

Residential

Residential

Residential/nonresidential Residential

Community college (Associate’s Degree)

Undergraduate liberal arts college Undergraduate professional-orientation Religious identity

Global citizenship

Special focus

2 Towards Quality Higher Education: Barriers and Enablers 11

12

H. E. Gardner and W. Fischman

exemplified in daily practices. Positive examples of “mission accomplished” should be highlighted. When the mission has not been realized, or has been realized imperfectly, or even undermined, that failure should be acknowledged and efforts should be undertaken to recalibrate course. With respect to the PLATO project, the achievement of positive learning within and across disciplines constitutes a laudable goal (for a definition of positive learning, see Zlatkin-Troitschanskaia et al. 2018b). • Character: As part of its mission, every institution should endeavor to form graduates who understand and seek to behave in desirable ways (Minnameier 2018). We use the term “moral” to denote how individuals treat one another in everyday life—we have termed this form “neighborly morality.” We use the term “ethical” to denote how individuals behave in their roles as worker and their roles as citizen—we have termed this desideratum “the ethics of roles” (Weber 1922/ 1946). Some of this moral and ethical formation can take place through courses and regulation of acceptable behavior on campus, widely known and scrupulously enforced. However, by far the most important ingredient of character development is the daily behavior—and, importantly, the reasons for that behavior—of the older persons on the campus: faculty, administrators, staff, and even (or perhaps even especially) students who are more senior than their peers in age or status. When these older persons on campus embody good character, they can exert a beneficent effect on students. Conversely, if bad character is not recognized as such, and if it is not dealt with firmly, then the students cannot know, and therefore cannot appreciate the differences between exemplary, acceptable, and deplorable character. • Curriculum: As noted, we believe that a central mission of every educational institution should be the creation and presentation of a high quality curriculum. We also believe that the creation and implementation of a curriculum should be the prerogative of the faculty—and particularly those faculty members who have a long-time association (and hence greater familiarity and identification) with the college or university. Parts of the curriculum will presumably resemble the offerings of secondary school, particularly those secondary schools that explicitly prepare students for higher education; parts of the curriculum will presumably resemble the offerings of professional education, which will likely occur in conjunction with or after the completion of non-vocational higher education. As we have argued in our earlier PLATO paper (Gardner 2018a), the curriculum should be fashioned with an eye toward those cognitive capacities that emerge (or at least flourish) toward the end of adolescence. In that vein, we urge that all students take at least one course that involves philosophical questions and modes of argumentation (Gardner 2018b); a course that highlights the various ways in which, and the media by which, individuals communicate with one another (this field of study is frequently called semiotics; see Klein et al. in this volume; for descriptions of various languages, media, and codes, see Wittum et al. 2018); courses that foreground interdisciplinary issues and models

2 Towards Quality Higher Education: Barriers and Enablers

13

of thinking (Meyer et al. 2018); and, relatedly, courses that call on syntheses of different types and kinds of knowledge (Ahrweiler 2017). Particularly in the era of “fake news” and “truthiness,” we need to pay special attention to learning that is better described as mislearning (Gardner 2011; Hartig and Goldhammer 2018; Maurer et al. in this volume) or negative learning (Zlatkin-Troitschanskaia et al. 2018b).

Barriers to the Achievement of Goals When we began our study of higher education in 2012, we were certainly cognizant of the many challenges that faced higher education in the United States—indeed, those challenges constituted a primary reason for undertaking this study. However, only as the study proceeded, did we become aware of the nature and potency of these challenges and, as a result, of the necessity to address these challenges as expeditiously and effectively as possible. In what follows, based on the findings from our study, we identify two challenges that we consider internal—challenges that, insofar as possible, arise and can (and should) be addressed on campus. As a complement, we identify two challenges that we deem external—challenges that will require mobilization beyond the walls, and outside the greens of individual campuses. In the cases of both internal and external challenges, we also offer suggestions about how these challenges might be approached. Internal Challenge “Mental Health and Well-Being”: As our study progressed, we were struck most powerfully by the incidence and the significance of mental health challenges across the range of campuses. On every campus that we studied, mental health was identified as the biggest problem. This identification occurred both with respect to open-ended questions “what are the biggest problems on campus?” and to a rank order request “please rank order five problems on campus” (the options were academic dishonesty, mental health, relationships with peers, substance abuse, and safety). Interestingly, though perhaps not surprisingly, those individuals who spend considerable time on campus (students, faculty, administrators) were more struck by the mental health issues than were those who are not currently on campus (parents, alumni/ae, trustees). These sheer quantitative results raise numerous questions, only some of which have been adequately investigated by researchers. How different is this pattern from earlier times? Are the mental health conditions, symptoms, and complaints that are voiced the same or different from periods in the past? If different, is that because individuals are more likely to seek help, or because more help is available, or because there is less stigma attached to seeking help? Do all individuals mean the same thing by “mental health problems”? (For example, does “stress” or “depression” denote similar or distinctive conditions? Is stress as a result of failure to gain admission to a club comparable to stress as a consequence of family or financial

14

H. E. Gardner and W. Fischman

hardship?), and, most hauntingly, are there many individuals in college now who would never have qualified, or even survived, in an earlier era? While we are intrigued by these questions, the fact remains that mental health problems are widespread, and that they need to be addressed as adequately as possible in as timely a fashion as possible, if well-being is achieved. For if a significant proportion of students are suffering from stress, anxiety, depression, or some other mental health symptomatology, they will not be able to benefit from the educational purposes for which colleges have presumably been instituted (see also Dormann et al. 2018). Needless to say, institutions of higher education have taken various approaches to dealing with mental health issues—and while the financial resources of the institution are clearly one factor, they are by no means the only one. Of those that we have encountered, we identify Peer Counselors at Kenyon College (Ohio) as especially promising. Formed in 2012, this network directly addresses students who need help, but are too nervous or afraid to seek it. Some students may not know where to go, others feel that walking into a mental health center will become a stigma—a mark of weakness, a sign of illness. While some institutions have a limited number of counseling sessions for students, Kenyon College offers an unlimited number of sessions, and yet some students are still reportedly reluctant to walk into the center. Accordingly, the College has instituted a program of Peer Counselors. These fellow students first take an oath of confidentiality, then go through training, and are finally empowered to work both informally (casual conversations around campus) and formally (student-led discussion groups in residence halls). This procedure is effective in breaking the culture of silence (Jurney 2012) by letting students know that it is okay and wise to ask for help (Thompson 2012). In addition, Kenyon Peer Counselors strive to create awareness of and connection to other resources so that students can find the help they need. Similarly, California State University, Northridge (CSUN) also utilizes students to “spread the word” about the prevalence of mental health issues on campus and the resources available to students. As this large university does not have sufficient staff to meet the demand for counseling sessions, the campus enlists support of students, and also of faculty and administrators. For example, through presentations in classes or at campus organizations, student peer educators help to educate students about various support groups and educational networks (e.g., ones that address eating disorders, depression and suicide, rape prevention). Students are also trained to work on “listening lines,” in which peer counselors listen to students’ struggles, and then make appropriate referrals. In addition to student support at CSUN, faculty and administrator are also encouraged to help students feel more comfortable sharing problems and anxieties. For instance, a campus-wide survey indicated that students did not feel a sense of community at CSUN. Accordingly, faculty and administrators were given a variety of stickers to display, each with particular words and phrases that describe certain personal traits—such as “I am a first generation student,” “I failed my first college exam,” “I identify as transgender.” The hope is that students will realize that even

2 Towards Quality Higher Education: Barriers and Enablers

15

faculty and administrators share vulnerabilities, and that these more senior individuals are available to talk about their own struggles and how they dealt with them. Internal Challenge “Loneliness, Alienation, Failure to Belong”: Clearly related to, but different from sheer mental health problems, are the widely reported feelings of alienation, estrangement, isolation, or “anomie” on the part of many students (as well as some faculty and some members of other constituents). Viewed externally, particularly as portrayed by the media, and also by the brochures and videos displayed at individual campuses, college is supposed to be a period of fun, enjoyment, pleasure, and indeed, among students whom we have termed “inertial” or “transactional,” partying and playing around are seen as the principal reasons for going to college. Yet, directly counter to that portrayed idyllic state, significant proportions of the college populace report that they feel they do not “belong” at the school in which they have enrolled. Again, the reasons for this “anomie” can be varied as are the proposed explanations: the rise of social media, which highlight the pleasures that others are presumably having (FOMO—fear of missing out—has become a powerful cry for help), the preoccupation of faculty members with their own professional careers to the detriment of initiating and maintaining meaningful relations with students, and the greater number of the so-called first gen (first generation) students, often from impoverished and/or minority backgrounds, who lack familiarity with the norms, expectations, and resources of college life today. The campus population is no longer restricted to prep school students enrobed in prep school clothing, and to “Mr. Chips” professors who have resided for their entire lives on campus and spend many hours each week interacting with students. Colleges can and should deal directly with these feelings of alienation—whether such a lack of belonging derives from a felt distance from fellow students, from the course of study, or from the institutional mission writ large. We have identified several approaches that seem promising in that regard. In each case, these approaches should be launched as early and explicitly as possible, continue throughout the college experience, and constitute an important part of the explicit and implicit (hidden curriculum) of the institution (see section “The “Three Cs” of Quality Higher Education”; also Gardner 2018a). As one example, the “La Verne Experience” at the University of La Verne (California) is a four-year curriculum designed to build community both within and beyond campus and to encourage civic engagement (Fischman 2018). The first year of the program (First Year Learning Experience) is the flagship initiative. From the first day of orientation, as students have just arrived on campus, they are grouped in small cohorts for the year (30 students and three faculty members). These “learning communities” consist of three academic classes as well as a service work component. This first year program helps each student identify not only as a member of a particular learning community, but also as a La Verne student, carrying out the university’s core values of civic responsibility and care for others in need. Though the curriculum in each of the succeeding years shifts in focus, the first year program helps students to develop long-lasting connections to fellow students, to the professors in the learning community, and to the institutional mission.

16

H. E. Gardner and W. Fischman

A second example is the Second-year Transformational Experience Program (STEP) at The Ohio State University. This program is specifically designed to foster connections between students and professors in the second year of college. In view of the well-documented “sophomore slump,” the University brought faculty and administrators together in a series of focus groups to discuss how to help students navigate the myriad of available resources. The resulting STEP program aims to bring students and faculty together in conversation to talk about things that matter outside of the classroom (e.g., personal finances, residential life, aspirations, challenges). The program has three primary goals: (1) to create time and space for students and faculty to connect with one another, (2) to inform students about how the school’s various offerings (e.g., research positions, study abroad programs, tutoring center) can help them achieve their goals, and (3) to remind faculty that helping students is their primary responsibility. Faculty members voluntarily sign up to be a “mentor” to students, committing to spending quality time with students; some faculty elect unstructured one-on-one meetings, other faculty may prefer structured small group discussions. As a secondary benefit of STEP, some faculty members report that participation in the program helps them to connect with other faculty across the university, whom they may have never had the chance to meet. As one more example, at Queens College (New York), the Center for Ethnic, Racial, and Religious Understanding (CERRU) focuses on encouraging dialogue between and among students from different backgrounds—including Christian, Muslim, and Jewish groups—those who often “silo” on college campuses. Among many activities, the Center is well-known for its training of Fellows—students who take on the responsibility of forging connections between diverse student groups by encouraging in-person conversation, often involving contentious topics, such as the religious and political conflict between Palestine and Israel. Fellows become ambassadors for the school community by facilitating book club meetings and lunchtime events. CERRU is well-known by students at Queens as the organization that brings students together, even those who, on the surface, do not appear to have values and interests in common. The program aims to help all students understand that they belong to Queens College, as well as the larger New York and American community, through work at elementary schools, religious institutions (e.g., church, temple), and other local colleges. While the La Verne and Ohio State programs may be particularly appropriate for American students in residential colleges, the CERRU program—which takes place on a commuter campus—seems relevant for institutions of higher education around the world. The twin challenges of mental health problems and feelings of alienation should constitute the highest priority issues for all institutions of higher education. Not only are these situations painful for the individuals involved (be they students, faculty, or administrators). Of equal importance, the colleges and universities are unlikely to be able to achieve, or even to address, their primary academic mission unless the members of their community feel reasonably healthy and can become meaningfully engaged with others in the community.

2 Towards Quality Higher Education: Barriers and Enablers

17

External Challenge “Cost of Education”: Compared to many commodities and services in modern post-industrial society, the costs of higher education have risen and continue to rise significantly. In contrast, for much of American (and also Western) society, income has remained relatively stagnant. While authorities differ on whether the costs of higher education constitute a crisis, there is no doubt that many individuals and constituencies question the reasons for and the need for a higher education which seems expensive or even overly or prohibitively expensive. Some qualifications should be stated. First of all, private education is far more expensive than public education (Gallup-Purdue 2015). (The average tuition at a 4 year public institution is $17,000, and the average tuition at a 4 year private institution is $38,589; College Board 2011.) The small number of well-endowed private colleges and universities in the United States can provide scholarship/ fellowship support that the far larger number of less endowed institutions cannot possibly provide. Moreover, in most cases, the so-called sticker price differs from what higher education actually costs (Konrad 2011), and, most importantly, the support from the state for public education has declined steadily in recent decades with no signs of any reversal of this trend (Gallup-Purdue, 2015). Pundits have put it succinctly, “education used to be seen as a public good; now it is seen as a private good” (Baum and McPherson 2011). Indeed prestigious large state universities currently receive less than 10% of their operating budget from the state. If they could legally become private universities, these comparatively wealthy “big publics” might choose to do so; but often legal or financial barriers prohibit this course of action (Bacow 2017). Without doubt, steps can be taken to make the cost of college less onerous. Institutions can drop expensive athletic programs and decline to build over-the-top facilities that are costly to construct and then costly to maintain. They can share facilities and even faculty members (and disciplinary programs) with one another. College could be shortened in length—say, to 3 years (with or without summer courses). Online education could become an effective supplement or replacement. (We do not endorse certain other cost-saving devices, like eliminating tenure or increasing the already high percentage of adjunct faculty.) One idea with promise is the commitment of future alums to “give back” to the school in proportion to the wealth that they will subsequently accumulate. A perennially popular remedy is for the government (local, state, federal) to assume much of the cost of higher education—as is done in most of the developed world. Indeed, dating back to the immediate post WW II period, President Harry Truman’s commission (1947) proposed that community colleges (often then called junior colleges) be free to students. This remedy does not seem feasible in the United States at present, and indeed, if the example of Britain is any indication, other developed countries may also begin to charge for higher education—and to increase the charges over time. External Challenge “Antipathy to Higher Education”: In the last few years, public opinion polling in the United States has documented a most disturbing trend: a significant proportion of the population (and a majority of those who describe

18

H. E. Gardner and W. Fischman

themselves as members of the Republican Party) question the value of higher education (Pew Research Center 2017). Yet again, this finding needs to be unpacked. First off, there is questioning of whether higher education yields an adequate ROI—return on investment. Nearly all indices concur that, until now, higher education has been a good financial investment; however, it is significant that a growing proportion of the population does not believe that is the case (Pew Research Center 2017). Trends in the future may support this growing public perception: many more jobs will be automated or handled by artificial intelligence, and those jobs that remain whether high status (computer programmer) or craft like (plumber, electrician) may not require a traditional college degree. Far more discomforting is the contention that higher education—and particularly education in the liberal arts and sciences—may be bad for the nation (Pew Research Center 2017). To be sure, some of these data are due to ignorance. Even in the target population of our study, many individuals do not know the meaning of the expression “liberal arts”—and some have not even heard the phrase! In an amazingly telling finding, matched groups of individuals rate a curriculum lower when the phrase “liberal arts” is used explicitly, than when exactly the same curriculum is presented without the telltale phrase being uttered (Busteed 2017). The apparent—and perhaps growing—antipathy toward higher education may reflect another, even more insidious state of affairs. It may be that those in the population who lack higher education—or even lack contact with those who have higher education—may feel ignored or condescended to by those who have spent 4 years or more in an institution of higher education. Presumably, this is what then candidate Donald Trump meant when he declared at one of his rallies “I love the poorly educated” and when he ridiculed candidate Hillary Clinton’s references to “the deplorables.” It may also be the case that our institutions of higher education— whose faculty as a whole lean leftwards—encourage ways of thinking and acting that alienate them from that portion of the population who have not been exposed to those “liberal arts and sciences” ways of thinking (Mehler et al. 2017). Clearly, this antipathy within the populace is deeply injurious to the status—and, indeed, to the plight—of institutions of higher education in the United States, and perhaps elsewhere as well. It is not helped when political leaders and other influential individuals who should know better embrace anti-intellectual attitudes. (Even President Obama was not immune from this trend when, during his presidency, he ridiculed a degree in art history—fortunately, he was rebuked for this thoughtless remark and apologized for it; Jaschik 2014; Madhani 2014).

Conclusion In the end, as is the case with other institutions that have lost public respect in recent decades, we cannot simply look to others to counter critiques of higher education. To paraphrase a famous injunction borrowed from Greek culture and from the Bible:

2 Towards Quality Higher Education: Barriers and Enablers

19

“Institutions of higher education—heal thy-selves.” If we are to gain, or regain, the financial and attitudinal support that institutions of higher education need, we must put our own epistemological, cultural, and financial houses in order. To the extent that the intrinsic and extrinsic challenges are related—for instance, if high costs cause psychological stress—we must attempt to ameliorate them. Above all, we must exemplify what we claim that we stand for—in terms of context, character, and curriculum—as enablers of top-quality higher education. As best we can, we must address the challenges outlined here—as well as those additional ones that may arise going forward, and we must make our actions known—making sure that we do so in a way that embodies our values, rather than cheapens or even undermines them. Here, we encounter the principal reasons that the PLATO project is so important. It is a scholarly endeavor, involving scholars of several disciplines working together or in tandem, and making findings known in appropriate formats. We should not pretend to be something other than we are. Yet at the same time, we must always keep in the forefront the indispensable roles that we play in creating new knowledge that may be of use to the wider society and in communicating that knowledge as clearly and as widely as possible. Perhaps, indeed, the examples of positive learning being developed and probed can be used across the entire population, so that bridges can be built between scholars and their institutions, on the one hand, and the wider world, on the other.

References3 Ahrweiler, P. (2017). Agent-based simulation for science, technology, and innovation policy. Scientometrics, 110(1), 391–415. https://doi.org/10.1007/s11192-016-2105-0 Bacow, L. S. (2017). The political economy of cost control on a university campus. Lecture presented at Center for Studies on Higher Education in Berkeley, CA. https://www.uctv.tv/ shows/The%20Political-Economy-of-Cost-Control-on-a-University-Campus%2032375 Baum, S., & McPherson, M. (2011). Is education a public good or a private good? The Chronicle of Higher Education. https://www.chronicle.com/blogs/innovations/is-education-a-public-goodor-a-private-good/28329 Busteed, B. (2017). Higher education: Drop the term “liberal arts”. Washington, DC: Gallup. https://news.gallup.com/opinion/gallup/216275/higher-education-drop-term-liberal-arts.aspx College Board Advocacy & Policy Center. (2011). Trends in College Pricing 2011 (Trends in Higher Education Series). https://trends.collegeboard.org/sites/default/files/College_Pricing_ 2011.pdf Dormann, C., Demerouti, E., & Bakker, A. (2018). A model of positive and negative learning. In O. Zlatkin-Troitschanskaia, G. Wittum, & A. Dengel (Eds.), Positive learning in the age of information – A blessing or a curse? (pp. 315–346). Wiesbaden: Springer. Fischman, W. (2018). Good lives: Students working together to build community. Howard Gardner. https://howardgardner.com/2018/07/23/the-good-life-integration-of-academics-andcivic-engagement/

3

We are grateful for Sophie Blumert’s assistance with these references.

20

H. E. Gardner and W. Fischman

Gallup & Purdue University. (2015). Great jobs, great lives: The relationship between student debt, experiences and perceptions of college worth (Gallup-Purdue Index Report 2015). https://www. wfyi.org/files/wfyi/files/gpi-report-2015-09-25-2015.pdf Gardner, H. (2011). Truth, beauty, and goodness reframed: Education for the virtues in the age of truthiness and twitter. New York: Basic Books. Gardner, H. (2018a). Higher education: A platonic ideal). In O. Zlatkin-Troitschanskaia, G. Wittum, & A. Dengel (Eds.), Positive learning in the age of information – A blessing or a curse? (pp. 9–21). Wiesbaden: Springer. Gardner, H. (2018b). Why we should require all students to take two philosophy courses. The Chronicle of Higher Education. https://www.chronicle.com/article/Why-We-Should-RequireAll/243871 Hartig, J., & Goldhammer, F. (2018). Digital information literacy as prerequisite for positive learning. Presented at the International PLATO Conference, Johannes Gutenberg-University Mainz. Jaschik, S. (2014). Obama vs. art history. Inside Higher Ed. https://www.insidehighered.com/news/ 2014/01/31/obama-becomes-latest-politician-criticize-liberal-arts-discipline Jurney, T. (2012). Counseling center needs to expand. The Kenyon Collegian. https://www. facebook.com/KenyonCollegePeerCounselors/photos/a.794995323873263/ 183343133321805/?type¼3&theater Konrad, M. (2011). Uncover the real costs of public and private colleges. U.S. News. https://www. usnews.com/education/blogs/the-scholarship-coach/2011/10/27/uncover-the-real-costs-of-pub lic-and-private-colleges Madhani, A. (2014). Obama apologizes for joking about art history. USA Today. https://www. usatoday.com/story/theoval/2014/02/19/obama-apologizes-to-texas-art-history-professor/5609089/ Mehler, A., Uslu, T., Hemati, W., Flöck, F., & Lücking, A. (2017). Language of the “elite” – How social dynamics imprints on knowledge resources. Presented at the International Summer Conference PLATO (Positive Learning in the Age of Information), Helmholtz Institute, Johannes Gutenberg-University Mainz. Meyer, O., Imhof, M., Coyle, D., & Banerjee, M. (2018). Positive Learning and Pluriliteracies). In O. Zlatkin-Troitschanskaia, G. Wittum, & A. Dengel (Eds.), Positive learning in the age of information: A blessing or a curse? (pp. 235–265). Wiesbaden: Springer. Minnameier, G. (2018). Reconciling Morality and Rationality. In O. Zlatkin-Troitschanskaia, G. Wittum, & A. Dengel (Eds.), Positive learning in the age of information: A blessing or a curse? (pp. 347–361). Wiesbaden: Springer. Pew Research Center. (2017). Sharp partisan divisions in views of national institutions: Republicans increasingly say colleges have negative impact on U.S. Pew Research Center. http://assets.pewresearch.org/wp-content/uploads/sites/5/2017/07/11101505/07-10-17-Institutionsrelease.pdf Thompson, M. (2012). Asking for help: Students form peer counseling service. The Kenyon Collegian. https://issuu.com/kenyoncollegian/docs/kenyoncollegian9-6-12/5 Truman, H. S. (1947). The president’s commission higher education for democracy. The American Presidency Project. http://www.presidency.ucsb.edu/ws/index.php?pid¼12802 Weber, M. (1922/1946). Science as a vocation. In H. H. Gerth & C. W. Mills (Eds.), From max weber: Essays in sociology (pp. 129–156). New York: Oxford University Press. Wittum, G., Jabs, R., Hoffer, M., Nägel, A., Bisang, W., & Zlatkin-Troitschanskaia, O. (2018). A concept for quantitative comparison of mathematical and natural language and its possible effect on learning. In O. Zlatkin-Troitschanskaia, G. Wittum, & A. Dengel (Eds.), Positive learning in the age of information: A blessing or a curse? (pp. 109–126). Wiesbaden: Springer. Zlatkin-Troitschanskaia, O., Wittum, G., & Dengel, A. (2018a). Editorial – About a ‘PLATO’. In O. Zlatkin-Troitschanskaia, G. Wittum, & A. Dengel (Eds.), Positive learning in the age of information: A blessing or a curse? (pp. 1–6). Wiesbaden: Springer. Zlatkin-Troitschanskaia, O., Schmidt, S., Molerov, D., Shavelson, R. J., & Berliner, D. (2018b). Conceptual fundamentals for a theoretical and empirical framework of positive learning. In O. Zlatkin-Troitschanskaia, G. Wittum, & A. Dengel (Eds.), Positive learning in the age of information: A blessing or a curse? (pp. 29–50). Wiesbaden: Springer.

Chapter 3

The Role of Modeling for “Seeking Truth” in an Educational Policy Classroom David C. Berliner

Introduction No less a person than the author of our declaration of independence and our third president, Thomas Jefferson, warned Americans about coming to Europe. He believed that Europe was corrupting. Writing at about that same time, Abbot Konrad Tanner of the Swiss Confederation identified what it was that might be corrupted (quoted in Trohler 2011): It may be good for [the young man] to visit foreign countries, but only after having been adopted to the way of living in his fatherland, after having absorbed the good principles with his mother’s milk, after being enabled by the domestic education to know himself and the world.

I think that both President Jefferson and Abbot Tanner worried less about wild woman, gambling, and drink as the corrupting influences on the young gentlemen who travelled in those days than they did about the challenge that travel provides to one’s social beliefs. Social beliefs—the kinds that have no easily determined “right answers” are hard to prove or falsify. Included among these are stereotypes. Each nation wants its versions of its social truths to be deep, deep, in the psyches of its young. As the Abbott notes, these are ways of thinking that are absorbed with mother’s milk! These beliefs, ideas, prejudices, and biases are often what makes for social cohesion in a community. Social cohesion for most citizens of a country is enhanced if most believe that Protestants think this way, Jews are like that, Catholics believe such and such, business men act only to maximize profits, factory workers are expendable, voting should be for men only, our leaders will not lie to us, and so forth. Social “truths” D. C. Berliner (*) Arizona State University, Tempe, AZ, USA e-mail: [email protected] © Springer Nature Switzerland AG 2019 O. Zlatkin-Troitschanskaia (ed.), Frontiers and Advances in Positive Learning in the Age of InformaTiOn (PLATO), https://doi.org/10.1007/978-3-030-26578-6_3

21

22

D. C. Berliner

like these—often stereotypes about others—are buried deep in the psyche’s of one’s own social group. It is a challenge to these communal beliefs that Jefferson feared. Obviously, some of these deeply buried beliefs about our social world are the roots of some unethical and immoral action, sometimes leading to cruel, inhuman, or degrading historical atrocities such as the holocausts in Germany and Rwanda, or the treatment of indigenous people in Australia and the USA. In the same way, but on the positive side, without thinking deeply, we often act ethically and morally because we hold, deep in our psyche’s, a set of shared beliefs about what is right and wrong. Much of what we learn in these areas, when young, are modeled for us. That is, many of the beliefs we hold are not explicitly taught. These might include how to treat strangers; the sharing of food with those who are hungry; watching out for the safety of infants; how to be a friend to another human; the offering of charity. Or, as easily learned from models, are such acts as the cruel treatment of pets; the dislike of those from another land or with another skin color; or avoidance of those with disabilities. These learned behaviors are rarely taught, though often modeled. Modeling is universally acknowledged as a powerful way to transmit ways of doing things, for example, the behaviors needed to change a tire on an automobile or on a bicycle. Less well recognized is that modeling is equally effective in teaching another person particular ways of thinking about, and making sense of, the world in which they live. The renowned psychologist Albert Bandura did his classic experiments on the power of models to influence behavior. Bandura showed how easy it was to get young children in a nursery school setting, though models, to either hug or beat up a big inflatable doll (Bandura 1986, 2008). Love and hate toward others, it seems, are easily learned by those who view displays of love and hate by prestigious others. Explicit teaching was not necessary to teach those behaviors. I was reminded about the power of modeling to influence how we think and act as I began to think more about the research completed, and the goals of the PLATO project (for an overview of PLATO, see Zlatkin-Troitschanskaia et al. 2018).

Role of Higher Education It is only a very self-aware person who realizes as a teenager, or adult, the damage that may have been done to their minds by absorbing ideas that appear to be clearly true, but which are not! Certain ideas appear to be unambiguously true because everyone of importance to that person [their parents, teachers, and political leaders] holds those ideas to be true, and those people who are important to them act on those perceived truths. That is why Jefferson and Tanner, among many others, saw foreign travel as something to be avoided, as it could well challenge the social truths—the memes, if you will—that are prevalent in, and often prop up a particular society. Travel provides new models. Travel sheds light on what is right and wrong; what is worthy of belief or disbelief. A simple example is that of driving patterns. Some

3 The Role of Modeling for “Seeking Truth” in an Educational Policy Classroom

23

people believe that other people drive on the “wrong” side of the road. But, of course, there really is no “right” side of the road. Yet, “they” are wrong and “we are right.” Nowhere did the confrontation of ideas about how people could and should live show up more clearly than when US African-American soldiers went to Germany in and after World War II. They discovered a far different set of social norms than they experienced at home. It is been acknowledged that at least some of the roots of America’s civil rights movement were in the much more positive treatment of Black soldiers in Europe by Germans and other Europeans. Travel, then, as Jefferson and Abbot Talbot correctly opined, was indeed dangerous. Travel provokes a comparative look at one’s own nations’ ways of thinking and acting and that is the source of the danger. American black soldiers came back with a different view about how things should be. An “iron curtain” and a Berlin Wall are only Jefferson and Tanner made visible in contemporary times. Nations that used these techniques want the knowledge and beliefs that their citizens hold to go unchallenged, and they recognize that travel outside their countries could change that. Which brings me to PLATO. It is my belief that a good liberal arts education is at least equal to travel in disturbing the ideas that one comes to believe in, merely by growing up in a particular family and nation (for an overview of liberal arts education in the USA, see Gardner and Fischman in this volume). If a liberal arts education is well taught and well learned, then the arts, humanities, and sciences will challenge many of the societally implanted social truths we walk around with. The replacement of one’s familial, local, and juvenile beliefs and thoughts, with more warranted and sophisticated knowledge, may even occur unknowingly. In the language of PLATO, a good college education is intended to foster the acquisition of much more warranted, verified knowledge, while also providing methods to minimize negative learning— our beliefs in unwarranted or incorrect knowledge (for the definition of negative learning, see Zlatkin-Troitschanskaia et al. 2018). As I thought about what Zlatkin-Troitschanskaia and other colleagues connected with PLATO were teaching me, and I looked at our contemporary situation in the USA, I decided to take on more responsibility to help my university students learn how to distinguish truth from fiction. I sought to be more conscious of this part of my professorial role than I had ever been before. The former Secretary of State of the USA, Rex Tillerson captured my concerns, and I suspect the concerns of many others. After being inside President Trump’s White House for over a year, and then fired by an erratic president with difficulty telling truth from fiction, he said (Tillerson 2018): If our leaders seek to conceal the truth, or we as people become accepting of alternative realities that are no longer grounded in facts, then we as American citizens are on a pathway to relinquishing our freedom.

24

D. C. Berliner A responsibility of every American citizen to each other is to preserve and protect our freedom by recognizing what truth is and is not,. . . . what a fact is and is not, and begin by holding ourselves accountable to truthfulness and demand [that] our pursuit of America’s future be fact-based, not based on wishful thinking, not hoped-for outcomes [related to] shallow promises.

A “good” college education has always exposed students to more nuanced and critical thinking about the world they live in, providing a bit more truth about their history, their society, and their place in that society. In the USA it is in college, not usually before, that we learn about our history of government supported lynching’s of Black people; our treatment of Irish immigrants; our massacres of American Indians; the horrors of our private jail systems; our torture of prisoners of war; and so forth. The proud, free, fair, generous American, a stereotype built up through our K– 12 schooling and learned from prestigious members of our community, is interrogated throughout a proper college education. It is a college education that often provides students with a more nuanced view of American’s and American life. We learn, sadly, that we Americans are thoroughly human, meaning that we sometimes have acted and continue to act, like the beasts we were taught to believe only existed in other countries. In my country, those holding narrow stereotypes about our glorious history and the benevolent social world in which we live are more often members of the Republican party. Not surprisingly, then, that party has dramatically turned against college education: Fig. 3.1 presents these data. We see in these data that at all ages, for all educational levels, and at all income levels, Republicans are far less likely than Democrats to believe that universities and colleges have a positive effect on our society. The Republican party, considered the more conservative of the two major political parties in the USA, dislikes our institutions of higher education and the substance of what we in the universities profess. Perhaps we in the university community should pat ourselves on the back a little, as we seem to be accomplishing some of the goals that academics have held for the university experience for a few hundred years! Universities deliberately hope to challenge some of what our students believed to be true before they first entered our institutions of higher education. We do our work so that our students might obtain a bit of enlightenment. In the PLATO lexicon, we promote positive learning and challenge the negative learning students have acquired from their homes, neighborhoods, and from their previous schooling. If our universities do their job correctly in my country, our college graduates will be more likely to hold a more nuanced and less self-congratulatory view of what it means to be an American. It is not unpatriotic to acknowledge mistakes, for how else might a nation guard against ever repeating them? Perhaps this fear of more positive learning is why the Republican controlled legislature in my state of Arizona have voted recently to spend zero dollars on the 300,000 students in our states’ community college system (Smith 2017). My states’ legislature also cut the funding of the states’ three universities to pre-2008 levels

3 The Role of Modeling for “Seeking Truth” in an Educational Policy Classroom Fig. 3.1 The perceived value of a college education by political affiliation in the USA (Pew Research Center 2017)

25

Across educational groups, Republicans give colleges & universities low ratings % who say colleges and universities have a positive effect on the way things are going in the country Dem/Lean Dem

Rep/Lean Rep

36

Total

72

52

18-29 39

30-49 50-64 65+ Postgrad College graduate Some college HS or less

73 72

29

72

27

72 35

82

32

76

37

69

37

68

Family income $75,000 or more $30,000-$74,999

31

79

34

Less than $30,000

75 46

66

Source: Survey conducted June 8-18, 2017. PEW RESEARCH CENTER

(Cano 2017). As I think about this, I have come to believe that my state legislature and a good many others throughout the USA are not interested in a world filled with people who know or know how to seek truth. Therefore, as a professor in these difficult times, I felt the need to increase my modeling of such truth seeking in the domains in which I teach.

Modeling Truth-Seeking Behavior In The Manufactured Crisis: Myths, Fraud, and the Attack on America’s Public Schools (Berliner and Biddle 1995), and in 50 myths and lies that threaten America’s schools (Berliner and Glass 2014), I attempted to help the educational profession and the general public to understand what is more or less likely to be true. Then my

26

D. C. Berliner

involvement with PLATO made me more concerned for modeling, not just writing about what truth seeking is like. I found that it is neither hard nor very time consuming, given access to the web, to model truth seeking behavior. After all, while the web can be a great deluder, it is at the same time the world’s best and fastest fact checker (Lynch 2016). I believe, as well, that modeling truth seeking behaviors makes me a more influential teacher of my students. I begin many of my classes with an examination of a current news report related to educational policy, with an emphasis on what is said by my President and his Secretary of Education. I then model what a search for truth, or for warrant, is like. My model is the fact checking done by highly regarded newspapers and magazines. Research on the impact of fact checking is sparse, but it appears that fact checking does correct misperceptions among citizens, and it certainly discourages politicians from spreading misinformation (e.g., Ciampaglia 2018). Not long ago, in a memo of September 25, 2017, concerned with increasing access to high quality STEM (Science, Technology, Engineering, and Mathematics) coursework, President Trump (2017) wrote: Today, too many of our Nation’s K-12 and post-secondary students lack access to highquality STEM education, and thus are at risk of being shut out from some of the most attractive job options in the growing United States economy. Courses in Computer Science are especially scarce in too many schools and communities, despite the job opportunities that these skills create. Nearly 40 percent of high schools do not offer physics and 60 percent of high schools do not offer computer programming.

The president is promoting STEM education and he worries as 40% of America’s high schools apparently do not offer physics courses. I fact checked and shared with my students what that the non-partisan American Institute for Physics had to say about physics course offerings in the USA (White and Tyler 2015) (Fig. 3.2). This credible source of data informs us that about 98% of the students whose parents are in the top third of the income distribution have access to physics courses every year. Moreover, about 95% of the students of middle-income parents have similar rates of access to physics courses. And, over 90% of the students from families in the lowest third of the income distribution have regular access to physics courses. The president and his staff appear to be quite wrong about access to physics courses. It seems clear that our current president and the president before him, often at the urging of America’s business community, have pushed vigorously for more STEM education and for more computer-related classes in all our schools. And perhaps our presidents and their supporters are right, since curriculum choices are always value choices. On this issue the current president is expressing his values quite clearly. To help my students understand the complexity of educational policy, I then ask them this: How is it that the president and his aides did not provide our educational research community with any guidance about what should be thrown out of the curriculum, so that we can put into the curriculum the recommended coding or computer science classes? Should we throw out music, or art, or physical education? Or maybe government? Or, should we reduce reading and language arts time? Or,

3 The Role of Modeling for “Seeking Truth” in an Educational Policy Classroom

27

How Often Physics is Offered By Socioeconomic Status* US Public High Schools Only, 2012-13 % of seniors attending a school where physics is available ...

Socioeconomic status

0%

20%

40%

60%

80%

100%

Better Off

Average

Worse Off

Every Year

Every Other Year

Never

* Principal/teacher assessment of student economic circumstances relative to those of students at other schools in the local area.

Fig. 3.2 Physics course offerings by student socioeconomic status

will the President and Secretary of Education tell us to keep all those courses and then add time to the school day for computer courses, and then offer a few billion dollars more for our nations’ schools to cover the increased educational costs of adding computer coding and/or computer science to the high school curriculum? I hope that my students come to realize that the President and the Secretary are not at all well versed enough in curriculum issues to be offering the educational community this kind of advice. My concerns about the president’s letter to the Secretary went even deeper and so I looked into that literature. Lohr (2017), science writer for the New York Times, charted data from the US Bureau of Labor Statistics to examine my countries need for STEM workers. As can be seen in Fig. 3.3, the STEM shortage seems to be imaginary. In life science (excluding medical personnel), we are producing 183,000 graduates for 12,000 jobs per year. In engineering, we are producing 169,000 graduates for 51,000 jobs per year; in the physical sciences and mathematics, the same situation holds, and only in computing is there is some match between degrees obtained and jobs available. Because of so many lies, by so many people, we get the situation described in Fig. 3.4. Forty-eight percent of STEM degree holders do not work in STEM fields (Graf et al. 2018). Their degrees in science and technology may help them do other jobs quite well, but they did not end up in a STEM field. This suggests that the whole shortage issue is a myth! But Because I teach an educational policy class, I point out that the overproduction of indigenous American technology job seekers is not the only the problem of contemporary training and employment: Our STEM graduates

28

D. C. Berliner

Life Sciences* 183k 12k Engineering 169k 51k Physical Sciences 43k 9k Mathematical Sciences 33k

So Many Degrees, So Little Demand The number of graduates with technical majors (shown: bachelor, master and Ph.D. degrees awarded in 2015-16) tends to outpace job openings (shown: 2014-24 projections, annualized). Computer science is the exception.

7k Computer Science 107K 108K *Does not include health care occupations. Bureau of Labor Statistics, National Center for Education Statistics

Fig. 3.3 Relationship of degrees earned to jobs available, from 2014 to 2024 projections, annualized (Lohr 2017)

have to compete with approximately 50- to 80,000 H-1B visa permits that are issued each year to foreign STEM personnel so they can work in the USA. The current STEM workforce now has an estimated 460,000 HI-B visa-holding immigrants on payroll, and they work for about 70% of the wages for which a domestically trained STEM worker will work. Moreover, visa-holding STEM workers are sought-after because they rarely complain about working conditions, fearing a loss of their job and being sent back to their native countries. I usually astound my class and everyone else to whom I present these data. But it is easy to obtain and to verify. My goal is to model fact checking. I want to instill in my students the understanding that uncritical belief in our industrialists, our government officials, and even our fellow educators, including me, is not warranted without fact checking. Positive learning—holding warranted knowledge—requires a considerable amount of effort. I also try to remind my students that in the middle of this push toward STEM, other academic outcomes provide our country with a great deal of value. Among the other talents in school worthy of development are reading, writing, art, music, acting, dance, historical inquiry, participation in government, sports, and serving on the school newspaper and school yearbook. I present data to show that engagement in these activities is just about as good at predicting successful employment in adult life as are scores on science and math examinations.

3 The Role of Modeling for “Seeking Truth” in an Educational Policy Classroom

29

52% of STEM-trained college graduates are employed in the STEM workforce Among workers who majored in science, technology, engineering or math, % currently employed in each type of job

Management business, finance Social services, legal, education

17% STEM 11%

52%

Other nonSTEM 20%

Note: Based on employed adults ages 25 and older completing a bachelor’s degree in STEM major field of study. Source: Pew Research Center analysis of 2014-2016 American Community Survey (IPUMS). “Women and Men in STEM Often at Odds Over Workplace Equity” PEW RESEARCH CENTER

Fig. 3.4 About half of all those prepared to be STEM workers actually work as STEM workers (Graf et al. 2018)

Sometime each semester, I present my students with the following quote and ask them to identify the speaker: According to school-testing experts’ rule of thumb, the average child’s achievement score declines about 1 percent for each year they’re in school. That gives the expression dumbing down a whole new meaning. Schools may be hazardous to your child’s intellectual health.

My students and I briefly discuss this quote, and we all agree it is a ludicrous statement. Who could have said that? What might be their motive? I then reveal that the quote is from our current president, Donald Trump, from his book: The America We Deserve, published in 2000. In our class discussion of this statement we almost unanimously decide that a person might only say something this ridiculous if their agenda is to destroy America’s public education system, and if they believe their readers are not very intelligent. If even a few people who vote end up believing something this

30

D. C. Berliner

ridiculous, it changes the level of support for public schooling within a community. Thus, outrageous statements such as these can add to the numbers who want to see a decrease in funding for public schools and an increase in private and charter schooling. It may be a patently ridiculous statement, but it is quite purposely directed at an audience that is predisposed to believe such nonsense. And, if a famous person who they may think of as smart says things like this, a small group of citizens get more ammunition for their argument about the need to defund our public schools. They may then put more of their faith and dollars into charter and private schools. Other direct quotes from our president that I share on different school days are: “Our schools aren’t centers of learning, they’re centers of crime.” And “Our schools aren’t safe, which is bad enough. On top of that, our kids aren’t learning. Too many are dropping out of school and into the street life. . . Schools are crime-ridden and they don’t teach.” (Trump 2000)

It was easy to check on the truth of the president’s statements and I shared both my method and findings with my students. For example, the USA is certainly a culture where it is easy to get guns. That is why we see tragedy every few months at one school or another. But there is really a more mundane story about our schools, as provided in data from the US Department of Education (Musu-Gillette et al. 2018). While no one believes these data are as accurate as we might prefer, because schools often suppress reports of such incidents, it is quite apparent that the percent of children who have brought a gun to school has dropped dramatically over the last 25 years. School counselors and other school personnel have been addressing this issue successfully for a long time. Even one gun in a school is frightening. But such incidents among our over 50 million students in school for 180 days a year are bound to happen, as we live in a country with easy access to guns. When this topic is discussed I teach some facts that are met, at first, with disbelief. I teach that the number of school shootings in the USA is essentially unchanged from the 1980s and 1990s (Nicodemo and Petronio 2018). Because of automatic weapons, the numbers killed and wounded in school shootings have risen, making headlines throughout my nation and the world. But the frequency of school shootings has not changed much at all (Duwe 2017). School shootings are actually an infrequent event, but made more noticeable of late by the numbers involved when automatic weapons are used. The government, as it pushes for more charter and voucher schools, apparently wants to scare people from attending public schools by providing false data about school shootings. Recently the Trump Department of Education was claiming that at least one incident of a school shooting occurred in 240 schools during the 2015–2016 school year (U.S. Department of Education 2018). But the investigative reporters at National Public Radio (NPR) followed that up and the number they could verify was 11 (Kamenetz et al. 2018). The government may well be promulgating false information because, on any given day, one of our nearly 100,000 American public schools is likely to be one of the safest places to be in America. I also show my students that that rates of victimization and violent victimization have been falling for around 25 years, as have rates of theft in schools. These data are

3 The Role of Modeling for “Seeking Truth” in an Educational Policy Classroom

31

Percent 50

40 Hispanic 30 Black 20 Total 10

1

White

0 1970

1975

1980

1985

1990

1995

2000

2005

2010

2013

Year

Fig. 3.5 School dropout rates 1970–2013 (McFarland et al. 2016). 1Includes other racial/ethnic categories not separately shown. NOTE: “Status” dropouts are 16-to 24-year-olds who are not enrolled in school and who have not completed a high school program, regardless of when they left school. People who have received GED credentials are counted as high school completers. Race categories exclude persons of Hispanic ethnicity. Data are based on sample surveys of the civilian noninstitutionalized population. Source: U.S. Department of Commerce, census Bureau, Current population survey (CPS), October, 1970 thorough 2013

all available from the US government, in agencies run by the executive branch of our government, that is, the presidents’ office (Musu-Gillette et al. 2018). I share with my students a 2018 credible survey that informs us that 97.3% of students ages 12–18, when asked about thefts, violent crimes, sexual assaults, and robberies during the previous 6 months reported no such criminal victimization at school (Yanez et al. 2018). I then ask my students why the president is demonizing our public schools? What is his motive? We spend considerable time talking about what is gained by presidential and governmental lies. Our president also said that too many of our kids are dropping out of schools and into “the street life.” The president may be correct that the dropout rate is too high, since that is a value statement that can never be refuted with data. However, I found when I looked at the current data that the dropout rate is remarkably low by historical standards (McFarland et al. 2016). Despite the president’s beliefs, dropout rates have been going down over the past few decades because school personnel took the problem of school dropouts seriously. Worth noticing in Fig. 3.5 is that the dropout rates for minorities have gone down rather dramatically, a goal that the educational leaders of our public schools set for themselves many decades ago. So, once again, the president seems not to have his facts straight. This keeps me asking my students: What is his goal? What is it that saying such easily disproved statements accomplishes for the president? There must be a goal or the president would not set out to pick a fight with three million

32

D. C. Berliner

educators and the 70% of parents who think the public school their child attends is actually a “good” school (Phi Delta Kappan Poll 2018). In fact, at just over 83%, the national high school graduation rate is at an all-time high! Because of America’s teachers, administrators, and counselors, just since 2001, 2.8 million more students have graduated from high school than might otherwise have been the case. This has resulted in significant benefits for young people, for the economy, and for the nation. So why criticize, rather than applaud our educators? What is the tactical aim of such blatant untruths? I also share with my students that when I was looking at graduation rates to obtain more trustworthy data than that supplied by the president I discovered that the president and Education Secretary DeVoss both stated that the Washington DC Voucher program had a 98% graduation rate from high schools (Ujifusa 2017). I told my students that to me, such a rate was as likely as encountering a unicorn. From experience I knew it had to be a bold face lie. So, this time I asked a few of my students to do the fact checking—to see if my modeling had some power. They came back a week later with credible data to support these statements: First, the voucher program was graduating 72% of its students, not 98%, according to the National Center on Educational Statistics (McFarland et al. 2018). Second, the graduates of the voucher schools were less likely than a matched sample of public school students to go to college within 2 years. Third, almost all of Washington DC’s public schools lied about graduation rates, falsifying the records of attendance and achievement for hundreds or thousands of students (Stein 2018a, b). The cause seemed to be pressure by administrators to give out high school degrees to as many students as they could, including some who came nowhere close to deserving them. I knew that if either the President or Secretary knew anything about schooling in America, they would have known that what they said was untrue. But, my class and I concluded, sadly, that perhaps they didn’t care at all about the truth. In another statement by our president, he said that those who do stay in school are not learning, and that their teachers are not teaching. Once again, I asked some of my students to examine this statement by sending them to the National Assessment of Educational Progress, our NAEP tests, rightly called the nations’ report card. The longitudinal NAEP tests, given every 4 years, plot the growth in student academic achievement on the conventional curriculum chosen for America’s youth. What my students easily found, as I knew they would, is presented in Table 3.1 (National Center for Education Statistics 2013). It seems impossible to claim that our public school students are not learning when we have evidence that reading and mathematics scores for fourth and eighth graders are up substantially, over the course of a good many years, on the best assessments we have to monitor growth in achievement. Further, as can be seen, these scores are up a lot more for Black and Hispanic students than they are for White students, showing that America’s educators took quite seriously the need to reduce the racial, ethnic, and income gaps in school achievement that hurt our nation. The data in Table 3.1 also refute another charge made by the president. That is, if our students’ scores are going up over the decades, as they are, then the president’

3 The Role of Modeling for “Seeking Truth” in an Educational Policy Classroom

33

Table 3.1 Improvement in NAEP reading and mathematics scores from 1970s to 2012 NA EP long term trends, by subject and by student age Mathematics age 9

Mathematics age 13

Reading age 9

Reading age 13

Race/ethnicity White students Black students Hispanic students White students Black students Hispanic students White studente Black students Hispanic students White students Black students Hispanic students

Change in score from 1971 to 2012 in reading and from 1973 to 2012 for mathematics 27 36 32 19 36 32 15 36 25 9 24 17

From: Trends in academic progress in reading 1971–2012|, and in mathematics 1973–2012.

charge of there being no teaching (to accompany his charge of no learning) is also likely to be baseless. The president is not the only one to make unsubstantiated comments about achievement in education. A few years ago, one of the world’s wealthiest individuals, Bill Gates, said: Over the last four decades, the per-student cost of running our K-12 schools has more than doubled, while our student achievement has remained flat (Gates 2011).

Table 3.1 demonstrates that Mr. Gates is absolutely wrong about test scores being flat. His other charge, that schooling now costs more, is true. I share with my students the fact that we went from educating roughly 3.6 million special education students per year in the late 1970s, to educating about 6.5 million special education students per year in the last few years (Institute for Education Sciences, National Center for Education Statistics 2016). This is over a 70% increase in the recipients of very costly school programs. Many of these educational interventions cost 50 and 60 thousand dollars, per child, per year. Further, technology costs are up dramatically in the years about which Mr. Gates is concerned. He needs to only look at what his company charges for their products over the years he is concerned about! So, we discuss how Mr. Gates was wrong on achievement, and how he is not thinking clearly about the costs of education. Then I ask a values question, which is always a part of policy analysis, if they believe that the richest nation in the world can afford this kind of commitment to America’s most vulnerable—our special needs population and their families? And should the richest country in the world invest in classrooms and schools that use the latest in technology, often invented here in the USA? My students are left to ponder the values inherent in these policy issues because fiscal data is not the only appropriate source of information relevant to the policies the USA chooses to adopt.

34

D. C. Berliner

Table 3.2 Costs of schooling a child from primary grades through to high school completion in 11 nations (OECD 2016) Nations Australia Austria Belgium Denmark Iceland Luxembourg Norway Sweden Switzerland United Kingdom United States of America

Cumulative amount spent per student on primary and secondary education (in 2013 converted dollars) 141,556 162,399 162,045 165,166 146,672 247,822 196,026 154,506 231,497 146,597 138,692

President Trump also talked about the costs of schooling. He said “People are tired of spending more money on education than any nation in the world per capita.” And he also said “We’re number one in terms of cost per pupil by a factor of, worldwide, by a factor of many. Number two is so far behind, forget it” (Ujifusa 2016).

The president appears not to care that we can easily check those statements. And, with my class, I did that in 5 min or less. The president lied (OECD 2016) (Table 3.2). By now I am sure you get the point. The President or Secretary is reported to say something about education, and then I, or my students, check for the warrant. They said that private schools provide a better education than do public schools. I show the class data from Lubienski and Lubienski (2014) arguing that they do not. I then have students read other articles critical of the supposed private school advantage (Wenglinksy 2007; OECD 2011). It turns out that private schools “look better” because of who goes to them, not because their teachers or curriculum are necessarily any better than that provided to public school children. The administration has argued that voucher supported private schools are highly successful. With very little effort I located data from Washington DC, Ohio, Louisiana, and Indiana (Dynarski and Nichols 2017). These data all show no success for voucher schools. I also provide my students a brief history of the negative effects of vouchers in New Zealand, Chile, and Sweden (Levin 2017). While my students are generally impressed with these many counterfactuals, the federal administration, along with many state legislators remain unconvinced. America’s love affair with vouchers appears to be cooling down a little, in part because data generally does not support their success, and they take considerable amounts of money away from neighboring public schools. Nevertheless, despite the data, these educational programs still command considerable support in legislatures across my country. As a wonderful Arizona politician once told me, in America’s legislatures “facts are negotiable, opinions are rock solid!”

3 The Role of Modeling for “Seeking Truth” in an Educational Policy Classroom

35

Table 3.3 Arizona state demographics and actual enrollments at four reputedly outstanding charter schools (Potterton 2013)

Percent of students: free and reduced lunch Percent of students: English language learners Percent of students: IEPs Percent of students: white/non-hispanic

Arizona Average (%) 35

Basis SchoolScottsdale 0

Basis SchoolTucson 0

Great Hearts AcademyChandler 0

Great Hearts AcademyVeritas 0

7.5

0

0

0

0

11.7

0.06

2.1

1.4

3.5

42.9

57.7

53.1

67.4

72.8

When the administration in Washington, or here in Arizona, informs the public how successful charter schools are, I bring to my class data for four of the nations’ and my states’ top ranked high schools (Potterton 2013). Table 3.3 shows quite convincingly that in a recent year what was occurring in each exemplary school was a good deal of skimming or creaming of the easier to teach students was occurring. Although public charters are supposed to take all applicants, most do not. Instead, most charters accept the easiest to teach students, and do not admit, or soon drop, those that are harder to teach. When the President or Secretary says cyber schools are wonderful, I provide data and arguments that they are frequently the single worst alternative that can be chosen for educating one’s children (Miron et al. 2018). When the President or Secretary say school funding doesn’t matter much, I come back with recent data from New Jersey showing that after a large influx of money for some of the poorest districts in the state, educational outcomes improve substantially (Jackson et al. 2015). There appears to be no end to unsupported statements about education, and that keeps me busy trying to prevent too much negative knowledge, often promulgated by prestigious public figures, from accumulating in our students’ psyches.

Conclusion More than ever before in my career I try to model how one searches for warrant. I consciously attempt to promote more positive learning and to keep negative learning to a minimum. Am I successful? I think so, but that is also difficult to fact check. I think I am successful because I help my students avoid being foolish. That is how I put it to my class. I tell them it is foolish, if not dangerous, to hold unwarranted, unsubstantiated beliefs in one’s

36

D. C. Berliner

professional field. No one likes to appear to be foolish! So, they seem to value what I teach them. Values are different. With my students, I am clear that they can hold political beliefs leading to their support for promulgation of charter and voucher schools. But what they should never do is make false claims about the superiority of those systems. And certainly, they should never assume that in comparisons with the regular public schools, the data obtained from voucher or charter schools are comparable. The populations served in these different kinds of schools are almost always quite different. No amount of statistical equating can actually make those different school populations comparable. In fact, the apparently simple question about what type or sector of schooling is “better,” may be impossible to answer with certainty. Similarly, one can value the STEM fields, believing that they are crucially important for our economy, and in these times. But one should not hold beliefs that there is a shortage of skilled workers in this area, or that students who want to study other fields are making a mistake. An educator can both value and work toward achieving a higher high school graduation rate, or higher scores on standardized achievement tests. But one should acknowledge the historically high graduation rate we now have, the huge improvement in achievement among Black and Hispanic Americans, and the horribly boring schooling associated with test preparation to obtain those higher test scores. While values cannot be empirically checked, they certainly can be argued about, perhaps forever, and this is appropriate. On the other hand, the truth of the statements made by our political and educational leaders can often be quickly, easily, and inexpensively checked. Arguments based on reliable data are potentially winnable, though that is never certain when the data obtained is related to political issues. As noted at the start of this chapter, travel teaches us that there are other ways of thinking and behaving than those we might now possess. Travelling can challenge us. A good education can do the same. In a world where false knowledge exists in ever greater abundance, university professors need more than ever to consciously promote the possession of warranted knowledge by their students. Modeling that process appears to one successful way to do that. Finally, interrogating the truthfulness of the claims made by those in power may not just be an instructional preference of mine, with particular relevance for our times, but instead, behaviors like mine may be a requirement of those of us who work in universities. In a 50-year-old article that was highly influential for the worldwide academic community, and perhaps more relevant than ever, Noam Chomsky (1967) expressed his thoughts about the professoriate this way: Intellectuals are in a position to expose the lies of governments, to analyze actions according to their causes and motives and often hidden intentions. In the Western world, at least, they have the power that comes from political liberty, from access to information and freedom of expression. For a privileged minority, Western democracy provides the leisure, the facilities, and the training to seek the truth lying hidden behind the veil of distortion and misrepresentation, ideology and class interest, through which the events of current history are presented to us. The responsibilities of intellectuals, then, are much deeper, given the unique privileges that intellectuals enjoy.

3 The Role of Modeling for “Seeking Truth” in an Educational Policy Classroom

37

References Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory. Englewood Cliffs: Prentice-Hall. Bandura, A. (2008). Social cognitive theory of mass communication. In J. Bryant & M. B. Oliver (Eds.), Media effects: Advances in theory and research (pp. 94–124). New York: Routledge. Berliner, D. C., & Biddle, B. J. (1995). The manufactured crisis: Myths, fraud, and the attack on America’s public schools. New York: Addison-Wesley. Berliner, D. C., & Glass, G. V (2014). 50 myths and lies that threaten America’s public schools. New York: Teachers College Press.No period after the V Cano, R. (2017). Arizona students get less state money now than in 2008, study says. Azcentral, Phoenix, AZ. Retrieved September 1, 2017, from https://www.azcentral.com/story/ news/politics/arizona-education/2017/11/30/study-arizona-students-get-less-state-money-nowthan-2008/903541001/ Chomsky, N. (1967). The responsibility of intellectuals. The New York Review of Books, 8(1). Retrieved August 24, 2018, from https://www.nybooks.com/articles/1967/02/23/a-special-sup plement-the-responsibility-of-intelle/ Ciampaglia, G. L. (2018). The digital misinformation pipeline: Proposal for a research agenda. In O. Zlatkin-Troitschanskaia, G. Wittum, & A. Dengel (Eds.), Positive learning in the age of information – A blessing or a curse? (pp. 413–421). Wiesbaden: Springer. Duwe, G. (2017). Mass shootings are getting deadlier, not more frequent. Politico Magazine, October 4, 2017. Retrieved August 23, 2018, from https://www.politico.com/magazine/story/ 2017/10/04/mass-shootings-more-deadly-frequent-research-215678 Dynarski, M., & Nichols, A. (2017). More findings about school vouchers and test scores, and they are still negative. Washington, DC: Brookings. Retrieved September 1, 2018, from https:// www.brookings.edu/research/more-findings-about-school-vouchers-and-test-scores-and-theyare-still-negative/ Gates, W. (2011). Flip the curve: Student achievement vs. school budgets. The Blog, Huffington Post. https://www.huffingtonpost.com/bill-gates/bill-gates-school-performance_b_ 829771.html?guccounter¼1 Graf, N., Fry, R., & Funk, C. (2018). 7 facts about the STEM workforce. Washington, DC: Pew Research Center. Retrieved September 2, 2018, from http://www.pewresearch.org/fact-tank/ 2018/01/09/7-facts-about-the-stem-workforce/ Institute for Education Sciences, National Center for Education Statistics. (2016). Digest of education statistics. Retrieved September 1, 2018, from https://nces.ed.gov/programs/digest/ d17/tables/dt17_204.30.asp Jackson, C. K., Johnson, R. C., & Persico, C. (2015). The effects of school spending on educational and economic outcomes: Evidence from school finance reforms. The Quarterly Journal of Economics, 131(1), 157–218. Kamenetz, A., Arnold, A., & Cardinali, E (2018). The school shootings that weren’t. Washington, DC: National Public Radio (NPR). Retrieved August 27, 2018, from https://www.npr.org/ sections/ed/2018/08/27/640323347/the-school-shootings-that-werent Levin, H. M. (2017). Worldwide, school choice hasn’t improved performance. US News and World Report. Retrieved September 1, 2018, from https://www.usnews.com/news/best-countries/arti cles/2017-01-30/little-global-evidence-suggests-school-choice-helps-performance Lohr, S. (2017). Where the STEM jobs are (and where they aren’t). New York Times. Retrieved September 1, 2018, from https://www.nytimes.com/2017/11/01/education/edlife/stem-jobsindustry-careers.html?smid¼fb-nytscience&smtyp¼cur&_r¼0 Lubienski, C., & Lubienski, S. T. (2014). The public school advantage: Why public schools outperform private schools. Chicago, IL: University of Chicago Press. Lynch, M. (2016). The internet of US: Knowing more and understanding less in the age of big data. New York: Liveright.

38

D. C. Berliner

McFarland, J., Stark, P., & Cui, J. (2016). Trends in high school dropout and completion rates in the United States: 2013 compendium report. Washington, DC: U.S. Department of Education, National Center for Education Statistics. https://nces.ed.gov/pubs2016/2016117rev.pdf McFarland, J., Hussar, B., Wang, X., Zhang, J., Wang, K., Rathbun, A., et al. (2018). The condition of education 2018. Washington, DC: U.S. Department of Education, National Center for Education Statistics. https://nces.ed.gov/pubs2018/2018144.pdf Miron, G, Shank, C., & Davidson, C. (2018). Full-time virtual and blended schools: Enrollment, student characteristics, and performance. National Education Policy Center, Boulder, CO. Retrieved August 5, 2018, from http://nepc.colorado.edu/publication/virtualschools-annual-2018. Musu-Gillette, L., Zhang, A., Wang, K., Zhang, J., Kemp, J., Diliberti, M., et al. (2018). Indicators of school crime and safety: 2017. Washington, DC: National Center for Education Statistics, U.S. Department of Education, and Bureau of Justice Statistics, Office of Justice Programs, U.S. Department of Justice. National Center for Education Statistics. (2013). The Nation’s report card: Trends in academic progress 2012 (NCES 2013–456). Washington, DC: National Center for Education Statistics, Institute of Education Sciences, U.S. Department of Education. https://nces.ed.gov/ nationsreportcard/subject/publications/main2012/pdf/2013456.pdf Nicodemo, A., & Petronio, L. (2018). Schools are safer than they were in the 90s, and school shootings are not more common than they used to be, researchers say. News @ NorthEastern. Retrieved August 23, 2018, from https://news.northeastern.edu/2018/02/26/schools-are-stillone-of-the-safest-places-for-children-researcher-says/ OECD. (2011). Private schools: Who benefits? PISA in focus, no. 7. Paris: OECD Publishing. Retrieved September 1, 2018, from www.oecd.org/pisa/pisainfocus/48482894.pdf OECD. (2016). Education at a glance 2016, OECD indicators. Paris: OECD Publishing. https:// doi.org/10.1787/19991487 Pew Research Center. (2017). Sharp partisan divisions in views of national institutions. Retrieved August 24, 2018, from http://www.people-press.org/2017/07/10/sharp-partisan-divisions-inviews-of-national-institutions/ Phi Delta Kappan Poll. (2018). Teaching: Respect but dwindling appeal. The 50th annual poll of the public’s attitudes toward the public schools. Bloomington, IN: Phi Delta Kappan International. http://pdkpoll.org/results Potterton, A. U. (2013). A Citizen’s response to the President’s charter school education proclamation: With a profile of two “Highly Performing” charter school organizations in Arizona. Teachers College Record, November 1, 2013. Retrieved September 2, 2018, from http://www. tcrecord.org/content.asp?contentid¼17309 Smith, A. A. (2017). Coping with zero in Arizona. Inside Higher Ed. Retrieved September 1, 2018, from https://www.insidehighered.com/news/2017/01/27/arizona-community-colleges-copestate-disinvestment-and-declining-enrollments Stein, P. (2018a). Public Schools graduation rate on track to decline this year. The Washington Post. Retrieved September 1, 2018, from https://www.washingtonpost.com/local/education/dcpublic-schools-graduation-rate-on-track-to-decline-this-year/2018/03/01/3429790a-1cdc-11e8b2d9-08e748f892c0_story.html?noredirect¼on&utm_term¼.aebd87c7f761 Stein, P. (2018b). Report calls into question validity of hundreds of diplomas. The Washington Post. Retrieved September 1, 2018, from https://www.washingtonpost.com/local/education/ report-calls-into-question-validity-of-hundreds-of-diplomas/2018/01/29/d86e5c82-0513-11e88777-2a059f168dd2_story.html?utm_term¼.e228b0895286 Tillerson, R. (2018). Speech at the Virginia Military Academy (May 16, 2018). Quoted from The Atlantic. https://www.theatlantic.com/ideas/archive/2018/05/rex-tillerson/560612/ Trohler, D. (2011). Languages of education. New York: Routledge. Trump, D. J. (2000). The America we deserve. Los Angeles, CA: Renaissance Books. Trump, D. J. (2017). Presidential memorandum for the secretary of education: Increasing access to high-quality science, technology, engineering, and mathematics (STEM) education. Presidential Memoranda. Retrieved September 1, 2018, from https://www.whitehouse.gov/presiden tial-actions/presidential-memorandum-secretary-education/

3 The Role of Modeling for “Seeking Truth” in an Educational Policy Classroom

39

U.S. Department of Education. (2018). School climate and safety: Data highlights on social climate and safety in our nation’s public schools. In 2015–16 civil rights data collection. Washington, DC: U.S. Department of Education, Office for Civil Rights. Ujifusa, A. (2016). Donald Trump says we spend a ton for bad test scores. Let’s check the numbers. Education Week. Retrieved August 15, 2018, from http://blogs.edweek.org/edweek/campaignk-12/2016/01/donald_trump_we_spend_a_ton_on.html Ujifusa, A. (2017). At school choice event, Trump praises D.C. vouchers for helping kids graduate. Bethesda, MD: Education Week. http://blogs.edweek.org/edweek/campaign-k-12/2017/05/ trump_praises_dc_vouchers_graduation.html Wenglinksy, H. (2007). Are private high schools better academically than public high schools? Washington, D.C.: Center on Education Policy. Retrieved July 13, 2013, from http://www. edline.com/uploads/pdf/PrivateSchoolsReport.pdf White, S., & Tyler, J. (2015). Underrepresented minorities in high school physics. Results from the 2012–13 Nationwide survey of high school physics teachers. College Park, MD: American Institute of Physics. Retrieved August 1, 2018, from https://www.aip.org/statistics/reports/ underrepresented-minorities-high-school-physics Yanez, C., Lessne, D., & Hansen, R. (2018). Student victimization in U.S. schools. Results from the 2015 school crime supplement to the National Crime Victimization Survey (p. 106). Washington, DC: U.S. Department of Education, Stats in Brief, NCES, 2018. Zlatkin-Troitschanskaia, O., Schmidt, S., Molerov, D., Shavelson, R. J., & Berliner, D. (2018). Conceptual fundamentals for a theoretical and empirical framework of positive learning. In O. Zlatkin-Troitschanskaia, G. Wittum, & A. Dengel (Eds.), Positive learning in the age of information – A blessing or a curse? (pp. 29–52). Wiesbaden: Springer.

Chapter 4

Literature, Simulation, and the Path Towards Deeper Learning Mita Banerjee

Why Teach Literature?1 Literature, and more importantly teaching literature, may be one of the core possibilities of “deeper learning” (Pellegrino and Hilton 2012). Yet, to argue that reading literature may enhance deeper learning is itself related to the concept of PLATO in ways which, at first sight, may seem mutually exclusive (for a detailed description of the PLATO program, see Zlatkin-Troitschanskaia et al. 2018). First, many approaches to deeper learning argue that instruction, in the humanities and elsewhere, has too often focused only on domain-specific teaching—such as the teaching of, say, reading literature—rather than on didactical approaches to conveying this domain-specific knowledge (Meyer et al. 2018). Recent approaches which have also been at the heart of the PLATO project have suggested, however, that we need to focus not only on domain-specific knowledge, but also on other core aspects at the center of teaching any given field. This leads to a “pluriliteracies model” which “focuses on the development of subject specific literacies and transferrable knowledge and skills as well as on personal growth” (Meyer et al. 2018, p. 238). What teaching models may we develop which fruitfully connect, for instance, domain-specific knowledge and didactics? The pluriliteracies approach outlined above is independent of the particularity of the given field at hand; it relates, first and foremost, to the interaction between and complementarity of domain-specific knowledge and other fields, stemming especially from didactics and educational 1 The ideas developed in this paper are part of a collaborative framework with Oliver Meyer and Margarete Imhof; my discussion of literature and simulation is greatly indebted to the dialogue with didactics and psychology.

M. Banerjee (*) Obama Institute of Transnational American Studies, Department of English and Linguistics, Johannes Gutenberg University of Mainz, Mainz, Germany e-mail: [email protected] © Springer Nature Switzerland AG 2019 O. Zlatkin-Troitschanskaia (ed.), Frontiers and Advances in Positive Learning in the Age of InformaTiOn (PLATO), https://doi.org/10.1007/978-3-030-26578-6_4

41

42

M. Banerjee

psychology. The pluriliteracies model hence proposes that we focus not only on subject specific knowledge, but also on aspects such as “affect” (Meyer et al. 2018, p. 243), “engagement,” “mastery,” “reflection” (p. 244), and “interdependence of growth areas” (p. 245). The aim of this paper is to show how domain-specific knowledge (here, American literature) has changed in conversation both with didactics and psychology. This paper proposes to connect the discipline of American studies with research from didactics and educational psychology to develop new models for teaching (American) literature. It hence implicitly argues that PLATO’s goal of fostering deeper learning can be achieved only through interdisciplinarity. Using the approach and the model of deeper learning as a starting point and baseline of reference (Meyer et al. 2018), this paper discusses a specific way in which domain-specific knowledge—i.e., what is taught—may in fact matter in a sense that it may give rise to particular forms of didactics. As Kosslyn (2018) has argued, “the utility of the humanities [may depend crucially] on how they are taught” (p. 27). In the context of deeper learning, we may thus ask what specific properties of the fields or genres we teach in the humanities lend themselves to developing particular teaching models enabling students to acquire the critical skills they need to cope with the infinity of information in the information age (see also Berliner in this volume). This paper inquires into the ways in which we can teach literature to achieve deeper learning. The phrase “deeper learning literature”2 is meant to capture the precise intersection of teaching literature through new didactic forms. This discussion in its turn is situated in a wider debate about the future—and the relevance—of the humanities. The humanities have recently come under attack for being irrelevant and for being much too self-absorbed for thinking about their own possible applications. As Kosslyn (2018) notes, “Often the humanities seem selfreferential. Teaching in the humanities often does not invite ways to extend the skills and knowledge outside the narrow confines of the materials at hand” (p. 27). It may thus be surprising, and highly relevant for the interdisciplinary work envisioned by PLATO, that the most outspoken and passionate defenders of the humanities have been researchers who do not work in the humanities. As Kosslyn begins his essay (2018), “I do not work in the humanities. Nevertheless, I am appalled to see the resurgence of negativity about the humanities and want to provide a perspective on why we should include them in any twenty-first century curriculum [. . .].” (p. 24). Taking its cue from Kosslyn, this paper suggests that the humanities may benefit from listening to how they are perceived by fields outside the humanities; such a dialogue between self-image and exterior view, in turn, may also be a remedy to the much-criticized self-absorption of the humanities. Fields such as medical humanities and narrative medicine have thus argued that the value of the humanities in general and of teaching literature in particular lies in

2

I am indebted to Oliver Meyer for this term.

4 Literature, Simulation, and the Path Towards Deeper Learning

43

the fact that they enable a change of perspective.3 In the field of narrative medicine, scholars such as Rita Charon, Maura Spiegel, and Danielle Spencer have proposed that in the didactics of medicine, for instance, the use of literature texts may be key (Spiegel and Spencer 2017). As Charon, the founder of narrative medicine, writes, literature and the techniques used by literary scholars may have much to offer to the medical curriculum. Her approach is based on the idea of “narrative knowledge”: [To be] empathetic and effective in the care of individual patients [. . .] requires what psychologists and literary scholars call narrative knowledge [. . .]. If narratives are stories that have a teller, a listener, a time course, a plot, and a point, then narrative knowledge is what we naturally use to make sense of them (2008, p. 9).

Not only philosophy but literature as well has thus asked some of the most fundamental questions about what makes us human. According to Kosslyn, “the humanities are the best way to learn some important skills and knowledge. Yes, they help us to appreciate what it is to be human and to live full and complete lives, but they also serve much more utilitarian, quotidian purposes [. . .]” (2018, p. 24). If PLATO is centrally concerned with fostering critical thinking and moral reasoning based on a core of specific values and ethics (Zlatkin-Troitschanskaia et al. 2018), then the humanities and literature in particular may have much to offer by asking some of the most fundamental questions. Yet, one of the qualities of literature which is not addressed in these contexts and which may be at the heart of PLATO is not so much or not only what literature is, but also how it may be taught. It is at this juncture critical for deeper learning that the concept of “deeper learning literature” situates itself. The model we propose takes the property of literature as enabling a change of perspective—a property which is also at the heart of the narrative medicine approach outlined above—as its starting point, and it inquires into the intersection between turn-taking or role play on the one hand and simulation on the other. As Kosslyn notes, “literature [. . .] exercises our capacity to see things from novel perspectives. We humans can learn by mentally simulating the world—which often involves visualizing specific scenarios—and by imitating what others do” (2018, p. 25; emphasis mine).

Literature as Simulation Capitalize on the intersection between what literature can do (ask as it does the most fundamental questions of what makes us human) and how it can be taught to unfold its fullest potential for deeper learning, we may need to develop new models for

3

At the same time, a caveat may be in order here. There is a sense in which the current debate is curiously lopsided. For establishing the relevance of, say, fields within the life sciences, for instance, no one would require life scientists to look into how the field is viewed by scholars of literature. There may thus be a key discrepancy here in arguing that the humanities need to prove their efficacy in the terms of another discipline, or that their value lies only in their transferability. In the face of such discussions, however, this paper takes a pragmatic approach, looking at both what the humanities have to offer and how and in what fields this knowledge may then be applied.

44

M. Banerjee

teaching literature. These models, I suggest here, may be built on the concept of simulation.4 As Shavelson puts it, “high-fidelity simulations of actual or ‘criterion’ situations to research, teach, and assess criterion’ are at the core of research and development in PLATO” (2018, p. 283). The idea of simulation, at first sight, seems to have nothing to do with literature or the teaching of literature; its provenance lies outside the humanities, in the field of psychology. In particular, the approach outlined here takes its cue from the work of Shavelson.5 When the USA first put a man on the moon, it was Shavelson who developed the main models of simulation meant to prepare the astronauts for their mission. These simulations ranged from low to high fidelity: the closer they were to the actual moon landing, the higher their degree of fidelity was. As Shavelson describes, “[in] the case of an extraterrestrial simulation of gravity, three alternatives were used in research and training. Moving from the lowest level of abstraction and highest level of fidelity, there are: (a) parabolic flight, (b) water flotation, and (c) counterbalance” (2018, p. 283). What would the teaching of literature, however, have to do with putting a man on the moon? My point here is not that the humanities have now succumbed to a form of megalomania, claiming that everything falls within their terrain, but rather that these models, originating in psychology, may fruitfully be applied to the didactics of teaching the humanities, and of teaching literature in particular. To be sure, simulation has already been at the core of teaching models in a variety of disciplines, among them the translator training outlined by Hansen-Schirra et al. (2018). It has also been central to didactics in law, where mock trials are a key part of the curriculum (Karraker 1993), or in political science (Ahmadov 2011), where students, for instance, simulate the admission process of new countries into the European union (Balas et al. 2017). Such models of simulation, at first sight, may seem more appropriate than the experiment of applying the simulation model to the teaching of literature. The advantage of using simulation in these contexts—putting a man on the moon, conducting a mock trial, or admitting a country into the EU—seems to be that there is a concrete material framework which can be simulated: the specific conditions, the atmosphere on the moon, and the effects of weightlessness; the working environment and time pressure of simultaneous translation; the specific interactions and protocols of a court room; and the protocols and dynamics of EU admission hearings. The study of teaching literature, on the other hand, seems to offer no such material frameworks, or so it would seem. I suggest, however, that the material conditions

4

The link between literature and simulation has often been made in the context of the opposition, or at least difference, between literature and other, usually digital media (e.g., Lem 1989). At the same time, the idea of literature as a medium of experimental or imagined action has also been taken up by literary authors, most prominently by German writer Wellershoff, who spoke of his fiction as a “literary space of simulation” (Bügner 1993). My aim in this paper is to ask how these approaches to both reading and writing literature might fruitfully be transformed into a different approach to teaching literature. 5 He is one of the expert advisers of the PLATO project.

4 Literature, Simulation, and the Path Towards Deeper Learning

45

which literature simulates are themselves highly akin to the simulations and interactions simulated in the frameworks outlined above, if only at second sight. For one of the central approaches to literature, and an approach which may be particularly fruitful for interdisciplinary work between literary studies and other fields, is the idea that literature constitutes soziales Probehandeln (Gendolla and Schäfer 2004) or “experimental action” in the social sphere (Wood 2018; Felman 1982). Literary texts simulate social interactions in all kinds of ways. For instance, they portray, in particular cultural and historical contexts, the encounter between the dominant culture and minority groups in the framework of a given nation state. Literary narratives thus help us come to terms with migration since they portray, not only in esthetic but also and especially in psychological terms, a society’s encounter with cultural or religious difference. Moreover, reading literature enhances empathy in the reader (Kosslyn 2018, p. 25; Kidd and Castano 2013). If this is the case, then, the question I ask in this paper is how we can develop new approaches to teaching literature as simulation. The present model proposes that we take the concept of literature as soziales Probehandeln or experimental social action as the basis for reading literature as a simulation of social interaction. The task becomes to develop models that capitalize on the idea of simulation—or rather, of multiple simulations with different layers and degrees of fidelity—not just in reading, but in teaching literature. On the scale from “surface learning” to “positive learning”, “reading literature”6 would establish a basic understanding of the text. Simulation I, the first stage of the actual simulation model would be the understanding of literature as a simulation of social interaction (soziales Probehandeln). This first stage of the simulation prepares the students for a different form of engagement with the text: an approach that is not only cognitive, but enables an affective, immersive understanding of a literary text. The role of affect, in turn, is central to a different approach to didactics in university education. As a rule, university education has been said to be a space form in which, at least on the level of teaching, all affect should be absent. Moreover, traditional forms such as the university lecture assume that a top-down approach with the university professor at the apex of the hierarchy. Recent models, on the other hand specify that the teaching process should be an interactive one, with students and trenchers sharing in what then becomes a co-production of knowledge. One way towards developing models of deeper learning, then, may be to systematically investigate and integrate the role of affect in the classroom (Banerjee and Meyer 2018, p. 11). The next stage of the simulation model, simulation II, is one of the center pieces of the model. Each literary text confronts its readers with an ethical dilemma. The characters represented in the texts, themselves representative, to a degree, of particular social groups in a given society at a particular point in time, are confronted with ethical choices. Simulation II requires students to enact a scene from the literary text they have read beforehand. They get to choose the characters they want to embody; they may enact characters which are already in the text or invent new ones. In

6

I am indebted to Oliver Meyer for this point.

46

M. Banerjee

enacting the scene—a scene which may or may not be found in the actual text, but which in any case needs to fit the parameters given in the text—students have to script their own dialogues; they should not use direct quotations from the original text. This form of simulation II has a number of advantages: Students immerse themselves into the text; they simulate social interactions in a particular society in a particular historical period. In this immersion, the simulation—far removed as tit may at first seem from their own context and their own life worlds—is now suddenly about them. Cognitive psychology has shown that learning models are more effective to the extent that learners have to make active choices (Markant and Gureckis 2014). This model of simulation II hence confronts students with the task of having to take myriads of choices: which characters to enact, which scene to play, whether to take existing characters or new ones. At the same time, this model of simulation II enables a particular form of in-group differentiation (Binnendifferenzierung). The role they play and the depth (of thought, argument, and affect) will be in line with the learner’s specific level of aptitude and expertise. In one and the same group, learners of different stages of learning can interact in fruitful and highly complex ways. Furthermore, each learner brings to the simulation his own biography, context, and experience. His or her choice of literary character, and the interpretation he or she brings to this character, will be based not only on the learner’s domain-specific knowledge (say, of the literary genre, historical period, the language used, etc.) but also on his previous biographical knowledge and affective experience. These “background” influences, however, will enter the simulation without the learner’s being aware of them (such an awareness will have to be spelled out only in subsequent stages of the simulation model). At the same time, the nature of the simulation, because the students have to act in the roles they have chosen, is more than just a task. It is meant to foster the students’ engagement with the material they study: Student engagement captures the degree to which a student is actively involved in activities designed for learning [. . .]. It is conceptualized as a multidimensional construct composed of behavioral, cognitive, emotional, and social patterns of behavior, which, in combination, contribute to learning and to developing successful learning habits and attitudes (Meyer et al. 2018, p. 244).

Moreover, there is a sense in which, in simulation II, students have to set their own goals and devise their own strategies for first understanding and enacting7 the text at hand. Simulation II is hence meant to invite self-reflection: Self-reflection is both a way of learning and a goal for learning. Current theories of selfregulated learning contain self-reflection as a core construct. Successful self-regulated learning needs students to understand how to set adequate goals for themselves, how to plan a learning episode including selecting learning strategies, to monitor reiterating goals when they meet difficulties and errors [. . .] (Meyer et al. 2018, p. 244).

7

Simulation II is not so much a re-enactment as it is an enactment, because students cannot simply replicate a scene from the text but have to write their own scripts and hence create their own interpretation of the text which in this sense becomes “their” text.

4 Literature, Simulation, and the Path Towards Deeper Learning

47

In simulation II, students decide, for instance, how much background research they want to conduct. To what extent, for instance, do they want to delve into the historical context from which the literary text arose, and to what extent is this knowledge important for the scene they want to enact? Moreover, they may be faced with difficulties in understanding the language or esthetic nuances of the text; these are obstacles in the learning process which they have to overcome, choosing their own path around them. Most important about the stage of simulation II, however, is that the students (presenting in groups of two or three) identify the ethical dilemma which, to their mind, the text confronts the reader with. The aim of simulation II is to convey the dilemma to the other students who serve as an audience to the performance, and to enable them to reflect on the ethical choices they would have made in this situation. If each literary text confronts the reader with multiple ethical dilemmas, then, the students of the group which performs the simulation identify—not explicitly, but implicitly, through the scene they are enacting—choose one particular dilemma which to them is the most salient or relevant. This relevance is itself key to the model of deeper learning literature: By first identifying, and then enacting and hence immersing themselves into an ethical dilemma in which they have to make choices, students become aware that this situation—remote as it may at first seem from their particular cultural and historical context and their own life worlds—suddenly comes to be about them. Even if the degree of fidelity is still low—they are enacting, as German learners of English, a seventeenth-century text written by a Puritan goodwife such as Mary Rowlandson’s The Sovereignty and Goodness of God (1648/ 1997), the process of enacting a dilemma that, for instance, Rowlandson’s character is faced with in the text no longer seems as remote. It is they who have to act in Rowlandson’s place. The enacting of an ethical dilemma, with regard to which students have to position themselves in a particular way, is itself key to PLATO’s overall goal (for a definition of positive learning in PLATO, see Zlatkin-Troitschanskaia et al. 2018). As Gardner (2018) has argued, it is especially important to take into account, apart from the curriculum itself, the relevance of both “character” and “context” (see also Gardner and Fischmann in this volume). In devising new forms of teaching, he writes, we need to pay attention to “Character (the kinds of human beings we hope to nurture); and Context (the educational environments conducive to these curricular and character goals)” (11; italics original). The simulation model thus confronts students with the task of having to first identify and then to take ethical choices. These choices, removed as they may at first seem from their own life worlds (low fidelity), are key to developing their identities as critically engaged citizens. This is especially important in view of the fact that learners from ages 16 or 17 to 25 can be seen as “emerging adults” (Meyer et al. 2018, p. 237). As Gardner (2018) notes, At this stage of life, young people are most open to cognitive broadening, least likely to be burdened by other commitments (full time work, taking care of their own household, starting a family) (p. 10).

48

M. Banerjee

Building on the above-described pluriliteracies model of deeper learning, the simulation approach to deeper learning literature thus emphasizes what Shavelson has referred to as the “moral component” (Shavelson 2018). He observes, Action also carries with it a moral component: “Should I do this?”, “In what way does it affect other people?”, “How does it affect the environment?”, “How does it affect me?” Indeed, this moral component of action is the focus of one of the four areas of research envisioned for PLATO. Hence the Teaching for Learning model might consider a moral facet in addition (p. 287).

After the scene from the text has been enacted in simulation II, students stay in their roles and their peers get to ask them questions. This Q&A is still part of the simulation, phase II. Immersed in the roles they have chosen, the presenters have to react to the questions posed by their fellow students. As studies have found, learners are much more likely to respond to feedback from their peers than from the instructor; this part of the simulation model creates a space where a conversation takes place among students, with the instructor being part of the group, asking her own questions, but by no means from a vantage point which is any different than that of the other audience members. The next stage of the simulation, simulation III, functions as an “abstraction through guided reflection.”8 Students have to zoom out of and reflect on the choices they made in the enactment (simulation II). What characters did they choose, and why? To what extent did they invent characters that were not present in the original text? Students are hence invited by the instructor to reflect on the choices they made. So far, the instructor has been part of the audience, he or she may now emerge as a mentor guiding the students’ reflection on their own actions and decisions during the enactment of simulation II. Key to simulation III is enabling students to abstract from their own performance in simulation II: In this abstraction, students learn to identify missing perspectives they “invent,” for instance, a Native American as a fullyfledged, complex character because such a character is absent from Rowlandson’s narrative. In so doing, students identify the historical context of the text, and the ideology underlying it. In Rowlandson’s captivity narrative, Rowlandson’s Puritan outlook as well as the publication conventions which accompanied the making public of her testimony of captivity made it impossible to envision Native American characters whose cultural signification was as sophisticated as her own. Central to the goal of deeper learning is the idea that students become “inoculated” (ZlatkinTroitschanskaia et al. 2018) to “fake news” and to texts in different media which manipulate them by using a particular kind of affect. The aim of simulation III is to enable students to identify the biases of a specific text (Zlatkin-Troitschanskaia et al. in this volume). It is important that this step at the core of simulation III may be adapted to other models of teaching (see also Berliner in this volume). Shavelson (2018) proposes with regard to deeper learning models in Internet research that students do not take a given website at face value but instead ask a seemingly simple question: “Where 8

I am indebted to Oliver Meyer for this term.

4 Literature, Simulation, and the Path Towards Deeper Learning

49

have I landed?” Shavelson refers to this step in the process of unpacking any given piece of information given, for instance, on a website as Taking Bearing: Rather than clicking randomly, readers actively try to figure out where they have landed, who is behind the information being provided, and what the creators of the information want the user to take from the site (p. 289; italics original).

This question enables students to identify the location of the website: The context it comes from, its sponsors, with their own specific ideology and motivation. In exploring the context and ideological background of a given source, students may also assess or question the “trustworthiness” of the site; and they may go on to “read laterally” (Shavelson 2018, p. 289) to find out the ideological agenda of the text or website they are confronted with (see also Wineburg et al. 2018). The “move” students make with regard to Rowlandson’s seventeenth-century Puritan text may be highly similar: They have “landed” as it were, on a textual site emblematic of the Puritan worldview; if they search laterally, they will be able to unpack both Rowlandson’s Puritan ideology and relate this ideology to other texts such as Winthrop’s “A Model of Christian Charity” (1630/1994). Interesting in the context of the PLATO model, moreover, is that the seeming remoteness of Rowlandson’s text (as a literary text from the seventeenth century, as a Puritan captivity narrative, and so on) may in fact be suited for teaching students the mechanisms of distrust or critical inquiry, which they can bring to apply other texts and media which they are more familiar with, such as websites. The fidelity degrees included in the simulation model may be essential here. Precisely because students may be unfamiliar with the genre given the fact that they are confronted, in their everyday lives, more with Internet-based texts or videos than with a literary texts, and will gain most of their information from these source, they may be better able to identify the biases of this literary text. Lack of familiarity with the genre and low fidelity may hence be assets in this context. Using a text which is not part of their daily repertoire, may thus enable students to identify the mechanisms of detecting ideological biases, as in a “dry run.” Simulation IV is located at the further end of the scale towards positive learning. At this stage, students translate the ethical conclusions they have drawn from the simulation in stage II into their own personal context. According to Pellegrino and Hilton (2012), deeper learning takes place when “‘the process through which an individual becomes capable of taking what was learned in one situation and applying it to a new situation (i.e. transfer)’” (quoted in Meyer et al. 2018, p. 239). What personal relevance does the choice made in the ethical dilemma presented in the text and the enactment in simulation II have for their own life worlds? Even more importantly, students in simulation IV may reflect on the learning constraints brought about by their own personal context. In the instance of enacting Rowlandson’s captivity narrative in an American studies classroom, one student said she realized during the performance how carried away she was by Rowlandson’s Puritan outlook, and how aversely she reacted to any criticism, by her peers, of the religious fervor of Rowlandson’s character in the text. She wondered whether her own highly religious background proved to be a learning

50

M. Banerjee

constraint in this particular context, a boundary which she could not go beyond.9 Her aim, she said, was to overcome these constraints in this particular context, hence calling into question the ideologies she had been brought up with. This step of simulation IV is crucial since, as Shavelson (2018) emphasizes, it is short-sighted to focus only on the learner as an individual; important is to look at the learner in her own particular context. The potential of simulation IV is to enable learners to reflect on the constraints and positionalities brought about by their own social contexts. According to Shavelson (2018), [An additional] point that researchers need to avoid is a narrow focus on “the person”—the student or the citizen—in studying the impact of media on negative and positive learning. [. . .] People live in environments that support and foster either PL [Positive Learning] or NL [Negative Learning]. We need to understand the nature of these environments and their impact on thought, feeling and action. This is not about the way news is manipulated but about micro-cultures that exist in neighborhoods, religious groups or in societies that afford fake news that support strongly held if untested or unethical beliefs (p. 290; my insertion).

As part of simulation IV and reaching out to a further stage in the simulation model, such as Simulation V, students may choose a text which is close to their own life worlds (and thus possesses a higher level of fidelity) in which an ethical dilemma unfolds which is similar to the one they have encountered in the previous simulation. As Kosslyn (2018) has emphasized, this process of applying the knowledge gleaned from reading literature as a form of soziales Probehandeln is crucial for deeper learning to occur. He proposes, “learning about human nature by reading novels must in turn apply to real-life situations that share crucial underlying characteristics with the fictional situation” (2018, p. 27).

This translation process would enable students to question the representation contained, say, in a rap song, through using the tools of critical inspection which they have acquired in the low fidelity simulation. This would be an additional benefit of low-level simulation models: Precisely because students are not familiar with the texts used in this part of the simulation, they be readier to call their messages into doubt and expose their ideological content. It is by moving from simulations of low fidelity to those of high fidelity that students may be more prepared to assess the trustworthiness of some sources as compared to others. As Gardner (2018) suggests, this points to the core of PLATO’s engagement with the concept of educating a future citizenry: “For the citizen a raft of analogous dilemmas arise. How does one inform oneself with respect to issues of the day? How does one know whom or what to trust and what to ignore?” (p. 15). At stake in the project envisioned by PLATO is not just that we need to develop new teaching models, models which help shape the “character” of emerging adults as future citizens of the nations we live in and the global communities we are part of, but that we devise models which are especially geared, as Zlatkin-Troitschanskaia

9 As Shavelson also notes, parents’ religious beliefs are transmitted to children such that they impact on how students interpret what they are learning (2018, p. 287).

4 Literature, Simulation, and the Path Towards Deeper Learning

51

has argued, the challenges of the information age. According to ZlatkinTroitschanskaia et al. (2018), While learning has constantly been an object of research in manifold disciplines and fields, it has generally been understood in a positive sense. In the Age of Information, we are witnessing an increasing number of phenomena in the context of knowledge construction and accumulation that we describe as “negative learning.” This includes, for example, the deliberate circulation of counterfactual knowledge leading to negative learning outcomes, i.e. deficient decision-making and acting [. . .]. On the other hand, following classical humanist ideals, we assume to transform negative learning into positive learning in both formal and informal education [. . .] (p. 1).

Modeling Deeper Learning At the same time, one of the major shortcomings of previous models of deeper learning used in a humanities context seems to be that from within the humanities alone, few tools had been available for in fact measuring the success of what we call deeper learning models here. How might we assess whether or not a given new teaching format has in fact led to deeper learning? The model I am proposing in this paper, as in previous publications (Meyer et al. 2018) is an interdisciplinary model which sets out to link literary studies, didactics, and educational psychology.10 It is in collaborating with psychological models and methods in particular that the success of deeper learning may be assessed. The simulation model I have been describing here, drawing on the work of Shavelson in particular, may thus be suited both for teaching literature and for assessing deeper learning. As Shavelson (2018) notes, simulations possess “usefulness not only for instruction but also for assessment of competence” (p. 282). From a psychological perspective, what would be the effects and interactions derived from our theoretical assumptions?11 What would the simulation model look like if aspects of the instructive mode and the role of moderators and mediators are taken into account? With regard to the mode of instruction, it is important that simulation models may give rise to what we may call “learning partnerships.” The aim is a “co-production of knowledge” (Meyer et al. 2015). As has been outlined above, the role of the instructor changes in the context of teaching literature as simulation, as s/he becomes a mentor of the student’s progress and his or her path towards developing an individual, self-directed interpretation of the text. This aspect may itself be key to the concept of deeper learning in the literature classroom. The ethical dilemma with which a literary text presents its readers is different from a task which needs to be solved; it is a situation in which, confronted with an ethical choice, both students and instructor in a sense have to reflect on their own ethical ideals. Key

10

This part of the essay is centrally based on the work of Margarete Imhof, to whom I am greatly indebted. 11 The following ideas are chiefly indebted to Margarete Imhof’s ideas.

52

M. Banerjee

to simulation II is that in a sense, both students and teacher involve together; the simulation confronts them with having to make an ethical choice, and by making that choice, they grow together. Because what is at stake is self-reflection and personal growth on the part of both the students and the teacher, the instructor’s learning is no different from the students’ since what is at issue in this stage of the simulation is not so much a detailed knowledge of the text or the historical background but rather ethical reasoning and the making of ethical choices. Thus, “personal growth [. . .] occurs both in the student and the teacher” (Meyer et al. 2018, p. 245). Within the simulations themselves, students develop empathy through identification with the roles they have created; simulations stimulate emotional and behavioral engagement. Perspective taking is key to the simulations that are being enacted. At the same time, students develop a high degree of self-efficacy. According to Meyer et al. (2018), Self-efficacy, defined as a strong belief in one’s ability to solve a problem and the expectation to succeed in a task is a ‘key personal resource in self-development, successful adaptation, and change’ (p. 243).

Key here is that the simulation model enables students to develop an assessment of their own performance which is, at first, independent of either the reaction of their peers or their instructor. The goal of the simulation is, first and foremost, to create their own interpretation of the ethical dilemma they enact; in the simulation, each student “authentically” stages his or her reading of the text. At the heart of the simulation model, there is a sense of “well-being” and a “goal-orientation”12 based on the goal which students set for themselves (see also Dormann and Guthier in this volume). There is hence an intrinsic motivation at the heart of the simulation model; students develop persistence in following the task through, in first identifying the ethical dilemma, then identifying potential roles, enacting them, etc. At the same time, the key transition from simulation II to simulations III and IV makes sure that students’ engagement with the text takes place not only on an emotional, but also on a cognitive level. As a result, students develop transferable subject knowledge and skills, both of which are key to forms of deeper learning (Pellegrino and Hilton 2012). They develop complex forms of media literacy, since they are able to transfer what they have learned in low-level forms of simulation (in, for instance, understanding a seventeenth-century captivity narrative from the USA as German learners of English) to texts in other media formats (say, their favorite blockbuster film or rap song). Even more importantly, students acquire complex problem-solving skills, and there is much room for creativity since they get to delineate and invent their own roles, set their own tasks, etc. In so doing, they develop autonomy, ownership, and interest.13

12

Both these terms are Margarete Imhof’s. My discussion of the potential outcome of the simulation is crucially based on Margarete Imhof’s assessment. 13

4 Literature, Simulation, and the Path Towards Deeper Learning

53

Conclusion The aim of this paper has been to show how the questions asked by PLATO may change the ways we think about discipline-specific knowledge and the ways in which we convey this knowledge. It has revisited, in its discussion of deeper learning in an American studies classroom, the idea of literature in dialogue with other media (such as Internet-based texts, YouTube videos, or hip hop songs). Most importantly, it has proposed that the model of simulation, which is widely used in classrooms in law, political science, or translator training, may also be suited for the literature classroom. Literature may teach us to “own up”: In our daily lives and as citizens of increasingly complex societies, we are constantly called upon to take ethical decisions. PLATO’s goal is to enable students—and ourselves as teachers—to develop ways to act responsibly, based on rational decisions which by no means have to be incompatible with our morals and affects (see also Minnameier 2018). PLATO’s advantage may thus be its courage to ask some of the “big questions,” questions which the humanities may usually shy away from asking, given as they may be to an attention to the minutest of details. This may be a time when the value of the humanities is being “rediscovered” by fields outside the humanities; such rediscovery, far from superfluous or irrelevant to the self-image of the humanities, may in fact be solace in that they may renew our self-esteem in the face of budget cuts and calls for both applicability and economic usefulness (Lill 2016). The value of the interdisciplinary work envisioned by PLATO may be in its chutzpah to try to counter all the forms of negative learning, including “false certainties” (ZlatkinTroitschanskaia et al. 2018). In a time when “fake news” and a widespread distrust of academia seems to newly have become accepted in the public sphere, the attempt to teach emerging adults how to inoculate themselves against such rhetoric may be a mammoth task, but it is deeply necessary. Oser (2018) has described this necessity as the “Trump effect”: Since Trump has been elected and has been reacting to current developments with irrational statements and deeds, in restaurants, schools, families, at workplaces, everywhere people discuss political issues, political values and political traditions. [. . .] Many begin to understand the endangered relationship between nations and cultures. And Trump’s controversial twittering brings together former politics enemies for the sake of an open society. Thus, to produce an inaccurate and even ‘fake’ behavior can be helpful against a political Sleeping Beauty [. . .] (p. 365).

Such resistance to the false certainties which now seem to have become newly acceptable under Trump’s presidency may be all the more necessary, moreover, for work in American studies, which can bring to an investigation of the Trump effect its knowledge of the genealogy of ideas which made Trump possible. Yet, this effect is far from being confined to the USA alone; precisely because it seems to engulf ever more parts of the world and ever more aspects of our lives both academic and personal, our work of necessity has to be interdisciplinary.

54

M. Banerjee

References Ahmadov, A. (2011). When great minds don’t think alike: Using mock trials in teaching political thought. Political Science and Politics, 44(3), 625–628. Balas, A, Ebner, N., & Kotelis, A. (2017). European council simulation “European agenda on migration” role-play negotiation. https://eeas.europa.eu/sites/eeas/files/sim1-european_coun cil_simulation_migration-110917-eu_file.pdf Banerjee, M., & Meyer, O. (2018). Deeper learning in promoting pluriliteracies development in American studies courses. Gutenberg University of Mainz. Unpublished manuscript. Bügner, T. (1993). Lebenssimulationen: Zur Literaturtheorie und Fiktionalen Praxis von Dieter Wellershoff. Wiesbaden: Springer. Charon, R. (2008). Narrative medicine: Honoring the stories of illness. Oxford: Oxford University Press. Felman, S. (Ed.). (1982). Literature and psychoanalysis: The question of reading - otherwise. Baltimore, MD: Johns Hopkins University Press. Gardner, H. (2018). Higher education: A platonic ideal. In O. Zlatkin-Troitschanskaia, G. Wittum, & A. Dengel (Eds.), Positive learning in the age of information: A blessing or a curse? (pp. 9–21). Wiesbaden: Springer VS. Gendolla, P., & Schäfer, J. (2004). Vernetztes Probehandeln: Literatur im Zeitalter permanenter Mutabilität. Philologie im Netz, Beiheft 2, 23–34. Hansen-Schirra, S., Hofmann, S., & Nitzke, J. (2018). Acquisition of generic competencies through project simulation in translation studies. In O. Zlatkin-Troitschanskaia, G. Wittum, & A. Dengel (Eds.), Positive learning in the age of information – A blessing or a curse? (pp. 267–280). Wiesbaden: Springer VS. Karraker, M. W. (1993). Mock trials and critical thinking. College Training, 41(4), 134–137. Kidd, D. C., & Castano, E. (2013). Reading literary fiction improves theory of mind. Science, 342 (6156), 377–380. Kosslyn, S. M. (2018). Why we should teach the humanities. In O. Zlatkin-Troitschanskaia, G. Wittum, & A. Dengel (Eds.), Positive learning in the age of information: A blessing or a curse? (pp. 23–28). Wiesbaden: Springer VS. Lem, S. (1989). Literatur als Simulation möglicher Welten: Stanislaw Lem im Gespräch mit Florian Rötzer. Kunstforum, 98, Video-Special: 100. Lill, F. (2016). Angriff auf die freien Denker: Universitäten in Japan, Großbritannien und den USA verabschieden sich von ihren Geisteswissenschaften. Die Zeit, 15(2016), 1–2. Markant, D., & Gureckis, T. (2014). Is it better to select or to receive? Learning via active and passive hypothesis testing. Journal of Experimental Psychology, 143(1), 94–122. Meyer, O., Coyle, D., Halbach, A., Schuck, K., & Ting, T. (2015). A pluriliteracies approach to content and language integrated learning – Mapping learner progressions in knowledge construction and meaning-making. Language, Culture, and Curriculum, 28(1), 41–57. Meyer, O., Imhof, M., Coyle, D., & Banerjee, M. (2018). Positive learning and pluriliteracies. In O. Zlatkin-Troitschanskaia, G. Wittum, & A. Dengel (Eds.), Positive learning in the age of information: A blessing or a curse? (pp. 235–265). Wiesbaden: Springer VS. Minnameier, G. (2018). Reconciling morality and rationality: Positive learning in the moral domain. In O. Zlatkin-Troitschanskaia, G. Wittum, & A. Dengel (Eds.), Positive learning in the age of information: A blessing or a curse? (pp. 347–362). Wiesbaden: Springer VS. Oser, F. (2018). Positive learning through negative learning: The wonderful burden of PLATO. In O. Zlatkin-Troitschanskaia, G. Wittum, & A. Dengel (Eds.), Positive learning in the age of information: A blessing or a curse? (pp. 363–370). Wiesbaden: Springer VS. Pellegrino, J., & Hilton, M. (Eds.). (2012). Education for life and work: Developing transferable knowledge and skills in the 21st century. Washington, D.C.: National Research Council of the National Academies.

4 Literature, Simulation, and the Path Towards Deeper Learning

55

Rowlandson, M. (1648/1997). The sovereignty and goodness of god. In N. Salisbury (Ed.), The sovereignty and goodness of god by Mary Rowlandson: With related documents (Bedford Series in History and Culture). Bedford. (Original work published 1648). Shavelson, R. J. (2018). Positive learning in the age of information (PLATO) – Critical remarks. In O. Zlatkin-Troitschanskaia, G. Wittum, & A. Dengel (Eds.), Positive learning in the age of information: A blessing or a curse? (pp. 281–291). Wiesbaden: Springer VS. Spiegel, M., & Spencer, D. (2017). The principles and practice of narrative medicine. A narrative medicine classroom/workshop (pp. 42–53). Oxford: Oxford University Press. Wineburg, S., Breakstone, J., McGrew, S., & Ortega, T. (2018). Why Google can’t save us – The challenges of our post-Gutenberg moment. In O. Zlatkin-Troitschanskaia, G. Wittum, & A. Dengel (Eds.), Positive learning in the age of information: A blessing or a curse? (pp. 221–228). Wiesbaden: Springer VS. Winthrop, J. (1630/1994). A model of Christian charity. In R. Gottesman, L. B. Holland, & N. Baym (Eds.), The Norton anthology of American literature (Vol. 1, 4th ed., pp. 170–180). New York: Norton. (Original work published 1630). Wood, S. (2018). Small experimental action. Paragraph, 40(3), 383–398. Zlatkin-Troitschanskaia, O., Wittum, G., & Dengel, A. (2018). Editorial – About a ‘Plato’. In O. Zlatkin-Troitschanskaia, G. Wittum, & A. Dengel (Eds.), Positive learning in the age of information – A blessing or a curse? (pp. 1–6). Wiesbaden: Springer VS.

Chapter 5

Successful and Positive Learning Through Study Crafting: A Self-Control Perspective Christian Dormann and Christina Guthier

Driven by the new developments in information and communication technology rewarding environments are accessible for anyone at any time. Particularly, social networks like Facebook, Twitter, and Instagram play a vital role when it comes to promoting yourself, connecting with people, and sharing accomplishments (Wilcox and Stephen 2012). Facebook alone had 2.20 billion monthly active users in March 2018, with 1.45 billion daily active users on average (Facebook 2018). Given the current world population of about 7.64 billion people (Worldometers 2018) more than a quarter of all people in the world are active users of Facebook alone. According to Brandwatch, four million likes (i.e., positive feedback respectively a reward,) are generated by Facebook users every single minute (Smith 2018). On the one hand receiving this kind of rewards for self-presentation on social networks can increase self-esteem (Valkenburg et al. 2006; Wilcox and Stephen 2012). On the other hand social network use can also result in decreased self-control regarding, for example, spending money, mental persistence, and health (Wilcox and Stephen 2012). It can therefore be assumed that Internet use in general and social network use in particular can also distract from decent academic learning and possibly lead to negative learning (Happ et al. 2016; Schmidt et al. 2018; Schmidt et al. in this volume). When it comes to successful and therefore positive academic learning self-control is a crucial self-regulatory skill. According to Zimmerman (1990, p. 14) “selfregulated students are distinguished by their systematic use of metacognitive, motivational, and behavioral strategies; by their responsiveness to feedback regarding the effectiveness of their learning; and by their self-perceptions of academic accomplishment.” Self-regulated students seek to actively gain new knowledge and academic skills rather than just being a recipient of the content taught by their C. Dormann (*) · C. Guthier Department of Business and Economics Education, Johannes Gutenberg University Mainz, Mainz, Germany e-mail: [email protected]; [email protected] © Springer Nature Switzerland AG 2019 O. Zlatkin-Troitschanskaia (ed.), Frontiers and Advances in Positive Learning in the Age of InformaTiOn (PLATO), https://doi.org/10.1007/978-3-030-26578-6_5

57

58

C. Dormann and C. Guthier

teachers. Hence, using self-regulation strategies results in a proactive learning process. A proactive, self-regulated learning process comprises three cyclical phases (Zimmerman 2002): (1) forethought phase, (2) performance phase, and (3) selfreflection phase. The three phases of self-regulated learning are characterized by two main processes each. The forethought phase comprises task analysis (e.g., goal setting) and self-motivation beliefs (e.g., intrinsic interest). The performance phase comprises self-control (e.g., self-instruction) and self-observation (e.g., selfrecording). The self-reflection phase comprises self-judgment (e.g., self-evaluation) and self-reaction (e.g., self-satisfaction; for more details, see Zimmerman 2002). Students who engage in these processes are academically more successful (Zimmerman and Martinez-Pons 1988). Since self-control is a crucial process in the performance phase, we need to make sure that self-control skills, which seem to be threatened by the ubiquitous influence of social media, are trained at university. Additionally, self-control is regarded as a key competency these days, and it was shown to relate to both students’ academic success as well as their performance in their jobs (Stumm et al. 2010). In terms of Zimmerman (2002, p. 68), self-control “refers to the deployment of specific methods or strategies that were selected during the forethought phase.” For example, implementing alternative courses of action when obstacles prevent carrying out the intended plan, such as switching from learning math to physics if a required book is unavailable, represents a self-control strategy. Imagery, selfinstruction, attention focusing, and task strategies are key self-control learning methods (Zimmerman 2002). In the present chapter, we adopt a slightly broader perspective of self-control, and refer to self-control as the ability to behave in a desired way resulting in intended goal attainment such as finishing homework on time or graduating from university (Tangney et al. 2018).

Self-Control Strength and Capacity for Self-Control Self-control itself requires self-control strength (Muraven and Baumeister 2000). The self-control strength model by Muraven and Baumeister (2000) is based on five assumptions: First, self-control strength is needed for actions that apply self-control strategies which, for example, cause behavioral changes. Second, self-control strength is depleted when self-control is exerted. For instance, many experiments demonstrated the performance in self-control tasks is reduced if the task is preceded by another (depleting) self-control task (Muraven and Baumeister 2000). Third, selfcontrol strength is a single resource that fuels various self-control operations (e.g., emotional, cognitive). So, if there is a need for self-control in any given situation, self-control strength will be depleted and less self-control strength will be available for all kinds of following tasks that need self-control strength resulting in worse performance. Fourth, persons vary in their capacity to exert self-control (Tangney et al. 2018). Capacity for self-control is supposed to be a trait-like factor and is

5 Successful and Positive Learning Through Study Crafting

59

sometimes also referred to as trait self-control (e.g., Diestel et al. 2015). Hence, the trait-like capacity for self-control and the state-like current level of self-control strength represent different albeit related concepts. Both, high capacity for selfcontrol and high self-control strength are positively related to goal attainment, but volitional acts required for goal attainment are always fueled by the currently available self-control strength. Fifth, during times when no self-control is required, self-control strength recovers to the level of a persons’ capacity for self-control. Sixth, capacity for self-control can be increased by regularly practicing self-control tasks similar to a muscle that needs training for strengthening. Thus, engaging in self-control temporarily, and quite strongly, reduces self-control strength and permanently, and less strongly, increases capacity for self-control. Two studies examining students’ levels of self-control already showed positive outcomes of self-control on individual and interpersonal level. Wolfe and Johnson (1995) reported self-control, measured with items that imply “task-orientedness, planfulness, promptness, persistence, efficiency or effectiveness, and self-reliance.” (p. 178), as the most robust predictor of a student’s grade point average. Tangney et al. (2018) showed that undergrad students with higher levels of capacity for self-control had better grades, fewer impulse control problems (e.g., binge eating, alcohol abuse), better psychological adjustment, higher self-acceptance and selfesteem, better interpersonal relationships, more secure attachment styles, better perspective-taking, better anger management, more guilt (guilt has predominantly motivating effects, see Tangney et al. 1996), and less shame. Hence, the key question to us is: how should academic learning environments be designed to improve students’ capacity for self-control and therefore encourage students to selfregulatory learning? Creating academic environments in which a lot of self-control strength is needed for performing involves the risk that students experience a severe loss of self-control strength. According to conservation of resources theory (Hobfoll 1989), a loss of resources in general and of self-control strength in particular can cause stress symptoms. Among employees, several studies already demonstrated that performing tasks that require self-control causes stress symptoms such as exhaustion (Diestel and Schmidt 2011; Neubach and Schmidt 2008). Interestingly, these effects were mitigated or even disappeared for employees with a high capacity for self-control (Schmidt et al. 2012). Hence, there is some risk of causing stress symptoms among students by providing learning environments where self-control strength is needed for performing. Nevertheless, training of students’ capacity for self-control should be aimed for because of two reasons. First, only by executing self-control, capacity for self-control can be expanded and therefore exhausting effects of self-control tasks be avoided in the long term. Second, there are, as already mentioned, manifold positive effects of high capacity for self-control and Tangney et al. (2018) showed that there were no curvilinear effects indicating a risk of being “overcontrolled.” The more self-controlled a student was the better were the outcomes including academic performance.

60

C. Dormann and C. Guthier

Lack of Identity and Depletion of Self-Control Strength To foster academic performance, it is important to prevent unnecessary depletion of self-control strength. The organizational psychology literature suggests that selfcontrol strength is depleted if employee experiences a low sense of identity. Kehr (2004) reasoned that incongruent motives cause psychological conflicts, and their resolution requires volitional regulation (i.e., self-control), and a depletion of selfcontrol strength. In a similar vein, Schmidt and Diestel (2015) proposed that goal incongruence in terms of person-organization fit depletes self-control strength, too. We propose this also applies to a wide range of phenomena among students that involve some sort of psychological discrepancy. Such discrepancies could come in the form of, for example, lack of need-supplies fit or need satisfaction, person-study fit, demands–abilities fit, low psychological ownership, or low identification with the tasks that one has to accomplish during studies or the people one has to deal with. We refer to all these discrepancies using the term lack of identity. We propose that when students experience a lack of identity, their self-control strength is depleted and less self-control strength is available for their academic studies. To avoid low levels of self-control strength during academic studies, students should be empowered to achieve a sense of identity. For example, if they are allowed to write their seminar papers after the exams period, the likelihood that the demands (writing exams and papers) overtax their abilities is reduced. So, they could invest more self-control strength in their academic learning. However, they could still decide to write the papers during the exams period, which they probably do if this fits their demands. Empowering students to achieve a sense of identity requires autonomy to some extent. Without being allowed to make autonomous decisions, for example, on which of a set of possible topics one could write a seminar paper, achieving identity becomes less likely. At the same time, without being allowed to make autonomous decisions, there are few opportunities for self-regulation in general and self-control in particular. For instance, if there is a single, pre-determined protocol or process students have to adhere to in the case illness-related missed lessons, students would not engage in self-regulated planning and execution of their plan to deal with the problem. They might engage in self-controlling their negative emotions, but opportunities to apply other types of self-control strategies are limited.

Self-Control Strategies Kuhl (1984) identified six self-control strategies. Probably most important for academic learning are: cognitive self-control in terms of attention control (e.g., not listening to other people while reading), encoding control (e.g., selectively encode those aspects of a topic that are relevant for exams), and information-processing control (e.g., self-directed “stop”-instructions to prevent a waste of time). Further,

5 Successful and Positive Learning Through Study Crafting

61

Kuhl (1984) identified a motivational, incentive escalation control strategy (e.g., overrating the pleasure of having a chocolate after learning). Another strategy suggested by Kuhl (1984) is emotional control (e.g., self-instructing to stop worrying). Finally, he identified that people engage in environmental control, which refers to acts that aim at changing the environment. We focus on environmental control, which is an extremely powerful strategy as it may substitute or reduce the need to engage in other self-control acts. By this, environmental self-control could save self-control strength. Regarding cognitive self-control, an environmental self-control strategy may be to removing distracting stimuli from the desk. This could reduce the need to engage in attentional selfcontrol. Some environmental self-control strategies may reduce more than one cognitive self-control effort. For instance, first summarizing book chapters and then using the summary for repeatedly reading and encoding the content could reduce both information-processing self-control efforts and encoding self-control efforts. Another environmental self-control strategy, which is particularly useful to substitute motivational self-control acts, is making social commitments. For example, telling other students that one is unavailable for a party tonight because one has to learn, may reduce the need to engage in motivation control when learning during the late evening. Finally, environmental self-control may also substitute the need to engage in emotional self-control. For example, emotions are contagious and spill over between persons (Hatfield et al. 1993). Avoiding negative emotional contagion, which could lead to displeasure triggering emotional self-control, by joining or founding learning groups with other happy students represents a promising environmental control strategy. In addition to substituting or reducing the need for other types of self-control, environmental self-control could also reduce students’ possible lack of identity, and, via this second pathway, save self-control strength. For instance, changing the university could enhance person-organization fit, changing the minor could enhance person-study fit, founding learning groups could lead to more social need satisfaction, choosing among alternative courses could enhance demands–abilities fit, and making choices in general could improve psychological ownership and increase students’ identification with the tasks they have to accomplish. Environmental selfcontrol requires self-control strength, but once it is done it may reduce the need to engage in other types of more intense self-control efforts over a longer time. Thus, environmental self-control could save more self-control strength than it does require, and more of the retained self-control strength could be invested in academic learning. Previous research on students’ learning has mainly focused on self-regulation and self-control on direct academic learning activities (e.g., self-instruction etc.). This book chapter mainly adds to the literature by proposing that environmental selfcontrol is crucial for successful learning outcomes. We propose that the effects of environmental self-control are transmitted via three pathways. First, according to the self-control strength model by Muraven and Baumeister (2000), enacting environmental self-control should enhance capacity for self-control in the longer run. Second, environmental self-control should increase students’ identity. By this,

62

C. Dormann and C. Guthier

they experience fewer psychological conflicts, which would have to be resolved by investing self-control strength (Kehr 2004). Thus, environmental self-control could save self-control strength via increased identity. Third, environmental self-control could directly substitute or reduce cognitive, motivational, or emotional self-control acts, which also saves self-control strength. The amount of self-control strength saved through environmental control could then be selectively used for those selfcontrol acts that are required for important long-term goal attainments, such as successful academic learning, a healthy life style, and a happy life.

Environmental Self-Control: Study Crafting Since students’ use of environmental control has been rarely studied thus far, we borrow insights from the organizational psychology and management literature. In this literature, employees’ acts of changing their work environment to achieve some form of identity are referred to as job crafting (Wrzesniewski and Dutton 2001). Crafting behavior is thought to be beneficial to individuals because jobs or in our case study programs are usually designed without considering workers’ (or students’) idiosyncratic needs and motives (Hornung et al. 2010; Wrzesniewski et al. 2013). However, individuals strive to satisfy their needs, finding meaning in what they are doing, and try to develop and maintain their identity (Rosso et al. 2010; Dutton et al. 2010), which is unlikely to be achieved in highly standardized contexts. Job crafting is a powerful way to individualize your work according to your needs. Several studies already provided evidence that job crafting has positive effects on need satisfaction and leads to better fits, performance, motivation, and health. Regarding need satisfaction and better fits, job crafting increases intrinsic need satisfaction (Slemp and Vella-Brodrick 2014), person–job fit (Chen et al. 2014; Niessen et al. 2016; Shenavar 2017; Tims et al. 2016), and demands–abilities fit (Lu et al. 2014). Regarding performance, job crafting results in higher quality job performance (Bakker et al. 2012; Tims et al. 2014), adaptive performance (Demerouti et al. 2017), higher creativity (Demerouti et al. 2015; Lin et al. 2017), and experiencing ‘flow’ (Ko 2012). Regarding motivation, job crafting increases work engagement (Rudolph et al. 2017; Qi et al. 2016; Yang et al. 2017) as well as job satisfaction, and commitment (Dierdorff and Jensen 2017). Finally, numerous studies showed that job crafting results in better health and well-being (e.g., Lichtenthaler and Fischbach 2016; Petrou et al. 2017; Tims et al. 2013; YepesBaldó et al. 2018). However, not all types of crafting may positively affect performance. Reducing demands, which is one type of crafting, was negatively related to task and contextual performance in a study by Gordon et al. (2015), while all other types of crafting had positive effects in their study. Interestingly, a study Dierdorff and Jensen (2017) showed moderate levels of crafting were associated with dysfunctional performance-related outcomes.

5 Successful and Positive Learning Through Study Crafting

63

Fig. 5.1 Study crafting-model

Crafting can take many different forms, which largely overlap Kuhl’s (1984) and extended environmental self-control strategies (Corno 2010) in the academic learning literature. Wrzesniewski and Dutton (2001) suggested that people may physically or cognitively change their tasks and the boundary conditions. According to them, physical changes refer to changes in the structure, content, or number of tasks to be accomplished, whereas cognitive changes refer to changes in peoples’ perceptions of their tasks and contextual conditions (Bakker et al. 2012). Task-related crafting (i.e., task-control; Corno 2010) may extend to contextual conditions of the tasks (setting control; Corno 2010), including technical (e.g., adapting software) as well as organizational variables (e.g., initiating changes in work procedures), or social relations such as relations with colleagues (peer control; Corno 2010) or with supervisors (teacher control; Corno 2010), Recent studies added to the types of changes that may occur by crafting. Lyons (2008) reported self-initiated skill development could occur, and Laurence (2010) showed that contraction could also be conceived as a form of crafting; contraction aims at decreasing stimulation or reducing the complexity of the task or environment. Figure 5.1 summarizes our major propositions. Self-control demands increase student’s experienced lack of identity, such as lack of person-environment fit or ability-demands fit. A lack of fit triggers crafting behaviors. As we discuss below, however, crafting behavior occurs only if some necessary preconditions are met, which motivate and enable students doing so. Study crafting represents an environmental self-control strategy that depletes self-control strength to some degree. The net effect is positive because students then have to invest less self-control strength in cognitive, emotional, and motivational self-regulation. Thus, more self-controlstrength is retained, which could be used for all volitional processes required for academic learning. The job crafting literature suggests this will results not only in better academic achievement and creativity, but also in higher study engagement, higher commitment to their studies, and better health and well-being. Finally, because of the continued use of crafting behavior and other forms of self-control, in the long run an increase in the capacity for self-control can be expected. After finishing their studies in the information age, students are likely to encounter many situations, in which their ability for crafting and their capacity for self-control, represent more important predictors of positive occupational learning and success than the academic knowledge they had acquired. The latter has limited half-life in

64

C. Dormann and C. Guthier

many disciplines and quickly becomes obsolete, whereas the former is unlikely to become outdated.

Improving Study Crafting Encouraging students to crafting behaviors is particularly important for successful academic studies and subsequent life outside universities. We do not neglect the important role of cognitive, emotional, and motivational control strategies. Indeed, such internal volitional strategies are trainable and can be changed, too. However, this has already been dealt with in the literature (Greenberg and Rhoades 2010), and the focus on the present chapter is on environmental self-control. In the remainder of the present chapter, we therefore develop some ideas about how crafting behavior could be encouraged in higher education. There are two important starting points to increase study crafting and therefore to enable training of self-control at universities. First, the structure of study programs and study tasks needs to enable crafting behavior (allowing crafting). Second, students need to be motivated for engaging in crafting behavior (wanting crafting).

Improving Structures of Study Programs and Study Tasks: Autonomy From a structural point a crucial aspect is providing students with autonomy. We regard this issue as extremely important as changes of study conditions during recent decades has reduced students’ autonomy more and more. In parts, this was probably triggered by the Bologna reform, which resulted in highly structured courses, modules, and studies, which have been leaving declining room for flexibility and choices and thus student autonomy (Zlatkin-Troitschanskaia et al. 2017). On the other hand, the overall increase in the number of students has not been mirrored in the increase of resources of universities. This leads to the problem that sometimes even meeting the minimum requirements laid down in the accredited study programs is nearly impossible for the staff. This is another cause of decreased student autonomy during recent decades. Conceptually, it is almost trivial that without a minimum level of autonomy, crafting behavior becomes nearly impossible. For example, if it is a formal requirement to attend a lecture, students simply do not have the autonomy to decide in what different kind of study-related activity they could invest their time. When attendance would not be required, students could use their autonomy and might decide to engage in an additional course, if their knowledge, skills, and abilities (KSAs) already go beyond what is delivered by the lecture. In addition to substantive reasons to increase autonomy to improve students’ crafting behaviors, several empirical

5 Successful and Positive Learning Through Study Crafting

65

studies already showed that autonomy leads to higher levels of crafting (e.g., Leana et al. 2009; Lyons 2008; Sekiguchi et al. 2017). Autonomy could be given to students in more than one way because different types of autonomy can be distinguished (Mullarkey et al. 1997), such as in the methods of study (e.g., learning alone vs. in groups), the timing (e.g., morning vs. evening; distributed vs. condensed before examinations), the goals (e.g., mastery vs. performance), or the contents (e.g., physics vs. biology; work psychology vs. educational psychology). Regarding the contents, for example, while it is obvious that students should not decide about the basic set of contents to be learned within a discipline, they should be provided with different options within a discipline. In many contexts, this could require additional resources such as more lecturers. In other instances, it may just require re-designing a course, for example, offering students the option to deal with the topic motivation from a work psychology vs. educational psychology perspective. To provide another example regarding the methods of study, there is certainly no single best way to learn, so students should be made familiar with different methods and tryout which methods fits their capabilities best. To summarize, autonomy represents a necessary precondition of study crafting. Above and beyond, student autonomy is also likely to motivate crafting behaviors. There could be, however, some drawbacks, which we deal with in the following subsection.

Motivating Students’ Crafting Behaviors: Setting Increasingly Complex Goals Critics of student autonomy might reason that too much autonomy could undermine learning. For instance, Vecchio (1987) and Fernandez and Vecchio (1997) showed that among newly hired teachers, a highly structured leadership style, which leaves low autonomy and possibilities for crafting on the side of the teachers, leads to better performance. Positive effects of structured and directive leadership seemingly contradict the proposed positive effects of autonomy and crafting. However, the possibly negative effects are limited to the initial phases of a newly entered job or course of study. In the early stages of a course of study, ambiguity about what to do is probably large, and too much autonomy could even increase the felt ambiguity. However, we assume this becomes less likely later. The studies by Vecchio (1987) and Fernandez and Vecchio (1997) yielded no evidence that a directive leadership style is superior at later stages of the teachers’ careers. Studies on autonomy-supporting by parents and teachers also provided evidence for positive effects. A meta-analysis among young students of the effects of autonomy support by parents demonstrated that higher autonomy support is related to better academic performance, higher autonomous motivation, and psychological

66

C. Dormann and C. Guthier

functioning (Vasquez et al. 2016). Similar evidence exists for autonomy supportive teaching (e.g., Furtak and Kunter 2012). Further, autonomy has been shown to consistently related to satisfaction and motivation among employees, (e.g., in the meta-analyses by Fried 1991 and Spector 1986). Thus, in general, there is reason to assume that autonomy should probably be limited in the early phases of a study, but should then be continuously increased. A challenge for teachers or lecturers at all stages, however, is how to increase autonomy without increasing ambiguity. A promising means to avoid ambiguity is to set goals. When goals are clearly set, students know exactly what they have to do. Goal setting theory (Locke and Latham 2006) posits that performance is increased by setting specific and high (difficult) goals. Higher goals lead to greater effort and persistence, which may deplete selfcontrol strength in the short run but cause an increase in capacity for self-control in the long run. In addition to triggering self-control efforts, goals also motivate students to use their relevant KSAs thereby consolidating them. Goals may also motivate students to acquire new knowledge and capabilities (Locke and Latham 2006). Thus, goal setting has the potential to achieve a whole bunch of desired outcomes. When setting goals teachers should probably focus on what has to be achieved, and in what time frame. It is frequently not necessary to prescribe which methods have to be used to achieve these goals. Regarding the methods, there are much more opportunities to provide students with autonomy compared to the content of academic goals. Still, the challenge for teachers is to figure out what the best level for autonomy for each student or for each group of students is. This is important because combining reasonably high goals with autonomy that matches students’ KSAs creates complex tasks that are intellectually stimulating. Complex tasks are challenging because they are not easy to solve; there are many degrees of freedom due to the level of autonomy. Several studies already revealed that complex and challenging tasks are related to higher levels of crafting (e.g., Berg et al. 2010; Ghitulescu 2007). Balancing goal setting and autonomy requires an appropriate leadership style among teachers. Several studies already showed that servant leadership and transformational leadership improved crafting (Kooij et al. 2017; Wang et al. 2017; Yang et al. 2017). Servant leaders focus on serving their followers, and so do servant teachers serve their students (Greenleaf 1977; Stone et al. 2004). Among other aspects, transformational leadership comprises inspirational motivation, intellectual stimulation, and individualized consideration (Bass 1985). Inspirational motivation refers to articulating appealing and inspiring visions, for example, what students could be able to achieve with their acquired KSAs. This could enhance students’ commitment to the goals set. Intellectually stimulating are teachers who take the risk and solicit students’ ideas. Finally, individualized consideration is the core component to ensure a balance between set goals and given autonomy: attending to students’ needs, to acts as a mentor, and to have an open ear to students’ concerns and needs.

5 Successful and Positive Learning Through Study Crafting

67

Excellent leadership of teachers does not mean that teachers need to assist students with every single step of the learning process. Rather, teachers should define clear and high learning goals, that should be achieved within a reasonable time frame, but with the appropriate degree of autonomy regarding the methods students use to achieve these goals. The goals should be pursued consequently by transparently communicating how and at which point of the learning process they are going to help and where students need to autonomously decide how to learn and solve their tasks.

Summary and Conclusions We concur with other scholars that for positive academic learning self-control is a crucial self-regulatory skill (e.g., Wolfe and Johnson 1995; Zimmerman 1990). In particular, this applies to students living in the information age, where self-control skills are required not only to focus on academic studies, but also to prevent overusing social media and other Internet-based services. Negative learning in the information age possibly occurs because information found on the Internet is no longer adequately processed and validated. People got ‘hungry’ for information (surveillance gratification seeking, Maurer et al. in this volume). Working memory gets easily flooded with new information, preventing validation of previously acquired information. Self-regulation plays an important role here as it is required to control the impulse to attend to new information (e.g., veto-control) until older information is better cognitively evaluated (e.g., correctly judging the trustworthiness and usefulness; Zlatkin-Troitschanskaia et al. in this volume) and the content of sources is questioned (e.g., reflecting their objectivity and reliability). Thus, selfregulation may be an important volitional component of digital information literacy. Self-control itself requires self-control strength, which depletes individuals’ capacity for self-control in the short run when facing self-control demands (Muraven and Baumeister 2000). In the long run, however, frequently exerting self-control increase the capacity for self-control like muscle strength increases with physical exercise. A particular self-control strategy that has been rarely dealt with in the literature is environmental self-control, which leads to changes in the task itself, the context (e.g., working materials), or the social relations that are relevant for the task (e.g., peers, teachers). In the work psychology literature, this is also known as job crafting, which has been studied extensively in recent years (e.g., Demerouti et al. 2017). We propose that environmental self-control through study crafting depletes self-control strength less strongly than cognitive, emotional, or motivational self-control because once the environment is changed so that it matches ones’ aims, desires, and motives, there is no need to continue. Students could then invest their remaining self-control strength into their academic learning. This claim is supported by several empirical studies on job design, which established a wide range of positive outcomes including increased person-job fit, performance, adaptive performance, commitment,

68

C. Dormann and C. Guthier

engagement and motivation, and health and well-being. Thus, we regard study crafting as an excellent means to improve positive learning and increasing the capacity for self-control. Study crafting shall lead to various forms of positive learning. Study crafting fosters engagement through identity and improved self-regulation. Similar propositions for positive learning are also made by Banerjee (in this volume), who suggest that role identification leads to improved cognitive engagement, intrinsic motivation, and longer persistence. On the other hand, as crafting sometimes could imply reducing complexity (Laurence 2010), one could reason that crafting may hamper positive academic learning and personal development. However, as Nagels et al. (this volume) noted for the case of reduced linguistic complexity, reducing complexity reduces cognitive effort, but it provides more capacities for memory and retrieval, and, thus, overall, makes conditions better so that more academic knowledge is acquired. To facilitate study crafting, we propose that combining goal setting with increasing levels of students’ autonomy to be a promising approach. Autonomy has already been established as an antecedent of crafting behavior in the work psychology literature (e.g., Leana et al. 2009). Although high levels of autonomy may sometimes cause feelings of ambiguity and lead to decreased performance (e.g., Vecchio 1987), this is likely to be limited to contexts in which students lack knowledge about the relevant tasks and goals. Therefore, we suggested that autonomy has to be matched with goal difficulty levels, which could be achieved by transformational behavior of teachers and lecturers. The more academically mature students are, the higher can be the goals and the level of autonomy assigned to them, which implies complex and challenging goals that are intrinsically motivating and trigger further crafting behavior (e.g., Berg et al. 2010). Further, it is a well-established finding that higher goals also boost performance (Locke and Latham 2006). Although extrapolating a lot from the work psychology literature, in terms of selfregulation there are many similarities between work and learning (Hacker and Sachse 2014). Therefore, we assume that autonomy and goal setting improve crafting and self-regulation skills including the capacity for self-control in both domains. Above and beyond, crafting and self-control skills are required in many life domains, and this is highly likely to remain valid for a long time. Unfortunately, the Bologna reform has led to a decrease in students’ autonomy (ZlatkinTroitschanskaia et al. 2017). Furthermore, the increasingly underfunded university system with reduced individual contact between lecturers and students, and with a possible trend to overuse various types of digital media (e.g., Patterson 2017) instead of facetime in teaching, has made individualized consideration and intellectual stimulation of students by their teachers difficult. We expect students could nevertheless be empowered to craft their study conditions so that they become motivated, healthy, and well-educated citizens.

5 Successful and Positive Learning Through Study Crafting

69

References Bakker, A. B., Tims, M., & Derks, D. (2012). Proactive personality and job performance: The role of job crafting and work engagement. Human Relations, 65(10), 1359–1378. https://doi.org/10. 1177/0018726712453471 Bass, B. M. (1985). Leadership and performance beyond expectations. New York, NY: Free Press. Berg, J. M., Wrzesniewski, A., & Dutton, J. E. (2010). Perceiving and responding to challenges in job crafting at different ranks: When proactivity requires adaptivity. Journal of Organizational Behavior, 31(2–3), 158–186. https://doi.org/10.1002/job.645 Chen, C. Y., Yen, C. H., & Tsai, F. C. (2014). Job crafting and job engagement: The mediating role of person-job fit. International Journal of Hospitality Management, 37, 21–28. Corno, L. (2010). Volitional aspects of self-regulated learning. In B. J. Zimmerman & D. H. Schunk (Eds.), Self-regulated learning and academic achievement (2nd ed., pp. 191–225). New York, NY: Routledge. Demerouti, E., Bakker, A. B., & Gevers, J. M. (2015). Job crafting and extra-role behavior: The role of work engagement and flourishing. Journal of Vocational Behavior, 91, 87–96. https://doi. org/10.1016/j.jvb.2015.09.001 Demerouti, E., Xanthopoulou, D., Petrou, P., & Karagkounis, C. (2017). Does job crafting assist dealing with organizational changes due to austerity measures? Two studies among Greek employees. European Journal of Work and Organizational Psychology, 26(4), 574–589. https://doi.org/10.1080/1359432X.2017.1325875 Dierdorff, E. C., & Jensen, J. M. (2017). Crafting in context: Exploring when job crafting is dysfunctional for performance effectiveness. Journal of Applied Psychology, 103(5), 463–477. https://doi.org/10.1037/apl0000295 Diestel, S., Rivkin, W., & Schmidt, K.-H. (2015). Sleep quality and self-control capacity as protective resources in the daily emotional labor process: Results from two diary studies. Journal of Applied Psychology, 100(3), 809–827. https://doi.org/10.1037/a0038373 Diestel, S., & Schmidt, K.-H. (2011). The moderating role of cognitive control deficits in the link from emotional dissonance to burnout symptoms and absenteeism. Journal of Occupational Health Psychology, 16(3), 313–330. https://doi.org/10.1037/a0022934 Dutton, J. E., Roberts, L. M., & Bednar, J. (2010). Pathways for positive identity construction at work: Four types of positive identity and the building of social resources. Academy of Management Review, 35(2), 265–293. https://doi.org/10.5465/amr.35.2.zok265 Facebook. (2018). Company info. Retrieved August 4, 2018, from https://newsroom.fb.com/com pany-info/ Fernandez, C. F., & Vecchio, R. P. (1997). Situational leadership theory revisited: A test of an across–jobs perspective. The Leadership Quarterly, 8(1), 67–84. https://doi.org/10.1016/ S1048-9843(97)90031-X Fried, Y. (1991). Meta-analytic comparison of the Job Diagnostic Survey and Job Characteristics Inventory as correlates of work satisfaction and performance. Journal of Applied Psychology, 76 (5), 690–697. https://doi.org/10.1037/0021-9010.76.5.690 Furtak, E. M., & Kunter, M. (2012). Effects of autonomy-supportive teaching on student learning and motivation. The Journal of Experimental Education, 80(3), 284–316. https://doi.org/10. 1080/00220973.2011.573019 Ghitulescu, B. E. (2007). Shaping tasks and relationships at work: Examining the antecedents and consequences of employee job crafting. Doctoral dissertation, University of Pittsburgh, Pittsburg. Gordon, H. J., Demerouti, E., Le Blanc, P. M., & Bipp, T. (2015). Job crafting and performance of Dutch and American health care professionals. Journal of Personnel Psychology, 14(4), 192–202. https://doi.org/10.1027/1866-5888/a000138 Greenberg, M. T., & Rhoades, B. L. (2010). Self-regulation and executive function: What can teachers and schools do? In C. L. Cooper, J. Field, U. Goswami, R. Jenkins, & B. J. Sahakian (Eds.), Mental capital and wellbeing (pp. 377–382). Hoboken, NJ: Wiley-Blackwell.

70

C. Dormann and C. Guthier

Greenleaf, R. K. (1977). Servant leadership: A journey into the nature of legitimate power and greatness. New York, NY: Paulist Press. Hacker, W., & Sachse, P. (2014). Allgemeine Arbeitspsychologie: Psychische Regulation von Tätigkeiten (3rd ed.). Göttingen: Hogrefe. Happ, R., Zlatkin-Troitschanskaia, O., & Schmidt, S. (2016). An analysis of economic learning among undergraduates in introductory economics courses in Germany. The Journal of Economic Education, 47(4), 300–310. https://doi.org/10.1080/00220485.2016.1213686 Hatfield, E., Cacioppo, J. T., & Rapson, R. L. (1993). Emotional contagion. Current Directions in Psychological Science, 2, 96–99. Hobfoll, S. E. (1989). Conservation of resources: A new attempt at conceptualizing stress. American Psychologist, 44(3), 513–524. https://doi.org/10.1037/0003-066X.44.3.513 Hornung, S., Rousseau, D. M., Glaser, J. R., Angerer, P., & Weigl, M. (2010). Beyond top-down and bottom-up work redesign: Customizing job content through idiosyncratic deals. Journal of Organizational Behavior, 31(1–2), 187–215. https://doi.org/10.1002/job.625 Kehr, H. M. (2004). Implicit/explicit motive discrepancies and volitional depletion among managers. Personality and Social Psychology Bulletin, 30(3), 315–327. https://doi.org/10.1177/ 0146167203256967 Ko, I. (2012). Crafting a job: Creating optimal experiences at work. Dissertation Abstracts International Section A: Humanities and Social Sciences, 72(10-A), 3914. Kooij, D. T. A. M., van Woerkom, M., Wilkenloh, J., Dorenbosch, L., & Denissen, J. J. (2017). Job crafting towards strengths and interests: The effects of a job crafting intervention on person–job fit and the role of age. Journal of Applied Psychology, 102(6), 971–981. https://doi.org/10.1037/ apl0000194 Kuhl, J. (1984). Motivational aspects of achievement motivation and learned helplessness: Toward a comprehensive theory of action control. In B. A. Maher & W. B. Maher (Eds.), Progress in experimental personality research (Vol. 13, pp. 99–171). New York, NY: Academic Press. Laurence, G. A. (2010). Workaholism and expansion and contraction oriented job crafting: The moderating effects of individual and contextual factors. Doctoral dissertation, Syracuse University, Syracuse. Leana, C., Appelbaum, E., & Shevchuk, I. (2009). Work process and quality of care in early childhood education: The role of job crafting. Academy of Management Journal, 52(6), 1169–1192. https://doi.org/10.5465/amj.2009.47084651 Lichtenthaler, P. W., & Fischbach, A. (2016). Job crafting and motivation to continue working beyond retirement age. Career Development International, 21(5), 477–497. https://doi.org/10. 1108/CDI-01-2016-0009 Lin, B., Law, K. S., & Zhou, J. (2017). Why is underemployment related to creativity and OCB? A task-crafting explanation of the curvilinear moderated relations. Academy of Management Journal, 60(1), 156–177. https://doi.org/10.5465/amj.2014.0470 Locke, E. A., & Latham, G. P. (2006). New directions in goal-setting theory. Current Directions in Psychological Science, 15(5), 265–268. https://doi.org/10.1111/j.1467-8721.2006.00449.x Lu, C. Q., Wang, H. J., Lu, J. J., Du, D. Y., & Bakker, A. B. (2014). Does work engagement increase person–job fit? The role of job crafting and job insecurity. Journal of Vocational Behavior, 84(2), 142–152. https://doi.org/10.1016/j.jvb.2013.12.004 Lyons, P. (2008). The crafting of jobs and individual differences. Journal of Business and Psychology, 23(1–2), 25–36. https://doi.org/10.1007/s10869-008-9080-2 Mullarkey, S., Jackson, P. R., Wall, T. D., Wilson, J. R., & Grey-Taylor, S. M. (1997). The impact of technology characteristics and job control on worker mental health. Journal of Organizational Behavior, 18(5), 471–489. https://doi.org/10.1002/(SICI)1099-1379(199709)18:53.0.CO;2-V Muraven, M., & Baumeister, R. F. (2000). Self-regulation and depletion of limited resources: Does self-control resemble a muscle? Psychological Bulletin, 126(2), 247–259. https://doi.org/10. 1037/0033-2909.126.2.247

5 Successful and Positive Learning Through Study Crafting

71

Neubach, B., & Schmidt, K.-H. (2008). Haupt- und Interaktionseffekte von Selbstkontrollanforderungen auf Indikatoren der Arbeitsbeanspruchung [Main and interaction effects of self-control demands on indicators of job strain]. Zeitschrift für Arbeits- und Organisationspsychologie, 52, 17–24. https://doi.org/10.1026/0932-4089.52.1.17 Niessen, C., Weseler, D., & Kostova, P. (2016). When and why do individuals craft their jobs? The role of individual motivation and work characteristics for job crafting. Human Relations, 69(6), 1287–1313. https://doi.org/10.1177/0018726715610642 Patterson, M. C. (2017). A naturalistic investigation of media multitasking while studying and the effects on exam performance. Teaching of Psychology, 44(1), 51–57. https://doi.org/10.1177/ 0098628316677913 Petrou, P., Demerouti, E., & Xanthopoulou, D. (2017). Regular versus cutback-related change: The role of employee job crafting in organizational change contexts of different nature. International Journal of Stress Management, 24(1), 62–85. https://doi.org/10.1037/str0000033 Qi, Y., Wu, X., & Wang, X. (2016). Job crafting and work engagement in primary and secondary school teachers: A cross-lagged analysis. Chinese Journal of Clinical Psychology, 24, 935–938. Rosso, B. D., Dekas, K. H., & Wrzesniewski, A. (2010). On the meaning of work: A theoretical integration and review. Research in Organizational Behavior, 30, 91–127. https://doi.org/10. 1016/j.riob.2010.09.001 Rudolph, C. W., Katz, I. M., Lavigne, K. N., & Zacher, H. (2017). Job crafting: A meta-analysis of relationships with individual differences, job characteristics, and work outcomes. Journal of Vocational Behavior, 102, 112–138. https://doi.org/10.1016/j.jvb.2017.05.008 Schmidt, K.-H., Hupke, M., & Diestel, S. (2012). Does dispositional capacity for self-control attenuate the relation between self-control demands at work and indicators of job strain? Work & Stress, 26(1), 21–38. https://doi.org/10.1080/02678373.2012.660367 Schmidt, K.-H., & Diestel, S. (2015). Self-control demands: From basic research to job-related applications. Journal of Personnel Psychology, 14(1), 49–60. https://doi.org/10.1027/18665888/a000123 Sekiguchi, T., Li, J., & Hosomi, M. (2017). Predicting job crafting from the socially embedded perspective: The interactive effect of job autonomy, social skill, and employee status. The Journal of Applied Behavioral Science, 53(4), 470–497. https://doi.org/10.1177/ 0021886317727459 Shenavar, F. (2017). Assessment the mediating role of person-job fit and psychological ownership in the relationship between job crafting and job satisfaction. Journal of Psychology, 20(4), 376–392. Slemp, G. R., & Vella-Brodrick, D. A. (2014). Optimising employee mental health: The relationship between intrinsic need satisfaction, job crafting, and employee well-being. Journal of Happiness Studies, 15(4), 957–977. https://doi.org/10.1007/s10902-013-9458-3 Smith, K. (2018). 47 Incredible Facebook statistics and facts. Brandwatch. https://www. brandwatch.com/blog/47-facebook-statistics/ Spector, P. (1986). Perceived control by employees: A meta-analysis of studies concerning autonomy and participation at work. Human Relations, 39, 1005–1016. https://doi.org/10.1177/ 001872678603901104 Stone, A. G., Russell, R. F., & Patterson, K. (2004). Transformational versus servant leadership: A difference in leader focus. Leadership and Organization Development Journal, 25(4), 349–361. https://doi.org/10.1108/01437730410538671 Stumm, S., Thomas, E., & Dormann, C. (2010). Selbstregulationsstärke und Leistung. Zeitschrift für Arbeits-und Organisationspsychologie A&O, 54, 171–181. https://doi.org/10.1026/09324089/a000029 Tangney, J. P., Boone, A. L., & Baumeister, R. F. (2018). High self-control predicts good adjustment, less pathology, better grades, and interpersonal success. In R. F. Baumeister (Ed.), Self-regulation and self-control: Selected works of Roy F. Baumeister (pp. 181–220). London: Routledge.

72

C. Dormann and C. Guthier

Tangney, J. P., Miller, R. S., Flicker, L., & Barlow, D. H. (1996). Are shame, guilt and embarrassment distinct emotions? Journal of Personality and Social Psychology, 70(6), 1256–1269. https://doi.org/10.1037/0022-3514.70.6.1256 Tims, M., Bakker, A. B., & Derks, D. (2013). The impact of job crafting on job demands, job resources, and well-being. Journal of Occupational Health Psychology, 18(2), 230–240. https:// doi.org/10.1037/a0032141 Tims, M., Bakker, A. B., & Derks, D. (2014). Daily job crafting and the self-efficacy–performance relationship. Journal of Managerial Psychology, 29(5), 490–507. https://doi.org/10.1108/JMP05-2012-0148 Tims, M., Derks, D., & Bakker, A. B. (2016). Job crafting and its relationships with person–job fit and meaningfulness: A three-wave study. Journal of Vocational Behavior, 92(February), 44–53. https://doi.org/10.1016/j.jvb.2015.11.007 Valkenburg, P. M., Peter, J., & Schouten, A. P. (2006). Friend networking sites and their relationship to adolescents’ well-being and social self-esteem. Cyber Psychology & Behavior, 9(5), 584–590. https://doi.org/10.1089/cpb.2006.9.584 Vasquez, A. C., Patall, E., Fong, C. J., Corrigan, A. S., & Pine, L. (2016). Parent autonomy support, academic achievement, and psychosocial functioning: A meta-analysis of research. Educational Psychology Review, 28(3), 605–644. https://doi.org/10.1007/s10648-015-9329-z Vecchio, R. P. (1987). Situational Leadership Theory: An examination of a prescriptive theory. Journal of Applied Psychology, 72(3), 444–451. https://doi.org/10.1037/0021-9010.72.3.444 Wang, H. J., Demerouti, E., & Le Blanc, P. (2017). Transformational leadership, adaptability, and job crafting: The moderating role of organizational identification. Journal of Vocational Behavior, 100(June 2017), 185–195. https://doi.org/10.1016/j.jvb.2017.03.009 Wilcox, K., & Stephen, A. T. (2012). Are close friends the enemy? Online social networks, selfesteem, and self-control. Journal of Consumer Research, 40(1), 90–103. https://doi.org/10. 1086/668794 Wolfe, R. N., & Johnson, S. D. (1995). Personality as a predictor of college performance. Educational and Psychological Measurement, 55, 177–185. https://doi.org/10.1177/ 0013164495055002002 Worldometers. (2018). Current world population. Retrieved August 4, 2018, from http://www. worldometers.info/world-population/ Wrzesniewski, A., & Dutton, J. E. (2001). Crafting a job: Revisioning employees as active crafters of their work. Academy of Management Review, 26(2), 179–201. https://doi.org/10.5465/amr. 2001.4378011 Wrzesniewski, A., LoBuglio, N., Dutton, J. E., & Berg, J. M. (2013). Job crafting and cultivating positive meaning and identity in work. In A. B. Bakker (Ed.), Advances in positive organizational psychology (Vol. 1, pp. 281–302). Bingley: Emerald Group Publishing. http://psycnet. apa.org/doi/10.1108/S2046-410X(2013)0000001015 Yang, R., Ming, Y., Ma, J., & Huo, R. (2017). How do servant leaders promote engagement? A bottom-up perspective of job crafting. Social Behavior and Personality: An International Journal, 45(11), 1815–1827. https://doi.org/10.2224/sbp.6704 Yepes-Baldó, M., Romeo, M., Westerberg, K., & Nordin, M. (2018). Job crafting, employee wellbeing, and quality of care. Western Journal of Nursing Research, 40, 52–66. https://doi.org/10. 1177/0193945916680614 Zimmerman, B. J. (1990). Self-regulated learning and academic achievement: An overview. Educational Psychologist, 25(1), 3–17. https://doi.org/10.1207/s15326985ep2501_2 Zimmerman, B. J. (2002). Becoming a self-regulated learner: An overview. Theory Into Practice, 41(2), 64–70. https://doi.org/10.1207/s15430421tip4102_2 Zimmerman, B. J., & Martinez-Pons, M. (1988). Construct validation of a strategy model of student self-regulated learning. Journal of Educational Psychology, 80(3), 284–290. https://doi.org/10. 1037/0022-0663.80.3.284 Zlatkin-Troitschanskaia, O., Pant, H. A., Lautenbach, C., Molerow, D., Toepper, M., & Brückner, S. (2017). Modeling and measuring competencies in higher education. Wiesbaden: Springer VS.

Chapter 6

On the Relationship Between “Education” and “Critical Thinking” Klaus Beck

Preliminary Remarks The question of what “critical thinking” means aims, first and foremost, at a definition. Put in the everyday context of small talk, the answer is usually hardly uniform and even less distinct as the colloquial language leaves room for interpretation and allows for a variety of uses.1 In the scientific language one should be able to expect a clear statement of meaning. Yet, there are several reasons why this is not necessarily the case. An unbiased speaker of German will probably point out, in an effort to clarify, that “thinking” is a mental process and that there are many kinds of “thinking,” among which the variant described as “critical” is probably motivated by some kind of doubt. And to “doubt,” in turn, could mean several things such as questioning the truthfulness of a statement, or denying the evaluation expressed in it or considering the prerequisites for its creation as problematic (see also Oser and Biedermann in this volume). Be that as it may, the reason why ideas about what “critical thinking” means vary can be mainly broken down into two more technical reasons as well as one strategic reason, which requires special attention.

1 This goes so far that each individual, in principle, is completely free to attribute a meaning to a sequence of letters or sounds (or to any sign at all), be it already established in written or spoken language or newly invented—a nominalistic position (Essler 1972, p. 198ff.). Supporters of a humanities viewpoint, on the other hand, represent a (hyper-)platonistic viewpoint in these matters, which assumes that all terms of our languages, including logic operators, are to be understood as names of “real” facts, thus including also immaterial entities, and that it is therefore an empirical matter to correctly describe the meaning of a term (ibid.).

K. Beck (*) Department of Business and Economics Education, Johannes Gutenberg University of Mainz, Mainz, Germany e-mail: [email protected] © Springer Nature Switzerland AG 2019 O. Zlatkin-Troitschanskaia (ed.), Frontiers and Advances in Positive Learning in the Age of InformaTiOn (PLATO), https://doi.org/10.1007/978-3-030-26578-6_6

73

74

K. Beck

The first technical reason, in short, is that almost all terms of colloquial language are necessarily surrounded by a more or less broad scope of meaning as they must have a certain semantic openness to be able to adapt flexibly to changing conditions of use. Their general functionality in everyday understanding is due to the frequently overlooked fact that communication takes place under a concrete non-linguistic framework and that possible, unspokenly conceivable understandings of terms are limited to what is meant in the respective situation and context.2 The second technical reason is that the social sciences in particular, in their intention to investigate facts concerning individuals and society as a whole, adopt colloquial terms into their scientific language to make the relationship to everyday life visible. Even if they re-define the colloquial terminology for their purposes, the different connotations associated with it usually do not disappear completely and thus open up possibilities for scientists to understand one and the same term differently.3 In principle, misunderstandings associated with these circumstances can be avoided by agreeing on careful definitions (e.g., Savigny’s principles of definition, 1971), but again with a possibly serious limitation that cannot be ignored in our globalized world and which, as will be shown in the last section, will gain considerable importance in this context. The question is whether the—possibly even very precise—information on the meaning of a term can be fully and accurately translated into another language or compared with the meaning it might already have in that language. This is not only due to the very fundamental reservations that have to be expressed from a linguistic-philosophical point of view.4 This reservation is also connected to the fact that the continuous development of language suggests unstable associations of meaning in colloquial speech, which are also constantly changing. The third strategic aspect of the ambiguity regarding the meaning of the term “critical thinking” and similar terms is of a completely different nature. One such related term, which must come into play in this context almost inevitably and for systematic reasons, is the term “education,” because otherwise, no clarity about “critical thinking” can be gained, at least in the German-speaking linguistic area and cultural sphere. The following considerations of demarcation and coherence are primarily dedicated to this concept. Both terms, “education” (from now on in the sense of “Bildung”) and “critical thinking,” do not merely serve to describe a fact (i.e., a descriptive use), for example, in this case to describe a person’s mental constitution, although they could be used exclusively for this purpose. In fact, the pragmatic context in which they are regularly used is much broader and more serious. The—inherently harmless—definition of

For example, the agreement to meet “at the bar” works out because, during the conversation, you are just crossing the railway tracks and not sitting in front at the counter of a tavern. 3 Examples include not only “critique” and “thinking” but also “morality”, “motivation” or even “theory”. 4 See on the “untranslatability thesis” Quine (1960/1980, Chap. II) and Davidson (1984) as well as Sukale (1988); on linguistic relativity the Sapir-Whorf hypothesis, presented in Gipper (1972). 2

6 On the Relationship Between “Education” and “Critical Thinking”

75

their meanings is almost always put into a normative context at the same time: it is provided with the claim, with the expectation, with the demand even that as many people as possible, especially adolescents, should reach the state which the two terms describe by definition of meaning (normative use).5 This fact makes such definitions momentous. Yet they remain indispensable. At the same time, and paradoxically, there is a purely descriptive task associated with the normative claim regularly imposed on the two terms: that it should be possible to reliably determine whether what “critical thinking” and “education” designate and what the “commitment makers” prescribe has actually been achieved in individual cases—a diagnostic question. Obviously, this task cannot do without a preliminary definition of terms. The same is true for the determination of the normative meaning of any terms of the same nature.6 Before we focus on the two concepts, it should also be noted that the aim is not to provide legitimacy for their normative use but rather to gain (more) clarity from a diagnostic point of view. Their determining characteristics will only be connected with the claim to be measurable, at least in principle, even if a measurement method must first be developed for some conceptual characteristics. But the normative claim associated with the inclusion of a certain feature in the definition should not be completely ignored. This would be countered by the pragmatic feasibility criterion, saying that what may be required can actually be realized.7 In this respect, our definitions are not only language norms that hope for acceptance and adoption by the scientific community; they may also be understood in the sense that we would not contradict their use in a normative context.8 Nevertheless, our considerations are not aimed at gathering justifications for such claims. Rather, we take up those elements of meaning that are counted as the core of “education” and “critical thinking” in the literature—although in this context mostly with a normative intention. However, we do so from a rational-

5

The connection between descriptive and normative meaning, i.e., between definition and requirement, is established by requiring that, with regard to the denotatum of the defined term, it should be brought about, maintained, removed, or similar. In many, if not most, contexts of definitions, such an application is not readily obvious. One thinks, for example, of elementary terms from the natural sciences such as “temperature” or “volume,” but also from social sciences, for example, “socialization,” “interaction,” or even in economics, for example, “balance” or “exchange rate.” It has been criticized on occasion that all terms inherently transport a normative meaning from the very beginning as even the definition itself is nothing more than a normative statement. However, this view mixes different language levels (definitions are of a meta-linguistic nature) on the one hand and confuses the functions of language regulation and language use on the other hand. 6 This ultimately includes all teaching and learning aims as well as all overarching pedagogical goals. 7 “Ought implies capability” or “ultra posse nemo obligatur”—a bridging principle between “be and ought”, as was recently tried to justify, with good reason, in Critical Rationalism (Albert 1980, p. 76f.). 8 However, it would require an own basis of legitimation, which is not given if what we propose as a language regulation were to be demanded of a certain group of people (e.g., pupils, students, applicants, voters, office holders) as a requirement to strive for or even achieve the defined state.

76

K. Beck

reconstructive perspective, i.e., in view of securing the internal consistency of each concept. In this respect, our considerations go in part beyond what the sources already provide.

A View on the Causal Range According to the German grammar rules, nouns ending in “-ung” usually refer to both a process and a state.9 In this context, we are initially concerned with “education”/“Bildung” as a term for an individual’s state that can change as a result of external and internal influences—however, this change cannot be situational but only occurs over longer periods of time. This simultaneously implies the assumption that “education” can have a weaker or stronger form in different people, i.e., that “education” must not be considered dichotomously (i.e., “present” or “not present”). This term thus describes—both in the widespread technical as well as the colloquial understanding—a relatively stable personal characteristic that varies from person to person. This understanding is associated with some interesting diagnostic questions, the answer to which depends on how the term is defined. Without addressing the all but countless proposed definitions in the literature, it can be stated that “education”/ “Bildung” is regularly understood as an internal personal state that cannot be dimensionally isolated and observed, such as “weight,” “blood pressure,” or even a certain knowledge content. Rather, “education” should be understood as an internal constellation of characteristics that is constituted by the interaction of several components, such as: • • • •

A well-structured and broad knowledge base, The ability to cross-reference the elements of this knowledge base, To understand the according implications, To make balanced judgments based on generally and highly acclaimed and stable, values, taking into account different perspectives at the same time, • To be open to new knowledge, new experiences, new developments, • To be able to make oneself understood to the addressee, etc. Such an understanding of the concept, no matter how it is differentiated and accentuated in detail, assumes that the concept of “education” is embedded in a complex causal structure within the individual (Beck 1987), which has, as it were, a synchronic and a diachronic extension—synchronic insofar as an internal network of interactions between cognitive and affective components is involved and diachronic

9

Corresponds approximately to the suffix “-tion” in English nouns.

6 On the Relationship Between “Education” and “Critical Thinking”

“internal processing”

synchronic

Throughput Input: “perception”

77

Output: “behavior”

diachronic

time

Fig. 6.1 Causal localization of “education” as part of throughput

insofar as it functions as an instance for processing external inputs (“perceptions”) and generating emissions directed “outwards” (“behavior”) (Fig. 6.1).10 For the closer determination of a construct of this causal range, questions arise as to what its components are in detail, in which causal relation they stand to one another, how they contribute to the development of the overall construct, under which conditions they (may) change, which minimum levels they must have to enable certain (desired) developments of the overall construct, and whether they have a—possibly partly—compensatory relation to one another. An attempt to approach a clearer understanding of “education” as previously outlined can be described as reconstructive insofar as, starting from a colloquial understanding of “education,” it asks about the components or elements and their interactions that can constitute the overall construct. This presupposes a sufficiently differentiated repertoire of individual psychological constructs that justifies the prospect or expectation of ultimately identifying a useful, i.e., diagnosable, constellation of cooperating “individual instances” whose description of effect comes close to what was meant by the colloquial preconception of “education.” In view of the current state of cognitive psychology (as well as neurobiology), however, it cannot be assumed that this approach currently holds much prospect of success, even if we are already able to map comparatively complex cognitive achievements such as processes of perception quite accurately, by using this approach.

Components of “Education” as Sub-Functions of “Critical Thinking” Despite this rather pessimistic situation, since time immemorial there has been an unbroken and strong pragmatic interest in developing a concept of “education” that makes it possible to establish differences in the characteristics of individuals and— The processes thus distinguished, including those of “throughput”, can in fact all be reconstructed as temporal sequences. The proposed distinction is only made at a psychological meso-level for reasons of clarity.

10

78

K. Beck

more importantly—to determine more precisely the goal that every individual is tasked with achieving to have the possibility of a “successful life” in the context of historical, cultural, and social standards.11 The history of ideas on the definition of education is rich in attempts to “bring to the concept,” which in this sense should be regarded as desirable and should therefore be strived for in the educational influence of young generations. Some lines of this discussion may also be highly relevant with regard to “critical thinking.”

Education and Social Differentiation As early as seventeenth-century England,12 there are concepts that can be regarded as forerunners of a modern European educational concept. They express aspects that are associated with “education” up to the present day. The term “virtuoso” and the concept of “politeness” that replaced it stood for the need to have categories for differentiating social strata. Both emerged in linguistic usage after the “Glorious Revolution” (1688/89) made noble origins and class affiliation no longer functional as labels for the politically relevant groups. Group membership was now constituted by characteristics of the individual, which could each be more or less pronounced and thus still allow hierarchical differentiation within the “upper” strata—whilst still excluding the “lower” strata (Horlacher 2011, p. 23f.). With this upgrading of personal qualities, which might have been acquired through education and personal effort, everything that is later associated with “education” gains broad social relevance. The idea that an achieved status can have something to do with individually developed abilities has continued to the present day where there is talk of, for example, an “educated middle-class,” of “educated elites” or of “educational advancers” and “educational privileges”—an aspect that has remained alive in “critical thinking” as a consideration of the social dimension of problem solutions and statements.

Education and Critique Only in mid-eighteenth century did the concept of education itself find its way into pedagogical terminology (Menze 1970, p. 135), where, from the very beginning, Ironically, Niklas Luhmann writes: “The word education [“Bildung”] provides the contingency formula of the educational system with an indisputably beautiful body of words. It flows easily from the tongue” (2002, p. 187; translated from German by DeepL [https://www.deepl.com/translator] and author). 12 The relevant deliberations of Greek and Roman antiquity can be dispensed with here as they are enclosed in the modern discussions on the concept of education. 11

6 On the Relationship Between “Education” and “Critical Thinking”

79

another function has been ascribed to it, which, although temporarily lost in the nineteenth century, was revisited in the first half of the twentieth century and advanced to its core importance. According to Menze (1970) in his recapitulation of early conceptual development, education “arises from the critical distance of rational man from theology, metaphysics, the ruling social class” (p. 136) and generally “from his own time” (p. 143). It’s the Age of Enlightenment. So it says in Immanuel Kant’s famous formulation, which addresses the foundation of all critique and at the same time combines it with an educational mission for each individual: Enlightenment is man’s release from his self-incurred tutelage. Tutelage is man’s inability to make use of his understanding without direction from another. Self-incurred is this tutelage when its cause lies not in lack of reason but in lack of resolution and courage to use it without direction from another . . . Sapere aude!. (Kant 1784/1981, vol. 9, p. 53; italics in the original spaces between letters).13,14

It is the critical impetus of Kant’s philosophy that the founders of the Frankfurt Institute for Social Research took up in the early twentieth century—despite all other criticism of Kant (Reitemeyer 2012)—and the socially critical impact of which continues to be incorporated into the “theory of education” of emancipatory pedagogy.15 Even though the criticism of this branch of educational science, which dominated in the 1970s and 1980s (Horlacher 2011, p. 81ff.) and aimed at the creation of egalitarian social conditions, has now subsided, the element of critique has been preserved as a constitutive feature of the concept of education up to the present day. However, it has largely cast off its egalitarian socio-political ballast and now stands—cum grano salis—as its purified constitution for a rather purely cognitively understood facet of the current understanding of “education” and thus also of “critical thinking.”

13

Translation German-English retrieved from http://ghdi.ghi-dc.org/sub_document.cfm?docu ment_id¼3589 (2018, June 15). 14 Similarly: “The most important revolution in the interior of man is: «the outcome of his selfincurred tutelage». Instead of others thinking f o r him until then and merely imitating him or letting himself be guided by leading-strings, he now dares to move forward with his own feet on the ground of experience, if still shaking” (Kant 1798/1981, vol. 10, p. 549; translation from German by DeepL and author). 15 Thus Bernhard (2011) speaks of a “concept of liberation education” for that time: ‘Western’ populations are dependent “on the cultural industries that incapacitate them” (p. 90). “Education (provides) an early warning system regarding the mechanisms of the incorporation of capital that must not be underestimated” (p. 99).—In contrast to the original function of serving as a category for distinguishing social stratification, a normative interpretation of the concept of education as a “battle concept” (Ribolits 2011) comes into play here and stands for the exact opposite, the leveling of all social differences.

80

K. Beck

Education and Knowledge Following Humboldt’s distinction between general and specialized education (1809/ 1960–1981)16 in the second half of the nineteenth century, as previously mentioned, a narrow understanding of education became widespread that placed the knowledge stocks that the educated17 should have and have available at the center of the discussion about what education means and demands.18 This turn of events was influenced by the French encyclopedists (d’Alembert, Diderot, Jaucourt and others) on the one hand, who wanted to document the collected knowledge of their time, including the methods of critical thinking,19 and make it accessible to the people, and by the neo-humanist Winckelmann on the other hand, who in his return to Greek antiquity as the ideal of humanity demanded the renewal of knowledge about the Hellenes and thus gave the concept of education a different note of meaning, which also goes hand in hand with the claim to mastering the “old” languages and the knowledge of Greek and Roman philosophy and literature as a “humanistic education” up to the present day. For pragmatic reasons alone, which relate to the organization of the education system as a whole and the canon of subjects and curricula as “directories” of the knowledge to be acquired in particular, “knowledge” had to persevere as an element of meaning in education. It had already been canonized into the “septem artes liberales” from antiquity until the Middle Ages and was later addressed by pedagogical celebrities from Comenius to Pestalozzi and Rousseau to Herbart as an

16 This—momentous—separation of general and specialized, i.e., vocational training, still takes place today, citing a passage in the Lithuanian school plan: “There is a certain amount of knowledge that must be general, and even more a certain formation of attitudes and character that no one should lack. Everyone is obviously only a good craftsman, merchant, soldier and businessman if he is a good, decent man and citizen, enlightened according to his status, in himself and without regard to his particular profession. If the school education gives him what is necessary for this, he subsequently acquires the special ability of his profession so easily and always retains the freedom, as so often happens in life, to pass from one to the other” (von Humboldt 1809/1960–1981, p. 218). 17 Women were far from being mentioned in this context at the time. In 1900, the physician Möbius was still able to publish a paper entitled “On the physiological imbecility of women”, which by no means brought him violent opposition (Steinberg 2005). In contrast, the “Memorandum” of the “First German General Assembly of Conductors and Teachers of the Higher Girls’ Schools” of 1872 proclaims: “It is necessary to allow the woman an education equal to the spiritual formation of the man in the generality of species and interests, so that the German man is not bored by the spiritual short-sightedness and narrow-mindedness of his wife in the domestic flock and the warmth of feeling for the same stands by his side” (cited from Lange and Bäumer 1901, p. 64f.; translation from German by DeepL and author)—a sign that “pedagogy” was a considerable step ahead of “medicine” at the time. 18 Whether Humboldt himself would have accepted this claim is still controversial and must remain unanswered (Zabeck 1974). 19 Keyword “Encyclopédia”. “La double vocation de cet ouvrage est de répertorier les connaissances et les savoirs de son siècle et aussi d’ouvrir une réflexion critique, de “changer la façon commune de penser”” (Wikipédia 2018).

6 On the Relationship Between “Education” and “Critical Thinking”

81

important facet of their educational thinking (Dolch 1982).20 Until then, the concept of “education” had always included the entire personality in its social context.21 Around the middle of the nineteenth century, it lost its “contemporary component” and was reduced “to a sum of knowledge that one must know” (Menze 1970, p. 149). Thus “education” is degraded to “general education,” understood as “knowledge of factual knowledge” to be assessed in state examinations, which, although it remains purely external, helps its graduates to gain certain privileges (Lichtenstein 1971, col. 927) and has therefore been referred to as “journalistic ‘half-education,’”22 as “philistine education” and as “smug” (ibid., quoted from Ruge 1843), even as a “preliminary stage of barbarism” (ibid., quoted from Nietzsche 1873). This understanding of education is subject to a continuing distrust of ideology (ibid.). The most recent and critically discussed attempt to canonize “educational knowledge” in such a sense in Germany was probably written by Schwanitz (2001). With the emergence of the Internet, the question was raised as to whether individual knowledge acquisition is still required in view of permanent and ubiquitous availability of information online (e.g., SPIEGEL online of August 21, 2009). However, the school curriculum tradition remained unaffected by this (Tenorth 1994); indeed, since “Bologna,” it has recently also conquered European universities in the form of “module manuals.” In the meantime, there is probably, a—partly continuous and partly newly formed–“consensus on education” today that without well-structured and substantive knowledge neither “education” nor “critical thinking” is possible and is therefore indispensable as a defining as well as a norming component of both.

Education and Morality A significant—for the present context last—line of tradition in the history of ideas has also survived into our present understanding of education. It is the idea of the “moral” man, which dominates educational thinking even in antiquity and thematizes his virtue as a central element. Comenius gave morality its own place in education as “ethics” (“Sittenlehre”) in his “Didactica magna” (1627–1657/1985) and can also be found in the educational novels of Rousseau (1762/1925) and

The regulations for the study of political science in the “Kingdom of Bavaria” state, for example: “The complete course of general sciences includes the following subjects: (1) philosophy (2) elementary mathematics (3) philology (4) general world history (5) physics (6) natural history” (Döllinger 1823, p. 204). “(T)o the study of special sciences...”count as “auxiliary sciences”. . .“encyclopedia and methodology” (ibid. s: 206). However, the students were forbidden, among other things, “all deliberative meetings” (ibid. p. 219), i.e., meetings under a motto that today we would describe as “critical thinking”. 21 Their “individual position” (“Individuallage”), as Pestalozzi put it (Lichtenstein 1971, Col. 925). 22 This term reappears in a polemic by Th. W. Adorno in the first half of the twentieth century, where it is positioned against the “cultural industry” (1959/1998). 20

82

K. Beck

Pestalozzi (1781/1972). In the early Enlightenment era, as part of an overall, integrative view, it became the subject of a—at that time by no means harmoniously discussed (Blankertz 1963)–“harmony” concept as the “highest and most proportional (“proportionierlichste”) formation (of all; author) abilities to a whole“(von Humboldt 1792/1995; translation from German by author). With the emergence of neo-humanism—also in preparation by Goethe and Schiller—an understanding of “education” was finally established which would be unimaginable without the inclusion of an ethical component. This basic idea of an educational concept that is also anchored in morality lives on. And in the first half of the twentieth century, Kerschensteiner (1917/1999) and Spranger (1921) raised it to its core meaning under the guiding principle of the human being matured to morality (“Sittlichkeit”). In the Critical Theory of the Frankfurt School, “education” is subjected to politically motivated ideologization (e.g., Mollenhauer 1968; Klafki 1976) and undergoes a “secularization” in postmodernism as an obligation of the educated to constitutional basic values, the observance and fulfillment of which requires action motivated by justice coupled with civil courage, empathy and consequentialist thinking. Without any doubt, the psychological research on morality initiated by Lawrence Kohlberg has also contributed to this renewed understanding of education (Kohlberg 1981; Oser and Althof 1992).

“Education” and “Critical Thinking” Today It has long been commonplace in educational science to point out that every age has its own idea of education to develop. Theodor Litt has attempted to abolish the inevitable variability of what is to be understood by each epoch in a formulation that can since then be regarded as almost “classical”: According to this, education is “that constitution of man which enables him to put himself and his relationship to the world in order” (1954, col. 11; translation from German by author). What is once again expressed in this formula above all is the all-embracing and comprehensive claim to validity, which “education” is to express in the tradition of the German history of ideas. In this respect, the scope of the concept of “education,” which knows no limits to domains, is in almost every historically existing version23 and to this day much broader than that of the concept of “critical thinking.” In which aspects and dimensions this applies in detail depends on the more elaborated provisions that both concepts are subject to in the respective context of use. In this understanding, “critical thinking” always requires guidance, stimulation and restraining by the cognitive and emotional instances included in the comprehensive understanding of education:

23

Except for the shortening it experienced during the second half of the nineteenth century, as described above.

6 On the Relationship Between “Education” and “Critical Thinking”

83

• Motivation in the form of civil courage, curiosity and doubt against dogmatic gullibility • Restraint through empathy, respect and tolerance against inhuman recklessness • Counteracting demagogic and ideological swaggering through knowledgesaturated, rules-guided and meta-cognitively controlled thinking • The solution-oriented creative imagination against utopian and unrealistic debauchery and • The moral competence of judgment embedded in stable self-esteem against indifferent and unprincipled radicalism. With regard to the US-American discourse on “Critical Thinking” (CT), the distinction from the German term “Kritisches Denken” (KD) must be seen in the fact that it has a history of meaning that goes back longer and is more widely reflected than its specialist German-speaking counterpart. In the US, CT is more comprehensively conceptualized than in Germany (Dewey 1916, 193324; Ennis 1962; Kennedy et al. 1990; O’Flahavan and Tierney 1990; Abrami et al. 2015; Huber and Kuncel 2016).25 However, it is much narrower and has a stronger focus on its cognitive side than the German-language concept of “education.” For international comparative research, a careful preclarification of the conceptual understanding within the two traditional lines of CT and KD is indispensable. Things are further complicated by the comparative use of the (German-language) concept of “Bildung,” for which there is no equivalent in English (and probably in many other languages as well, Benner 2018, p. 10). At best in word combinations such as “[skilled and] well educated person,” “literate,” “lettered,” or “sophisticated person,” there are echoes of the German concept of education which are not able to reflect the German discussion (Ash 2006; Bauer 2006; Horlacher 2004; Løvlie and Standish 2002; Mogensen and Schnack 2010; Prange 2004). Without going into psychological detail here, “critical thinking” must be reconstructed in colloquial and technical language use as an element of meaning and function of “education,” whose “achievement”—comparable to a “module”— refers to “education” (Fig. 6.2): “education” includes the ability of “critical thinking,” but goes far beyond it in several respects. Thus, “perception” and “behavioral control” precede resp. follow as diachronically operating process stages of “critical thinking” and not to be conceptualized as constituent functional parts of this achievement. Civil courage and tolerance, respect and self-esteem as categories for attitudes in social interaction may also function as components of “education,” but they must not be accepted as constitutive of “critical thinking.” Critical thinking, meanwhile, will have to access knowledge, moral judgment, creativity and

Dewey instead still uses the term “reflective thinking”: “Active, persistent, and careful consideration of a belief or supposed form of knowledge in the light of the grounds which support it and the further conclusions to which it tends” (Dewey 1916, p. 9). 25 See the “Watson-Glaser Critical Thinking Appraisal (WGCTA)” (Watson and Glaser 1964), for which a German adaptation exists (Sourisseaux et al. 2007) and which is sometimes labeled to be the “gold standard” in measuring CT. 24

84

K. Beck

Fig. 6.2 “Critical thinking” as an element of “education”

metacognition as cognitive, and motivation and empathy as affective, independent performance instances in the overall construct of “education.” Logical thinking operates on the regions of knowledge of facts and processes, creativity produces virtual images and constellations of mentally modified reality on a trial basis, moral judgment tests, the admissibility of imagined conclusions, metacognition controls the use of cognitive “instruments,” motivation provides in particular energy for curiosity and for doubt and empathy enables the adoption of perspectives to make alternative results of “critical thinking” accessible for moral evaluation. At the core of “critical thinking” will be a weighing, alternative comparative judgment of the validity of circumstances that have become doubtful or in need of assessment, a judgment the formation of which still requires a differentiated reconstruction and operationalization (also in differentiation from “problem solving”). Irrespective of such outstanding works it must be emphasized that our current understanding of education must absorb an existential context that is characterized by the plurality of competing world interpretations, by the global relevance of social problems, by the simultaneous distribution of communicative information in the media and by comparatively rapid changes in the framework conditions for the individual execution of life. In this mixed situation, a normative concept of education must refer to basic anthropological facts that make local as well as global processes of understanding and problem solving seem possible because they can be used as genre-specific equipment. In this regard, if one follows the fundamental assumption expressed by Karl Popper (1994) with the phrase “all life (be) problem solving,” it seems obvious to base the understanding of education on the hope that

6 On the Relationship Between “Education” and “Critical Thinking”

85

man is capable of rationality. This would justify the expectation that he has a disposition, as Popper (1994) and Habermas (1981) expressed it, to bow to the better argument. If this is the case, “critical thinking” as a functional element of education would have the task of seeking, identifying and elaborating this argument—against all information and fact concealments and against all distortions and confusions of argumentation, but under consideration of and with respect for the singular contextual conditions and the legitimate plural ideas of all those involved and affected—of a “successful life” in the one world.

References Abrami, P. C., Bernard, R. M., Borokhovski, E., Waddington, D. I., Wade, C. A., & Persson, T. (2015). Strategies for teaching students to think critically: A meta-analysis. Review of Educational Research, 85(2), 275–314. Adorno, T. W. (1959/1998). Theorie der Halbbildung. In Gesammelte Schriften. Soziologische Schriften I (Vol. 8, pp. 93–121). Darmstadt: Wissenschaftliche Buchgesellschaft. Albert, H. (1980). Traktat über kritische Vernunft. Tübingen: Mohr. Ash, M. G. (2006). Bachelor of what, master of whom? The Humboldt myth and historical transformations of higher education in German-speaking Europe and the US. European Journal of Education. Research, Development and Policy, 41(2), 245–276. Bauer, W. (2006). Education, Bildung and post-traditional modernity. Curriculum Studies, 5(2), 163–175. https://doi.org/10.1080/14681369700200012 Beck, K. (1987). Allgemeinbildung als Objekt empirischer Forschung–Methodologische Aspekte der Gegenstands- und Begriffskonstitution. In H. Heid & H.-G. Herrlitz (Hrsg.), Allgemeinbildung. Beiträge zum 10. Kongreß der Deutschen Gesellschaft für Erziehungswissenschaft (pp. 41–49). Zeitschrift für Pädagogik. 21. Beiheft. Weinheim: Beltz. Benner, D. (2018). Der Beitrag der Erziehungswissenschaft zur Bildungsforschung, erörtert aus der Perspektive der Allgemeinen Erziehungswissenschaft und der Erziehungs- und Bildungsphilosophie. Erziehungswissenschaft. Mitteilungen der Deutschen Gesellschaft für Erziehungswissenschaft, 29(56), 9–18. Bernhard, A. (2011). Elemente eines kritischen Begriffs der Bildung. In B. Lösch & A. Thimmel (Eds.), Kritische politische Bildung. Ein Handbuch (pp. 89–100). Schwalbach: WochenschauVerlag. Blankertz, H. (1963). Berufsbildung und Utilitarismus. Problemgeschichtliche Untersuchungen. Düsseldorf: Schwann. Comenius, J. A. (1627–1657/1985). Große Didaktik (6th ed.). Stuttgart: Klett-Cotta. Davidson, D. (1984). Wahrheit und interpretation (inquiries on truth and interpretation). Frankfurt: Suhrkamp. Dewey, J. (1916). Democracy and education: An introduction to the philosophy of education. New York, NY: Macmillan. Dewey, J. (1933). How we think (2nd ed.). Boston, MA: DC Heath. Dolch, J. (1982). Lehrplan des Abendlandes. Darmstadt: Wissenschaftliche Buchgesellschaft. Döllinger, G. (1823). Repertorium der Staats-Verwaltung des Königreichs Baiern. In Unterricht und Bildung (Vol. IV, 2nd ed.). München: Fleischmann. Ennis, R. H. (1962). A concept of critical thinking. Harvard Educational Review, 22(1), 81–111. Essler, W. K. (1972). Analytische Philosophie I. Stuttgart: Kröner. Fisher, A. (2001). Critical thinking. An introduction. New York, NY: Cambridge University Press. Gipper, H. (1972). Gibt es ein sprachliches Relativitätsprinzip? Untersuchungen zur Sapir-WhorfHypothese. Frankfurt: Fischer.

86

K. Beck

Habermas, J. (1981). Theorie des kommunikativen Handelns. Frankfurt: Suhrkamp. Horlacher, R. (2004). Bildung–A construction of a history of philosophy of education. Studies in Philosophy and Education, 23(5–6), 409–426. Horlacher, R. (2011). Bildung. Bern: Haupt. Huber, C. R., & Kuncel, N. R. (2016). Does college teach critical thinking? A meta-analysis. Review of Educational Research, 89(2), 431–468. Kant, I. (1784/1981). Beantwortung der Frage: “Was ist Aufklärung?”. In Kant Werke (Vol. 9, pp. 53–61). Darmstadt: Wissenschaftliche Buchgesellschaft. Kant, I. (1798/1981). Anthropologie in pragmatischer Absicht. In Kant Werke (Vol. 10, pp. 397–690). Darmstadt: Wissenschaftliche Buchgesellschaft. Kennedy, M., Fisher, M. B., & Ennis, R. H. (1990). Critical thinking: Literature review and research needed research. In L. Idol & B. F. Jones (Eds.), Educational values and cognitive instruction: Implications for reform (pp. 11–40). New York, NY: Routledge. Kerschensteiner, G. (1917/1999). Das Grundaxiom des Bildungsprozesses und seine Folgerungen für die Schulorganisation. Heinsberg: Dieck. Klafki, W. (1976). Aspekte kritisch-konstruktiver Erziehungswissenschaft. Weinheim: Beltz. Kohlberg, L. (1981). The philosophy of moral development. In Essays on moral development (Vol. 1). New York, NY: Harper & Row. Lange, H., & Bäumer, G. (Eds.). (1901). Handbuch der Frauenbewegung (Vol. 1). Berlin: Moeser. Lichtenstein, E. (1971). Stichwort “Bildung”. In J. Ritter (Ed.), Historisches Wörterbuch der Philosophie (Vol. 1, pp. 921–937). Darmstadt: Wissenschaftliche Buchgesellschaft. Litt, T. (1954). Naturwissenschaft und Menschenbildung (2nd ed.). Heidelberg: Quelle & Meyer. Løvlie, L., & Standish, P. (2002). Introduction: Bildung and the idea of a liberal education. Journal of Philosophy of Education, 36(3), 317–340. Luhmann, N. (2002). Das Erziehungssystem der Gesellschaft. Frankfurt: Suhrkamp. Menze, C. (1970). Stichwort “Bildung”. In J. Wehle & G. Speck (Eds.), Handbuch pädagogischer Grundbegriffe (Vol. I, pp. 134–184). München: Kösel. Mogensen, F., & Schnack, K. (2010). The action competence approach and the ‘new’ discourses of education for sustainable development, competence and quality criteria. Environmental Education Research, 16(1), 59–74. https://doi.org/10.1080/13504620903504032 Mollenhauer, K. (1968). Erziehung und Emanzipation. Polemische Skizzen. München: Juventus. Norris, S., & Ennis, R. (1989). Evaluating critical thinking. Pacific Grove, CA: Midwest Publications. O’Flahavan, J. F., & Tierney, R. J. (1990). Reading, writing, and critical thinking. In L. Idol & B. F. Jones (Eds.), Educational values and cognitive instruction: Implications for reform (pp. 41–64). New York, NY: Routledge. Oser, F., & Althof, W. (1992). Moralische Selbstbestimmung. Stuttgart: Klett-Cotta. Pestalozzi, H. (1781/1972). Lienhard und Gertrud I. In A. A. Steiner (Ed.), Ein Buch für das Volk. Werke (Vol. 1). Zürich: Consortium. Popper, K. R. (1994). Alles Leben ist Problemlösen. Über Erkenntnis, Geschichte und Politik. Darmstadt: Wissenschaftliche Buchgesellschaft. Prange, K. (2004). Bildung: A paradigm regained? European Educational Research Journal, 3(2), 501–509. Quine, W. v. O. (1960/1980). Wort und Gegenstand (word & object). Stuttgart: Reclam. Reitemeyer, U. (2012). Das Verhältnis der Kritischen Theorie zur Philosophie Kants. Retrieved July 19, 2018, from https://www.uni-muenster.de/imperia/md/content/ew/forschung/feuerbach/ das_verh__ltnis_der_kritischen_theorie_zur_philosophie_kants.pdf Ribolits, E. (2011). Bildung–Kampfbegriff oder Pathosformel: Über die revolutionären Wurzeln und die bürgerliche Geschichte des Bildungsbegriffs. Wien: Löcker. Rousseau, J. J. (1762/1925). Emile. Bielefeld: Velhagen & Klasing. Schwanitz, D. (2001). Bildung. Alles, was man wissen muß. Frankfurt: Eichborn. Sourisseaux, A., Felsing, T., Müller, C., Stübig, S., Schmücker, J., Heyde, G., et al. (2007). Watson Glaser critical thinking appraisal. Frankfurt: Harcourt Test Services.

6 On the Relationship Between “Education” and “Critical Thinking”

87

SPIEGEL Online. (2009, August 21). Internet als Bildungsplattform. Wie Wissen zu globaler Klugheit wird. Retrieved July 23, 2018, from http://www.spiegel.de/wissenschaft/natur/inter net-als-bildungs plattform-wie-wissen-zu-globaler-klugheit-wird-a-644007.html Spranger, E. (1921). Lebensformen. Geisteswissenschaftliche Psychologie und Ethik der Persönlichkeit (2nd ed.). Halle: Niemeyer. Steinberg, H. (Ed.). (2005). “Als ob ich zu einer steinernen Wand spräche.” Der Nervenarzt Paul Julius Möbius. Eine Werkbiografie. Huber: Bern. Sukale, M. (1988). Denken, Sprechen und Wissen. Logische Untersuchungen zu Husserl und Quine. Tübingen: Mohr. Tenorth, H.-E. (1994). “Alle alles zu lehren”. Möglichkeiten und Perspektiven allgemeiner Bildung. Darmstadt: Wissenschaftliche Buchgesellschaft. von Humboldt, W. (1792/1995). Ideen zu einem Versuch, die Grenzen der Wirksamkeit des Staates zu bestimmen. Stuttgart: Mogensen Reclam. von Humboldt, W. (1809/1960–1981). Bericht der Sektion des Kultus und Unterrichts an den König. In A. Flitner & K. Giel (Eds.), Werke in fünf Bänden (Vol. IV, pp. 210–238). Darmstadt: Wissenschaftliche Buchgesellschaft. von Savigny, E. (1971). Grundkurs im wissenschaftlichen Definieren (5th ed.). München: dtv. Watson, G. B., & Glaser, E. M. (1964). Test manual: The Watson Glaser critical thinking appraisal. San Antonio, TX: Psychological Corp. Wikipédia. (2018). Heading “encyclopédia”. Retrieved from June 14, 2018, from https://fr. wikipedia.org/wiki/Encyclop%C3%A9die#Dictionnaire_et_ encyclop%C3%A9die Zabeck, J. (1974). Allgemeinbildung und Berufsbildung–Über den Widersinn der Restauration eines Gegensatzes mit der Absicht, ihn zu überwinden. In H. Schanz (Ed.), Grundfragen der Berufsbildung (Vol. 1, pp. 41–56). Stuttgart: Holland + Josenhans.

Chapter 7

A Three-Level Model for Critical Thinking: Critical Alertness, Critical Reflection, and Critical Analysis Fritz K. Oser and Horst Biedermann

And don’t criticize What you can’t understand. . . (From Bob Dylan: The times they are a-changin’)

Introduction: Three Levels of Critical Thinking In his famous essay “On Inequality” the philosopher H. G. Frankfurt starts with a statement of President Obama’s State of the Union address. Obama mentions that income inequality would be the defining challenge of our time. Frankfurt answers: “It seems to me, however, that our most fundamental challenge is not the fact that the incomes of Americans are widely unequal. It is, rather, the fact that too many of our people are poor” (2015, p. 3). This example is a prototype of critical thinking (CT). Frankfurt does not say that president Obama is lying or faking facts, rather he emphasizes three other things: (a) With respect to justice, there are some criteria of reversibility and autonomy that must be taken into account, (b) Justice is not a question of taking something away from someone by force and giving it to the poor; this would produce new and immediate injustice, (c) Any moral authority must be questioned “given the fact of reasonable pluralism, citizens cannot agree on any moral authority, whether it is a sacred text or institution. Nor do they determine the order of moral values or the dictates of what some regard as natural law” (Rawls 1993, p. 97). This example elicits CT with respect to justice. The level is CT as analysis, the most complex and deepest level. For this level a critical thinker needs (a) knowledge about a specialist field, in this case on ethical theories, knowledge of an explicit F. K. Oser (*) Department of Educational Science, University of Fribourg, Freiburg, Switzerland e-mail: [email protected] H. Biedermann University of Teacher Education St. Gallen, St. Gallen, Switzerland e-mail: [email protected] © Springer Nature Switzerland AG 2019 O. Zlatkin-Troitschanskaia (ed.), Frontiers and Advances in Positive Learning in the Age of InformaTiOn (PLATO), https://doi.org/10.1007/978-3-030-26578-6_7

89

90

F. K. Oser and H. Biedermann

subject matter, and (b) procedural analytical knowledge (deduction, inclusion, induction, deconstruction, comparison, etc.). As a prototypical example we can see a journalist who is an expert for music critique in a newspaper. To be accepted he/she must have expert knowledge and use it critically. A second example: A representative of the liberal party holds an address saying that the American defense industry has sold weapons for 10 billion $ to foreign countries, thus supporting war and aggression around the whole world—this should be stopped. A respondent who has checked the issue on the Internet counters that most weapons were sold for protecting people, children, families, or for supporting our interests in foreign countries or for helping to foster democratic thinking in suboptimal areas. In this second example, we find a different form of CT, namely an immediate reflection of information given societal situation, a control of it from the same point of view of the speaker or from the opposite side, a framed questioning of what has been stated. This represents the second level of CT, a questioning of a concept or of a point of view or of a conviction from a general reflection standpoint. It is generic. The topic can be any societal issue that becomes ethically, politically, socially, or religiously framed. Persons doubt not only the validity of information or the validity of its sources but they question the societal, ethical, religious, etc. connectedness of it. For being critical on this second level, we do not need first a specialists’ knowledge but moral, political, social, etc. engagement. (e.g., for discussing the death penalty or the abortion issue, medical knowledge is secondary). Finally, a third example: A young couple negotiates a health insurance they are to sign. In that moment one of them asks for being able to wait one more day and thus for postponing the final signature. They want to study the small print again to discuss the offer with friends, colleagues, and parents and to make sure that they are able to meet the demands of the insurance company. This is a prototype for critical alertness. It is a different level of CT: The involved persons have to balance epistemic doubt with epistemic trust and thus search for a cognitive equilibrium that helps to make a decision. A sense of critical alertness has a motivational character. It drives the subject to check what has been done from a functional point of view. It can be developed by asking about what is necessary in order not to fail. In the philosophical area, this is called “skepsis” (the skeptical eye, see Marquard 2007). So far, we have demonstrated that there are different levels of CT. Critical analysis, the deepest level, is an expert-oriented doing. The second level, critical reflection, is a basic attitude that must be taken into consideration if (new) information is questioned to be true or false, reliable or not reliable, moral or immoral, etc. It is necessary for every responsible member of a society (Zlatkin-Troitschanskaia et al. 2019). It is generic. The third level we call critical alertness. It states that people are more or less motivated and ready to take a step back and reconstruct and question any own or foreign thinking result from a skeptical point of view. A holistic approach to and a test of CT should take into consideration all three levels. To be precise, we distinguish

7 A Three-Level Model for Critical Thinking

91

1. CT as alertness 2. CT as immediate reflection 3. CT as analysis. In addition to these levels, each CT-act can have mostly more than one target, it can be biological and religious, economical and moral, technical and political, chemical and moral, etc. Hence, it includes, besides CT within the field (first level), different aspects, which are demanded by societal expectations (second level). Not only the logic of the field and its demanded reliability is important but also societal expectations. Furthermore a framing and a check-up in considering the 7 Social Domains (SDs) are at stake. We understand the following domains by it: namely: (1) moral thinking, where justice is central; (2) the relationship domain, where the form of human interaction dynamic is questioned; (3) the political judgment, where public power is controlled; (4) the judgment of professional ethos, where a certain form of engagement and accountability is questioned; (5) religious faith and judgment, where the relationship to an ultimate being is questioned; (6) esthetics, where the free expression is questioned; and (7) conventions, where cultural rules of nicety and common communicative sense are demanded. In these domains, the basic question is that of Kant’s “self-inflicted dependency” (selbstverschuldete Unmündigkeit), unlike that of truth and reliability of information as normally reflected with respect to news, to the scientific community or daily orientation-knowledge. When these domains are touched, CT must take into consideration (a) a completely different questioning, (b) different criteria (norms, standards, and mindsets), and (c) a different methodology. It is expected on the second level. For this, we can learn from developmentalists and philosophers.

Critical Thinking in the USA and in Europe: A Critical Approach Critical Thinking (CT) according to John Dewey is a skill that everybody needs for organizing and ordering his/her own thinking and the thinking and judgments of others (Glaser 1941; Dewey 1925; Abrami et al. 2015; Huber and Kuncel 2016; ERIC 1990; Hyslop-Margison 2003). If an idea, a concept, or any information persists after our critical analysis, we can say that it has become part of our truth, that it is more secure, or that it is functionally better protecting our beliefs. Many scientific statements about CT do not differentiate between general thinking processes, a kind of purposeful thinking, and criticism based on specific questioning processes and framed by one or more of the three mentioned levels. In the first case, we are trying to find the best way to get from Paris to London. This is goal-oriented thinking, not critical thinking. In the second case, we are wondering why someone suddenly has a breakdown. This is not critical thinking, but rather epistemic astonishment. In the third case, someone is searching the right formula for a mathematical problem. This is not critical thinking, but perhaps it is creative thinking. Deduction,

92

F. K. Oser and H. Biedermann

induction, and logical conclusion are not forms of critical thinking, but rather logical thinking. Problem solving is not critical thinking, but trial and error. All these forms of thinking and these logical technics are not per se CT, but all these techniques can be used critically. Then they receive a different value. From this perspective, the definition by Scriven and Paul (1987), does not really help: Critical thinking is the intellectually disciplined process of actively and skillfully conceptualizing, applying, analyzing, synthesizing, and/or evaluating information gathered from, or generated by, observation, experience, reflection, reasoning, or communication, as a guide to belief and action. In its exemplary form, it is based on universal intellectual values that transcend subject matter divisions: clarity, accuracy, precision, consistency, relevance, sound evidence, good reasons, depth, breadth, and fairness (p. 766, see also Scriven and Paul 1996).

Facione (2004, cited in Walter and Leschinky 2007, p. 9) also terms CT as “inference, explanation, interpretation, evaluation, analysis, self-regulation”. Both definitions aim more at general thinking than at specifically CT; we do not see the specific critical agency in it. Here a—or even the—fundamental hiatus in the discussion about CT comes up: Different authors see different characteristics in this construct—some see CT, for example, as adequate, complex, deductive, inductive, fast and slow, logical, finetuned, creative, clear, definition-oriented thinking. Others, see it rather as questioning and evaluating. This is why Walter and Leschinsky (2007) state that no single definition is used for CT, yet people use it for all kinds of rational arguments. Thus, if we take seriously the definition of Scriven and Paul and the description of Walter and Leschinsky, CT is—except for evaluation—just average thought. It comprises all forms of thinking and is not distinguishable from any form of building knowledge and reflection. Walter and Leschinsky state that in the USA, traditional CT is just appropriate thinking, but not CT. The same holds true for Astleitner (1998), who edited one of the few books on CT in the German area. But he rather belongs to the American school of thought on the matter and even developed a method for making students and teachers “critical.” Exercises by Astleitner include linguistic components. Here, we find differences between sentences, statements and propositions, conjunction recognition, disjunction recognition, conditional clauses, contradictions, opposites and consistencies, simple statements, premises and conclusions, etc. Furthermore, there are exercises for arguing and drawing conclusions, reconstructing and assessing arguments, recognizing mistakes in arguing, thinking inductively, hypothesizing, generalizing and reasoning statistically, and finally, exercising bundles on the range of values. These are all examples of techniques for thinking, referring to the acquisition of coherent, logical, and complex thinking in general. In this respect, they are prerequisites for CT, but not yet CT. One could also speak of clever thinking. Actually—as stated—only the evaluation of arguments (Astleitner 1998, p. 74ff.) is really CT. All in all, Astleitner presents a mixture of adequate, correct, and complex thinking on the one hand and real evaluative CT on the other. And back to Scriven and Paul: They stated that CT “entails the examination of those structures or elements of thought implicit in all reasoning: purpose, problem, or question-at-

7 A Three-Level Model for Critical Thinking

93

issue; assumptions; concepts; empirical grounding; reasoning leading to conclusions; implications and consequences; objections from alternative viewpoints; and frame of reference”. This is quite different to the above-mentioned definitions. If CT is like thinking in general, then we do not need a special concept; everything would be tautological. But if we speak about examination, control, prove, questioning, testing the limits, etc., the notion of CT makes sense. It becomes a useful philosophical meta-concept with respect to inclusion, differentiation, etc. But we also accept: General thinking technics can be used in two forms. The first form is, for instance, deduction in general. To understand Kant’s categorical imperative it helps to understand step by step its basic preconditions. The second form is different from this. Here, a philosopher may use the same technique of deduction for something else, namely for an examination of a construct or a concept. The examination means to look critically to it. Hence techniques like deduction, etc. are not per se CT, but they can be used as such. This relates to a tradition that will be more strongly represented in the European area. Here CT entails a structured or unstructured examination of information or a concept or an idea. Questions of a critical thinker are • • • • • • • •

Is this information, concept, or theory adequate? Is this information, concept, or theory valid and truthful? Does this information, concept, or theory produces fear and insecurity? Can I see the opposite of this information, concept, or theory? (Has this theory been falsified enough?) Where are the limits of this piece of work? Who decides about the relevance of it? How does this information, concept, or theory relate to societal justice? Why do religious persons reject the permission of this technique?

In this tradition, CT is not just thinking; it rather demonstrates how to contest the rightness (or falseness), the usefulness (or uselessness), and the value (the valuelessness) of any product or construct human thinking produces and it must be related to some of the seven social domains. But even this is not enough. Additionally we must take into consideration: Being a critical thinker involves more than cognitive activities such as logical reasoning or scrutinizing arguments for assertions unsupported by empirical evidence. Thinking critically involves us recognizing the assumptions underlying our beliefs and behaviors. It means, we can give justifications for our ideas and actions. Most important, perhaps, it means, we try to judge the rationality of these justifications. We can do this by comparing them to a range of varying interpretations and perspectives. We can think through, project, and anticipate the consequences of those actions that are based on these justifications. And we can test the accuracy and rationality of these justifications against some kind of objective analysis of the ‘real’ world as we understand it (Brookfield 1987). And further questions (in addition to the above ones) would be • What are the basic assumptions of a statement and does it fit the justification of a given argument if it is attacked?

94

F. K. Oser and H. Biedermann

So far, we differentiate between general thinking (learning about something or someone) and CT (questioning something or someone) and its embeddedness within social domains. Some preconditions for the last one are, on one hand, examination and doubt, and, on the other, to put oneself in the shoes of others, see with the glasses of others, and then restart to examine the information. The second and third levels of CT are central for this. In Europe, CT can be found above all in connection with philosophical thinking, and this primarily includes criticism of society, religion and ideology, criticism of systems such as schools, the military, capitalism, criticism of systems of thought, theories and concepts of norms, etc. Important representatives are Kant (1999), The Critic of the Pure Reason, Adorno (1957/1958), the Theory of Knowledge and Critique of Knowledge, consciously juxtaposed to be able to relate one to the other, and Popper (1934), Fallibilism. Here, CT is not just an individual demand, it represents an institutional norm: According to Popper, critique, if you will, is a social institution; publicity and openness are part of this inter-subjective cooperation, and only from it can objectivity arise—but never from the isolated effort of an individual ‘to be objective’. . . According to Popper, a critical method is therefore possible to the extent that each criticizing authority is open to criticism (Schäfer 1988, p. 71; translation by the authors).

Critical Thinking and Domain Specificity Above we presented three levels of CT. Here, with respect to a strong philosophical tradition, we have to question if CT is easier to realize within a domain than generic and domain comprehensive. Within a certain domain CT is normal; students on the tertiary level learn—after having absorbed—to question narratives, theories, definitions, techniques, concepts, procedural rules; and as these are part of their academic domain they already have a basic understanding of the inner logic of this domain. But what about the generic part where CT is seen rather as a disposition (second level)? This question—as Abrami (2015, p. 280f.) shows—was posed already by Glaser (1941, p. 175) who speaks about “being disposed to consider in a thoughtful way the problems and subjects that come within the range of one’s experience”. Abrami et al. (2015, p. 280) conclude: Some 70 years after Glaser, there is little consensus about whether CT is a set of generic skills that apply across subject domains (engineering, arts, science) or depends on the subject domain and context in which it is taught.

In addition, Abrami et al. (2015) show that, on the one hand, psychologists like Woolfolk (1998), and, on the other hand, philosophers like Ennis (1989), Siegel (1988), Grovier (1985), rather view CT as skills (or disposition) applicable in different contexts and to different domains, whereas McPeck (1981), cited again in Abrami et al. (2015), believes that all thinking is thinking about something and that CT is fully related to a subject matter. He argues, that CT is not an object in itself, but related to a distinct subject; otherwise it would be an empty concept.

7 A Three-Level Model for Critical Thinking

95

More lucidity is needed for general or generic CT. Here, Adorno (1957) gives us some hints. In his lectures 1957/1958, he states that CT means “decision” on which epistemic awareness is valid and important. He says, in particular, that non-critical awareness is just awareness without reflection about truth, meaning, importance, quality, reversibility, freedom, etc. That is why we named the second level of CT Critical Reflection. Stressing the second more generic level, we know, that often both go together, for example, surgery and accountability, teaching and ethos of the teacher, architectural skill and political will.

Some Basics for Critical Thinking and Epistemic Respect CT means to question and to analyze the situational aspects, the transformational dynamics, and the limits of information whatever form it takes and relate it to some of the 7 SDs. However, how can we do this with high respect for the other, without being cynical, without destroying what others want to communicate to us? There are the following basic conditions for CT: • An object, a process, a product, an argument, a problem solution, an information, or a theory (a construct) is just given, is there, or has been produced or is delivered to us through communication or is hidden by people in power, etc. Our first assumption is that the given construct is—even if we have high respect for it—not complete (limited), partly mistaken, not public enough, constitutes something unclear, or even false (sense of critical alertness). • The second condition is that none of these hunches can be proved without understanding the construct. Like being a reviewer of a scientific article of an A-journal, we must try on the first level to understand before taking a critical stance. Even on the second level, without understanding there is no criticism. (Sometimes these two processes go together, but we should always be aware of it; or sometimes the information is very general so that we already have an implicit critical stance). • CT needs a careful distance, a kind of deconstruction and reconstruction of the construct. Procedurally, we should be concerned that CT has a solid method (a good example is the falsification process). • Since a construct in all its mentioned forms is seldom totally (like in the falsification process of a theory) mistaken, our task is thus to see right and lucid parts of it (We can, for instance, stress that this information is central for nurses but it is not complete. • It is never enough to question if the respective construct in its different forms is true or is valid. We must say from which standpoint (mindset) and from which norms we argue. This standpoint or these norms can be—as Popper mentioned it—criticized themselves. This standpoint is always a particular point of view.

96

F. K. Oser and H. Biedermann

• The mindset, which we use for our CT, can be of high complexity, reversibility, autonomy, density, vitality, or can be even an alternative theory (information). These questions, if taken into consideration, prevent from cynical critical behavior. It would be arrogant to just criticize without recurring to these pre-conditions. In a basic mental act, humans have the task to believe what the other says, states, writes, argues. We call this basic epistemic trust. With this consideration, we think that the demand to be critical needs high sensibility not to destroy. On the one hand, we know that truths are never negotiable, on the other hand, we learn to accept what our mother and father told us as being true and important. To say that something is bullshit (see the philosopher Frankfurt “On bullshit”) presupposes a basic acceptance of the one who criticizes, and vice versa. CT and self-centered attitudes often go together. Cynical remarks after having proved the falseness of the other’s argument, the other’s fake information of a tweet or a primitive under-complex social belief are unnecessary. To find out about the weakness of what we believe does not give us the right to tease or disregard the other as a person and does not give us the right for antisocial behavior and slight. We rather propose the line of the concept of moral disagreement that states that even if we do not accept the conviction of the other, seeing it as morally disqualified, we still have to respect his/her person and his/her judgment (Gutman and Thompson 1996). Even if we accept that the fallacy of correcting the other is a constitutive part of CT, it has to be acceptable and transformative by the others’ thinking load. CT as thinking and acting must lead to responsibility. That is why we speak about a sense of critical prudence. Its generic roots demand a kind of carefulness, of alertness with respect to how we present and deal with information, and how we create new thinking patterns. Critical alertness is something else than critical thinking and critical thinking is something else than critical analysis (see also Muis et al. 2018).

Critical Thinking in Connection with the 7 Social Dimensions CT often relates field-specific knowledge to moral, ethical, social, or religious mindsets. How does this work? We consider the following examples: • A member of the National Security Committee states: We do fear, that with an increase of migrants the unemployment rate in our country raises. • An urologist writes in the newspaper that he believes that it is immoral to transplant prostates because of a zero chance to succeed. • An art student rejects to see Picasso as one of the greatest artist of the 20th century because she does not believe that his technique was precise and that he was never a socially engaged human being. • A young couple states that they do not want to celebrate their marriage religiously because they do not believe in God, eventhough their parents would like them to.

7 A Three-Level Model for Critical Thinking

97

What does CT mean in these cases where some of the 7 SDs are combined with evidences, factual or scientific information?—Behind these kinds of statements we do discover personal convictions, beliefs, interests, feelings, and we cannot ask about truth or falseness, reliability or non-reliability. Interests cannot be true or false, they can only be evinced, presented, and coordinated with other interests. Convictions cannot be true or false, they can be more or less flexible and reasonable. This is especially seen within the psychology of negotiation. There, interests must be differentiated from positions proclaiming power (Fisher 2013). Behind interests humans rely on basic values like relatedness, financial security, autonomy, competence, and professional recognition. Behind convictions there are life experiences, which cannot be true or false but authentic or less authentic. This makes CT within these domains quite difficult. For any judgment, we need a different mindset than truth or falseness. One of the simplest questions one may ask is (a so-called type 1 question): What does this information tell us. All mentioned examples are justifiable through the credibility of the information and the validity of the sources (see also Berliner in this volume). All four we do encounter with openness and trust. We can also question (type 2): (a) Is this belief justified, or (b) do this motivation and other emotional states (e.g., a strong self-concept, religious belief) make sense, or (c) is this interest or attitude convincing. Beliefs, motivational and emotional issues, and interests can be more or less reversible, complex, and autonomous. They can impede or foster progress. They can be situationally adequate, present, or not present. They can be weak or strong. They can be negative (e.g., aggressive) or positive (e.g., happiness-supporting), transparent, or hidden. Thus, with such arguments CT means looking if they impair, detract, or debase something or someone. They are not justifiable, but “judgable” as worthy or unworthy or appraisable. CT means here a seeing of all these fragile elements. Finally, we can ask (type 3): (a) Is this caring behavior good or bad in the sense of societal expectations, (b) does this justice act help, (c) does this truthfulness display more transparency. Virtues cannot be true or false; morality cannot be true or false. We can have too less, too much, or a sense-making middle proportion of it (Aristotle). From any developmental point of view, they can be more or less reversible, more or less autonomous, more or less complex, and more or less justifiable by criteria of tradition and deepness. And the most important issue: everyone in a society can discuss, question, and deny/support it. Type 3 could be an economical–moral mixture; it questions tolerance towards traditional economic burden (like in the case of Frankfurt’s booklet on inequality). This gives us a different view on what CT can be. It goes often beyond seriousness and usefulness of information. It has to do with meaning and well-being, often with happiness and trust. This is why in the European tradition we speak about “Bildung” (education) (see Beck in this volume).

98

F. K. Oser and H. Biedermann

Measurement Issues Established Test Formats There are two difficult questions with regard to the richness of the literature on CT-tests. First, is it true that CT-tests often do not measure CT respective to the specific aspects of CT but rather aspects of clever thinking, logical thinking, or analytical thinking? And, what do these tests predict? We will start the discussion of these questions with some examples: The HEIgthen test of CT contains information on a certain topic (Educational Testing Service 2018). Based on a multiple-choice test format the students are asked to evaluate the statements according to their correctness. This is approached in two ways: (a) analytically and (b) synthetically. Analytical means “evaluating evidence and its use, including evaluating the evidence itself and evaluating it in light of its larger context, its relevance to the argument, appropriateness of sources, possibilities of bias and the degree of support the evidence lends to the claims made in the argument”. Synthetically means “understanding implications and consequences, including identifying unstated conclusions or implications and consequences that go beyond the original argument” (www.ets.org/heighten/about/critical_thinking/). As interesting as the test looks, it is rather a test of proving the knowledge of a student in a classical multiple-choice format (to find out what is the right answer). The better a student knows the information given, the better he/she will perform. The Cap Critical Reasoning test (version 2018) is similar. The student has to read some information and then choose the right answer from a set of 4 or 5 different possibilities. It is rather a knowledge-based analytical thinking and multiple-choice test (http://practice.cappassessments.com). Assessment of Higher Education Learning Outcomes (AHELO) (OECD 2015) is targeted towards “students’ capacity to use, apply and act on the knowledge and reasoning they have gained from their degrees”. Students should apply their knowledge to real-world problems with respect to “Generic Skills, Economics and Civil Engineering, as well as the development of contextual instruments to aid with the interpretation of assessment data” (https://www.acer.org/aheloau/ahelo). This test is a classical performance-oriented test, but CT is only implicitly involved in it. The CT test for high school students from Sarigoz (2012) is an instrument where students do believe more or less that they are critical. One item example is “While a matter is explained, I can analyse it by thinking the datas regarding that matter.” Or another: “By means of explanations regarding a matter, I can predict the ideas unexplained.” This test is a belief test and not a CT test. The most well-known test is the Watson-Glaser Critical thinking Appraisal (Watson and Glaser 2002). The test contains inferences, recognition of assumptions, deduction, interpretation, and evaluation of arguments. Even though this test is highly valid, it is mostly a thinking (not a critical thinking) test. The critical part is hidden within the difficulty to solve a thinking problem.

7 A Three-Level Model for Critical Thinking

99

All in all, we think that what Abrami et al. state, namely “the opportunity for dialogue, the exposure of students to authentic or situated problems and examples, and mentoring had a positive effect on CT skills” (275) is a hint for proving whether thinking and CT are similar competences. The respective training recommendations are often just for better thinking in general. Then one has to look at the prognostic value of these tests. If Abrami et al. (ibid.) state that CT skills and student achievement yields an effect size of 0.30, we ask why should CT lead to higher achievement? We rather think that we should not measure student achievement with regard to CT but rather the relationship between CT and moral sensitivity, CT and religious judgment, CT and political awareness, CT and societal responsibility, and similar. This is only possible with a performance test format for CT.

Performance Testing Performance testing consists of the tendency to use real-life situations for measuring the knowledge, the competencies, and the attitudes of a person in certain fields (see Shavelson et al. 2019). To be tested, a person does not have to fill out any multiplechoice questions, but he/she rather models and transforms the respective situation. Through this modeling and forming, the subject elicits his/her capacity to deal with the situation, to understand it, and to work with it adequately and critically. Performance testing is a new field, especially in the field of CT. Its complexity produces some difficulties with respect to measurement techniques. In our ongoing study, we aim to stimulate CT of university or college students precisely not within their study domain, but within political, moral, ethical, and religious dimensions of a real-life problem: the so-called kidney transplantation story. We think that this situation is located on the second level of CT; it stimulates the generic part of thinking. It stimulates critical reflection and critical alertness.

Back to the Three Levels of Critical Thinking: How to Undertake Questioning in Each of It Questioning in the sense of CT means “posing questions” and contesting something rather than solving problems, or find out if a statement is more or less important, or analyze the correctness of a logical inclusion. Since the deepest level is contentspecific, we question a theory, a concept, a data analysis, a statistical calculation, etc. by the means of our specialist field. Thus, measurement frames would be what is accepted as more or less significant with respect of the limits of this theory, this concept, this data analysis, etc. One example would be a generalizability question: Is this theory transferable to a different age? Does this concept hold if the subjects are

100

F. K. Oser and H. Biedermann

put into a stress situation? And the main question is how this concept is structured. Moral, ethical, social, political, and religious questions with respect to this first and deepest level are treated only on level two. They are generic. In the second level, critical reflection, we ask why this person, for instance, feels treated unjustly. Here, we have to understand how the 7 SDs are related to societal facts. The question is not about the possibility to realize, for example, a certain surgery; it is rather about the political, social, and religious consequences if we do so. We question the hidden interests of people’s acts and we reflect, value, and coordinate the respective consequences. We do not so much look at the truth and the validity of a human decision but to its value with respect to other not yet transparent values. Questions in this second level are generic. The third level is about being wide-awake. You listen with a skeptical attitude to societal events like political decisions, practical lifestyle implementations, religious expressions, moral indignations, etc. Subjects develop something like a sense of critical alertness. The basic question here lies in the conflict between one’s own convictions and the ones of others and has the form of a permanent wish to consolidate different points of view. Subjects ask again and again about the opposite opinion. Imagining the opposite of an idea becomes what we call a basic positive skepticism. A test question would be about the interest on the opinion of the other party.

Critical Thinking and the Kidney Story: Towards a Performance Test Within the 7 Social Dimensions For the application of what presented above, we developed the so-called kidney story. The focus of this problem lies on the question how can we conceive CT with respect to a societal problem encompassing knowledge, political judgments, ethical issues, and religious concepts? How can we apply questions with respect to the second level?1 Most of what we present is level two material; it is very well suitable for stimulating generic CT.

The Basic Situation Stimulating Critical Thinking The test is located within iPAL (Zlatkin-Troitschanskaia et al. 2019), an encompassing project for developing CT as a observed performance. We propose a classical medical situation, a kidney shortage when transplantation is needed, with justice aspects, religious aspects, ethical issues, political concepts, and social

1 We would like to thank Richard Shavelson for the help he gave us in construction of the most important parts and to reduce its length.

7 A Three-Level Model for Critical Thinking

101

elements. Each of these domains has a different value element leading to different measurement means. The starting point is that Gerhard desperately needs a kidney. He would be placed in a juste order according to medical criteria, but mostly according to the time he is diagnosed a strong kidney disease. Gerhard is a wealthy person. He and his brother and sister inherited a fortune from their parents. So there are maybe other ways to get a kidney.

Critical Thinking in Three Opinions on Justice The following CT questions are posed: (a) If you have the money, do what you need to do to get around the waiting list and to receive a kidney. Others can profit from your deal (Brother Max). (b) The whole family should be included in the decision process, and doing such things for money could become public. In addition, we cannot influence our destiny. The other patients are part of this (Sister Elisabeth). (c) Put yourself in the shoes of others and try to keep the law. We cannot act against the law. As long as we keep the law we can use the money as we like (Mother). Task 1: The student has to select one of these statements spontaneously as best and criticize the others. He/she has to justify their choice positively and negatively. Task 2: He/she has to justify their choice positively and negatively (reflection and eventually change of the position). Task 3: The student has to apply the theory of justice of Rawls (1993) in its basic tenets to the situation. • The first critical rule here is the amount of reversibility B > A; C > B, Rawls > C • The second points to the indirect P-score (Rest 1986): If seen critically, how fare goes the proposed solutions with respect to the total “veil of ignorance”?

Critical Thinking and Medical and Psychological Stress Factors The content of this part relates to the well-being of the donator and the receiver after surgery. There are past-surgery problems like fatigue, pain, or immunosuppressant problems. There is information that yields many medical and psychological stress factors. Task 1: The student reacts spontaneously, “I am convinced that. . . .” Task 2: The student has to find out how important this information is for the decision to donate an organ. Here, the critical analysis relates to the usefulness, on the one hand, and the reliability, on the other.

102

F. K. Oser and H. Biedermann

Task 3: The student has to deconstruct the information and question on each part its truthfulness (by using additional Internet information). The critical rule standards are scientific testing and expirience in post-operational health-care. For this student has to apply an important text on care of Noddings (1992) to the criticism of Dr. Kröncke, who informs about the phenomenon of fatigue or pain or immunosuppressant problems. The rule system is: Whatever happens in a transplantation situation—without any guilt attribution—both shall receive the optimal care. The critical part consists of the acceptance of negative consequences (hurting) for the sake saving a life.

Critical Thinking and Professional Morality The health conditions required of a kidney recipient are well known. Recipients undergo testing to ensure these conditions are met for their safety during the operation and their ability to tolerate the anti-rejection medication necessary after transplantation. Dr. Singer is Gerhard’s family doctor and his medical consultant. She knows that Gerhard was a longtime substance abuser and as a consequence he had—in the past—severe neurological impairment that is not problematic at the moment. Gerhard is also suffering from a skin disease called mycosis. But for kidney transplantation a patient needs to be—besides the kidney problem—completely healthy. Task 1: Students are asked to react spontaneously about the importance of such health conditions. Here, is an example of such a statement: “I would—in the case of Gerhard—just make a gut decision. I do think what I feel is always what is right. I do not need to care so much about facts, rules, and regulations. I think a doctor knows spontaneously what is good for a patient and normally he/she has no time to consider all the consequences that could arise. In this case, I would just give the current information, nothing else. In Gerhard’s case, justice means to care about him whatever information you have.” Task 2: Students are confronted with statements from medical doctors with respect to Dr. Singers’ dilemma. They have to analyze these opinions critically. The statements are related to levels of forms on how to bring justice, care, and truthfulness together (see concepts of professional balancing of duty aspects in Oser 1998). The statements represent: (a) the concept of avoiding, (b) the concept of delegating, (c) the concept of single-handed decision making, (d) the concept of incomplete discourse, and (e) the concept of complete discourse. This hierarchy is critical insofar that it represents a value system with a differentiated complex and a different responsibility distributed discourse orientation. Students choosing a lower form (like avoiding) are less inclined to take responsibility for others; students choosing higher level statements are better able to balance justice, care, and truthfulness. Students not only choose the “better” option, they also critically reject the obviously less adequate option. In this part, we only work with two critical levels, namely 1 and 3.

7 A Three-Level Model for Critical Thinking

103

Critical Thinking and Religious Issues Based on different religious statements (speaking of the pope, religious experience of a strong believer, religious commandment of a sect), we present to the students five statements that are related to the stage theory of religious development. The basic tenet is that each higher stage is religiously more integrated and less related to external coercions of religious systems. Each higher stage sees religious belief as a matter of bringing together dependency versus freedom, fear and hope, immanence and transcendence, time relatedness and eternity, reality and symbolic transformation, fragility versus security (Oser and Gmünder 1991). Task 1: The student expresses his/her spontaneous sympathy with one of the statement. Task 2: The student expresses why one of these statements is best. Task 3: The student has to compare these statements, to choose one, and to justify the choice is a typical CT matter. The content is given and must be proven to be less or more worthy with regard to the two dimensions: (a) the intensity of “the lifeaccepting and free-making” relationship with an ultimate being, and (b) liberty with respect to religious constraints. The procedural rules are the distance from being obliged to give or do something for the ultimate and to see any metaphysical sign as a possibility to think in terms the insecurity of any given situation. Students justify the non-chosen answers.

Critical Thinking and Political Decision-Making Transplantation issues require technically highly developed hospitals. Since there are many of such clinics in Swiss cantons, some had—because of economical reasons—to be closed. Because in the canton of St. Gallen, two hospitals are in a business competition with each other, the government has to find out how to solve the problem and which hospital shall be favored. One hospital is private; it is technically best equipped for transplantation processes. The other one is state supported, huge, and has no specific highly developed specialities. Task 1: The students have to react spontaneously to the issue. Task 2a: The students have to analyze the situation and have to try to crystallize some of the issues that can never be solved with negotiation but only by governmental decision and some that can easily be solved through negotiation. They have to use all given information to formulate alternative options, evaluate each alternative, and, finally, recommend and justify a course of action. They have to write some first recommendation pieces, some own ideas on how the conflict could be solved (critical analysis of a political conflict where the social-democrats rather vote for the state-hospital, the liberal rather for the private clinic). The main concept is that a common inclusive will is necessary to put the private clinic into the state-hospital by warranty further privacy (conditional inclusive analysis).

104

F. K. Oser and H. Biedermann

Task 2b: One argument of the bigger hospital is their immediate connections to university research and to the students control through the medical professors. This is precisely a positive point for the small clinic. They argue that because of their engines and their very modern technical tools they would be better able to guide students. Thus, the task is to solve this issue by generating own arguments for one or the other or a common solution from an educational point of view (critical exclusion of different educational point of views). Task 3: The student has to apply critically the concept of moral disagreement to the situation. If the deliberation does not lead to a solution deliberation must be considered continuous—as Gutman & Thompson (1996) say—and reciprocity, even seen as futile, and should not be left behind. All this is precisely leaning to live with moral disagreement. In all these tasks, the test takers look for CT itself. We are not striving for the right deduction, induction, etc. We rather ask how the student questions the situation and we developed scoring material that helps to value each student’s statements.

Conclusion To develop a new assessment instrument of CT, new reflection on how CT is distinct from thinking in general is necessary. The critical part of CT means questioning fixed and generally accepted information and screening it in positive and negative arguments (first level). To connect this competence with societal dimensions such as moral, political, religious, etc. elements is part of the seriousness of problems in our time (second level). One of the most difficult stances in the construction of a performance test on CT is critical alertness (first level), profound skepticism, or doubt (second level) and examination (third level), which respects, on the one hand, the idea of others, but, on the other hand, demonstrates the fragility of their theories, beliefs, and convictions. A three-level model of CT can help to find an orientation with respect to field-oriented analysis, general critical reflection (generic), and the motivation to be skeptical in a positive sense. To measure CT means to measure how people question and doubt. We understand why the other side, namely general thinking, in many tests is seen as a precondition for CT. But we think that the act to be skeptical is a competence in itself; and it should be measured in itself. We should find a way to model and measure what helps to bring society subtly ahead of just what is publically accepted and given. To measure real CT would be a new and challenging task. All in all, we must learn to measure the questioning and its thinking stream but not the general thinking with all its technical increments.

7 A Three-Level Model for Critical Thinking

105

References Abrami, P. C., Bernard, R. M., Borokhovski, E., Wddington, D. I., Wade, C. A., & Persson, T. (2015). Strategies for teaching students to think critically: A meta-analysis. Review of Educational Research, 85(3), 275–314. Adorno, T. W. (1957/1958). Vorlesung zur Einleitung in die Erkenntnistheorie. Frankfurt: Junius Drucke. Astleitner, H. (1998). Kritisches Denken. Basisqualifikation für Lehrer und Ausbildner. Innsbruck: Studienverlag. Brookfield, S. D. (1987). Developing critical thinkers: Challenging adults to explore alternative ways of thinking and acting. Ballmoor: Open University Press. Cap Critical Reasoning Test. (version 2018). http://practice.cappassessments.com Dewey, J. (1925). Experience and nature. Chicago: Open Court. Educational Testing Service. (2018). HEIghten®Critical Thinking Assessment. Retrieved from https://www.ets.org/heighten/about/critical_thinking/ Ennis, R. H. (1989). Critical thinking and subject specificity: Clarification and needed research. Educational Researcher, 18(4), 4–10. Facione, P. A. (1990). Critical thinking: A statement of expert consensus on educational assessment and instruction. Research findings and recommendations. Newark, NJ: American Philosophical Association (ERIC). Facione, P. A. (2004). Critical thinking: What it is and why it counts. Milbrae, CA: Academic Press. Fischer, L. (2013). Presidential War Power (3rd.ed.). Lawrence: University Press of Kansas. Frankfurt, H. G. (2015). On inequality. Princeton, NJ: Princeton University Press. Glaser, E. M. (1941). An experiment in the development of critical thinking. Doctoral dissertation (UMI No. 0156200) Grovier, T. (1985). A practical study of argument. Belmont, CA: Wadsworth. Gutman, A., & Thompson, D. (1996). Democracy and disagreement. Why moral conflict cannot be avoided in politics, and what should be done about it. Cambridge, MA: Belknap Press. Huber, C. R., & Kuncel, N. R. (2016). Does college teach critical thinking? A meta-analysis. Review Educational Research, 86(2), 431–468. Hyslop-Margison, E. J. (2003). The failure of critical thinking: Considering a virtue epistemology pedagogy. Philosophy of Education Society Yearbook, 2003, 319–326. Kant, I. (1999). Critique of pure reason (The Cambridge Edition of the Works of Immanuel Kant) (P. Guyer & A. W. Wood, ed., Trans.). Cambridge: Cambridge University Press. Marquard, O. (2007). Skepsis in der Moderne. Stuttgart: Reclam. (UB 18524). McPeck, J. (1981). Critical thinking and education. Toronto: Oxford University Press. Muis, K. R., Chevrier, M., & Singh, C. A. (2018). The role of epistemic emotions in personal epistemology and self-regulated learning. Educational Psychologist, 53(3), 165–184. Noddings, N. (1992). The challenge to care in schools: An alternative approach to education (Advances in contemporary educational thought series) (Vol. 8). New York: Teachers College Press. OECD. (2015). AHELO Main Study. Retrieved from https://www.oecd.org/education/skillsbeyond-school/ahelo-mainstudy.htm Oser, F. (1998). Ethos: die Vermenschlichung des Erfolgs. Opladen: Leske + Budrich. Oser, F., & Gmünder, P. (1991). Religious judgment. A developmental approach. Birmingham: Religious Education Press. Popper, K. (1934/2005 11. Aufl.). Logik der Forschung. Wien: Akademie Verlag. Rawls, J. (1993). Political liberalism. New York: Columbia University Press. Rest, R. J. (1986). Moral development. Advances in research and theory. New York: Praeger. Sarigoz, O. (2012). Assessment of the high school students’ critical thinking skills. Procedia: Social and Behavioral Sciences, 46, 5315–5319. Schäfer, L. (1988). Karl R. Popper. München: Beck.

106

F. K. Oser and H. Biedermann

Scriven, M., & Paul, R. (1996). Defining critical thinking: Critical thinking as defined by the National Council for Excellence in Critical Thinking. Retrieved from https://www. criticalthinking.org/pages/defining-critical-thinking/766 Scriven, M., & Paul, R. (1987). https://www.criticalthinking.org/pages/defining-critical-thinking/ 766 Shavelson, R. J., Zlatkin-Troitschankaia, O., & Marino, J. P. (2018). International performance assessment of learning in higher education (iPAL): Research and development. In O. ZlatkinTroitschanskaia, M. Toeper, H. A. Pat, C. Lautenbach, & C. Kuhn (Eds.), Assessment of learning leaning outcomes in higher education (pp. 193–214). Springer. Shavelson, R. J., Zlatkin-Troitschanskaia, O., Beck, K., Schmidt, S., & Marino, J. (2019). Assessment of University Students’ Critical Thinking: Next Generation Performance Assessment. International Journal of Testing. https://doi.org/10.1080/15305058.2018.1543309 Siegel, H. (1988). Educating reason: Rationality, critical thinking, and education. New York: Routledge. Walter, P., & Leschinky, A. (2007). Critical thinking und migrationsbedingte Bildungsbenachteiligung: Ein Konzept für die subjektive Auseiandersetzung mit schulstrukturellen Merkmalen? Zeitschrift für Pädagogik, 1, 1–15. Watson, G., & Glaser, E. (2002). Critical thinking appraisal. London: Pearson Assessment. Woolfolk, A. E. (1998). Educational psychology. Boston: Allyn & Bacon. Zlatkin-Troitschanskaia, O., Toepper, M., Molerov, D., Buske, R., Brückner, S., Pant, H. A., et al. (2019). Adapting and validating the collegiate learning assessment to measure generic academic skills of students in Germany: Implications for international assessment studies in higher education. In O. Zlatkin-Troitschanskaia, M. Toeper, H. A. Pat, C. Lautenbach, & C. Kuhn (Eds.), Assessment of leaning outcomes in higher education (pp. 245–266). Wiesbaden: Springer. Zlatkin-Troitschanskaia, O., Shavelson, R. J., Schmidt, S., & Beck, K. (2019). On the complementarity of holistic and analytic approaches to performance assessment scoring. The British Journal of Educational Psychology. Advance online publication. https://doi.org/10.1111/bjep. 12286

Part II

Learning with New Media and Technology

Chapter 8

Positive and Negative Media Effects on University Students’ Learning: Preliminary Findings and a Research Program Marcus Maurer, Christian Schemer, Olga Zlatkin-Troitschanskaia, and Judith Jitomirski

Introduction Extant research has provided valuable insights into how students learn what they are supposed to learn (Pashler et al. 2007; Zlatkin-Troitschanskaia et al. 2019). However, learning in higher education often occurs in informal or unintentional learning environments (Maurer et al. 2018). Specifically, students can learn from representations in journalistic news media (newspapers, television news, news websites, etc.) or social media, including multiple heterogeneous sources (e.g., social network sites, wikis, blogs; Dalton and Crosby 2013; Kimmerle et al. 2015). It is assumed here that the learning material or the format in which it is presented can support or impede students’ learning in the sense of positive or negative learning (ZlatkinTroitschanskaia et al. 2018) depending on various factors, for example, the credibility of the information source, the relatedness of the representation to the domain of interest, the amount of material to be learned, or formal aspects of presentation (textbased, audio, or audiovisual mode) (e.g., Sundar 2003; Shavelson et al. 2019). For instance, informal learning might facilitate positive learning when students are M. Maurer (*) · C. Schemer Institute of Journalism and Communication Studies, Johannes Gutenberg University of Mainz, Mainz, Germany e-mail: [email protected]; [email protected] O. Zlatkin-Troitschanskaia Department of Business and Economics Education, Johannes Gutenberg University Mainz, Mainz, Germany e-mail: [email protected] J. Jitomirski Institute for Educational Studies, Humboldt University of Berlin, Berlin, Germany e-mail: [email protected] © Springer Nature Switzerland AG 2019 O. Zlatkin-Troitschanskaia (ed.), Frontiers and Advances in Positive Learning in the Age of InformaTiOn (PLATO), https://doi.org/10.1007/978-3-030-26578-6_8

109

110

M. Maurer et al.

exposed to learning material that is reliable and consistent with and supportive of the material learned in more formal settings. However, informal learning might also impede positive learning or increase negative learning when information learned in the context of mass media or social media is inconsistent with what students have learned in formal settings (Maurer et al. 2018). In this context, the implementation of new media as teaching-and-learning resources might (albeit not intentionally) have negative effects on the acquisition of knowledge and skills. So far, such informal or unintentional learning has received little scholarly attention. Research in communication science focuses predominantly on the knowledge acquisition of the overall population (e.g., Graber 2001), while research into students’ knowledge acquisition is largely limited to formal learning (e.g., Zlatkin-Troitschanskaia et al. 2016). Overall, we know comparatively little about how informal learning can promote and support self-directed learning in the sense of students’ positive learning (ZlatkinTroitschanskaia et al. 2018). Therefore, in this paper, we (1) develop a theoretical framework systemizing the factors causing positive or negative media effects on students’ learning, (2) present findings of a pilot study dealing with media effects on students’ learning at university, and (3) discuss a broader research program intended to gain further insights into the processes influencing positive and negative student learning when using mass and social media during studies.

Positive and Negative Media Effects on Learning The present paper seeks to build a coherent framework to explain the interplay between formal learning and informal or unintentional learning caused by mass or social media use. The present study also aims to identify the essential elements underlying sources of knowledge and their specific medial and information representations as well as to assess in detail the effects on students’ learning in higher education. The positive and negative impact of media use on learning can theoretically be explained by the following three factors: 1. Information presented (media content): Mass and social media can influence learning if used to review, consolidate, or complement knowledge acquired in higher education. Due to minimal temporal and spatial limitations as well as hypertextuality, interactivity, and multimodality, online media generally have an advantage over offline media, as they can convey additional and more in-depth information (e.g., Maurer and Oschatz 2016). However, news media as well as non-journalistic sources, such as interest group websites or social networks, may disseminate abstruse, incomplete, or even factually incorrect information (e.g., Maurer and Reinemann 2006). This raises the question to which extent different sources disseminate information completely and correctly. 2. Choice of information sources (media use): Students use information selectively, which is reinforced by the available diversity of information and the individualization of online services through algorithms (filter bubble, see Pariser 2011) and

8 Positive and Negative Media Effects on University Students’ Learning

111

is assumed to generally discourage the development of a common societal body of knowledge (fragmentation hypothesis; e.g., Fletcher and Nielsen 2017). In addition, it is conceivable that students might specifically avoid information related to their subject and tend to use media to distract themselves from studying (procrastination hypothesis; e.g., Madge et al. 2009). Moreover, the majority of relevant information, especially when found online, might be ignored by the students. Thus, the most relevant issue is a detailed analysis of how and to what extent students use the information presented in various media, and which media are suited to improve learning. 3. Understanding and processing information (media effects): The effects of media on knowledge in various domains remain widely disputed in research (e.g., Maurer, 2017). Especially the use of the internet can promote or inhibit thorough processing of information and knowledge acquisition (Maurer and Oschatz 2016). When using different media, students are often confronted with an abundance of information, which may in itself result in an information overload that reduces a learner’s capacities to focus on relevant aspects of the material to be learned (Ziegler et al. 2015). To some extent, smart searching strategies can enable them to differentiate between reliable and unreliable sources and to distinguish relevant from irrelevant content (Shavelson et al. 2019). There may also be a tendency to use the internet as an online transactive memory, which can foster creative problem solving, however, at the same time, students may mistake the knowledge that is accessible online as their own (Ward 2013). Thus, different searching strategies might result in enhanced or positive learning, but may also have serious repercussions on students’ memory and social cognition (Sparrow et al. 2011). Moreover, learning can be inhibited by a tendency of students to use and process information in a way that helps them confirm already existing worldviews (motivated reasoning, Kunda 1990), which might result in both positive and negative learning.

Pilot Study For the initial investigation of mass and social media effects on students’ learning, a collaborative pilot study was carried out as part of the ongoing research project WiWiKom (Zlatkin-Troitschanskaia et al. 2019). This overall project covers the period from September 2015 to November 2019, during which a total of five surveys are conducted. In the first measurement, at the beginning of the 2016/17 winter semester, 9055 first-year students of economics and social studies at 54 universities were surveyed across Germany. In addition to information on the students’ sociobiography and previous education, study subject-related previous knowledge (testbased) and general cognitive abilities (intelligence) were measured (ZlatkinTroitschanskaia et al. 2016, 2019). In the summer semester of 2017, 1418 of these test takers participated in an online survey, which was designed by the project partners. Fifty-five percent of the participants studied business and economics, and

112

M. Maurer et al.

13% studied sociology (32% missing) (at t1, 20% had studied sociology and 80% economics). On average, the students were in their second semester or at the end of their first year of studies. In particular, the use of media during studies was examined in a differentiated manner and further psychological constructs such as overload, need for evaluation, critical thinking, self-regulation, and self-efficacy were measured. In the subsequent survey at the beginning of the 2017/18 winter semester, the university students were again surveyed on site, and particularly study domainspecific knowledge (test-based) and further study (mis)success indicators (Shavelson et al. 2018) such as the number of courses attended, CPs achieved, and examinations passed were assessed. The data that will be presented in this paper resulted from the second survey conducted in the summer semester of 2017 and focuses on the descriptive specification of media resources that students use during their studies. It will be interesting to explore whether the media use of sociology students is different from the media use of economics students. In the first step, we focus on the question which media (news media, social media, as well as traditional university print and online media) students usually use to inform themselves about issues related to their study field (in this case: the economy). Therefore, we asked two different questions: (1) which media students generally use to inform themselves about economic issues; and, more specifically, (2) which media they use to prepare for their courses and exams. In each case, the frequency of using 14 different media was asked on a six-point scale ranging from “never” to “several times a day.” For our analyses, we consider students who use a medium at least “once a week” as users of that medium. In a block of questions, the students were asked to state to what extent they trust these 14 different media and consider them reliable sources of information. In addition, the students were asked to indicate in an open format the medium they use most frequently in their studies. As expected, our analysis shows that students use a variety of media in their university studies. This holds true not only for their general media use but also for their exam preparation. As Fig. 8.1 shows, students use television news, Wikipedia, and even video platforms such as YouTube more often than textbooks to inform themselves about economic issues. Considering that almost all of the students surveyed are in their second semester, these figures are quite remarkable, as both sociology and especially economics degree courses are strongly structured in the first year of studies and are based on compulsory lectures with corresponding mandatory textbooks. Therefore, we would have expected students in their second semester to use textbooks most frequently. Interestingly, the study shows that regarding other media, students use news media more often than science databases like Google Scholar. Finally, even tabloid newspapers are used as frequently as science journals. Overall, there is a trend towards students using non-scientific sources of information more frequently than scientific ones. Interestingly, the open-ended question about the medium most frequently used is also answered with “Google.” To further investigate this phenomenon, it would be particularly important to know more specifically which content related to which subject areas students use (see Section “Discussion: Limitations and Outline for a Broader Research Program”).

8 Positive and Negative Media Effects on University Students’ Learning

113

Online Encyclopedias Science Data Bases Course Scripts Textbooks Science Journals Video Platforms Economic TV Magazines Television News News Magazines Weekly Newspapers Economic Newspapers Tabloid Newspapers Regional Newspapers Nationwide Newspapers 0

10

20

30

Preparing for exams

40

50

60

70

80

General use

Fig. 8.1 Students’ use of economic information in general and when preparing for exams (in %)

These additional content analyses are all the more important as data shows that even when preparing for exams, students do not rely solely on university teaching material such as mandatory textbooks. Instead, Wikipedia is the students’ second most important information source in this case as well. When looking at a few Wikipedia articles on some key economic topics (such as the supply-demand model, etc.), a heterogeneity in the quantity and quality of the presented information becomes apparent at first glance. It would therefore be urgently necessary to know, for instance, which specific articles and which contents are used by students. This also applies to other media used by the students, as about one third of them use various kinds of news media (e.g., newspapers, television news) and still more than half of the students rely on video platforms as a source of learning during their studies. These results become even more interesting when comparing the data with the students’ answers regarding their assessment of the reliability of these sources. The textbooks and nationwide newspapers in particular are regarded by students as the most reliable sources (nationwide newspapers—(F (1.706) ¼ 4.70, p ¼ 0.0001, adjusted R2 ¼ 0.0190) and textbooks—(F (1.706) ¼ 13.31, p ¼ 0.0003, adjusted R2 ¼ 0.0171). In contrast, the majority of students rate the source tabloid papers as less reliable (F (1.706) ¼ 6.32, p ¼ 0.0121, adjusted R2 ¼ 0.0075). This shows that the students use several sources for their studies, although they assessed the reliability of some of the sources as critical. Further studies would have to investigate the reasons for this (see Section “Discussion: Limitations and Outline for a Broader Research Program”). In an additional question, students were asked about using social media to inform themselves about economic questions. Using the same scale as explained above, we asked how often they use social media information from various sources like classmates, friends, political parties, NGOs, trade associations, or labor unions.

114

M. Maurer et al.

others unions celebrities NGOs & other organisations management associations political parties colleagues friends and aquaiantances other students 1

2

3

4

5

6

1 - never to 6 - multiple times per day

Fig. 8.2 Students’ use of social media during their studies

Particularly information from classmates (66%) and other friends (53%) is often used. However, Pause exp). The SPM Anatomy Toolbox (Version 2.2c; Eickhoff et al. 2007; www.fz-juelich.de/ime/spm_anatomy_toolbox) was used to exactly determine the anatomical localization of neural activations.

Results Behavioural Data ID.SWT scores were higher than ID scores for all subjects and speech samples (mean ID.SWT ¼ 5.62; mean ID ¼ 5.02) and the subject ranks for mean ID scores differed between the two calculation methods for some subjects. Statistical tests revealed that the difference of ID ranks and ID.SWT ranks was not statistically significant ( p ¼ 0.71, r ¼  0.099), suggesting that the overall relation between subjects with regard to ID scores was retained when changing the calculation type.

Functional Imaging Data To determine the general task effect—that is, activation for overt speech production during picture description—brain activation patterns for overt speech production vs. experimental pauses were calculated. Patterns of neural activations encompassed

13

The Neural Basis of Idea Density During Natural Spoken Language

159

Fig. 13.2 The effect of increased linguistic complexity as measured by ID (contrast ID+ > ID) on the neural processing associated with speech production

the well-known “language network” including frontal and temporal as well as occipital regions in both hemispheres. This image was used as a functional mask to find out if parametric modulations of ID (+ vs. / vs. +) were reflected in activation patterns within the language network, and/or whether ID-associated BOLD enhancements recruit particular regions not primarily associated with speech production per se. For the effect of linguistic complexity as measured by ID, enhanced BOLD responses were observed in five brain regions in the left hemisphere and five regions in the right hemisphere; these were positively associated with measures of ID (contrast ID+ > ID). We found clusters of activation bilaterally in the inferior parietal lobule (IPL) and superior parietal lobule (SPL), and in the left middle frontal gyrus (MFG), left hippocampus (HPC), right superior occipital gyrus (SOG), right middle temporal gyrus (MTG), and right cerebellum (CE). The results for the parametric ID effects are presented in Fig. 13.2. The inverse contrast, ID vs. ID+, revealed different neural responses for increased ID in the right medial prefrontal cortex (mPFC), encompassing the anterior cingulate cortex (ACC), extending to the medial frontal gyrus.

160

A. Nagels et al.

Discussion Behavioural Data The ID.SWT measure yielded constantly higher absolute values as compared to ID, though the results suggest that overall, the relation among subjects regarding individual mean idea density was preserved. This indicates that ID.SWT represented idea density in the study population in a comparable way to ID. The observed differences still require an explanation. In this study, the difference between ID scores obtained with the two calculation methods results from the application of the SWT. Scoring ideas which overlap window boundaries leads naturally to a higher calculated ID score, i.e., higher values for ID.SWT were to be expected.

Functional Neuroimaging Data The results of the functional imaging data revealed a network of brain regions showing more activation for semantically more complex speech, as indexed by increased idea density scores (Fig. 13.2). Interestingly, the application of the functional mask for brain activity correlating with overt speech production in general revealed that the activity patterns observed for the contrast ID+ > ID did not overlap with the general language network. Thus, it appears that brain regions involved in the processing of linguistic complexity (as measured by idea density) differ from those that support speech production in general. In the network of brain regions, whose activation was associated with a parametrical variation of idea density, those in the hippocampus can be regarded as of primary interest. The hippocampus is usually not among the principal brain areas reported in the context of speech production, such as Broca’s or Wernicke’s areas. However, the activation observed in the hippocampus in the left hemisphere is in line with a recent proposal (Duff and Brown-Schmidt 2012, 2017) concerning the memory-language interface. The researchers argue that in addition to its role in supporting memory, the hippocampus contributes to effective language processing as well (Duff and Brown-Schmidt 2012, 2017). The notion of idea density, which has been derived from the theory of Kintsch (1974) on the representation of meaning in memory, is by definition inherently related to memory load. Hence, the production of semantically more complex speech can be expected to place higher demands on processes supported by the memory system. Each idea, be it a predication, modification or a connective, encompasses a predicator and one or more arguments (Kintsch 1974). It is widely assumed that language is produced incrementally, necessitating an integration of ideas and relations within the unfolding speech stream, thereby placing demands on processes supported by memory of different kinds. This process involves lexical planning

13

The Neural Basis of Idea Density During Natural Spoken Language

161

ahead of speech (Lee 2015), while the scope of lexical planning has been observed to vary across contexts (Ferreira and Swets 2002). The present observation of hippocampal activation in response to increased semantic/linguistic complexity in overtly produced speech provides additional converging evidence for a direct supportive function of the hippocampus in online language processing.

Limitations and Future Directions The present study is exploratory, so its results must be treated with caution. Still, these findings suggest that idea density may be an appropriate measure for investigating the neural underpinnings of the processing of semantic complexity. This study tested a group of healthy young and middle-aged adults; thus the subject population was rather homogeneous in terms of language ability. Furthermore, the group of participants was rather small (N ¼ 14). Hence, subtle effects of increased linguistic complexity on neural activity might have gone unnoticed in this sample. Future research on the neural correlates of idea density might benefit from larger and more heterogeneous study populations, for example, by contrasting a patient group and healthy controls. As noted by Ferguson et al. (2014) and Spencer et al. (2015), a speech sample length below a certain threshold—Ferguson and Spencer recommend a minimum sample length of 60 words—adds to variability in the ID measure. In the present study, this issue was mitigated through the application of the sliding window procedure, but a residual influence of small text sizes cannot be completely excluded. Future studies using a similar design might benefit from extending the duration of speech elicitation. Since ID correlates with a variety of verbal and non-verbal factors, our findings could serve as a starting point for a comprehensive model of neural information processing. Within the PLATO project, factors associated with the learning process should prove particularly useful in shedding light on the relation between ID and positive (or negative) learning (Zlatkin-Troitschanskaia et al. 2018). The present findings indicate that a fundamental understanding of knowledge representation is necessary to explain linguistic processes utilized during learning, as resources outside the already known networks may be integrated in language perception and production. A basic understanding of the neural mechanisms engaged during the processing of high- or low-ID linguistic structures will therefore be essential in developing an innovative learning approach as envisioned in PLATO.

162

A. Nagels et al.

References Chand, V., Baynes, K., Bonnici, L., & Farias, S. T. (2012b). Analysis of idea density (AID): A manual. Davis, CA: University of California. Chand, V., Baynes, K., Bonnici, L. M., & Farias, S. T. (2012a). A rubric for extracting idea density from oral language samples. Current Protocols in Neuroscience, 58(1), 10.5.1–10.5.15. https:// doi.org/10.1002/0471142301.ns1005s58 Cheung, H., & Kemper, S. J. (1992). Competing complexity metrics and adults’ production of complex sentences. Applied PsychoLinguistics, 13(1), 53–76. https://doi.org/10.1017/ S0142716400005427. Duff, M. C., & Brown-Schmidt, S. (2012). The hippocampus and the flexible use and processing of language. Frontiers in Human Neuroscience, 6, 69): 1–69):11. https://doi.org/10.3389/fnhum. 2012.00069 Duff, M. C., & Brown-Schmidt, S. (2017). Hippocampal contributions to language use and processing. In D. E. Hannula & M. C. Duff (Eds.), The hippocampus from cells to systems: Structure, connectivity, and functional contributions to memory and flexible cognition (pp. 503–536). Cham: Springer. Eickhoff, S. B., Paus, T., Caspers, S., Grosbras, M.-H., Evans, A. C., Zilles, K., et al. (2007). Assignment of functional activations to probabilistic cytoarchitectonic areas revisited. NeuroImage, 36(3), 511–521. https://doi.org/10.1016/j.neuroimage.2007.03.060 Engelman, M., Agree, E. M., Meoni, L. A., & Klag, M. J. (2010). Propositional density and cognitive function in later life: Findings from the Precursors Study. The Journals of Gerontology. Series B, Psychological Sciences and Social Sciences, 65(6), 706–711. https://doi.org/10. 1093/geronb/gbq064 Farias, S. T., Chand, V., Bonnici, L., Baynes, K., Harvey, D., Mungas, D., et al. (2012). Idea density measured in late life predicts subsequent cognitive trajectories: Implications for the measurement of cognitive reserve. The Journals of Gerontology. Series B, Psychological Sciences and Social Sciences, 67(6), 677–686. https://doi.org/10.1093/geronb/gbr162. Ferguson, A., Spencer, E., Craig, H., & Colyvas, K. (2014). Propositional idea density in women’s written language over the lifespan: Computerized analysis. Cortex, 55, 107–121. https://doi.org/ 10.1016/j.cortex.2013.05.012 Ferreira, F., & Swets, B. (2002). How incremental is language production? Evidence from the production of utterances requiring the computation of arithmetic sums. Journal of Memory and Language, 46(1), 57–84. Kemper, S. J., LaBarge, E., Ferraro, F. R., Cheung, H., Cheung, H., & Storandt, M. (1993). On the preservation of syntax in Alzheimer’s disease: Evidence from written sentences. Archives of Neurology, 50(1), 81–86. https://doi.org/10.1001/archneur.1993.00540010075021 Kintsch, W. (1974). The representation of meaning in memory. The experimental psychology series. Hillsdale, MI: Erlbaum. Lee, C. (2015). How to cite an article with an article number instead of a page range. APA Style Blog. Retrieved August 21, 2018, from http://blog.apastyle.org/apastyle/2015/05/how-to-citean-article-with-an-article-number-instead-of-a-page-range.html Lee, E.-K., Brown-Schmidt, S., & Watson, D. G. (2013). Ways of looking ahead: Hierarchical planning in language production. Cognition, 129(3), 544–562. https://doi.org/10.1016/j.cogni tion.2013.08.007. Mitzner, T. L., & Kemper, S. J. (2003). Oral and written language in late adulthood: Findings from the Nun Study. Experimental Aging Research, 29(4), 457–474. https://doi.org/10.1080/ 03610730303698. Murray, H. A. (1943). Thematic apperception test. Cambridge: Harvard University Press. Nagels, A., Kircher, T., Steines, M., & Straube, B. (2015). Feeling addressed! The role of body orientation and co-speech gesture in social communication. Human Brain Mapping, 36(5), 1925–1936. https://doi.org/10.1002/hbm.22746.

13

The Neural Basis of Idea Density During Natural Spoken Language

163

Oldfield, R. C. (1971). The assessment and analysis of handedness: The Edinburgh inventory. Neuropsychologia, 9(1), 97–113. Riley, K. P., Snowdon, D. A., Desrosiers, M. F., & Markesbery, W. R. (2005). Early life linguistic ability, late life cognitive function, and neuropathology: Findings from the Nun Study. Neurobiology of Aging, 26(3), 341–347. https://doi.org/10.1016/j.neurobiolaging.2004.06.019. Schmidt, T., & Wörner, K. (2014). EXMARaLDA. In J. Durand, U. Gut, & G. Kristoffersen (Eds.), Oxford handbooks in linguistics. The Oxford handbook of corpus phonology (pp. 402–419). Oxford: Oxford University Press. Smolík, F., Stepankova, H., Vyhnálek, M., Nikolai, T., Horáková, K., & Matejka, Š. (2016). Propositional density in spoken and written language of Czech-speaking patients with mild cognitive impairment. Journal of Speech, Language, and Hearing Research, 59(6), 1461–1470. https://doi.org/10.1044/2016_JSLHR-L-15-0301. Snowdon, D. A., Greiner, L. H., & Markesbery, W. R. (2000). Linguistic ability in early life and the neuropathology of Alzheimer’s disease and cerebrovascular disease: Findings from the Nun Study. Annals of the New York Academy of Sciences, 903(1), 34–38. https://doi.org/10.1111/j. 1749-6632.2000.tb06347.x Snowdon, D. A., Kemper, S. J., Mortimer, J. A., Greiner, L. H., Wekstein, D. R., & Markesbery, W. R. (1996). Linguistic ability in early life and cognitive function and Alzheimer’s disease in late life. Findings from the Nun Study. JAMA: The Journal of the American Medical Association, 275(7), 528–532. https://doi.org/10.1001/jama.275.7.528 Spencer, E., Craig, H., Ferguson, A., & Colyvas, K. (2012). Language and ageing – Exploring propositional density in written language – Stability over time. Clinical Linguistics & Phonetics, 26(9), 743–754. https://doi.org/10.3109/02699206.2012.673046 Spencer, E., Ferguson, A., Craig, H., Colyvas, K., Hankey, G. J., & Flicker, L. (2015). Propositional idea density in older men’s written language: Findings from the HIMS study using computerised analysis. Clinical Linguistics & Phonetics, 29(2), 85–101. https://doi.org/10.3109/02699206. 2014.956263. Stephens, G. J., Silbert, L. J., & Hasson, U. (2010). Speaker-listener neural coupling underlies successful communication. Proceedings of the National Academy of Sciences, 107(32), 14425–14430. https://doi.org/10.1073/pnas.1008662107. Turner, A., & Greene, E. (1977). The construction and use of a propositional text base. Technical report: Vol. 63. Boulder, CO: University of Colorado, Institute for the Study of Intellectual Behavior. Zlatkin-Troitschanskaia, O., Schmidt, S., Molerov, D., Shavelson, R. J., & Berliner, D. (2018). Conceptual fundamentals for a theoretical and empirical framework of positive learning. In O. Zlatkin-Troitschanskaia, G. Wittum, & A. Dengel (Eds.), Positive learning in the age of information – A blessing or a curse? (pp. 29–52). Wiesbaden: Springer.

Part III

Innovative Analytical Approaches for Modeling and Measuring of Learning

Chapter 14

TextInContext: On the Way to a Framework for Measuring the ContextSensitive Complexity of Educationally Relevant Texts—A Combined Cognitive and Computational Linguistic Approach Alexander Mehler and Visvanathan Ramesh

Introduction In every learning situation, the assessment of the complexity (Campbell 1988; Liu and Li 2012, 2014; Stouten and Größler 2017) of linguistic input (text, speech, multimodal documents, etc.) is influenced by a multitude of variables interacting with the learning context in a non-linear way. From a linguistic point of view, these variables range from low-level structures (as studied by graphematics and phonology) via mid-level ones (concerning morphology, lexis, and syntax) to high-level structures (as studied by semantics and pragmatics) (Levelt 1989; Pickering and Garrod 2004). This is supplemented by non-linguistic variables relating to the interplay and multi-layeredness of discourse, resource, and described situations— to name some of the context variables according to the nomenclature of situation semantics (Barwise and Perry 1983). A direct consequence of this contextual dynamics is that the same text can be understood differently by the same learner in different contexts (Sweller et al. 2011), while different texts can be understood in a similar way, even if their linguistic or multimedia manifestations are very different. Similarly, the assessment of a text as being complex cannot be reduced to the assessment of language data contained in the text, but has to take into account non-linguistic contexts to arrive at learner-dependent judgments. Thus, in order to assess learning material along variables such as complexity, readability, comprehensibility, degree of difficulty, or adequateness, the interplay of linguistic expressions A. Mehler (*) Department of Computer Science and Mathematics, Goethe University Frankfurt am Main, Frankfurt am Main, Germany e-mail: [email protected] V. Ramesh Department of Computer Science and Mathematics, Goethe University Frankfurt am Main, Frankfurt am Main, Germany © Springer Nature Switzerland AG 2019 O. Zlatkin-Troitschanskaia (ed.), Frontiers and Advances in Positive Learning in the Age of InformaTiOn (PLATO), https://doi.org/10.1007/978-3-030-26578-6_14

167

168

A. Mehler and V. Ramesh

on the one hand and aspects of the (short-term) situational and the (long-term) extrasituational (Goodwin and Duranti 1992) context of learning on the other hand has to be modeled. From the point of view of the analysis of context effects in linguistics, this diagnosis appears as a commonplace (Auer 1996). However, while context sensitivity is analyzed as a characteristic of the semantics (Barwise and Perry 1983) and pragmatics (Gumperz 1992) of natural languages, the issue of constructing this sensitivity by linguistic means, that is the contextualization (Gumperz 1992) of text and speech (Auer 1992), has been a topic for decades without being under control by computational linguistics. This is the starting point of the present chapter. It aims at a computational model based on a cognitive re-construction of context sensitivity. To this end, we assume 1. that this sensitivity is mediated by the Long-Term Memory (LTM) of the agents involved in communication. 2. More specifically, we assume that extrasituational contexts become accessible via the LTMs of the speakers and hearers, of the authors and readers or learners involved. 3. Thirdly, we assume that this concerns in particular schematic knowledge, which is represented in the LTM and is now to be reconstructed in parts with the help of Machine Learning (ML). By making this knowledge an object of ML, that is, by developing a machine learner for reconstructing schematic knowledge used by agents to contextualize texts, we finally aim at establishing a kind of procedural semantics that closes the gap to extensional semantics (as exemplified by situation semantics). This can be seen as a long-term goal of our research. To formalize our scenario of context sensitivity a little further, we start from the notion of relational meaning in situation semantics. On the most general level, it states that the meaning of an expression x is a relation d, c k x k r, e

ð14:1Þ

in which the variable d denotes a discourse situation of uttering x, c stands for connections of the corresponding speaker (e.g., of deictic expressions to real-world entities), e denotes a situation described by x in d, and r represents a so-called resource situation helping to reconstruct what is described by x in terms of e, for example, by means of the common ground of the speaker and hearer involved in d (Barwise and Perry 1983). An implication of this relational concept of meaning is that the same expression x can be interpreted differently (in the sense of describing different situations or events e) depending on the variation of contextual variables such as d, c, and r. By utilizing this notion, we aim to reconstruct assessments of text complexity also as a relational predicate. To this end, we modify Formula (14.1) as follows:

14

TextInContext: On the Way to a Framework for Measuring the Context. . .

d, c k x k r 1 , . . . , r k , e, φðxÞ

169

ð14:2Þ

where the variable d denotes the situational context of processing expression x (i.e., a text) in an educational situation by the learner L involved in d to perform a particular task described by x, e stands again for a situation or event described by x, and r1, . . ., rk denote resource situations utilized by L to interpret x in terms of e, that is, to contextualize x in conjunction with c. Finally, the expression φ(x) denotes the complexity assessment of x in relation to the context defined by the other variables of Formula (14.2), so that φ(x) is assumed to vary in analogy to the described situation e depending on L. Our idea now is to refer to cognitive and sociolinguistics to further specify the resource situations r1, . . ., rk. According to this approach, any ri, 1  i  k, denotes an extrasituational context (outside the scope of d ) that is represented in the form of knowledge that the learner L can retrieve from his or her LTM to interpret x in terms of a (preferably coherent) mental model (Kintsch 1998; Schnotz 1994; Van Dijk and Kintsch 1983).1 This knowledge can be primarily of a social, interactional, epistemological, ontological, linguistic, or any other nature (Barwise and Perry 1983; Goodwin and Duranti 1992, for enumerations of contextual resources). In any event, in accordance with the openness of the notion of context we do not assume a fixed set of types of knowledge-related resources. However, whatever is considered an extrasituational context that influences the assessment of a text in the sense of Formula (14.2) is understood by us to be mediated by the cognition of the learner L involved in d, that is, by his or her long- and short-term memory. This is exactly the starting point for our ML, since the knowledge represented in the LTM can be understood as being learned as a result of the processing of (linguistic and other) data. By modeling this learning with the help of ML, we want to contribute to establishing a novel semantics that reconstructs extrasituational contexts via the detour of automatically learned (linguistic) schemata derived from this data. In this way, we aim at grounding the otherwise indeterminable context units, which, for example, make situation semantics an operationally opaque undertaking, with the help of ML and the schemata learned by it: extensional definitions of contexts are then replaced by schemata learned with the help of ML (similar to a procedural semantics (Eikmeyer 1985; Rieger 1985)). Regardless of the interplay of linguistic and non-linguistic context variables, as considered so far, approaches to the evaluation of, for example, positive or negative learning (in the sense of Zlatkin-Troitschanskaia et al. 2018) either concentrate on a single (independent variable of a single) level of linguistic manifestation or train their classifiers by simultaneously considering different variables of different levels while excluding context effects (Mehler et al. 2018d). Direct consequences of this approach are that the resulting classifiers have to be retrained each time the learning context changes, that they are usually very data-intensive (with rare events not being taken into account or being underrepresented), and that they cannot necessarily be

1 This means that we understand e as a mental model and not as a situation in terms of an extensional semantics.

170

A. Mehler and V. Ramesh

transferred to comparable but slightly different learning contexts. With a given task descriptions, for example, such classifiers would always arrive at the same result for assessing its complexity, regardless of the learning situation and the extrasituational context of the learner. From this point of view, complexity is seen as a functional attribute of the given data stream and not as a relational attribute that depends on the context. Our hypothesis is that this is one reason for the still high amount of unexplained variance, for example, in studies on assessing learning outcomes (Mehler et al. 2018d). It relates to problems of approaches to Natural Language Processing (NLP) in educational research (Litman 2016) such as the predominance of black box models, the bottleneck problem as a result of data sparseness, and the lack of semantic interpretability of the models involved (Marcus 2018). In Mehler et al. (2018d), we demonstrated that quantitative linguistic approaches allow for predicting certain knowledge test results of students. We demonstrated that a certain amount of unexplained variance can be attributed to the linguistic manifestation of the underlying task description. Our analysis indicates a strong correlation between linguistic features of task descriptions and rates of correct student responses on these tasks (Mehler et al. 2018d, p. 180). This is a strong argument for computational linguistic analyses, which seem to be valid resources for better predicting student responses and thus negative or positive learning. The aim of the present work is to further develop this research and to establish a theoretical basis for it by accounting for context effects which are out of reach for the feature model presented in Mehler et al. (2018d). The chapter is organized as follows: in section “Requirement Analysis”, we present our requirements analysis for narrowing down relevant implementations of our model of context-sensitive interpretations of educationally relevant texts, henceforth called TextInContext. In section “An Architecture for Modeling the ContextSensitive Assessment of the Complexity of Texts”, we describe the modular architecture of TextInContext in detail and discuss a number of cognitive theories as candidates for implementing its building blocks. In section “A Systems Engineering View of Multimodal Cognition”, we briefly discuss a multimodal extension of our approach while section “On the Way to Modeling Networks of Learners: A Methodical Discussion” adds a brief methodological discussion. Finally, in section “Conclusion”, we evaluate TextInContext in the light of the requirements analysis from section “Requirement Analysis” and provide an outlook on future work to implementing TextInContext.

Requirement Analysis In this section, we briefly list requirements that a model of assessing the contextsensitive complexity of texts as sketched in section “Introduction” should meet. Rather than presenting a detailed evaluation by means of empirical data, the chapter concentrates on the significance and validity of our proposal to assess its quality. To this end, we consider seven requirements (Table 14.1):

14

TextInContext: On the Way to a Framework for Measuring the Context. . .

171

Table 14.1 Requirements for a model that can process and evaluate educational texts and their complexity in a context-sensitive way

3 4

Attribute Automated Capable of processing big data Responsive to data small Modular

5 6 7

Expressive Context sensitive Simulative

1 2

Requirement The model is based on machine learning The model is corpus analytic The model allows for interpreting rare events The model implements cognitive theories as modular building blocks The model maps various dimensions of complexity The model accounts for various context effects The model enables simulative predictions

1. Automation: as mentioned above, our proposal is based on ML for modeling context effects mediated by knowledge components retrieved from LTM. For this purpose, we aim at a kind of ML, which recursively builds on the output of NLP to derive set-, sequence-, tree-, and graph-like structures as building blocks of a text grammar, whose expressions are linked to units of knowledge representation, such as those provided by projects like Wikidata. These ontological units are required, for example, to classify and identify the expressions of such a grammar thematically, and thus to model their aboutness (Yablo 2014) relations as an elementary level of semantic representation. An important part of such a model concerns the embeddings of the expressions of the text grammar, which are computed to represent their syntagmatic and paradigmatic associations. Ideally, this whole procedure results in a hierarchical graph, whose nodes in turn can represent graphs (patterns) and whose edges can correspond to association or constituent relations of these nodes. In Mehler et al. (2018b), we described such a model based on genetic algorithms (see Mehler 2006 for an early proposal in this regard). 2. Capability of processing big data: Instead of being based on an extensional semantics where the modeler endows the model with his or her knowledge of the part of reality to be modeled (Rieger 2001), the ML we envision should necessarily be based on the analysis of large text corpora (this and related requirements have been described by Rieger 2001). From the point of view of ML, this requirement is a truism. It is also true that the semantics that can be modeled in this way necessarily falls short of a linguistic semantics in the classical sense. However, due to the bottleneck problem of educational data, we need to go beyond these truths. This will be done in section “An Architecture for Modeling the Context-Sensitive Assessment of the Complexity of Texts” with regard to random learners and in regard to transfer learning. 3. Responsivity and sensitivity to small data: The smaller the training data, the lower the quality of the generalization performed by ML, especially in the case of rare events. ML is good in cases where large training data are available so that rare events are less well covered. Due to Zipf’s first law (Zipf 1972), however, we know that regardless of the size of a corpus, the percentage of its hapax legomena

172

A. Mehler and V. Ramesh

is around 50%. If this were a problem for a human reader, natural language texts would be largely incomprehensible. Thus, it is required that an adequate ML addressing the program described in section “Introduction” is capable of interpreting hapax legomena or hapax events of whatever sort. 4. Modularity: A model of context-sensitive text complexity requires a model of text comprehension whose modules are each based on cognitive theories. To narrow down such a modular architecture we address the following research questions (RQs): RQ1. Which cognitive theory can be used as a starting point for measuring the complexity of texts in a context-sensitive manner? To answer this question, we consider the cognitive load theory (CLT) of Sweller et al. (2011). Thus, we distinguish between intrinsic and extraneous cognitive load, where the former is affected by prior knowledge (i.e., learned schemata). From this point of view, the so-called element interactivity among the constituents of the learning input is a reference point for measuring its contextsensitive complexity. This leads us to the second question: RQ2. Which model of text comprehension can be used to identify components of texts whose interactivity must be evaluated in terms of CLT or related approaches to assess the complexity of these texts? We tackle this question by considering the Construction Integration Model (CIM) of Kintsch (1998). Due to its dual instantiation in the form of statistical and propositional models (Kintsch 1998, 2008), it can help to build a bridge between these two approaches and thus to semantically ground the entire measurement process. RQ3. Starting from a learning theory and a theory of text comprehension, are there alternatives to the dominant feature model and in particular to “endto-end” approaches in ML? To answer this question, we consider the theory of Conceptual Spaces (CS) (Gärdenfors 2000) by distinguishing between quality dimensions, domains, and concepts derived from them. In this way, a layer is introduced between the layer of elementary characteristics on the one hand and the target variables (success rate, degree of complexity, etc.) on the other, which could help to increase the interpretability of the resulting classifiers and reduce the black box character of ML. Ideally, this feature model itself is based on a relational semantics that makes each of the factors of the complexity of task descriptions context-sensitive (see Requirement 6). 5. Expressiveness: Measuring text complexity is an established topic in quantitative linguistics—usually examined in readability studies (Anderson and Davison 1988; Islam and Mehler 2013; Mikk 1995; Tuldava 1993). A situation semantic account of complexity requires going beyond such purely linguistic approaches. This may include dimensions such as the multitude of task goals and the ways to achieve them in the sense of “complexity as a function of objective

14

TextInContext: On the Way to a Framework for Measuring the Context. . .

173

characteristics” (Campbell 1988, p. 44), prior knowledge (considering complexity in relation to task–performer interactions), and even personal interest and arousal (Campbell 1988). Analogously, with Liu and Li (2012) we can distinguish goal- and output-oriented dimensions from those that refer to the input, its presentation mode, the available time for task completion, and alternative methods for task fulfillment. A third alternative is proposed by Stouten and Größler (2017) who distinguish size, variety, redundancy, ambiguity, variability, inaccuracy, novelty, incongruity, connectivity, and temporal demand as complexity dimensions. Regardless of which model serves as a basis for leaving the narrow path of purely linguistic approaches, a model of task complexity should account for dimensions regarding the interplay of task descriptions, learners, situational features (e.g., time budget, presentation mode, availability of resources), and described situations (e.g., from the perspective of input–output relationships and their mediation through procedural alternatives). 6. Context sensitivity of complexity dimensions: We aim to measure the complexity of task descriptions to better predict or even to explain test results of different learners. Regarding the notion of task complexity, Campbell (1988) and Liu and Li (2012) distinguish objective approaches (whose complexity assessments are independent of the performer or learner) from subjective approaches (for which the complexity depends on the interaction between task and performer). We adhere to the latter group of approaches. That is, rather than relying on a concept of task complexity as a “function of objective task characteristics” (Campbell 1988, p. 40) we adhere to a dynamic notion of complexity as a result of the interaction of task- and context-, of task- and learner-related characteristics (Campbell 1988). Therefore, we need to relate each dimension of the complexity of task descriptions, which we consider a relevant unit of measurement, to the various contextual factors that we assume influence text comprehension. Liu and Li (2012) distinguish, for example, clarity, redundancy, and quantity as dimensions of the complexity of a task. From the perspective of task–person interaction, Campbell (1988) distinguishes, for example, experience, familiarity, and capacity of the short-term memory as dimensions influencing the complexity of a task (in our case: of a task description). Modeling complexity assessments according to a subjective approach then means showing how such dimensions are influenced by what we model as the short- and long-term context of the learning of a particular (type of) learner. 7. Simulation: In addition to reconstructing test results as collected by Test of Understanding in College Economics (TUCE) (Zlatkin-Troitschanskaia et al. 2014) and related projects, the model should be able to operate on simulated data to enable predictions based on unforeseen texts. This is necessary, for example, to enable transfer learning in relation to areas of education for which test data are out of reach or consist only in the form of small data. Another reason concerns text design, which involves creating learner-adapted texts, or intelligent tutors who support the learner in processing complex texts.

174

A. Mehler and V. Ramesh

In addition to using ML to better adapt texts to the needs of learners (according to the program of adaptive hypertexts (Brusilovsky 2001), another reason to investigate text complexity in simulated environments is to examine language change as a function of the development of text structures—be it in view of their shortening, for example, in online media such as Twitter or in view of the collaborative writing of documents in online media such as Wikipedia, Overleaf, and Google Docs. We will come back to this point in section “On the Way to Modeling Networks of Learners: A Methodical Discussion”.

An Architecture for Modeling the Context-Sensitive Assessment of the Complexity of Texts This section describes a conceptual model for the integration of building blocks (Requirement 4) for modeling the context-sensitive assessment of the complexity of texts (Requirements 5 and 6). This abstract model is schematically depicted in Fig. 14.1: it contains an exemplary instantiation by means of CLT, CIM, and CS (Requirement 4). We call any such instance of our conceptual framework for implementing the research program of sections “Introduction” and “Requirement Analysis” a model of TextInContext. The multi-layer architecture of our proposal for instantiating TextInContext distinguishes five modules: a framing learning theory, a discourse theory embedded into this learning theory, a feature theory for linking the discourse theory with models of text complexity, a schema learning theory for capturing long-term contextual effects (Requirement 1), and a big data model

Fig. 14.1 The architectural setting of TextInContext including five modules (left side) and their instantiation (right side) by means of CLT (Sweller et al. 2011), CIM (Kintsch 1998), CS (Gärdenfors 2000), graph embeddings, and models of collaborative writing

14

TextInContext: On the Way to a Framework for Measuring the Context. . .

175

(Requirement 2), which could provide a way out of the data bottleneck problem in the area of educational text data mining.

Module M1: Learning Theory The underlying learning theory to be specified by Module M1 concerns the core of TextInContext, from which all other modules are derived one after the other. It aims to mediate between short- and long-term aspects of the context sensitivity of the processing of learning objects and task descriptions based on them. Any instantiation of M1 has to specify how the processing of input texts is influenced by the activation of knowledge from the learner’s Long-Term Memory (LTM) and by the learning situation itself. This dual “provenance” of information can be located with the help of Schnotz’ (1994) text understanding model: When processing a text, the learner integrates information as the result of top-down processes of information processing on the one hand and bottom-up processes of transforming input data into information on the other. The former are expectation-driven processes based on prior knowledge retrieved from the LTM. The latter are data-driven processes for converting elementary data units of the respective input stream into increasingly complex data representations. By integrating both types of processes, the reader creates a mental model as a cognitive representation of his or her understanding of the input text. In terms of our reference to situation semantics, the mental model relates to what Barwise and Perry (1983) understand as the described situation, while the resource situation relates to what is retrieved from the LTM to constrain the generation of the mental model. Finally, the discourse situation concerns the situational (e.g., medial) context of learning. Using this cognitive terminology, the statement about context sensitivity of section “Introduction” remains unchanged: The same learner may understand the same task description differently on the basis of different prior knowledge or situational learning contexts. A learning theory which describes this context sensitivity to assess the complexity of learning material is the Cognitive Load Theory (CLT) (Sweller 1994, 2003; Sweller et al. 2011). It distinguishes three factors that influence a learner’s cognitive load when learning information from a particular learning material (here we only consider the first two of them): intrinsic, extraneous, and germane cognitive load. Intrinsic Cognitive Load (ICL) refers to the complexity inherent in the information units to be learned. It is induced by what is called the interactivity of input elements (i.e., “material that needs to be learned” (Sweller 1994, p. 304): If the elements can be learned independently, they induce low interactivity and, vice versa, if they have to be learned in relation to each other, they induce high interactivity. The load on the working memory induced by this interactivity can be reduced by schemes stored in and retrieved from the LTM (Sweller 1994, p. 299; Sweller 2003, p. 221; Sweller et al. 2011, p. 64). That is, prior knowledge in the form of learned schemata can then

176

A. Mehler and V. Ramesh

reduce ICL, as it gives the learner a prior understanding of the relationships of components (elements) of the learning material, the interactivity of which must otherwise be understood and processed in detail. As a consequence, we arrive at a learner-dependent theory: “A precise measure of element interactivity that is independent of the learner is unobtainable because [. . .] what constitutes an element is affected by the knowledge of the individual” (Sweller 1994, p. 306). Extraneous Cognitive Load (ECL) relates to the presentation or manifestation of the learning material to the learner, which can disrupt learning (e.g., because of its multimodal complexity): it “is a function of the cognitive activities required of students because of the manner in which the information is presented” (Sweller 1994, p. 306). Ideally, learning scenarios support the acquisition of appropriate schemata (by inducing germane cognitive load (Sweller et al. 2011, p. 57) while reducing ECL. According to Kirschner et al. (2011a), a central problem of CLT is the lack of valid complexity measures that reflect the variety of complexity dimensions assumed by this theory. Although we will not close this gap, we will at least provide an outline of how this theory can be instantiated in terms of a schema theory (in sections “Module M4: Schema Learning Theory” and “Module M5: Transfer Learning for Enabling Big Data Analysis of Educational Text Data”) and a theory of text comprehension (in section “Module M1: Learning Theory”) to link to ML. The reference to models of text comprehension is necessary to operationalize the measurement of element interactions as predicted by CLT as a central factor of ICL. Our idea is essentially that the cohesion and coherence relations between text components induce element interactivity to facilitate or inhibit the generation of a coherent mental representation of the input text. A theory that models such processes is the construction integration model of Kintsch (1998).

Module M2: Discourse Theory Modeling the interactivity of text components on the one hand and knowledge units retrieved from the LTM on the other requires a model of text comprehension that takes into account the interpretation of ambiguous components and their semantic integration with what the learner has already stored in his or her memory. Such a model is proposed by Kintsch (1988, 1998) in terms of the so-called Construction Integration Model (CIM). Roughly speaking, CIM distinguishes between two processes of text comprehension, which are not necessarily linearly ordered, but can run in parallel to generate a mental model as a preferably coherent interpretation of the respective text input: During the construction phase, largely context-free, bottom-up processes generate a potentially incoherent network of meaning representations that may contain competing interpretations of the same input. In the subsequent integration phase, context-adequate constructions are strengthened and inadequate ones

14

TextInContext: On the Way to a Framework for Measuring the Context. . .

177

suppressed by recourse to the processing context—especially through the impact of knowledge retrieved from LTM.2 Since construction processes do not necessarily produce a coherent mental representation of a text, integration processes are required to reinforce coherent, strongly connected nodes or sub-graphs in STM based on a spreading activation process that operates on the outcome of the construction phase while suppressing incoherent nodes. In this model, the context does not work as a filter that only allows for constructing context-adequate representations, but as a system of constraints that constructions have to meet to “survive” the integration process. In this sense, CIM can be assumed to describe a constraint satisfaction process which reminds of approaches in the area of parallel constraint satisfaction (John and McClelland 1992) and neural networks, which finally come into consideration as candidates for implementing CIM. This makes this model even more interesting for us: While older implementations of CIM are based on propositional text semantics in the tradition of van Dijk and Kintsch (1983), newer implementations and extensions of CIM are based on numerical semantics to model processes of spreading activation and constraint satisfaction based on association relations of interactive elements (Kintsch 2001, 2008). This is exactly where the integration of CLT and CIM in the sense of Requirement 1 becomes relevant, as far as the automation of text representation using ML is concerned: that is, instead of propositional networks and their experience-based elaborations we consider numerical text representation networks and their elaborations according to Requirement 1 as models of text bases and the mental models derived from them (Fig. 14.2).3

Module M3: Feature Theory Although CIM provides a conceptual framework for modeling element interactivity in accordance with CLT, it does not specify reference points for the context-sensitive measurement of task complexity. What is additionally required is a feature model that correlates various influencing factors with complexity itself (see Requirement 5 of section “Requirement Analysis”) and the result of learning along its poles of positivity and negativity as distinguished in Zlatkin-Troitschanskaia et al. (2018). At this point of our analysis, we must avoid falling into the trap of classical feature approaches, as they prevail in computational linguistics. Along this predominant paradigm texts are quantified by feature vectors whose dimensions are implicitly seen as separable. Thus, while (syntactic, semantic, etc.) classes of the features

2 While the LTM is constraining and contextualizing the construction-integration process, the STM is considered to be the locus of this process (Wharton and Kintsch 1991). 3 See Mehler (2007) for an early version of a related model in the context of a numerical re-construction of the principle of compositionality that starts from Kintsch’s (2001) predication theory.

178

A. Mehler and V. Ramesh

Fig. 14.2 The visual depiction of CIM according to Wharton and Kintsch (1991) and its modification by reference to numerical semantics: Instead of propositional networks and their experiencebased elaborations, numerical text representation networks and their elaborations according to Requirement 1 are regarded as models of text bases and the mental models derived from them

that define the latter dimensions are assumed, these classes are usually considered irrelevant beyond feature selection. Based on this approach, ML selects subsets of partially integral or separable features during training, making learning ultimately opaque so that we do not know which features are effective in which way by interacting with which other features when applying the trained classifiers. Modeling is then reduced to extending feature spaces by adding more and more features, and then optimizing hyperparameters. Ultimately, feature modeling is replaced by endto-end learning so that modeling in the classical sense becomes irrelevant, while the engineering of increasingly complex architectures based on neural networks gets central.4 A way out of this fallacy is provided by the theory of Conceptual Spaces (CS) of Gärdenfors (2000, 2014). Roughly speaking, this theory distinguishes between dimensions as qualities of objects endowed with a topological structure, domains as subspaces of conceptual spaces, which are spanned by integral dimensions (while 4 See Lee et al. (2017) as an example of an approach to end-to-end learning by example of coreference resolution, which is, however, very successful.

14

TextInContext: On the Way to a Framework for Measuring the Context. . .

179

separable dimensions are considered to belong to different domains), properties as convex regions of corresponding domains, concepts as correlated convex subregions of several domains, prototypes as centers of certain regions representing concepts, learning as the formation and hierarchization of domains and concepts, and finally vagueness as the weighting of dimensions, fuzzy regions, and dynamic prototypes. The difference between Gärdenfors’ semantics and classical black box approaches to ML is illustrated in Fig. 14.3. The upper part shows a ML-based approach (black box) to automatically assessing positive (+) or negative () learning using a set of feature dimensions F1, . . ., Fn that are assumed to be separable and ideally orthogonal. From the point of view of feature selection, evaluation concerns the modification of the feature space itself. The lower part exemplifies a conceptual space by interrelating the complexity dimensions of Stouten and Größler (2017) to outcomes of positive or negative learning. This example starts from five levels (code, medium, mode, situational or short-term context (ST-context), and extrasituational or longterm context (LT-context)) to induce a range of dimensions D11, . . ., D51, . . . In this model, integral dimensions of the latter range are integrated by means of domains Dom1, Dom2, . . ., which serve to represent concepts as representations of the contents of the original texts, and which in turn are used as reference points for the assessment of the corresponding complexity dimensions according to Stouten and Größler (2017), finally to be related to the learning outcomes of the learner under consideration. In this approach, the levels of area, dimension, domain, concept, and complexity become reference points of evaluation or revision of the operational model, which makes the whole process less opaque compared to the standard model of ML (Fig. 14.3). In addition, several of these levels can be viewed individually or in combination to interpret the result of ML based on this architecture. In any case, through this approach, models of the short- and long-term context (ST-context, LT-context) gain influence on complexity dimensions that are differentiated in this example, just as it is demanded by Requirement 6. An example of this approach, which illustrates the role of prototypes in conceptual spaces, shows Fig. 14.4 relying on the task of assessing the complexity of treelike structures: using the approach of Mehler et al. (2018a, c) for modeling topological structures of trees, this example distinguishes a range of dimensions as, for example, the order, dependency, depth, or imbalance structure of trees as different dimensions (for the formal details, see Mehler et al. 2018a, c) that are integrated to different domains. The next step is to derive from these domains a series of distinct types of trees, each of which is represented by one or more prototypes. In this way, we distinguish, for example, the concept of a balanced binary tree (C1), a line graph (C2), a caterpillar graph (C3), and a star graph (C4). Finally, the complexity of any input tree T is assessed by measuring its similarity to the latter prototypes (for related concepts of graph similarity, see Koutra et al. 2011, 2014). In this way, the entire process gains in interpretability. The reason is that the prototypes stand (or can stand) for well-known classes of trees used as reference points for assessing the complexity of an input tree. It is obvious how this approach can be transferred to the evaluation of natural language texts.

180

A. Mehler and V. Ramesh

Fig. 14.3 Above: Depiction of a ML-based approach (black box) to automatically assessing positive (+) or negative () learning using a set of feature dimensions F1, . . ., Fn that are assumed to be separable and ideally orthogonal. Below: Exemplification of conceptual spaces by means of assessing the complexity of task descriptions in relation to positive and negative learning

14

TextInContext: On the Way to a Framework for Measuring the Context. . .

181

Fig. 14.4 Left: Deriving concepts from domains of characterizing trees in terms of their dependency, depth and distance structure, etc., where these domains are in turn derived from a series of dimensions all of which are utilized to characterize the topology of trees. Right: four concepts of trees, each of which represented by a prototype, are referred to assess the complexity of an input tree T

According to this approach, we can distinguish three levels of models that increasingly allow for interpretability: 1. Zero-level models concern the classical approach of feature engineering in ML (Fig. 14.3). 2. First-level models concern approaches that are derived from conceptual spaces and offer a higher level of interpretability than zero-level models by referring to concepts as functions of domains as functions of quality dimensions: While the latter dimensions resemble the feature spaces of zero-level models, interpretability can be established by recourse to domains and concepts (as well as prototypes) derived from them. 3. Second-level models get into reach by extending conceptual spaces in terms of Zadeh’s approach to computing with words (Zadeh 1997), in which text resources can be modeled by means of fuzzy granules (concepts), their attributes (dimensions or domains), and by the values these attributes take. In such a model, we can think of learning fuzzy rules of the sort imbalancedðT Þ ^ deepðT Þ ^ stratifiedðT Þ ) complexðT Þ

ð14:3Þ

for representing, for example, the complexity of tree-like structure. Ideally, textual resources are modeled on the basis of a second-level approach, so that any assessment of their complexity can be interpreted directly on the basis of fuzzy rules, as illustrated by Formula (14.3).

182

A. Mehler and V. Ramesh

Module M4: Schema Learning Theory The task of module M4 is to operationalize contextual units that do not belong to the situational context of a learning situation, but refer to what has to be retrieved from LTM to influence the processing of the current task description. This is necessary because CLT, CIM, and CS are underspecified regarding the computational specification of such units. Though CLT provides a conceptual framework for reconstructing the situation semantic analysis of the context-sensitive interpretation of task descriptions, it says not very much about instantiating its (computational) building blocks. It does not introduce, for example, a computational model of schema knowledge as part of the LTM: neither in terms of the type of schemata learned nor in terms of their scope, interactions, or types of acquisition. Conversely, the descriptive framework provided by situation semantics is underspecified with respect to the exact types of resource situations and their constitutional and delimiting conditions. One oscillates, so to speak, between context theories that emphasize the sensitivity of language comprehension (e.g., of understanding task descriptions) to various aspects of context (e.g., the learning situation), but do not offer measurement procedures for delimiting observational instances of these theoretical terms. At this point, we achieve a predetermined breaking point with our approach. The reason for this is that, due to our computational orientation, we do not aim to model the schema knowledge of a given learner experimentally, but to automatically reconstruct such knowledge by analyzing large corpora of textual as well as spoken and non-textual data. Thus, rather than modeling the cognitive state of only of single learners, we also aim at inferring context providing schema knowledge from text corpora to model types of different learners. This approach runs the risk of creating as large a corpus of texts as possible to model an unrealistically overactive learner who is supposed to have read everything such a corpus offers. To avoid this pitfall, we propose instead to distinguish types of learners whose instances perform different but characteristic random walks through such corpora. The background to this approach is that all text-based information that a person has represented in her LTM must have been constituted as a result of processing written or spoken data streams. Of course, focusing on such data streams falls short of a theory of what a person mentally represents through text comprehension, since it omits, for example, the modeling of any kind of embodied information processing. According to the symbolic orientation of projects based on big text data, however, such an approach aims at those data streams which are accessible as symbolic data at all for implementing elementary context models. Furthermore, we assume that the more elementary the LTM-related schemata are in the linguistic sense (starting from phonological/graphemic representations in the direction of what cognitive linguists call situation models), the more valid our model of these schemata is as a result of applying a corpus-analytical approach. A positive example of this, which has recently received much attention, are so-called word embeddings as models of syntagmatic or paradigmatic associations of lexical elements (Komninos and Manandhar 2016; Levy and Goldberg 2009; Ling et al. 2015; Mikolov et al.

14

TextInContext: On the Way to a Framework for Measuring the Context. . .

183

2013). This is precisely our approach, that is, the representation of linguistic information through the exploration of as large a corpus of linguistic data as possible as a starting point for modeling schema knowledge that contextualizes situated processes of information processing by being partially retrieved from the LTM. More specifically, we are proposing to create a new type of learning corpus that includes all the possible texts that learners in a particular field of education (e.g., economics) might have read, prior to the processing of corresponding task descriptions. Such a corpus would contain a wide range of data of Computer Mediated Communication (CMC) as, for example, of social news aggregation and discussion sites, social networking sites, question-and-answer sites (e.g., Reddit, Twitter, and Stack Exchange), slide sharing sites, MOOCs from E-learning platforms, etc. It would also include online encyclopedia and related resources (Wikipedia, etc.), but also newspaper sites and video platforms (e.g., YouTube). The reason for this is to expand our comparison base and capture a wide range of online data that learners access when searching for information on the web to get an understanding of their learning tasks. However, the kind of learning corpus that we conceive should also contain textbooks and related written or spoken material in the respective field of education (Litman 2016). Ideally, such a corpus also contains entertainment texts, the reading of which interrupts the reading of task-related ones. The idea is now to infer types of random learners under the condition of approximating the test outcomes of real learners as reported, for example, in the Test of Understanding in College Economics (TUCE) (Zlatkin-Troitschanskaia et al. 2014) corpus of task descriptions and test datasets. That is, we think of random walks through our learning corpus which is represented as a network of intertextually linked texts, dialogs, etc. That is, in analogy to the notion of a random surfer (Brin and Page 1998) we conceive random learners whose validity is based on their ability to predict the outcomes of tests such as those collected by TUCE. In this way, we want to answer questions of the following kind: Which time series of texts spanning which textual subnetwork of the underlying learning corpus must a learner have read with what probability to which degree to derive which schema knowledge based on which he or she achieves which result in which test based on which task description?

The spectrum of such textual time series and the schemata derived from them—as models of the reading history and LTMs of respective types of learners—would then provide different contexts for achieving comparable outcomes: a variation of this spectrum leading to a different test result would provide a model of a form of context change that appears to be reflected in a significant change in the interpretation of the underlying task description—in accordance with Formula (14.2). From this perspective, the time series defined over the respective text networks are then to be explored as data streams for inferring schema knowledge, for example, on the level of lexical embeddings (see above), multi-word expressions (Sag et al. 2002), phrasal schemata, and constructions as described, for example, by pattern grammar (Hunston and Francis 2000) or construction grammar (Van Trijp et al. 2012). Furthermore, we can consider schematic knowledge relating to semantic and

184

A. Mehler and V. Ramesh

thematic patterns as described, for example, by research on textual entailment (Dagan et al. 2013). In all these cases, computational linguistics has reached a level that makes it likely that such schemata can be automatically learned to reliably predict the targeted learning outcomes. However, no research has yet been conducted along the research line and the research questions dealt with by TextInContext.

Module M5: Transfer Learning for Enabling Big Data Analysis of Educational Text Data Module M5 is a framing module that addresses the problem of data acquisition inherent in any approach to assessing the context-sensitive complexity of natural language texts. The starting point is the constrained accessibility of learning histories and in particular of reading histories as addressed by Module M4. Since our goal is to model knowledge stored in long-term memory as a starting point for reconstructing resource situations, it is necessary to process everything a reader has read or communicated in detail. This is necessary to capture effects of context-sensitive text comprehension beyond the situational context. In fact, the corresponding data streams are usually not accessible. To find a way out of this data bottleneck problem, we can employ transfer learning. Transfer learning is an approach to machine learning in which knowledge acquired in one area is transferred to a related area for which there are no (sufficiently) trained models (Silver and Bennett 2008). In the case of task descriptions in the context of TUCE, we observe that such descriptions resemble text snippets rather than longer texts and that the learning history of the respective learner is just as inaccessible as his or her reading context or group-related learning data in cases where learning takes place in groups. In such scenarios, machine learning can hardly go beyond the data of a mostly small set of task descriptions. In fact, machine learning in such an area is based on rather small text data (Mehler et al. 2018d). Although we can use feature expansion or large-scale pre-trained classifiers that allow detailed annotations, for example, of the topics addressed in text snippets (Uslu et al. 2018), the underlying data is essentially limited to the “small possible worlds” described by such snippets. From the point of view of model-theoretical semantics, this is an advantage. From the point of view of machine learning, however, this is a major drawback: based on such a data set, a ML-based approach would inevitably fall short of a theory of context-sensitive text comprehension. To overcome this bottleneck problem, we suggest deploying transfer learning by transferring as many hypotheses as possible about learning in educational environments to the field of large online social network data available through collaborative online media such as Wikipedia and related systems. Such a transfer scenario is schematically depicted in Fig. 14.5: Starting from a text (task description) T, the complexity of which is to be assessed in a certain, not fully controllable situational

14

TextInContext: On the Way to a Framework for Measuring the Context. . .

185

Fig. 14.5 Transfer learning as a means to solve the problem of data sparseness of studies of the reading histories of learners as access points for modeling long-term knowledge representations as models of resource situations

context St of reading T at time t by the reference learner L, we assume that L has processed a corpus of texts T1. . .Tm and participated in a series of communication events with the interlocutors L1. . .Li before t, in some order and depth, to acquire a certain but not fully accessible knowledge base from which L retrieves information to process T in the situation St. Since the time series of the networked texts T1. . .Tm and the communications with interlocutors L1. . .Li are not fully accessible to us, we can hardly model the schema knowledge that is available to L in St. This means that we cannot directly assess the complexity of T with respect to L in relation to the longterm knowledge of this learner. Therefore, we suggest—in addition to the random learner model of section “Module M4: Schema Learning Theory”—to model this knowledge on the basis of a transfer scenario in which the learner L is related to an author A of a collaborative online medium. In line with this approach, the time series of networked texts T1. . .Tm is mapped to a time series of networked web documents D1. . .Dm, to which A and his or her collaborators A1. . .Aj contributed before or during the writing of D in the uniquely determinable time span u, so that in this analogy the web document D corresponds to the text T. In this way, we gain access to

186

A. Mehler and V. Ramesh

the editing histories of a huge set of texts and their collaborative writing and discussion by a huge set of authors and wikilocutors (Mehler et al. 2018c) in correspondence to the hidden data on dialogical and multilogical communication among the learners involved. Since we are interested in types of random learners (see section “Module M4: Schema Learning Theory”) that allow for predicting test outcomes as reported by TUCE, we will need to study types of random authors A. In this way, types of random authors A will be finally related to types of random learners L corresponding to them (Fig. 14.5). Beyond Wikipedia, resources are to be explored in this context, which also play a central role in the development and testing of models of random learners according to section “Module M4: Schema Learning Theory”. This includes not only releases of Wiktionary (Mehler et al. 2017), Wikibooks, Wikiversity, and Wikinews in various languages (e.g., Chinese, English, French, German, Japanese, Korean), but also special wikis and communication platforms for knowledge communication and related areas. Furthermore, this includes Wikidata as a terminological ontology which is unified for all languages for which there are releases of Wikipedia and, thus, allows for identifying entities and concepts across a large number of languages. In each of these cases, the corresponding article, category, and discussion spaces are to be explored to get access to written language as well as to conceptual orality in the sense of Koch and Oesterreicher (1994). In addition to linguistic data, this approach accounts for social data on interacting agents who cooperate in writing online documents or participate in their online discussions. In this way, our approach combines the observed use of language with aspects of the underlying interaction behavior, so that the complexity of textual data can be related to social contexts. In any event, our procedure requires a reformulation of hypotheses about learners of the sort of L (L) in terms of collaborative writing by authors of the sort of A (A) (Fig. 14.5). This can be exemplified by transferring a sample hypothesis of Sweller et al. (2011) according to which group learning facilitates the processing of complex tasks (based on respective task descriptions) (see also Kirschner et al. 2011a, b). By transferring this hypothesis to the area of collaborative writing we may ask, for example, whether writing in groups facilitates the collaborative writing of complex articles about difficult topics. More specifically, we may ask: Is a text more understandable (and, thus, less complex) if many authors are involved in its writing or if it is discussed intensively or even controversially? A second example that relates directly to TUCE concerns the question whether the distance correlations between linguistic variables on the one hand and test outcomes on the other as observed in Mehler et al. (2018d) can also be observed for other languages not contained in the TUCE corpus. More precisely, we can ask whether these correlations are in line with Nisbett’s hypothesis (Nisbett 2003) about cultural differences as manifested by Asian in contrast to Indo-European languages—apparently online data open the field for the study of whole language families and not only of individual language pairs. A third scenario that also relates to TUCE concerns the impact of using specific linguistic constructions, for example, on the sentence level on the complexity of task descriptions (in the sense of the notion of element interactivity of CLT). In this context, we can ask, for example, whether the use of a certain pattern (e.g.,

14

TextInContext: On the Way to a Framework for Measuring the Context. . .

187

Fig. 14.6 Average geodesic distance hdi as a function of the number of wikilocutors per wikicussion in a sample of 10,000 discussions of the German Wikipedia (see Mehler et al. 2018c for details about this corpus)

if. . .then constructions) reduces or does not reduce the complexity of task descriptions in the field of education in economics. Once transferred to the field of online writing, we can explore the extremely rich databases for far more than 100 languages provided by Wikipedia to explore these and similar questions. Given the amount of data on collaborative authoring in online media such as Wikipedia, such questions are relatively well answerable (see Mehler et al. 2018c for a recent study on available data, for example, on online discussions and their law-like hierarchical structure). Figure 14.6 illustrates these data about wikicussions, that is about discussions of Wikipedia articles by means of a sample of 10,000 social networks, which manifest the networking among the underlying interlocutors. Figure 14.6 shows the average geodesic distances hdi (y-axis) as a function of the number of different wikilocutors participating in these wikicussions (x-axis). In each of the social networks on which these statistics are based, a wikilocutor is linked to another participant in the same discussion if and only if the former responds to a post of the latter. Although these networks are directed, hdi is remarkably small and oscillates around 3: regardless of the number of participants in a wikicussion, we can assume that, on average, there are very short communication paths between participants in the same discussion, which makes it likely that there is a closely connected core of wikilocutors that dominates the distribution of a discussion’s threads. Whether the same applies to groups of learners preparing for the same test is still unknown. However, by analyzing communication behavior in online media, we obtain evidence of the regular organization of the formation of communication groups, which can eventually be transferred back to scenarios of group learning. This

188

A. Mehler and V. Ramesh

can help to derive conditions, for example, for the pre-selection of random learner models as suggested in section “Module M4: Schema Learning Theory”. Finally, by employing such an approach we gain a strong and easily accessible testbed to test hypotheses transferred from the field of educational data to the field of collaborative writing, which would significantly broaden our empirical base far beyond what is currently available for training machine learning routines using small text data.

A Systems Engineering View of Multimodal Cognition Educational data is multimodal and includes diagrams, videos, slide presentations, animations, and multimedia content. The methodology articulated in the above sections for fusing cognitive approaches to address the design of context-sensitive, explainable systems for natural language content analysis has relationships to parallel activities in computer vision. In our model-based systems engineering philosophy for computer vision (Greiffenhagen et al. 2001), the vision system is seen as producing a mapping from a scene instance to semantics. The process for real-time inference is viewed as “indexing” involving a massively parallel process that produces hypotheses followed by a sequential and deliberative “estimation” step that refines these hypotheses. This paradigm is analogous to a “coarse-to-fine” paradigm of sequential statistical estimation. Formal design of this coarse-to-fine estimation engine requires the essential elements of: “Bayesian Network specification,” “Design of Hypothesis generators motivated from the Bayesian network,” “Systems analysis of hypothesis generator to develop a statistical likelihood model for the features it generates as a function of the Bayesian network variables,” and a final estimation step involving belief propagation or fusion to perform global inference. Indexing addresses computational speed requirements, while the “estimation” step addresses accuracy requirements. Our original work (Ramesh 1995) allowed for more general approaches to the design of inference engines and used the Bayesian modeling formalism only as a mechanism to specify the application context. We essentially assumed that an expert designer provides the structure of the system design that is appropriate to solve a given vision problem and the task of a systems engineer was to analyze how a given design performs via the use of formal statistical generative models of context and to optimize the tuning parameters in the designed system. In this work, the space of possible system designs is considered to be independent of the formal models of contexts. This uses a “separation principle,” that is, the design space for composing systems is independent of the model space for applications. Since 2010, the advent of Big Data, increased computational power, and advanced training algorithms, deep learning methods enable a wide range of applications, albeit with a black box property (as articulated in the earlier sections of this paper). ML systems development nowadays is supported by a plethora of open-source platforms, programming libraries, and by GPU/high performance computing and cloud-based computing. Rapid applications engineering in this context

14

TextInContext: On the Way to a Framework for Measuring the Context. . .

189

involves: data preparation, curation and annotation, neural network architecture selection by human expert based on intuition and trial-and-error, loss-function specification, and optimization. The challenge in data-driven approaches is thus abstracted away from a standardized training and optimization methodology, onto choices of end-to-end neural network architectures and their hyperparameters and an intensive dataset assembly process. Meta-learning of architectures is an active research topic aiming to automate the selection process of deep neural network architectures and its individual mathematical modules. A number of recent efforts address the design of interpretable vision systems with deep learning (see review by Zhang and Zhu 2018). Our recent work has been on the use of simulation tools for cognitive vision and on illustrating model-based design for cognitive vision involving anomaly extraction (Veeravasarapu et al. 2017a, b; Weis et al. 2017). The focus of our current work in cognitive vision is to adapt and fuse systems engineering principles with modern ML advances to address the challenge of designing systems that are context sensitive, explainable by design, and whose performance can be quantified in context fusion of this perspective with the integrative approach presented in the earlier sections is a potential direction for approaching the synthesis and analysis of context-sensitive, explainable multi-modal cognitive systems for education. In this way, our research proposal finally goes beyond assessments of purely textual learning material and also addresses the extraneous cognitive load as distinguished by CLT.

On the Way to Modeling Networks of Learners: A Methodical Discussion In this section, methodological requirements and possible extensions of our approach are briefly discussed to prepare it for measuring learning in larger groups of cooperating agents. To this end, we start from the notion of multi-layered socialsemiotic networks (Mehler 2008; Mehler et al. 2018d) to model the interaction of long- and short-term learning contexts on the one hand and the complexity of linguistic information on the other. According to this approach, we assume a circular process of mutual constitution between language use and the underlying subsystem of language being distributed over a community of speakers/hearers generating the latter system in a (hyper-)incursive/recursive manner (Andersen 2002; Leydesdorff 2009). As a result of this circular process (allowing for short- and long-term learning (Gärdenfors 2000; Kintsch 1998; Sweller et al. 2011)), for short- and long-term alignment among interacting agents (Pickering and Garrod 2004), the complexity of the same linguistic input varies depending on the learning history of the agents under consideration. Ideally, this means that complexity assessments become relational attributes that include individual and groups of learners and models of their learning histories as additional arguments. Starting from this theoretical approach, we have to assume at least three layers of multi-layered social-semiotic networks: regarding the

190

A. Mehler and V. Ramesh

underlying community of agents (e.g., learners, writers, discussants, etc.), the corresponding language subsystem (e.g., biology, chemistry, economics), and situations of language use (e.g., writing or task completion). The circular process by which the latter two systems are embedded in the first is associated with mechanisms of self-organization of natural languages, which are known to evolve into fluent equilibria (Köhler 1987). A typical manifestation of such an equilibrium is given by core-periphery structures in conjunction with scale-free distributions of the operative elements within the same layer or between different layers. To model multi-layered networks, one can deploy ML for inducing (attributes of) their vertices and edges, respectively. Any such approach integrates methods of at least three areas of computational modeling: 1. Computational linguistics: rather than processing simple lexical features derived directly from textual input (e.g., task descriptions), one can experiment with feature grammars for generating sequence-, tree-, or graph-related features feeding evolutionary searches for the best performing feature model (Mehler et al. 2018b). Instead of concentrating on optimizing classifiers in relation to learning functions, one can look for valid feature models that guarantee the interpretability of our ML. 2. Multimodal modeling: feature modeling is extended by multimodal embeddings of triads of social, situational (location, time, etc.), and linguistic units (words, phrases, etc.) for learning two kinds of associations among units of the same or different modality: syntagmatic associations by contiguity and paradigmatic associations by similarity (see Requirement 1). For each modality, one has to additionally consider type formation with respect to social roles (e.g., of editors), semantic classes (e.g., regarding the formation of terminologies), or text types (e.g., discussions vs. forums vs. wiki articles). This approach to multimodal embeddings in social-semiotic networks can be considered as a prerequisite for learning interdependencies between semantic units and behavioral or temporal patterns of interacting agents (e.g., learners, writers). 3. Network modeling: one can further utilize methods of network theory (Newman 2010) to analyze core-periphery structures and community building within multilayered networks (see above). This can be done to explore context-related features beyond the narrow limits of internal features derived directly from task descriptions. In this way, we may consider interaction oriented features (who is co-authoring a text, who is cooperating in completing a task) as well as intertextual features (regarding a range of semantically similar texts being processed by learners). While methods of computational linguistics serve for modeling micro-level units of communication events, multimodal modeling is deployed to model meso-level units (e.g., social roles, text types, topics) and their relations. Finally, network modeling is deployed to model (the covariation of) macro-level units of structure formation in multi-layer social-semiotic networks.

14

TextInContext: On the Way to a Framework for Measuring the Context. . .

191

Conclusion The chapter developed a conceptual framework for assessing the complexity of natural language texts and especially of task descriptions in a context-sensitive manner. To this end, it considered three theories of learning, of text comprehension, and of concept representation in conceptual space. The chapter also aimed to build a bridge between ML-based models of schematic knowledge stored in LTM and affecting the processing of texts on the one hand and extensional semantic notions of context as provided by situation semantics on the other. From a theoretical point of view, the achievements of the chapter are as follows: 1. We made a concrete proposal for a modular architecture that combines a learning theory (i.e., Cognitive Load Theory (CLT), Sweller et al. 2011) with a theory of text comprehension (i.e., Construction Integration Theory (CIM), Kintsch 1998), and a theory of cognitive semantics (i.e., Conceptual Spaces (CS), Gärdenfors 2000) (Requirement 3). 2. We referred to CLT as a model framework for integrating the learning of schemata as units of the long-term, extrasituational context (Requirement 1 and 6), while reference is made to CIM to integrate element interactivity and schema learning according to CLT (Requirement 1). 3. Conceptual spaces were utilized to make different dimensions of complexity sensitive to models of short- and long-term contexts (Requirements 5 and 6). 4. By additionally elaborating the analogy of completing learning tasks and text authoring, we offered a hypothetical solution to the problem of data bottlenecks in NLP-based educational research by addressing big textual data (Requirement 2). To this end, we proposed to employ transfer learning to allow for accessing large text databases (of textbooks and off- and online documents) for implementing our approach. 5. This approach may finally help to pave the way for developing simulation models to better predict the learning outcomes of learners, for example, in terms of negative learning (Requirement 7). Our approach may take an important step towards a more realistic language model for measuring the complexity of learning tasks that goes beyond black box models in NLP, computational linguistics and quantitative linguistics. It may help to bridge between cognitive science, computational linguistics, and data science (using big linguistic data) to avoid pitfalls of computational modeling (e.g., lack of context sensitivity, lack of interpretability as a result of opaque black box models, overfitting, etc.). Thus, our framework may be seen as a blueprint of a multidisciplinary cooperation among educational researchers, (computational) linguists, and data scientists. Much remains to be done: This concerns primarily the implementation and testing of the conceptual model developed here. In this way, we may also find solutions for Requirement 2 (responsiveness to small data) and Requirement 7 (simulation and prediction), which have remained largely unaffected by our modeling so far.

192

A. Mehler and V. Ramesh

According to section “A Systems Engineering View of Multimodal Cognition”, this also concerns multimodal data that has to be analyzed to capture the full range of information that is necessary to tackle the notion of extraneous cognitive load.

References Andersen, P. B. (2002). Dynamic semiotics. Semiotica, 1(4), 161–210. Anderson, R. C., & Davison, A. (1988). Conceptual and empirical bases of readability formulas. In R. C. Anderson & A. Davison (Eds.), Linguistic complexity and text comprehension (pp. 23–54). Hillsdale, MI: Erlbaum. Auer, P. (1992). Introduction: John Gumperz’ approach to contextualization. In P. Auer & A. Di Luzio (Eds.), The contextualization of language (pp. 1–37). Amsterdam: Benjamins. Auer, P. (1996). From context to contextualization. Links & Letters, 3, 11–28. Barwise, J., & Perry, J. (1983). Situations and attitudes. Cambridge: MIT Press. Brin, S., & Page, L. (1998). The anatomy of a large-scale hypertextual web search engine. Computer Networks and ISDN Systems, 30, 107–117. Brusilovsky, P. (2001). Adaptive hypermedia. User Modeling and User-Adapted Interaction, 11(1), 87–110. Campbell, D. J. (1988). Task complexity: A review and analysis. Academy of Management Review, 13(1), 40–52. Dagan, I., Roth, D., Sammons, M., & Zanzotto, F. M. (2013). Recognizing textual entailment: Models and applications. Synthesis lectures on human language technologies, San Rafael, CA: Morgan and Claypool, 6(4): 1–220. Eikmeyer, H.-J. (1985). Prozedurale Semantik. In B. Rieger (Ed.), Dynamik in der Bedeutungskonstitution, Papiere zur Textlinguistik (Vol. Bd. 46, pp. 31–45). Hamburg: Buske. Gärdenfors, P. (2000). Conceptual Spaces. Cambridge: MIT Press. Gärdenfors, P. (2014). The geometry of meaning: Semantics based on conceptual spaces. Cambridge: MIT Press. Goodwin, C., & Duranti, A. (1992). Rethinking context: An introduction. In A. Duranti & C. Goodwin (Eds.), Rethinking context: Language as an interactive phenomenon (pp. 1–42). Cambridge: Cambridge University Press. Greiffenhagen, M., Comaniciu, D., Niemann, H., & Ramesh, V. (2001). Design, analysis, and engineering of video monitoring systems: An approach and a case study. Proceedings of the IEEE, 89(10), 1498–1517. Gumperz, J. J. (1992). Contextualization and understanding. In A. Duranti & C. Goodwin (Eds.), Rethinking context: Language as an interactive phenomenon (pp. 1–42). Cambridge: Cambridge University Press. Hunston, S., & Francis, G. (2000). Pattern grammar. a Corpus-driven approach to the lexical grammar of English. Amsterdam: John Benjamins. Islam, Z., & Mehler, A. (2013). Automatic readability classification of crowd-sourced data based on linguistic and information-theoretic features. In 14th International Conference on Intelligent Text Processing and Computational Linguistics (CICLing 2013). John, M. F. S., & McClelland, J. L. (1992). Parallel constraint satisfaction as a comprehension mechanism. In R. G. Reilly & N. E. Sharkey (Eds.), Connectionist approaches to natural language processing (pp. 97–136). Hove: Erlbaum. Kintsch, W. (1988). The role of knowledge in discourse comprehension: A construction-integration model. Psychological Review, 95(2), 163–182. Kintsch, W. (1998). Comprehension. A paradigm for cognition. Cambridge: Cambridge University Press. Kintsch, W. (2001). Predication. Cognitive Science, 25, 173–202.

14

TextInContext: On the Way to a Framework for Measuring the Context. . .

193

Kintsch, W. (2008). How the mind computes the meaning of metaphor: A simulation based on LSA. In R. W. Gibbs (Ed.), The Cambridge handbook of metaphor and thought (pp. 129–142). Cambridge: Cambridge University Press. Kirschner, F., Paas, F., & Kirschner, P. (2011a). Task complexity as a driver for collaborative learning efficiency: The collective working-memory effect. Applied Cognitive Psychology, 25(4), 615–624. https://doi.org/10.1002/acp.173025 Kirschner, P. A., Ayres, P., & Chandler, P. (2011b). Contemporary cognitive load theory research: The good, the bad and the ugly. Computers in Human Behavior, 27(1), 99–105. Koch, P., & Oesterreicher, W. (1994). Schriftlichkeit und Sprache. In H. Günther & O. Ludwig (Eds.), Schrift und Schriftlichkeit: Ein interdisziplinäres Handbuch internationaler Forschung (Vol. 1, pp. 587–603). Berlin: De Gruyter. Köhler, R. (1987). Systems theoretical linguistics. Theoretical Linguistics, 14(2/3), 241–257. Komninos, A., & Manandhar, S. (2016). Dependency based embeddings for sentence classification tasks. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (pp. 1490–1500). Koutra, D., Parikh, A., Ramdas, A., & Xiang, J. (2011). Algorithms for graph similarity and subgraph matching. Retrieved from https://www.cs.cmu.edu/jingx/docs/DBreport.pdf Koutra, D., Kang, U., Vreeken, J., & Faloutsos, C. (2014, April 24–26). VoG: summarizing and understanding large graphs. In Proceedings of the 2014 SIAM International Conference on Data Mining, Philadelphia, Pennsylvania, USA (pp. 91–99). Lee, K., He, L., Lewis, M., & Zettlemoyer, L. (2017). End-to-end neural coreference resolution. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing (pp. 188–197). Levelt, W. J. M. (1989). Speaking: From intention to articulation. Cambridge: MIT Press. Levy, O., & Goldberg, Y. (2009). Dependency-based word embeddings. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers, pp. 302–308). Leydesdorff, L. (2009). The non-linear dynamics of meaning processing in social systems. Social Science Information, 48(1), 5–33. Ling, W., Dyer, C., Black, A., & Trancoso, I. (2015). Two/too simple adaptations of word2vec for syntax problems. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Litman, D. (2016). Natural language processing for enhancing teaching and learning. In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence (AAAI-16) (pp. 4170–4176). Liu, P., & Li, Z. (2012). Task complexity: A review and conceptualization framework. International Journal of Industrial Ergonomics, 42(6), 553–568. Liu, P., & Li, Z. (2014). Comparison of task complexity measures for emergency operating procedures: Convergent validity and predictive validity. Reliability Engineering & System Safety, 121, 289–293. Marcus, G. (2018). Deep learning: A critical appraisal. Clinical Orthopaedics and Related Research. Retrieved from https://arxiv.org/abs/1801.00631 Mehler, A. (2006, June 26). In search of a bridge between network analysis in computational linguistics and computational biology—a conceptual note. In H. R. Arabnia & H. Valafar (Eds.), Proceedings of the 2006 International Conference on Bioinformatics & Computational Biology (BIOCOMP’06), Las Vegas, USA (pp. 496–500). Mehler, A. (2007). Compositionality in quantitative semantics. A theoretical perspective on text mining. In A. Mehler & R. Köhler (Eds.), Aspects of automatic text analysis, studies in fuzziness and soft computing (pp. 139–167). Berlin: Springer. Mehler, A. (2008). Structural similarities of complex networks: A computational model by example of wiki graphs. Applied Artificial Intelligence, 22(7&8), 619–683. Mehler, A., Gleim, R., Hemati, W., & Uslu, T. (2017). Skalenfreie online soziale Lexika am Beispiel von Wiktionary. In S. Engelberg, H. Lobin, K. Steyer, & S. Wolfer (Eds.), Proceedings

194

A. Mehler and V. Ramesh

of 53rd Annual Conference of the Institut für Deutsche Sprache (IDS), March 14-16, Mannheim, Germany (pp. 269–291). Berlin: De Gruyter. Mehler, A., Hemati, W., Uslu, T., & Lücking, A. (2018a). A multidimensional model of syntactic dependency trees for authorship attribution. In J. Jiang & H. Liu (Eds.), Quantitative analysis of dependency structures (pp. 315–348). Berlin: De Gruyter. Mehler, A., Hemati, W., Gleim, R., & Baumartz, D. (2018b). VienNA: Auf dem Weg zu einer Infrastruktur für die verteilte interaktive evolutionäre Verarbeitung natürlicher Sprache. In H. Lobin, R. Schneider, & A. Witt (Eds.), Forschungsinfrastrukturen und digitale Informationssysteme in der germanistischen Sprachwissenschaft (Vol. 6, pp. 149–176). Berlin: De Gruyter. Mehler, A., Gleim, R., Lücking, A., Uslu, T., & Stegbauer, C. (2018c). On the self-similarity of Wikipedia talks: A combined discourse-analytical and quantitative approach. Glottometrics, 40, 1–45. Mehler, A., Zlatkin-Troitschanskaia, O., Hemati, W., Molerov, D., Lücking, A., & Schmidt, S. (2018d). Integrating computational linguistic analysis of multilingual learning data and educational measurement approaches to explore student learning in higher education. In O. Zlatkin-Troitschanskaia, G. Wittum, & A. Dengel (Eds.), Positive learning in the age of information (PLATO) – A blessing or a curse? (pp. 145–193). Wiesbaden: Springer. Mikk, J. (1995). Methods for determining optimal readability of texts. Journal of Quantitative Linguistics, 2(2), 125–132. Mikolov, T., Yih, W. & Zweig, G. (2013). Linguistic regularities in continuous space word representations. In Proceedings of NAACL 2013 (pp. 746–751). Newman, M. E. J. (2010). Networks: An introduction. Oxford: Oxford University Press. Nisbett, R. E. (2003). The geography of thought: How Asians and Westerners think differently . . . and why. New York: Free Press. Pickering, M. J., & Garrod, S. (2004). Toward a mechanistic psychology of dialogue. Behavioral and Brain Sciences, 27, 169–226. Ramesh, V. (1995). Performance characterization of image understanding algorithms. PhD thesis, Department of Electrical Engineering, University of Washington, Seattle. Rieger, B. (1985). Semantische Dispositionen: Prozedurale Wissensstrukturen mit stereotypisch repräsentierten Wortbedeutungen. In B. Rieger (Ed.), Dynamik in der Bedeutungskonstitution, Papiere zur Textlinguistik (Vol. Bd. 46, pp. 163–228). Hamburg: Buske. Rieger, B. (2001). Computing granular word meanings: A fuzzy linguistic approach in computational semiotics. In P. Wang (Ed.), Computing with words (pp. 147–208). New York: Wiley. Sag, I. A., Baldwin, T., Bond, F., Copestake, A., & Flickinger, D. (2002). Multiword expressions: A pain in the neck for NLP. In Proceedings of the 3rd. International Conference on Intelligent Text Processing and Computational Linguistics (CICLing-2002, pp. 1–15). Schnotz, W. (1994). Aufbau von Wissensstrukturen: Untersuchungen zur Kohärenzbildung beim Wissenserwerb mit Texten. Beltz, Weinheim. Silver, D. L., & Bennett, K. P. (2008). Guest editor’s introduction: Special issue on inductive transfer learning. Machine Learning, 73(3), 215. Stouten, H., & Größler, A. (2017). Task complexity in individual stock control tasks for laboratory experiments on human understanding of dynamic systems. Systems Research and Behavioral Science, 34(1), 62–77. Sweller, J. (1994). Cognitive load theory, learning difficulty, and instructional design. Learning and Instruction, 4(4), 295–312. Sweller, J. (2003). Evolution of human cognitive architecture. Psychology of Learning and Motivation, 43, 215. Sweller, J., Ayres, P., & Kalyuga, S. (2011). Cognitive load theory (Explorations in the learning sciences, instructional systems and performance technologies). New York: Springer. Tuldava, J. (1993). The statistical structure of a text and its readability. In L. Hřebíček & G. Altmann (Eds.), Quantitative text analysis (pp. 215–227). Trier: Wissenschaftlicher Verlag.

14

TextInContext: On the Way to a Framework for Measuring the Context. . .

195

Uslu, T., Mehler, A., Niekler, A., & Baumartz, D. (2018). Towards a DDC-based topic network model of Wikipedia. In Proceedings of 2nd International Workshop on Modeling, Analysis, and Management of Social Networks and their Applications (SOCNET 2018). Van Dijk, T. A., & Kintsch, W. (1983). Strategies of discourse comprehension. New York: Academic Press. Van Trijp, R., Steels, L., Beuls, K., & Wellens, P. (2012). Fluid construction grammar: The new kid on the block. In Proceedings of the Demonstrations at the 13th Conference of the European Chapter of the Association for Computational Linguistics (pp. 63–68). Veeravasarapu, VSR, Rothkopf, C., & Ramesh, V. (2017a). Model-driven simulations for computer vision. In Applications of Computer Vision (WACV), 2017 IEEE Winter Conference (pp. 1063–1071). Veeravasarapu, VSR, Rothkopf, C., & Ramesh, V. (2017b). Adversarially tuned scene generation. In IEEE CVPR (pp. 6441–6449). Weis, T., Mundt, M., Harding, P., & Ramesh, V. (2017). Anomaly detection for automotive visual signal transition estimation. In Intelligent Transportation Systems (ITSC), 2017 IEEE 20th International Conference (pp. 1–8). Wharton, C., & Kintsch, W. (1991). An overview of construction-integration model: A theory of comprehension as a foundation for a new cognitive architecture. ACM SIGART Bulletin, 2(4), 169–173. Yablo, S. (2014). Aboutness. Princeton: Princeton University Press. Zadeh, L. A. (1997). Toward a theory of fuzzy information granulation and its centrality in human reasoning and fuzzy logic. Fuzzy Sets and Systems, 90, 111–127. Zhang, Q., & Zhu, S.-C. (2018). Visual interpretability for deep learning: A survey. Clinical Orthopaedics and Related Research. Retrieved from https://arxiv.org/abs/1802.00614 Zipf, G. K. (1972). Human behavior and the principle of least effort. an introduction to human ecology. New York: Hafner. Zlatkin-Troitschanskaia, O., Förster, M., Brückner, S., & Happ, R. (2014). Insights from a German assessment of business and economics competence. In H. Coates (Ed.), Higher education learning outcomes assessment - International perspectives (pp. 175–197). Frankfurt am Main: Peter Lang. Zlatkin-Troitschanskaia, O., Schmidt, S., Molerov, D., Shavelson, R. J., & Berliner, D. (2018). Conceptual fundamentals for a theoretical and empirical framework of positive learning. In O. Zlatkin-Troitschanskaia, G. Wittum, & A. Dengel (Eds.), Positive learning in the age of information – A blessing or a curse? (pp. 29–50). Wiesbaden: Springer.

Chapter 15

From Cognitive Structures to Positive and Negative Learning in a Dialogue Semantics Perspective Andy Lücking

The Need for Semantic Structures Natural language sentences can be ambiguous because of lexical or structural (syntactic) ambiguity (subsequent examples are taken from Chierchia and McConnell-Ginet 2000). For instance, the sentence in (15.1) is ambiguous because the form ‘bull’ is ambiguous between male cow, papal communication, or nonsense (lexical ambiguity). You should have seen the bull we got from the pope:

ð15:1Þ

Structural or syntactic ambiguity is exemplified in (15.2) Competent women and men hold all the good jobs in the firm:

ð15:2Þ

The sentence in (15.2) is ambiguous because of different ranges of the adjective “competent”: it can modify only “women”, or it can modify “women and men”. Lexical ambiguity is accounted for by lexical entries that pair one form with several meanings; structural ambiguity is accounted for by assigning several syntactic derivation trees to one input sentence. There is, however, another source of ambiguity that cannot be analysed in terms of the lexicon or syntactic parses, namely so-called scope ambiguities. An example is given in (15.3) (taken from Dwivedi 2013):

A. Lücking (*) Faculty of Computer Science and Mathematics, Goethe University Frankfurt am Main, Frankfurt am Main, Germany Université Paris Diderot (Paris 7), Laboratoire de Linguistique Formelle (LLF), Frankfurt am Main, Germany e-mail: [email protected] © Springer Nature Switzerland AG 2019 O. Zlatkin-Troitschanskaia (ed.), Frontiers and Advances in Positive Learning in the Age of InformaTiOn (PLATO), https://doi.org/10.1007/978-3-030-26578-6_15

197

198

A. Lücking

Every schoolgirl crossed a road: a: The roads were flat and paved:

ð15:3Þ

b: The road was flat and paved: The indefinite noun phrase a road of the initial sentence in (15.3) can act as an antecedent expression of both a plural (15.3a) and a singular (15.3b) continuation. The continuations explicate two readings associated with the initial sentence, namely that there is one road that was crossed by all schoolgirls, or that the roads co-vary with the schoolgirls (so that there are at most as many roads as schoolgirls). This ambiguity cannot be ascribed to the lexicon since the words in the sentence in question do not have multiple readings. This ambiguity cannot either be ascribed to syntax since the sentence in question is syntactically unambiguous. Therefore, the apparent ambiguity has to be addressed on some other level, namely in terms of semantic representations. Traditionally, scope ambiguities are represented by means of logical forms, where the scope taking expressions may differ with respect to their relative order. This is illustrated in (15.4) in terms of Minimal Recursion Semantics (Copestake et al. 2005). The underspecified scope relations in (15.4a) can be resolved in two ways, corresponding to the “plural”, narrow scope reading of a road in (15.4b) and to the singular, wide scope reading in (15.4c). every(y) schoolgirl(y)

a(x) ...

road(x)

cross(y,x)

...

ð15:4Þ

Some kind of semantic representation is required to analyse examples like (15.3). In fact, an essential part of the semantic business is to derive semantic structures (Kamp 1979). Now, it is tempting to equate or at least relate semantic and cognitive structures. However, some care has to be taken to prevent formal grammar and psycholinguistics from diverging to an irreconcilably degree (Ferreira 2005).

The Cognitive Role of Semantic Structures One psycholinguistic problem with the scope analysis just sketched is that it is difficult to reconcile with findings on incremental processing. Language comprehension proceeds word by word (or even morpheme by morpheme), where each new expression is integrated on a syntactic or a thematic pathway (Schlesewsky and Bornkessel 2004). Incremental processing entails that the words are interpreted

15

From Cognitive Structures to Positive and Negative Learning. . .

199

in situ, that is, at their surface position in the input stream, which is in some conflict with the inverse scope analysis given in (15.4c), where the sentence final argument is raised to the initial position within logical form. Accordingly, modern approaches to formal grammar aim at representations that are compatible with incremental parsing models. Substantial contributions on this topic come from dialogue processing (Rieser and Schlangen 2011), but also syntactic, sentence-based processing approaches draw on psycholinguistic evidence (Demberg et al. 2013). In this general context, it has also been shown that sentences involving quantified noun phrases (QNPs) such as our example in (15.3) are interpreted incrementally (Urbach and Kutas 2010). The results are “inconsistent with any hypothesis according to which the processing of quantifier semantics is entirely deferred or delayed” (op. cit., p. 19). However, a delayed interpretation is exactly what seems to be entailed by inverse scope analyses such as (15.4c): the interpretation of the subject QNP (every schoolgirl) is delayed until the object QNP (a road) is processed. These issues have been incorporated in recent theories of QNP semantics (Lücking and Ginzburg 2018), formulated within Type Theory with Records (TTR; Cooper 2012; Cooper and Ginzburg 2015). On this account, a QNP denotes a set triplet consisting of a maximal set (maxset), a reference set (refset) and a complement set (compset), where the refset provides the witness set and the compset the set {maxset\refset}. What has been ascribed to scope ranges above is analysed here as dependent functions (see also Jacobson 2000; Zeevat 2018), which allow for an in situ interpretation. For instance, the “plural”, narrow scope interpretation of a road is brought about by a function that depends on some individual labelled “x”, as illustrated in (15.5): 2

phon : 6 6 6 6 6 6 6 q‐params : 6 6 6 6 6 6 6 6 6 6 6 6 6 6 cont ¼ 6 6 6 4

3 Listðevery schoolgirl crossed a roadÞ 3 2 7 refset s : SetðIndÞ 7 7 7 6 dist 7 7 6 7





! 7 6 7 7 6c s : schoolgirl ðrefset sÞ 7 7 6 " " " ### 7 7 6 7 refind : Ind 7 6 7 5 4f ¼ λr : ½x : Ind  r  x : q‐params : 7 7 c : roadðrefindÞ 7 7 3 2 sit ¼ s1 : Rec 7 7 3 2 7 6 7 q‐cond s : refset s ¼ maxset s j j j j 7 6 7 7 6 7 7 6 7 6 7 7 6 dist 7 : Prop 7 6 7 6 1 , 2

! ðrefset s, f ðrefset sÞ:q‐params:refindÞ 7 6 sit‐type ¼ 6 nucl 7 7 : RecType : cross 7 6 7 7 6 7 6 7 7 6 5 4 5 5 4 dist 1, 2

! anti‐nucl : Øcross ðcompset s, f ðcompset sÞ:q‐params:refindÞ

ð15:5Þ Arrow types indicate a plural type, arrow types with a “dist” superscript indicate a distributive interpretation, which may be restricted to certain argument positions by a subscript notation (e.g. “1,2” for subject and object). On this account, the “singular”, wide scope interpretation corresponds to a relation between the witness sets, respectively, witness individuals, refset and refind, involved (the notion of a witness set is taken from Barwise and Cooper 1981, though it is used here in a different way to indicate that a witness is of type set of individuals):

200 2

phon

6 6 6 6 6 6 6 q‐params 6 6 6 6 6 6 6 6 6 6 6 6 6 6 cont 6 6 6 4

A. Lücking :

:

Listðevery schoolgirl crossed a roadÞ 3 2 refset s : SetðIndÞ 7 6 dist 7 6





! 6c s : schoolgirl ðrefset sÞ 7 7 6 7 6 7 6 refind o : Ind 5 4 2

¼

c o : sit ¼ s1 :

6 6 6 6 6 6 sit‐type 6 6 4

roadðrefindÞ Rec 2 q‐cond s : 6 6 6 ¼ 6 : 6 nucl 6 4 anti‐nucl

:

jrefset sj ¼ jmaxset sj dist



!1 ðrefset s, refindÞ cross dist



!1 ðcompset s, refindÞ Øcross

3 7 7 7 7 7 7 7 7 7 7 7 7 7 3 7 7 7 3 7 7 7 7 7 7 7 7 7 7 : Prop 7 7 7 7 : RecType 7 7 7 7 7 7 7 7 5 5 5

ð15:6Þ Given the integrated psycholinguistic, philosophical and formal modelling, semantic structures like those given in (15.5) and (15.6) can be claimed to cover cognitive aspects in the sense of Ferreira (2005). As such, they provide representations that capture part of the situation models built during semantic sentence processing (related to mental models, see Johnson-Laird 1989) under normal conditions (on the notion of normal conditions, see Millikan 1984).

Situation Modelling The cognitive construable semantic structures motivated in the preceding section are couched in a situation-semantic framework (Barwise and Perry 1983) and embed Austinian propositions (Austin 1950). The complete semantic structure can be witnessed by a concrete situation, labelled “s1” in (15.6) and (15.5), which is of the type specified as “sit-type”. The structures used to represent the semantics of natural languages can also be used to model (the informativity supported by) situations (Barwise and Seligman 1997). The tool offered by TTR for representing situations are the eponymous records. A record which provides a possible witness for the sit-type called for by the abovegiven example (15.5) is given in (15.7), omitting proof objects for the “schoolgirl events” within the record for simplicities sake (see Cooper 2012 on proof objects):

15

From Cognitive Structures to Positive and Negative Learning. . .

3 a : Ind 6 sa : schoolgirlðaÞ 7 7 6 7 6 7 6 b : Ind 7 6 6 sb : schoolgirlðbÞ 7 7 6 7 6 7 6 c : Ind 7 6 3 6 2 7 a ¼ an 6 sc : schoolgirlðcÞ 7 7 6 b ¼ ka 7 6 7 d : Ind 7 6 6 7 7 6 6 6 6 c ¼ nu 7 6 sd : schoolgirlðdÞ 7 7 7 6 s1 ¼ 6 7 7 : 6 x : Ind 7 6 d ¼ fr 7 6 7 6 7 6 7 4 x ¼ rd1 5 6 7 6 rx : roadðxÞ 7 6 7 6 y : Ind y ¼ rd2 7 6 7 6 ry : roadðyÞ 7 6 7 6 6 c1 : crossða, xÞ 7 7 6 6 c2 : crossðb, xÞ 7 7 6 7 6 4 c3 : crossðc, xÞ 5

201

2

ð15:7Þ

c4 : crossðd, yÞ Situation s1 hosts six individuals, labelled “a”, “b”, “c”, “d” and “x” and “y”, where the former four are schoolgirls and the latter two are roads. The schoolgirls labelled “a”, “b” and “c” cross the road labelled “x”, the schoolgirl labelled “d” crosses the road labelled “y”, and there are no further schoolgirls. Every schoolgirl crossed a road, though not in every case the same one. Accordingly, the situation can be classified with the linguistic content from (15.5), which captures the desired meaning and therefore correctly classifies the example situation from (15.7): 2

2 sit ¼ s1 : 6 6 6 6 6 6 6 6 6cont ¼ 6 6 6 sit‐type ¼ 6 6 4 4

3 3 Rec 3 2 7 7 q‐cond s : jrefset sj ¼ jmaxset sj 7 7 7 6 7 7 dist 7 6 7 7 1 , 2 : Prop 7 6 7 7

! : cross ðrefset s, f ðrefset sÞ:q‐params:refindÞ 7 : RecType 7 6 nucl 7 7 6 7 7 5 4 5 5 dist 1, 2

! anti‐nucl : Øcross ðcompset s, f ðcompset sÞ:q‐params:refindÞ

ð15:8Þ The semantic representation in (15.8) embeds a “negative predication”, the antinucl. Here the negated verb is predicated of the compset, which is empty in case of every, however. This would be different when another quantifier is substituted, say most. In this case, there would be few schoolgirls left that did not cross a road. Now a negated type is any type that is incompatible with the underlying positive type (Cooper and Ginzburg 2011). In this sense, semantics resting on a Type Theory with Records provides a content-related notion of negation. Both negation and correct classification (true judgment) play a different role in learning, which is briefly clarified subsequently.

202

A. Lücking

Positive and Negative Learning Learning usually takes place in a dialogical set-up. Dialogue and its progressing in terms of conversational moves can be modelled with KoS, the dialog framework developed by Ginzburg (2012). Dialogue unfolds on a Dialogue Game Board (DGB). A DGB is an information state-based sheet for describing communicative interactions which is updated dynamically. The DGB from KoS tracks the interlocutors (spkr and addr fields), a record of the dialogue history (Moves), dialogue moves that are in the process of grounding (Pending), the question(s) currently under discussion (QUD) and the assumptions shared among the interlocutors (Facts). The TTR representation of a DGB following Ginzburg (2012) is given in (15.9), where LocProp is the type of a locutionary proposition (token-type couples from utterance events) and poset abbreviates “partially ordered set”. 2

3

spkr : Ind

7 6 addr : Ind 7 6 7 6 7 6 utt‐time : Time 7 6 6 c‐utt : addressingðspkr, addr, utt‐timeÞ 7 7 6 7 6 7 6 Facts : SetðPropÞ 7 6 7 6 Pending : listðLocPropÞ 7 6 7 6 5 4 Moves : listðLocPropÞ

ð15:9Þ

QUD : posetðQuestionÞ Two qualifications are required. The first is that the Facts field labels a public common ground, which is an apt simplification for many investigations. In case of learning, however, it seems to be more useful to employ a more sophisticated common ground story, which distinguishes a private and a shared part from each interlocutor’s perspective (Poesio and Rieser 2010). This is displayed in (15.10), following Ginzburg (2012): "

"

"

spkr : facts : "

" addr : facts :

"

shared : SetðPropÞ private : SetðPropÞ shared : SetðPropÞ

### ###

ð15:10Þ

private : SetðPropÞ

Based on this more fine-grained Facts structure, a basic dialogical notion of learning can already be developed: Learning corresponds to a proposition which initially is only part of spkr.facts.private that becomes part of spkr.facts.shared and addr.facts.shared. The second qualification is required with respect to speaker and addressee: being a dialogue theory both are assumed to be human beings that engage in natural

15

From Cognitive Structures to Positive and Negative Learning. . .

203

language interaction. However, nothing prevents one from thinking, for instance, as the speaker as a book or a web page, from which the addressee gains knowledge. Respective follow-up modification may in this case be necessary. Now given the distinction in positive and negative types as exemplified by the nucl and anti-nucl conditions (see (15.5) and others), positive and negative propositions can be grounded, that is, added to facts.shared.1 In both cases, learning according to the above-given working notion has taken place. The first learning instance is one of learning from the positive, where a state of affairs is claimed to be of a positive, non-negated type. The addressee learns that a certain state-of-affairs gives rise to classifying information which was not aware to him beforehand. The second instance is learning from the negative, where a state of affairs is characterized as being incompatible with the type offered. The addressee learns by counterexample or exclusion. Thus, the semantic-cognitive program pursued here eventually allows to capture an important distinction in pathways of learning. In shortcut notation, the difference in learning content is this: s: T versus s: ØT. Both are inhabitants on equal footing in TTR semantics. They are called learning from the positive and learning from the negative, respectively (see also Oser and Spychiger 2005). Furthermore, going beyond semantics proper, we might question the status of s, the situation learned about. s may be part of the actual world, in which case we learn something substantial, what PLATO calls positive learning (for a definition, see Zlatkin-Troitschanskaia et al. 2018). s may also be part of an alternative world, in which case the gained knowledge remains fictional to some degree.2 While learning from the positive and learning from the negative are semantic notions, positive and negative learning apply to the level of ontology or truth. To this end, in philosophy the difference between the actual and the alternative is attributed to the body of knowledge provided by the best of our scientific theories (e.g. Beckermann 1999). Accordingly, scientific consensus has to be one if not the decisive criterion of PLATO to address positive and negative learning.

Outlook: Pointers to Multilingual Settings Given the semantic-cognitive twist of the preceding sections, some further learning issues in particular with respect to multilingual learning shall be noted. The first is the renewed interest in iconicity in languages (Dingemanse et al. 2015). Following this direction, languages can map state-of-affairs with a greater or lesser degree of 1

The counterpart of grounding is posing a clarification request, which is skipped here (see Ginzburg 2012 for a detailed discussion). 2 Putting it this way, we switched to a possibilistic, Kratzer-style situation model (Kratzer 1989). We do not see any severe obstacles that stand in the way of a situation abstraction rendering more in line with the cognitive construal of TTR, which would, however, complicate matters for present purposes.

204

A. Lücking

structure preservation between event representation and grammatical encoding. A rather detailed account in this respect is the linking competence, which rests on a figure-ground distinction as well as on agency (Kasper 2013). Both aspects have already been testified experimentally: there is some evidence that when speakers’ visual attention is directed to the patient (in contrast to the agent), they are more likely to produce a passive voice sentence (Gleitman et al. 2007). If the grammatical encoding is isomorphic to the visual encoding of a scene, parsing and understanding proceed faster on average. There may be also language inherent grammatical means that lead to a better adaptation of speakers to various tasks, which are not due to epistemic differences across languages, but to speakers being more trained due to grammatical habits of their language (Bisang 2015; Clark 1996). How incremental processing is realized in particular in head-final languages is a renowned field of study (e.g. Yoo 2007). Remarkably, speakers are capable of incremental processing even if the main predicate comes at the end of the input stream. Semantic models still have to deal with this phenomenon, but prediction models seem to provide promising results in this respect (e.g. Kutas et al. 2011). Finally, interactions with different kinds of learning still have to be explored.

References Austin, J. L. (1950). Truth. In Proceedings of the Aristotelian Society. Supplementary. Vol. xxiv. (Reprinted in Austin, J. L. (1970). Philosophical papers (2nd ed., pp. 111–128). Oxford: Clarendon Press. Barwise, J., & Cooper, R. (1981). Generalized quantifiers and natural language. Linguistics and Philosophy, 4(2), 159–219. https://doi.org/10.1007/BF00350139 Barwise, J., & Perry, J. (1983). Situations and attitudes (The David Hume Series of philosophy and cognitive science reissues). Stanford: CSLI Publications. Barwise, J., & Seligman, J. (1997). Information flow: The logic of distributed systems (Cambridge tracts in theoretical computer science) (Vol. 44). Cambridge: Cambridge University Press. Beckermann, A. (1999). Analytische Einführung in die Philosophie des Geistes. Berlin: de Gruyter. Bisang, W. (2015). Hidden complexity – The neglected side of complexity and its implications. Linguistics Vanguard, 1(1), 177–187. https://doi.org/10.1515/lingvan-2014-1014 Chierchia, G., & McConnell-Ginet, S. (2000). Meaning and grammar – An introduction to semantics (2nd ed.). Cambridge: MIT Press. Clark, H. H. (1996). Communities, commonalities, and communication. In J. J. Gumperz & S. C. Levinson (Eds.), Rethinking linguistic relativity (pp. 324–355). Cambridge: Cambridge University Press. Cooper, R. (2012). Type theory and semantics in flux. In R. Kempson, T. Fernando, & N. Asher (Eds.), Philosophy of linguistics (Handbook of philosophy of science) (Vol. 14, pp. 271–323). Oxford: Elsevier. Cooper, R., & Ginzburg, J. (2011). Negation in dialogue. In: Proceedings of the 15th Workshop on the Semantics and Pragmatics of Dialogue (pp. 130–139). SemDial 2011, Los Angeles. Cooper, R., & Ginzburg, J. (2015). Type theory with records for natural language semantics. In S. Lappin & C. Fox (Eds.), The handbook of contemporary semantic theory (2nd ed., pp. 375–407). Oxford: Wiley-Blackwell.

15

From Cognitive Structures to Positive and Negative Learning. . .

205

Copestake, A., Flickinger, D., Pollard, C., & Sag, I. A. (2005). Minimal recursion semantics: An introduction. Research on Language and Computation, 3(2-3), 281–332. Demberg, V., Keller, F., & Koller, A. (2013). Incremental, predictive parsing with psycholinguistically motivated tree-adjoining grammar. Computational Linguistics, 39(4), 1025–1066. Dingemanse, M., Blasi, D. E., Lupyan, G., Christiansen, M. H., & Monaghan, P. (2015). Arbitrariness, iconicity and systematicity in language. Trends in Cognitive Sciences, 19(10), 603–615. https://doi.org/10.1016/j.tics.2015.07.013 Dwivedi, V. D. (2013). Interpreting quantifier scope ambiguity: Evidence of heuristic first, algorithmic second processing. PLoS One, 8(11), e81461. https://doi.org/10.1371/journal.pone. 0081461 Ferreira, F. (2005). Psycholinguistics, formal grammars, and cognitive science. The Linguistic Review, 22(2-4), 365–380. https://doi.org/10.1515/tlir.2005.22.2-4.365 Ginzburg, J. (2012). The interactive stance: Meaning for conversation. Oxford: Oxford University Press. Gleitman, L. R., January, D., Nappa, R., & Trueswell, J. C. (2007). On the give and take between event apprehension and utterance formulation. Journal of Memory and Language, 57(4), 544–569. https://doi.org/10.1016/j.jml.2007.01.007 Jacobson, P. (2000). Paycheck pronouns, Bach-Peters sentences, and variable-free semantics. Natural Language Semantics, 8(2), 77–155. Johnson-Laird, P. N. (1989). Mental models. In M. I. Posner (Ed.), Foundations of cognitive science (pp. 469–499). Cambridge: MIT Press. Kamp, H. (1979). Events, instants and temporal reference. In R. Bäuerle, U. Egli, & A. von Stechow (Eds.), Semantics from different points of view (Springer series in language and communication) (Vol. 6, pp. 376–417). Berlin: Springer. Kasper, S. (2013). The culture and nature of the linking competence. How action and perception shape the syntax-semantics relationship. PhD thesis, Philipps-Universität Marburg. Kratzer, A. (1989). An investigation of the lumps of thought. Linguistics and Philosophy, 12(5), 607–653. Kutas, M., DeLong, K. A., & Smith, N. J. (2011). A look around at what lies ahead: Prediction and predictability in language processing. In M. Bar (Ed.), Predictions in the brain: Using our past to generate a future (pp. 190–207). Oxford: Oxford University Press. Lücking, A., & Ginzburg, J. (2018). ‘Most people but not Bill’: Integrating sets, individuals and negation into a cognitively plausible account of noun phrase interpretation. In Proceedings of Cognitive Structures: Linguistic, Philosophical and Psychological Perspectives. CoSt’18. Millikan, R. G. (1984). Language, thought and other biological categories. Cambridge: MIT Press. Oser, F., & Spychiger, M. (2005). Lernen ist schmerzhaft: Zur Theorie des Negativen Wissens und zur Praxis der Fehlerkultur. Weinheim: Beltz. Poesio, M., & Rieser, H. (2010). Completions, coordination, and alignment in dialogue. Dialogue and Discourse, 1(1), 1–89. https://doi.org/10.5087/dad.2010.001 Rieser, H., & Schlangen, D. (2011). Introduction to the special issue on incremental processing in dialogue. Dialogue and Discourse, 2(1), 1–10. https://doi.org/10.5087/dad.2010.001 Schlesewsky, M., & Bornkessel, I. (2004). On incremental interpretation: Degrees of meaning accessed during sentence comprehension. Lingua, 114(9), 1213–1234. https://doi.org/10.1016/ j.lingua.2003.07.006 Urbach, T. P., & Kutas, M. (2010). Quantifiers more or less quantify online: ERP evidence for partial incremental interpretation. Journal of Memory and Language, 63(2), 158–179. Yoo, D.-G. (2007). Syntax und Kontext: Satzverarbeitung in kopffinalen Sprachen. PhD thesis, Universität Bielefeld. Zeevat, H. (2018). Interpreting dependent NPs. In Proceedings of Cognitive Structures: Linguistic, Philosophical and Psychological Perspectives. CoSt’18. Zlatkin-Troitschanskaia, O., Schmidt, S., Molerov, D., Shavelson, R., & Berliner, D. (2018). Conceptual fundamentals for a theoretical and empirical framework of positive learning. In O. Zlatkin-Troitschanskaia, G. Wittum, & A. Dengel (Eds.), Positive learning in the age of information – A blessing or a curse? (pp. 29–50). Wiesbaden: Springer VS.

Chapter 16

Performance in Knowledge Assessment Tests from the Perspective of Linguistic Typology Walter Bisang and Patryk Czerwinski

Setting the Stage: Cross-Linguistic Variation and Cognitive Skills Language is an important means of communication and a powerful tool for conceptualising, categorising and interpreting the world. From a global perspective, there are some 7000 different languages1 which serve exactly these purposes. As everybody can experience easily when travelling around the world, languages differ, and communication across language borders can be pretty hard if not impossible without the help of an interpreter. In this context, linguistic typology aims at modelling cross-linguistic variation as it manifests itself in the grammar of the world’s languages. It investigates the question of whether and to what extent this variation follows regular patterns (Comrie 1981; Croft 2003; Song 2011; Velupillai 2012). An important pattern of regularity which was found by Greenberg (1963) are implicational universals, i.e., correlations between two (or more) grammatical properties of the type ‘If a language has property P, it also has property Q’ (see also Bisang 2018). A precondition for solid typological findings is the detailed analysis of the grammatical properties of individual languages with their structures and categories—a fact that has been highlighted in the programmatic paper by Evans and Levinson (2009). The present paper focuses on that second and more basic aspect of linguistic typology. As has become quite clear in the course of typological research,

1

See www.ethnologue.com, last access August 22, 2018.

W. Bisang (*) · P. Czerwinski Department of English and Linguistics, Johannes Gutenberg University Mainz, Mainz, Germany e-mail: [email protected]; [email protected] © Springer Nature Switzerland AG 2019 O. Zlatkin-Troitschanskaia (ed.), Frontiers and Advances in Positive Learning in the Age of InformaTiOn (PLATO), https://doi.org/10.1007/978-3-030-26578-6_16

207

208

W. Bisang and P. Czerwinski

grammars vary considerably more than speakers of Indo-European languages like English or German may think (for a good survey, see Velupillai 2012). Linguistics in general has developed various methods and terminological tools for detecting and describing fine-grained differences in how the grammars of individual languages work. The present paper is based on the following well-known statement of Roman Jakobson (1896–1982): ‘Languages differ essentially in what they must convey and not in what they may convey’ (Jakobson 1992 [1952], p. 149). The question of what categories must be obligatorily expressed in a language as it is asked by Jakobson can be nicely illustrated by an example from Edward Sapir (1884–1939), who starts out from the following seemingly simple and straightforward English sentence (for a more elaborate discussion of obligatoriness, see Lehmann 1995, p. 124; Bisang 2009, 2014, 2015): 1. Sapir (1921, p. 82): The farmer kills the duckling. As Sapir (1921, p. 90) points out, there are several grammatical categories in (1) which are ‘absolutely indispensable’. Thus, the grammar of English forces its speakers to express tense, i.e., speakers have to select a particular tense value from a set of grammatical forms or markers indicating tense values like present, past, etc. In (1), the verb kill is marked for present. A normal English main clause is ungrammatical if there is no tense marking. Similarly, count nouns like farmer or duckling must be marked for number (singular, plural) and reference (definiteness, indefiniteness, specificity). While this may sound straightforward to a speaker of English, the obligatoriness of these categories is by no means universal. Languages vary considerably with regard to the categories which are compulsory and the extent to which a given grammatical category is obligatory. To give an idea of how this works, we look at a possible translation of (1) into Chinese: (2)

Chinese: 农民杀死了小鸭子。 nóngmín shāsǐ-le farmer kill-PFV ‘The farmer kills the duckling’.

xiǎo yāzi. small duck

As this translation shows, there is no obligatory marking of plural or (in)definiteness on the noun. In fact, the nouns nóngmín ‘farmer’ and yāzi ‘duck’2 are not marked for any grammatical category. The question of whether they are singular/ plural or definite/indefinite must be inferred from context, the grammar does not provide that information. The only category we find in (2) is the category of aspect, expressed by the perfective marker -le (but even that information is not strictly

2 The diminutive derivational morphology in duckling in the English version can only be expressed lexically in Chinese with xiǎo ‘small’.

16

Performance in Knowledge Assessment Tests

209

obligatory but necessary in a short sentence out of context, which is assumed to report a concrete situation). In addition to the question of obligatoriness of grammatical categories, languages also vary with regard to the fine-grainedness of the distinctions made within a given grammatical category. Thus, the English system of number is limited to singular and plural. In other languages, we find additional values like dual (‘two items’), trial (‘three items’), quadral (‘four items’) and paucal (‘a few items’), which add to the fine-grainedness of the grammatical system of a language (for more details, see Corbett 2000). The cases discussed so far are just a few simple examples of the overall observation that the inventory of grammatical categories and the constructions we find cross-linguistically is not limited to what we find in English or German. If language is a powerful tool for conceptualising, categorising and interpreting the world, as pointed out above, one may ask to what extent cross-linguistic differences in terms of obligatoriness and fine-grainedness may affect human cognitive skills. With this question, we do not want to take up the debate of how language determines our worldview or our thinking (see the statement in Bisang 2018 on the Sapir–Whorf hypothesis). We are rather interested in the much more practical question of whether the grammatical categories which are relevant in a language with their properties of obligatoriness and their semantic fine-grainedness affect performance in cognitive tasks like problem solving. In this context, international higher-education competence tests conducted in different languages provide an excellent testing ground for checking if cross-linguistic grammatical variation enhances or inhibits cognitive skills needed for specific ways of reasoning for successful performance. For that purpose, we analysed data from the US Test of Understanding in College Economics (TUCE) (for further information, see Walstad et al. 2007; ZlatkinTroitschanskaia 2014; Brückner et al. 2015a). This test is of particular interest for the purpose of this paper because it shows language/culture-specific differences in performance (Brückner et al. 2015b; Zlatkin-Troitschanskaia et al. 2016a, b). The test consists of 60 test items (multiple-choice questions) and is available in English, Japanese, Korean and German. The data analysed in this paper are from English, Japanese and Korean. Japanese and Korean are structurally similar with regard to various properties and structures which significantly deviate from English. While English has the basic word order of subject-verb-object (SVO), Japanese and Korean have subject-object-verb (SOV). Relative clauses precede the noun in Japanese and Korean, while they follow it in English. Japanese and Korean have a rich inventory of case markers, while English has almost none (except for personal pronouns). Even though differences like these are relevant for typological research, the research task of this paper needs to focus on the more specific differences having to do with the existence/absence of certain categories and their fine-grainedness. For that reason, the following typologically well-known domains of grammar were selected: • Information structure and topicality • Converbs and clause combining • Modality and evidentiality.

210

W. Bisang and P. Czerwinski

The detailed functions of these domains will be discussed further below. For the purpose of this introduction it is important to see if and how they are realised in the three languages under scrutiny. The category of topicality exists in all three languages but there is a significant difference. Japanese and Korean are topic-prominent languages (Li and Thompson 1976), English is not—it is a subject-prominent language. Converbs are prominent and frequently used for producing complex clauses by combining individual clauses in Japanese and Korean, while their functionally equivalent ing-participles (gerunds) are much less frequent in English. Moreover, both East Asian languages have large inventories of converbs. Japanese has 13 converbs according to Alpatov and Podlesskaya (1995), Korean ultimately has some 30. Finally, the category of evidentiality is only very marginally developed in English. In contrast, Japanese and Korean have rich sets of markers which are specialised in expressing that function. Thus, each of the above three domains are clearly less developed in English than in Japanese and Korean. In the case of converbs and evidentials, both East Asian languages have highly developed inventories of grammatical distinctions with Korean clearly showing a higher degree of fine-grainedness than Japanese. Moreover, the meanings expressed by these structures are optimally suited for assessing the semantic relations between different propositions and for evaluating their truth status and the source of information on which they are based. In that sense, the domains selected here are also relevant for scientific reasoning in general. The 60 test items in the three languages of English, Japanese and Korean are clearly not sufficient for proving that the presence/absence of grammatical categories and their fine-grainedness enhance or inhibit cognitive skills for problem solving. In spite of this, we argue that our paper can show two things: 1. The individual test items in their English, Japanese and Korean versions do not state the same thing. In that sense, language clearly matters in higher-education competence tests even beyond the trivial fact that translators often add culturespecific explanations. The Japanese and the Korean versions make ample use of the grammatical inventories available in these languages for helping students understand the problem in context. It will be shown in as much detail as is possible in a short paper how the relevant categories function in Japanese and Korean. 2. Even if it is not possible to clearly prove effects of grammar on cognitive skills in problem solving with the TUCE data, we will provide evidence showing that this is a highly promising research question. For showing this, the paper describes each of the three domains in an individual section: Section “Information Structure: Japanese and Korean as Topic-prominent Languages” on information structure and topicality, Section “Clause Combining and Converbs” on converbs and clause combining, and Section “Evidentials and Modality” on modality and evidentiality. Section “Potential Effects on Performance” presents some preliminary indicators pointing to the relevance of the three domains for the students’ performance in the TUCE.

16

Performance in Knowledge Assessment Tests

211

Information Structure: Japanese and Korean as Topic-Prominent Languages Successful communication crucially depends on the adequate encoding of the information that is available to the speaker and the hearer. Thus, speakers have to assess the extent to which individual elements of their linguistic production are activated in the hearer’s mind and they have to structure their linguistic productions accordingly. The grammatical structures which are needed for that purpose are summarised under the term ‘information structure’ (Prince 1981; Lambrecht 1994; Féry and Ishihara 2016). Chafe (1976), who used the somewhat more concrete term of ‘information packaging’, offered the following well-known picture of how the informational statuses of linguistic units are assessed: The statuses to be discussed here have more to do with how the content is transmitted than with the content itself. Specifically, they all have to do with the speaker’s assessment of how the addressee is able to process what he is saying against the background of a particular context. Not only do people’s minds contain a large store of knowledge, they are also at any one moment in certain temporary states with relation to that knowledge ... Language functions effectively only if the speaker takes account of such states in the mind of the person he is talking to. (Chafe 1976, p. 27)

One important component of information structure is the topic. Even though there are many different definitions, there is general agreement that the topic is associated with ‘aboutness’ as in the following definition: By topicality we understand everything pertaining to ‘what the clause is about’, given the informational setting in which it occurs. (Dik 1997, p. 68)

While the topic generally expresses a concept that the speaker assumes is activated in the hearer’s mind, its counterpart—the comment—provides information which the speaker assumes to be absent in the hearer’s mind within a given communicative situation. The following somewhat technical quotation shows one way of modelling the interaction of topic and comment: An entity, E, is the topic of a sentence, S, if in using S the speaker intends to increase the addressee’s knowledge about, request information about, or otherwise get the addressee to act with respect to E. A predication, P, is the comment of a sentence S, if in using S, the speaker intends P to be assessed relative to the topic of S. (Gundel 1988, p. 210)

In many languages of East and mainland Southeast Asia, among them Japanese, Korean and Chinese, the notion of topic is syntactically broader than in Indo-European languages like English or German. It is for that reason that they are called ‘topic-prominent languages’ in a seminal paper by Li and Thompson (1976). In the case of Japanese and Korean, the topic is expressed by a specific marker, which is -wa in Japanese and –(n)un in Korean. In both cases, the information which precedes the topic marker is the topic and what follows it is the comment. Since there are remarkable similarities between topic marking in Japanese and Korean, we shall start with a somewhat more detailed description of Japanese which will be followed by the relevant data on Korean.

212

W. Bisang and P. Czerwinski

In Japanese, normal word order is subject-object-verb (SOV). In sentences in which the whole information is assumed to be inactivated, the subject gets its nominative marker -ga, while the object is marked by the accusative marker -o with a verb like ka-u ‘buy’: (3) a.

Japanese: 田中さんが車を買った。 Tanaka-san-ga Tanaka-Mister-NOM ‘Mister Tanaka bought a car’.

kuruma-o car-ACC

kat-ta. buy-PST

If either the subject or the object has the informational status of a topic, it loses its case marker, takes the sentence-initial position and gets the topic marker -wa (see (3b, c)). The remaining information that follows -wa is the comment, i.e., the information which is supposed to need activation in the hearer’s mind (for more on information structure in Japanese, see some classics like Kuroda 1972, Maynard 1987; Shibatani 1990): (3) b.

c.

Japanese: 田中さんは は、車を買った。 Tanaka-san-wa, kuruma-o Tanaka-Mister-TOP car-ACC ‘As for Mr. Tanaka, he bought a car’. Japanese: 車は田中さんが買った。 kuruma-wa, Tanaka-san-ga car-TOP Tanaka-Mister-NOM ‘As for the car, Mr. Tanaka bought it’.

kat-ta. buy-PST

kat-ta. buy-PST

In contexts of questions in assessment tests, the topic-comment structure is often used instead of an interrogative clause. In such a structure, the information given in the topic corresponds to the question and the student’s task is to simply pick the item from A to D in a multiple-choice test that completes the sentence (i.e. the comment). There are quite a few items of this type in TUCE. One of them is item 53, which is translated into Japanese as follows: (4)

Japanese (TUCE, item 53): 銀行全体で増やせる預金の最大額は ginkoo zentai-de fuyaseru yokin-no saidai-gaku-wa bank entire.system-in increase deposit-GEN maximum-amount-TOP ‘The maximum amount by which banks can increase deposits in the entire banking system TOP...’

This form of topicalisation is straightforward and it does not generate additional costs triggered by the use of an interrogative pronoun as in the English version:

16

Performance in Knowledge Assessment Tests

213

5. What is the maximum amount by which banks can increase deposits in the entire banking system? To illustrate the difference between the Japanese and the English version, one may start from the literal English translation of (4), repeated below as (5a). At a next stage, the topic marker (TOP) is replaced by is what (5b). From that basis, the targeted English version in (5) can be reached by first moving the copula is to the clause-initial position (5c) and then the interrogative pronoun what (5d): (5)

(a) The maximum amount by which banks can increase deposits in the entire banking system TOP. (b) The maximum amount by which banks can increase deposits in the entire banking system is what? (c) is the maximum amount by which banks can increase deposits in the entire banking system what? (d) What is the maximum amount by which banks can increase deposits in the entire banking system?

Like Japanese, Korean is an SOV language, whose subjects and objects are marked for case. Example (6a) is analogous to (3a) with no topic marker. Similarly to Japanese, case-marked subjects and objects lose their case marking in the sentence-initial topic position. In (6b), the topicalised noun is the subject, in (6c) it is the object (Jun 2015, p. 184): (6) a.

b.

c.

Korean: 자라가 토끼를 물었다. cala-ka thokki-lul turtle-NOM rabbit-ACC ‘The turtle bit the rabbit’. Korean (Jun 2015, p. 184): 자라는 토끼를 물었다. cala-nun thokki-lul turtle-TOP rabbit-ACC ‘As for the turtle, it bit the rabbit’. Korean (Jun 2015, p. 184): 토끼는 자라가 물었다. thokki-nun cala-ka rabbit-TOP turtle-NOM ‘As for the rabbit, the turtle bit it’.

mul-ess-ta. bite-PST-DECL

mul-ess-ta. bite-PST-DECL

mul-ess-ta. bite-PST-DECL

The topic is also used for forming questions in Korean. The following example from the TUCE has the same topic structure as the Japanese example in (4):

214

(7)

W. Bisang and P. Czerwinski

Korean (TUCE, item 53): 은행 전체에서 증가시킬 -수 있는 예금의 최대 금액은, unhyang cenchey-eyse cungka sikhi-l swu iss-nu-n yeykum-uy bank entire.system-in increase-MODAL.IPF.PART deposit-GEN choytay kumayk-un maximum amount-TOP Lit.: ‘The maximum amount by which banks can increase deposits in the entire banking system TOP...’

Clause Combining and Converbs In English, relations of adverbiality between clauses are mostly expressed with conjunctions (or adverbial subordinators) like when, after, before, while, because, if, even if, although, etc. Events which take place in a temporal sequence are mostly linked by the conjunction and (or more complex expressions like and then) as in John sat down and (then) read the newspaper. Japanese and Korean also use markers which can be compared with conjunctions but these markers occur at the end of the clause and, what is more important, are used rather rarely. By far more common are converbs (see below), nominalisations and clause-final particles (e.g. ga ‘but’, to ‘if’). Given the complexity of the domain of clause combining, we will focus on converbs and, in the case of Japanese, on clause-final particles. The selection of these phenomena is motivated by their frequency in the TUCE data. For the same reason, the fine-grainedness of the individual markers will be illustrated for conditionals in the case of Japanese and for temporal relations in the case of Korean. Converbs are roughly defined as verb forms which cannot be used alone and express an adverbial or a temporal relation to the event expressed by the main verb (for questions of definition and cross-linguistic differences and similarities, see Haspelmath 19953; Nedjalkov 1995; Bisang 1995, 2016). Such forms also exist in English and other languages of Europe but their number and function is rather limited and they are not nearly as frequent in texts as they are in Japanese and Korean. In English, converbs are generally discussed under the term of ‘gerund’. They are expressed by participles as in Watching with his cold yellow eyes, D. understood the anger of the blind woman perfectly (König 1995, p. 63). Romance languages like French, Spanish or Italian have specific forms for expressing the function of gerunds.

3

In Haspelmath’s (1995) definition, converbs only express adverbial relations. In most other approaches, among them Nedjalkov (1995) and Bisang (1995), converbs can express adverbial as well as sequential relations between two propositions.

16

Performance in Knowledge Assessment Tests

215

Japanese and Korean have significantly more developed system of converbs. Some of them are functionally rather versatile and cover a broad range of possible semantic relations to the proposition they precede (also see the situation in English as presented in Kortmann 1995). They are called ‘contextual converbs’ by Haspelmath (1995; see also Alpatov and Podlesskaya 1995). Other converbs, the specialised converbs, express very specific interclausal semantic relations. With this inventory the two languages have excellent tools for defining the relation between two propositions with exactly the right precision as it is needed in a given context. In the case of assessment tests like TUCE, the detailed clarification of how different clauses and their contents are semantically related can be an important strategy for enhancing students’ performance in solving test problems. Japanese has two contextual converbs, the converb in -te and the converb in -i (Kuno 1973: 195ff.; Alpatov and Podlesskaya 1995; Hasegawa 1996). The following well-known examples (8a) and (8b) from Kuno (1973) only differ with regard to the converb form of the verb nug-u ‘take off’ (te-form: nui-de vs. i-form: nug-i): (8) a.

b.

Japanese (Kuno 1973, p. 195): ジョーンは上着を脱いで でハンガーに掛けた。 John-wa uwagi-o nui-de John-TOP jacket-ACC take.off-CVte ‘John took off his jacket and put it on a hanger’. ジョーンは上着を脱ぎハンガーに掛けた。 John-wa uwagi-o nug-i John-TOP jacket-ACC take.off-CVi ‘John took off his jacket and put it on a hanger’.

hangaa-ni kake-ta. hanger-on hang-PAST

hangaa-ni kake-ta. hanger-on hang-PAST

Even though both sentences can have the same English translation, they differ in meaning. In many contexts, the converb in -te indicates that the proposition it marks is completed before the next proposition can take place, while the converb in -i signals simultaneity (Kuno 1973, p. 199). Even though this is a good approximation, the situation is more complex. In an insightful paper, Ono (1990) shows that the te-form implies greater continuity between the two propositions involved than the i-form. The continuity manifests itself in temporal distance, spatial distance and the identity of the nominal concepts expressed in the two clauses involved.4

4

This semantic difference is also reflected in the syntactic status of the clauses marked by the te-form and the i-form. Converb forms of the Japanese and Korean type are often discussed in theoretical approaches for which the distinction between coordination and subordination is not sufficient from a typological perspective (Croft’s 2001, p. 320 coordination-subordination continuum or the three nexus types of coordination, co-subordination and subordination in Van Valin Jr., 2004, p. 183ff.). Seen from the perspective of a continuum, the i-form is more coordinative than the te-form (see also Alpatov & Podlesskaya 1995, p. 474). The syntactic difference between the two forms shows up in various syntactic tests, among them relativization, the use of the interrogative particle ka, the replacement of the subject by an interrogative pronoun (Kuno 1978, p. 123ff.), pseudo-clefting, equi-NP deletion, subject honorification and the replacement of subject and object by a pronominal copy (Tamori 1976).

216

W. Bisang and P. Czerwinski

Thus, the temporal interval and the spatial distance between the two propositions of ‘taking off his jacket’ and ‘putting it on a hanger’ are necessarily short with the te-form in (8a), while the two propositions may not take place in the same room or may take place after a somewhat longer period of time with the i-form in (8b). The semantic versatility of the i-form lies in the indeterminacy of continuity as defined above. In contrast, the versatility of the te-form has more to do with the semantic quality of the relation between the propositions involved. Thus, the te-form can express temporal sequentiality (He drinks coffee and then reads a newspaper), temporal overlap/simultaneity (He reads the newspaper while drinking coffee), modification/manner (Smiling, he opens the door) or cause (The water being too hot, he did not take a bath). The following two examples from TUCE excellently illustrate the difference between the te-form and the i-form in terms of the difference between temporal sequentiality (10) and simultaneity (11). In (10), we present the relevant part from item 8, which is answer D. The overall contexts run as follows: 9. English (TUCE, item 8): A recent hurricane destroyed half of the orange crop. Consumers are responding to an increase in the price of oranges by buying more apples. This change is expected to increase the price and quantity of apples sold. In terms of basic supply and demand analysis, there has been a: D: shift in the demand curve for oranges and a movement along the demand curve for apples. (10)

Japanese (TUCE, item 8D): オレンジの需要曲線に沿って価格が変化し、 orenji-no zyuyoo-kyokusen-ni-sotte kakaku-ga henka-si, orange-GEN demand.curve-DAT-along price-NOM change-make:CVi 2 リンゴの需要曲線がシフトした。 ringo-no zyuyoo-kyokusen-ga sihuto si-ta. apple-GEN demand-curve-NOM shift do-PST ‘Movement along the demand curve for oranges and a shift in the demand curve for apples’. 1

In (10), the simultaneity of the two events of ‘price change along the demand curve of oranges’ and the ‘shift in the demand curve of apples’ as it is expressed by the converb in -i is crucial for answering the test question. In contrast, in (11), it is crucial that the ‘increase of interest rates’ occurs before ‘more saving is encouraged’. For that reason, the converb in -te is used: (11)

Japanese (TUCE, item 39A): 市場金利を引き上げて て貯蓄を促すこと。 Sizyookinri-o hikiage-te tyotiku-o interest rates-ACC increase-CVte savings-ACC ‘Interest rates increase, encouraging more saving’.

unagas-u koto. encourage-NPST NMLZ

16

Performance in Knowledge Assessment Tests

217

Of particular interest is the following example from TUCE, in which both converb forms are used. The first event of ‘producing as much as possible’ must clearly be finished and is therefore marked by the converb in -te before the next two simultaneous events of ‘selling all the output’ and ‘trying to reduce the fixed unit costs’ can take place, which are marked by the i-form accordingly: (12)

Japanese (TUCE, item 29D): できる限り多く生産して て生産物をすべて売り、 dekiru kagiri ooku seisansi-te seisanbutu-o subete ur-i As much as possible produce-CVte output-ACC all sell-CVi 2 生産物 1 単位当たりの固定費用負担を下げようとしている。 seisanbutu 1 tan’i atari-no kotei hiyoo futan-o sageyoo-to si-te i-ru. output 1 unit per fixed cost burden-ACC try.to.reduce-PROG Lit.: ‘Produce as much as possible and then sell all the output while keeping down the fixed unit costs’. 1

The English version as reproduced in (13) presents the relevant information in different portions and does not express the temporal relations between them as in the Japanese version: 13. D: Try to sell all the output it can produce, to spread fixed costs across the largest possible number of units. There is also a considerable inventory of specialised converbs in Japanese. The following list presents the most important ones: 14. Japanese: specialised converbs: -ba: -tara: -tari: -temo: -nagara:

conditional (see below for more details; if) conditional (see below for more details; if) coordination of alternating events (and) concessive (even though) (1) simultaneous (while), (2) concessive (although)

Conditionals are expressed by the converbs in -ba and -tara and, additionally, by the two clause-final particles to and nara. For the understanding of the specific meaning of these markers, it is important to be aware that they never express pure conditionality. They are always combined with meaning components from other grammatical categories (e.g. modality and temporality). The conditional particle nara differs from the other markers inasmuch as the speaker does not take her/his perspective but rather the perspective of the hearer or some other people. Thus, it presents the proposition it marks as ‘the assertion by the hearer (or people in general) without completely agreeing with it’ (Kuno 1973, p. 176). This can be illustrated by item 32 from TUCE, which runs as follows in the English version:

218

W. Bisang and P. Czerwinski

15. If workers or businesses anticipate that an expansionary monetary policy will increase inflation, the effects of this policy on real output will be: Answer C: Smaller, if workers demand and receive higher wages. The conditional in answer C is marked by nara in the Japanese translation: (16)

Japanese (TUCE, item 32C): 1 労働者が賃金引き上げを要求し、その通りに roodoosya-ga tinginhikiage-o yookyuusi, sono toori-ni workers-NOM wage.increase-ACC demand:CV that way-in 2 なるな なら小さい。 nar-u nara tiisa-i. become-NPST if small-NPST Lit.: ‘[It will be] small, if [we assume that] workers demand and receive higher wages’.

If marked by nara, the propositions of ‘demanding and getting higher wages’ in (16) are marked as situations people would generally agree on even though the speaker, or the writer in this case, does not commit her/himself to this statement. This specific semantic relation is not expressed in the English version. The other three markers reflect conditions which are based on the speaker’s assessment of a proposition. The converb in -ba generally marks propositions as hypothetical, while the converb in -tara can also express that a proposition is known to take place by the speaker (presumptive) or that the speaker believes that it takes place (predictive; Alpatov and Podlesskaya 1995, p. 478). In contrast to the conditional markers -ba and nara, the converb in -tara expresses a specific temporal relation in which the proposition it marks must be finished before the next proposition can take place (Kuno 1973, p. 183). Finally, the particle to prototypically expresses two types of conditionals (Kuno 1973, p. 193f.). It can mark general statements (e.g. When the rain is falling, people don’t like walking in the streets.) or it indicates that the proposition it marks is a specific single event which is in no ‘logical antecedent-consequent relation’ (Kuno 1973, p. 194) to the proposition expressed by the subsequent clause. There are good examples for the conditional markers -tara and to in the TUCE data. The English version of TUCE item 11 as indicated in (17) is translated with the converb -tara as in (18): 17. If tariffs are increased, the long-run effect is most likely to be: ... (18) Japanese (TUCE, item 11): 1 関税が引き上げられた たら、その長期的な影響として kanzei-ga hikiage-rare-tara, sono tyookiteki-na eikyoo tosite tariffs-NOM increase-PASS-CV its long-run-ADJ effect as for 2 最も考えられるのは mottomo kangae-rare-ru no-wa mostly think.about-can-NPST NMLZ-TOP Lit.: ‘If tariffs are increased [and this proposition is entirely plausible, and would take place before the next one], the long-run effect can be thought to be...’

16

Performance in Knowledge Assessment Tests

219

In (18), the information that the event of ‘increased tariffs’ is entirely plausible or even likely, and must take place before ‘the long-term effect’ to be found out by the students is explicitly added. It is not stated in the English version (17), in which it must be inferred from context. In the example from TUCE item 57, the marker to is used for expressing a general statement (20). In the English version, the conditional clause is simply marked by if and follows the main clause: (19) Reducing inflation will require the monetary authorities to make larger changes in the money supply if most people expect a rapidly rising price level. In the Japanese version, the conditional clause is moved to the beginning of the sentence and is marked by to (for reasons of space, the translation of the main clause is omitted in (20)). (20)

Japanese (TUCE, item 57): 多くの人々が急激な物価上昇を予想すると と、 ooku-no hitobito-ga kyuugeki-na bukka-zyoosyoo-o yosoosur-u to, ... most people-NOM rapid-ADJ price-increase-ACC predict-NPST COND ‘[Generally,} if most people expect a rapidly rising price level...’

In this context, to specifies that the conditional relation expressed in (20) holds generally. Thus, whenever the proposition ‘most people expect a rapidly rising price level’ is true, the situation expressed in the main clause will take place. Again, this specification is not expressed in the English version in (19). Korean has a much larger inventory of converbs than Japanese (Sohn 1999, 2009; Hatcher Jr., 2013). Even in a small text sample as the data from TUCE, we find 13 different forms. As in Japanese, some converbs cover a broad range of functions (contextual converbs), while others express more specific functions (specialised converbs). Each of the 13 converbs will be briefly presented before looking at temporal relations in more detail. There are three contextual converbs in the TUCE data. The first one is the converb in -ko. Its main function is ‘to connect two independent events or states’ (Sohn 2009, p. 300). As can be seen from the following example, the temporal relation between the two propositions may be sequential (first proposition X, than proposition Y), simultaneous or neutral to temporal order: (21)

Korean (Sohn 2009, p. 300): 기호가 노래하고 미아가 춤추어요. Kiho-ka nolayha-ko Mia-ka Kiho-NOM sing-CV Mia-NOM (a) Sequentiality: ‘Kiho sings and then Mia dances’. (b) Simultaneity: ‘Kiho sings and Mia dances’. (c) Neutral: ‘Kiho sings and Mia dances’.

chwumchw-eyo. dance-POL

220

W. Bisang and P. Czerwinski

The converb in -a/e(se) is always combined with main clauses in the declarative form. It generally states that the proposition in the converb clause directly affects the matrix predicate. Thus, the converb clause is understood as a cause or a precondition for the proposition of the main clause. If the converb in -a/e(se) occurs with verbs expressing dynamic events, it signals that the proposition must be completed before the proposition expressed by the main verb can take place. With stative prepositions (e.g. be young, be tall), it signals that the proposition of the main predicate ‘takes place in the state produced by’ the proposition of the converb clause (Lee 1993, p. 454). In this context, the converb in -a/e(se) can express causal, instrumental, purposive and, to a certain extent, even simultaneous relations. The converb in -mye signals either simultaneity or, similar to the converb in -ko, the semantic coordination of two independent clauses (Hatcher Jr., 2013, p. 26). The specialised converbs are listed in (22) with some short indications of their functions: (22) Korean: specialised converbs: -tolok:

-myen:

-e/ato: -(u)mulo: -(nu)ntey:

-(u)lye(ko):

-key:

This converb has two specialised functions: (1) Terminative function: The proposition of the main predicate takes place until the proposition of the subordinated verb takes place. (2) Purposive function, translated as ‘so that’ (Sohn 2009). Conditional converb: Different subtypes of conditionality between possible and counterfactual depend on the occurrence of additional markers (Sohn 2009, p. 305f.). The main clause must be in the declarative.5 Concessive converb, translated as ‘although, even though’ Causal relation, translated as ‘because’. This converb signals that its proposition has the status of background information (see Sohn’s 2009, p. 312 term ‘background information provider’). It may be translated by ‘given that’. If the main clause is in the imperative, the same marker indicates a causal relation. This converb marks purpose. It is translated as ‘in order to’. The short form in -(u)lye only occurs if the verb in the main clause is a movement verb. There is no such restriction with the long form -(u)lyeko (Sohn 2009, p. 310). Moreover, the subject of the converb clause must be identical to the subject of the main clause. This converb marks purpose like -(u)lye(ko) (Sohn 2009) but its subject may also differ from the subject of the main clause. If it occurs with adjectives it marks result (see the adjective short in He cuts his hair short, Hatcher Jr., 2013, p. 39).

5 There is another conditional converb in -ketun which is used if the main verb is an imperative, a propositive or an intentive clause.

16

Performance in Knowledge Assessment Tests

-telato:

221

This converb marks concessives (‘although, even though’) and concessive conditionals (‘even if’).

Finally, there are two more markers, whose status is controversial. The marker -kena signals alternative disjunction (‘or’) and may thus be interpreted as a coordinative suffix (Sohn 2009, p. 312f.), while the marker -ciman oscillates between the functions of a concessive converb (‘although’) and an adversative coordination (‘but’) as is illustrated by the following example: (23)

Korean (Sohn 2009, p. 289): 존은 오래 독일어를 배우지만 잘 못 해요. John-un olay tokile-lul John-TOP long German-ACC

paywu-ciman that.way-in

cal mos hay-yo. well not do-POL (a) Converb: ‘Although John has been studying German for years he does not speak it well’. (b) Adversative coordination: ‘John has been studying German for years but he does not speak it well’.

Korean has a particularly rich inventory of temporal converbs, including anterior, simultaneous and the typologically rare posterior converb in -tolok. In the TUCE sample they are used consistently to denote the specific temporal relationship between the two clauses (propositions). In each of the examples presented below, this information is missing in the English versions of the TUCE as they are presented in the translations. In the example below, the temporal converbs in -ko (simultaneous) and -a/e(se) (anterior) specify that the reduction in import tax together with the parallel increase in export subsidies leads to an increase in external trade. (24)

Korean (TUCE, item 31C): -수입품에 대한 관세가 내려가고 고, -수출품에 대한 보조금이 늘어나서, 무역의 규모 가 확대되므로. swuipphwum-ey tayha-n kwansey-ka naylyeka-ko imports-DAT concern-PART import.tax-NOM decrease-CVsimult swuchwulphwum-ey tayha-n pocokum-i nulen-ase exports-DAT concern-PART subsidy-NOM increase-CVanterior mwuyek-uy kyumo-ka hwaktaytoy-mulo. external.trade-GEN scale-NOM expand-CVcausal ‘Promotes higher levels of international trade by reducing taxes on imports and increasing subsidies for exports’.

In the following example, the simultaneous converb in -mye specifies that the company can sell ice cream at that price while employing workers at the wage specified:

222

(25)

W. Bisang and P. Czerwinski

Korean (TUCE, item 12): 이 회사는, 1인당 4,500원의 일당으로 노동자를 고용할 -수가 있으며, 1개당 100원으로 아이스 크림을 판매할 -수가 있다. i

hoysa-nun

1 intang

4,500 wen-uy

1 person.per

4,500 won-GEN per.day-INSTR worker-ACC

koyongha-l swu-ka issu-mye

1 kaytang

wen-ulo

aisukhulim-ul

employ-MODAL-CONVsimult

1 unit.per

won-INSTR

ice.cream-ACC

This company-TOP

iltang-ulo

notongca-lul

phanmayha-l swu-ka iss-ta. sell-MODAL-DECL ‘It can hire workers for $45 a day and sell ice cream cones for $1.00 each’.

Finally, the posterior converb specifies that the action continues until the event denoted by the converb occurs: (26)

Korean (TUCE, item 14D): 생산량을 제한하여, 제품가격이 한계비용을 상회하도록 하기 때문이다. sayngsanlyang-ul ceyhanha-ye ceyphwumkakyek-i hankyeypiyong-ul produced.amount-ACC limit-CVsimul product.price-NOM marginal.cost-ACC sanghoyha-tolok ha-ki ttaymwuni-ta. exceed-CVposter do-NMLZ reason.be-DECL ‘Restricting output to levels at which their products are valued more than the marginal cost of producing them’.

For each of the above examples, the English version lacks the specific temporal distinctions and instead uses conjunctions with very broad meaning like ‘and’.

Evidentials and Modality Propositions can be classically defined in terms of their factuality. This leads to a distinction between propositions which are factual and propositions which belong to the domain of modality, which is further subdivided into the domains of epistemic modality (necessity and possibility) and deontic modality (obligation and permission). Epistemic modality reflects the speaker’s assessment of the truth status of a proposition from the perspective of her/his knowledge, while deontic modality addresses the degree of compulsion associated with a proposition from the perspective of social norms that exist in a speech community.6 In English, modality as a grammatical category is primarily expressed by modal verbs like can, may, must, should, ought to, etc. Evidentiality is a special category which addresses the source of the information provided by the proposition. In various languages with obligatory evidential 6 This is a simplified presentation of a highly complex topic (for some further information, see Palmer 1986; De Haan 2012; Narrog 2012, Nuyts & van der Auwera 2016).

16

Performance in Knowledge Assessment Tests

223

markers, speakers have to select from a set of markers which indicate the type of evidence they have for the information they present (Willett 1988; Aikhenvald 2004). Is the information based on direct (e.g. visual) evidence or is it based on indirect evidence (e.g. inference or hearsay)? In the Amazonian language Tariana, the verb in a simple clause like ‘John has played football’ can have five different markers, depending on whether the evidence of the speaker is direct visual (The speaker has seen John playing football), direct non-visual (The speaker has heard John playing football), inferred from context, assumed because of previous experience or hearsay (Aikhenvald 2004, p. 2f.). As Aikhenvald (2004: 2) points out for Tariana, the use of evidential marking is obligatory, ‘[o]mitting an evidential results in an ungrammatical and highly unnatural sentence’. English does not have a grammatical system specifically for marking evidentiality nor is its expression compulsory but it can use adverbs like allegedly and there are some specific contexts in which modal verbs may be interpreted in the sense of an evidential: 27. John must have arrived yesterday [The speaker has not observed this event but s/he infers it from the fact that John’s suitcase is in front of the door]. In German, the deontic modal verb sollen ‘should’ can also express the evidential function of hearsay: (28)

German: Hans soll John should ‘It is said that John will arrive today’. [hearsay]

heute ankommen. today arrive

The relation between modality and evidentiality is the subject of hot debate in linguistics. While evidentiality is a subcategory of epistemic modality for some linguists (Palmer 1986), other linguists see evidentiality as a separate linguistic category (Aikhenvald 2004; De Haan 2006). A third type of linguists sees some areas of overlap between evidentiality and modality (van der Auwera and Plungian 1998). What seems to be quite clear is that evidentiality looks at the truth status of a preposition from the perspective of the source of information available to the speaker, a perspective which is not a necessary precondition for factuality judgements in terms of epistemic modality. An important reason for the debate on the status of evidentiality is the observation that one and the same marker can express functions of both categories (see must have been in (27) and German sollen ‘should’ in (28)). Since this is ultimately no problem for the present paper, which is mainly interested in how a system of highly specified markers can enhance cognitive skills in problem solving, we shall not attempt any solutions to this general problem. However, what can be said from the perspective of Japanese and Korean is that these two languages have an impressive number of markers which specifically express evidentiality. Even though these markers are not obligatory as in Tariana, they are used rather frequently in certain contexts, among them in contexts in which detailed information concerning the origin and reliability

224

W. Bisang and P. Czerwinski

of information is relevant. For that reason, evidential marking is relatively frequent in the context of examinations and assessments like TUCE. Since the category of evidentiality is less known outside of linguistics, the focus of this paper will be on that category. Modality will be described in a more selective way for explaining specific cases found in TUCE. Japanese has at least the following evidential markers which generally take the last position in the paradigm of Japanese verb forms: 29. Japanese evidential markers: -soo:

-yoo:

-rasii:

-hazu:

-daroo:

7

This marker originates from a noun meaning ‘appearance’. It has two meanings. (1) Based on its historical origins, it denotes ‘the appearance of an object, primarily based on visual impressions, but secondarily also on auditive impressions or knowledge that the speaker has otherwise gained’ (Narrog 2009, p. 119). With dynamic verbs its meaning extends to ‘a “general” prediction about things to happen in the near future’ (Narrog 2009, p. 120). In this first function, -soo is deductive. (2) In its second function, it indicates reported information (reportative; Narrog 2009, p. 116f.; also see -rasii below). Visual or accessible inference: The inference associated with this marker and the next one is based on abduction, i.e., it marks ‘an inference about some reason or grounds p, based on evidence q, which is the result of q’ (Narrog 2009, p. 118).7 In the case of -yoo, the conclusion is based on the evidence of an immediate, mostly visual impression. Non-accessible inference: This marker is also based on abduction. In contrast to -yoo, the conclusion is based on evidence which is ‘inaccessible to the speaker perception or unexpected’ (Narrog 2009, p. 118). The meaning of -rasii goes beyond the reportative function of the second function of -yō inasmuch as it is not strictly limited to quoting the proposition it marks. It indicates rather that ‘the speaker has added some reasoning to it’ (Narrog 2009, p. 117). Expectation: This marker ‘denotes that a proposition is held to be true relative to strong beliefs, general rules or knowledge in the mind of the speaker, which functions as the presuppositions for the conclusion modified by hazu’ (Narrog 2009, p. 103). In that sense, hazu is associated with deductive reasoning. This marker is called ‘speculative’ by Narrog (2009, p. 100, 105ff.). Its main function lies rather in the expression of what the speaker infers or thinks about a given state of affairs than in the expression of its specific position in the epistemic continuum between certainty and probability.

An example of abduction from Narrog (2009, p. 118): The speaker knocks at someone’s door and gets no answer. In such a situation s/he may conclude that the person living in that place is not at home and mark the verb expressing that state of affairs with -yoo or -rashii.

16

Performance in Knowledge Assessment Tests

225

Evidentials are found in various items of TUCE. The following two examples illustrate the markers -soo (30) in its first function (function (1) in (29)) and -hazu (32) as an indicator of expectation. In (30), the marker -soo is embedded into a nominalised construction formed by the nominaliser -no followed by the topic marker -wa8: (30)

Japanese (TUCE, item 7): ある企業が、市場を最も Aru kigyoo-ga sizyoo-o mottomo Some firm-NOM market-ACC most 2 独占化しそ そうなのは、 dokusenkasi-soo-na-no-wa monopolise-PREDICT-ADJ-NMZ-TOP Lit.: ‘A firm is generally predicted to be most likely to monopolize a market TOP’. 1

While the English version of (30) as given in (31) does not address the source of the statement or the likelihood of market monopolisation, the Japanese version clearly indicates that the proposition can be predicted on the basis of general knowledge. (31) A firm is most likely to monopolize a market whenever ... In example (32), the marker hazu signals that the effect of long-run cutbacks in production becoming much smaller matches general expectations: (32)

Japanese (TUCE, item 16): 長期的には、生産量の減少はずっと tyookiteki-ni-wa, seisanryoo no gensyoo-wa zutto long-run-in-TOP production.rate GEN cutbacks much 2 小さくなるは はずである。 tiisaku-naru-hazu-dearu. small-become-EXPECT-be.PRES Lit.: ‘In the long run, cutbacks in production are expected to become much smaller’. 1

A look at the English version in (33) shows that the information that cutbacks are to be expected under the conditions specified in test item 16 is missing. (33) [If the tax was levied directly on the amount of pollution generated], the long-run cutbacks in production would be much smaller. Even though modality is a very broad field, we only present the grammatical marking of modality in terms of ability and situational possibility as it is expressed by the potential construction (Narrog 2009 p. 96ff.). This function is basically expressed by the marker -(ra)re9 and the verb deki-ru ‘can’. Both express the

On the use of the topic construction for expressing questions in tests, see section “Information Structure”. 9 Depending on its morphological environment, the marker –(ra)re- can be realised in various forms. We don’t discuss these details in this paper. 8

226

W. Bisang and P. Czerwinski

same meaning even though they occur in different morphosyntactic environments. The verb deki-ru stands in for the verb su-ru ‘make, do’, which cannot take the potential marker -(ra)re-. Both markers occur frequently in the TUCE corpus. In (34), the marker -(ra)reinforms the reader that what is said is possible in the situation presented by the test item. While the English version in (35) presents the propositions with the verbs heras-u ‘reduce’ and kaseg-u ‘earn’ as facts, the Japanese version describes them as possible in a situation in which flu shots are received: (34)

Japanese (TUCE, item 28): 病気で休む日数が減らせ せるから、 byooki-de yasum-u nissuu-ga heras-er-u kara illness-because take. days-NOM reduce-POSSIBLE-NPST because time.off 2 より多く稼げるようになること。 yori ooku kaseg-er-u yooni nar-u koto. more a lot earn-POSSIBLE-NPST CMPL become-NPST NMLZ Lit.: ‘The fact that [If flu shots are received] it is possible to earn more because it is possible to reduce sick days’. 1

35. [Flu shots reduce sick days] allowing those who get flu shots to earn more income. In (36), the marker deki-ru expresses exactly the same meaning as -(ra)re-: (36)

Japanese (TUCE, item 10): 1 天然ガスは、石油よりもずっと tennen gasu-wa, sekiyu yori-mo zutto natural gas-TOP oil than-FOC much 2 安価に暖房に利用できる。 anka-ni danboo-ni riyoo deki-ru. cheap-ADV heating-for profit can.NPST Lit.: ‘It is possible that natural gas may provide heat at a much lower cost than oil’.

The English original in (37) simply asserts the lower costs induced by natural gas as a fact. In contrast, the Japanese marker deki-ru creates a relation to the preceding information (given in brackets in the English version in (37)) and states that in such a situation it is possible that costs will be much lower. 37. [In an economy where heating oil is the primary source of heat for most households, new supplies of natural gas, a substitute for heating oil, are discovered.] Natural gas provides heat at a much lower cost. In Korean, evidential markers occur in different positions of its rather complex verbal paradigm. Sohn (1994, p. 60) presents the following slots, starting from

16

Performance in Knowledge Assessment Tests

227

the verbal root (V) to the left of the form (SH and AH stand for different honorific forms, SH ¼ subject honorific, AH ¼ addressee honorific)10: 38. V – SH – TENSE/ASPECT – MODALITY – AH – MOOD – terminal Evidential markers occur either in the terminal11 position or in one of the non-terminal positions of TENSE/ASPECT, MODALITY or MOOD. The following lists present the most widely used and widely recognised terminal (39) and non-terminal (40) evidential markers with a short functional description: 39. Terminal evidential markers of Korean: -kwun -ci

-na/-nka/-lkka

-ney -tay

Mirative (on the term, see DeLancey 1997): the information of the proposition is unexpected. Commitment: ‘the speaker is committed to the truth of the propositional content conveyed’ (Lee 2015, p. 348; see also Lee 1999). Abduction: ‘a source or cause is conjectured based on a situation that is known to be its consequence’ (Lee 2015, p. 254). In contrast to -ci, the speaker is not committed to the truth of the proposition (Lee 2015, p. 257). Factual realisation: the speaker ‘has just become aware of the propositional content’ (Lee 2015, p. 257). Hearsay: ‘the propositional content is obtained as hearsay rather than through direct experience’ (Lee 2015, p. 253).

40. Non-terminal evidential markers of Korean: -te-

-keyss-(u)li-

-la-

first-hand evidential: ‘refers to a past moment at which the speaker views or experiences the situation described, ..., and it implies that the information about the situation described was obtained through the speaker’s perceptual experience’ (Lee 2015, p. 254). deductive reasoning: ‘prediction or conjecture made through deductive reasoning’ (Lee 1991, p. 124, Lee 2015, p. 253). presumptive: This marker ‘expresses the speaker’s simple presumption or speculation about situations the speaker cannot get access to’ (Lee 1991, p. 124f.). Its use is limited to literary style (poems, poetic prose). introspective evidential: This marker expresses ‘the speaker’s introspection in soliloquy’ of an event ‘he or she has just perceived’ (Lee 1991, p. 430f.).

The following three examples illustrate the use of the non-terminal deductive reasoning marker (41) and the two terminal markers for expressing hearsay (42) and abduction (43): 10

For the sake of readability, we have slightly modified Sohn’s (1994) template. An important group of markers in the terminal slot indicate whether the status of a clause is that of a declarative, a propositive (or hortative), an interrogative or an imperative. 11

228

W. Bisang and P. Czerwinski

(41)

Korean: deductive reasoning (Jun 2015, p. 253): 오늘 비가 오겠다. Onul pi-ka o-keyss-ta. Today rain-NOM come-DEDREAS-DECL Looking at the cloudy sky: ‘[I gather] it will rain today’.

(42)

Korean: hearsay (Jun 2015, p. 253): 오늘 비가 온대요. Onul pi-ka Today rain ‘It is said that it will rain today’.

(43)

o-n-tay-yo. come-NONPAST-HEARSAY-DECL

Korean: abductive reasoning (Jun 2015, p. 253): 피곤한가 봐요. phikonha-nka pwa-yo. tired-ABDREAS see-POL [Seeing someone who yawns] ‘You must be tired’.

In the TUCE sample, a broad range of evidential markers are used to convey additional information about the source, and different levels of commitment to the veracity of the information provided. (44)

Korean (TUCE, item 2): 만약 시장에 규제가 없다면, 이들 재화의 효율적인 산출-수준과 비교해서, X재와 Y재의 생 산은 과잉이 되겠는가, 아니면 과소가 되겠는가? manyak sicang-ey if

kyucey-ka

market-in regulation-NOM

epsta-myen

i-tul

not.be-CVcond

this-PL goods-GEN

cayhwa-uy

hyoyulceki-n

sanchwulswucwun-kwa

pikyoha-yse

X cay-wa Y cay-uy

efficient-PART

output.level-in.relation.to

compare-CV

X axis and Y axis-GEN

sayngsan-un

kwaing-i

toy-keyss-nunka,

ani-myen

production-TOP

excess-NOM

become-DCT.RE-NCOMM

not-CVcond

kwaso-ka

toy-keyss-nunka?

dearth-NOM

become-DCT.RE-NCOMM

‘Would unregulated markets produce too much or too little of Good X and Good Y, compared to the efficient output levels for these products?’

The non-terminal suffix -keyss- ‘indicates that the propositional content conveyed is a prediction or a conjecture made through deductive reasoning; that is, based on a normative expectation of cause/source-effect/consequence relation, the speaker predicts a result or consequence out of a known cause or source’ (Lee 2015 after Chang 1985; Choi 2015). Additionally, the terminal suffix -nunka indicates that the speaker is not committed to the truth of the propositional content conveyed. It is used

16

Performance in Knowledge Assessment Tests

229

in indirect questions in which the speaker indicates his or her uncertainty about the truth of the propositional content conveyed (Lee 2015). Compare this with the example below. (45)

Korean (TUCE, item 9C): 그 산업의 개개의 기업의 장기적 이윤이 증가할 것이다. ku sanep-uy kaykay-uy kiep-uy cangkicek iyun-i this industry-GEN individual-GEN company-GEN long.run profit-NOM cungkaha-l kesi-ta. increase-PRESUM-DECL ‘The long-run economic profits of individual firms in the industry will decrease’.

The periphrastic construction -(u)l ke(s)-i- expresses conjecture. The irrealis adnominal form -(u)l ‘is used to refer to a situation that is yet to occur or one that the speaker only presumes to exist’. Compared to -keyss- with which the speaker’s conjecture is based on deductive reasoning, the conjecture expressed with -(u)l ke(s)-i- does not require any basis. It is simply the speaker’s presumption (Lee 2015). In both examples, in the English version the additional information provided by the evidential markers is entirely missing. (46) Korean (TUCE, item 31): 과거 10년 간 고율의 인플레이션에 휩싸였던 작은 나라가, 과거 50년 간 아주 낮은 인플레이션 밖에 없었던 큰 나라의 통화에, 자국 통화를 링크하기로 결정했다고 하자. kwake 10 nyen kan koyul-uy inphulleyisyen-ey past 10 year period high.rate-GEN inflation-DAT hwipssay-ess-te-n caku-n nala-ka, kwake 50 nyen be.affected-RETROS-REALIS.PART small-PART country-NOM past 50 year kan acwu nacu-n inphulleyisyen pakkey eps-ess-te-n period very small-PART inflation only not.be-PAST-RETROS-REAL.PART khu-n nala-uy thonghwa-ey cakwuk thonghwa-lul big-PART country-GEN currency-DAT own.country currency-ACC lingkhuha-ki-lo kyelcengha-yss-tako ha-ca. link-NMLZ-INSTR decide-PAST-QUOT do-HORT ‘A small country that has experienced high inflation for the past decade decides to set the value of its currency equal to the value of a currency in a large nation that has had very low inflation for the past 50 years’.

The retrospective (non-terminal) suffix -te- ‘implies that the situation described was obtained through the speaker’s perceptual experience’ (Lee 2015). Here, in combination with the realis adnominal marker -n it indicates that we know for a fact that those countries had high/low inflation in the past. Additionally, the addressee is alerted to the fact that we are talking about a hypothetical scenario by the ‘let us assume’ construction, which again is missing in the English version. Finally, as we have shown for Japanese, evidential and modal distinctions seem highly relevant to speakers of both languages. This is evident not only in the finite verb forms as illustrated above. Non-finite verbs as well carry obligatory realis/

230

W. Bisang and P. Czerwinski

irrealis distinctions, -m and -ki, respectively, in the case of deverbal nouns, and -(u)n and -(u)l in the case of participles. As in the Japanese example (36) (which uses the potential form of the verb ‘to do’), Korean uses the irrealis participial form -(u)l to specify that natural gas can, but does not always, or does not always have to, substitute heating oil: (47)

Korean (TUCE, item 10): 석유를 대신할 천연가스가 새로 발견되었다고 하자. sekyu-lul taysinha-l chenyenkasu-ka saylo oil-ACC replace-IRR.PART natural.gas-NOM newly palkyentoy-ess-tako ha-ca. discover-PAST-QUOT do-HORT Lit.: ‘Let’s assume that natural gas, which can substitute oil, has been newly discovered’. English version: ‘New supplies of natural gas, a substitute for heating oil, are discovered’.

Potential Effects on Performance We identified the three grammatical categories described above based on a sample of TUCE items where the difference in performance between English-speaking students on the one hand and Japanese and Korean students on the other was particularly jarring. Looking closely at the sample, it is evident that the Japanese and Korean translations add a wealth of information that is missing from the original English version. On the other hand, for the grammatical distinctions that are obligatory in English but not in Japanese and Korean like definiteness and number (see Section “Information Structure: Japanese and Korean as Topic-prominent Languages”), the translators consistently make those distinctions through lexical means whenever omitting them would lead to ambiguity, for example, using demonstratives like ‘this’ and ‘that’ in place of the English definite article ‘the’, and ‘one’ or ‘certain’ instead of the indefinite ‘a(n)’. There are other ways in which the Japanese and Korean versions are consistently more explicit and less ambiguous than the English version. For example, in relative and adverbial subordinate clauses, there is never any ambiguity as to which argument within the sentence is referred to.12 Those linguistic factors, together with the overall quality of the Japanese and Korean texts as compared to English, the general tendency of translations to add relevant information, as well as various cultural factors (e.g., Japanese and Korean

12

While this may be true for this type of academic text, it is certainly not generally true for either language. To the contrary, they are both well known for extensive use of ‘pro-drop’, and reliance on pragmatic inference in text processing (Bisang 2006, 2009).

16

Performance in Knowledge Assessment Tests

231

students’ unfamiliarity with the inner workings of the American professional sports leagues that one question pertains to), seem to largely account for the difference in performance in the sample. As for the broader sample of 60 questions each for Japanese and Korean, the correlation with performance is less clear, but nevertheless seems to generally point in the same direction. The two tables below show the differences in results between Japanese and Korean versus US students, juxtaposed with the counts of grammatical factors we identified for each item. The tables below list each TUCE item in which at least one of the factors is present. Even though there are some exceptions, the Japanese and the Korean students outperformed the students from the USA. This tendency is clearly stronger with Korean which has larger grammatical inventories and higher frequencies of individual features than Japanese (Figs. 16.1 and 16.2). Korean students’ performance improvement over US students (percentage) Number of identified grammatical factors

Fig. 16.1 Korean students’ performance improvement over US students vs number of identified grammatical factors

Japanese students’ performance improvement over US students (percentage) Number of identified grammatical factors

Fig. 16.2 Japanese students’ performance improvement over US students vs number of identified grammatical factors

232

W. Bisang and P. Czerwinski

These results require further elaboration. As can be seen from the above charts, for Japanese in particular the picture is less clear cut at this stage. This is certainly due to the small size of the sample, as well as the multiplicity of the various linguistic and non-linguistic factors involved. Accounting for all those variables, as well as applying statistical tools to reveal the potential correlations in more detail will be the next stage of our research into this topic. Nevertheless, we believe that these preliminary results confirm our initial assumptions, and attest to the relevance of the categories we identified for cognitive tasks like comprehension and problem solving.

Conclusions The focus of this paper was on cross-linguistic differences in the grammatical inventories of individual languages. The clarification of this issue is crucial for the question of how differences in grammar (including differences in actual use of those different features by translators, another language-related factor that we did not focus on for this paper) may enhance or inhibit students’ ability of problem solving. For that purpose, we presented a detailed description of (1) information structure and topicality, (2) converbs and clause combining and (3) modality and evidentiality. In future work, it will be necessary to analyse how these differences interact with other factors in determining the effect of cross-linguistic differences on reasoning and problem solving. As we have argued in Section “Potential Effects on Performance”, there is some good evidence for correlations. On a linguistic level, it will be necessary to see how the three grammatical domains interact and if they have the same weight or if some of them are more important than others. Our current database is clearly not sufficient to come up with statistically reliable generalisations. In addition to these linguistic issues, it will be necessary to see how the linguistic factors are used in the teaching materials used by the students. Thus, one can ask how different text types and media (e.g. online encyclopaedias, course scripts, textbooks, newspaper articles, etc.) affect the use of the grammatical inventory of a language and how the type of texts consulted by the students affect students’ performance. From another perspective, linguistic factors which turn out as good candidates for affecting student performance may be further used in cognitive experiments at the level of individual students. As we hoped to show, the combination of detailed linguistic insights in extensive interdisciplinary cooperation with other fields has the potential to contribute to a new field of research which aims at understanding how individuals end up acquiring knowledge which is based on evidence and scientific reasoning.

16

Performance in Knowledge Assessment Tests

233

References Aikhenvald, A. Y. (2004). Evidentiality. Oxford: Oxford University Press. Alpatov, V. M., & Podlesskaya, V. (1995). Converbs in Japanese. In M. Haspelmath & E. König (Eds.), Converbs in cross-linguistic perspective: Structure and meaning of adverbial verb forms – Adverbial participles, gerunds (pp. 465–485). Berlin: De Gruyter Mouton. Bisang, W. (1995). Verb serialization and converbs—Differences and similarities. In M. Haspelmath & E. König (Eds.), Converbs in cross-linguistic perspective: Structure and meaning of adverbial verb forms – Adverbial participles, gerunds (pp. 137–188). Berlin: De Gruyter Mouton. Bisang, W. (2006). From meaning to syntax—Semantic roles and beyond. In I. Bornkessel, M. Schlesewsky, B. Comrie, & A. D. Friederici (Eds.), Semantic role universals and argument linking (pp. 191–236). Berlin: De Gruyter Mouton. Bisang, W. (2009). On the evolution of complexity—Sometimes less is more in East and mainland South East Asia. In G. Sampson, D. Gil, & P. Trudgill (Eds.), Language complexity as an evolving variable (pp. 34–49). Oxford: Oxford University Press. Bisang, W. (2014). Overt and hidden complexity—Two types of complexity and their implications. Poznan Studies in Contemporary Linguistics, 50(2), 127–143. Bisang, W. (2015). Hidden complexity—The neglected side of complexity and its consequences. Linguistics Vanguard, 1(1), 177–187. Bisang, W. (2016). Finiteness and nominalization—Convergence and divergence. In C. Chamoreau (Ed.), Finiteness and nominalization (pp. 13–41). Amsterdam/Philadelphia, PA: Benjamins. Bisang, W. (2018). Knowledge representations and cognitive skills in problem solving—A view from linguistic typology. In O. Zlatkin-Troitschanskaia, G. Wittum, & A. Dengel (Eds.), Positive learning in the age of information (PLATO) – A blessing or a curse? (pp. 127–144). Wiesbaden: Springer. Brückner, S., Förster, M., Zlatkin-Troitschanskaia, O., Happ, R., Walstad, W. B., Yamaoka, M., et al. (2015a). Gender effects in assessment of economic knowledge and understanding: Differences among undergraduate business and economics students in Germany, Japan, and the United States. Peabody Journal of Education, 90(4), 503–518. https://doi.org/10.1080/ 0161956X.2015.1068079. Brückner, S., Förster, M., Zlatkin-Troitschanskaia, O., & Walstad, W. B. (2015b). Effects of prior economic education, native language, and gender on economic knowledge of first-year students in higher education. A comparative study between Germany and the USA. Studies in Higher Education, 40(3), 437–453. https://doi.org/10.1080/03075079.2015.1004235 Chafe, W. L. (1976). Givenness, contrastiveness, definiteness, subjects, topics, and point of view. In C. N. Li (Ed.), Subject and topic (pp. 25–55). New York, NY: Academic. Chang, K.-H. (1985). Hyentay kwuke-uy yangsang-pemcwu yenkwu (A study of modality categories in Modern Korean). Seoul: Tower Press. Choi, S. (2015). First language acquisition. In L. Brown & J. Yeon (Eds.), The handbook of Korean linguistics (pp. 339–354). West Sussex: Wiley. Comrie, B. (1981). Language universals and linguistic typology. Oxford: Blackwell. Corbett, G. G. (2000). Number. Cambridge: Cambridge University Press. Croft, W. A. (2001). Radical construction grammar. Syntactic theory in typological perspective. Oxford: Oxford University Press. Croft, W. A. (2003). Typology and universals (2nd ed.). Cambridge: Cambridge University Press. De Haan, F. (2006). Typological approaches to modality. In W. Frawley (Ed.), The expression of modality (pp. 27–70). Berlin: De Gruyter Mouton. De Haan, F. (2012). Typology of tense, aspect and modality systems. In J. J. Song (Ed.), The Oxford handbook of linguistic typology (pp. 445–464). Oxford: Oxford University Press. DeLancey, S. (1997). Mirativity: The grammatical marking of unexpected information. Linguistic Typology, 1, 33–52.

234

W. Bisang and P. Czerwinski

Dik, S. C. (1997). The theory of functional grammar, part I: The structure of the clause (2nd, revised ed.). In K. Hengeveld (Ed.), Functional grammar series 20. Berlin: De Gruyter Mouton. Evans, N., & Levinson, S. C. (2009). The myth of language universals: Language diversity and its importance for cognitive science. Behavioral and Brain Sciences, 32, 429–448. Féry, C., & Ishihara, S. (2016). The Oxford handbook of information structure. Oxford: Oxford University Press. Greenberg, J. H. (1963). Some universals of grammar with particular reference to the order of meaningful elements. In J. H. Greenberg (Ed.), Universals of language (pp. 73–113). Cambridge: MIT Press. Gundel, J. K. (1988). Universals of topic–comment structure. In M. Hammond, E. Moravcsik, & J. Wirth (Eds.), Studies in linguistic typology (pp. 209–239). Amsterdam/Philadelphia, PA: Benjamins. Hasegawa, Y. (1996). A study of Japanese clause linkage. The connective TE in Japanese. Stanford: Center for the Study of Language and Information (CSLI). Haspelmath, M. (1995). The converb as a cross-linguistically valid category. In M. Haspelmath & E. König (Eds.), Converbs in cross-linguistic perspective: Structure and meaning of adverbial verb forms – Adverbial participles, gerunds (pp. 1–55). Berlin: De Gruyter Mouton. Hatcher, R. J. Jr. (2013). A description of Korean converbs and their Northeast Asian context. MA Thesis, Seoul National University. Jakobson, R. (1992). On linguistic aspects of translation. In R. Schulte & J. Biguenet (Eds.), Theories of translation: An anthology of essays from Dryden to Derrida (pp. 144–151). Chicago/London: University of Chicago Press. Originally 1959 in R. A. Brower (Ed.), On translation. Cambridge: Harvard University Press. Jun, Y. (2015). Focus, topic and contrast. In L. Brown & J. Yeon (Eds.), The handbook of Korean linguistics (pp. 179–195). West Sussex: Wiley. König, E. (1995). The meaning of converbs. In M. Haspelmath & E. König (Eds.), Converbs in cross-linguistic perspective: Structure and meaning of adverbial verb forms – Adverbial participles, gerunds (pp. 57–95). Berlin: De Gruyter Mouton. Kortmann, B. (1995). Adverbial participial clauses in English. In M. Haspelmath & E. König (Eds.), Converbs in cross-linguistic perspective: Structure and meaning of adverbial verb forms – Adverbial participles, gerunds (pp. 189–237). Berlin: De Gruyter Mouton. Kuno, S. (1973). The structure of the Japanese language. Cambridge: MIT Press. Kuno, S. (1978). Japanese: A characteristic OV language. In W. P. Lehmann (Ed.), Syntactic typology (pp. 57–138). Austin: University of Texas Press. Kuroda, S.-Y. (1972). The categorical and thetic judgment: Evidence from Japanese. Foundations of Language, 9, 153–185. Lambrecht, K. (1994). Information structure and sentence form: Topic, focus and the mental. Cambridge: Cambridge University Press. Lee, H. S. (1991). Tense, aspect and modality. A discourse-pragmatic analysis of verbal affixes in Korean from a typological perspective. PhD thesis, University of California, Los Angeles. Lee, K. (1993). A Korean grammar on semantic-pragmatic principles. Seoul: Hankwuk Mwunhwasa. Lee, H. S. (1999). A discourse-pragmatic analysis of the committal -ci in Korean. Journal of Pragmatics, 20, 327–358. Lee, H. S. (2015). Modality. In L. Brown & J. Yeon (Eds.), The handbook of Korean linguistics (pp. 249–268). West Sussex: Wiley. Lehmann, C. (1995). Thoughts on Grammaticalization: A programmatic sketch. Munich: Lincom. Li, C. N., & Thompson, S. A. (1976). Subject and topic: A new typology of language. In C. N. Li (Ed.), Subject and topic (pp. 457–489). New York, NY: Academic. Maynard, S. K. (1987). Thematization as a staging device in the Japanese narrative. In J. Hinds, S. K. Maynard, & S. Iwasaki (Eds.), Perspectives on Topicalization, the case of Japanese wa (pp. 57–82). Amsterdam/Philadelphia, PA: Benjamins.

16

Performance in Knowledge Assessment Tests

235

Narrog, H. (2009). Modality in Japanese. The layered structure of the clause and hierarchies of functional categories. Amsterdam/Philadelphia, PA: Benjamins. Narrog, H. (2012). Modality, subjectivity, and semantic change. Oxford: Oxford University Press. Nedjalkov, V. P. (1995). Some typological parameters of converbs. In M. Haspelmath & E. König (Eds.), Converbs in cross-linguistic perspective: Structure and meaning of adverbial verb forms – Adverbial participles, gerunds (pp. 97–136). Berlin: De Gruyter Mouton. Nuyts, J., & van der Auwera, J. (Eds.). (2016). The Oxford handbook of Mood and modality. Oxford: Oxford University Press. Ono, T. (1990). Te, I and Ru clauses in Japanese recipes: A quantitative study. Studies in Language, 14(1), 73–92. Palmer, F. R. (1986). Mood and modality. Cambridge: Cambridge University Press. Prince, E. (1981). Toward a taxonomy of given-new information. In P. Cole (Ed.), Radical pragmatics (pp. 223–255). New York, NY/London: Academic. Sapir, E. (1921). Language. An introduction to the study of speech. New York, NY: Harcourt, Brace and Company. Shibatani, M. (1990). The languages of Japan. Cambridge: Cambridge University Press. Sohn, H.-M. (1994). Korean. London: Routledge. Sohn, H.-M. (1999). The Korean language. Cambridge: Cambridge University Press. Sohn, H.-M. (2009). The semantics of clause linking in Korean. In R. M. W. Dixon & A. Y. Aikhenvald (Eds.), The semantics of clause linking: A cross-linguistic typology (pp. 285–317). Oxford: Oxford University Press. Song, J. J. (2011). The Oxford handbook of linguistic typology. Oxford: Oxford University Press. Tamori, I. (1976). The semantics and syntax of the Japanese gerundive and infinitive conjunctions. Papers in Japanese Linguistics, 5, 307–360. Van der Auwera, J., & Plungian, V. A. (1998). Modality’s semantic map. Linguistic Typology, 2, 79–124. Van Valin Jr., R. D. (2004). Exploring the syntax-semantics Interface. Cambridge: Cambridge University Press. Velupillai, V. (2012). An introduction to linguistic typology. Amsterdam/Philadelphia, PA: John Benjamins. Walstad, W. B., Watts, M., & Rebeck, K. (2007). Test of understanding in college economics: Examiner's manual (4th ed.). New York, NY: National Council on Economic Education. Willett, T. L. (1988). Across-linguistic survey of the grammaticalization of evidentiality. Studies in Language, 12, 51–97. Zlatkin-Troitschanskaia, O., Förster, M., Brückner, S., & Happ, R. (2014). Insights from a German assessment of business and economics competence. In H. Coates (Ed.), Higher education learning outcomes assessment: International perspectives (pp. 175–197). Frankfurt am Main: Lang. https://doi.org/10.3726/978-3-653-04632-8 Zlatkin-Troitschanskaia, O., Brückner, S., Schmidt, S., & Förster, M. (2016a). Messung ökonomischen Fachwissens bei Studierenden in Deutschland und den USA – Eine mehrebenenanalytische Betrachtung der hochschulinstitutionellen und individuellen Einflussfaktoren. Unterrichtswissenschaft, 44(1), 73–88. https://doi.org/10.3262/UW1601073 Zlatkin-Troitschanskaia, O., Schmidt, S., Brückner, S., Förster, M., Yamaoka, M., & Asano, T. (2016b). Macroeconomic Knowledge of Higher Education Students in Germany and Japan – A Multilevel Analysis of Contextual and Personal Effects. Assessment & Evaluation in Higher Education, 41(5), 787–801. https://doi.org/10.1080/02602938.2016.1162279

Chapter 17

IRT Modeling of Decomposed Student Learning Patterns in Higher Education Economics Susanne Schmidt, Olga Zlatkin-Troitschanskaia, and William W. Walstad

Introduction Measuring change in student learning poses a challenge for educational measurement. This applies in particular to higher education, where, for practical reasons, multiple-choice (mc) tests are most commonly used and overall test scores and their differences between measurement points are used to assess the students’ change. In the first measurement, the pretest, initial knowledge is assessed. For each mc item, students select the one (correct) answer from the stated alternatives. The pretest score sums the number of correct responses to the test items. In the second measurement, the conclusion of the unit or course, students take the same mc test to measure their final knowledge. This posttest score sums the number of correct responses to the test items. To measure the change in knowledge for individual students, the pretest score is subtracted from the posttest score. This score is also referred to as a difference score. Based on this difference score, it is possible to evaluate to what extent students have improved or how much they have learned. In cases where the difference score is more moderate than large, the conclusion is that students did not learn as much as they were supposed to learn. However, this conclusion might be wrong, as students who already had a high initial knowledge level do not have the chance to improve greatly. Due to this and other factors, the difference score has often been criticized in the past (Cronbach and Furby 1970). This criticism is based predominantly on the argument that difference scores are merely regarded as total S. Schmidt (*) · O. Zlatkin-Troitschanskaia Department of Business and Economics Education, Johannes Gutenberg University Mainz, Mainz, Germany e-mail: [email protected]; [email protected] W. W. Walstad Department of Economics, University of Nebraska-Lincoln, Lincoln, NE, USA e-mail: [email protected] © Springer Nature Switzerland AG 2019 O. Zlatkin-Troitschanskaia (ed.), Frontiers and Advances in Positive Learning in the Age of InformaTiOn (PLATO), https://doi.org/10.1007/978-3-030-26578-6_17

237

238

S. Schmidt et al.

or composite measures without considering patterns in the responses to the test items. To avoid this, we followed the approach of Walstad and Wagner (2016) and decomposed the difference scores using four distinct patterns of learning (see also Walstad et al. in review). This decomposition represents an alternative approach to difference score for assessing changes in student learning. This study shows that when an mc test is given to economics students as a pretest to measure their prior knowledge of a subject before a course begins, and again at the end of the semester as a posttest, the students’ item responses across two test measurement points indicate distinguishable learning patterns that can then be differentiated after decomposing the aggregated difference scores. These patterns reflect two types of student learning outcomes: (1) Positive learning (PL) (giving incorrect answers in the pretest and correct answers in the posttest) and (2) Negative learning (NL) (correct pre and incorrect post).1 The decomposition of test scores demonstrates that the difference score is constructed from the two conflicting types of student learning (PL and NL) and enables an analysis of how aggregated test scores are influenced by these different types of learning. To date, PL and NL have only been modeled in accordance with classical test theory. To learn more about the properties of the newly developed approach of decomposing a test score from repeated measurements, we conducted an analysis based on probabilistic test theory. Four research questions (RQ) are investigated. RQ1: How reliable is the decomposition of the test score in PL and NL? RQ2: Are there differences in item difficulties and discriminations between the PL and NL models? RQ3: Do items with a quantitative reasoning (QR) component correspond more strongly to the pattern of NL while items without any QR demands correspond more strongly to a PL pattern? RQ4: Do items at lower cognitive levels correspond to PL while items at higher cognitive levels tend to correspond to NL? Section “Conceptual Framework” provides an overview of the theoretical framework for PL and NL, particularly from the perspective of learning. In Section “Test Instrument and Sample,” we describe the Test of Understanding of College Economics (TUCE), which was used to assess PL and NL. This section is followed by a description of the sample including some descriptive statistics. Sections “Latent Analysis and IRT Modeling of Student Learning Patterns with Macroeconomics Data” and “Latent Analysis and IRT Modeling of Student Learning Patterns with Microeconomics Data” present the results of the probabilistic models for PL and NL

1 Walstad and Wagner (2016) also distinguish two more learning patterns: retained and zero learning, which represent no learning at all. Retained learning means giving correct answers and zero learning means giving only incorrect answers in both the pretest and the posttest. Therefore, these two learning patterns will not be discussed in this paper.

17

IRT Modeling of Decomposed Student Learning Patterns

239

based on the investigating research questions RQ1 to RQ4. The results are summarized and discussed in Section “Discussion and Conclusion.”

Conceptual Framework We investigated PL and NL using probabilistic test theory, with data from students who were repeatedly assessed with regard to their content knowledge in the domain of economics. The first response pattern from matched pretest and posttest data is answering test items on the pretest incorrectly but correctly answering the same items on the posttest. This pattern indicates an improvement in student knowledge; an appropriate label for this outcome is thus positive learning (PL). In the second response set, students give correct answers to items on the pretest, but then give incorrect answers on the posttest. This response set indicates loss in understanding from pretest to posttest. In contrast to the previously described outcome, it therefore represents negative learning (NL). From a learning perspective, a difference score measures the total gain (incorrect to correct) (PL), which is after subtracting the total loss (correct to incorrect) (NL), or the “net” change in knowledge. In the case of PL, it is generally assumed that the students could not give the correct answer to an item due to a lack of knowledge before starting a course. This knowledge was acquired in the course so that students could give the correct answer to the item by the end of the course. The conceptual support for NL is less plausible than for PL as NL moves from knowing to not knowing. There could be three reasons for this learning outcome (see also ZlatkinTroitschanskaia et al. 2018): 1. Forgetting: If the questions in a test do not target (memorized) knowledge and definitions, but rather deductive reasoning in one domain or finding an answer to complex cases, as presented here in the TUCE, forgetting is unlikely. Knowledge, however, may not only be forgotten but it could become inert instead (e.g., Renkl 1994). 2. Guessing: Guessing is a common test strategy in students’ processing of mc tests (e.g., Brückner 2017; Walstad et al. 2018). When guessing is incorporated as a modeling parameter in the regression analysis with pretest and posttest variables, only a very small part of the variance in the test result can be explained by guessing (Walstad et al. 2018). Thus, NL cannot be substantially explained, even after accounting for guessing. 3. Acquisition of domain-specific (here economics) misconceptions: The cognitive interviews using the think-aloud method with the TUCE items show that students draw on incorrect domain-specific knowledge, or misconceptions, in solving some items (Brückner, 2017). This process leads to mistakes in reasoning and finding the solution for the tasks (see also Brückner and Pellegrino 2016). These misconceptions could be one of the significant reasons for NL. From a test-theory perspective, subtracting NL from PL simply reduces the amount of total knowledge improvement and the estimate of PL. Therefore,

240

S. Schmidt et al.

technically and conceptually, the difference between PL and NL is exactly equal to the difference of the posttest score minus the pretest score, which, by definition, represents the difference score. In contrast, PL alone is less confounded by other factors and assumptions than a difference score, and thus can serve as a more direct or unique measure of improvement in student knowledge.2 Therefore, we investigate PL and NL as a decomposed measure of change in economic knowledge.

Test Instrument and Sample The instrument selected for the learning and test score analysis in this study was the fourth edition of the Test of Understanding of College Economics (TUCE) (Walstad et al. 2007; Walstad and Rebeck 2008). The TUCE was developed to measure students’ knowledge about the principles of economics in higher education classrooms as well as in research studies on undergraduate students in the USA. The TUCE is a nationally normed and standardized mc test with the features of the test meeting the recommended standards for educational tests (AERA et al. 2014). It can therefore be used in both cross-sectional and longitudinal settings. Furthermore, the TUCE can be used for cross-national comparisons and has also been translated and adapted for testing and research in five other nations—Germany, Japan, South Korea, New Zealand, and the Philippines (e.g., Brückner et al. 2015; Förster et al. 2015; Yamaoka et al. 2010). The content domain for principles of economics is defined as a combination of macroeconomics and microeconomics. Accordingly, the TUCE covers these two subdomains, each of which comprises 30 mc items (Walstad et al. 2007). The microeconomic items focus on the economic problem, markets and prices, theory of the firm, factor markets, the role of government, and international microeconomics. The macroeconomics items assess the measurement of macroeconomic performance, aggregate supply and demand, money and financial markets, monetary and fiscal policies, policy applications, and international macroeconomics. Studies with the posttest data show that both parts are highly correlated, but separable and with sufficient reliabilities (microeconomics: α ¼ 0.70; macroeconomics: α ¼ 0.77) (e.g., Brückner et al. 2015). The TUCE items are based on a well-established taxonomy but modified into three cognitive categories (Bloom et al. 1956; Walstad et al. 2007). The first category assesses recognition and understanding of economic content. The largest percentage of the items (80%) consists of the second and third cognitive categories, respectively, and requires explicit or implicit application of economic knowledge to correctly answer the economics questions.

2 For technical details about the decomposition of the difference score, see Walstad and Wagner (2016).

17

IRT Modeling of Decomposed Student Learning Patterns

241

Table 17.1 Descriptive statistics: Macro and Micro TUCE

Variables Total Male Female White Non-white Business major Other major English speaker Non-English speaker Took college economics courses No college economics courses Took high school economics courses No high school economics courses

Macro sample Mean N pretest 2789 9.8 1651 10.2 1125 9.2 1973 10 804 9.3 1453 9.5 1309 10.1 2439 9.8 335 10 947 10.5

Mean posttest 14.2 14.8 13.4 14.8 12.8 13.9 14.5 14.1 14.7 15.4

Micro sample Mean N pretest 3255 9.4 1848 9.6 1384 9 2204 9.5 1015 9 1637 9.1 1549 9.7 2744 9.3 475 9.8 1276 9.7

Mean posttest 12.8 13.1 12.3 13.2 11.8 12.2 13.4 12.6 13.6 12.4

1835

13.6

1958

9.2

13

14.4

1415

9.7

13

14

1789

9.1

12.6

1107 1657

9.5 10 9.7

The data for the analysis comes from a longitudinal study. The panel sample for the macroeconomics part consists of 2789 students and the microeconomics panel sample comprises 3255 students (Walstad et al. 2007). The sample was pretested at the beginning of the fall semester and post-tested at the end of the same semester. The students were taught by 62 instructors at 44 US institutions of higher education (53% at master’s level in universities, 27% at doctoral-granting institutions, 14% at baccalaureate colleges, and 7% at associate’s colleges). Typically, the “principles of economics” course (macro or micro) is taken before many undergraduates declare a major or they take it as a requirement for other majors (e.g., business), which is why only 6% of the sample were majoring in economics. Table 17.1 provides more information on the sample.

Latent Analysis and IRT Modeling of Student Learning Patterns with Macroeconomics Data Prior research has studied PL and NL based on classical test theory (Walstad and Wagner 2016). To learn more about the properties of the newly developed approach to decomposing a test score from repeated measures, a latent analysis of PL and NL

242

S. Schmidt et al.

in IRT models was conducted with Mplus (Muthén and Muthén 1998–2012) using the WLSMV3 estimator. Table 17.2 presents the global fit criteria for the PL and the NL models with the macroeconomic data, indicating that both models fit the data well. The ratio of χ 2 to degrees of freedom (df) is below 1.5 in both cases, which indicates a good fit. The RMSEA for both PL and NL is not significant and below 0.05 (in addition, the 90% confidence interval (C.I.) is below 0.05), which also indicates a good fit. Although the CFI and TLI results show that neither PL or NL meet the benchmark of 0.95, indicating that the fit is insufficient, they can more or less be ignored in this case, as they are sensitive to large sample sizes, such as that used for this TUCE analysis. While the WRMR statistics show that only the NL model meets the benchmark of being below 1.0, Muthén and Muthén (1998–2012) suggest that this fit statistic is experimental and can be ignored as long as the other global fit criteria are met. Overall, we can assume that the model fits the data well. These finding provide initial support for RQ1 about whether the decomposition of test scores into PL and NL is reliable. To gain deeper insights into the overall model fit and identify any ill-fitting items, we analyze the local fit of the Confirmatory Factor Model by investigating PL and NL at the item level using the IRT parameterization. The reason for an imperfect fit might be due to specific items that do not or hardly contribute to the overall model. This analysis provides further insights that can help to answer RQ1 and investigate RQ2 with regard to possible differences in item discrimination and item difficulties within and between the PL and NL models. Table 17.3 supplies the results of the item discriminations αi and item difficulties βi. Table 17.2 Global model fit criteria of macro PL and NL CFA models (n ¼ 2789) Macro χ 2 (df) p value χ 2/df RMSEA (90% C.I.) p value CFI TLI WRMR

Positive learning 536.67 (405) 0.000 1.325 0.011 (0.008/0.013) 1.000 0.898 0.890 1.049

Negative learning 456.080 (405) 0.040 1.126 0.007 (0.002/0.010) 1.000 0.795 0.780 0.982

The Mplus WLSMV (short form of “robust weighted least squares“) estimator is used for confirmatory factor analysis (CFA) with categorical data. The parameterization is different from an IRT model; however, Mplus provides both CFA and IRT parameterization in case of one-dimensional models with dichotomous responses because in this case, both models are equivalent (Glöckner-Rist and Rist 2008). According to Brown (2006), the WLSMV estimator is the best choice here for various reasons. One reason is that with the WLSMV estimator the typical model evaluation criteria are reported (such as RMSEA). With a maximum likelihood estimation, which is the usual estimator for IRT models and which are not based on tetrachoric correlation, but on raw data, there are no such criteria. 3

17

IRT Modeling of Decomposed Student Learning Patterns

243

Table 17.3 IRT parameterization of TUCE macro items Macro Item i1 i2 i3 i4 i5 i6 i7 i8 i9 i10 i11 i12 i13 i14 i15 i16 i17 i18 i19 i20 i21 i22 i23 i24 i25 i26 i27 i28 i29 i30

Positive learning Discriminations αi p value 0.275 0.000 0.123 0.001 0.279 0.000 0.204 0.000 0.417 0.000 0.243 0.000 0.171 0.000 0.160 0.000 0.324 0.000 0.136 0.000 0.227 0.000 0.135 0.000 0.171 0.000 0.349 0.000 0.145 0.000 0.149 0.000 0.240 0.000 0.359 0.000 0.202 0.000 0.107 0.003 0.302 0.000 0.406 0.000 0.314 0.000 0.285 0.000 0.222 0.000 0.269 0.000 0.189 0.000 0.286 0.000 0.294 0.000 0.252 0.000

Difficulties βi 1.332 5.272 1.432 3.547 20.085 2.085 3.693 4.025 2.317 5.287 1.963 3.650 3.876 1.288 4.026 4.174 3.043 1.050 3.412 6.359 1.821 1.910 2.186 2.756 2.243 2.846 3.640 1.905 2.665 2.603

p value 0.000 0.001 0.000 0.000 0.170 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.003 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000

Negative learning Discriminations αi p value 0.179 0.013 0.155 0.005 0.298 0.000 0.096 0.064 0.182 0.030 0.210 0.000 0.142 0.007 0.192 0.000 0.057 0.272 0.131 0.010 0.341 0.000 0.127 0.014 0.322 0.000 0.136 0.013 0.188 0.001 0.068 0.189 0.144 0.004 0.232 0.000 0.171 0.001 0.405 0.000 0.126 0.041 0.160 0.013 0.120 0.022 0.116 0.025 0.328 0.000 0.100 0.057 0.203 0.001 0.163 0.003 0.140 0.011 0.256 0.000

Difficulties βi p value 8.825 0.011 6.884 0.004 4.131 0.000 11.135 0.063 10.058 0.025 4.603 0.000 6.559 0.006 4.864 0.000 19.893 0.271 7.066 0.009 4.365 0.000 8.041 0.013 2.911 0.000 9.118 0.012 5.283 0.000 14.321 0.188 6.458 0.003 6.012 0.000 5.443 0.001 2.681 0.000 11.030 0.038 8.511 0.012 8.455 0.021 8.681 0.024 3.315 0.000 10.618 0.055 6.517 0.001 6.576 0.002 8.031 0.010 4.014 0.000

One remarkable finding is that almost all parameters are significant (see p values in Table 17.3). Only one item difficulty in the PL model (i5) is not significant. Only two pairs of item discriminations and difficulties are not significant in the NL model (i9 and i16). The item difficulty of i5 in the PL model could be interpreted as this item having been too easy for this particular sample. Thus, i5 shows more PL, irrespective of the students’ abilities. The probability of not solving the item in the pretest, but solving the item in the posttest is highest for this item i5, even for students with lower ability levels. Item i5, however, can still differentiate between high and low performing students, as shown by its significant item discrimination

244

S. Schmidt et al.

parameter. Furthermore, i5 has the highest discrimination parameter in the PL model. At the same time, the item discrimination for i5 is not particularly high in NL. In contrast to the PL model, the item difficulty in i5 indicates that this is a very difficult item in the NL model. This finding suggests that i5 could differentiate less between high and low performing students. The probability of solving the item correctly in the pretest and giving a wrong answer in the posttest is very low, even for students with high ability levels. Similarly, i9 shows higher discriminations for PL, but low and non-significant discrimination in NL. At the same time, the item discriminations are relatively high in the PL model and in the NL model. This result indicates that i9 is better at differentiating between high and low performing students in macroeconomics for PL than for NL, while it is nonetheless relatively difficult to give a wrong answer in the pretest and a correct answer in the posttest. For NL, item i9 cannot differentiate well between high and low performing students and the difficulty level is the highest in the entire NL model. This result suggests that NL in i9 is very improbable NL. The item results also offer insights for RQ3. Mehler et al. (2018) conducted further linguistic and content analyses of TUCE items and found a significant correlation between linguistic and content-related characteristics, such as the amount of terminology in an item, and students’ performance. Furthermore, Brückner et al. (2015) and Shavelson et al. (2019) found a significant correlation between items with numerical content and students’ performance (according to RQ3 about whether or not the TUCE items require QR). Although some parts of the variance in overall test performance could be explained based on these types of analyses, none of these previous studies assessed different learning patterns such as PL and NL. Figure 17.1 presents i5 and i9. Both items are short, and only contain text and no pictures or other representational (e.g., mathematical) formats, and the content is relatively direct. By contrast, i20, presented in Fig. 17.2, has a relatively low discrimination parameter in PL and a higher discrimination parameter in NL. There are obvious

Fig. 17.1 High PL discriminations, but low and non/significant NL discriminations in macroeconomics items 5 and 9

17

IRT Modeling of Decomposed Student Learning Patterns

245

Fig. 17.2 Low PL discriminations but high NL discriminations in macroeconomics item 20

differences in the formulation and structure of this item and the response options. Item 20 is a longer question and contains numbers that requires students to think in terms of numerical operations. This feature makes the item less likely to show PL than NL. As for RQ4, according to the test developers (Walstad et al. 2007), the TUCE items can also be sorted into three cognitive categories based on the taxonomy of Bloom et al. (1956): (1) recognition and understanding; (2) explicit application; and (3) implicit application. To answer RQ4, it can be noted that items with high PL discriminations, but low and non-significant NL discriminations, belong in lower taxonomy levels, while items with low PL discriminations but high NL discriminations tend to be at higher taxonomy levels. Thus, easier items correspond more strongly with PL and more difficult items correspond more strongly with NL. Remarkably, this sorting based on PL and NL item discrimination and difficulty corresponds to the taxonomic classification of these items based on the original content review: Items i5 and i9 indeed belong to the lowest cognitive category of recognition and understanding, while item i20 belongs to the highest cognitive category of implicit application (Walstad et al. 2007, p. 5). Item i16, which also has a low discrimination in PL, but at the same time a low and non-significant discrimination in NL (with moderate to high difficulties), belongs to the middle cognitive category of explicit application (Walstad et al. 2007, p. 5). This item does not contribute much to either PL or NL. Therefore, i16 could be eliminated completely from both models, which would improve the model fit slightly (for PL: χ 2 ¼ 502.179 ( p ¼ 0.000); χ 2/df ¼ 1.33; RMSEA ¼ 0.011 ( p ¼ 1.000); CFI ¼ 0.901; TLI ¼ 0.893; WRMR ¼ 1.047 and for NL: χ 2 ¼ 423.746 ( p ¼ 0.048); χ 2/df ¼ 1.12; RMSEA ¼ 0.007 ( p ¼ 1.000); CFI ¼ 0.809; TLI ¼ 0.795; WRMR ¼ 0.978).4

4 There are also other items which do not contribute much to either the PL or the NL model (such as items i2 or i10), and the global fit criteria could be improved by eliminating them. However, as the focus of this study is on latent modeling of PL and NL as well as on identifying specific items that show remarkably high values for PL or NL, we will not go into further detail here.

246

S. Schmidt et al.

Latent Analysis and IRT Modeling of Student Learning Patterns with Microeconomics Data Similar results in the global fit evaluation as well as on the local level of the model (RQ1) were found in the analysis of the microeconomics data, but they are slightly worse overall (Table 17.4). However, the ratio of χ 2 and df is also below the benchmark of 2 here and is therefore an acceptable fit. In addition, the RMSEA statistic indicates a relatively good fit. The TLI, CFI, and WRMR again indicate an insufficient fit, but they are less useful fit measures than previously discussed. For the PL model, the CFI and TLI are higher than for the NL model, whereas the NL model shows better values in all other fit criteria. This difference might be due to several non-significant item parameters (RQ2), as shown in Table 17.5. The NL model shows nine pairs of non-significant item discriminations and difficulties (i1, i2, i4, i8, i10, i11, i19, i21, i26). The PL model only has three pairs of item discriminations and difficulties that are not significantly different from zero (i10, i19, and i22). This finding indicates that these items cannot differentiate between high and low performing learners for PL in microeconomics. Item i22, for example, has a low and non-significant discrimination for PL, but a higher and significant discrimination for NL. The difficulties are also different between PL and NL in item i22. This result indicates that in PL, high and low performing learners with a PL pattern have an equal probability of solving this item. It further indicates that the item is difficult for all participants, regardless of ability level. Conversely, when it comes to NL, the discrimination parameter indicates that high performing learners with a NL pattern have a higher probability of NL in this item than low performing learners with a NL pattern. A closer analysis of i22 reveals that it is a more complex item lingusitically and content-wise than, for instance, i5 and i9 (Fig. 17.3). The item stem is longer and also includes a salient representation in terms of a table and numerical representations with a mathematical task (RQ3). Compared to i5 and i9, which require verbal reasoning (VR) in economics, quantitative reasoning (QR) is required for this item (for the QR/VR model, see Shavelson et al. 2019). As previously described, preliminary studies of the TUCE data have shown that the QR items in the sample are often Table 17.4 Global model fit criteria of micro PL and NL CFA models (n ¼ 3255) Micro χ 2 (df) p value χ 2/df RMSEA (90% C.I.) p value CFI TLI WRMR

Positive learning 631.237 (405) 0.000 1.559 0.013 (0.011/0.015) 1.000 0.801 0.787 1.149

Negative learning 486.935 (405) 0.003 1.202 0.008 (0.005/0.010) 1.000 0.641 0.614 1.021

17

IRT Modeling of Decomposed Student Learning Patterns

247

Table 17.5 IRT parameterization of TUCE micro items Micro Item i1 i2 i3 i4 i5 i6 i7 i8 i9 i10 i11 i12 i13 i14 i15 i16 i17 i18 i19 i20 i21 i22 i23 i24 i25 i26 i27 i28 i29 i30

Positive learning Discriminations αi p value 0.143 0.000 0.165 0.000 0.202 0.000 0.336 0.000 0.151 0.000 0.427 0.000 0.158 0.000 0.326 0.000 0.253 0.000 0.047 0.183 0.501 0.000 0.267 0.000 0.134 0.000 0.338 0.000 0.289 0.000 0.100 0.005 0.302 0.000 0.147 0.000 0.019 0.604 0.356 0.000 0.107 0.003 0.047 0.185 0.163 0.000 0.120 0.001 0.214 0.000 0.147 0.000 0.227 0.000 0.274 0.000 0.116 0.002 0.158 0.000

Difficulties βi p value 4.603 0.000 4.608 0.000 2.763 0.000 0.113 0.106 4.355 0.000 1.282 0.000 5.226 0.000 2.428 0.000 3.116 0.000 15.420 0.182 1.418 0.000 1.674 0.000 4.132 0.000 1.547 0.000 2.725 0.000 6.769 0.005 2.168 0.000 4.413 0.000 42.168 0.603 2.040 0.000 7.084 0.003 15.102 0.184 5.055 0.000 5.189 0.001 3.715 0.000 5.084 0.000 2.931 0.000 2.648 0.000 6.538 0.001 4.148 0.000

Negative learning Discriminations αi p value 0.079 0.136 20.023 0.664 0.163 0.003 0.066 0.263 0.152 0.002 0.295 0.000 0.132 0.010 0.044 0.517 0.100 0.060 0.058 0.265 0.106 0.118 0.180 0.001 0.086 0.098 0.177 0.001 0.265 0.000 0.128 0.010 0.281 0.000 0.139 0.011 0.065 0.164 0.164 0.003 0.078 0.109 0.247 0.000 0.215 0.000 0.284 0.000 0.235 0.000 0.041 0.402 0.122 0.022 0.267 0.000 0.196 0.000 0.267 0.000

Difficulties βi 13.595 243.062 6.543 23.588 5.586 4.888 7.484 34.795 11.169 17.074 15.109 6.510 11.898 5.671 4.908 7.160 3.839 7.581 12.614 7.549 10.596 3.504 5.223 3.291 5.183 22.084 8.863 4.247 5.114 3.675

p value 0.134 0.664 0.002 0.261 0.002 0.000 0.009 0.516 0.058 0.264 0.114 0.001 0.096 0.000 0.000 0.009 0.000 0.010 0.163 0.003 0.108 0.000 0.000 0.000 0.000 0.401 0.020 0.000 0.000 0.000

more difficult compared to the VR items (Brückner et al. 2015). Accordingly, the cogntive taxonomy item 22 belongs to is the highest category, namely implicit application (Walstad et al. 2007, p. 4). The investigation of item parameters in the micro TUCE also reveals that there are two items with higher discriminations in the PL model, but low and non-signficant discriminations in the NL model (i4 and i11; Table 17.5). These items are similar to i5 and i9 in Fig. 17.1; the text is short and there are no other numerical oder image-based representation formats (Fig. 17.3).

248

S. Schmidt et al.

Fig. 17.3 Low or non-significant PL discriminations, but higher NL discriminations in microeconomics item 22

Fig. 17.4 High PL discriminations, but low or non-significant NL discriminations in microeconomics items 4 and 11

Item i4 falls in the lowest cognitive category of recognition and understanding and item i11 belongs to the middle category, explicit application (Walstad et al. 2007, p. 4) (RQ4). This could be the reason why i4 has a low level of difficulty and is not significant in PL, whereas i11 has a high level of difficulty and is significant (items 4 and 11; Fig. 17.4). These items are similar to items 5 and 9 in Fig. 17.1:

17

IRT Modeling of Decomposed Student Learning Patterns

249

The text is short and there are no other numerical oder image-based representation formats (Figs. 17.1 and 17.4). This finding indicates that the cognitive ability level of participants needs to be higher for item i11 than for i4 in order for students to have the same likelihood of solving both items.

Discussion and Conclusion The latent analysis and IRT modeling of the two student learning patterns in economics (PL and NL) provide relatively good fit statistics and supply item results that can be interpreted in a meaningful way. These overall results align with the findings of latent analyses with aggregated test scores (Walstad et al. in review). The analysis also offers insights with which to address RQ1 and RQ2. Regarding RQ1, a decomposition of test scores into PL and NL appears to be a reliable measures for analyzing different learning patterns over time. The IRT modeling and analysis also enable identification of items that correspond more closely to PL or NL. For example, items with weak discrimination parameters in PL also have higher discriminations in NL, and vice versa. This result indicates that we can confirm our assumption of differences between PL and NL item paramaters for RQ2. For some items, we found remarkable differences between PL and NL discrimination parameters, which shows that some items support PL while others correspond more strongly to NL (e.g., i22 in Fig. 17.3). This item is difficut in terms of the probability to give a correct answer in the posttest when the answer in the pretest was incorrect (which is defined as PL). Conversely, the probability of giving an incorrect answer to this item in the posttest if the answer in the pretest was correct (which is defined as NL) is not low, even for participants with lower ability scores in NL. Students probably correctly guessed the answer to i22 on the pretest, but acquired some misconceptions about the item content during the course or did not learn the content. Finding an explanation as to why NL actually occurs requires more in-depth analyses of the curriculum as well as of instructional and learning practice to explore what students actually learn when faced with such content. One next step towards explaining PL and NL would be to include different predictor variables in our latent models. Such potential influence factors of PL and NL can also be derived from our initial results regarding RQ3. Our findings indicate that items with QR demands correspond more closely to NL than items without QR demands. Thus, students’ QR abilities could have a significant influence on PL and NL. Future studies should assess such generic yet domain-related skills such as QR as well as verbal and spatial reasoning as separate constructs and examine their correlation to PL and NL. This analysis could not only provide important insights into explaining students’ learning patterns, but might also become the foundation for domain-specific educational optimization processes with which to effectively foster PL.

250

S. Schmidt et al.

The results of RQ4 confirm our assumption that items at higher cognitive levels tend to correspond with NL and items at lower cognitive levels with PL. To further explore these findings, in-depth analyses of students’ cognitive processes during item testing are necessary. From an educational perspective, this finding again indicates learning difficulties when it comes to items that go beyond mere rote understanding and require more complex reasoning. This result could be an indication that such skills must be fostered more intensively and more effectively in higher education economics. In conclusion, learning scores can be used to analyze tests and items as well as to investigate factors influencing students’ change in regression analyses. The PL score is an alternative measure of growth that avoids the problems associated with difference scores and simultaneously allows for a more precise measurement of change in student learning over the course of studies. Thus, additional personal parameters of students (e.g., gender, nationality, prior knowledge) and of the instututions (e.g., target curricula, instruction) should be included in the explanatory models. For this purpose, the IRT parametrization provides an adequate framework that should be extended to a MIMIC (multiple indicators and multiple causes) model (Jöreskog and Goldberger 1975; Brown 2006, pp. 304ff.). By doing so, we can evaluate the results not only at the item-level, but also at the student-level as it will be possible to identify which students have the highest probability of PL (for an item-student-model, see Mislevy 2016; Brückner and Pellegrino 2016). To ensure the external validity of the findings, the modeling should also be tested with the other samples of students in different nations.

References American Education Research Association (AERA), American Psychological Association (APA), & National Council on Measurement in Education (NCME). (2014). Standards for educational and psychological testing. Washington, DC: American Education Research Association. Bloom, B. S., Engelhart, M. D., Furst, E. J., Hill, W. H., & Krathwohl, D. R. (1956). Taxonomy of educational objectives. Handbook 1: Cognitive domain (Vol. 19). New York, NY: David McKay. Brown, T. A. (2006). Confirmatory factor analysis for applied research. New York, NY: The Guildford Press. Brückner, S. (2017). Prozessbezogene Validierung anhand von mentalen Operationen bei der Bearbeitung wirtschaftswissenschaftlicher Testaufgaben. Landau: Empirische Pädagogik. Brückner, S., & Pellegrino, J. W. (2016). Integrating the analysis of mental operations into multilevel models to validate an assessment of higher education students’ competency in business and economics. Journal of Educational Measurement, 53(3), 293–312. Brückner, S., Förster, M., Zlatkin-Troitschanskaia, O., Happ, R., Walstad, W. B., Yamaoka, M., et al. (2015). Gender effects in assessment of economic knowledge and understanding: Differences among undergraduate business and economics students in Germany, Japan, and the United States. Peabody Journal of Education, 90(4), 503–518. Cronbach, L. J., & Furby, L. (1970). How we should measure "change" – Or should we? Psychological Bulletin, 74(1), 68–80.

17

IRT Modeling of Decomposed Student Learning Patterns

251

Förster, M., Zlatkin-Troitschanskaia, O., Brückner, S., Hambleton, R. K., Walstad, W. B., Yamaoka, M., et al. (2015). Validating test score interpretations by cross-national comparison: Comparing the results of students from Japan and Germany on an American test of economic knowledge in higher education. Zeitschrift für Psychologie, 223(1), 14–23. Glöckner-Rist, A., & Rist, F. (2008). Modelle mit latenten Variablen für dichotome und polytome Indikatoren. In J. Reinecke & C. Tarnai (Eds.), Klassifikationsanalysen in Theorie und Praxis (pp. 117–139). Münster: Waxmann. Jöreskog, K. G., & Goldberger, A. S. (1975). Estimation of a model with multiple indicators and multiple causes of a single latent variable. Journal of the American Statistical Association, 70(351a), 631–639. Mehler, A., Zlatkin-Troitschanskaia, O., Hemati, W., Molerov, D., Lücking, A., & Schmidt, S. (2018). Integrating computational linguistic analysis of multilingual learning data and educational measurement approaches to explore student learning in higher education. In O. Zlatkin-Troitschanskaia, G. Wittum, & A. Dengel (Eds.), Positive learning in the age of information (PLATO) – A blessing or a curse? (pp. 145–196). Wiesbaden: Springer. Mislevy, R. J. (2016). How developments in psychology and technology challenge validity argumentation. Journal of Educational Measurement, 53(3), 265–292. Muthén, L. K., & Muthén, B. O. (1998–2012). Mplus user’s guide (7th ed.). Los Angeles: Muthén & Muthén. Renkl, A. (1994). Träges Wissen: Die "unerklärliche" Kluft zwischen Wissen und Handeln (Forschungsbericht Nr. 41). LMU München: Lehrstuhl für Empirische Pädagogik und Pädagogische Psychologie. Shavelson, R. J., Marino, J., Zlatkin-Troitschanskaia, O., & Schmidt, S. (2019). Reflections on the assessment of quantitative reasoning. In B. L. Madison & L. A. Steen (Eds.), Calculation vs. context: Quantitative literacy and its implications for teacher education (2nd ed.). Washington, DC: Mathematical Association of America. Walstad, W. B., & Rebeck, K. (2008). Test of understanding of college economics. American Economic Review, 98(2), 547–551. Walstad, W. B., & Wagner, J. (2016). The disaggregation of value-added test scores to assess learning outcomes in economic courses. The Journal of Economic Education, 47(2), 121–131. Walstad, W. B., Watts, M., & Rebeck, K. (2007). Test of understanding in college economics: Examiner’s manual (4th ed.). New York, NY: National Council on Economic Education. Walstad, W. B., Schmidt, S., & Zlatkin-Troitschanskaia, O. (in review). Alternative measures of change in student learning when using multiple-choice tests. Walstad, W. B., Schmidt, S., Zlatkin-Troitschanskaia, O., & Happ, R. (2018). Pretest posttest measurement of the economic knowledge of undergraduates – Estimating guessing effects. Paper presented at the Allied Social Science Association annual meeting. Philadelphia, PA: AEA. Yamaoka, M., Walstad, W. B., Watts, M. W., Asano, T., & Abe, S. (Eds.). (2010). Comparative studies on economic education in the Asia-Pacific region. Tokyo: Shumpusha. Zlatkin-Troitschanskaia, O., Schmidt, S., Molerov, D., Shavelson, R. J., & Berliner, D. (2018). Conceptual fundamentals for a theoretical and empirical framework of positive learning. In O. Zlatkin-Troitschanskaia, G. Wittum, & A. Dengel (Eds.), Positive learning in the age of information – A blessing or a curse? (pp. 29–52). Wiesbaden: Springer.

Chapter 18

Assessing Mathematics Knowledge and Skill: What College Students Actually Know and Can Do? Marta K. Mielicki, Mara V. Martinez, Louis V. DiBello, Alexa W. C. Lee-Hassan, and James W. Pellegrino

Introduction and Background The PLATO Project is concerned with identifying and understanding the conditions that promote positive learning in postsecondary education and is engaged in multiple inquiries about what students know and can do in various programs of study (Zlatkin-Troitschanskaia et al. 2018). Among the project’s domains of inquiry is mathematics that may be essential for success in various academic and technical degree programs. In the USA there has been long standing concern about the mathematics knowledge and skill that is prerequisite for academic success in a variety of degree programs in both 2-year and 4-year higher-education institutions. For example, the need for postsecondary remediation in mathematics has been a subject of ongoing debate among policymakers and educators for a long time (Martorell and McFarlin 2010). Every year, a sizeable percentage of new college students arrive on campuses in the USA without the academic skills necessary for

Research in this paper is based upon work supported by the National Science Foundation under grant number DRL-1316736. Any opinions, findings, and conclusions or recommendations expressed in this paper are those of the authors and do not necessarily reflect the views of the National Science Foundation. M. K. Mielicki Department of Psychology, University of Illinois at Chicago, Chicago, IL, USA e-mail: [email protected] M. V. Martinez Department of Mathematics, Statistics, and Computer Science, University of Illinois at Chicago, Chicago, IL, USA e-mail: [email protected] L. V. DiBello · A. W. C. Lee-Hassan · J. W. Pellegrino (*) Learning Sciences Research Institute, University of Illinois at Chicago, Chicago, IL, USA e-mail: [email protected]; [email protected] © Springer Nature Switzerland AG 2019 O. Zlatkin-Troitschanskaia (ed.), Frontiers and Advances in Positive Learning in the Age of InformaTiOn (PLATO), https://doi.org/10.1007/978-3-030-26578-6_18

253

254

M. K. Mielicki et al.

college-level work (Chen 2016). For instance, Greene and Winters (2005) found that two-thirds of public school students in the 2002 high-school graduating class were not ready for college-level work. In this context, the Common Core State Standards Initiative (2010) in the USA set expectations for what students should know at different grade levels in the areas of English language arts and mathematics with the ultimate goal of ensuring college and career readiness. However, students’ post-school lives will vary, and the required content knowledge for success may vary as well. The question of what mathematics knowledge and skill is required for college and career readiness has been addressed by several researchers. For example, Conley et al. (2011) tackled this question by asking a broad sample of college instructors in a variety of disciplines and from a range of higher-education institutions, including two- and four-year colleges, about the applicability and importance of the high-school-level Common Core State Standards in Mathematics (CCSSM). Conley et al. (2011) found that Algebra and Functions standards were among the standards rated most applicable by instructors, particularly in math and science domains. The algebra standards pertaining to interpreting expressions, creating equations, and solving linear equations in one variable were rated as most important, as were the Functions standards pertaining to modeling the relationship between quantities, interpreting graphs and tables, graphing symbolic functions, writing a function, and using function notation. The National Center on Education and the Economy (NCEE 2013) took a different approach in their attempt to understand which CCSSM are required for success in courses and programs at the two-year community college level. Rather than asking instructors about what they perceived to be relevant content, the NCEE engaged in a systematic analysis of textbooks, exams, and assignments from introductory courses for a variety of programs, including the required math courses for those programs. Their results indicate that, for the majority of programs at the community college level, the most relevant standards were from the 6th–8th grade band and pertained to ratios, proportionality, expressions, and simple equations. Both the Conley et al. (2011) and NCEE (2013) studies provide information about what postsecondary students should know in mathematics and how that might vary as a function of program of study. Together the two studies identify middleschool algebra and pre-algebra topics as the set of mathematical knowledge and skills that are relevant for success in typical higher education courses and programs of study. However, neither study was designed to actually assess the level at which students have mastered the content reflected in either the middle-school or highschool algebra standards from the CCSSM. We set out to construct an assessment to examine the extent to which undergraduates in a large, research-intensive university in the Midwestern United States had mastered the standards that were identified as pertinent by the aforementioned studies (Conley et al. 2011; NCEE 2013). We analyzed performance on a broad set of tasks according to whether: (1) students were self-identified as Science, Technology, Engineering, and Mathematics (STEM) majors versus those who were non-STEM majors; (2) the general mathematical strands the tasks represented

18

Assessing Mathematics Knowledge and Skill

255

(i.e., groupings of standards according to algebra content sub-domains); and (3) the types of representations involved (e.g., tabular, algebraic, graphical, etc.). Specifically, we analyzed the extent to which students pursuing STEM- and non-STEMrelated career paths in a 4-year public university have mastered Algebra and Functions standards pertaining to interpreting expressions (and parts of expressions), creating equations, solving linear equations in one variable, modeling the relationship between quantities, interpreting graphs and tables, graphing symbolic functions, writing a function, and using function notation. Since these standards were judged to be highly relevant by math and science instructors specifically, these standards may be most relevant and show higher levels of mastery for students pursuing STEM-related career paths. In addition, based on the standards that were identified as highly relevant for success in community college level courses by the NCEE (2013), we included questions in the assessment to determine whether students have mastered algebrarelated standards in the 6th-8th grade band pertaining to ratios, proportionality, expressions, and simple equations. These standards should be relevant for students pursuing both STEM- and non-STEM-related career paths. We were concerned with the overall level of performance exhibited by each group of students for each of the content strands as well as the types of tasks within each content strand that appeared relatively easy versus those that were more challenging and what that might imply about areas of knowledge and skill where students might need additional instructional support and remediation.

Method Participants A total of 363 undergraduates at a large midwestern, research-intensive public university in the USA participated in the study for partial credit in an introductory psychology course. A total of 5 participants were excluded from the analysis, resulting in a final sample size of 358. Three participants did not complete nine or more items, and two students had total scores lower than chance. Of the final sample, 61% was female. Participants reported all of the math courses they had taken during high school. Eighty-two percent reported taking Algebra 1, 89% reported taking Algebra 2, 90% reported taking Geometry, 84% reported taking Pre-calculus/ Trigonometry, 11% reported taking non-AP calculus, and 25% reported taking AP Calculus. Fifteen percent reported taking other math courses including Statistics and International Baccalaureate Math courses. Twenty percent of participants completed some of their K-12 schooling outside of the USA. Participants were classified as pursuing a STEM-related career path if they reported majoring in science (i.e., biology, chemistry, physics, and neuroscience), engineering, computer programming, information technology, or mathematics. Based on this classification, 35% of the participants reported pursuing a STEM-related career path. Twenty-four

256

M. K. Mielicki et al.

Table 18.1 Math-related demographic data Math ACT score (out of 36) Math SAT score (out of 800) Number of math courses taken in college Grade in math courses (out of 4.0) Math confidence (out of 6)

N 328 42 355 212 353

M (SD) 24.68 (4.56) 627.12 (121.12) 1.04 (1.02) 2.62 (1.07) 3.27 (0.07)

Range 15–36 300–800 0–7 0.0–4.0 2–5

Table 18.2 Proportion of participant reported parental education levels and annual income Less than high school High school Professional training Some college College Some graduate school Masters Ph.D./M.D./J.D. Under $45,000 $45,000–$50,000 $50,000–$60,000 Over $60,000

Mother’s level of education 0.14 0.25 0.02 0.23 0.24 0.02 0.09 0.01 Reported parents’ annual income 0.33 0.22 0.15 0.30

Father’s level of education 0.16 0.28 0.02 0.14 0.21 0.04 0.11 0.05

percent of participants either had taken or were taking math courses that are considered remedial by the university. Additional demographic data for the participants are presented in Tables 18.1 and 18.2.

Materials and Procedure Following Conley et al. (2011) and NCEE (2013), we decided to include algebra and pre-algebra middle-school topics as the mathematical content of the assessment. For mathematical internal coherence, we grouped CCSSM into three strands: Variables and Patterns, Linear Equations, and Linear Functions. Thus, an assessment was developed that is composed of 136 multiple-choice items in three content strands: Variables and Patterns (VP—49 items), Linear Equations (LE—43 items), and Linear Functions (LF—44 items). The VP strand measured relations, variables, patterns, ratios, and rates and was not restricted to linear relationships. The LE strand emphasized solving linear equations and algebraic manipulation. The LF strand included the concept of a linear function as a relationship between two quantities. Assessment items were selected from various sources: 27 items from practice tests for the College Board’s SAT examination, 19 items from practice tests

18

Assessing Mathematics Knowledge and Skill

257

for the ACT examination, 8 items from publicly disclosed Partnership for Readiness for College and Careers items (PARCC), 59 items from National Assessment of Educational Progress (NAEP), and 23 items from Third International Mathematics and Science Study (TIMSS). The assessment items were mapped to relevant middleschool and high-school-level Common Core Mathematics standards. There were 26 standards represented across items in the LE strand, 20 standards in the LF strand, and 21 standards in the VP strand. Assessment items were classified as pertaining to the high-school (HS) grade band if they mapped to any HS standards.1 The assessment was presented to each student in a booklet constructed within a balanced incomplete block (BIB) design. The assessment items were divided into 4 blocks per strand, and booklets were then constructed containing a block from each strand. There were 12 booklets total, ranging from 33 to 35 items per booklet. Each student responded to the items in one randomly assigned booklet. This BIB design ensured that sufficient data could be collected for each item without having to administer every item to every student. The overlap of items across booklets made it possible to model all items and all students on the same underlying measurement scale using a 2PL IRT analysis (Baker 2001; Crocker and Algina 2006). The booklets were administered via computer in a lab setting using the ASSISTments platform (https://www.assistments.org/).

Results Means for performance on the three content strands are presented in Fig. 18.1. As can be seen in Fig. 18.1, overall performance on the items within each strand was relatively good with means above 0.75 for both the Variables and Patterns and Linear Equations strands. Performance was lower for the Linear Functions strand with a mean of 0.65. There was considerable variability in performance on individual tasks within each of the content strands. Across content strands, performance on items pertaining to 6th–8th grade standards (M ¼ 0.78, SD ¼ 0.17) was higher than for items pertaining to high-school standards (M ¼ 0.69, SD ¼ 0.21). As shown in Table 18.3 this effect was driven by a difference in the LE and LF strands and not the VP strand. The weakest performance was shown on the Linear Functions items, all of which mapped against high-school standards. Not surprisingly, independentsamples t-tests revealed that students pursuing a STEM-related career path have mastered more mathematical content pertaining to all three content standards than their counterparts who are not pursuing STEM careers (Table 18.4). However, even non-STEM majors appear to have a good grasp of content pertaining to the VP strand considered essential for success in two-year college programs of study, whereas they appear to have a much weaker understanding of the content in the

1 A description of each item and its mapping to the CCSSM standards can be obtained from the authors in addition to item performance data.

258

M. K. Mielicki et al.

Fig. 18.1 Mean proportion correct for individual items within each content strand. Note: Error bars represent standard deviation Table 18.3 Proportion correct for items by content strand and grade band Strand Variables and patterns Linear equations Linear functions

6th–8th grade standards N M (SD) Range 30 0.76 (0.19) 0.22–0.99 13 0.83 (0.11) 0.60–0.98 0 – –

High-school standards N M (SD) Range 18 0.76 (0.13) 0.49–0.99 30 0.70 (0.20) 0.14–0.93 44 0.65 (0.23) 0.07–0.93

Table 18.4 Proportion correct for items by major and strand Strand Variables and patterns Linear equations Linear functions

STEM M (SD) 0.80 (0.16) 0.81 (0.18) 0.71 (0.21)

Range 0.33–1.00 0.27–1.00 0.00–1.00

Non-STEM M (SD) 0.74 (0.16) 0.70 (0.21) 0.62 (0.20)

Range 0.25–1.00 0.09–1.00 0.09–1.00

t-test 3.48** 4.71** 4.13**

**p < 0.01

LF strand. According to NCEE (2013), mastery of the mathematical content represented by the LF strand may be less important for non-STEM majors. A 2-parameter logistic (2PL) item response theory (IRT) model with missing data was used to provide estimated difficulty and discrimination parameters for each item and an ability estimate, theta, for each examinee (Baker 2001; Crocker and Algina 2006). Estimated ability scores ranged between 1.94 and STEM majors had a higher average ability estimate consistent with the results presented in Table 18.4 based on item p values. A comparison of item difficulty based on the IRT analysis did not reveal significant differences based on content strand, F(2, 132) ¼ 1.28, p ¼ 0.28, or standards grade band, F(1, 133) ¼ 2.17, p ¼ 0.14. The failure to find overall differences at these general levels of item classification is not surprising

18

Assessing Mathematics Knowledge and Skill

259

given the high degrees of variability in task difficulty shown within and across the content strands and grade bands. While a detailed description of performance variation within each content strand is beyond the current presentation, we were concerned with characterizing the types of mathematical knowledge and skill associated with higher versus lower degrees of accuracy. For purposes of qualitative analyses of performance on low- and highdifficulty items, we selected a subset of 12 items. Six items were selected from the set of all low-accuracy items (proportion correct between 0.6 and 0.7), and a second set of six items were selected from the high-accuracy items (proportion correct between 0.9 and 1.0). The six high-accuracy items used in the qualitative analysis are shown in Table 18.5 and the six low-accuracy items are shown in Table 18.6. Within each of these accuracy ranges, two items per strand were selected if they instantiated multiple standards. Each of the final 12 items was linked to between 2 and 6 standards. All of the standards represented in the items used for qualitative analysis are listed in Table 18.7. The qualitative analysis of specific sample items provided an opportunity for a finer-grained determination of the mathematical skills and knowledge that predict high and low performance by college students, especially in light of the high variability in item performance within each strand and the lack of substantial overall differences between strands. As can be seen by examining the specific sample items, we concluded that college students are successful at translating between verbal and symbolic representation to construct expressions with one variable (VP01, LF13), expressions with two variables (VP04), and equations with one variable (LE28, LF08). However, this ability to translate between verbal and symbolic representations appears to break down in the case where a variable appears on two sides of the equal sign (LE19). Students can construct equations involving ratios and solve them when the position of the unknown is explicit (LE29); however, they struggle with ratios more generally (VP15). Students have trouble translating between tabular and symbolic representation (LF33, LF40). Students also have not fully mastered the concepts associated with the Cartesian plane (LE13), and struggle with translating between verbal and graphical representation (VP16). It is worth noting that of the 23 standards represented by this subset of items, only 4 standards were unique to high-accuracy items and only 9 standards were unique to low-accuracy items. This suggests that there was much overlap in the standards associated with high- and low-accuracy items. Performance depends less on the specific standard(s) represented in an item and more on the way in which that standard is tapped by the task and in relationship to other standards. These results relating item performance to specific standards are suggestive of the specific areas of strength and weakness that characterize the mathematical proficiency of typical students in the early stages of their college careers. But additional work would be needed to develop a comprehensive assessment program that was more diagnostic with respect to particular patterns of strengths and weaknesses. In addition, such a system would need to be used with students from a range of higher-education institutions to establish absolute and relative levels of performance and specific patterns of strengths and weaknesses.

260

M. K. Mielicki et al.

Table 18.5 High-accuracy items for each strand Strand Variables and patterns

Linear equations

Linear functions

Item VP01. On a car trip Sam drove m miles, Kara drove twice as many miles as Sam, and Darin drove 20 fewer miles than Kara. In terms of m, how many miles did Darin drive? a. 2m + 20 b. 2m  20 c. m/2 + 20 d. (m + 20)/2 e. m/2  20 VP04. If notebooks cost $2 each and backpacks cost $32 each, which of the following represents the cost, in dollars, of n notebooks and b backpacks? a. 16nb b. 34nb c. 34(n + b) d. 2n + 32b e. 2(n + 32b) LE28. n is a number. When n is multiplied by 7, and 6 is then added, the result is 41. Which of these equations represents this relation? a. 7n + 6 ¼ 41 b. 7n  6 ¼ 41 c. 7n  6 ¼ 41 d. 7(n + 6) ¼ 41 LE29. If the ratio 7 to 13 is the same as the ratio x to 52, then what is the value of x? a. 7 b. 13 c. 28 d. 364 LF08. The cost, C, of printing greeting cards consists of a fixed charge of 100 cents and a charge of 6 cents for each card printed. Which of these equations can be used to determine the cost of printing n cards? a. C ¼ (100 + 6n) cents b. C ¼ (106 + n) cents c. C ¼ (6 + 100n) cents d. C ¼ (106n) cents e. C ¼ (600n) cents LF13. A car can seat c adults. A van can seat 4 more than twice as many adults as the car can. In terms of c, how many adults can the van seat? a. c + 8 b. c + 12 c. 2c – 4 d. 2c + 4 e. 4c + 2

18

Assessing Mathematics Knowledge and Skill

261

Table 18.6 Low-accuracy items for each strand Strand Variables and patterns

Item VP16. Two hoses are used to fill the pool. Twice as many gallons of water per minute flow through one of the hoses as through the other. Both hoses had been on for 12 h and had filled the pool to the 4-foot mark then the hose with the slower flow then finished filling the pool to the 5-foot mark. Which of the following graphs shows the relationship between the time spend filling the pool and the height of the water in the pool? a

height of water (in feet)

height of water (in feet)

d

time (in hours) b

time (in hours) height of water (in feet)

height of water (in feet)

e

time (in hours)

time (in hours)

height of water (in feet)

c

time (in hours)

Linear equations

VP15. In a certain string ensemble, the ratio of men to women is 5:3. If there are a total of 24 people in the ensemble, how many women are there? a. 12 b. 11 c. 10 d. 9 e. 8 LE13. At which y-coordinate does the line described by the equation 6y  3x ¼ 18 intersect the y-axis? a. 18 b. 9 c. 6 d. 3 e. 2 LE19. Brandy has a collection of comic books. If she adds 15 to the number of comic books in her collection and multiplies the sum by 3, the result will be 65 less than 4 times the number of comic books in her collection. How many comic books are in her collection? a. 50 b. 85 c. 110 d. 145 e. 175 (continued)

262

M. K. Mielicki et al.

Table 18.6 (continued) Strand Linear functions

Item LF33. As a class experiment, a cart was rolled at a constant rate along a straight line. In the chart below, Shawn recorded the cart’s distance, x, in feet from a reference point at the start of the experiment and for each of 5 times in t seconds. Which of the following equations represents the data? t

0

1

2

3

4

5

x

10

14

18

22

26

30

a. x ¼ t + 10 b. x ¼ 4t + 6 c. x ¼ 4t + 10 d. x ¼ 10t + 4 e. x ¼ 14t LF40. The table below gives values of the function f for several values of t. If the graph of f is a line, which of the following defines f(t)? t

0

1

2

3

f (t)

-1

1

3

5

a. b. c. d. e.

f(t) ¼ t  1 f(t) ¼ t + 1 f(t) ¼ 2t + 1 f(t) ¼ 2t – 1 f(t) ¼ 1  2t

Discussion This study explored the extent to which students at a 4-year university pursuing either STEM or non-STEM related career paths demonstrate mastery of the mathematics knowledge and skills associated with three algebra content strands: Variables and Patterns, Linear Equations, and Linear Functions. As expected, STEM majors outperformed non-STEM majors across all content strands, and had higher average ability estimates based on a 2PL IRT model fit to the data. However, non-STEM majors demonstrated an adequate understanding of variables, patterns, ratios, and rates, which, according to NCEE (2013), may be sufficient mathematical content for success in programs of study associated with many diverse college majors. The qualitative analysis of specific items associated with high and low performance across both STEM and non-STEM majors revealed students’ strengths and weaknesses with respect to specific content. Students were successful at translating between verbal and symbolic representation when expressions or equations had one variable and when expressions had two different variables. However, students were less successful at translating between verbal and symbolic representations in the case where a variable appeared on two sides of the equal sign. This is consistent with other work demonstrating that students struggle with algebra problems where the

18

Assessing Mathematics Knowledge and Skill

263

Table 18.7 CCSSM for items used for qualitative analysis by Strand Strand Variables and patterns

Linear equations

Linear functions

CCSSM • Use variables to represent numbers and write expressions when solving a real-world or mathematical problem • Understand the concept of a ratio and use ratio language to describe a ratio relationship between two quantities • Understand the concept of a unit rate, and use rate language in the context of a ratio relationship • Use ratio and rate reasoning to solve real-world and mathematical problems • Recognize and represent proportional relationships between quantities • Identify the constant of proportionality (unit rate) in tables, graphs, equations, diagrams, and verbal descriptions of proportional relationships • Represent proportional relationships by equations • Graph proportional relationships, interpreting the unit rate as the slope of the graph. Compare two different proportional relationships represented in different ways • Write, read, and evaluate expressions in which letters stand for numbers • Write expressions that record operations with numbers and with letters standing for numbers • Evaluate expressions at specific values of their variables • Solve real-world and mathematical problems by writing and solving equations of the form x + p ¼ q and px ¼ q for cases in which p, q, and x are all nonnegative rational numbers • Use variables to represent quantities in a real-world or mathematical problem, and construct simple equations and inequalities to solve problems • Solve linear equations in one variable • Solve linear equations with rational number coefficients, including equations whose solutions require using the distributive property and collecting like terms • Create equations and inequalities in one variable and use them to solve problems • Understand that the graph of an equation in two variables is the set of all its solutions plotted in the coordinate plane • Use variables to represent two quantities in a real-world problem that change in relationship to one another; write an equation to express one quantity in terms of the other quantity. Analyze the relationship between the dependent and independent variables using graphs and tables, and relate these to the equation • Construct a function to model a linear relationship between two quantities. Determine the rate of change and initial value of the function from a description of a relationship or from two (x, y) values, including reading these from a table or from a graph. Interpret the rate of change and initial value of a linear function • Use function notation, evaluate functions for inputs in their domains, and interpret statements that use function notation in terms of a context • Write a function that describes a relationship between two quantities • Recognize situations in which one quantity changes at a constant rate per unit interval relative to another • Construct linear functions, given a graph, a description of a relationship, or two input-output pairs (include reading these from a table)

264

M. K. Mielicki et al.

unknown value is referenced twice in the quantitative relationships described in the problem (e.g., Koedinger et al. 2008). Students were also less successful in translating between other pairs of representation types, such as from tabular to symbolic representation and from verbal to graphical representation. This trouble translating across representations in algebra is a well-documented difficulty that students experience (e.g., Ainsworth 2006; Brenner et al. 1997; Leinhardt et al. 1990; Moschkovich et al. 1993). Students generally struggled with ratios except when the position of the unknown was made explicit. Both Conley et al. (2011) and NCEE (2013) used the Common Core standards as a framework for exploring what mathematical content college students should know. Conley et al. (2011) found that standards pertaining to modeling the relationship between quantities, interpreting graphs and tables, graphing symbolic functions, writing a function, and using function notation were among the standards rated most applicable by math and science instructors at four-year colleges. The findings of the present study indicate that students pursuing STEM-related career paths have mastered the skills described in these standards more so than students pursuing nonSTEM-related career paths. In contrast, an analysis of assignments, textbooks, and exams from community college courses (NCEE 2013) revealed that the most relevant standards at that level were ratios, proportionality, expressions, and simple equations. The results of this study show that even non-STEM majors demonstrate an adequate grasp of the content and skills related to these standards. The present results suggest that the specific mathematical skills and knowledge that college students have mastered is only weakly associated with three broad algebra-related content strands identifiable within the Common Core State Standards for Mathematics. This is not surprising given that each of the broad content strands encompassed multiple separate standards, and items varied with respect to the number of relevant standards and the specific ways in which the underlying knowledge and skills were applied. The findings from the present study point to the importance of focusing on identifying the substantive mathematical knowledge and cognitive skills that remain challenging for college students early in their higher-education career, that are strongly predictive of college student performance, and that are likely to influence their success in various programs of study. More detailed descriptive and diagnostic work is needed, especially since some reports have highlighted that many college students with limited academic skills do not take remedial coursework, while substantial numbers of students with strong high-school backgrounds do enroll in remedial courses (Attewell et al. 2006). Future research should explore precisely what differentiates such students, whether such differences matter, and whether the presence or absence of remediation impacts academic success. In this context, we posit the need for further development of an assessment system that is sensitive to various factors identified by previous studies of mathematical knowledge and skill, such as those discussed in this report, and its deployment in several sites with subsequent validation and refinement of the instrument for educational planning purposes. Using a diagnostic assessment system representative of the latest research on mathematics knowledge and skill as the basis of placement of students into remedial or regular education courses might lead to more effective

18

Assessing Mathematics Knowledge and Skill

265

educational decision making and permit a focus on positive learning that draws upon foundational mathematics knowledge and skills.

References Ainsworth, S. (2006). DeFT: A conceptual framework for considering learning with multiple representations. Learning and Instruction, 16, 183–198. https://doi.org/10.1016/j.learninstruc. 2006.03.001 Attewell, P., Lavin, D., Domina, T., & Levey, T. (2006). New evidence on college remediation. The Journal of Higher Education, 77(5), 886–924. https://doi.org/10.1080/00221546.2006. 11778948. Baker, F. (2001). The basics of item response theory. ERIC Clearinghouse on Assessment and Evaluation, University of Maryland, College Park. Retrieved October 30, 2004, from http:// echo.edres.org:8080/irt/baker/ Brenner, M. E., Mayer, R. E., Moseley, B., Brar, T., Durán, R., Reed, B. S., et al. (1997). Learning by understanding: The role of multiple representations in learning algebra. American Educational Research Journal, 34(4), 663–689. https://doi.org/10.3102/00028312034004663 Chen, X. (2016). Remedial coursetaking at U.S. public 2- and 4-year institutions: Scope, experiences, and outcomes. Washington, DC: U.S. Department of Education. Common Core State Standards Initiative. (2010). Common core state standards for mathematics. Retrieved from http://www.corestandards.org/assets/CCSSI_Math%20Standards.pdf Conley, D., Drummond, K., de Gonzalez, A., Rooseboom, J., & Stout, O. (2011). Reaching the goal: The applicability and importance of the common core state standards to college and career readiness. Eugene, OR: Educational Policy Improvement Center. Crocker, L., & Algina, J. (2006). Introduction to classical and modern test theory. New York: Wadsworth. Greene, J. P., & Winters, M. A. (2005). Public high school graduation and college-readiness rates: 1991–2002. New York, Manhattan: Institute for Policy Research, Center for Civic Innovation. Koedinger, K. R., Alibali, M. W., & Nathan, M. J. (2008). Trade-Offs between grounded and abstract representations: Evidence from algebra problem Solving. Cognitive Science, 32, 366–397. https://doi.org/10.1080/03640210701863933 Leinhardt, G., Zaslavsky, O., & Stein, M. K. (1990). Functions, graphs, and graphing: Tasks, learning, and teaching. Review of Educational Research, 60(1), 1–64. https://doi.org/10.3102/ 00346543060001001 Martorell, P., & McFarlin, I. (2010). Help or hindrance? The effects of college remediation on academic and labor market outcomes. Review of Economics and Statistics, 93(2), 436–454. Moschkovich, J., Schoenfeld, A. H., & Arcavi, A. (1993). Aspects of understanding: On multiple perspectives and representations of linear relations and connections among them. In T. A. Romberg, E. Fenemma, & T. P. Carpenter (Eds.), Integrating research on the graphical representation of functions (pp. 69–100). New York: Erlbaum. National Center on Education and the Economy (NCEE). (2013). What does it really mean to be college and work ready? The English and Mathematics required by first year community college students. Retrieved from http://www.ncee.org/wp-content/uploads/2013/05/NCEE_ MathReport_May20131.pdf Zlatkin-Troitschanskaia, O., Wittum, G., & Dengel, A. (Eds.). (2018). Positive learning in the age of information (PLATO) – A blessing or a curse? Wiesbaden: Springer.

Part IV

Perspectives

Chapter 19

On the Way of Developing a Holistic Explanatory Model of Positive Learning Andrey Podolskiy

Introduction and Background An exclusively impressive and powerful constellation of methods and approaches involved in the PLATO project focuses on the key question: “How can we build on existing research to achieve significant progress in learning research by explaining negative and positive learning in the age of information” (ZlatkinTroitschanskaia et al. 2018, p. 38). It is evident that to answer such ambitious question requires combining different approaches in an integrative model. Accordingly, creating such a model is considered to be one of the most important tasks in PLATO (Zlatkin-Troitschanskaia et al. 2018). The first, but far not simple and practically important step in that direction has been taken by Zlatkin-Troitschanskaia et al. (2018) who introduced four main cornerstones for collaborative research in PLATO, where the first cornerstone (a) focuses on information structures and information processing, the second cornerstone (b) touches upon broad and various expertise from several areas of education, communication and media science, psychology, sociology, the third cornerstone (c) is formed by a focus on morals and ethics as normative fundamentals of positive learning. At last, the fourth cornerstone (d) deals with development, opportunity, and risk evaluation of technology-supported learning. Already initial, probably even superficial acquaintance with the PLATO project, it highly impresses with an ambitious effort to “start collecting and linking the first pieces of the puzzle so that, over the years, PLATO can make progress towards a more complete picture of positive and negative learning in the age of information” (Zlatkin-Troitschanskaia et al. 2018, p. 44). I completely agree with the authors that “more than a decade of intensive, fundamental research is still necessary to reach the goals of PLATO: A holistic explanatory model is to be developed and established as A. Podolskiy (*) National Research University “Higher School of Economics”, Moscow, Russia e-mail: [email protected] © Springer Nature Switzerland AG 2019 O. Zlatkin-Troitschanskaia (ed.), Frontiers and Advances in Positive Learning in the Age of InformaTiOn (PLATO), https://doi.org/10.1007/978-3-030-26578-6_19

269

270

A. Podolskiy

a framework for the study of positive learning and negative learning in the age of information” (ibid). To evaluate how complex such a task is, one may bear in mind the efforts to build a satisfactory meta-theory of learning that are still “relatively immature compared to many recognized elemental learning theories” (Swanson 2012, p. 2252), although the task for meta-learning theorists—to fuse the behaviorist, cognitivist, humanist, social, constructiveness, and holistic orientations to learning theory (Swanson and Holton 2009, p. 195)—is sufficiently narrower and does not require an interdisciplinary collaboration with all obvious complications as the PLATO mega-task does. It is unlikely that any of the distinguished scholars who initially joined the PLATO enterprise would claim that they know the way to reach a final project goal—“to form a holistic, multilayered, multiscale, environment- and valuesensitive model of learning” (Zlatkin-Troitschanskaia et al. 2018, p. 45). The main resource to reach it is considered interdisciplinary cooperation and advanced computational possibilities. In general, that is correct. However, we all know how difficult it is to find a “common language” for the representatives of different fields of knowledge. It is my firm belief that one of the important steps is a broad and intensive discussion of PLATO’s “stake holders” aiming to come to the consensus about the conceptual rules of the interdisciplinary cooperation. The recent situation is the following: The consensus—more or less—exists concerning the general understanding of the concepts “positive” and “negative” learning with a lot of semi-tones and shades (i.e., as demonstrated by different authors in this volume). Certainly, a long-term interdisciplinary collaboration should bring with it its results. However, I suppose that the way of developing a holistic explanatory model of positive learning might be optimized and accelerated by the following means: trying to select theoretical schemes which either exist already or will probably emerge in the nearest future and that are mostly close to the solution of two important tasks: First, to describe in holistic and integral terminology the main cases of both positive and negative learning in their contraposition: “successful acquisition of subject-specific and generic concepts, abilities, and skills, and the learning environments, and social and societal contexts in which knowledge, skills and abilities are acquired, and false disciplinary or interdisciplinary concepts (misconceptions or alternative conceptions) as well as attitudes and prejudices that impede learning and information processing according to common quality standards (e.g., introducing bias), or contradict universal societal values and norms” (Zlatkin-Troitschanskaia et al. 2018, p. 33). Second, to ensure, at least potentially, the effects of positive learning and to avoid effects of negative learning in a more or less wide area. It is evident clear that even complete and detailed description does not automatically provide a guaranteed shift from negative to positive learning.

19

On the Way of Developing a Holistic Explanatory Model of Positive Learning

271

Activity Theory of Learning and “Positive Learning in the Age of Information”: Could Traditional Theories of Learning Still Work? The aim of this paper is to offer an optional scheme, which is based on the activity theory of learning and might be applied to a broad field of influences, listed in the description of the four “cornerstones” (Zlatkin-Troitschanskaia et al. 2018). “Activity theories of learning” is a generalized term for learning theories that are based on the general “activity approach (paradigm, outlook, framework)” initially introduced by Russian/Soviet psychologists L. Vygotsky, A. Leontiev, S. Rubinstein, A. Luria and further developed by their disciples and followers both in Russia and in the West Countries (e.g., Davydow, Zinchenko, Wertsch, Cole, Engeström) (Podolskiy 2012a). The representatives of the first generation of the activity-oriented learning theorists (Elkonin, Davydow, Galperin, Lompscher, Talyzina) considered human learning processes in two interconnected but nevertheless different respects: 1. learning as a universal mechanism for the appropriation of social experience by an individual (Galperin 1992; Talyzina 1981; Davydow 1999). 2. learning activity as a special form of the social activity of personality. The second (current) generation of the activity-oriented learning theorists pays additional attention to the extension and expansion of the approach elaborated by their predecessors. Due to the fact that activity theories of learning went international in the 1980s, the cultural dimension of learning is taken into account not only declaratively by these theorists but also with respect to learning in concrete cultural settings (Elbers, Hedegaard, Wardekker). In addition, several authors have called attention to the necessity of developing conceptual tools to understand dialogue, multiple perspectives and voices, and networks of interacting activity systems (Sannino et al. 2009; van Oers et al. 2008). Engeström (1987, p. 64) considers expansion as “a form of learning that transcends linear and socio-spatial dimensions of individual and short-lived actions, . . . learning is understood in the broader and temporally much longer perspective of a third dimension, that is, the dimension of the development of activity.” Meanwhile, in recent decades, Russian scholars have been continuing to concentrate on exploring the age-related peculiarities of learning processes (Obuchova, Burmenskaya), clarifying the role of adult–child and child–peer communication in the facilitation of learning processes (Rubtsov; Zuckerman), and exploring the problem of bridging a gap between theoretical activity-related learning models and real instructional technology (Podolskiy 2012b). New directions, which open broad perspectives for further theoretical, empirical, and applied research in the frame of the activity approach, have appeared, such as “learning to learn” (Hautamäki et al. 2002), an application of the activity framework to the area of human–computer interaction (Kaptelinin, Nardi).

272

A. Podolskiy

The basic assumption of the activity theory of learning is that “types of knowledge towards which the learning process is directed then appear both as the motivation, in which the student’s need for learning has become objectified, and the activity’s objective. In cases where students do not have such a need, they either will not be engaged in learning or else will be learning to satisfy some other need. In such a case, learning ceases to be an activity since instead of meeting a particular need – the acquisition of knowledge – it merely serves as an intermediary objective. In such a case learning is an action realizing some other activity; the knowledge that serves as the action objective does not serve as a motivation, since it is not knowledge which activates the learning process” (Talyzina 1981, p. 45). Learning is understood within the framework of the activity paradigm as a universal mechanism for the appropriation of social experience by an individual. The core element of the learning process is formation (appropriation) of actions (mental, perceptual, motor, and verbal). Human actions and images reflect, and are the product of both human needs and the demands and conditions of the objective situation. Any human action may be characterized by a set of primary and secondary properties. The following properties are considered to be primary: (a) the composition of the action’s objective content, (b) the extent to which essential elements of the problem situation are differentiated from nonessential elements, (c) the degree to which the action has been internalized, and (d) “energetic” (speed and enforcement) parameters. The secondary properties are: (a) (b) (c) (d)

reasonability, generalization, consciousness, and criticism.

The secondary properties are the result of specific combinations of primary properties. Both primary and secondary properties represent socially estimated and evaluated qualities of human activities and may refer to any sort of activity, whether individual or collective, material or mental (Galperin 1992). The final values of these properties determine the specific action and/or image that is formed. Galperin (1992) considered the values of the properties to be the direct outcomes of action formation conditions. He, therefore, defined a system of conditions that guarantees the achievement of prescribed and desired properties of action and image. It is called the system of planned, stage-by-stage formation of mental actions or the PSFMA system and includes four subsystems: 1. the conditions that ensure adequate motivation for the subject’s mastering of the action, 2. the conditions that establish the necessary orientation base of action,

19

On the Way of Developing a Holistic Explanatory Model of Positive Learning

273

3. the conditions that support the consecutive transformations of the intermediate forms of action (materialized, verbal) and the final end transformation into the mental plan, and 4. the conditions for cultivating or “refining through practice” the desired properties of an action (Galperin 1989). Each subsystem contains a detailed description of related psychological conditions, which include the motivational and operational areas of human activity. The PSFMA system represents a complete nomothetic set of psychological conditions which stand behind the learning processes, and any specific case of learning may be considered as a result of “subtracting” one or the other condition from the complete list. Accordingly, absent elements of the PSFMA system may be easily found and inserted. In the nomothetically orienting role of the general PSFMA system, the successful application of the PSFMA does not imply a literal reproduction of some abstract, extremely general procedure. Rather, it refers to the creative design of a system of necessary and sufficient psychological conditions for instruction. The elaboration of such a procedure occupies an intermediate position between fundamental psychological knowledge and the actual process of schooling, instructing, or training. This intermediate position is operationalized in the consecutive elaboration of three models of the instructional situation. These are the psychological, the psychological–pedagogical, and the procedural, or technological, models (Podolskiy 2014). The psychological model includes: 1. a description of the knowledge and skills to be acquired on the basis of the learner’s mental actions, images, and concepts, 2. a description of the macro- and microstructure of the multilevel learner’s orientation as the basis for a new mental action, concept, or image to be formed,1 3. a description of age-related and individual characteristics of the learner that are relevant to instruction and schooling, and 4. a description of the specific system of psychological conditions needed for the formation of the planned action. It is evident that in different implementations of the PSFMA system, application emphasis should be placed on different constituents of the psychological model. The main function of the psychological–pedagogical model is to project the psychological model onto the specific objective and subjective conditions of schooling and teaching. Such conditions include instructional activities and 1 Three psychologically different but interconnected levels of orientation base may be distinguished in considering mental activities of learning: (a) the executive orientation base, a scheme of human orientation on how to do something; (b) the goal- orientation base, a scheme of human orientation on what to do; (c) the sense orientation base, a scheme of human orientation on why ( for what) to do something. The three levels of orientation base are connected to each other in both ascending and descending order: Human understanding on how to do something also affects higher level sense and goal representations and is in turn affected by the possibilities and execution of the sense and goalorientation bases (Podolskiy 2014).

274

A. Podolskiy

the organization and distribution of different organizational forms during a lesson or a sequence of lessons; in-class and homework activities along with individual, small-group, and whole-class learning activities; use of available technical aids for teaching (e.g., computer-assisted learning). One might declare that the psychological–pedagogical model represents the “art of the possible”—that is, it reaches an optimal compromise between the strict requirements of the psychological model and the restrictions constructed by objective and subjective components of reality. Sometimes it is necessary to reduce such strict requirements (at least part of them) in favor of implementation, and sometimes they are necessary to overcome resistance in the traditional learning environment to implement innovation. The last, procedural, or technological model of instructional situations includes a detailed description of the teaching process distributed between units of definite form and time, with a precise description of the goal of each unit and the means to achieve it. It also includes a complete list of teaching documentation: schemes, different types of learning and assessment tasks, a description of the order in which technical aids should be applied, and a number of other materials specified for different types and kinds of schooling/instructional situations. The procedural model looks like the traditional well done “teacher’s lesson plan”; however, one has to remember that this model is based on the considerations outlined in the psychological and psychological–pedagogical models. It is also necessary to consider the three-model framework as an intellectual tool, not just as an algorithm that prescribes how a teacher should act. This framework, when used in an appropriate and sophisticated way, gives a teacher the ability to orient, plan, control themselves completely, and correctly design, arrange, and carry out different instructional activities. This framework may provide us with an applied psycho-educational theory that occupies an intermediate position between fundamental psychological knowledge and educational/instructional practice (Podolskiy 2014).

Conclusion As a conclusion, I would like to raise an almost obvious question: How does the activity-based approach comply with the PLATO project’s set of challenges? If one was to limit an exploration of this question by the PSFMA approach only, one may say that, it has been convincingly demonstrated by hundreds of experimental and applied studies that a whole set of the main objectives of any schooling effort have been reached through this approach. For example, 1. the guaranteed acquisition of the curriculum by all learners with the necessary level of preliminary knowledge and skills is achieved without prolonging the time allocated and with essentially no additional cost;

19

On the Way of Developing a Holistic Explanatory Model of Positive Learning

275

2. the separation of instruction into the acquisition of knowledge and its application is minimized or wholly disappears; 3. learners are able to transfer acquired abilities to new situations and are also able to transfer the process for acquiring new knowledge and skills; 4. by becoming aware of these newly formed abilities, learners become more and more interested in the processes of acquiring knowledge and in knowledge itself. On the one hand, it is not difficult to see that these indicators are matching at least some of the indicators of positive learning. However, on the other hand, the vast majority of both laboratory and applied cases of the PFSMA (with a few exceptions) has been designed and performed in, so to say, “traditional” environments. Would this circumstance become crucial for the efforts to apply the PFSMA approach in the conditions of the “Age of Information”? There are at least two options for getting the answer to this question, which lie on the surface. 1. The first is an empirical one. Select one case of the unambiguously negative learning, and develop the proper PFSMA model to provide a shift to the positive one. 2. The second is a theoretical one. Select two interrelated cases of positive and negative learning, and provide a comparison of the conditions that tentatively stand behind the opposite cases. After getting a solid theoretical result, it might be possible to make an empirical check similar to the previous one. Certainly, both—empirical and theoretical—cases will require as a prerequisite of the study a proper explicit description of both phenomena—positive and negative learning in terms of which features are present and which features are absent. Then, it will be possible to impose one model (phenomenological) to another model (PSFMA-based theoretical), and clarify what phenomena are not covered by the theoretical model. One may also expect to get one or two comments on the statement about an insufficiency of traditional theories of learning axiomatically introduced by Zlatkin-Troitschanskaia et al. (2018). In other words: Should we abandon the traditional theories of learning only because they are created in the pre-information age without serious exploration of their contemporary explanatory potentials?

References Davydow, V. (1999). What is real learning activity? In M. Hedegaard & J. Lompscher (Eds.), Learning activity and development (pp. 123–139). Aarhus: Aarhus University Press. Engeström, Y. (1987). Learning by expanding: An activity-theoretical approach to developmental research. Cambridge: Cambridge University Press. Galperin, P. (1989). Organization of mental activity and effectiveness of learning. Journal of Soviet Psychology, 27(3), 65–82. Galperin, P. (1992). Stage by stage formation as a method of psychological investigation. Journal of Russian and East European Psychology, 30(4), 60–80.

276

A. Podolskiy

Hautamäki, J., Arinen, P., Eronen, S., Hautamäki, A., Kupiainen, S., Lindblom, B., et al. (2002). Assessing learning-to-learn: A framework. Helsinki: National Board of Education. Podolskiy, A. (2012a). Activity theory of learning. In N. M. Seel (Ed.), International encyclopedia of the sciences of learning (pp. 83–85). New York: Springer. Podolskiy, A. (2012b). Development and learning. In N. M. Seel (Ed.), International encyclopedia of the sciences of learning (pp. 944–950). New York: Springer. Podolskiy, A. (2014). “There is nothing so practical as a good theory”: How to let it work in practice (the case of Galperin’s theory). Psychology in Russia: State of the Art, 7(3), 4–12. Sannino, A., Daniels, H., & Gutiérrez, K. D. (Eds.). (2009). Learning and expanding with activity theory. New York: Cambridge University Press. Swanson, R. (2012). Metatheories of learning. In N. M. Seel (Ed.), International encyclopedia of the sciences of learning (pp. 2251–2255). New York: Springer. Swanson, R. A., & Holton, E. F. (2009). Foundations of human resource development (2nd ed.). San Francisco: Berrett-Koehler. Talyzina, N. (1981). Psychology of learning. Moscow: Progress Publishers House. Van Oers, B., Wardekker, W., Elbers, E., & Van der Veer, R. (Eds.). (2008). The transformation of learning: Advances in cultural historical activity theory. Cambridge: Cambridge University Press. Zlatkin-Troitschanskaia, O., Schmidt, S., Molerov, D., Shavelson, R., & Berliner, D. (2018). Conceptual fundamentals for a theoretical and empirical framework of positive learning. In O. Zlatkin-Troitschanskaia, G. Wittum, & A. Dengel (Eds.), Positive learning in the age of information – A blessing or a curse? (pp. 29–52). Wiesbaden: Springer.

Chapter 20

PLATO in Search of Identity Richard J. Shavelson

PLATO’s Brief History PLATO was conceived at the Johannes Gutenberg-University of Mainz (JGU) in response to the German Research Foundation (DFG) (2016) “Call for proposals in the Clusters of Excellence and Universities of Excellence.” While the JGU had an excellence funding line in precision physics, fundamental interactions, and structure of matter (PRiSMA; https://www.prisma.uni-mainz.de/), PLATO’s research agenda sought to integrate and build on the educational, behavioral, neural, social, and computer sciences. The abstract for its “Draft Proposal for the Establishment and Funding of the Cluster of Excellence” (JGU 2016) reads as follows: The internet offers extremely effective new opportunities for manipulation; there are countless examples of misinformation and false data being spread in mass media, which can have grave effects on human thinking and behavior. In this light, deficits in higher education and research – both providers and generators of new knowledge and learning technologies – become apparent. The aim of the PLATO Cluster of Excellence is to unite outstanding and highly innovative expertise on the fundamental structures of information processing and human learning, and to investigate the fundamentals of learning in the digital age, which are of existential importance to our modern society (p. 2).

The aim of PLATO’s research today remains closely related to its initial conception but with a sharper focus (https://www.plato.uni-mainz.de/research-focus/; accessed 9/2/18): While information and communication technology has an immense influence on our lives, little is understood about its particular effects on the way we learn. In the Age of Information, scientific knowledge has to compete against misinformation and biased data spread through social and mass media. These conditions pose hazards for learning in formal and informal environments.

R. J. Shavelson (*) Stanford University, Stanford, CA, USA e-mail: [email protected] © Springer Nature Switzerland AG 2019 O. Zlatkin-Troitschanskaia (ed.), Frontiers and Advances in Positive Learning in the Age of InformaTiOn (PLATO), https://doi.org/10.1007/978-3-030-26578-6_20

277

278

R. J. Shavelson

The international research program “Positive Learning in the Age of Information” (PLATO), established in 2016, unites outstanding expertise with innovation in information processing to investigate positive learning in our knowledge-based society in a vital scientifically, morally and ethically oriented approach.

A definition of positive learning has been elusive (for a definition of positive learning, see Zlatkin-Troitschanskaia et al. 2018b). The definition of its opposite, negative learning, has proven to be easier to come by: knowledge acquired, intentionally or unintentionally (without awareness), from biased or false information in conflict with democratic norms and values (for a definition of negative learning, see Zlatkin-Troitschanskaia et al. 2018b; Zlatkin-Troitschanskaia et al. in this volume). While funding is always a concern for university research, PLATO offered a strong academic incentive to build in areas not only of student demand (social and behavioral sciences) but of high societal value in the “Age of Information.” PLATO was seen by the JGU Mainz as a means of building its social and behavioral research as well as its teaching infrastructure. It turned out that student demand for courses of study in these areas had burgeoned at JGU and the university was concerned about building faculty capacity in particular in the humanities and social sciences to meet the demand. In the end, the DFG did not fund PLATO or any new Excellence Cluster proposals from the educational sciences and humanities in 2017. So an outstanding idea for a research center linking various universities and research institutes in the pursuit of reason in the Age of Information was left with few resources for pursuing its agenda. Small amounts of university funding supported the research presented in the first PLATO volume (Zlatkin-Troitschanskaia et al. 2018a) and in this volume. This funding was used by individual and small groups of the involved universities to pursue their research that was tangentially linked to the PLATO vision. This volume presents the current state of PLATO’s research. Organizations, like individuals, search for identity: Who am I? Why am I here? What is my purpose? How do I add value? PLATO, the ambitious research center dubbed “Positive Learning in the Age of Information,” is searching for its identity (for an overview, see Zlatkin-Troitschanskaia et al. 2018a, b). The search for PLATO’s identity is made more difficult than usual as PLATO is a consortium of many academic institutions—Johannes Gutenberg-University of Mainz, the hub; Goethe University Frankfurt, Technical University Darmstadt, and the Technical University Kaiserslautern. As academic institutions are habited by independent thinking professors with their predictably narrow research agendas, the search for identity is exacerbated—the phrase, “like herding cats,” comes to mind. The complexity of this search for identity is documented in this volume and the contributions included herein. As a member of PLATO’s scientific advisory board and having been involved with PLATO since its inception (roughly May 2016), I was asked to critically discuss and comment on the current state of PLATO’s research and its perspectives. What follows are my curmudgeonly conclusions.

20

PLATO in Search of Identity

279

PLATO As Evidenced in This Volume One way to try to capture PLATO’s identity is to look at the organization and nature of the contributions included in this volume. This book does so at the ground level; I take a birds-eye view to try to capture identity. Overall, twelve of PLATO projects and eight contributions by PLATO’s international partners are presented in this book. The projects presented in the book’s chapters are interdisciplinary in nature. However, only few projects are based on cross-institutional collaboration. A key strength of PLATO is and should be interdisciplinary and crossinstitutional research. This gives rise to the question of whether PLATO is adequately harnessing its potential strengths. Indeed, there is nothing unique in an identity of individual or small groups of university members pursuing their research interests under a common umbrella that post-hoc links them together. Of the twelve research projects presented here, nine dealt with cognition, eight with communication and interaction, nine with language and representations, four with teaching methods/instruction, eight with new analytic methods, and eight with the use of innovative computational methods. Needless to say, a project might be counted more than once. Of the projects, a few dealt with educational practice or technical applications. Having enjoyed reading a wide range of fascinating research I found that a couple of questions kept buzzing in my head. Where’s the PLATO vision? Where’s the PLATO coherence?

PLATO’s Vision and Mission As far as I can tell, PLATO has, at least on paper, a vision and mission. From its most recent proposal we find that PLATO is committed to the following: Every day, abundant information on mass and social media reaches billions of people around the globe. However, much of the information presented online is biased, false, inaccurate, conflicting, unverified, preselected or algorithmically obscure, often collides with fundamental humanistic values and poses moral and ethical problems. When Internet users generate knowledge based on such information, negative learning (NL) occurs, which can manifest, e.g., in the acquisition of domain-specific misconceptions or counter-factual knowledge. NL is difficult to avoid, as it often occurs unintentionally and unconsciously. Studies reveal that most Internet users do not have the necessary skills to autonomously select and consciously process (online) information and are vulnerably to being misled and unwittingly acquiring false information. In the Information Age, individual beliefs or prejudices become more important than factual knowledge as most Internet users struggle to filter and process vast amounts of information and tend to rely on the first few hits of an online search. To date, research on human learning has not kept up with the rapid development of ICT and little is known about the effects of ICT use on learning. In PLATO, we examine the influence of new ICT realities and upcoming technological developments and how they affect positive learning (PL) [italics mine].

280

R. J. Shavelson

Fig. 20.1 PLATO research areas (https://seafile.rlp.net/d/4fa2215e936b4d4da534/; accessed 8/27/18)

On the PLATO website the vision of PLATO’s research is described as follows (see also Fig. 20.1): PLATO is a research initiative among multiple German universities and research institutes as well as numerous cooperating universities and researchers globally. PLATO unites outstanding expertise with innovation in research investigating the representation of information and its effects on students’ positive learning as a vital, scientifically, morally and ethically oriented learning approach in a knowledge-based society. We seek to study the impact of the Internet and especially social media on university students’ learning – with implications for citizens’ learning in democratic societies. The research covers studies of basic informational and cognitive structures and processes in domain-related learning from sources of high or low information quality. Outcomes have implications for action in applied research in education and design of tools for students and citizens to responsibly handle information in digital media and mitigate their negative impacts on student learning and citizen engagement in democracies.

The vision is consistent with my consideration of the series of fascinating, somewhat unconnected research projects presented in this volume that could be conducted by individual and small groups of researchers from a wide range of disciplines and professions. However, identity needs a coherent, connected set of projects that aim at some higher common purpose. That purpose, as highlighted in the PLATO’s vision statement, raised some questions in my head. What vision underlies the different project areas? How is PLATO organized to realize that vision? Moreover, I wondered about coherence: How are the different areas related to produce research toward the vision? Where are the collaborations that are necessary to realize the vision? Bottom line: What is PLATO’s value added?

20

PLATO in Search of Identity

281

Toward a Possible Identity for PLATO In summarizing the contributions included in this volume I realized that PLATO is not only about basic research as required in the DFG Excellence Cluster proposal. PLATO is also about applied research on educational practice at the university and new technological developments and applications to support basic and applied research and development. Basic, applied, and technological research and development seemed to be needed to address the PLATO mission: “to investigate positive learning morally and ethically in a knowledge society”. As these thoughts emerged in my mind new questions arose. Should basic, applied, and technical research and development be pursued in parallel? Should they build on one another? How might they fit with the whole vision of PLATO? Stokes (1997), in reviewing the history of scientific research, found the linear model of research from basic to applied to practical application to be misconceived and empirically unsupportable. The great discoveries were found in Pasteur’s Quadrant (Fig. 20.2)—use inspired basic research. This notion rang a bell and seemed to be a way to conceptualize and integrate PLATO research. The questions of identity—vision, mission, and research agenda—just might be driven by “use inspiration.” Ultimately what does PLATO want to achieve? In a real sense, it seeks to achieve positive learning at the demise of negative learning in schools, colleges, universities, and society at large. Working backwards the question is what mix of basic and applied research is needed to realize this vision?

Fig. 20.2 Graphic representation of Pasteur’s quadrant (Stokes 1997)

282

R. J. Shavelson

Taking Stock In looking at this volume, I could see the possibility of developing PLATO into a coherent program of research, development, teaching, and public outreach. To do so, some stock taking is needed.

Basic and Applied Research The papers presented in this volume and their organization demonstrate two strengths. They present important collaborative projects that bore more or less on negative and/or positive learning. Moreover, they use innovative analytic methods. The weaknesses, overall, are that they largely focused on fundamental, basic research, were loosely coupled inter-disciplinarily and methodologically, lacked connections across participating institutions and the promise of unique contributions faculty from each would add to an interdisciplinary team of researchers. It seemed to me that, with tough decisions, PLATO could develop a clear vision of purpose so that PLATO’s research and technological development (R&D) would lead to positive learning and reduce negative learning for education and society. Moreover, with this vision, research projects could be linked across disciplines, sites, methods, and “areas.”

Education Research into Practice In the end, although originally conceived as a basic social and behavioral science research institute, PLATO aims to impact education practice—educating students pre-university and in university and educating the public in positive learning whether it be for academic achievement or participation in a democratic society. This is difficult for research universities to stomach. The practical is for practitioners (although Stokes and I would demur). Research on education practice should build on the basic/applied research carried out in PLATO and feedback to inform that research. Indeed, education practice could very well be the test bed for ideas and findings from these projects and even more importantly feedback finding and so affect the direction of the basic/applied research—that is working in Pasteur’s Quadrant. Such a test bed might include randomized trials testing hypotheses in a somewhat controlled environment. I say somewhat controlled as teaching and learning are harder to control than the force of gravity in PRiSMA research (https://www.prisma.uni-mainz.de/). Moreover, PLATO could make use of naturally occurring quasi-experiments or observational studies where innovations in education practice that impact negative

20

PLATO in Search of Identity

283

and positive learning are being tried outside of or closely related to the current PLATO research agenda. Such projects might identify new areas for more basic/ applied research in PLATO as well as a reality test of the constraints inherent in educating students in schools and universities, and adults in society. By linking PLATO’s agenda with education practice, teacher education, schools, and higher education institutions become involved. A sort of “clearinghouse” might be established. PLATO’s research would inform practice and practice would inform PLATO’s research and development agenda.

Technological Development The challenges presented by the profiling of individual Internet users for the purpose of sending fake news, misinformation, falsely advertised websites, etc. that play to their profiles will take more than educating students and citizens to be critical thinkers and reflective users of the Internet. Individuals live within multiple contexts which significantly influence their thinking, emotions, and behavior (see also Berliner, in this volume). These externalities need to be considered in fostering positive learning. One clear use of technology is in teaching and learning. PLATO should make full use of the technology in this way. Importantly technology should be used to teach students how to deal with information gathered from the Internet. However there are other pressing uses of technology that PLATO might embrace. Technology, for example, could be built to “fight” unethically used and misused technology if PLATO is to enhance positive learning and reduce negative learning. Such technology would build on current developments of “alternative facts detectors,” artificial intelligence, and the analysis of big data. Perhaps PLATO should create a special technology laboratory that would work with and build on developments in its research and development areas. Collaborators in PLATO, for instance, the German Research Institute for Artificial Intelligence, provide such potential expertise.

Public Outreach The most obvious arena for PLATO in reaching out to the public is in education, both pre-college and university education. The developments within PLATO’s R&D programs might very well be expected to provide useful concepts and tools to improve teaching and learning through the wise use of the Internet. Business and policy appear to fall outside PLATO’s purview judging from the papers presented in this volume. Nevertheless, both are big players in fostering positive learning and blunting negative learning. PLATO’s basic and applied

284

R. J. Shavelson

research and its technological developments will have important but smaller effects than needed without coordination of its agenda with business and policy. In the business world, the Internet and in particular social media have become one of the largest money makers in the world. The open access Internet has permitted anyone or entity to post whatever it wants without responsibility. Facebook, Google, and the likes are beginning to crack down on users acting irresponsibly but only under great pressure from government and the public. Business leaders cannot be expected to ignore their bottom lines and stock values given the economic pressures of the global economy. However, they can be expected to act responsibly and ethically. Anyway, without external pressure, as we have seen, they will not do so wholeheartedly. Government needs to step up and develop Internet policies that sanction sources of negative learning. We are beginning to see this happen in part by the European Union’s efforts. Perhaps some aspect of PLATO should focus on business and policy as levers for increasing students’ and the public’s access to positive learning and curtailing sources of negative learning.

Toward Establishing PLATO’s Identity PLATO needs to go beyond its current rhetoric to build a coherent vision of its purpose and value added. It needs to formulate a mission and R&D program to move it toward that vision. This in contrast to seeing who the researchers are and creating an inclusive vision post hoc. The vision has to be bigger than simply producing academic research that is published in the top peer reviewed journals. It must demonstrate improvement in teaching and learning in the German education system as well as have an impact on teaching and learning for citizens at large. To this end, PLATO may need to study the role of business and government in its quest. It must, in the end, influence policy makers, business leaders, educators, and the public to meet its vision and promise. PLATO needs adequate funding which is something it does not have. With insufficient funding its vision, mission, and identity will be warped by incoherent funding opportunities. PLATO also needs an organizational structure and set of processes that support its mission. Perhaps this is a cart and horse problem—which comes first: organization or funding? What I’ve seen is that without clear leadership and an organization to support it, PLATO will flounder; its identity will continue to be elusive.

PLATO’s Research PLATO’s research areas would be derived from its mission. I’m not sure what this might look like. Right now the division seems to include the: (a) person or

20

PLATO in Search of Identity

285

small group (neurological, psychological, and sociological research), (b) situation or context characterized by social media and the like (linguistics, communications), (c) educational where person meets situation/context, and (d) technological (computer scientists, statisticians, mathematicians). PLATO might expand its current purview to include policy and business expertise, perhaps as a small area of research in addition to its current conceptualization.

PLATO’s Organization PLATO needs to organize itself to accomplish this ambitious research agenda. Here I briefly sketch one possible organization for PLATO. PLATO would be led by a director who is highly regarded by academics, business leaders and policy makers. The director would focus on PLATO’s overall vision and mission—a Mr. or Ms. “outside.” PLATO’s day-to-day business would be run by a chief operating officer. He or she would be a highly regarded academic with a broad view of the social and behavioral sciences—a Mr. or Ms. “inside.” Each area of PLATO’s research would be comprised of interdisciplinary/interuniversity researchers overseen by an associate director. PLATO would also have an Office of Public Relations headed by an associate director. This is where business leaders, policy makers, educators, and citizens would come together not only to learn of PLATO’s work but to inform PLATO’s work. This likely would garner important support for PLATO’s mission. If PLATO is to realize its vision and come to have a clear, coherent identity, it needs commitment from: • Presidents and provosts of participating universities, and especially from those at the lead university JGU Mainz. • Faculty—to paraphrase a famous US President, one held in high esteem by Germany, John F. Kennedy—should: Ask not what PLATO can do for you and your research; rather ask what you and your research can do for PLATO. • Leadership from within PLATO whose authority to act is fully established with the budget needed to act. • Adequate and stable funding to permit PLATO to stay with its vision and not stray to tangential projects with ready funding.

Concluding Comment Organizations, like individuals, search for identity: Who am I? Why am I here? What is my purpose? How do I add value? PLATO is searching for its identity. Its cause is just and noble—Positive Learning in an Age of Biased and False Information—the

286

R. J. Shavelson

ultimate result of relativism. Its cause is timely—democracies all over the world are reeling from the impact of instantaneous mass media and social networking with forceful political agendas and no responsibility for warranted knowledge let alone “truth.” I don’t know if the identity sketched here has any traction. My intent is to stimulate discussion and action on PLATO’s identity and its very existence. PLATO is far too important an endeavor to blow it. Now is the time to get PLATO’s act and funding together.

References German Research Foundation (DFG). (2016). Call for proposals in the Clusters of Excellence and Universities of Excellence. Retrieved August 27, 2018, from http://www.dfg.de/download/pdf/ foerderung/programme/exzellenzstrategie/schedule_excellence_strategy.pdf Johannes Gutenberg-University Mainz (JGU). (2016). Draft proposal for the establishment and funding of the Cluster of Excellence: Positive Learning Typology (PLATO). Mainz: Johannes Gutenberg-University Mainz. Stokes, D. E. (1997). Pasteur’s Quadrant – Basic science and technological innovation. Brookings: Institution Press. Zlatkin-Troitschanskaia, O., Schmidt, S., Molerov, D., Shavelson, R. J., & Berliner, D. (2018b). Conceptual fundamentals for a theoretical and empirical framework of positive learning. In O. Zlatkin-Troitschanskaia, G. Wittum, & A. Dengel (Eds.), Positive learning in the age of information – A blessing or a curse? (pp. 29–52). Wiesbaden: Springer. Zlatkin-Troitschanskaia, O., Wittum, G., & Dengel, A. (Eds.). (2018a). Positive learning in the age of information – A blessing or a curse? Wiesbaden: Springer.

Chapter 21

What Can We Learn from Theoretical Considerations and Empirical Evidence on Learning in Higher Education? Implications for an Interdisciplinary Research Framework Olga Zlatkin-Troitschanskaia, Sebastian Brückner, Dimitri Molerov, and Walter Bisang

Contexts and Developments in Knowledge Building in the Internet Age Digital Media Revolution and Co-construction of Knowledge in Higher Education. In 370 BC, Plato wondered how the limitations of human perception influenced understanding and knowledge of the environment or reality. Today, in modern learning environments, student learners are confronted with an overload of non-validated online information, and need to interpret decontextualized information, for example, images (or even ideas) (Behrens et al. 2010). All universities direct students to the Internet for sources of information, and students rely heavily on the Internet to complete assignments (Cheon et al. 2012; Fossland 2015). Academic learning is increasingly beyond the influence of formal educational institutions as it becomes more location- and time-independent, individualized, multimodal, and self-controlled than ever before (e.g., Dabbagh and Kitsantas 2012). The Internet, considered “both the world’s best fact-checker and the world’s best bias confirmer” (Lynch 2016), has become the main source of information for students (Brooks 2016; Maurer et al. in this volume). Choosing from a flood of dynamic, globally connected, partly unwarranted information within a multitude of sources is a major challenge for learners. While the Internet offers new benefits and opportunities for learning and teaching, it has also enabled profound manipulations O. Zlatkin-Troitschanskaia (*) · S. Brückner · W. Bisang Johannes Gutenberg University Mainz, Mainz, Germany e-mail: [email protected]; [email protected]; [email protected] D. Molerov Humboldt University of Berlin, Berlin, Germany e-mail: [email protected] © Springer Nature Switzerland AG 2019 O. Zlatkin-Troitschanskaia (ed.), Frontiers and Advances in Positive Learning in the Age of InformaTiOn (PLATO), https://doi.org/10.1007/978-3-030-26578-6_21

287

288

O. Zlatkin-Troitschanskaia et al.

and dissemination of countless examples of misinformation and false data through social and mass online media (Lazer et al. 2018). Little research has been conducted on the key challenges and potential negative effects of online learning in formal academic education (Newman and Scurry 2015). In response, only additive measures, such as providing more computers and offering new modules on digital literacy, have been offered even though the educational impact of such additive measures has been scarcely investigated and is debatable. Academic Learning Using “Alternative Facts”. The rapid development of two parallel learning systems, or worlds, can be observed in academic practice: students use traditional resources such as course scripts along with mass and social online media as sources for learning—using the latter often at the same time and more frequently and intensively than lecture notes or textbooks (Gikas and Grant 2013; Persike and Friedrich 2016; Maurer et al. in this volume; Mushtaq and Benraghda 2018). They depend on the Internet to write papers and prepare for exams. Thus, students’ knowledge can be based on content acquired from traditional learning media and/or mass online (and offline) media sources, including warranted facts alongside unwarranted or didactically unvetted information and even fabricated information. The responsibility to judge the quality of information and distinguish trustworthy from distorted, unwarranted, or biased information rests with the learner. Online, students’ selection of information sources, content, representations, and quality depends on both their individual prior knowledge and beliefs as well as on—hardly researched—algorithmically selected information subsampling (e.g., by ranking algorithms from search engines such as Google). Negative Learning in Higher Education. Negative learning (NL) is an unexplored phenomenon that occurs in higher education when students, for instance, use academically unwarranted, unvalidated information or draw false conclusions while developing domain-specific knowledge. NL is reflected in a decrease in students’ disciplinary knowledge or understanding from the beginning to the end of their academic studies, that is, students leave university with biased or even false knowledge and conceptions in their subjects without being aware of it (Zlatkin-Troitschanskaia et al. 2018; Schmidt et al. in Chapter 16). The decrease is more pronounced among students who use online sources more often when studying. Today, many bachelor students do not use lecture notes or textbooks for learning in higher education (Gikas and Grant 2013; Persike and Friedrich 2016; Maurer et al. in this volume). Moreover, studies of biases (e.g., eye witness testimonies) indicate that even information identified as untrustworthy at a particular moment may still be remembered as vaguely correct, and false certainty can be attributed to imagined events (Bernstein et al. 2011; Wilson et al. 2013; Somerville et al. 2008). Thus, further knowledge construction can be jeopardized as belief updates occur unsystematically. Even though the mechanisms of knowledge acquisition from various types of (online) information are dramatically under-researched, it is evident that learners subjectively select and process information from diverse sources (Von der Mühlen et al. 2015, 2016) and that the risk of acquiring misconceptions increases with the intensity with which low-quality information is used (Steffens et al. 2017).

21

What Can We Learn from Theoretical Considerations and Empirical Evidence

289

Typical misconceptions are related to domain-specific concepts, remain stable over time, arise from individual experiences, and contradict scholarly subject understanding (Verkade et al. 2017). To prevent academic domain learning from becoming meaningless or leading to NL, it is vitally important to understand how combinations of “parallel learning systems” that include both formal teaching and informal “uncontrolled” sources of information affect the formation of students’ misconceptions and biases. The Online Information Era. Developments in the online information era make positive learning (PL) outcomes—concisely defined as a desired outcome reflected in students’ increase in warranted knowledge and understanding between the beginning and end of their academic studies (Zlatkin-Troitschanskaia et al. 2018; Schmidt et al. in this volume)—particularly difficult to achieve because of the challenges arising from the endless and often questionable stream of information assessed in highly complex and dynamic communication systems. Two further developments complicate PL attitudes and outcomes: the increase in non-transparency and the spread of distorted information. Both erode societal trust in knowledgegenerating institutions as a whole and contribute to a concomitant rise in hostility toward science and in populist anti-intellectualism (Hocevar et al. 2014). Sampling of information for learning by monopolistic information access providers is not transparent. Students engaged in a heuristically guided search for trustworthy sources are unaware of the algorithms that providers use to generate their ranked selection of “hits”, and the biases and interferences that those hits may represent. Furthermore, in the unclear boundaries of online spaces, new (anonymous) intermediaries, gatekeepers, filter bubbles, or social bots can now pre-select and manipulate information presented to learners. In this complex, uncertain arena, attitudes demoting the value of scholarly rigor pose an acute threat to academic learning. Dissemination of untrustworthy information is to be expected in a pluralistic communication space, yet the greater its presence and influence, the more PL is at risk. This influence challenges academic learning that aims for critical, deliberative, and pluralistic use of information. Providing students with effective tools for fostering PL may operate against populism. Current misinformation scandals (e.g., Cadwalladr 2018) have generally illustrated citizens’ unawareness of the origin, distribution mechanisms, and flow of online information as well as its potential for intentional manipulation or unintentional bias (e.g., through customized Google search hits). While both researchers and developers have issued warnings that popular online social (and mass) media are designed to exploit mental weak points that may present judgmental traps or promote weak reasoning (e.g., Ciampaglia 2018; Oeberst et al. 2018), the effects of students’ use of new information and communication technologies (ICTs) for learning, even in formal higher education, have barely been studied to date. Undermining Mental Prerequisites for Information Processing. Audiovisual interactive media offer perceptually rich information, creating strongly engaging experiences (Kise in this volume). Online media providers have advanced possibilities to quantify users’ (micro)interaction behavior and gain detailed insights into

290

O. Zlatkin-Troitschanskaia et al.

human behavior. Although these possibilities can be used to develop ever-improved representations and optimize sequencing of learning content, information processing systems can also be exploited for business, political, or unethical purposes. The Internet economy relies on drawing and maintaining users’ attention. This has led to the creation of highly addictive, targeted interaction mechanisms such as attention grabbers and gratification loops. While industry leaders such as Google and Facebook have just recently pledged to adopt TimeWellSpent design principles, the exact implementation and the scope of the necessary shift in their business models remain unclear. In online information sources, students’ habitual expectancy of gratification is shorter than the time needed for critical deliberation (Kahneman 2011; Gigerenzer 2015), which may have long-term, yet under-investigated, negative effects on various parts of mental processing and on learning behavior. The Role of Academic Education in the Internet Age. As a generator and disseminator of warranted knowledge, a user of online information, and a developer of digital technologies, academic education confronts the challenges of NL in online learning in multiple ways: When anticipating the tasks and challenges that accompany the diversity and asymmetric distribution of information, academic institutions have reacted with additive measures (e.g., preliminary courses) rather than integrative measures in practice and in research. While these measures may address short-term deficits, in the long term, they are not far-reaching enough if academic institutions are expected to react in a proactive, scholarly sound way to frequently changing technological and social requirements and to historic challenges resulting in NL. To approach the challenges of learning from information found on the Internet, it is necessary first and foremost to describe adequately the initial situation of the learner in the online information arena. Overall, the radius and speed of the distribution of distorted and false information is much greater today and the dissemination mechanisms are not transparent. Students’ mental strategies for selecting, processing, and learning are often insufficient for what is demanded for effective understanding and participation in a complex and ever changing environment (Wineburg and McGrew 2016). When students are unable to recognize biased or false information on the Internet and use it to build knowledge, NL is to be expected. Epistemic beliefs and domainspecific (e.g., naïve) mental models act as filters to more advanced information processing and knowledge acquisition (Alexander et al. 2018; Alexander and List 2017; Alexander 1997). Contemporary online learning environments contribute dramatically to cognitive overload and cognitive dissonance (Hahnel et al. 2019b), increasing the danger that learners neglect complex, abstractly presented knowledge (e.g., in textbooks) and rely more on lower-quality information as may be encountered on familiar social media sites that is consistent with their beliefs and biases and easy to comprehend (Shrivastav and Hiltz 2013). Thus, learning in academic domains (“warranted facts” instead of “alternative facts”) is at risk.

21

What Can We Learn from Theoretical Considerations and Empirical Evidence

291

Theoretical Considerations on Learning in Higher Education Earlier developed learning theories such as deeper learning (Pellegrino and Hilton 2012), higher order learning (Paas et al. 2003), insightful learning (Köhler 1972), conceptual learning (Gagné 1985), connected learning (Gardner 1999, 2011), multimedia learning (Mayer 2005), domain learning (Alexander 1997), transformational learning (Meyer et al. 2010), and others (e.g., Bandura 2001; Bronfenbrenner 1995; Bruner 1966; Dewey 1997; Mislevy 1996; Skinner 1953; Snow 1989; Weinert 2001) have proven to be useful in traditional instruction. Various models used to explain academic learning in the past can now be used to map affective and cognitive dispositions and processes for adaptation to learning in different disciplines generally characterized by both interdisciplinary and subject-specific components (e.g., logical reasoning, domain-specific content). To explain and predict academic learning (validly and reliably), however, these mid-range theories must be further specified and integrated. Numerous theories such as systems theory (Luhmann 2012), heuristics and bounded rationality (Simon 1959; Gigerenzer and Todd 2001), conceptual change (Posner et al. 1982), naïve theories (Piaget 1972), conceptual understanding and misconceptions (Marton and Svensson 1979), expert-novice (Ericsson 2008), problem solving (Newell and Simon 1972), and interconnected building blocks for different concepts of learning already established in several disciplines (e.g., physics, economics) could be used as conceptual foundations to systematically structure the concepts related to academic learning, as described concisely below. Learning is generally based on the interaction between individual internal information representations and the external information representations contained in learning environments (Demetriadis 2004). Contexts including other students and multimedia determine these environments (e.g., Wikipedia articles, learning blogs, textbooks) are externally represented in interpersonal and multimodal formats including linguistic, numeric-mathematical, and spatial representations. Learning depends on individual dispositions (beliefs, attitudes, biases) and abilities (cognitive, motivational, emotional), with which a learner selects, reconfigures, and processes external information internally. Here, we refer to Mayer’s cognitive theory of multimedia learning and its specifications and assumptions (Mayer 2005). For instance, changes in information processing are expected as learners become more familiar with (domain-specific) content and representation formats. Learning is fundamentally a non-linear, intra-individual activity (level I) that is strongly influenced by what the individual brings to the learning situation and, according to Bronfenbrenner’s model (Bronfenbrenner 1977), also by those with whom the individual interacts (level II) and the environment where those interactions occur (level III) with its affordances and constraints on the learning process. Thus, we can further differentiate between these three levels of learning systems: The microlevel refers to the individual student, his/her prior knowledge in a learning situation (often in the form of a “mental model” or scheme), beliefs, and biases, and his/her uses of these as the basis for constructing new knowledge—either adding information (even if distortion is needed) or reconstructing the scheme (even if

292

O. Zlatkin-Troitschanskaia et al.

the result is inaccurate). At the mesolevel, different social dynamics can occur while building knowledge, for example, the individual convictions of the group leaders can significantly determine the selection, processing, and evaluation of information. At the macrolevel, context-specific learning environments influence information processing and knowledge building across individuals and groups. Even in formal academic education, where knowledge acquisition is guided by preselected sources (teachers), access to digital (learning) resources on the Internet leads to completely different learning scenarios and diverging intra- and inter-individual learning patterns (e.g., Dabbagh and Kitsantas 2012). Moreover, as learning often starts with an issue or problem, it creates emotional or motivational tension (e.g., interest or conflict) that initiates further development of situational and mental models. Thus, learning environments need to be analyzed to determine whether their components (e.g., conflicting information, clauses, phrases, pictures) are constructed to generate more or less affective situations that ultimately initiate intended learning and promote deeper learning (Pellegrino and Hilton 2012). Learning is a relatively permanent change in behavior that occurs over time (process). In accordance with fundamental theories of learning, we assume genuine learning in academic domains occurs when PL supersedes NL and students are aware of and vigilant about the risks of NL; we define PL and NL as follows: Positive learning is the acquisition of academically warranted conceptual, procedural, and transferable knowledge and understanding that has a long halflife and is in line with scientifically substantiated knowledge, subsumes ethical norms and moral values, and is flexible in adapting to the availability of new information in a certain age. Negative learning is the acquisition of erroneous conceptual and procedural knowledge and understanding from unwarranted information, which leads to faulty mental models and reasoning, and significant resources are needed to change or unlearn such misconceptions or biases. Learning can be intentional, unintentional, incidental, and occur more or less consciously. This is especially true for learning that occurs informally on mass and social media, which is currently dramatically increasing in formal education (e.g., Dabbagh and Kitsantas 2012; Mushtaq and Benraghda 2018). The notion of PL vs. NL refers to a result of a learning process. A desirable or undesirable result can be manifested in either correct or incorrect content knowledge or more or less elaborated epistemic beliefs, respectively. Such learning, in turn, functions as a filter when processing subsequent information and the acquisition of new knowledge. NL is resistant to change and indicates that students do not recognize the errors in understanding technical facts, concepts, and models, and are convinced that this incorrect knowledge is correct. In models of domain learning (Alexander et al. 2018; Alexander and List 2017; Alexander 1997), expertise (i.e., the amalgam of learning processes) in a study domain (e.g., economics, physics) is analytically considered as development in different phases. Here, several influencing factors are examined, for example, whether expertise develops differently depending on the domain or its sub-areas (e.g., subjects, topics, contents), learning objects (e.g., principles,

21

What Can We Learn from Theoretical Considerations and Empirical Evidence

293

concepts, arguments, rules), mental dispositions, and/or variable input conditions (e.g., prior knowledge, motivation, attitudes). Taking into account theories of multimodal learning (Mayer 2005), the model of domain learning provides a core conceptual framework for analyzing changes in learning processes in academic domains. Since domains are ever-evolving (Kuhn 1999), first, we can differentiate between well-defined and well-structured domain topics (e.g., “break-even point” in economics) as well as “ill-defined” domain topics (e.g., anticipating non-rational learning behavior). When students learn multimodally, as is characteristic of today’s academic education, their choice of information impacts their learning and depends on (quasi-)invariant disciplinary and cross-disciplinary core features, including, according to Goldman et al. (2016) framework, their epistemological beliefs of what constitutes trustworthy and sufficiently comprehensive information (epistemology), their mental processing patterns (inquiry practices/strategies of reasoning), the structuring of the object of investigation (concepts, themes, and frameworks), the form and nature of the externally represented information (representation/types of texts), and linguistic factors (discourse and linguistic structures). In addition to the disciplinary core principles (e.g., epistemology, research practices, overarching concepts, discourse and language structures), the analytical framework is complemented by the perspective of consequences of decision-making and acting that can be evaluated against ethical values and moral concepts for learning, referring to, for instance, conscious learning among individuals, implicit or explicit values conveyed by groups to individuals, and embedded in, for example, discipline-specific ethics in formal and informal teaching-and-learning environments. This perspective addresses the different value qualities in knowledge, skills, beliefs, and attitudes within and across learning systems and their interactions as the determinants of learning and its PL or NL outcomes; however, focus is limited to the “non-controversial” norms or ethics within the scholarly community such as trustworthiness and honesty and less consideration is given to the societal ethical, normative principals which also underlie knowledge. In online environments, learners mentally draw on specific patterns of reasoning and representation, and apply them to potentially incomplete or unreliable information from multimodal, information-rich contexts. When operationalized in a discipline-specific way, we can identify structures that indicate the potential positivity (or negativity) of learning (e.g., the number and quality of claims, backings, propositions). If students think quickly in situations where information requires careful analysis, this can lead to NL, as they can be easily duped by faulty or “nudging” information sources (Thaler 2009). If students recognize the need to think slowly and question their prior knowledge and beliefs to critically evaluate the implications of the information, they are likely to experience PL (Kahneman 2011). Various dynamic learner-centered actions and their axioms, for example, from research on heuristics, cognitive biases, and bounded rationality (Stanovich 2011; Gigerenzer 2015), thus, need to be specified and integrated into learning theories.

294

O. Zlatkin-Troitschanskaia et al.

Empirical Research Research on learning in the new Internet environment has been fragmented and has not systematically considered developments in information and communication technologies (ITC). Instead, it has been conducted almost exclusively from a pedagogical, psychological, technological, or philosophical perspective, usually in intradisciplinary approaches. Most studies focusing on higher education do not consider students’ use of new ICTs or their impact on domain learning. For the new research progam PLATO (for details, see Zlatkin-Troitschanskaia et al. 2018), we have included findings from various disciplines based on both large-scale data and small-scale data (mostly in experimental designs with small samples).

Sources of Information Used for Learning in Higher Education In accordance with studies on the general use of media by learners (e.g., in higher education: Persike and Friedrich 2016; in schools: Feierabend et al. 2017), recent studies (Maurer et al. in this volume; Mushtaq and Benraghda 2018; Jitomirski et al. 2019) show that higher education students acquire domain-specific and generic knowledge, understanding, skills, beliefs, and attitudes not only through formal learning at university, including from lectures, tutorials, and “didactically approved” learning and instructional materials such as textbooks and scripts, but also through informal and partly unintentional learning from innumerable, easily accessible online sources such as online encyclopedias (e.g., Wikipedia articles), video platforms (e.g., YouTube), or other social media. Based on surveys of media use for academic learning, most of bachelor students prepare for exams using online media (e.g., Wikipedia articles) instead of lecture notes and textbooks (Maurer et al. in this volume). Initial studies indicate both the positive and negative effects of online media use on the acquisition of interdisciplinary and domain-specific knowledge over the course of university studies (Jitomirski et al. 2019; Gadiraju et al. 2018).

Knowledge Development in Higher Education Generally, information becomes knowledge only when integrated into the cognitive system and connected to existing knowledge elements (Alexander 1997). In this sense, the acquisition and development of further domain-specific knowledge may be jeopardized even if students have only one element of troublesome and erroneous knowledge in their mental model (Meyer et al. 2010). We defined the acquisition of erroneous knowledge, faulty mental models and biases without sufficient intention or ability to change them as negative learning (NL). Exemplary empirical evidence of NL includes a decrease in accurate domain-specific knowledge over the course of studies in higher education (Fig. 21.1) (e.g., Schmidt et al. in this volume; Happ et al.

21

What Can We Learn from Theoretical Considerations and Empirical Evidence

295

Fig. 21.1 Intra-individual differences (left) and inter-institutional differences in economics knowledge change over the course of bachelor studies (right) (Happ et al. 2016; Schmidt et al. 2016; Zlatkin-Troitschanskaia et al. 2013, 2018; for US, see Walstad and Wagner 2016; Schmidt et al. in this volume)

2016) or an increase in the acquisition of counterfactual misconceptions and false disciplinary concepts (e.g., Brückner 2017). NL might be facilitated by the fact that students struggle to handle easily accessible, growing, quasi unlimited information bases (Wineburg et al. 2018). For instance, analyses of university curricula and learning materials used in Germany and the USA for teaching and learning domainspecific core concept and models in economics, such as “scarcity” or “supply and demand,” show that higher education students are exposed to many scientifically backed concepts, models, and evidence-based findings as well as plentiful unwarranted or conflicting information in various less curated online media (Brückner et al. 2015; Zlatkin-Troitschanskaia et al. 2015). Current models of learning do not offer sophisticated explanations for NL outcomes such as decreasing domain-specific knowledge. Generally, factors related to higher education alone usually explain only 15–25% of variance in results on standardized assessments in large-scale studies (see, e.g., the two rows focused on Pseudo-R2 in Table 21.1). While these results are staggering from an educational point of view, the impact of non-educational factors on learning has rarely been examined from other disciplinary perspectives. Assessment research has indicated that, despite thorough validation, at least an equally large portion (if not larger) of variance in students’ learning outcomes is correlated with other aspects, for instance, language features such as linguistic structures and their complexity (Mehler et al. 2018; Bisang and Czerwinski in this volume; Allalouf 2019). Representationally more complex and cognitively more demanding information is also associated with remarkably higher NL values than less complex and less demanding information (Kise in this volume; Schmidt et al. in this volume). Overall, these findings provide an insight into what students have learned (or have not learned) over the course of their studies. However, we do not know why students have learnt or forgotten the contents of the subject and what the major predictors are that can explain these learning patterns. So far, no definite explanatory factors have been determined, and even the general question—why such learning

296

O. Zlatkin-Troitschanskaia et al.

Table 21.1 Explained variance in change of economics knowledge over bachelor studies (Förster et al. 2015; Happ and Zlatkin-Troitschanskaia 2014; Zlatkin-Troitschanskaia et al. 2016) Parameter Fixed effects Intercept Commercial VET (1 ¼ yes) Gender (1 ¼ male) Mother tongue (1 ¼ not German) Final school grade Commercial upper secondary school attended Finance course completed Mathematics course completed Number of semesters Type of institution (1 ¼ University)

Pseudo-R2 (Institutional) Pseudo-R2 (Personal)

Model 1

Model 2

Model 3

Model 4

Model 5


0.244 –


0.309 0.276


0.485 0.327


0.789 0.329


0.821 0.315

– –

– –

0.299
0.105

0.299
0.114

0.294
0.118

– –

– –


0.142 0.200


0.165 0.188


0.166 0.186

– –

– –

– –

0.187 0.187

0.207 0.176

– –

– –

– –

0.021 –

0.017 0.138

0.022 0.295 – –

0.017 0.284 0.223 0.037

0.016 0.251 0.280 0.150

0.000 0.230 0.993 0.220

0.000 0.229 0.999 0.222

Note: p  0.05, p  0.01, p  0.001

patterns such as NL occur in the first place, given that all university students usually have above-average cognitive dispositions such as intelligence—remains unsolved. To investigate this question, the focus should be not only on learning outcomes (such as student responses to domain-specific tests), but especially on the learning processes, particularly online. For example, we need real-time process data of students’ online search behavior and processing of online information when they are working on domain-specific tasks.

Online Reasoning and Critical Handling of Online Information In theory, the Internet enables unrestricted individual and collaborative knowledgebuilding from multiple sources, which approximates ideal conditions of democratic knowledge generation. In practice, Internet users bear the responsibility of judging the quality of information based on academic standards, and handling information from partly conflicting sources. Recent studies indicate that higher education students (both undergraduates and graduates) are not sufficiently prepared, literate, or disposed to critically judge the credibility of information or to deal with conflicting information in particular.

21

What Can We Learn from Theoretical Considerations and Empirical Evidence

297

This became evident, for example, in a study with a sample of 7804 students from 12 US states who were asked to critically assess the credibility of information (Wineburg et al. 2018). The latest studies with undergraduate and graduate students in Germany show similar findings (Hahnel et al. 2019b; Shavelson et al. 2019; Zlatkin-Troitschanskaia et al. 2019b). Overall, studies reveal that most university students do not have sufficient skills to autonomously select and consciously process online information and are vulnerable to being misled and to unwittingly acquiring false knowledge through biased information. In the Internet Age, this vulnerability arises as most students tend to rely on the first few hits presented in an online search and struggle to filter, process, evaluate, and synthesize vast amounts of information (Gadiraju et al. 2018; Kammerer and Gerjets 2012; Wineburg and McGrew 2016). Indeed, the studies show that students seek information that confirms their prior beliefs and not (previous) knowledge (Zlatkin-Troitschanskaia et al. 2019b; Von der Mühlen et al. 2015, 2016). Think-aloud studies found deficits in perception and classification of evidence-based argumentation (Von der Mühlen 2015, 2016). Results point out that students of psychology have deficits in their use of sources for evaluating the credibility of multiple texts as well as in accurately judging the plausibility of arguments and recognizing common argumentation fallacies. Moreover, students often relied on their intuition or opinion alone regarding the acceptability of a claim. The findings indicate that, although students do validate scientific information against their knowledge and beliefs, their judgments are often erroneous, in part because their strategies are immature (Von der Mühlen 2015, 2016). Overall, the findings indicate that both bachelor and master students have insufficient skills when it comes to critically handling information, even at the end of their studies (Fig. 21.2). Overall, previous research indicates that NL becomes especially problematic when unwarranted or misleading information is unwittingly used as the basis for further action such as information selection, knowledge building, and professional or civic decision-making. Preliminary studies indicate that both domain-specific misconceptions and individual beliefs can lead to selective use of information, as information that does not correspond to previous knowledge or beliefs is ignored in decision-making and action in the domain-specific task, resulting in incorrect

Fig. 21.2 Critically dealing with multiple, partly conflicting sources of information (Shavelson et al. 2019; Zlatkin-Troitschanskaia et al. 2019b). Note: 1¼lowest performance, 6¼highest performance

298

O. Zlatkin-Troitschanskaia et al.

solutions (Brückner and Zlatkin-Troitschanskaia 2018; Klein et al. 2019; Van Lehn and van de Sande 2009).

Selective and Erroneous Visual Perception NL is difficult to avoid, as it often occurs unintentionally and unconsciously. When learners are convinced that their knowledge is correct and factual and do not realize that they have acquired a misconception, for example, in economics, this can lead to reasoning errors in tasks tapping the corresponding concept (Brückner and Zlatkin-Troitschanskaia 2018; Kise in this volume). In addition to textual representations, graphic representations in a variety of media play a significant role. The effects of NL can be observed in eye-tracking studies investigating the change in domain-specific knowledge among students in physics and economics, for example, in the analysis of diagram progressions, axis labels, extreme values, scaling (Elkasrawi et al. 2016; Klein et al. 2019; see also Kise in this volume). Before each analysis, areas of interest (AOI) are defined which, from the point of view of the researcher, reflect the construct-relevant (e.g., rotation of figures to solve problems in a mental rotation test) and construct-irrelevant areas (e.g., guessing to solve tasks in a mental rotation test) (Messick 1989) for solving tasks. The assumptions for the construct(ir)relevance result from the construct and test definition (Mislevy and Haertel 2006) and are not trivial; they require a comprehensive knowledge of the construct to be measured (e.g., domain-specific knowledge in physics) and its operationalization (for knowledge in business and economics, see ZlatkinTroitschanskaia et al. 2014). For NL, it can be assumed that students who look more often at construct-relevant AOIs direct their attention to potential misconceptions and are therefore more sensitive to NL. Figure 21.3 shows different areas which can be marked as relevant (blue) and irrelevant (red) AOIs by comparing two graphs, A and B. In future studies, it will be necessary to investigate reasons for NL and to systematically evaluate texts, pictures, graphics, signals, sounds, etc. with regard to their construct(ir)relevance. Some studies indicate that such effects on NL are correlated both to the level of expertise in a domain (e.g., number of study semesters) and to the level of self-confidence (Klein et al. 2019; Kise in this volume). Further studies should also examine the significance of additional factors such as reading skills, level of attention, specific task, and/or information characteristics as well as domain-specific influences.

Cognitive Interviews Based on results of cognitive laboratory experiments (CogLabs), undergraduate and graduate students sometimes seek to confirm what they already know and believe

21

What Can We Learn from Theoretical Considerations and Empirical Evidence

299

Fig. 21.3 Heat map of students’ visual perception of non-textual representations (possible construct-relevant AOIs in blue; construct-irrelevant AOIs in red)

instead of critically questioning their previous knowledge and beliefs in cases where their knowledge is perceived as insufficient to solve a task (e.g., Brückner 2017). Both domain-specific misconceptions and individual beliefs can lead to a selective use of information, resulting in NL (Brückner and Zlatkin-Troitschanskaia 2018). Studies suggest that while students may be familiar with certain domain-specific content, they tend to become insecure when solving concrete tasks. This tension between familiarity and certainty may vary from subject to subject, but often indicates greater underlying difficulty in understanding and is partially based on the formal (e.g., school) and informal (e.g., domain-specific newspapers) learning opportunities as well as the personality of the student. For example, a study on students’ economic knowledge indicated that with regard to certain subject concepts, students rated their familiarity with a concept higher than their certainty in applying it appropriately in a certain context (Confidence) (Brückner and ZlatkinTroitschanskaia 2018; Land 2016) (Fig. 21.4). It also became evident that this difference significantly affects the item solution process and correlates with students’ NL. What remains open, however, is how the long-term development of knowledge is affected by the familiarity-confidence gap.

300

O. Zlatkin-Troitschanskaia et al.

Fig. 21.4 Confidence and familiarity of university students solving tasks in business and economics (Brückner and Zlatkin-Troitschanskaia 2018)

Summary In summary, NL was initially measured among students as a decrease in accurate domain-specific knowledge in business and economics over the course of studies in higher education (e.g., Fig. 21.4). In the meantime, we have found further evidence of NL in other forms of academic learning outcomes such as generic skills (e.g., online reasoning and critical handling of online information, Zlatkin-Troitschanskaia et al. 2019b), and expect to find more evidence of NL when examining students’ beliefs (e.g., epistemic orientation), and attitudes (on effects of attitudes on well-being, see Gardner 2018; Dormann and Guthier in this volume). Finally, if we conceive academic performance as action based on a continuum of competence, which includes both individual dispositions (e.g., knowledge and intelligence) and their situational application (e.g., situation-specific skills) acquired in higher education (Zlatkin-Troitschanskaia et al. 2019a) then NL could lead to unfounded, incompetent, or professionally or civically unsuitable action (see also Oser and Biedermann in this volume). Overall, much is known about information processing under laboratory conditions, but not in agile online learning environments. For instance, we do not know

21

What Can We Learn from Theoretical Considerations and Empirical Evidence

301

how learners deal with conflicts between new online information and previously acquired knowledge or assumptions about a domain; i.e., detecting inconsistencies and awareness of the necessity to resolve the inconsistencies to obtain or maintain a consistent set of beliefs. The underlying cognitive principles are only partially understood for simple instructional and learning situations; in contrast, Internetsupported learning is a complex phenomenon that occurs in a minimally controlled setting where intentional misinformation can be expected and should be investigated. The multitude of already existing data (e.g., from the aforementioned largeand small-scale studies) has to be linked to draw a coherent picture of Internet learning. However, this also requires structure-exploring procedures that can show correlations in the comprehensive databases (e.g., between the results from eye-tracking data, curricula, and Internet log files). In particular, the use of learning analytics, AI-supported algorithms, as well as inner- and cross-disciplinary collaboration in the research of Internet learning are central components of PLATO. Yet, despite many current speculations in policy and practice, the influence of available media, especially social and mass media, the characteristics of these media, and the reasons for students to use unwarranted rather than warranted information have hardly been investigated. In addition, an important question will be to what extent the perception of these multimodally represented media (e.g., through diagrams and texts on websites) (Subrahmanyam et al. 2014) can be described, promoted, and predicted across disciplines through features in the curricula. The efficient perception and critical exploration of these media as part of digital literacy will become increasingly important, but many curricula do not yet adequately implement this (Working Group Curricula 4.0 2018). At the same time, this complicates research into the phenomena, since many still implicitly take place in informal situations outside the university.

Integration and Further Specification of Learning Models in an Interdisciplinary Research Framework Considering learning theories and current research, we propose a framework that combines several fragmented strands of research in an integrated model to describe and explain online learning in modern academic education and its positive or negative results (PL/NL). On the basis of the abovementioned findings from largeand small-scale assessments, it becomes evident that while there are various approaches to research NL, at the same time, approaches must be chosen that are sufficiently explorative, process-oriented, real-time, and comprehensively represent the multiple facets of the learner and their interaction with a particular learning medium and information representation. This goal accounts for the wide range of contributing disciplines. Ultimately, learning is the constructive mental process of acquiring new knowledge and beliefs, integrating them into different memory systems, and recalling them in later use

302

O. Zlatkin-Troitschanskaia et al.

(Alexander 1997). At an interpersonal level, learning is a social phenomenon, as knowledge and beliefs are shared by groups or institutions (Vergès and Ryba 2012). As learning processes occur on different and multiple time scales, the description and explanation of learning requires time-differentiated analyses, tracing effects over time and in real time. The analysis of academic learning and its results at these various levels requires a combination of approaches that encompass, besides content and didactic analyses in a certain domain (e.g., economics, physics), empirical and experimental methods from education, social science, psychology, cognitive sciences, communication and media studies, linguistics and intersecting disciplines (e.g., neurolinguistics), narrative analyses from cultural studies, and algorithmic and computational methods from mathematics and computer science, including artificial intelligence (AI). In this framework, we propose to examine the same learning subjects and objects and underlying processes based on unified data pool, at various degrees of granularity and levels of temporal resolution, taking empirically and theoretically driven approaches, employing established disciplinary methods and developing new interdisciplinary theories and models to explain and predict NL/PL. As a result, researchers from all involved disciplines can focus on a common research object: academic learning, which occurs intra- or inter-individually in interactive, (non-) cooperative environments and includes digital and non-digital learning sources. Researchers should model various determinants of learning (e.g., cognitive skills, linguistic, narrative, visual, content structures) in these complex and dynamic learning environments as well as their positive or negative effects and take (cross-) disciplinary and mixed-methods approaches to analyze them separately and in their interactions. For instance, one and the same Wikipedia article on an economic topic may be evaluated narratively by literature experts, linguistically by text mining experts, technically and didactically by economics education experts, and contentanalytically by media and communication experts. Students’ information processing when reading the article is recorded in log files of their Internet search behavior and using eye tracking during real-time Internet use by psychologists and learning analytics experts, while sociologists use network modeling methods to examine the discourse on this text in a learning group. These data are used by AI researchers to train diagnostic or predictive systems as well as for comparative analyses by experts from other disciplines (e.g., physics). Depending on the modeling approaches, the research focus would be on (1) academic learning processes or results, (2) their relation to internal (mental) or external (physical/digital) information representations, (3) at one, two, or across multiple levels, (4) at different and multiple time scales (in real time, over longer periods of time), and (5) in specific domains, starting with two curricularly “welldefined” domains (e.g., economics and physics), then, progressively including mathematics, computer science, and medicine and expanding to less well-defined domains such as sociology. To explore the theoretically expected interactions and (causal) relations among (1)–(5), the framework can be complemented by a panel study over the course of

21

What Can We Learn from Theoretical Considerations and Empirical Evidence

303

bachelor and/or master degree programs and by a technological project to integrate (big) educational data computationally, enabling cross-format analyses of the above facets; both will help develop predictive models of student learning. The coordinated interdisciplinary research cooperation, building on the current preliminary studies, theoretically and methodologically incorporates the latest findings and expertise in a given field (e.g., linguistics) into those of other fields (e.g., economics, physics education) and vice versa, results in a continuously innovating research program.

Conclusion The dramatic historical developments and new phenomena described briefly above highlight the learning and teaching challenges that have arisen in recent years and are likely to expand over time. So far no systematic research has been conducted on these new teaching-learning environments or the mechanisms of learning in these environments; only general research on digital literacy and the development of new digital technologies has been conducted (Steffens et al. 2017). Unique to this era is that there are no beginning students in any discipline without years of informal learning and knowledge gained on the Internet, which by its nature does not observe disciplinary boundaries, that is, at least a quantitative difference, which has become qualitative. Misconceptions are nothing new, but these days they are far more entrenched and thus harder to eliminate.1 Established theories and concepts aiming to explain, predict, or even influence academic learning stem from an era in which learning was primarily institutionalized, medially and technologically limited, highly disciplinary, and characterized by minor variations in teaching methodology. To address proactively and effectively the challenges posed by Internet learning, we urgently need to integrate theories and models that adequately describe and explain academic learning in ever-changing digital learning environments. In PLATO, we examine how conditions of the Internet Age lead to PL and NL, including analyses of relationships between students’ use of ICTs and PL and NL to explain recent empirically observed phenomena such as NL curves over the course of higher education studies. We aim to determine how NL can be avoided or transformed into PL. Recognizing that a significant part of domain-relevant learning happens outside of controlled formal education, PLATO aims to increase students’ awareness and autonomy in responsibly handling their learning process and outcomes. Based on the proposed interdisciplinary research framework, we seek to teach them to become aware of unintentionally acquired knowledge and be strategic in their acquisition of domain knowledge and (epistemic) beliefs in line with both discipline-specific warrants and moral and ethical reflections. PLATO aims

1

We would like to thank Howard Gardner for calling out awareness to this aspect.

304

O. Zlatkin-Troitschanskaia et al.

to deliver scientifically sound and evidence-based learner-centered education that meets academic quality requirements in and across disciplines, is abreast of technological advances, and integrates multidisciplinary findings on learning fundamentals. Acknowledgements. We thank all PLATO colleagues and researchers for their valuable input to this contribution, in particular, Patricia Alexander, Michael Posner, Richard Shavelson, and William Walstad.

References Alexander, P. A. (1997). Mapping the multidimensional nature of domain learning: The interplay of cognitive, motivational, and strategic forces. In M. L. Maehr & P. R. Pintrich (Eds.), Advances in motivation and achievement (Vol. 10, pp. 213–250). Greenwich, CT: JAI. Alexander, P., & List, A. (2017). Analyzing and integrating models of multiple text comprehension. Educational Psychologist, 52(3), 143–147. https://doi.org/10.1080/00461520.2017.1328309 Alexander, P., Murphy, K., & Sun, Y. (2018). Knowledge and belief change in academic development. In H. Fives & D. Dinsmore (Eds.), The model of domain learning. Understanding the development of expertise. New York: Routledge Taylor & Francis. Allalouf, A. (2019). Thirty years of research on the sources of DIF in translated tests. Presentation at the NCME, Toronto, 4–8 April. Arbeitsgruppe Curriculum 4.0. (2018). Curriculumentwicklung und Kompetenzen für das digitale Zeitalter – Thesen und Empfehlungen der AG Curriculum 4.0 des Hochschulforum Digitalisierung (Arbeitspapier: Vol. 39). Berlin: Hochschulforum Digitalisierung. Bandura, A. (2001). Social cognitive theory of mass communication. Media Psychology, 3(3), 265–299. Behrens, J. T., Mislevy, R. J., DiCerbo, K. E., & Levy, R. (2010). An evidence center design for learning and assessment in the digital world. CRESST Report 778. Los Angeles: UCLA-National center for Research on Evaluation, Standards, and Student Testing. Retrieved August 25, 2016, from http://files.eric.ed.gov/fulltext/ED520431.pdf Bernstein, D. M., Erdfelder, E., Meltzoff, A. N., Peria, W., & Loftus, G. R. (2011). Hindsight bias from 3 to 95 years of age. Journal of Experimental Psychology, Learning, Memory, and Cognition, 37(2), 378–391. Bronfenbrenner, U. (1977). Toward an experimental ecology of human development. American Psychologist, 32(7), 513–531. https://doi.org/10.1037/0003-066X.32.7.513 Bronfenbrenner, U. (1995). Ecological models of human development. In T. Husén (Ed.), The international encyclopedia of education (2nd ed., pp. 1643–1647). Oxford: Pergamon. Brooks, D. C. (2016). ECAR Study of Undergraduate Students and Information Technology. Louisville. Retrieved from https://library.educause.edu/~/media/files/library/2016/10/ers1605.pdf Brückner, S. (2017). Prozessbezogene Validierung anhand von mentalen Operationen bei der Bearbeitung wirtschaftswissenschaftlicher Testaufgaben. Landau: Empirische Pädagogik. Brückner, S., & Zlatkin-Troitschanskaia, O. (2018). Threshold concepts for modeling and assessing higher education students’ understanding and learning in economics. In O. Zlatkin-Troitschanskaia, M. Toepper, H. A. Pant, C. Lautenbach, & C. Kuhn (Eds.), Assessment of learning outcomes in Higher Education: Cross-National comparisons and perspectives (pp. 103–121). Basel: Springer International. Brückner, S., Förster, M., Zlatkin-Troitschanskaia, O., Happ, R., Walstad, W. B., Yamaoka, M., et al. (2015). Gender effects in assessment of economic knowledge and understanding:

21

What Can We Learn from Theoretical Considerations and Empirical Evidence

305

Differences among Undergraduate Business and Economics Students in Germany, Japan, and the United States. Peabody Journal of Education, 90(4), 503–518. Bruner, J. S. (1966). Toward a theory of instruction. New York: W.W. Norton. Cadwalladr, C. (2018, March 18). ‘I made Steve Bannon’s psychological warfare tool’: meet the data war whistleblower. The Guardian. Retrieved from https://www.theguardian.com/news/ 2018/mar/17/data-war-whistleblower-christopher-wylie-faceook-nix-bannon-trump Cheon, J., Lee, S., Crooks, S. M., & Song, J. (2012). An investigation of mobile learning readiness in higher education based on the theory of planned behavior. Computers & Education, 59(3), 1054–1064. Ciampaglia, G. (2018). The digital misinformation pipeline. In O. Zlatkin-Troitschanskaia, G. Wittum, & A. Dengel (Eds.), Positive learning in the age of information (pp. 413–422). Wiesbaden: Springer. Dabbagh, N., & Kitsantas, A. (2012). Personal learning environments, social media, and selfregulated learning: A natural formula for connecting formal and informal learning. The Internet and Higher Education, 15(1), 3–8. Demetriadis, S. (2004). Interaction between learner’s internal and external representation in multimedia environment: A state-of-the-art. Research report of the Kaleidoscope Network of Excellence (IST 507838), Deliverable 21.1.1. State-of-the-art report, Kaleidoscope Network of Excellence. Dewey, J. (1997). Experience and education. The Kappa Delta Pi Lecture Series. New York: Simon & Schuster. Elkasrawi, S., Abdelsamad, A., Bukhari, S. S., & Dengel, A. (2016). What you see is what you get? Automatic image verification for online news content. In: DAS IAPR International Workshop on Document Analysis Systems (DAS-2016), 12th IAPR International Workshop on Document Analysis Systems, located at DAS, April 11–14, Santorini, Greece, IEEE-Xplore digital library. Ericsson, K. A. (2008). Deliberate practice and acquisition of expert performance: A general overview. Academic Emergency Medicine, 15(11), 988–994. https://doi.org/10.1111/j.15532712.2008.00227.x Feierabend, S., Plankenhorn, T., & Rathgeb, T. (2017). JIM 2017 Jugend, Information, (Multi-) Media: Basisstudie zum Medienumgang 12- bis 19-Jähriger in Deutschland. Stuttgart. Förster, M., Brückner, S., & Zlatkin-Troitschanskaia, O. (2015). Assessing the financial knowledge of university students in Germany. Empirical Research in Vocational Education and Training, 7(6), 1–20. https://doi.org/10.1186/s40461-015-0017-5 Fossland, T. (2015). Moving beyond learning in net-based higher education? In Academic bildung in net-based higher education: Moving beyond learning (pp. 19–30). Abingdon: Routledge. Gadiraju, U., Yu, R., Dietze, S., & Holtz, P. (2018). Analyzing knowledge gain of users in informational search sessions on the web. In C. Shah, N. J. Belkin, K. Byström, J. Huang, & F. Scholer (Eds.), CHIIR’18: Proceedings of the 2018 Conference on Human Information Interaction & Retrieval: March 11-15, 2018, New Brunswick, NJ, USA (pp. 2–11). New York, NY: The Association for Computing Machinery. https://doi.org/10.1145/3176349. 3176381 Gagné, R. M. (1985). The conditions of learning and theory of instruction. New York: CBS College Publishing. Gardner, H. (1999). Intelligence reframed: Multiple intelligences for the 21st century. New York: Basic Books. Gardner, H. (2011). Frames of mind: The theory of multiple intelligences. New York: Basic Books. Gardner, H. (2018). Higher education: A Platonic ideal. In O. Zlatkin-Troitschanskaia, G. Wittum, & A. Dengel (Eds.), Positive learning in the age of information (pp. 7–22). Wiesbaden: Springer. Gigerenzer, G. (2015). Simply rational: Decision making in the real world. Oxford series in evolution and cognition. New York: Oxford University Press. Gigerenzer, G., & Todd, P. M. (2001). Simple heuristics that make us smart: Evolution and cognition. Oxford: Oxford University Press.

306

O. Zlatkin-Troitschanskaia et al.

Gikas, J., & Grant, M. M. (2013). Mobile computing devices in higher education: Student perspectives on learning with cellphones, smartphones & social media. The Internet and Higher Education, 19, 18–26. Goldman, S. R., Britt, M. A., Brown, W., Cribb, G., George, M., Greenleaf, C., et al. (2016). Disciplinary literacies and learning to read for understanding: A conceptual framework for disciplinary literacy. Educational Psychologist, 51(2), 219–246. https://doi.org/10.1080/ 00461520.2016.1168741 Hahnel, C., Kröhne, U., Goldhammer, F., Schoor, C., Mahlow, N., & Artelt, C. (2019a). Validating process variables of sourcing in an assessment of multiple document comprehension. British Journal of Educational Psychology. https://doi.org/10.1111/bjep.12278 Hahnel, C., Schoor, C., Kröhne, U., Goldhammer, F., Mahlow, N., & Artelt, C. (2019b). The role of cognitive load for university students’ comprehension of multiple documents. Zeitschrift für Pädagogische Psychologie, 33, 105–118. https://doi.org/10.1024/1010-0652/a000238. Happ, R., & Zlatkin-Troitschanskaia, O. (2014). Assessing the quality of the Bachelor Degree Courses: An analysis of their effects’ on students acquisition of economic content knowledge. Zeitschrift für Hochschulentwicklung, 9(2), 64–77. Happ, R., Zlatkin-Troitschanskaia, O., & Schmidt, S. (2016). An analysis of economic learning among undergraduates in introductory economics courses in Germany. The Journal of Economic Education, 47(4), 300–310. https://doi.org/10.1080/00220485.2016.1213686 Hocevar, K. P., Flanagin, A. J., & Metzger, M. J. (2014). Social media self-efficacy and information evaluation online. Computers in Human Behavior, 39, 254–262. https://doi.org/10.1016/j.chb. 2014.07.020 Jitomirski, J., Zlatkin-Troitschanskaia, O., Schlax, J., Kühling-Thees, C., Happ, R., Schipolowski, S., et al. (2019). The relation between general intellectual ability and studyrelated media use among University Students in Economics. Paper presented at the American Education Research Association (AERA) Annual Meeting 2019 in Toronto, Canada. Kahneman, D. (2011). Thinking, fast and slow. New York: Farrar, Straus and Giroux. Kammerer, Y., & Gerjets, P. (2012). Effects of search interface and Internet-specific epistemic beliefs on source evaluations during Web search for medical information: An eye-tracking study. Behaviour & Information Technology, 31(1), 83–97. https://doi.org/10.1080/0144929X. 2011.599040 Klein, P., Küchemann, S., Brückner, S., Zlatkin-Troitschanskaia, O., & Kuhn, J. (2019). Student understanding of graph slope and area under a curve: A replication study comparing first year physics and economic students. Physical Review of Physics Education Research, 14, 020109. Köhler, W. (1972). The task of Gestalt psychology. Princeton: Princeton University Press. Kuhn, D. (1999). A developmental model of critical thinking. Educational Researcher, 28(2), 16–46. https://doi.org/10.3102/0013189X028002016 Land, R. (2016). TOIL AND TROUBLE: Threshold concepts as a pedagogy of uncertainty. In R. Land, J. H. F. Meyer, & M. T. Flanagan (Eds.), Threshold concepts in practice (pp. 11–24). Rotterdam: Sense. Lazer, D. M. J., Baum, M. A., Benkler, Y., Berinsky, A. J., Greenhill, K. M., Menczer, F., et al. (2018). The science of fake news. Science, 359(6380), 1094–1096. https://doi.org/10.1126/ science.aao2998 Luhmann, N. (2012). Introduction to systems theory. Cambridge: Polity. Lynch, M. P. (2016). Googling is believing: Trumping the informed citizen. New York Times. Retrieved from http://opinionator.blogs.nytimes.com/2016/03/09/googlingis-believingtrumping-the-informed-citizen/ Marton, F., & Svensson, L. (1979). Conceptions of research in student learning. Higher Education, 8(4), 471–486. https://doi.org/10.1007/BF01680537 Mayer, R. E. (2005). Cognitive theory of multimedia learning. In R. E. Mayer (Ed.), The Cambridge handbook of multimedia learning. New York: Cambridge University Press. Mehler, A., Zlatkin-Troitschanskaia, O., Hemati, W., Molerov, D., Lücking, A., & Schmidt, S. (2018). Integrating computational linguistic analysis of multilingual learning

21

What Can We Learn from Theoretical Considerations and Empirical Evidence

307

data and educational measurement approaches to explore learning in higher education. In O. Zlatkin-Troitschanskaia, G. Wittum, & A. Dengel (Eds.), Positive learning in the age of information (pp. 145–196). Wiesbaden: Springer. Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 13–103). New York: Macmillan. Meyer, J., Land, R., & Baillie, C. (2010). Threshold concepts and transformational learning. Educational futures: rethinking theory and practice. Rotterdam: Sense. Mislevy, R. J. (1996). Test theory reconceived. Journal of Educational Measurement, 33(4), 379–416. https://doi.org/10.1111/j.1745-3984.1996.tb00498.x Mislevy, R. J., & Haertel, G. D. (2006). Implications of evidence centered design for educational testing. Educational Measurement: Issues and Practice, 25(4), 6–20. Mushtaq, A. J., & Benraghda, A. (2018). The effects of social media on the Undergraduate Students’ Academic Performance. Library Philosophy and Practice. Newell, A., & Simon, H. A. (1972). Human problem solving. Englewood Cliffs: Prentice-Hall. Newman, F., & Scurry, J. E. (2015). Higher education and the digital Rapids. International Higher Education, (26). https://doi.org/10.6017/ihe.2002.26.6968 Oeberst, A., de Vreeze, J., & Cress, U. (2018). The norm of neutrality in collaborative knowledge construction. In O. Zlatkin-Troitschanskaia, G. Wittum, & A. Dengel (Eds.), Positive learning in the age of information (pp. 209–220). Wiesbaden: Springer. Paas, F., Renkl, A., & Sweller, J. (2003). Cognitive load theory and instructional design: Recent developments. Educational Psychologist, 38(1), 1–4. https://doi.org/10.1207/ S15326985EP3801_1 Pellegrino, J. W., & Hilton, M. L. (2012). Education for life and work: Developing transferable knowledge and skills in the 21st century. Washington, D.C.: National Academies Press. https:// doi.org/10.17226/13398 Persike, M., & Friedrich, J. D. (2016). LERNEN MIT DIGITALEN MEDIEN AUS STUDIERENDENPERSPEKTIVE: Sonderauswertung aus dem CHE Hochschulranking für die deutschen Hochschulen. Arbeitspapier: [LERNING WITH DIGITAL MEDIA FROM A STUDENT PERSPECTIVE: Special analysis of the CHE higher education institution tanking for German HEIs. Working paper:] 17. Berlin: Hochschulforum Digitalisierung. Piaget, J. (1972). The psychology of the child. New York: Basic Books. Posner, G. J., Strike, K. A., Hewson, P. W., & Gertzog, W. A. (1982). Accommodation of a scientific conception: Toward a theory of conceptual change. Science Education, 66(2), 211–227. Schmidt, S., Zlatkin-Troitschanskaia, O., & Fox, J.-P. (2016). Pretest-Posttest-Posttest multilevel IRT modeling of competence growth of students in Higher Education in Germany. Journal of Educational Measurement, 53(3), 332–351. https://doi.org/10.1111/jedm.12115 Shavelson, R. J., Zlatkin-Troitschanskaia, O., & Mariño, J. P. (2018). International performance assessment of learning in higher education (iPAL): Research and development. In O. Zlatkin-Troitschanskaia, M. Toepper, H. A. Pant, C. Lautenbach, & C. Kuhn (Eds.), Assessment of learning outcomes in higher education: Cross-national comparisons and perspectives (Vol. 66, pp. 193–214). Cham: Springer. https://doi.org/10.1007/978-3-31974338-7_10 Shavelson, R. J., Zlatkin-Troitschanskaia, O., Beck, K., Schmidt, S., & Marino, J. P. (2019). Assessment of University Students’ critical thinking: Next generation performance assessment. International Journal of Testing, 37(9), 1–20. Shrivastav, H., & Hiltz, S. R. (2013). Information overload in technology-based education: A metaanalysis. Americas Conference on Information Systems, Chicago, Illinois. Simon, H. A. (1959). Theories of decision-making in economics and behavioral science. The American Economic Review, 49(3), 253–283. Skinner, B. F. (1953). Science and human behavior. New York: Free Press. Snow, R. E. (1989). Cognitive-conative aptitude interactions in learning. In R. Kanfer, P. I. Ackermann, & R. Cudeck (Eds.), Abilities, motivation, and methodology (pp. 435–474). Hillsdale, NJ: Lawrence Erlbaum.

308

O. Zlatkin-Troitschanskaia et al.

Somerville, M. M., Smith, G. W., & Smith Macklin, A. (2008). The ETS iSkills TM Assessment: A digital age tool. The Electronic Library, 26(2), 158–171. Stanovich, K. E. (2011). Rationality and the reflective mind. New York: Oxford University Press. Steffens, Y., Schmitt, I. L., & Aßmann, S. (2017). Mediennutzung Studierender: Über den Umgang mit Medien in hochschulischen Kontexten. Systematisches Review nationaler und internationaler Studien zur Mediennutzung Studierender. https://doi.org/10.13154/rub.106.95 Subrahmanyam, K., Michikyan, M., Clemmons, C., Carrillo, R., Uhls, Y. T., & Greenfield, P. M. (2014). Learning from paper, learning from screens: Impact of screen reading and multitasking conditions on reading and writing among college student. International Journal of Cyber Behavior, Psychology and Learning, 3(4), 1–27. Thaler, R. (2009). Nudge. Improving decisions about health, wealth and happiness. London: Penguin. Van Lehn, K., & van de Sande, B. (2009). Acquiring conceptual expertise from modeling: The case of elementary physics. In K. A. Ericsson (Ed.), Development of professional expertise (pp. 356–378). New York: Cambridge University Press. Vergès, P., & Ryba, R. (2012). Social representations of the economy. In A. S. De Rosa (Ed.), Cultural dynamics of social representation. Social representations in the “Social Arena” (1st ed., pp. 233–245). London: Routledge. Verkade, H., Mulhern, T. D., Lodge, J. M., Elliott, K., Cropper, S., Rubinstein, B., et al. (2017). Misconceptions as a trigger for enhancing student learning in higher education: A handbook for educators. Melbourne: The University of Melbourne. Von der Mühlen, S., Richter, T., Schmid, S., Berthold, K., & Schmidt, L. M. (2015). Use of sourcerelated strategies in evaluating multiple psychology texts: A student-scientist comparison. Reading and Writing (online first). https://doi.org/10.1007/s11145-015-9601-0 Von der Mühlen, S., Richter, T., Schmid, S., Schmidt, L. M., & Berthold, K. (2016). Judging the plausibility of arguments in scientific texts: A student-scientist comparison. Thinking & Reasoning, 22, 221–246. Walstad, W. B., & Wagner, J. (2016). The disaggregation of value-added test scores to assess learning outcomes in economics courses. The Journal of Economic Education, 47(2), 121–131. Weinert, F. E. (2001). Concept of competence: A conceptual clarification. In D. S. Rychen & L. H. Salganik (Eds.), Defining and selecting key competencies (pp. 45–65). Ashland: Hogrefe & Huber. Wilson, J. P., Hugenberg, K. J., & Bernstein, M. J. (2013). The Cross-Race effect and eyewitness identification: How to improve recognition and reduce decision errors in eyewitness situations. Social Interest and Policy Review, 7, 83–113. Wineburg, S., & McGrew, S. (2016). Why students can’t Google their way to truth. Education Week. Retrieved from http://www.edweek.org/ew/articles/2016/11/02/why-studentscant-goo gle-their-way-to.html Wineburg, S., Breakstone, J., McGrew, S., & Ortega, T. (2018). Why Google Can’t Save Us: The challenges of our post-Gutenberg moment. In O. Zlatkin-Troitschanskaia, G. Wittum, & A. Dengel (Eds.), Positive learning in the age of information (pp. 221–228). Wiesbaden: Springer. Zlatkin-Troitschanskaia, O., Happ, R., Förster, M., Preuße, D., Schmidt, S., & Kuhn, C. (2013). Analyse der Ausprägung und Entwicklung der Fachkompetenzvon Studierenden der Wirtschaftswissenschaften und Wirtschaftspädagogik. In O. Zlatkin-Troitschanskaia, R. Nickolaus, & K. Beck (Hrsg.), Kompetenzmodellierung und Kompetenzmessung bei Studierenden der Wirtschaftswissenschaften und der Ingenieurwissenschaften [Sonderheft]. Lehrerbildung auf dem Prüfstand, 6(1), 69–92. Zlatkin-Troitschanskaia, O., Förster, M., Brückner, S., & Happ, R. (2014). Insights from the German assessment of business and economics competence. In H. Coates (Ed.), Assessing learning outcomes: Perspectives for quality improvement (pp. 175–197). Frankfurt am Main: Lang.

21

What Can We Learn from Theoretical Considerations and Empirical Evidence

309

Zlatkin-Troitschanskaia, O., Shavelson, R. J., & Kuhn, C. (2015). The international state of research on measurement of competency in higher education. Studies in Higher Education, 40(3), 393–411. https://doi.org/10.1080/03075079.2015.1004241 Zlatkin-Troitschanskaia, O., Schmidt, S., Brückner, S., Förster, M., Yamaoka, M., & Asano, T. (2016). Macroeconomic knowledge of higher education students in Germany and Japan - A multilevel analysis of contextual and personal effects. Assessment & Evaluation in Higher Education, 41(5), 787–801. https://doi.org/10.1080/02602938.2016.1162279 Zlatkin-Troitschanskaia, O., Schmidt, S., Molerov, D., Shavelson, R. J., & Berliner, D. (2018). Conceptual fundamentals for a theoretical and empirical framework of positive learning. In O. Zlatkin-Troitschanskaia, G. Wittum, & A. Dengel (Eds.), Positive learning in the age of information (pp. 29–50). Wiesbaden: Springer. https://doi.org/10.1007/978-3-658-19567-0 Zlatkin-Troitschanskaia, O., Kuhn, C., Brückner, S., & Leighton, J. P. (2019a). Evaluating a technology-based assessment (TBA) to measure teachers’ action-related and reflective skills. International Journal of Testing., 19(2), 148–171. Zlatkin-Troitschanskaia, O., Shavelson, R. J., Schmidt, S., & Beck, K. (2019b). On the complementarity of holistic and analytic approaches to performance assessment scoring. The British Journal of Educational Psychology. https://doi.org/10.1111/bjep.12286

Index

A Academic achievement, 282 Academic domain, 94 Academic education Internet Age, 290 Academic learning, 60, 287, 288 Academic study environments, 3 Activity theories of learning assumption, 272 description, 271 framework, 272 human actions, 272 learning to learn, 271 PLATO project, 274 primary properties, 272 procedural/technological model, 274 PSFMA system, 272, 273 psychological model, 273 psychological–pedagogical model, 273, 274 second generation, 271 secondary properties, 272 specific action/image, 272 Adequateness, 167 Algebra, 254–256, 262 Alzheimer’s disease, 154 Ambiguity lexical, 197 structural, 197 Analytical approach, 98 Analytical thinking, 98 Areas of interest (AOI), 298 Aristotelian concept, 132 definition, 128, 130, 131 differentia, 130

genus, 130 pedagogical tool, 132, 133 tool for CT, 132 Artificial intelligence (AI), 5, 302 Assessment of Higher Education Learning Outcomes (AHELO), 98 Assessment tests, 215 Attention control, 60 Attentional self-control, 61 Augmented reality (AR), 122, 136 Autonomy, 97 academic goals, 66 empowering students, 60 and goal setting, 66, 68 levels of students, 68 study programs and tasks, 64–65 supportive teaching, 66

B Balanced incomplete block (BIB), 257 Bayesian Network specification, 188 Beginning-level learners, 133 Biased information, 277–279, 285 Bildung, 74, 76, 83, 97 Biology education aristotelian (see Aristotelian) computational approaches, 128 conceptual descriptions, 127 controlled vocabulary, 128 intelligent textbook, 131, 133 logic (see Logic) universally true statements, 128 Brain imaging methodology, 154

© Springer Nature Switzerland AG 2019 O. Zlatkin-Troitschanskaia (ed.), Frontiers and Advances in Positive Learning in the Age of InformaTiOn (PLATO), https://doi.org/10.1007/978-3-030-26578-6

311

312 C Capacity, self-control, 58, 59 Capp Critical Reasoning test, 98 Cell, 132 Center for Ethnic, Racial, and Religious Understanding (CERRU), 16 Clause combining, 214–222 Clause-final particles, 214 Clever thinking, 98 Cognitive load theory (CLT), 172, 174–177, 182, 186, 189, 191 Cognitive processing, 4 Cognitive psychology, 46 Cognitive skills, 207–210 Cognitive structures, 198 Cognitive taxonomy level, 238–240, 245, 248, 250 Colloquial and scientific language, 73, 74, 76, 77, 83 Color gradients, 139 Common Core State Standards in Mathematics (CCSSM), 254, 264 Common language, 270 Communication events, 185 Complex nominal compounds (CNCs) academic texts, 143 concepts, 143 EEG, 144 eye tracking, 144 high information density, 145 scientific literature, 145 self-regulated learning, 143 waste water treatment facility, 143 Comprehensibility, 167 Computational linguistic analyses, 170 Computational linguistics, 184, 190 Computational modeling, 190 Computational power, 188 Computer mediated communication (CMC), 183 Concept of liberation education, 79 Conceptual history, 75, 79, 83 Conceptual learning, 291 Conceptual spaces, 172, 178–181, 191 Confidence, 124 Confirmatory factor analysis (CFA), 242 Conflicting information, 136, 137, 141 Connected learning, 291 Connectional understanding, 135 Construction grammar, 183 Construction integration model (CIM), 172, 174, 176–178, 182, 191 Context-adequate constructions, 176

Index Context sensitivity, 171–173, 175, 189, 191 characteristics, 168 CIM, 174 CLT, 174 cognitive re-construction, 168 complexity dimensions, 173 discourse theory, 176–177 educationally relevant texts, 170 feature theory, 177–181 learning theory, 175–176 LTM (see Long-term memory (LTM)) multi-layer architecture, 174 relational meaning, 168 requirement analysis, 170–174 schema learning theory, 182–185 transfer learning average geodesic distance, 187 CLT, 186 communication behavior, 187 communication events, 185 context-sensitive text, 184 in educational environments, 184 framing module, 184 hypothesis, 186 LTM, 184 ML, 184 online media, 184, 187 online writing, 187 random learner model, 185 small text data, 184 social data, 186 social networks, 187 TUCE, 184, 186 wikicussions, 187 Wikidata, 186 Wikipedia, 186 Contextual converbs, 215 Converbs, 214–222 Corpus-analytical approach, 182 Correct understanding, 123 Counter-factual knowledge, 279 Crafting behaviors, 62–64 contextual conditions, 63 internal volitional strategies, 64 job, 62 motivating students’ behaviors, 65–67 perceptions of their tasks, 63 physical changes, 63 skill development, 63 structures of study programs, 64, 65 study crafting-model, 63 study tasks, 64, 65

Index task-related crafting, 63 types, 62 volitional processes, 63 Critical alertness, 90, 95, 96, 99, 100, 104 Critical analysis, 90, 91, 96, 101, 103 Critical handling, 296, 297 Critical prudence, 96 Critical reflection, 90, 95, 99, 100, 104 Critical Theory of the Frankfurt School, 82 Critical thinking (CT), 112 analysis, 89 characteristics, 92 colloquial language, 73, 74 commitment makers, 75 concept, 82 conceptual history, 83 conditions, 95, 96 critical alertness, 90 critical analysis, 90 critical reflection, 90, 95 definition, 73, 128 developmentalists and philosophers, 91 and domain specificity, 94, 95 and education (see Education) established assessment formats, 98–99 fragile elements, 97 functional element, 84, 85 general/generic, 95 guidance, 82 health insurance, 90 immediate reflection, 90 income inequality, 89 inherently harmless, 74 knowledge, 83 liberal party, 90 logical thinking, 84 measurement method, 75 moral evaluation, 84 moral judgment, 83 normative claim, 75 perception/behavioral control, 83 performance assessment (see Performance assessment) prerequisites, 92 principle, 74 prototype, 89 psychological, 83 questioning, 99, 100 questions, 93 reservation, 74 scientific language, 73 SDs (see Social domains (SDs)) and self-centered attitudes, 96

313 self-inflicted dependency, 91 social dimension, 78 social sciences, 74 societal expectations, 91 technical reasons, 73 transformational dynamics, 95 truths, 96 validity, 90 Cynical critical behavior, 96

D Daily orientation-knowledge, 91 Data-driven approaches, 189 Data-driven processes, 175 Data preparation, 189 Deeper learning, 291 instruction, 41 juncture critical, 43 literature, 42, 43 methods, 188 PLATO’s goal, 42 pluriliteracies model, 48 role of the humanities (see Role of the humanities for deeper learning) technologies, 123 Degree of difficulty, 167 Democratic society, 282 Depletion, 60 Described situation, 175 Descriptive framework, 182 Descriptive meaning, 75 Descriptive statistics, 238, 241 Dialog Game Board (DGB), 202 Dialogical learning rule, 202 Dialogue semantics, 199 Difference scores, 237–239 Digital media, 122, 287, 288 Digital technologies, 290 Directive leadership style, 65 Discipline-specific knowledge, 53 Discourse situation, 175 Discourse theory, 176–177 Distrust/critical inquiry, 49 Domain learning, 291, 292 Domain-specific concepts, 289 Domain-specific knowledge, 46 Domain-specific misconceptions, 279 Domain-specific teaching, 41, 42 Dropout rates, 31, 32 Dynamic media, 122

314 E Echo planar image (EPI), 156 Economics knowledge, 296 Economics learning IRT (see Item response theory (IRT)) Education Bildung (see Bildung) causal localization, 77 characteristics, 76 childhood, 9 cognitive and emotional instances, 82 cognitive psychology, 77 commitment makers, 75 components, 83 concept, 76, 77, 82, 84 definition, 78 dichotomously, 76 German grammar rules, 76 German history of ideas, 82 German-language, 83 and knowledge, 80–81 liberal arts and sciences, 9 practice, 282 psychological constructs, 77 reconstructive insofar, 77 semi-structured interviews, 9 vocational education, 10 vocational preparation, 9 Educational advancers, 78 Educational concept, 78 Educational knowledge, 81 Educational leaders, 36 Educational measurement, 237 Educational objectives, 78 Educational privileges, 78 Educational science, 79 Educational text mining, 175 Educational thinking, 81 Electroencephalogram (EEG), 144, 149 Electronic textbook, 133 Electrostatics concept, 141 materials, 138 participants, 137 results, 140 study design, 138–140 Element interactivity, 172 Emerging adults, 47 Emotional control, 61 Emotional self-control, 61 Encoding control, 60 Environmental self-control crafting (see Crafting)

Index emotional, 61 learning outcomes, 61 self-control strength, 61 strategies, 61 students’ identity, 61 Eukaryotic cell, 132 Evidentials, 222–230 Expectation-driven processes, 175 Extraneous cognitive load (ECL), 172, 176, 189, 192 Eye gaze data, 124, 141 Eye tracking, 125, 137, 138, 140, 144, 150

F Fact checking, 26, 28, 32, 35 Fake news, 53, 128 Fakes vs. facts, 288 False certainties, 53 False information, 30 False knowledge, 36 Fear of missing out (FOMO), 15 Feature theory balanced binary tree, 179 CIM, 177 CLT, 177 computational linguistics, 177 concepts, 179 conceptual spaces, 179, 181 dimensions, 178 domains, 178 end-to-end learning, 178 first-level models, 181 levels of area, 179 ML, 178–180 properties, 179 prototypes, 179 reference points, 179 second-level models, 181 textual resources, 181 tree-like structures, 179 vagueness, 179 zero-level models, 181 Fidelity, 44 Final knowledge, 237 Financial security, 97 Forethought phase, 58 Formal learning, 109, 110, 117

G General education, 81 Generative adversarial networks (GAN), 123

Index Generic yet domain-related skills, 249 Genetic algorithms, 171 German education system, 284 German grammar rules, 76 Glorious Revolution, 78 Goal setting, 58, 65–68 Grammatical categories, 208–210, 217 222, 230 Grammatical properties, 207 Greyscale gradient, 140 Group membership, 78

H Health insurance, 90 Heatmaps, 140 Heighten critical thinking assessment, 98 Higher education, 254 academic learning, 291 characteristics, 293 cognitive interviews, 298–301 digital media, 287, 288 domain learning, 292 filter, 292 knowledge development, 294–296 learning, 291, 292 the liberal arts, 23 macrolevel, 292 Mayer’s cognitive theory, 291 mid-range theories, 291 NL, 23, 288, 289, 292 non-controversial norms, 293 nuanced and critical thinking, 24 online environments, 293 perceived truths, 22 perceived value, college education, 24, 25 PL, 292 principles, 293 social norms, 23 sources of information, 294 state legislature, 25 systems theory, 291 travel, 22, 23 Higher order learning, 291 Holistic explanatory model of PL activity theories, 271–274 collaborative research, 269 common language, 270 and integral, 270 and NL, 270 objectives, 274 PLATO project, 269

315 stake holders, 270 subject-specific and generic concepts, 270 Human–computer interaction, 271 Humanistic education, 80

I Idea density (ID) aging, 154 behavioural data, 158, 160 cognitive impairment, 154 education, 154 experimental procedure, 155 fMRI data analysis, 158 functional imaging data, 158, 159 functional neuroimaging data, 160, 161 gender, 154 innovative learning approach, 161 linguistic complexity, 153, 161 MRI data acquisition, 156 natural discourse, 155 neural information processing, 161 participants, 155 propositional base structure, 153 semantic complexity, 161 transcriptions meta-discourse, 156 speech data, 157 statistical analysis, 157 Identity, 60–63, 68 Identity for PLATO DFG Excellence Cluster, 281 education research into practice, 282, 283 history of scientific research, 281 mission, 281, 284 organization, 285 organization and nature, 279 organizational structure, 284 Pasteur’s quadrant, 281 public, 283–284 questions, 281 R&D, 282, 284 research areas, 284, 285 stock taking basic and applied research, 282 technological development, 283 vision, 281 Incentive escalation control strategy, 61 Incorrect understanding, 123 Indirect P-score, 101 Indo-European languages, 186, 208 Inferior parietal lobule (IPL), 159

316 Informal learning, 109, 110, 117 Information age, 277–279 Information and communication technologies (ITC), 2, 57, 277, 294 Information density CNCs, 143, 144 comprehension and memory encoding 149, 150 individual differences, 144 memory traces, 147, 148 neuroscience, 144 psycholinguistics, 144 reader characteristics, 145, 146 reading goals, 146, 147 structure, 144, 145 student learning, 150 Information packaging, 211 Information processing, 269 Information-processing control, 60 Information structure assessment tests, 212 clause-initial position, 213 component, 211 grammatical, 211 information packaging, 211 interrogative pronoun, 213 language functions, 211 SOV, 212, 213 technical quotation, 211 topicalisation, 212 topic-prominent languages, 211 TUCE test, 213 Inhomogeneous, 140, 141 Initial knowledge, 237 Insightful learning, 291 Instructors, 48 Insufficient funding, 284 Intellectuals, 36 Intentional learning, 147 Interactivity, 175 Internal volitional strategies, 64 International research program, 278 Internet, 1, 277, 279, 280, 283, 284 environment, 294 learning, 301, 303 PL and NL, 303 real-time, 302 users, 296 Internet Age academic education, 290 knowledge development (see Knowledge development) Internet-based texts/videos, 49

Index Internet research, 48 Internet use, 57 Intra-individual differences, 295 Intrinsic cognitive load (ICL), 175, 176 Inverse scope analysis, 199 Item difficulties, 238, 242–244 Item response theory (IRT) domain-specific misconceptions, 239 forgetting, 239 guessing, 239 learning patterns macroeconomics data, 241–245 microeconomics data, 246–249 modeling and analysis, 249 parametrization, 250 probabilistic test theory, 238 RQ, 238 student learning patterns, 4 test instrument and sample, 240, 241 theoretical framework, 239–240

J Job crafting, 62

K Kidney transplantation story, 99 Knowledge characteristics, 127 consolidate/complement, 110 content, 115 domain-specific, 116, 117 effects of media, 111 and skills, 110 societal body, 111 sources, 110 students, 110, 116 test-based, 111, 112 Knowledge development academic learning, 288 in higher education (see Higher education) online information, 289 Knowledge, skills and abilities (KSAs), 64 Knowledge-related resources, 169 Kritisches Denken (KD), 83 K–12 schooling, 24

L Lack of identity, 60 Language comprehension, 198 Language learners, 132

Index Latent analysis and IRT (see Item response theory (IRT)) Learned schemata, 172, 175 Learner-dependent theory, 176 Learning economics students, 114 environments, 117 in higher education, 109, 116 informal, 109, 110 material, 109 media effects (see Media effects) online, 117 positive/negative, 109 resources, 114 source of, 113 unintentional, 110 university students (see Learning) Learning communities, 15 Learning environments, 287 Learning from the negative, 203 Learning from the positive, 203 Learning partnerships, 51 Learning patterns in economics, 249 NL, 238 PL, 238 predictor variables, 249 regression analyses, 250 Learning theory, 175–176 Lexical ambiguity, 197 Lexical embeddings, 183 Liberal arts, 9, 18, 23 Life science, 27 Linear equations (LE), 256 Linear functions (LF), 256 Linguistic typology clause combining, 214–222 cognitive skills, 207–210 converbs, 214–222 evidentials, 222–230 modality, 222–230 performance, 231 variation, 207–210 Linguistic variation, 207–210 Linking competence, 204 Literature as experiment in social interaction, 45, 46 Logic pedagogical tool, 130 photosynthesis, 129 tool for CT, 129, 130 universal truths, 129 universally quantified sentence, 128

317 Logical thinking, 98 Long-term memory (LTM), 168, 169, 171, 175–177, 182–184, 191 Low-level simulation models, 50

M Machine learning (ML), 168, 169, 171, 172, 184, 188 Macroeconomics data explicit application, 245 global fit criteria, 242 high positive learning discriminations, 244 i9 shows, 244 implicit application, 245 IRT parameterization, 242, 243 item difficulties, 243, 244 linguistic and content-related characteristics, 244 low positive learning discriminations 244, 245 Mplus WLSMV estimator, 242 NL (see Negative learning (NL)) PL (see Positive learning (PL)) properties, 241 recognition and understanding, 245 RMSEA, 242 TUCE, 245 WRMR statistics, 242 Mass media, 110, 111, 116, 117, 288 Mass online media, 288 Mathematical logic, see Logic Mathematical proficiency, 259 Mathematics knowledge and skills assessment, 256, 257 BIB design, 257 CCSSM, 254, 263 college and career readiness, 254 comprehensive assessment program, 259 content strand and grade band, 258 educational planning, 264 function notation, 254, 264 high-accuracy items, 259, 260, 262 low-accuracy items, 259, 261 NCEE, 254 participants, 255, 256 performance, 257, 259 postsecondary remediation, 253 qualitative analysis, 259 remedial courses, 264 STEM and non-STEM career paths, 255, 262, 264 verbal and symbolic representation, 262

318 Math-related demographic data, 256 Mayer’s cognitive theory, 291 Media content, 110, 116, 117 Media conversion AR, 122 automatic method, 123 deep learning technologies, 123 disadvantage, 122 dynamic media, 122 equation, 122 eye-tracking, 125 and features, 122 graphics, 122 intelligent learning system, 126 manga, 122 manually and automatically, 121 medium of information, 122 objective understanding, 121, 123, 124 picture-to-text conversion, 123 PL/NL, 121 reverse conversion, 123 sensing states, 124, 125 states of learners, 123 subjective understanding, 121, 123, 124 text explanation, 122 video, 122 Media credibility, 109 Media effects causal relationships, 116 content analysis, 116 credibility, 109 descriptive specification, media resources, 112 domain-specific knowledge, 112 institutional differences, 115 knowledge (see Knowledge) limitations, 116–117 mass, 110 media use, 116 news (see News media) online media, 115 open-ended question, 112 positive and negative, 110–111 social (see Socila media) sociology and economics, 114, 115 students’ use of economic information 112, 113 survey data, 116 surveys, 111 tabloid newspapers, 112 teaching-and-learning resources, 110 Wikipedia, 113 Media formats, 52

Index Media use, 110–117 Medical criteria, 101 Medical humanities, 42 Medical/psychological stress-factors, 101, 102 Memory encoding, 144, 148–150 Memory retrieval, 144, 146, 148 Mental model, 175 Metabolic heat, 129 Meta-learning of architectures, 189 Meta-learning theorists, 270 Micro-cultures, 50 Microeconomics data cognitive category, 248 global model fit criteria, 246 high positive learning discriminations, 248 IRT parameterization, 246, 247 low/non-significant positive learning discriminations, 246, 248 NL, 246 PL, 246 QR, 246 representation formats, 247, 249 RMSEA statistics, 246 TUCE, 247 VR, 246 Micro-level units, 190 Minimal recursion semantics, 198 Mixed reality, 137 Modality, 222–230 Modeling description, 22 higher education, 22 influence behavior, 22 learned behaviors, 22 PLATO project, 22 social beliefs, 21 social cohesion, 21 social truths, 21 Module manuals, 81 Moral component, 48 Moral disagreement, 96 Multi-layered social-semiotic networks, 189 Multilingual learning, 203 Multimedia learning, 135, 136, 291 Multimodal modeling, 190 Multimodality, 188–189 Multiple-choice (mc) tests, 237, 238 Multiple-choice question, 124 Multiple-choice test, 212 Multiple external representations (MERs), 136 Multiple indicators and multiple causes (MIMIC) model, 250 Multi-word expressions, 183

Index N Narrative knowledge, 43 Narrative medicine, 42, 43 National Assessment of Educational Progress (NAEP) tests, 32, 33, 257 National Center on Education and the Economy (NCEE), 254 National Public Radio (NPR), 30 Nations’ report card, 32 Natural language processing (NLP), 170 171, 191 Negative learning (NL), 1–2, 50, 51, 53, 109–111, 115–117, 121, 123, 124, 153, 161, 202, 203 conceptual support, 239 definition, 23 discrimination parameters, 249 in higher education, 288, 289 influence factors, 249 macroeconomics data (see Macroeconomics data) microeconomics data (see Microeconomics data) test score, 238 theoretical framework, 238–240 Network modeling, 190 Network theory, 190 Neural activations, 158 Neuronal oscillations, 149 Neuroscience, 144 News media, 109, 110, 112, 113, 115 Nisbett’s hypothesis, 186 Nominal compounds (NCs), 144, 150 Nominalisations, 214 Non-critical awareness, 95 Non-linguistic contexts, 167 Normative meaning, 75

O Objective approaches, 173 Objective understanding, 121, 123, 124 Online encyclopedias, 115 Online environments, 117 Online information, 287, 290, 296, 300, 301 critical handling, 296, 297 dissemination, untrustworthy information, 289 ICTs, 289 mental prerequisites, 289, 290 non-transparency, 289 online reasoning, 296, 297

319 PL outcomes, 289 sampling, 289 Online learning, 117, 132 in higher education (see Higher education) integration, 301–303 selective and erroneous visual perception, 298, 299 sources of information, 294 specification, 301–303 Online media, 115, 187 Online reasoning, 296, 297 Ontology, 128, 133 See also Aristotelian Open-source platforms, 188

P Panel survey, 116, 117 Parallel learning systems, 288, 289 2-Parameter logistic item response theory (2PL IRT), 257, 258, 262 Partnership for Readiness for College and Careers (PARCC), 257 Pattern grammar, 183 Performance assessment application, 100 CT iPAL, 100 and medical/psychological stressfactors, 101, 102 opinions on justice, 101 and political decision-making, 103, 104 and professional morality, 102 and religious issues, 103 measurement techniques, 99 real-life situations, 99 Performance phase, 58 Performance test, 99, 104 Person-organization fit depletes self-control strength, 60 Philosophical ontology, see Aristotelian Photosynthesis, 129 Phrasal schemata, 183 Physics education, 136, 137, 141 Picture bias AR, 136 conflicting information, 136 connectional understanding, 135 electrostatics (see Electrostatics) multimedia learning, 135, 136 people’s beliefs, 135 pictures to text, 135

320 Picture bias (cont.) project aim, 136–137 research questions, 136–137 seductive details effect, 135 Picture-to-text conversion, 123 Pluriliteracies model, 41, 48 Policy makers, 284 Policy research, 22, 26 Politeness, 78 Political affiliation, 25 Political beliefs, 36 Political decision-making, 103, 104 Political leaders, 22, 36 Political liberty, 36 Political parties, 24 Political science, 53 Positive learning (PL), 2, 45, 47, 49–51, 109–111, 115–117, 153, 161, 202, 203 definition, 278 discrimination parameters, 249 fundamentals, 269 holistic explanatory model (see Holistic explanatory model of PL) influence factors, 249 macroeconomics data (see Macroeconomics data) media conversion (see Media conversion) meta-learning theorists, 270 microeconomics data (see Microeconomics data) test score, 238 theoretical framework, 238–240 Positive learning in the age of information (PLATO) advantages, 53 concept, 41 history, 277–278 humanities, 278 identity (see Identity for PLATO) language, 23 lexicon, 24 research areas, 280 social sciences, 278 strength, 279 theoretical framework, 117 Predictive theory, 127 Pre-requisite knowledge, 132 Pretest posttest design, 237–241, 243, 244, 249 Primitive vocabulary, 132 Principles of economics, 240, 241 PRiSMA research, 282 Private and charter schooling, 30

Index Probabilistic test theory, 238, 239 Procedural analytical knowledge, 90 Procedural/technological model, 274 Professional morality, 102 Prokaryotic cell, 132 Psycholinguistics, 144 Psychological conflicts, 60 Psychological diagnostics, 77, 82, 83 Psychological discrepancy, 60 Psychological model, 273 Psychological–pedagogical model, 273, 274

Q Quality higher education argumentation, 12 external challenges antipathy to higher education, 17, 18 cost of education, 17 PLATO project, 19 psychological stress, 19 internal challenges loneliness, alienation, failure to belong, 15, 16 mental health and well-being, 13–15 secondary schools, 12 semiotics, 12 three Cs character, 12 context, 10 curriculum, 12 Quantified noun phrases (QNPs), 199 Quantitative linguistic approaches, 170 Quantitative reasoning (QR), 246 Quasi-multiple-choice test format, 98

R Random learner models, 188 Readability, 167 Real-time inference, 188 Real-time response (RTR) measurement, 117 Regular patterns, 207 Religious statements, 103 Representation formats, 244, 246, 247, 249 Republican party, 24 Research questions (RQ), 238 Resource situation, 168, 175 Return on investment (ROI), 18 Reverse conversion, 123 Role of the humanities for deeper learning concept, 43 co-production of knowledge, 51

Index developing models, 45 domain-specific knowledge, 42 domain-specific teaching, 41, 42 fields, 42 instruction, 41 interdisciplinary, 42, 51 learning partnerships, 51 literature classroom, 51 personal growth, 52 pluriliteracies model, 48 properties, 42 psychological perspective, 51 self-absorbed, 42 self-efficacy, 52 self-image, 53 self-reflection, 52 teaching format, 51 transferable subject knowledge and skills, 52

S Scale-free distributions, 190 Schema learning theory CIM, 182 CLT, 182 CMC, 183 computational linguistics, 184 corpus-analytical approach, 182 descriptive framework, 182 experimentally, 182 learning corpus, 183 LTM, 182, 183 random learners, 183 reading history, 183 schemata learned, 182 time series, 183 TUCE, 183 types of learners, 182 word embeddings, 182 School shootings, 30 Science databases, 115 Science journals, 115 Science, Technology, Engineering and Mathematics (STEM), 26, 254 Scientific community, 91 Second-year Transformational Experience Program (STEP), 16 Self-control academic performance, 59 capacity, 58, 59, 61 deployment, 58 environmental (see Environmental selfcontrol)

321 information-processing, 61 internet use, 57 learning methods, 58 performance phase, 58 positive academic learning, 57 self-regulated learning (see Self-regulated learning) self-regulated students, 57 social networks (see Social networks) strategies, 60–62 strength (see Self-control strength) student’s grade point average, 59 students’ identity, 61 students’ learning, 61 students’ levels, 59 temporarily, 59 types, 61 Self-control strength academic environments, 59 actions, 58 lack of identity and depletion, 60 single resource, 58 state-like current level, 59 stress symptoms, 59 tasks, 58 Self-directed learning, 110 Self-efficacy, 52, 112 Self-inflicted dependency, 91 Self-organization of natural languages, 190 Self-reaction, 58 Self-reflection, 46 Self-reflection phase, 58 Self-regulated learning, 46 characterization, 58 cyclical phases, 58 general and self-control, 60 negative emotions, 60 Self-regulated students, 57 Self-regulation, 112 Semantic knowledge, 147, 148 Semantic representations, 198 Semantic structures ambiguity, 197, 198 cognitive, 198–200 continuations, 198 psycholinguistics, 198 Sentence structure, 146, 150 Separation principle, 188 Simulation abstraction, 48 advantages, 44 characters, 45, 47 cognitive psychology, 46 concept of, 44

322 Simulation (cont.) deeper learning models, 48 domain-specific knowledge, 46 educating, 50 experimental action, 45, 50 fidelity, 44, 47 form of engagement, 45 functions, 48 identify, 47 in-group differentiation, 46 internet-based texts/videos, 49 learning, 49, 51 low fidelity, 50 material frameworks, 44 moral component, 48 personal relevance, 49 positive learning, 45 self-reflection, 46 self-regulated learning, 46 social interaction, 45 student engagement, 46 students identify, 48 surface learning, 45 teaching models, 50 teaching of literature, 44 text confronts, 45, 46 translation, 49 translation process, 50 trustworthiness, 49 university education, 45 Situation modelling, 200, 201 Skepticism, 100, 104 Sliding window technique (SWT), 157 Small text data, 184 Social beliefs, 21 Social cohesion, 21 Social domains (SDs) academic domain, 94 beliefs, motivational/emotional issues, 97 CT, 94 economical-moral mixture, 97 evidences, factual/scientific information, 97 field-specific knowledge, 96 framing and check-up, 91 general thinking, 94 interests, 97 performance assessment (see Performance assessment) questions, 97 sense-making middle proportion, 97 subject domains, 94 tertiary level learn, 94

Index Social media, 58, 67, 109–117, 279, 280, 284, 285, 288 Social networks, 57, 187 Social online media, 288 Social truths, 21, 23 Stanford History Education Group (SHEG), 2 Statistical Parametric Mapping software (SPM8), 158 Structural ambiguity, 197 Student and citizen education, 280, 283 Student socioeconomic status, 26, 27 Students’ knowledge, 288 Subject domains, 94 Subjective approaches, 173 Subjective understanding, 121, 123, 124 Subject-object-verb (SOV), 209, 212, 213 Superior parietal lobule (SPL), 159 Surface learning, 45 Synthetically approach, 98 System of planned stage-by-stage formation of mental actions (PSFMA system), 272–275 Systems theory, 291

T Task-related crafting, 63 Teaching literature concept of PLATO, 41 deeper learning (see Role of the humanities for deeper learning) humanities, 43 pluriliteracies model, 41 qualities, 43 as simulation (see Simulation) Technological development, 283 Technologies against biased information, 277 Test of Understanding of College Economics (TUCE), 173, 183, 184, 186, 238–247 Test-theory perspective, 239 Text complexity, 168, 172, 174 TextInContext architectural setting, 174 computational linguistic analyses, 170 computational model, 168 context sensitivity (see Context sensitivity) definitions, 169 functional attribute, 170 knowledge-related resources, 169 linguistic and non-linguistic context variables, 169 LTM, 169

Index meaning of an expression, 168 methodological requirements, 189–190 ML, 168 multimodal cognition, 188–189 NLP, 170 quantitative linguistic approaches, 170 requirement analysis automation, 171 capability of processing big data, 171 CIM, 172 CLT, 172 conceptual spaces, 172 context sensitivity, complexity dimensions, 173 ECL, 172 expressiveness, 172, 173 modularity, 172 responsivity and sensitivity to small data, 171, 172 simulation, 173 requirements analysis, 170, 171 task descriptions, 170 variables, 167 Text snippets, 184 Text-to-picture conversion, 123 Thematic apperception test (TAT), 155 Think-aloud method, 239 Third International Mathematics and Science Study (TIMSS), 257 Three models framework of the learninginstruction situation, 273, 274 Time series, 183 Topicality, 209–211, 232 Topic-prominent languages, 211 Transformational learning, 291 Translation process, 50 Translations, 208, 213, 215, 218, 219, 221, 230 Translator training, 53 Travel, 22, 23 Trump effect, 53 Truth-seeking administration, 34, 35 costs of schooling, 34

323 degrees earned to jobs available, 27, 28 dropout rates, 31, 32 educational policy, 26 educational profession, 25 fact checking, 26, 28, 32 NAE tests, 32, 33 physics course, 26, 27 private and charter schooling, 30 public schools, 30 rates of victimization, 30 school counselors/personnel, 30 school days, 30 school shootings, 30 school-testing experts’ rule, 29 STEM education, 26 STEM workers, 27–29 time consuming, 26 violent victimization, 30 Type theory with records (TTR), 199–201 Typing analysis, 125 Typing behavior, 125 Typing questions, 125

U Unintentional learning, 109, 110 Untranslatability thesis, 74 Unwarranted/incorrect knowledge, 23 Use-inspired research, 281

V Variables and patterns (VP), 256 Verbal reasoning (VR), 246 Video platforms, 115

W Watson-Glaser Critical Thinking Appraisal (WGCTA), 83, 98 Work vs. educational psychology, 65 Working memory, 146