Narrative Complexity: Cognition, Embodiment, Evolution 2019001146, 9780803296862, 9781496214904, 9781496214911, 9781496214928

Table of contents :
Cover
Title Page
Copyright Page
Contents
List of Illustrations
List of Tables
Acknowledgments
Introduction
Part 1
1. Narrative as/and Complex System/s
2. Caution, Simulation Ahead
3. The Wave-Crest
4. Complexity and the Userly Text
5. The Complexity of Informative Autobiographies
Part 2
6. Sources of Complexity in Narrative Comprehension across Media
7. Structural Complexity in Visual Narratives
8. Simplicity, Complexity, and Narration in Popular Movies
9. Heteronomy of Narrative
Part 3
10. Narrative Here-Now
11. Body Forth in Narrative
12. Between Distancing and Immersion
13. Intersubjectivity, Idiosyncrasy, and Narrative Deixis
14. Jazz as Narrative
Part 4
15. The Predictive Mind, Attention, and Cultural Evolution
16. Necessary Fictions
17. In Hindsight
Contributors
Index

Citation preview

Narrative Complexity

Frontiers of Narrative

Se rie s E ditor

Jesse E. Matz, Kenyon College

Narrative Complexity Cognition, Embodiment, Evolution Edited by Marina Grishakova and Maria Poulaki

University of Nebraska Press  |  Lincoln

© 2019 by the Board of Regents of the University of Nebraska All rights reserved

Publication of this volume was supported by the Estonian Research Council (grants 192, “Emergent Stories: Storytelling and Joint Sensemaking in Narrative Environments,” and 1481, “The Role of Imaginary Narrative Scenarios in Cultural Dynamics”) and the European Regional Development Fund (Center of Excellence in Estonian Studies). Library of Congress Cataloging-­in-­Publication Data Names: Grishakova, Marina, editor. | Poulaki, Maria, editor. Title: Narrative complexity: cognition, embodiment, evolution / edited by Marina Grishakova and Maria Poulaki. Description: Lincoln: University of Nebraska Press, [2019] | Series: Frontiers of narrative | Includes bibliographical references and index. Identifiers: lccn 2019001146 isbn 9780803296862 (cloth: alk. paper) isbn 9781496214904 (epub) isbn 9781496214911 (mobi) isbn 9781496214928 (pdf) Subjects: lcsh: Discourse analysis, Narrative. Classification: lcc p302.7 .n3664 2019 | ddc 401/.41—­dc23 lc record available at https://lccn.loc.gov/2019001146 Set in Minion Pro by E. Cuddy.

Contents

List of Illustrations  ix List of Tables  xi Acknowledgments xiii Introduction: Narrative Complexity  1 Marina Grishakova and Maria Poulaki Part 1. Narrative C omplexit y and Media

1. Narrative as/and Complex System/s  29 Marie-­L aure Ryan



2. Caution, Simulation Ahead: Complexity and Digital Narrativity  56 David Ciccoricco and David Large



3. The Wave-­Crest: Narrative Complexity and Locative Narrative  73 Emma Whittaker



4. Complexity and the Userly Text  98 Noam Knoller



5. The Complexity of Informative Autobiographies  121 Ulrik Ekman Part 2. C o gnition and Narrative C omprehension



6. Sources of Complexity in Narrative Comprehension across Media  149 Joseph P. Magliano, Karyn Higgs, and James Clinton



7. Structural Complexity in Visual Narratives: Theory, Brains, and Cross-­Cultural Diversity  174 Neil Cohn



8. Simplicity, Complexity, and Narration in Popular Movies  200 James E. Cutting



9. Heteronomy of Narrative: Language Complexity and Computer Simplicity  223 Hamid R. Ekbia

Part 3. Experience, Subjectivit y, and Emb odied C omplexit y

10. Narrative Here-­Now  247 Mieke Bal 11. Body Forth in Narrative  270 Ellen J. Esrock 12. Between Distancing and Immersion: The Body in Complex Narrative  291 Maria Poulaki 13. Intersubjectivity, Idiosyncrasy, and Narrative Deixis: A Neurocinematic Approach  314 Pia Tikka and Mauri Kaipainen 14. Jazz as Narrative: Narrating Cognitive Processes Involved in Jazz Improvisation  338 Martin E. Rosenberg Part 4. Narrative C omplexit y and Cultural Evolu tion

15. The Predictive Mind, Attention, and Cultural Evolution: A New Perspective on Narrative Dynamics  367 Marina Grishakova 16. Necessary Fictions: Supernormal Cues, Complex Cognition, and the Nature of Fictional Narrative  391 James Carney

17. In Hindsight: Complexity, Contingency, and Narrative Mapping  414 José Angel García Landa Contributors 437 Index 445

Illustrations



1. Network of an episodic or epic narrative  35



2. Network of an it-­narrative  35



3. Network of Jean Racine’s Phèdre 36



4. Presentational sequence versus causal network for Phèdre 41



5. Modes of realization of narrativity  47



6. Prying open the text  62



7. Simulated Braille used as a haptic metaphor  69

8. The Lost Index: natmus 82 9. Journeyer’s Guidebook 83 10. The userly text  108 11. Sequential two-­panel excerpt from The Fantom of the Fair 159 12. Three pictures from A Boy, a Dog, and a Frog 165 13. Narrative sequence with two narrative constituents  177 14. Basic narrative schemata within Visual Narrative Grammar  178 15. Narrative conjunction using repeated narrative categories  180 16. Narrative patterns created by the interaction between narrative schemata  183 17. Mean panels per book between groups for each narrative pattern  187 18. Correlation coefficients between each narrative pattern  189 19. Narrational shifts in movies released from 1940 through 2010  213 20. Movies ranked within release year according to gross income  215

ix

21. Emotional capitalism in practice  264 22. Emma triangulated-­strangulated 

264

23. Two retentive perspectives  321 24. Deixis in terms of the narrative nowness model  323 25. Narrative structure of Memento 328 26. The Wundt curve  400

x  Illustrations

Tables



1. Basic constructional patterns in Visual Narrative Grammar  176



2. Data from books analyzed from the Visual Language Research Corpus  185

xi

Acknowledgments

This volume would not have been possible without the invaluable input of friends and colleagues with whom we had the pleasure to share and discuss our ideas over the preceding years. We are grateful for their encouragement and support. We thank Jesse Matz, our series editor, and Alicia Christensen, our editor, for believing in our book and all staff members of the University of Nebraska Press who assisted us with editing. Our deepest gratitude to N. Katherine Hayles, Jens Brockmeier, and John Pier for reading the manuscript and providing their generous endorsement. We would also like to thank the anonymous reviewers for their advice and comments.

xiii

Introduction Narrative Complexity Marina Grishakova and Maria Poulaki

A recent large-­scale international study has shown that different societies tend eventually to evolve toward complexity.1 Complexity inheres in human cognition and the tools that humans use to change their social and technological environments. There is a feedback loop between the development of technologies and cultural practices, on the one hand, and the ways of conceptualizing complexity, on the other (Hayles 1990, xiv). Indeed, human cognition and action, as anthropogenic “disturbances,” affect environments by amplifying the range of uncertainty and complexity. Building explanatory systems to capture and explain the emergent complexity leads to an acknowledgment of the limits to and gaps in knowledge, which, in turn, prompts building ever-­newer explanatory systems ad infinitum. The complexification of cognition and behavior and the complexification of society are mutually reinforcing. At the same time, Edgar Morin warns against the tendency to simplify knowledge in contemporary society, with its desire for narrow specializations, resulting in fragmentation and quantification of knowledge, a huge amount of anonymous information manipulated by media and nation-­states, and a spread of simplistic ideologies. Morin argues for the “need for complex thought” as the basis for political strategies that could be efficient only by “working with and against uncertainty, chance, the multiple play of interactions and retroactions” (2008, 5). Driven by the problems of complexity and inspired by the emerging research on complex narratives in various narratological quarters, this volume takes another step toward establishing narrative studies of complexity and explores how narrative complexity differs in terms of mind and body engagement and embeddedness in social and cultural prac1

tices. Challenging the classical distinction between simplicity as primary and prototypical and complexity as secondary and derived from simplicity, we maintain that complexity, in its various guises, is an ongoing state of human societies and human minds. This volume is an attempt to reach beyond the dichotomies of primary and secondary by introducing the complexity of homo cogitans and homo narrans behavior, as guided by intention and judgment but also as embedded in various biological, social, and cultural systems. In this way we aim at the complexification of our knowledge of both narrative representation and intelligent behavior, not by discarding current approaches but rather by striving for a new synthesis, including the interplay of agent-­and system-­oriented perspectives. Formal, Systemic, and Processual Complexity Interest in complexity has been common to the arts and sciences. The history of artistic forms of complexity, from prehistoric cultures through the baroque style and romanticism to avant-­garde and postmodernism, is inextricably intertwined with philosophical, scientific, and technological developments and epistemic shifts. The formal perfection of an artistic masterpiece has long served as the epitome of complexity. From the formal, aesthetic perspective, complexity amounts to an organization and patterning involving a multiplicity of elements and their connectivity and variety, evoking surprise and wonder. Sonia Zyngier, Willie Van Peer, and Jèmeljan Hakemulder refer to attempts by the mathematician Garrett Birkhoff to quantify aesthetic complexity in 1933; in discussing the etymology of the word complex from the Greek plektos (braided) and the Latin complexus (surrounding, encompassing), these authors note that “complexity involves the perception by the reader of a multiplicity of parts or units, forming patterns. In a literary text, the reader must first perceive language features and patterns and then somehow construe their interrelatedness, or how linguistic elements are ‘braided together’” (2007, 656). As William Paulson demonstrates—­by drawing on theories of aesthetics from Denis Diderot (who thought that the beautiful depends “on the idea and perception of relations,” fundamental to humans living in “a culture of machines and devices, of made things”) to Karl Phillip Moritz (and his discussion of complexity in the 1780s in relation to self-­organization) to Yuri Lotman (who argued that art complicates its own structure through interaction with its environment)—­ 2  Grishakova and Poulaki

the idea of art and artistic authenticity has always been one of organization and formal innovation and, with it, complexity (1991, 41–­42). But when perceived merely as a formal organization and patterning, complexity may easily become redundant or predictable. On the one hand, the formalist-modernist picture of the evolution of artistic forms as a seesaw swinging between “automatization” and “de-­automatization,” convention and experimentation, operates with a limited inventory of formal constituents. On the other hand, the excessive accumulation of novel extravagant features may generate informational overload and the overcomplexification that impedes communication or makes it impossible.2 Dividing descriptions of complex items into limited sets of simple formal constituents ultimately triggers recursive mechanisms and brings back complexity (Eco 1986, 46–­54). Kurt Gödel’s incompleteness theorems demonstrate that the limits of formalization are vicarious: “There are, in addition to formalized levels of knowledge, distinct ‘semi-­formal’ or ‘semi-­intuitive’ levels” not formalizable within the given system (Piaget 1970, 35). Human sciences develop on the assumption of incomplete, changing knowledge that depends “on interplay of anticipation and feedback” (16). This kind of open-­ended dynamics introduces the factor of time into descriptions of complexity. With open-­ended dynamics and resistance to closure, modern conceptualizations of complexity include such features as informational richness or density, openness, and a multiplicity of interpretations provoked by the work of art. In this way the qualitative complexity is separated from the quantitative complexity and translates into the complexity of response. The number of formal units, their interrelatedness, and the size of the work of art do not prove to be of primary importance in defining complexity. For example, adventure stories or soap operas may include a huge number of constituents and connections without necessarily being complex: what matters is density and richness of information stemming from the interplay of predictability and indeterminacy. In information theories semantic density has been called “effective complexity,” which refers to the length of the description of an entity’s regularities or to the logical depth, that is, the number of levels or the length of the computer program describing these regularities. In early semiotic discourses informed by cybernetics and information theories, the complexity of a work of art was depicted as a multiple encodIntroduction  3

ing or interplay of multiple semiotic languages. Lotman argued that, similar to a work of art, the game “is a particular reproduction of the conjunction of regular and random factors.” Likewise, art has the capacity to pull in random elements and is thereby a nexus of two opposing tendencies: one seeking to subordinate the elements of the work of art to a definite structure or “grammar,” another loosening and destabilizing and striving to “make the structure itself convey information,” that is, become meaningful (1977, 64).3 The interplay of heterogeneous elements, each element belonging to two or more semiotic languages and taking on more than one meaning, propels an increase in complexity and semantic richness. Consequently, what is deemed to be random in real life becomes “poly-­ systemic” in art (72). From 1986 onward, when Ilya Prigogine’s work on complex systems became a key influence on Lotman, Prigogine’s idea of a particular significance of the critical points of disorganization and indeterminacy where future forms of order germinate had been extended to the study of culture.4 In this way pattern or structure could be perceived as synonymous with either simplification or complexification. Whereas Katherine Hayles relates “noise” or entropy (disorganization) to the possibility of a system to reorganize into a higher order of complexity, the order being synonymous with complexity (1999, 25), Lotman adopts Kolmogorov’s concept of complexity, which discriminates between entropies of various types and levels (1977, 26). For Lotman, what Claude Shannon’s information theory considers entropy may be perceived as meaningful complexity in art. In this way the limitations of early cinematic techniques, for example, were turned by filmmakers into advantages and developed into unique features of the emergent film medium. Similarly, the missing arms of Venus de Milo add to the statue’s aesthetic value. Twentieth-­century systems theories, from Talcott Parsons’s to structuralism, introduced a new, holistic view of complexity, and the study of complex systems added self-­organization and emergence to system descriptions. The complex, emergent processes in both natural and cultural “complex systems,” which cover a wide range of phenomena, from ant colonies to airport traffic, are difficult to measure and prognosticate: systems behave as complex wholes, and their properties are not reducible to the sum of their parts. The characteristic features of such systems are self-­organization, nonlinearity, weak top-­down control, fusion and 4  Grishakova and Poulaki

interconnection of systemic layers, recursion (feedback loops), nonhierarchical coupling, and embedding. Complex-­systems theories have been extended to the study of social systems and society (Niklas Luhmann, Immanuel Wallerstein, David Byrne) and to the study of culture (Katherine Hayles, Yuri Lotman, Siegfried J. Schmidt). With regard to human systems and behavior, these studies reveal a potential clash between the classical scientific world picture, with a rational controlling agency at the center, and complex, overdetermined, largely unpredictable, and irreversible systemic change, resulting in splitting time-­space scales, nonlinear causality, and unintended side effects (Urry 2003; Castells 1996–­98). The side effects, digressions, and failures seem to be inscribed in the systemic dynamics. Hence, the complex-­system perspective reveals limitations of the idea of linear development, progress, and, ultimately, any kind of segregation (historical, linguistic, or national), because natural and social systems are enmeshed in complex and dynamic relationships. The principle of systemic coupling and embedding or, put differently, functioning in view of the ever-­present coevolving and nested systems—­“moving the boundaries outwards to include the external properties which engage the broader context-­sensitivities”—­prevent a system’s ultimate closure. These processes, in turn, produce an unprecedented need for the complex knowledge necessary to account adequately for the state of a system, such as contemporary society: “We must work back and forth between the ontologies of different levels to check that features crucial to upper level phenomena are not simplified out of existence when modelling it at the lower level” (Wimsatt 2008, 100). The connection between aesthetic complexity and systemic complexity has been noted by a number of authors, including in this century by John Casti and Anders Karlqvist and contributors to their edited volume, Art and Complexity (2003). A productive encounter with complexity science allows refurbishing of the tradition connecting aesthetics with the idea of complexity, which goes back some decades, even centuries, depicting art as a complex pattern emerging out of randomness. Art’s patterns would translate, in terms of dynamic systems theory, into “attractors,” constraining the dynamics of signification and multiplication of meanings in the perceiver’s response (see Varela 1999; Spivey and Dale 2004; Spivey 2007). Introduction  5

The complexity of response (or process complexity; see Cutting, this volume), which introduces the agentive dynamics in systemic descriptions, is the reverse of formal (aesthetic or informational) complexity. In encountering complex artistic forms, the recipient (the reader, viewer, or listener) may not be able to develop a quick response—­the kind of response that has been conceptualized by different strands in cognitive psychology and narratology as neuronal “mirroring,” simulation or a mental model. In his famous manifesto “Art as Technique” (1917), Viktor Shklovsky defined art in terms of difficulty, deautomatization, and the increased length of perception necessary to experience its artfulness. Formal complexity—­the complexity of organization and pattern—­translates into complex forms of recipient engagement and becomes synonymous with the difficulty of perception and interpretation. In the arts complexity may be puzzling and challenging and provoke cognitive disorientation and frustration (Hogan 2003, 23; Grishakova 2014). The arts engage the human ability to tolerate ambiguity or lack of certainty and support interest in exploring multiple alternate meanings (see, e.g., Empson 1947; Steiner 1978). The temporality of artistic works and representations inheres in a viewer’s or reader’s perception: the perceiver fleshes out indeterminacies and gaps or takes them in stride, insofar as certain indeterminacy of communication is expected and accepted. The reception of a work of art often borders on the real-­life experiences of the uncertain, challenging, extreme, sublime, or repulsive, but, ultimately, owing to their artfully impeded, deautomatized nature, as described by Shklovsky, such experiences invite reflection and bear on the ways in which they appear to us or are brought to our awareness. Our perceptions and sensations are primary takes or hypotheses about reality in which our sense of reality and truth or beliefs is grounded: judgment and higher cognition stem from these primary bodily sensations and develop through layers and complex interplay of signaling and response. This is a central thesis of the embodied cognition paradigm on which many chapters of this volume capitalize. The embodied cognition paradigm, which is currently influential in research on complexity in both sub-­and suprapersonal (social embodied cognition) systems, problematizes the traditional foundational paradigm of cognitive science, which is focused on mental activity at the expense of perception, sensing, and acting (see Shapiro 2011, 2014). Emphasizing 6  Grishakova and Poulaki

the mind-­body’s engagement with the environment as constitutive of the embedded and enactive process of cognition extending beyond the confines of the body to include aspects of the physical world, this paradigm disputes the existing computational models of the mind and the so-­called sandwich model of cognition, which considers the latter a separate stage between the input and output of information, between perception and motor behavior, with the brain viewed as a computational machine and processor of information (for example, as in Jerry Fodor’s Representational Theory of Mind). Seminal works such as The Embodied Mind, by Francisco Varela, Evan Thompson, and Eleanor Rosch (1991), or George Lakoff and Mark Johnson’s Philosophy in the Flesh (1999), as well as the influence of ecological theory of perception (James J. Gibson) cast doubts on the universal validity of cognitive-­mentalist approaches. Parallel advances in neuroscience, such as dynamic systems models of the brain and its functions, together with neurological and philosophical accounts of perception, affect, and emotion and their role in cognition (from Antonio Damasio’s Descartes’ Error [1994] to Alva Noë’s Action in Perception [2004]), also contributed to the paradigm change. The embodied approaches have been brought to the study of discourses, artifacts, and culture in general. Research on the reader, viewer, or user engagement, mental imagery, empathy, emotion, memory, attention, and enactment became prominent in linguistics, psychology, literary, art, and media studies since the 1990s (for instance, Gerrig 1993; Esrock 1994; Plantinga and Smith 1999; Louwerse and Kuiken 2004; Miall 2006; Grodal 2009; Plantinga 2009; Herman 2013; Caracciolo 2014; Caracciolo and Kukkonen 2014; Coёgnarts and Kravanja 2015; Hven 2017, among others). The classical Western concept of the rational disembodied subject had already been problematized by Sigmund Freud and Jacques Lacan and their disciples in the early twentieth century. Recent research in cognitive neuroscience further complicates the image of the rational controlling “self” by revealing both the multiplicity of self-­positions and the possibility of channeling these into a mimetically compelling, albeit illusory, coherent “self” in normal brain activity. Emotionally distinct subject positions are underpinned by attentional systems activated in different regions of the brain (Lewis 2002). From the neurobiological perspective, there is no difference between perceiving, imagining, Introduction  7

and remembering. No neurocognitive indicators exist that separate the content of perception from the content of memory or imagination. To discern them, a reflexive act of the “autonoetic consciousness” is needed, which loops back on its own workings (Brockmeier 2015, 48; see also Grishakova 2014). But as neuroscientific research suggests, the processes of interpretation and narrativization develop already in the low preconscious levels, the left hemisphere functioning as an “interpreter” of the right hemisphere’s activity and weaving together fact and invention to maintain the self’s sense of continuity (Gazzaniga 1998; see also Tikka and Kaipainen, in this volume). Daniel Kahneman’s (2011) conception of slow and fast thinking offers a rather skeptical view of the role of rational judgment in decision making and action, the latter preceding the former rather than being guided by it in neuropsychological dynamics (see also Wegner 2002). The role of the fast preconscious processing of information in human action proves to be much more significant than presumed by classical science and rationalist philosophy. These discoveries, so discouraging for the Cartesian agent-­centered world picture, are not merely deconstructive, however; they testify to the fact that alterity is part of self-­ constitution and that the cognizing agency is actually an embodied and distributed agency. Rather than being imprisoned in a solitary Cartesian mind, it not only embraces individual neuropsychological and somatic systems but also may extend to an entire group of individuals. The living systems, building and rebuilding themselves through an open exchange of information and energy with the environment, seem to be functioning in spite of the second law of thermodynamics and teetering on the verge of disappearance and chaos. Likewise, maintaining the continuity of the “self” or the act of self-­authoring requires some effort. It is always a trade-­off, a compromise between agentic and systemic dynamics. The idea of system-­agent interplay and tension is fundamental to the structuring of this volume, which seeks to maintain and emphasize this tension rather than taking a single, agent-­based or systems-­based perspective. By bringing together these perspectives, we highlight their dynamic embedding on multiple scales of narrativity. Narrative Complexity The variety in philosophical and aesthetic thinking as well as in experimental scientific research on complexity has not been sufficiently theo8  Grishakova and Poulaki

rized or adopted by narratology. In trying to fill this gap, we focus particularly on the issues of comprehension and meaning making, which are central to the self-­definition of narratology and afford the “conditions of possibility” for narratology to be a discipline, theory, methodology, or interpretive practice that various scholars consider it to be (see Kindt and Müller 2003). As Tom Kindt and Hans-­Harald Müller (2003) argue, the major challenge to narratology in becoming a theory of interpretation is formulating a conception of meaning that a standard theory of interpretation is expected to be based on. Recent developments and the spread of narrative research to various fields and disciplines testify that narratology tends to reinvent itself permanently by adopting various theories of meaning, from possible worlds and reception theories to cognitivist, evolutionary, and biocultural frameworks. These attempts have not always been integrated or aimed at full-­fledged theory building. Nevertheless, they have contributed to the remodeling of the practices of interpretation contingent on respective theoretical frameworks and extended narratology beyond the formal analysis of narrative. Recent decades have witnessed growing interest in complex, nonconventional narrative forms; complex types of temporality and causality; nonlinear and multiplot narratives; and various forms of polyvocal narration in literature (Richardson 2015; Alber 2016; McHale 1987, 1994; Grishakova 2012; Gibbons 2012; Bray, Gibbons, and McHale 2015, to name just a few); electronic literature and video games (Ciccoricco 2007, 2014; Eskelinen 2012; Ensslin 2014); cinema (Staiger 2006; Bordwell 2006, 2007; Poulaki 2011a; Buckland 2009, 2014; Kiss and Willemsen 2016; Hven 2017); television (Mittell 2015); and across media (Ryan 2004, 2006; Grishakova and Ryan 2010; Ryan and Thon 2014). Narrative complexity has been understood as the lack of a classical (Aristotelian) structure, with a beginning, middle, and end, replaced instead by fragmented, distributed, interactive narratives (e.g., Hyvärinen et al. 2010; Lehtimäki, Karttunen, Mäkelä 2012). The idea of narrative coherence was also contested by research on illness narratives (Hydén and Brockmeier 2008), conversational and online storytelling (Ochs and Capps 2001; Page 2012), and “vast”—­ overarching or serial—­narratives spreading across media (Harrigan and Wardrip-­Fruin 2009). These studies reveal that various narrative practices, genres, and media avail themselves of different types of complexiIntroduction  9

ty. While focusing on the narrative forms themselves and their complex features, the studies also took a cognitive and experiential direction, seeing their complexity as lying equally in the texts and the representations being structured unconventionally and in the impact that these complex structures have on the mind of the recipient. For instance, Jan Alber’s work (2016) links the “unnatural” and cognitive directions of narratology, initially perceived as antagonistic. Recasting the unnatural or, rather, complex in terms of reader engagement seems to be a promising trend. On the other hand, in the framework of narrative poetics, research on complex narratives risks recapitulating old dichotomies, such as conventional or habitual versus artistic and innovative. Narrative complexity has typically been defined against the backdrop of some canonical linear or mimetic narrative, which, one could argue, has never existed in a pure form; nevertheless, it has been considered primary and prototypical regarding more complex and thus “deviant” forms. While capitalizing on the formalist distinction between artistic and ordinary language, the “unnatural” trend in narrative theory (Richardson 2015) seeks to integrate the centrifugal antinarrative tendencies into the narrative discourse by forming a typology of “deviant” features found in antimimetic or nonmimetic narratives. At the same time, complex experimental narratives have been considered weakly narrative or “low in narrativity” (Ryan 2007, 30). In fact, there are many mixed forms on both sides of the divide between the everyday and the artistic: both “commonplace” and “surprise” contribute to experimental poetics (Grishakova 2009b). Although the process of narrativization has sometimes been perceived as a simplification (Morson 1994, 6) or, rather, a compromise between the overarching narrative logic and an effort to capture experience in all of its complexity (Ochs and Capps 2001, 4), ongoing exploration of various narrative practices and discovery of ever-­new narrative forms transmitted through various semiotic channels eventually embraced a more flexible understanding of narrativity. For instance, in his innovative research on narrative in music, Byron Almén espouses a “sibling” rather than a “descendant” (parasitic in literary narrative) model of musical narrativity (2017, 12). What he calls “narrative” is a weakly referential, but meaningful, pattern of interrelated musical units, evolving over time. Although not yet sufficiently conceptualized and integrated, attempts at developing sibling models and emancipatory moves in narrative research are 10  Grishakova and Poulaki

concomitant with the development of process philosophy, process linguistics, and an emergentist view of language with similar theoretical and methodological concerns and similar interest in nonlinear dynamic processes, self-­organization, and embodied cognition (Ballmer and Wildgen 1987; MacWhinney and O’Grady 2015). Narrative and Systems Some systematic attempts to link narrative complexity with complex systems theory have been made by literary and media scholars, for example, in contributions to the volume Chaos and Order, edited by Hayles (1991), linking literature with chaos theory (see also Argyros 1992; Gillespie 2003; Parker 2007). In particular, self-­referential embedding and metanarrative strategies in literature have been linked to narrative complexity by authors such as Joseph Tabbi (2002) and Bruce Clarke (2008). In film and media studies Jan Simons (2008) as well as Maria Poulaki (2011a, 2011b, 2014, 2018) have specifically addressed the connection between complexity theory and narrative. Richard Walsh (2011) and, more recently, Walsh and Susan Stepney (2018) attempt to rethink the (in)compatibility of narrative with complex systems, reintroducing a problem that had been raised in organization studies and sociology (Doreian 2001; Boje 2001) as well as in literary studies (Abbott 2008). The applicability of the complex-­systemic approaches and the concept of emergence to narrative have been contested by H. Porter Abbott in his 2008 article “Narrative and Emergent Behavior.” Abbott contends that, owing to the “centralized controlling instance,” the role of narrative in grasping complex emergent processes such as evolution is limited: arguably, narrative fails in the representation of certain kinds of complex causality at the risk of disintegration and loss of narrative form. He concedes, however, that many complex narratives (tragedies, films, long novels) lack centralized causal control in spite of the human proclivity to see the controlling agency, even in situations where there is none. The “controlling agency,” as defined by Abbott, seems to refer indiscriminately to authors, narrators, or characters in the storyworlds—­potentially, a multitude of interacting agents of different levels and degrees of control—­a situation that may, in fact, result in increased complexity. Marie-­Laure Ryan contends in her chapter in this volume that authorial control invalidates the complexity approach Introduction  11

to text-­author relations. Conversely, as Ryan observes, the comparison between narrative and a complex system remains productive on the level of the plot, where the controlling instance is seemingly absent. A parallel has been drawn between narrative dynamics and principles of nonequilibrium thermodynamics, such as stochastic processes, dissipative structures, and spontaneous self-­organization (Pier 2017; see also Hayles 1991; Weissert 1991; Poulaki 2011a). However, the traveling concept of a complex or dynamic system becomes a modeling metaphor when applied to linguistic or narrative representations (see multiple works after Hayles on “chaotic” or nonlinear narratives mimicking certain features of complex systems, such as openness and interaction of system’s constituents, e.g., Demastes 1998; Mackey 1999; Werner 1999; Parker 2007, among others). The heuristic value and applicability of the concept are contingent on the interpretive insights it yields. While narrative poetics is in need of a foundational theory of meaning and representation, both cognitive and systemic approaches to narrative provide such a theory. The complex-­systemic approaches may therefore be adopted as a kind of narrative semantics whose explanatory power, however, extends only to certain aspects of narrative production and reception. Instead of exploring the analogy between narrative and complex systems, this volume adopts a pragmatic perspective, one which embraces both the complexity of narrative forms and the complexity of their production and experience in various contexts. We do not consider narrative representation to be a simple restriction of environmental complexity. By reducing environmental complexity, narratives may become internally more complex, and higher internal differentiation of a system will further enable new interactions with other interrelated systems, constituting a more complex environment for their own organization, and so on. On the other hand, human agency and tension between systemic and agentic dynamics remain important for understanding narrative complexity. Narration as a process of meaning making—­evolving at the interfaces of natural and social systems and mediated by various author-­or narrator-­like instances, including performers, gatekeepers, and conveyors—­may lend a top-­down control to systemic dynamics and convey a systemic rather than a solely agentic complexity.5 A weakened cen12  Grishakova and Poulaki

tral control is characteristic of multiagent systems, such as folk and online communities, improvisation groups in theater and music, and film or television storytelling, with the authorial function distributed among distinct actors. Whole communities and societies may function as complex systems, in a state of disequilibrium evolving toward chaos or a higher order of complexity (see, e.g., Byrne 1998). However, the agent’s role and impact are much more significant in social systems compared to natural systems, and the logic of biological systems cannot be smoothly extended to societies or groups. The focus of our interest lies in narrative as a tool that may reveal, enhance, or suppress complexity by participating in agentic-­systemic dynamics. Respectively, narrative complexity in the title of this volume refers to variability, change, and open-­endedness on both agentic and systemic scales and amounts to the interplay of micro-­and macrosystems lending complexity to narrative representations. From this perspective various features and functions of narrative complexity can be subsumed under two dimensions:



1. Narrative representation and embodied mind: complex relations between narrative (verbal, graphic, visual, auditory, or mixed) representation and mind; the production and processing of narrative in terms of mind and body engagement (perception, memory, emotion, affect, and somatosensory effects); and complex scripts of communication and behavior related to narrative experience and sense making. 2. Narrative representation and environments: the complexity of narrative representation contingent on media and technological affordances; the circulation, repurposing, and reuse of narratives in social and technological environments: shared, distributed, and emergent narratives; artificial, machine-­generated narratives; and multimodal narratives, including the interplay of various semiotic modes and media.

Andy Clark refers to the human mind as “a complex distributed cognitive device whose problem-­solving routines are defined over an unruly mass of biological and non-­biological circuits and pathways” (2003, 141). Within this frame of reference, narrative complexity relates to what Introduction  13

Clark calls “the mind-­body-­scaffolding problem” as “the problem of understanding how human thought and reason is born out of looping interactions between material brains, material bodies, and complex cultural and technological environments. We create these supportive environments, but they create us too” (11). Complexity involves embeddedness and an interplay of systems that contribute to production, processing, and the experience of narratives as mediators among these systems. Embeddedness, however, is not synonymous with static containment, nor is it mere speculative abstraction. In complex-­systems theory, and especially in its autopoietic strands, from biology (Varela and Maturana) to sociology (Luhmann), systems and environments are dynamically co-­ constituted, rather than just enveloping one another. This dynamic co-­ constitution involves both bottom-­up and top-­down streams. To narrow and flesh out the concept of the system and to concretize it for our purposes, the term could be replaced with a more specific concept of narrative environment introduced by the social scholars Jaber Gubrium and James Holstein in their analysis of “narrative reality” co-­constructed by various agents.6 Narrative environments refer to spaces where storytelling takes place, mediated and shaped by local semiotic relevancies (2009, 123–­24). Different narrative environments foster different types of storytelling. In a combination of the complex-­ systems framework with Gubrium and Holstein’s approach, narrative environments would be conceived of as the top-­down influence of participation formats, material platforms, communication channels, and institutional or informal settings that mediate storytelling, determine participants’ roles and offer scripts for their behavior, affect the internal organization of stories, and foster the emergence of a definite type of narrative—­settings in which “boundless stories” develop and in turn shape, from the bottom up, their narrative environments: “They are boundless in that regard, are told and retold, with no definitive beginnings, middles, or ends in principle, even while a sense of narrative wholes and narrative organization is always in tow. This obliges us to analyze stories and storytelling in an extended framework, with the aim of documenting what the communication process and its circumstances designate as meaningful and important, together with the various purposes stories serve” (2009, 2). Various chapters in this volume highlight the interlaced, embedded 14  Grishakova and Poulaki

form of the communication and storytelling processes, the continuity between the environment and the processes of “storying,” the latter dispersed across environments that lend their complexity and blend with them. Narratives travel across various contexts; they are retold and repurposed in their trajectories and link to one another on various levels and scales (e.g., Agha 2007; Wortham and Reyes 2015). About This Volume This volume seeks to integrate analytical effort and conceptual insight from different disciplines—­narrative studies, media studies, psychology, computer science—­to find a better match between our theories of narrative representation and the complexities of intelligent behavior. To account for the complex narrative forms, we maintain the scalar concept of narrativity and consider narrative dynamics as a process embracing various levels of representation and response, seamlessly integrating heterogeneous constituents of narrativity into a provisional and evolving whole (see Ryan 2004, 2006 on the difference between being a narrative and possessing “narrativity”; and Esrock and Grishakova, both in this volume). We subscribe to the view that (proto-­)narrativity, originating in early ontogenetic communication practices, may be inherent in a multitude of narrative-­like forms, such as locative narratives merging verbal and audio narratives with bodily movement and parts of landscapes (see Whittaker, in this volume); stories processed on various platforms and mobile devices (see Ekman; Knoller; and Ciccoricco and Large, all in this volume); and gameplay in videogames, jazz improvisation, or even dreams and reveries (for instance, Flanagan [2000, 12] suggests that producing “increasingly complex self-­expressive and self-­revealing narratives” may be one of the dream’s functions). Narrative functions as a blueprint calling for fleshing out by the recipient. Within this framework of experiential aspects and “embodiment effects” (“a seemingly physical sense of rich perceptual and motor descriptions,” Sanford and Emmott 2012, 268), the whole rich array of iconic-­indexical connections weaving together the human body and mind in the living fabric of reality becomes constitutive of narrative comprehension. Narrative comprehension involves integration of different layers of information—­rich percepts, sensorimotor experiences, attentional structuring, retrieval of memory images, and complex meaningful contexts stitched into patterns. While Introduction  15

drawing on the vocabulary of thermodynamics to describe the ability of artful narratives to heighten our vision through “iconic increase,” Paul Ricoeur calls this effect “scaling the entropic slope of ordinary perception, inasmuch as perception tends to level out differences and soften contrasts” (1994, 124). The embodied approaches to narrative involve what Mark Johnson (2007) calls the “non-­representational” dimensions of meaning—­ affective, perceptual, and experiential—­which contribute to its complexity. However, the human capacity of operating with symbols (signs) that include a representational component and are thereby distinguished from natural “signals” or “stimuli”—­the capacity that Piaget calls the “semiotic” or “symbolic” function, growing out of bodily action and affective-­ sensorimotor patterns (e.g., Wagoner 2010) and, ultimately, undergirding cultural habits and conventions—­creates a distance between organism and environment and, by seeking simultaneously to articulate and cover the distance, mediates the human’s and environment’s relations. What is perceived as meaningful is therefore always symbolically mediated and representational, even if it involves representations of different kinds and levels.7 The complex-­systemic view, however, casts a fresh light on the dynamics between narrative and environments. If we see narrative itself as an interface of complex adaptive systems, then interesting questions arise, not only about the idea of narrative as an emergent product of dynamic feedback between a representation and a mind in a sort of closed loop but also, from a broader perspective, as the emergent product of multiple interconnections of the embodied mind with a lived physical, social, and technological environment. In her chapter in this volume, Ryan particularly addresses the issue of narrative emergence, arguing that all narratives are about the emergent properties of complex networks of interconnected characters; however, she distinguishes between narratives of emergence and emergent narratives, that is, interactive and open-­ended stories, pointing to a distinction between epistemological and ontological planes of narrative complexity. In the complex coevolution of narrative systems and environments, mental, social, and technical, new media constitute new affordances for narrative and new forms of user engagement, triggering further evolution and adaptation, as shown in Ulrik Ekman and colleagues (2015). Ubiquitous computing and the embeddedness of smart devices in 16  Grishakova and Poulaki

everyday environments expand even more the horizons of “narrative reality” (Gubrium and Holstein 2009) and narrative intelligence (see Ekbia, in this volume) and challenge a conception of narrative based on closure. Narratives in new media interfaces (small and mobile devices and touch screens) are discussed in the chapters by David Ciccoricco and David Large and by Emma Whittaker, who see the new kinds of narrative representation not just as simulations of reality or Baudrillardian simulacra, but as bearing truth value (in the sense of Jamesian pragmatism, as pointed out by Whittaker) and embracing the body more holistically by means of metaphor, as shown in the discussion of the hybrid media novel Pry by Ciccoricco and Large. Such novel kinds of representation could perhaps be considered a “userly text” from Noam Knoller’s perspective, who in his chapter suggests this term and a model for interactive storytelling able to address environmental complexity without reducing it. In a similar vein, Ekman inquires how the disorganized complexity of cybernetic media ecology reconstitutes the self in the form of distributed autobiographical traces captured by intelligent devices, in particular the “intelligent assistants” that have received little narratological attention so far. Narrative complexity as affordance of human or artificial language, and the issues of linguistic and multimodal interpretation arising in the encounter with narratives in different media, are addressed in the chapters by Joseph P. Magliano, Karyn Higgs and James Clinton; Neil Cohn; James E. Cutting; and Hamid R. Ekbia. Magliano, Higgs, and Clinton consider the complexity inserted into the trans-­symbolic “back-­end” processing of narrative by the differential impact of distinct media and their own modalities. Cohn’s chapter argues for a “complex hierarchic embedding” of the culturally specific grammar of visual narratives, which, through recursion, can generate infinite constructions. Cutting is interested in the interplay between formal narrative complexity associated with multiplicity, interconnectedness and embedding, and process complexity, considering both textual and audiovisual narratives. Last, Ekbia assesses the (im)possibility of artificial intelligence systems to reach the heteroglossia of human language and the complexity of its narratives, refuting the idea of a “universal grammar” on which a mentalist paradigm for artificial intelligence development is based. The subpersonal level of cognitive-­embodied interactions constituting narrative at the interface of agents and representation systems is adIntroduction  17

dressed in the next cluster of chapters. Temporal, embodied, and subjective dimensions of the experience of narrative come under the spotlight. Mieke Bal considers the case of direct discourse (“narrative embedding”), which, traversing historical categorizations of realism, modernism, and postmodernism, produces “subjective fusion,” thereby undermining the single narrating voice and manifesting subpersonal as well as suprapersonal levels of complexity. Linking narrative representation and the embodied response, Ellen J. Esrock coins the concept of “transomatization” to capture how bodily rhythms and physiological patterns synchronize with the patterns of representation and weave narrative experience and comprehension as an emergent product of different systemic processes. The case of film narrative is discussed in the next chapter, where Poulaki considers how the interplay of distancing and immersive effects in cinema creates an expanded sense of embodiment feeding into the narrative experience. Pia Tikka and Mauri Kaipainen shift the focus to the temporality of narrative experience, also using a film case to suggest a computational model for the shared experience of narrative and its neurological manifestations. Drawing on Edmund Husserl’s model of nowness and Varela’s extension of it in what he called the “specious present,” the model of Tikka and Kaipainen seeks to capture in experimental conditions the complex ontospace of narrative experience, the space between retention and protention. A similar theoretical framework is developed by Martin Rosenberg to address the complex top-­down and bottom-­up cognitive processes involved in jazz and its emergent narrativity “in the moment.” The last set of chapters reaches to the macrolevel of narrative complexity and considers the evolution of narrative as a cognitive mode evolving and mutating in a constant interweaving with larger, historically evolving biocultural systems. The chapters by Marina Grishakova and James Carney draw on natural and cultural evolutionary theories: Grishakova takes a post-­Darwinian approach to narrative and the doubleness inherent in its biocultural co-­constitution and considers its predictive-­ corrective logic as contributing to the eventual complexification of the relations between narrative form, attention, and cognitive attunement. Carney combines evolutionary psychology with cognitive psychology to investigate the evolutionary function of fictional narrative. José Angel García Landa’s chapter takes a broader perspective: he sees evolution itself as a protonarrative, embedding local narratives in a complex histor18  Grishakova and Poulaki

ical morph, which is graspable only “in hindsight.” Hence, the chapter raises the issue of narrative explanation, which has been tackled by historiographers over decades, and offers a toolkit of concepts necessary to deal with the narrativity of complex events. It is not an accident that ambiguity and uncertainty, both from a phenomenological and a complex-­systemic point of view, is an underlying theme connecting many chapters in this volume. The “doubleness” and “double take” of narrative, its inhabiting of complex temporalities (between retention and protention, in hindsight); bodies (between internal and external, mental and media representation, metaphor and simulation); and planes of organization (suggested by complex hierarchic embedding, bottom-­up and top-­down determination) are key concepts and recurrent themes that weave the volume’s own interpretive pattern. Such concepts capture the undercurrents that introduce ambiguity into constructed interpretations and subjective formations, and the processes that alter existing trajectories by creating instability, which is perhaps the pivotal force of complexity. Prigogine, with Isabelle Stengers, describes bifurcation points as manifestations of “an intrinsic differentiation between parts of the system itself and its environment” (1997, 69), and the volume’s various chapters try to arrest the complexity of narrative found at a given bifurcation point, the intrinsic uncertainty of any process of meaning making and any encounter with an artwork. Ernst Gombrich said that “the very effort after meaning . . . will tend to hide ambiguity from us as long as possible” ([1960] 2004, 396). To approach complexity means to dive into this ambiguity and explore its convolutions. Notes 1. The study was based on an analysis of archaeological and historical data from 414 societies, spanning the past ten thousand years; see Turchin (2017). 2. David Letzler (2017) calls this overcomplexification cruft in his book on “mega-­novels.” Letzler’s title plays with Percy Lubbock’s classic The Craft of Fiction. Etymologically, cruft originates in technical slang and is “defined by The New Hacker’s Dictionary as ‘Excess; superfluous junk; used esp. of redundant or superseded code’” (qtd. in Letzler 2017, 5). 3. Compare neuroscientists J. A. Scott Kelso and David Engstrøm’s “philosophy of complementary pairs” (2006), that is, seemingly contradictory, mutually exclusive tendencies of behavior, such as integration and segregation Introduction  19

that are manifested by complex systems (human minds and brains) and are necessary to maintain their dynamics and coordination. 4. See the “bifurcation points” in Prigogine or “explosions” in Lotman. On the impact of Prigogine’s conceptions and terminology on Lotman’s work of the 1980–­90s, see Grishakova’s afterword (2009a). 5. Systemic logic ascribes certain roles and scripts of behavior to the participating agents. 6. Earlier, the new media artist and philosopher Mark Amerika (1997) referred to his web-­based projects, such as grammatron, as “public-­ domain narrative environments”: “grammatron depicts a near-­future world where stories are no longer conceived for book production but are instead created for a more immersive networked-­narrative environment that, taking place on the Net, calls into question how a narrative is composed, published and distributed in the age of digital dissemination.” By “narrative environment” Amerika means “the stream of information that flows through cyberspace, waiting to be harnessed into ‘a nomadic narrative’” (Ryan 2006, xvi). 7. Cf. Charles Sanders Peirce’s assumption that there are, possibly, representamens (signifiers) that are not signs; an example of such a representamen is a sunflower turning toward the sun and thereby indirectly representing it. However, Peirce also concedes that “thought is the chief, if not the only, mode of representation” (1932, para. 274)—­that is, the turning sunflower appears to be a representation (a sign) only for an interpreting mind. References Abbott, H. Porter. 2008. “Narrative and Emergent Behavior.” Poetics Today 29 (2): 227–­44. Agha, Asif. 2007. Language and Social Relations. Cambridge: Cambridge University Press. Alber, Jan. 2016. Unnatural Narrative: Impossible Worlds in Fiction and Drama. Frontiers of Narrative. Lincoln: University of Nebraska Press. Almén, Byron. 2017. A Theory of Musical Narrative. Bloomington: Indiana University Press Amerika, Mark. 1997. “grammatron.” Accessed April 5, 2018. www​ .grammatron​.com​/about​.html. Argyros, Alex. 1992. “Narrative and Chaos.” New Literary History 23 (3): 659–­73. Ballmer, Thomas T., and Wolfgang Wildgen. 1987. Process Linguistics: Exploring the Processual Aspects of Language and Language Use, and the Methods of Their Description. Tübingen: Niemeyer Verlag. 20  Grishakova and Poulaki

Boje, David. M. 2001. Narrative Methods for Organizational and Communication Research. London: Sage. Bordwell, David. 2006. The Way Hollywood Tells It: Story and Style in Modern Movies. Berkeley: University of California Press. —. 2007. Poetics of Cinema. New York: Routledge. Bray, Joe, Alison Gibbons, and Brian McHale, eds. 2015. The Routledge Companion to Experimental Literature. London: Routledge. Brockmeier, Jens. 2015. Beyond the Archive: Memory, Narrative, and the Autobiographical Process. Oxford University Press. Buckland, Warren, ed. 2009. Puzzle Films: Complex Storytelling in Contemporary Cinema. Chichester-­Malden ma: Blackwell. —, ed. 2014. Hollywood Puzzle Films. London: Routledge. Byrne, David. 1998. Complexity Theory and Social Sciences. London: Routledge. Caracciolo, Marco. 2014. The Experientiality of Narrative: An Enactivist Approach. Berlin: De Gruyter. Caracciolo, Marco, and Karin Kukkonen, eds. 2014. Cognitive Literary Study: Second Generation Approaches. Special issue, Style 48:3. Castells, Manuel. 1996–­98. The Information Age. Vols. 1–­3. Oxford-­Malden ma: Blackwell. Casti, John, and Anders Karlqvist, eds. 2003. Art and Complexity. Amsterdam: Elsevier. Ciccoricco, David. 2007. Reading Network Fiction. Tuscaloosa: University of Alabama Press. —. 2014. Refiguring Minds in Narrative Media. Lincoln: University of Nebraska Press. Clark, Andy. 2003. Natural Born Cyborgs: Minds, Technologies, and the Future of Human Intelligence. New York: Oxford University Press. Clarke, Bruce. 2008. Posthuman Metamorphosis: Narratives and Systems. New York: Fordham University Press. Coёgnarts, Maarten, and Peter Kravanja. 2015. Embodied Cognition and Cinema. Leuven: Leuven University Press. Damasio, Antonio R. 1994. Descartes’ Error: Emotion, Rationality and the Human Brain. New York: Avon Books. Demastes, William W. 1998. Theatre of Chaos: Beyond Absurdism, into Orderly Disorder. Cambridge: Cambridge University Press. Doreian, Patrick. 2001. “Causality in Social Network Analysis.” Sociological Methods and Research 30 (1): 81–­114. Eco, Umberto. 1986. Semiotics and the Philosophy of Language. Bloomington: Indiana University Press. Introduction  21

Ekman, Ulrik, Jay David Bolter, Lily Diaz, Morten Søndergaard, and Maria Engberg, eds. 2015. Ubiquitous Computing, Complexity and Culture. London: Routledge. Empson, William. 1947. Seven Types of Ambiguity. New York: New Directions. Ensslin, Astrid. 2014. Literary Gaming. Cambridge ma: mit Press. Eskelinen, Markku. 2012. Cybertext Poetics. The Critical Landscape of New Media Literary Theory. London: Continuum. Esrock, Ellen. 1994. The Reader’s Eye: Visual Imaging as Reader Response. Baltimore: Johns Hopkins University Press. Flanagan, Owen. 2000. Dreaming Souls: Sleep, Dreams, and the Evolution of the Conscious Mind. Oxford: Oxford University Press. Gazzaniga, Michael S. 1998. The Mind’s Past. Berkeley: University of California Press. Gerrig, Richard J. 1993. Experiencing Narrative Worlds: On the Psychological Activities of Reading. New Haven: Yale University Press. Gibbons, Alison. 2012. Multimodality, Cognition, and Experimental Literature. London: Routledge. Gillespie, Michael Patrick. 2003. The Aesthetics of Chaos: Nonlinear Thinking and Contemporary Literary Criticism. Gainesville: University Press of Florida. Gombrich, Ernst H. (1960) 2004. Art and Illusion: A Study in the Psychology of Pictorial Representation. London: Phaidon. Grishakova, Marina. 2009a. “Afterword: Around Culture and Explosion.” In Lotman 2009, 175–­88. —. 2009b. “Beyond the Frame: Cognitive Science, Common Sense, and Fiction.” Narrative 17 (2): 188–­99. —. 2012. The Models of Space, Time and Vision in V. Nabokov’s Fiction: Narrative Strategies and Cultural Frames. The Hague: Open Access Publications in European Networks. www​.oapen​.org​/search​?identifier​=​ 421498. —. 2014. “Fiction as a Cognitive Challenge: Explorations into Alternative Forms of Selfhood and Experience.” In Cognition, Literature, and History, edited by Mark J. Bruhn and Donald R. Wehrs, 190–­208. New York: Routledge. Grishakova, Marina, and Marie-­Laure Ryan, eds. 2010. Intermediality and Storytelling. New York: De Gruyter. Grodal, Torben. 2009. Embodied Visions: Evolution, Emotion, Culture and Film. Oxford: Oxford University Press. Gubrium, Jaber F., and James A. Holstein. 2009. Analyzing Narrative Reality. London: Sage. Harrigan, Pat, and Noah Wardrip-­Fruin. 2009. Third Person: Authoring and Exploring Vast Narratives. Cambridge ma: mit Press. 22  Grishakova and Poulaki

Hayles, Katherine N. 1990. Chaos Bound: Orderly Disorder in Contemporary Literature and Science. Ithaca: Cornell University Press. —. ed. 1991. Chaos and Order: Complex Dynamics in Literature and Science. Chicago: University of Chicago Press. —. 1999. How We Became Posthuman: Virtual Bodies in Cybernetics, Literature and Informatics. Chicago: University of Chicago Press. Herman, David. 2013. Storytelling and the Sciences of Mind. Cambridge ma: mit Press. Hogan, Patrick Holm. 2003. Cognitive Science, Literature, and the Arts: A Guide for Humanists. London: Routledge. Hven, Steffen. 2017. Cinema and Narrative Complexity: Embodying the Fabula. Amsterdam: Amsterdam University Press. Hydén, Lars-­Christer, and Jens Brockmeier. 2008. Health, Illness, and Culture: Broken Narratives. London: Routledge. Hyvärinen, Matti, Lars-­Christer Hydén, Marja Saarenheimo, and Maria Tamboukou, eds. 2010. Beyond Narrative Coherence. Amsterdam: Benjamins. Johnson, Mark L. 2007. The Meaning of the Body: Aesthetics of Human Understanding. Chicago: University of Chicago Press. Kahneman, Daniel. 2011. Thinking, Fast and Slow. New York: Farrar, Straus and Giroux. Kelso, J. A. Scott, and David A. Engstrøm. 2006. The Complementary Nature. Cambridge ma: mit Press. Kindt, Tom, and Hans-­Harald Müller. 2003. “Narrative Theory and/or/as Theory of Interpretation.” In What Is Narratology? Questions and Answers Regarding the Status of a Theory, edited by Tom Kindt and Hans-­Harald Müller, 205–­19. Berlin: De Guyter. Kiss, Miklós, and Steven Willemsen. 2016. Impossible Puzzle Films. A Cognitive Approach to Contemporary Complex Cinema. Edinburgh: Edinburgh University Press. Lakoff, George, and Mark Johnson. 1999. Philosophy in the Flesh. New York: Basic Books. Lehtimäki, Markku, Laura Karttunen, and Maria Mäkelä, eds. 2012. Narrative, Interrupted: The Plotless, the Trivial and the Disturbing in Literature. Berlin: De Gruyter. Letzler, David. 2017. The Cruft of Fiction: Mega-­Novels and the Science of Paying Attention. Lincoln: Nebraska University Press. Lewis, Marc D. 2002. “The Dialogical Brain: Contributions of Emotional Neurobiology to Understanding the Dialogical Self.” Theory and Psychology 12:175–­90. Introduction  23

Lotman, Yuri. 1977. The Structure of the Artistic Text. Translated by Gail Lenhoff and Ronald Vroon. Ann Arbor: University of Michigan. —. 2009. Culture and Explosion. Edited by Marina Grishakova. Translated by Wilma Clark. Semiotics, Communication, Cognition 1. New York: De Gruyter. Louwerse, Max, and Don Kuiken, eds. 2004. The Effects of Personal Involvement in Narrative Discourse. Special Issue, Discourse Processes 38:2. Luhmann, Niklas. 1995. Social Systems. Translated by John Bednarz Jr. and Dirk Baecker. Stanford: Stanford University Press. Mackey, Peter Francis. 1999. Chaos Theory and James Joyce’s Everyman. Gainesville: University Press of Florida. MacWhinney, Brian, and William O’Grady. 2015. The Handbook of Language Emergence. Malden ma: Wiley-­Blackwell. McHale, Brian. 1987. Postmodernist Fiction. London: Routledge —. 1994. Constructing Postmodernism. London: Routledge. Miall, David S. 2006. Literary Reading: Empirical and Theoretical Studies. New York: Lang. Mittell, Jason. 2015. Complex tv: The Poetics of Contemporary Television Storytelling. New York: New York University Press. Morin, Edgar. 2008. On Complexity. Translated by Robin Postel. Cresskill nj: Hampton. Morson, Gary Saul. 1994. Narrative and Freedom: The Shadows of Time. New Haven: Yale University Press. Noë, Alva. 2004. Action in Perception. Cambridge ma: mit Press. Ochs, Elinor, and Lisa Capps. 2001. Living Narrative: Creating Lives in Everyday Storytelling. Cambridge: Harvard University Press. Page, Ruth. 2012. Stories and Social Media: Identities and Interaction. New York: Routledge. Parker, Jo Alison. 2007. Narrative Form and Chaos Theory in Sterne, Proust, Woolf, and Faulkner. New York: Palgrave Macmillan. Paulson, William. 1991. “Literature, Complexity, Interdisciplinarity.” In Hayles 1991, 37–­53. Peirce, Charles Sanders. 1932. Collected Papers. Vol. 2. Boston: Harvard University Press. Piaget, Jean. 1970. Structuralism. Translated by Chaninah Maschler. New York: Basic Books. Pier, John. 2017. “Complexity: A Paradigm for Narrative?” In Emerging Vectors of Narratology, edited by Per Krogh Hansen, John Pier, Philippe Roussin, and Wolf Schmid, 533–­66. Berlin: De Gruyter.

24  Grishakova and Poulaki

Plantinga, Carl R. 2009. Moving Viewers: American Film and the Spectator’s Experience. Berkeley: University of California Press. Plantinga, Carl R., and Greg M. Smith, eds. 1999. Passionate Views: Film, Cognition, and Emotion. Baltimore: Johns Hopkins University Press. Poulaki, Maria. 2011a. “Before or Beyond Narrative? Towards a Complex Systems Theory of Contemporary Films.” PhD diss., University of Amsterdam. —. 2011b. “Implanted Time: ‘The Final Cut’ and the Reflexive Loops of Complex Narratives.” New Review of Film and Television Studies 9 (4): 415–­34. —. 2014. “Network Films and Complex Causality.” Screen 55 (3): 379–­95. —. 2018. “Emergent Causality in Complex Films and Complex Systems.” In Walsh and Stepney, 213–­31. Prigogine, Ilya. 1997. The End of Certainty: Time, Chaos, and the Laws of Nature. With Isabelle Stengers. New York: Free Press. Richardson, Brian. 2015. Unnatural Narrative: Theory, History, and Practices. Columbus: Ohio State University Press. Ricoeur, Paul. 1994. “Imagination in Discourse and in Action.” In Rethinking Imagination: Culture and Creativity, edited by Gillian Robinson and John F. Rundell, 118–­35. London: Routledge. Ryan, Marie-­Laure, ed. 2004. Narrative across Media: The Languages of Storytelling. Lincoln: University of Nebraska Press. —. 2006. Avatars of Story. Minneapolis: University of Minnesota Press. —. 2007. “Toward a Definition of Narrative.” In The Cambridge Companion to Narrative, edited by David Herman, 22–­35. Cambridge: Cambridge University Press. Ryan, Marie-­Laure, and Jan-­Noёl Thon. 2014. Storyworlds across Media: Towards a Media-­Conscious Narratology. Lincoln: University of Nebraska Press. Sanford, Anthony J., and Catherine Emmott. 2012. Mind, Brain and Narrative. Cambridge: Cambridge University Press. Shapiro, Lawrence. 2011. Embodied Cognition. London: Routledge. —, ed. 2014. The Routledge Handbook of Embodied Cognition. London: Routledge. Simons, Jan. 2008. “Complex Narratives.” New Review of Film and Television Studies 6 (2): 111–­26. Spivey, Michael J. 2007. The Continuity of Mind. Oxford: Oxford University Press. Spivey, Michael J., and Rick Dale. 2004. “On the Continuity of Mind: Toward a Dynamical Account of Cognition.” Psychology of Learning and Motivation 45:87–­142. Staiger, Janet, ed. 2006. Film Criticism. Special issue, Complex Narratives 31 (1–­2). Steiner, George 1978. On Difficulty and Other Essays. Oxford: Oxford University Press.

Introduction  25

Tabbi, Joseph. 2002. Cognitive Fictions. Minneapolis: University of Minnesota Press. Turchin, Peter, Thomas E. Currie, Harvey Whitehouse, Pieter François, Kevin Feeney, Daniel Mullins, Daniel Hoyer, et al. 2017. “Quantitative Historical Analysis Uncovers a Single Dimension of Complexity That Structures Global Variation in Human Social Organization.” Proceedings of the National Academy of Sciences. Accessed February 1, 2018. www​.pnas​.org​/cgi​/doi​/10​ .1073​/pnas​.1708800115. Urry, John. 2003. Global Complexity. Oxford-­Malden ma: Polity. Varela, Francisco J. 1999. “The Specious Present: A Neurophenomenology of Time Consciousness.” In Naturalizing Phenomenology: Issues in Contemporary Phenomenology and Cognitive Science, edited by Jean Petitot, Francisco J. Varela, Bernard Pachoud, and Jean-­Michel Roy, 266–­329. Stanford: Stanford University Press. Varela, Francisco, Evan Thompson, and Eleanor Rosch. 1991. The Embodied Mind: Cognitive Science and Human Experience. Cambridge ma: mit Press. Wagoner, Brady 2010. “Introduction: What Is a Symbol?” In Symbolic Transformation: The Mind in Movement through Culture and Society, edited by Brady Wagoner, 1–­16. London: Routledge. Walsh, Richard. 2011. “Emergent Narrative in Interactive Media.” Narrative 19 (1): 72–­85. Walsh, Richard, and Susan Stepney, eds. 2018. Narrating Complexity. Cham: Springer. Wegner, Daniel M. 2002. The Illusion of Conscious Will. Cambridge ma: mit Press. Weissert, Thomas. 1991. “Representation and Bifurcation: Borges’s Garden of Chaos Dynamics.” In Hayles 1991, 223–­43. Werner, Hans C. 1999. Literary Texts as Nonlinear Patterns: A Chaotics Reading of “Rainforest,” “Transparent Things,” “Travesty,” and “Tristram Shandy.” Göteborg: Acta Universitatis Gothoburgensis. Wimsatt, William C. 2008. “Aggregativity: Reductive Heuristics for Finding Emergence.” In Emergence: Contemporary Readings in Philosophy and Science, edited by Mark A. Bedau and Paul Humphreys, 99–­110. Cambridge ma: mit Press. Wortham, Stanton, and Angela Reyes. 2015. Discourse Analysis beyond the Speech Event. New York: Routledge. Zyngier, Sonia, Willie Van Peer, and Jèmeljan Hakemulder. 2007. “Complexity and Foregrounding: In the Eye of the Beholder?” Poetics Today 28 (4): 653–­82.

26  Grishakova and Poulaki

Part 1

Narrative Complexity and Media

1

Narrative as/and Complex System/s Marie-­L aure Ryan

What does it mean to say “narrative is complex” or “narrative is a complex system”? Complex compared to what: ordering a beer, following a recipe, proving a theorem? Are all narratives complex or only some of them—the ones considered “literary” or, to extend the idea to all media, the ones considered artistic? Is the scientific notion of complexity useful to narrative theory, or should narrative theory develop its own notion of complexity? Such are the questions that I propose to address in this chapter. In systems theory complexity is associated with concepts such as emergence, nonlinearity, decentralized control, feedback loops, recursion, self-­ organization, simulation, and distributed intelligence and with formulas such as “the whole is more than the sum of its parts” or “small events can have vast consequences” (the famous butterfly effect). The so-­called science of complexity is a loosely defined field that extends into many disciplines and covers a wide variety of phenomena: the organization of ant colonies and beehives; the mode of operation of the brain; the fluctuations of the stock market; the evolution of species; the development of the weather; the organization of the World Wide Web; and the functioning of the immune system, to name only a few. What these systems have in common, according to scientist Melanie Mitchell, is (a) collective behavior: they are made of a number of individual elements that follow relatively simple rules, without the guidance of a leader; (b) signal and information processing: all these systems produce signals, exchanged both internally (i.e., between their components) and externally (between the system and its external environment); and (c) adaptation: these systems change their behavior according to the circumstances, to improve their chance of survival (2011, 12–­13). Though there is no unanimously accept29

ed definition of complexity, nor a fixed set of transdisciplinary criteria for measuring and comparing complexity across systems, these conditions lead Mitchell to the following definition: a complex system is “a system in which large networks of components with no central control give rise to complex collective behavior, sophisticated information processing and adaptation via learning or evolution” (13). (Note, however, the redundancy, which points to the difficulty of defining complexity: a complex system is one that “give[s] rise to complex collective behavior” and “sophisticated [or complex] information processing.”) Other scientists dispute the necessity of adaptation: the late John Holland, a leading authority in the field, argued that there are two types of complexity: complex physical systems, or cps, and complex adaptive systems, or cas (2014, 6–­9). An example of cps is a cellular automaton, such as John Conway’s famous Game of Life: in a cellular automaton, a set of rules is applied over and over again to dots arranged on a grid, causing the global state of the system to change and resulting in unpredictable visual patterns, but the rules remain the same, and the system evolves on its own without input from the designer (“Conway’s Game of Life” 2018). In a cas, by contrast, “the elements, usually called agents, learn to adapt in response to interaction with other agents” (Holland 2014, 8). Moreover, “as the agents adapt to each other, new agents with new strategies usually emerge” (9). Whether or not the individual components of a system are capable of adaptation, the trademark of complex systems is the lack of a central controlling entity, comparable to the cpu of a computer, that pursues deliberate goals. Out of the simple rules followed by the components of complex systems arise (or more precisely, emerge) patterns and behaviors that could not be predicted by simply looking at the rules or by solving an equation. Therein resides the difference between a simple, linear system and a complex system. With a simple system there is a formula (an equation) that makes it possible to calculate the state of the system at a given time without calculating all the previous states. For instance, if a car moves at a steady speed of a hundred kilometers per hour, we know how far it has gone at a certain time by giving a value to t in the equation x =100t, where t represents the number of hours the car has been traveling and x the distance, without having to calculate how far it has gone at time t-­1, t-­2, and so on. But with a complex system, such as a cellu30  Ryan

lar automaton, there is no equation that predicts future states. To know what the global state of the system will be like at time t, it is necessary to compute its state at all preceding times, by recursively applying its rules to their own output (or new rules if there is adaptation). In other words, the state of a complex system at a future time t cannot be directly calculated, but it can be obtained through a simulation. There is consequently a close relation between emergence and simulation: we run a simulation to discover emergent properties; conversely, a property is emergent, when it can be discovered only through a simulation. An often cited example of a complex, emergent system is the ant colony (Johnson 2001). An ant colony consists of large numbers of individuals who follow very simple rules: they forage for food, they fight intruders, they leave chemical signals when they find food, and they respond to the signals left by other ants by joining in the trail. No ant has a global plan for the building and maintenance of the colony—­in fact no ant may even be aware of its existence. Yet, as Mitchell explains, “the ants in a colony, each performing its own relatively simple actions, work together to build astoundingly complex structures that are clearly of great importance to the survival of the colony as a whole” (2011, 4). A stunning example of this cooperation is the building of bridges, where ants use their own bodies to allow the passage of the colony over a gap. When the units of one level combine to form units of a higher level, the whole becomes more than the sum of its parts, and the system is hierarchical. This property is regarded by scientists (i.e., Holland 2014, 4) as the trademark of nonlinearity. Given this all-­too-­brief survey of the key concepts of complexity theory, how can one address the issue of narrative complexity? The notion of complexity was not invented by scientists, and it is quite possible to develop an approach that owes nothing to them. Moreover, the idea of narrative complexity can be conceived in two ways: as a property of narrative in general or as a property of certain narratives, so the set of all stories can be ranked on a continuum running from simple (minimal) stories to maximally complex ones. By cross-­classifying dichotomies we can divide the study of narrative complexity into four categories:

1. An approach, not inspired by science, to the complexity inherent to all narratives Narrative as/and Complex System/s  31



2. An approach, not inspired by science, to the complexity of a subset of all narratives 3. An approach, inspired by science, to the complexity inherent to all narratives 4. An approach, inspired by science, to the complexity of a subset of all narratives

Here I concentrate on points 3 and 4, leaving their distinction to the conclusion. But the criteria for considering an approach as inspired by science are rather loose. Complexity fascinates, and some of the keywords of complexity science, such as emergence, nonlinearity, and feedback loops have infiltrated literary-­critical discourse (for instance, see Hayles 2005), but it is very difficult to draw a line between those uses that are too highly metaphorical to deserve serious consideration and those that involve a reasonably close, interesting analogy between two domains that remain far apart in their objects and methods. If we compare the concepts of complexity theory to those of physics, such as force, mass, energy, gravity, entropy, superposition, and so on, we find that they are often not original to the field and that their meaning is extremely vague: there are at least thirty-­one different scientific definitions of complexity (Horgan 1995, 107). This vagueness explains, in part, their popularity with literary critics, but it also questions their scientific status—­a questioning that actually extends to the whole field of complexity theory.1 My standard of “scientificity” will therefore rest on the use of two concepts that theorists regard as constitutive of complex systems: (1) decentralized control and (2) emergence. It may seem artificial to study them separately, since decentralized control is a prerequisite for emergence: “Emergence refers to the spontaneous creation of order and functionality from the bottom up” (Page 2009, 20). Yet although emergence presupposes decentralized control, these two concepts have distinct narrative applications. Decentralized Control At first sight narrative and complex systems are polar opposites: complexity rejects the idea of a system controlled from the top down by a central authority, but narratives are the work of an author or authorial collective. Complex systems constantly evolve, often in unpredictable 32  Ryan

ways, but narrative, once inscribed on the page or in celluloid, no longer changes. Yet if we move from the level of author-­text relations to the level of the plot, the comparison of narrative and complex systems become much more productive. For a narrative application of complexity theory, I propose to go back to Mitchell’s (2011) definition of complex systems quoted on the first page of this chapter. The elements mentioned in condition (a) can be associated with characters and the rules that control their behavior with the principles that motivate character actions: goals, plans, desires, fears, and values. The exchange of signals of condition (b) correspond to the circulation of information between characters (mostly through dialogue) and between characters and the world at large, and the adaptive behavior of condition (c) is represented by the characters’ ability to react to changes in the global state of the storyworld. From the three conditions we can thus derive a reasonably accurate model of plot. Prototypical narratives are about dynamic networks of human (or humanlike) relations, and these networks can be conceived, at least to some extent, as complex systems. The analogy requires, however, some adaptations: the number of elements is far smaller in narrative than in most complex systems; the rules that motivate the behavior of characters are not nearly as simple as those that govern the behavior of ants or of dots in a cellular automaton; the elements of the system follow their own personal rules rather than all behaving according to the same principles; and adaptation means that the rules can change as the characters evaluate new situations. Humans, unlike ants, are able to alter their behavior, which is why human societies are much more susceptible to change (and narratively much more interesting) than ant colonies. While authors control characters and plots from the top down, using them to pursue certain artistic goals or to demonstrate certain moral or ideological theses, within the fictional world there is no central controlling instance.2 Characters think of themselves as freely acting human beings, and they know nothing of authorial designs. Plots may be dominated by one character, namely the hero, but this does not mean that heroes are central controlling units, because there are lots of events that they cannot control, such as the machinations of the villain. Good plots emerge out of the conflict between different personal goals: if the goals of the characters were fully compatible, there would be no story worth Narrative as/and Complex System/s  33

telling. Narrative is therefore a top-­down, centrally controlled system on the level of authorial design, but it must give the impression of an emergent, bottom-­up system on the level of plot. A proper balance between these two types of control is essential to the aesthetic success of narrative. Readers must be aware that characters are authorial creations in the service of a global design, but they must also regard them as autonomous agents. When the actions of characters are too obviously dictated by the interest of the plot, this is perceived as a “cheap plot trick” (Ryan 2009), this is to say, as an authorial failure. Human interactions, the proper stuff of narrative, may be too complex to be generated by cellular automata (at least in their current state of development), but by insisting on a systemic interconnection of elements, complexity theory can inspire a network-­based approach to narrative (as developed in Poulaki 2014).3 One type of relation that lends itself particularly well to network analysis is who among the characters interacts with whom in direct acts of communication. The importance of a character for a plot is at least in part a function of what is called its “degree” in network theory, that is, the number of connections leading in or out of the node that represents this character. In general, the higher the degree, the more important the character. Diagramming interactions between characters makes it also possible to distinguish several types of plot. The network for an episodic or epic narrative (figure 1), such as the “journey of the hero” pattern or a bildungsroman, would show a heavily connected character who encounters many different characters, but connections between groups of characters will be very few or nonexistent: each episode tends to have its own cast.4 The network of the so-­called it-­narrative, or circulation narrative (a genre popular in the eighteenth century that retraces the travel of an object, such as a bank note, through multiple owners), would show several reasonably dense networks (corresponding to the various stories), connected to one another through the possession of a common object or through a single connection, corresponding to a character who serves as mediator (figure 2; note that to account for change of possession of the object within a story it would take a different kind of diagram). This structural type also describes narratives with multiple, distinct stories taking place in the same storyworld, such as Cloud Atlas, by David Mitchell. The networks of dramatic narratives (figure 3, for Jean Racine’s Phèdre) are more densely connected: charac34  Ryan

Fig. 1. Network of an episodic or epic narrative. Created by the author.

Fig. 2. Network of an it-­narrative. Created by the author.

Fig. 3. Network of Jean Racine’s Phèdre, a dramatic narrative. The number of arrows corresponds to the number of exchanges between the characters. The weight of the arrows suggests their dramatic importance. Numbers represent degrees of connectivity to other characters. Created by the author.

ters are strongly interrelated, except for some marginal confidante or messenger types, and the system of connections, rather than individual characters, is the heart of the plot. The degree of connectivity of a character node gives a rough idea of its importance for the plot, but this is not an absolutely reliable criterion, because this would make Thésée the main character of Phèdre. We also need to map the number of scenes in which characters interact with one another, as shown in figure 3. But a computer examining this network would again think that Thésée is more 36  Ryan

important than Phèdre, because the diagram makes no distinction between dramatically significant exchanges (such as the unique exchange between Phèdre and Hippolyte in which she confesses her love for him) and short, purely utilitarian exchanges where characters transmit news. While a computer could create the diagrams of figures 1 and 2, it takes human interpretation to create a diagram that shows the dramatic weight of exchanges, as I have tried to do in figure 3. This shows the limits of a computational approach to narrative complexity. Still, plot diagrams based on interpersonal relations are not useless, because they can show a certain type of complexity, which can be used as the basis of a plot typology. By the standard of connectivity, the complexity of a plot is not a function of the number of characters or of events, but a matter of interconnection. The more difficult it is to cut out parts of the network, the more they contribute to a plot’s complexity. For instance, it is easy to cut away the episodes of an it-­narrative by severing one link. By this standard, the dramatic narrative is the most complex, followed by the epic narrative and the it-­narrative. The network diagram may tell about the complexity of the system of human relations that underlies a narrative, but it does not tell anything about the nature of the interactions that constitute the plot. For such a model, I propose to turn to the metaphor of the landscape introduced by Scott E. Page, a specialist of complex systems. Page distinguishes three kinds of landscapes, which correspond to three kinds of systems of increasing complexity (2009, 6–­9). Imagine that with every type of landscape there is a problem to solve, the problem of finding the highest point in the landscape. Now keep in mind that virtually all narratives are about finding solutions to problems, if by the pair “problem/solution” one understands patterns such as the pursuit of desires, the fulfillment of obligations, the restauration of a broken equilibrium, or adaptation to changed circumstances. The first type is a simple system represented by the shape of Mount Fuji. In this kind of system, it is very easy to find the highest peak in the landscape. In a narrative shaped like Mount Fuji, there is a problem with an obvious solution; this solution is adopted, and the story is not very interesting. An example would be a fox who is hungry. He buys a wheel of cheese, he eats it, and he is no longer hungry. An efficient solution but a boring narrative, because the system lacks complexity. Narrative as/and Complex System/s  37

The second type of landscape is compared by Page (2009) to the profile of the Appalachian Mountains. It is a rugged landscape, made of many ridges, and it is much more difficult to find the highest peak than in a Mount Fuji landscape. In this kind of system there are conflicting goals. Page uses the example of an agent with conflicting personal goals (for instance, taking a job in a nice place versus taking a job that makes lots of money), so the optimal solution must involve compromise. But the idea of conflicting goals can also be applied to the case of many agents who have different personal goals. The solution of the main problem must take all these goals into consideration, whether it is through compromise, or—­more frequently in narrative—­through deceit or competition. In the case of deceit, some characters will be made to act against their own interests. Let’s return to the case of the hungry fox. Rather than buying cheese, he notices a crow in a tree who holds a piece of cheese in his beak. The crow is also hungry, and he certainly won’t part with the cheese voluntarily, so the two characters have incompatible goals. To get the cheese the fox must take the crow’s desire to hold onto the cheese into consideration. So, rather than asking the crow to share it, he resorts to deceit. The conflict between the fox’s and the crow’s goals, and the ingenuity of the fox’s solution, make the story a far more interesting narrative than the previous example, where the fox buys cheese. The story of the fox and the crow may rest on a conflict between the goals of two characters, but the landscape is relatively stable: the goals do not change drastically during the story. In the third kind of landscape, the configuration of the system is highly unstable. Page (2009) calls it a “dancing landscape,” and he compares it to a landscape that is being shaken by an earthquake, so its global state changes continuously and so does the highest peak. This kind of system contains multiple interdependent actors with various goals. Some characters pursue the same goals (so that they are in a helper relation); other characters have competing goals, and if one of them succeeds, another will fail. The global situation evolves throughout the narrative as the result of two types of events. The first type is external accidental events, such as an earthquake. Though these events are not under the control of characters, they have important consequences on their situation and interpersonal relations. The second type of events is intentional actions performed by the characters in the pursuit of their goals. As Page writes, “Interdependence between our actions and the actions of others are what makes a landscape 38  Ryan

dance” (8). In a dancing-­landscape narrative, characters must adapt their behavior to a constantly changing situation. Let me illustrate this idea with an analysis of Phèdre, a tragedy with a remarkably tight and logical plot, even if the actions of characters are anything but rational. The causal and temporal networks of Phèdre are shown on figure 4. Here is how the plot develops: Situation 1: Thésée is king of Trézène. He is absent, but he holds power. Phèdre, his wife, secretly loves Hippolyte, Thésée’s son by a previous marriage. This love is forbidden for two reasons: she is married, and he is her stepson. (Her love is considered incestuous by the standards of the play.) Hippolyte loves Aricie, and she loves him, but he cannot marry her because she comes from an enemy family. Phèdre’s adaptation to the situation: She decides to let herself die. But this decision is abandoned when an external event changes the whole situation: Thésée’s death is announced. Situation 2: An obstacle to Phèdre’s and Hippolyte’s respective loves is removed. Phèdre’s adaptation: She sees hope in her love for Hippolyte and confesses it to him, but he does not respond. This is a failed action. Then another external event changes the landscape: it turns out that Thésée is not dead, and he returns to Trézène. Situation 3: Phèdre feels dishonored by her confession and fears that Hippolyte will tell Thésée. Phèdre’s adaptation: She allows her nurse, Oenone, to tell Thésée that Hippolyte has tried to force himself on her. Situation 4: Thésée believes Oenone. Thésée’s adaptation: He curses Hippolyte, banishes him from the kingdom, and asks the god Neptune to punish him. Situation 5: Hippolyte learns of Thésée’s curse. Hippolyte adaptation to 5: He tries to show his innocence by telling Thésée of his love for Aricie, but, the gentleman that he is, he does not tell anything about Phèdre’s love confession. Thésée does not believe him: this is a failed action. Meanwhile, in situation 6, Phèdre feels guilty about the severity of Hippolyte’s punishment; she decides to save him and to confess Narrative as/and Complex System/s  39

everything to Thésée. But in the course of the exchange Thésée inadvertently reveals to her that Hippolyte loves Aricie. This accidental event leads to situation 7: Phèdre, who thought that Hippolyte was incapable of love, realizes that she has a rival, and she is furious. Phèdre’s adaptation to 7: She keeps silent about Oenone’s lie, possibly causing Hippolyte’s later death, for if he had learned the truth, Thésée could perhaps have done something to save him. Meanwhile, in situation 8, Aricie is upset about the curse on Hippolyte, and she pleads Hippolyte’s cause to Thésée. This action leads to situation 9: Thésée begins to have doubts about the crime of Hippolyte. He asks for the return of Hippolyte to question him. But then Théramène arrives and announces that Hippolyte has been killed by a sea monster as a result of Thésée’s curse. Théramène’s arrival is an external event, but Hippolyte’s death is not, because it is a result of Thésée’s curse and Phèdre’s silence. This revelation leads to situation 10. Phèdre feels guilty of Hippolyte’s death. She confesses everything to Thésée, takes poison, and dies. This action leads to situation 11: Thésée realizes Hippolyte’s innocence. To honor Hippolyte’s memory, he adopts Aricie as a daughter. For clarity’s sake I have numbered the situations sequentially, and this corresponds to their order of occurrence in the play. One could pull the situations and the events that follow them directly into a line; however, one cannot pull the system of causal relations into a line, because it would form knots. The linear sequence of situations and events can be explained only by a nonlinear network of causal relations. Let’s take a closer look at why there are discrepancies between the temporal sequence and the causal network. On the level of story, narrative can be described as a linear succession of events that cause changes in a storyworld. But a world can contain many agents who act more or less simultaneously, creating the causal chains of parallel plot lines. A state can be the result of the accidental convergence of different plot lines: for instance, John goes to a party and drinks too much. On his 40  Ryan

Fig. 4. Presentational sequence versus causal network for Phèdre. The presentational sequence is indicated by the numbering of events: 1a, 1b, 2a, 2b, 3a, 4a, and so on. The causal network consists of the causal relations between situations and the adaptive actions of characters. The content of situations is explained in the text. Created by the author.

drive home he hits Mary, who was walking her dog, and kills her. In this case the state “Mary is dead” is caused by the coincidence of two chains of events. Similarly, the death of Hippolyte in Phèdre is arguably the result of a double causality, Thésée’s curse and Phèdre’s silence about Oenone’s lie. While a situation can have multiple causes, it can affect multiple agents in different ways and have multiple effects. In Phèdre we see Hippolyte, Phèdre, and Aricie reacting separately (though pretty much in the same way) to Thésée’s curse. Quite often characters react to the same situation in opposite ways: what is good for the hero is bad for the villain, and while the hero basks in triumph the villain may be plotting a revenge. Finally, the effects of events may be delayed: for instance, Jason may be unable to marry Amanda because it is revealed that long ago his father had a secret affair, and Amanda is his half sister. In Phèdre Thésées’s curse has a delayed effect, since several events intervene before Hippolyte is killed (at an indeterminate time and not onstage) by a sea monster. My claim, then, is that discrepancies between the tempoNarrative as/and Complex System/s  41

ral sequence and the causal network are a major factor of narrative complexity. Narratives are almost always linear in their presentation, since events must be represented by most media one after the other, but their plots can be highly nonlinear on the level of causal relations. The greater the discrepancy between the temporal sequence and the causal network, the more complex the plot. Emergence Emergence, in its strongest form, is a property of phenomena that we do not fully understand: how the individual elements of a system organize themselves into larger functional patterns without the top-­down guidance of a controlling authority.5 For instance, to say that consciousness emerges from the activity of individual neurons is another way of saying that we cannot bridge the gap between neuronal activity and (self-­) conscious thought. What then does it mean to speak of narrative emergence, and can we say something meaningful about it? Emergence can appear on two narrative levels: the level of content and the level of form. On the level of content, the play Arcadia, by Tom Stoppard (1993), deals explicitly with the issue of complex systems and therefore, more or less directly, with emergent phenomena. Emergence is not only discussed but also manifested in the play through the unpredictable network of sexual attraction that links the characters. But it does not take explicit reference to complexity theory for a narrative to illustrate the phenomenon of emergence. Every tale of machines revolting against their maker and taking over the world—­from the Golem to Frankenstein’s monster to The Matrix—­is a tale of emergence, since the human-­programmed machine develops abilities and goals that its creators could not foresee. What would a genuinely emergent narrative—­as opposed to a narrative of emergence—­be like? We must remain in the domain of the fictional to find a working example: in Neal Stephenson’s The Diamond Age, a novel about a poor girl in a repressive society who finds a magical (or rather, digital) book, the Primer, created by a computer wizard to educate a rich girl. The Primer tells a dynamic story that “ages” with the reader, adapting itself to her needs and becoming more and more sophisticated in style and plot as she grows older so that a single book can educate her from childhood into adulthood. On the technological level 42  Ryan

the Primer combines the distributed intelligence of the ant colony with the solidity of a well-­connected network that will not be disabled when some links are broken. The smart paper of its pages is made of a billion microprocessors that aren’t “especially smart or fast” and are very susceptible to being disabled, but “even with those limitations the smart paper still constituted . . . a very powerful graphical computer” (1996, 64). The actual implementation of emergent narratives is infinitely more problematic than their fictional description. In my discussion of this issue I start with nonlinearity, a feature widely associated with emergence. In mathematics, nonlinearity is defined as the property of an equation that traces a curve rather than a straight line: any exponent higher than the number one will result in nonlinearity. Since one cannot produce narratives through equations (at least not yet), this definition is not useful to narratology. In complexity science, nonlinearity and its opposite, linearity, are defined in operational terms: “a linear system is one you can understand by understanding its parts individually and then putting them together” (Mitchell 2011, 22). For instance, the computation of the surface of the United States is a linear problem because it can be achieved by calculating the surface of every state and adding them together. A nonlinear system, by contrast, cannot be understood by looking at the properties of its basic elements, because it presents a hierarchical structure: individual elements group themselves into meaningful configurations on a higher level. For instance, we cannot describe the organization of the ant colony by describing the behavior of each ant, no more than we can pass directly from a certain configuration of excited neurons in the brain to the idea of a horse (Hofstadter 2007). According to this conception, language is nonlinear, since we don’t interpret the sentence “the fox stole the cheese from the crow” by creating a dictionary with the entries “the,” “fox,” “steal, past tense,” “cheese,” and “crow.” A proper interpretation would include a network showing a transfer of possession of the cheese from the crow to the fox as the result of a morally reprehensible action (but how does one show moral value on a network?). If language is nonlinear, narrative takes the property to a higher level, since it is not the sum of the meanings of its component sentences. Moreover, we don’t understand a story by storing in memory a list of all the characters, objects, and events in the storyworld. For nonlinearity to become a property of particular narratives, rather Narrative as/and Complex System/s  43

than a general feature, we must adopt a different definition, a definition that relies on the idea of sequentiality. Here, for instance, is how Espen Aarseth defines nonlinearity in his classic Cybertext: “A non-­linear text is an object of verbal communication that is not simply one fixed sequence of letters, words, and sentences but one in which the words or sequences of words may differ from reading to reading because of the shape, conventions, or mechanisms of the text” (1997, 41).6 According to Aarseth this property of nonlinearity appears in certain print texts that allow some freedom of choice (“Choose Your Own Adventure” stories, the I Ching, Vladimir Nabokov’s Pale Fire, Julio Cortázar’s Rayuela), but it is in digital texts (computer games, hypertext fiction) that it reaches its full potential because of the ability of the computer to let the user determine the sequence. In the early days of digital textuality, the advocates of hypertext fiction regarded linearity as a limiting structure from which narrative should be liberated. George Landow (1997) promoted hypertext as the implementation of a mode of writing that Roland Barthes called “the writerly.” Barthes conceived the writerly as a nonlinear “galaxy of signifiers” that could be entered in multiple ways and explored through multiple chains of associations; yet he regarded the plurality of the writerly as incompatible with narrativity: “for the plural text, there cannot be a narrative structure, a grammar or a logic; thus, if one or another of these are sometimes permitted to come forward, it is in proportion (giving this expression its full quantitative value) as we are dealing with incompletely plural texts, texts whose plural is more or less parsimonious” (1974, 5, 6; see discussion in Ryan 2016). It is easy to see why narrativity conflicts with nonlinearity: on the story level, narrative is a temporally ordered series of states, mediated by events that cause changes in the storyworld. Time, unlike space, is unidimensional and consequently linear. Causality, the glue that holds states and events together, adds unidirectionality to the sequence, since causes must precede their effects. How then can hypertext reconcile the inherent linearity of narrative meaning with a relative freedom of navigation through the text? Hypertext narrative can be conceptualized in two ways: spatial or temporal. In the spatial conception there is a storyworld in which a certain sequence of events happens. The user’s actions determine the order of presentation of the text, but they do not determine the temporal suc44  Ryan

cession of events in the storyworld. The text is like a “garden of forking paths” (title of a short story by Borges often invoked in hypertext theory) that allows a large number of itineraries, some long, some short, but it is always the same garden, containing the same story that the user explores more or less fully. The spatial model maintains narrative coherence, even when causes are discovered after their effect, because it allows the reader to mentally rearrange elements in a proper chronological sequence. But the freedom of navigation prevents effects that rely on a fixed temporal disclosure of information, such as suspense and surprise. In the temporal model, by contrast, the order in which readers discover elements corresponds to their chronological order in the storyworld. The reader who reaches A and then B reads a different story than the reader who reads B and then A, and these two readers “produce” different storyworlds. In contrast to the spatial model, the temporal model is supposed to yield many different stories, but when the underlying network is densely connected, the author cannot control the long-­term succession of elements, and there is no guarantee of logical coherence. For every traversal to result in a well-­formed story, the underlying network should be a tree, because in a tree structure, there are no circuits and consequently only one way to reach a certain node. The danger of encountering the effect before the cause is therefore eliminated. For this reason, the only working examples of the temporal structure are the tree-­based “Choose Your Own Adventure” children stories (Ryan 2001). But with a story tree the individual stories are predetermined by the author; they are limited in number and length (they stop when the reader reaches the end of a branch), and there is consequently no real emergence, though children who identify with the hero may imagine that the story develops in real time as the result of their decisions. In the spatial model, by contrast, the number of paths through the text is unlimited, and emergence takes place in the many ways of experiencing the narrative discourse. For a text to present emergence on the level of story, a sequence of events must be generated during the live performance of the text or during the run of the program that operates it, and different performances or runs should produce different stories. As Richard Walsh observes, the best available model for emergent narrative is dramatic improvisation, as found in commedia dell’arte: “The story produced by a Narrative as/and Complex System/s  45

group of improvising actors is not determined top-­down, by a playwright or director, nor is it the creation of one actor. Instead, it emerges from the interaction among the members of the group—­that is, the elements of the system” (2011, 76). However, the spectator is external to the performance, and, while knowing that the actors are improvising enhances aesthetic appreciation, it does not lead to greater immersion or participation. For emergence to reach its fullest manifestation, the user must be part of the system, so as to have a say in the development of the story. In other words, the system must be interactive. Whereas in hypertext the algorithm does no more than guide the user through a database of prewritten textual fragments, in more “intelligent” types of digital narratives, such as video games, interactive drama, or automated storytelling program, the computer generates stories by performing a simulation of the storyworld. By simulation I mean that the computer keeps track of the state of the storyworld, allowing only those actions that are relevant to the current state and updating the world’s state on the basis of the user’s choices (or on the basis of a random choice among several possible alternatives). The system “knows” to some extent what is going on, in contrast to hypertext, where sense making is entirely left to the user. In a simulation different choices at the decision points will result in different sequences of states and events, this is to say, in different stories.7 Yet the narrativity of the output of a simulator is far from being universally recognized by those who have given thoughts to the issue. To understand why, we must go back to the definition of simulation proposed by Gonzalo Frasca, one of the pioneers of computer-­game studies. According to Frasca, “to simulate is to model a (source) system through a different system which maintains to somebody some of the behaviors of the original system” (2003, 223). While in the conception previously outlined, simulation can create imaginary worlds and processes, Frasca’s definition presents simulations as models of existent systems, such as flying an airplane.8 There is consequently no way a fictional story can be regarded as the product of a simulation engine. Another of Frasca’s reasons for regarding simulation as incompatible with narrative is explained by Walsh, who agrees with Frasca on this point: “Simulation and narration, as modes of representation, are different in kind: a simulation represents a system, globally, while a narrative represents a discrete tempo46  Ryan

Fig. 5. Modes of realization of narrativity. Created by the author.

ral sequence” (2011, 78). A flight simulator can indeed represent all the flights that can happen with a certain airplane, while a standard narration (print novel, comic, and movie) can represent only one particular flight. But rather than driving a wedge between simulation and narration, why not regard narration and simulation as two ways to communicate a certain type of semantic material that fulfills the basic conditions of narrativity? There are at least three ways to represent narrative content: enactment, as in drama and film; narration, as in novels; and simulation, as in computer games and story generators (figure 5). While narration produces only one story, and the multiple live enactments of the theater (as opposed to the single enactment of film) produce only small variations on a common script, simulation generates many stories, thereby fulfilling the basic condition of emergence. It could, however, be argued that the various runs of a computer game do not “tell” any story since they have no narrator nor narration.9 But neither do film and drama, except in the case of voiced-­over narration. This is why for a long time these media have been excluded from narratology. To solve the problem of the narrativity of simulations and restore the possibility of narrative emergence in digital media, I propose to invoke the distinction I made in Ryan (2004) between “being a narrative” and “having narrativity.” Even when they do not explicitly “tell” stories, simulative engines can produce outputs that activate in the playNarrative as/and Complex System/s  47

er’s mind the cognitive template that defines narrativity, just as certain life situations activate this template in the mind of the observer or participant. Whether it happens in the real world or in a fictional world, a situation contemplated by a mind can have narrativity without being a narrative, this is to say, without being encoded in a discourse that explicitly narrates past events. This does not mean that the narratives of simulation-­driven games are necessarily emergent. On the contrary, most computer games implement a predefined story, and variation occurs only on the microlevel, the level of how the player passes from fixed point A to fixed point B. The actions performed by players to solve individual problems can be said to possess narrativity, because it is by telling a story about them that players brag to other players about their exploits in the game world. They also possess some degree of emergence, because every player will perform a slightly different set of actions. But these moments of narrative emergence are framed by invariable events implemented by the code. Playing the game thus means progressing along a fixed plot line. Genuine narrative emergence, by contrast, requires the real-­time generation of a sequence of events that the user or interactor will interpret narratively, and these events must not be foreseen by the designer. Perhaps the most successful step in this direction is the computer game The Sims (2000–­14). The name of the game indicates the importance of simulation in the algorithm that runs it: the game constructs a world; fills it with objects and characters; specifies their properties and affordances; gives aspirations, fears and goals to the characters; and proposes to the player menus of actions that characters can perform in a given situation. After an action has been performed, the system calculates its effects, not only on the physical world, but also on the inner world of the characters and on their personal relations. For instance, if Lisa tries to kiss Tina, this will affect Tina’s feelings toward Lisa either positively or negatively, depending on how her personality has been set up by the code. Following the advice of Sandy Louchard and colleagues (2015, 186) to let emergent narratives be driven by characters rather than by plot (for plots are preset story arcs: how could they be emergent?), the game’s basic idea consists of throwing a number of characters (a group of friends or a family) with strong but distinct personalities in a common arena (a house or neighborhood) and letting them interact with one another 48  Ryan

so that a narratable sequence of events will emerge from their actions, fueled by their individual aspirations. Players can either select actions randomly, to see what the system will do, or try to implement certain narrative patterns, such as Greg getting rich, buying a fancy house, and successfully courting Jen. (The getting-­rich narrative is the game’s implicit script, but it can be subverted.) Because the range of possible actions in a given situation is always determined by the code, players are never in total control of the narrative development, but neither is the system. When the code throws in events, such as death claiming a character, it plays the role of blind fate, rather than trying to implement a script. Through its combination of goal-­oriented action and random happenings, The Sims is a credible simulation of life. On the negative side the game has been blamed for giving far too much importance to the repetitive chores of daily life (this is not drama, this is housekeeping, complained Chris Crawford [2004]) and for being unable to produce narrative closure: The Sims is a system for never-­ending soap operas, not for stories with a dramatic arc. Let me conclude this section by asking whether or not interactivity is a necessary feature of emergent narrative. If interactivity means dynamic relations between the elements of the system, which means between the characters, then the answer is definitely yes, but this is a feature of all narratives. On the other hand, if interactivity means the user’s ability to control the system, either by manipulating characters from a godlike perspective (as in The Sims) or by impersonating one of them, then the answer is a cautious no, because there are, or could be, simulation-­ based story-­generating programs that churn out different stories without external intervention, such as James Meehan’s Tale-­Spin (1981), Scott Turner’s minstrel (1984), or Selmer Bringsjord and David Ferrucci’s Brutus (1999). Yet the number of different stories that these systems generate is very limited (often no more than half a dozen, though huge numbers of lines of code are needed to produce them), and none of them is worth reading for pleasure. Artificial intelligence has made tremendous advances in the past few years in many domains, but automated story generation is not one of them. While noninteractive but emergent narrative systems are at least conceivable, user interactivity is a strong factor of emergence, because it injects systems with variable information and therefore guarantees that Narrative as/and Complex System/s  49

different runs of the program will produce different outputs. But the combination of interactivity and narrativity is the greatest problem that faces designers of digital art and entertainment, because narrative, as a global pattern, requires top-­down control, while interactivity provides bottom-­up input. Computer scientists Ruth Aylett and Sandy Louchard call this situation the interactive paradox: “On the one hand the author seeks control over the direction of a narrative in order to give it a satisfactory structure. On the other hand a participating user demands the autonomy to act and react without explicit authorial constraint” (2004, 25). If the task of generating reasonably interesting noninteractive stories is already beyond the ability of current ai, the difficulty of the task is elevated to a higher power if the stories have to incorporate unpredictable user input. It is only by limiting the range of choices of the user (for instance, by proposing a menu of possible actions) that a compromise can be achieved between interactivity and narrativity, but this limitation presupposes a predetermined global narrative script. As the user’s freedom is restricted, so is the system’s degree of narrative emergence. Since 2000 intensive work has been devoted to the creation of interactive narrative, but the formula that would ensure a successful compromise between the top-­down design of narrative and the bottom-­up input of the user still eludes designers.10 To this day, with the possible exception of The Sims, there is no commercially viable system that produces truly emergent interactive narratives. But who says that emergent narrative has to be computer assisted? Artificial intelligence has yet to approach human intelligence in narrative creativity, so why not rely on natural intelligence to adapt character behavior to unpredictable, evolving situations? In tabletop role-­playing games (Caïra and Tosca 2014), a naturally intelligent game master coordinates the behavior of a group of players who impersonate the characters of a partly predefined, partly improvised narrative script described in a rule book. Here is how tabletop games work: Using storytelling skills and relying on the rule book, the game master describes the current situation of the game world and the actions that can be taken in this particular situation. The players choose an action and roll a dice to determine its outcome. The game master narrates the event, and a new situation is created, allowing new actions. Creative game masters can make new rules on the fly when the players take the story in an unforeseen direc50  Ryan

tion; they can overrule the book when they can imagine more exciting developments; and they encourage the players to act out their roles by improvising dialogue. Tabletop role-­playing games are commedia dell’arte without spectators, since the beneficiaries of the performance are integrated into the storyworld. But a more emergent narrative does not necessarily mean a better, more satisfying narrative. Many people prefer to watch and read a totally prescripted story than to participate actively in an emergent narrative. Conclusion Let me return to the question of whether complexity is a universal feature of narratives or a distinctive property, that is, a property that creates distinctions within the set of all stories. In systems theory complexity is considered distinctive rather than universal: there are simple systems whose behavior can be predicted by a fixed formula, and complex systems whose behavior can be described only by applying their rules over and over again, just as there are stable landscapes that look like Mount Fuji, and dancing landscapes whose highest points continually change. In narrative matters the answer depends on how one conceives complexity. Here I have given two particular interpretations inspired by complexity science, namely, complexity as lack of a central controlling unit and complexity as emergence, but even these two ideas can be interpreted in many different ways. Emergence, for instance, can be conceived as the dynamic unfolding of narrative meaning through time (e.g. Walsh 2011), which makes it a universal property, since there cannot be narratives without a world undergoing changes, and there cannot be changes without a temporal frame; but this property is shared by all temporal arts and therefore does not say anything specific about narrativity. Emergence could also be conceived as unpredictability, which means the ability of a plot to create surprise, but this would not make it a distinctive property, since there are narratives whose ending is entirely predictable. In this chapter I have interpreted emergence as the ability of some systems to produce multiple stories. This property is scalar and distinctive, at least when story is taken literally and in a strong sense, since most narrative texts are not generative (Aarseth 1997 would say ergodic) systems with multiple outputs.11 The lack of a central controlling unit on the level of story, by contrast, can be regarded as typical of all narratives, even of Narrative as/and Complex System/s  51

narratives with a clearly central character, since no character absolutely controls the world, not even God, for God has to put up with the forces of evil. (Without such forces there would be no narrative.) The lack of a central controlling unit can lead to plots of variable complexity, depending on the number of characters, the compatibility of their desires, their ability to adapt to changing circumstances, and, above all, whether the causal system follows the temporal sequence or creates knots that tie together nonadjacent situations. Notes 1. John Horgan (1995) compares complexity theory to other, mostly short-­ lived twentieth-­century scientific movements such as cybernetics (Norbert Wiener), catastrophe theory (René Thom), information theory (Claude Shannon), and the predecessor of complexity, chaos theory. One may also place in this category Stephen Wolfram’s New Kind of Science, based on cellular automata. According to Horgan, most of these movements were the brainchild of one individual who envisioned the expansion of an idea into a “theory of everything.” For a rejoinder, see Mitchell (2011, 292–­303). 2. By regarding plot as a decentralized system, I challenge the claim made by H. Porter Abbott in “Narrative and Emergent Behavior.” Abbott argues that evolution, the prototype of emergent processes, cannot be narrated, because narrative is concerned with a central controlling instance, while evolution results from many “blind” microprocesses. He regards narratives of central control (such as explaining evolution through intelligent design or history through theories of conspiracy) as reassuring because they attribute an identifiable, unique cause to phenomena and “allow the perceiver to achieve a sense of cognitive control” (2008, 239). Abbott admits, however, that central control is not a necessary feature of narrative: “Canonical narratives like Hamlet generally resist the reduction to some form of central control, and long novels like Eliot’s Middlemarch, Mann’s Buddenbrooks, and Proust’s A la Recherche du Temps Perdu are monuments of such narrative complexity. But even soap operas can be mightily complex as well” (232). In my view this claim does not go far enough: even simple stories, such as “Little Red Riding Hood,” do not normally explain events through a central controlling instance, as do so-­called grand narratives. In most stories causality, from which narrative derives its explanatory power, works only between individual events. It would take a grand narrative to try to explain why the whole chain of events happened at all. Still, I agree with 52  Ryan

Abbott that evolution does not lend itself to narrativization, because of the lack of deliberate agents. Pure randomness does not make good stories. 3. Maria Poulaki (2014) (following Bordwell 2006) discusses one type of story as “network narrative”: a story exhibiting a “small world” pattern (a favorite concept of complexity science), where a large number of characters belonging to different groups are suddenly connected through random, external events. Her example is the 2008 film Burn after Reading. Here, however, I argue that all narratives are built on a network structure, though in the simplest cases this network may be linear. (A series of nodes connected to one another by a single edge is still a network.) 4. What about characters who appear in several episodes? Since the diagram shows character-­to-­character relations, but not the number of interactions, once characters are linked to the hero, they may appear many times in the plot. 5. This mysterious nature of emergence could explain why H. Porter Abbott (2004, 2008) regards emergent phenomena as unnarratable, since narration, to a large part, is explanation. 6. Actually, there is no need to restrict the definition to verbal communication. 7. The term simulation is also widely used to designate the mental modeling of storyworlds that takes place in the minds of readers, spectators, and players as they go through a narrative text. This kind of simulation is a fundamental condition of narrative comprehension. 8. By this criterion computer games can be regarded as simulations only on the microlevel of individual actions, such as shooting, driving cars, and collecting objects. 9. An exception is provided by those games in which a narrator, or announcer character, describes the events that happen in the game as the result of the player’s actions. This feature is found in sports games and also in some massively multiplayer online role-­playing games such as EverQuest. Story-­ generating programs such as Tale-­Spin (Meehan 1981) also combine simulation with narration by narrating what they produce through simulation. These examples provide an argument against Frasca’s (2003) claim that simulation and narration are mutually incompatible modes of representation. 10. This work is pursued by organizations such as icids (International Conference for Interactive Digital Narrative), cmn (Computational Models of Narrative), or a branch of aaai (Association for the Advancement of Artificial Intelligence) devoted to “narrative intelligence.” 11. In a weak sense one could say that every narrative text generates multiple readings, hence multiple stories. Narrative as/and Complex System/s  53

References Aarseth, Espen. 1997. Cybertext: Perspectives on Ergodic Literature. Baltimore: Johns Hopkins University Press. Abbott, H. Porter. 2004. “Unnarratable Knowledge: The Difficulty of Understanding Evolution by Natural Selection.” In Narrative Theory and the Cognitive Sciences, edited by David Herman, 143–­62. Stanford ca: Center for the Sudy of Language and Information. —. 2008. “Narrative and Emergent Behavior.” Poetics Today 29 (2): 227–­44. Aylett, Ruth, and Sandy Louchard. 2004. “The Emergent Narrative: Theoretical Investigation.” Proceedings of the Narrative and Learning Environments Conference, 25–­33. nile04. Edinburgh, Scotland. www​.nicve​.salford​.ac​.uk​ /sandy​/ENFramesetPage​.htm. Barthes, Roland. 1974. S/Z. Translated by Richard Miller. New York: Farrar, Strauss and Giroux. Bordwell, David. 2006. The Way Hollywood Tells It: Story and Style in Modern Movies. Berkeley: University of California Press. Bringsjord, Selmer, and David Ferrucci. 1999. Artificial Intelligence and Literary Creativity: Inside the Mind of Brutus, a Storytelling Machine. Hillsdale nj: Erlbaum. Caïra, Olivier, and Susana Pajares Tosca. 2014. “Role-­Playing Games.” In The Johns Hopkins Guide to Digital Media, edited by Marie-­Laure Ryan, Lori Emerson, and Benjamin J. Robertson, 433–­37. Baltimore: Johns Hopkins University Press. “Conway’s Game of Life.” 2018. Wikipedia, the Free Encyclopedia. Last modified September 21, 2018. https://​en​.wikipedia​.org​/wiki​/Conway​%27s​_Game​_of​_Life. Crawford, Chris. 2004. Chris Crawford on Interactive Storytelling. Berkeley: New Riders. Frasca, Gonzalo. 2003. “Simulation versus Narrative: Introduction to Ludology.” In The Video Game Reader, edited by Mark J. P. Wolf and Bernard Perron, 221–­35. New York: Routledge. Hayles, Katherine N. 2005. My Mother Was a Computer: Digital Subjects and Literary Texts. Chicago: University of Chicago Press. Hofstadter, Douglas. 2007. I am a Strange Loop. New York: Basic Books. Holland, John H. 2014. Complexity: A Very Short Introduction. Oxford: Oxford University Press. Horgan, John. 1995. “From Complexity to Perplexity.” Scientific American 272 (6): 104–­9. Johnson, Steven. 2001. Emergence: The Connected Lives of Ants, Brains, Cities, and Software. New York: Scribner. 54  Ryan

Landow, George P. 1997. Hypertext 2.0: The Convergence of Contemporary Critical Theory and Technology. Baltimore: Johns Hopkins University Press. Louchard, Sandy, John Truedale, Neil Sutie, and Ruth Aylett. 2015. “Emergent Narrative: Past, Present and Future of an Interactive Storytelling Approach.” In Interactive Digital Narrative, edited by Hartmut Koenitz, Gabriele Ferri, Mads Haahr, Diğdem Sezen, and Tonguç Sezen, 185–­99. New York: Routledge. Meehan, James. 1981. “Tale-­Spin.” In Inside Computer Understanding, edited by Roger Schank, 197–­225. Hillsdale nj: Erlbaum. Mitchell, Melanie. 2011. Complexity: A Guided Tour. Oxford: Oxford University Press. Page, Scott E. 2009. Understanding Complexity: Course Guidebook. Chantilly va: Teaching Company. Poulaki, Maria. 2014. “Network Films and Complex Causality.” Screen 55 (3): 379–­95. Ryan, Marie-­Laure. 2001. Narrative as Virtual Reality: Immersion and Interactivity in Literature and Electronic Media. Baltimore: Johns Hopkins University Press. —. 2004. Introduction to Narrative across Media: The Languages of Storytelling, edited by Marie-­Laure Ryan, 1–­40. Lincoln: University of Nebraska Press. —. 2009. “Cheap Plot Tricks, Plot Holes, and Narrative Design.” Narrative 17 (1): 56–­75. —. 2016. “Sequence, Linearity, Spatiality, or Why Be Afraid of Fixed Narrative Order.” In Narrative Sequence in Contemporary Narratologies, edited by Raphaël Baroni and Françoise Revaz, 176–­93. Columbus: Ohio State University Press. The Sims. 2000–­14. Videogame. Designed by Will Wright. Redwood Shores ca: Maxis. Stephenson, Neal. 1996. The Diamond Age, or A Young Lady’s Illustrated Primer. New York: Bantam. Stoppard, Tom. 1993. Arcadia. London: Faber and Faber. Walsh, Richard. 2011. “Emergent Narrative in Interactive Media.” Narrative 19 (1): 72–­85.

Narrative as/and Complex System/s  55

2

Caution, Simulation Ahead Complexity and Digital Narrativity David Ciccoricco and David Large

We begin with a proposition: complexity is a virtue of literary texts and arguably a vital element of any definition of the same. We follow with a proverb: still waters run deep. Thus, if scholars opt to seek out the deepest waters of literature they may do worse than exploring those that appear at first glance to be the most placid. The black mirror of the capacitive touchscreen would appear well placed to expand these proverbial waters. Both complexity, in the form of computing power and potential, and simplicity, in the form of well-­w rought, intuitive, and even “natural” interface design, are virtues of a touchscreen and smartphone era characterized by precisely milled, if featureless, slabs of glass and aluminum. Pry, an ios application released in 2014 by media artists Samantha Gorman and Danny Cannizzaro under the art collective Tender Claws, is billed as “an app hybrid of cinema, game and novel,” though the address of its promotional website, http://​prynovella​.com, in referring to a “novella,” hints at more concision. The app uses elements of all three media—­ video clips, achievement-­based interaction, and layered textuality—­to tell its story through the failing eyes of James, a Gulf War veteran. We are told to open James’s eyes by using two fingers in an outward spreading motion and to retreat inward to explore his memories by pinching the screen, which yields his “infinitely scrolling” thoughts for us to read (though infinitely looping would be more accurate). The app indulges the concept of seamless movement among and between words and images to explore the layers of a character’s consciousness (Gorman and Cannizzaro 2014). Through the reader’s deft (or fumbling) interactions, James’s past 56

intrudes on his present, and our handling of the text becomes something more akin to assisted entropy than piecing together a narrative puzzle. Narrative complexity, in the sense afforded by digital fiction, requires the reader to apprehend both discourse and medium in the same breath: its “verbal, discursive and/or conceptual complexity” is bound to its medium (Bell et al. 2010). One might productively consider digital fiction’s systematic treatment of tekhnē and logos as being in persistent pursuit of complexity through technological dependency. Put simply, technology interfaces the reader’s apprehension of complex systems of thought by way of simple—­often embodied—­metaphor. Furthermore, like any work of fiction, digital fiction engenders complex imaginative (mental) simulations as the reader cognitively calls forth its storyworld(s). Indeed, mental simulation is at the core of contemporary cognitive science’s debates on theory of mind, specifically questions regarding the extent to which we use our own mental machinery to simulate the mental states of others to ascribe those states reliably.1 At the same time, with their operational potential to act as rule-­based systems that model behavior in some way, works of digital fiction can also be complex simulations in and of themselves. More specifically, they can be examples of what we might call artifactual simulations, which carry the capacity to model anything from atmospheric or physical systems to the social systems of storyworlds.2 In fact, works of digital art and literature and no doubt contemporary video games remind us of the many ways simulations can be harnessed for creative and artistic ends, suggesting the subcategory of aesthetic simulations (Ciccoricco 2015, 23). If the complex intersubjective transaction between author, text, and reader marked the site of critical attention for much of literary theory’s own lifetime, then perhaps it is the transactive site between mental and aesthetic simulation to which we should direct our critical attention now. That same nexus of mind and machine, of course, marks contestable territory; it is the source of intense cross-­disciplinary tension, as reductive scientism seeks to quantify the very literary complexities that literary theory seeks to preserve and celebrate. For the same reason that site of transaction is instructive for a study in narrative complexity. After all, even though simulations are computationally encoded, they are representative (selective) rather than exhaustive (totalizing), which is also to say they are no less culturally encoded than prose. Simulations, by defiCaution, Simulation Ahead  57

nition, must be less complex than their target or source system, and, given the number of nebulous unknowns when dealing with social systems, the degree of complexity of that target is arguably much higher than, say, a climate simulation, even if one is simulating a moment of social interaction compared to a hundred years of climate change. Digital fiction is thus unique in that it must balance familiarity (pattern recognition and representation) and novelty (defamiliarization by means of its distinct interface) just as it, and its interface, must balance simplicity and complexity in its aesthetic treatment of narrative. With such a balance in mind, and factoring in the fluidity of the digital medium, one must question whether a complex interface necessarily implies or necessitates simple discourse, and the converse, whether complex narrative discourse requires a simple interface. With regard to Pry in particular, we first consider the fiction’s complex and literary narrative discourse in concert with its complex and affective interface. We then locate a tension between the intuitive ease of Pry’s touchscreen input and the difficulty of its achievement-­based interaction, which complicates the more immediate intuitive affordance of the interface. We use empirical examples from students using Pry in a class workshop to convey how the ludic quality of the text complicates their reading experience in significant ways. Finally, we consider the metaphorical apparatus of Pry and the multifaceted embodied gestures it both portrays through its storyworld and demands in its readers. In turn, we caution against the illusive complexity of totalizing simulations and argue for a form of conceptual complexity that extols the literary virtues of delimited metaphor in digital fiction—­in both tactile and cognitive terms. Complexity in Concert In the Psychomachia of Prudentius, readers of late antiquity find the first iteration of the now often repeated notion that Patience is a virtue. Set in graceful opposition to Wrath, Patience is assailed with words and weapons but remains unharmed; she is, Prudentius writes, the only virtue who willingly allies herself with others and lends her assistance: “No Virtue enters on the hazard of the struggle without this Virtue’s aid, for she has nought to lean upon, whose strength [Patience] does not uphold” (Thomson 1949, 291). In proposing complexity as a literary virtue, we must also recognize the value of patience. Who else but the pa58  Ciccoricco and Large

tient reader—­the habitual (re)reader—­might appreciate, for instance, the imbrications and intricacies of Virginia Woolf’s or James Joyce’s narratives? Well-­wrought narrative complexity is arguably the end goal of high modernism. Unlocking the enduring treasures of the likes of T. S. Eliot and Ezra Pound in turn became the end goal of New Criticism, with its fixation on the literary work as both autonomous and necessarily difficult. Eliot, in “The Metaphysical Poets” (1921), claims that the poet must become increasingly comprehensive, allusive, and indirect to dislocate language into its meaning. Such indirection is discursive, however; even though the New Critical focus falls more firmly on the poetic, the printed materiality of the text or the signifier under examination escapes inquiry (Ciccoricco 2012). Literary theory thus carries its long-­standing if unwitting bias toward representational complexities into digital environments, often at the expense of the kinds of operational complexities that arise from computational texts. Early theorists of digital textuality have emphasized the heightened complexity of the digital signifier compared to its print counterpart, especially in the way that it is a representation of an occluded code: “The signifier in a digital environment has a complex internal structure quite unlike the flat mark of print. Moreover, this complexity requires for its implementation constant encoding and decoding” (Hayles 2000). On a more holistic level theorists have identified the need to refine worn-­out conceptions of form and content by factoring in considerations of medium among more familiar distinctions of formal and discursive interplay. The return is a newfound critical awareness of the way in which materiality informs signification (Hayles 2002; Kirschenbaum 2008). As the early history of the field also suggests, it may have been tempting to see the complex linguistic or semiotic instability served up by poststructuralism reflected in or even fulfilled by the complex material instability of digital textuality. We are now careful, however, not to conflate the two or to make the (technological determinist) error of seeing discursive and formal complexity as merely an expression of longing for a new medium (Ciccoricco 2007). Nonetheless, other questions put forth over a decade ago might be more relevant now than ever. Some wondered, for instance, how far digital artists could push the complexity of narrative discourse in light of increasingly sophisticated interfaces: “Is it possible to create a body-­centric interactive piece with the storytelling capacity Caution, Simulation Ahead  59

of an epic novel or a play?” (Matt Gorbet, cited in Stefans 2005). Others questioned whether or not fiction “can take on such multi-­dimensionality and performativity and retain its cognitive richness,” locating a tension between the “performative complexity” of the interface and the “verbal and conceptual complexity” of literary texts (Tabbi 2003). The adjective complex, connoting both undesirable difficultly and desirable intricacy, already straddles a positive and negative reception. And it is clear that in literary matters, complexity, however well wrought, is far from a populist pleasure. Nevertheless, a digital and computational fiction moves conceptions of complexity beyond hermeneutic difficulty to account for the demands it makes perceptually, in terms of how we see it, and operationally, in terms of how we use it. Pry begins with a concerted avoidance of any reader interaction beyond touching the screen to play a cinematic prologue. As we watch passively, we see a suburban house, its comfortably domestic interior, a young man, James, in turn examining and packing his largely utilitarian belongings: clothes, toiletries, and personal effects, all neatly arranged in an early signal of a life accustomed to—­or anticipating—­military precision. The only early indicators of James’s preoccupation with memory or our awareness of his interior world are a Polaroid camera, a small notebook and pencil, and an intercut sequence of childhood videos in and around what appears to be the same house (and, perhaps, his deliberate pause to cut a lock of his dog’s hair before his final departure). In successively quicker cuts, we catch glimpses of military aircraft, army personnel on a windy desert plain, a woman smiling, a series of physical tussles, and a campfire, all intercut with flashes of the woman’s green eyes. Chapter 1 (“Below and Above”) opens with the intertitle “Six Years Later,” which fades into the tactile memory of a Polaroid photograph depicting the green-­eyed woman, superimposed on the image of an eye. We descend from the ceiling to James lying in bed; the camera eye closes in on his face, zooming in on his right eye and dilated pupil as the eyelids close around us, the visual metaphor suggesting we are engulfed by his perspective. We discover, as his thoughts flicker onscreen in the mode of rapid serial visual presentation, that he is “awake but not fully.” But so too are we. We have to be told to “spread and hold open to see through James’ eyes”; upon completion of this task we—­and he—­are reminded of his stasis (“Right, can’t move”). Or, we are told to “pinch and hold closed to enter 60  Ciccoricco and Large

James’ subconscious.”3 Therefore, the only act we complete that physically mirrors prying is opening James’s eyes. When we enter the protagonist’s thoughtscape, we are in fact pinching our fingers together, despite the fact that this is the moment we might be considered to be “prying.” The instructions set up a curious inversion from the outset, positioning the act of prying somewhere between a physical action (can one literally pry images on a screen?) and a metaphorical one, requiring that we forcefully close one’s eyes to pry open their consciousness. After all, we can literally pry into one’s personal business without any physical force, as the verb denotes, but literally prying into one’s consciousness or memory—­ be it a stream or a storage facility—­is a vexed prospect. The second chapter, “South Bay Implosion,” reinforces the tripartite division of the narrative into an inward, outward, and a kind of default mode. That is, without interaction, the display shows a largely black screen, with a reddish pulsing on its edges, suggesting that James’s conscious default amounts to considering the world with his eyes closed. Externally, James attempts to oversee a controlled civilian demolition, but, prying inward, we see his thoughts flicker almost too rapidly to read: “regulations / devils in the details / Hell is paved / with good exceptions.” As David Jhave Johnston suggests, the technical virtuosity of Pry’s interactive gestures is so adroit in its treatment of what he calls “plot psychology” that it overshadows or even erodes the reader’s simple engagement with the narrative: “The gestures are so elegant that experiencing them generates a pleasure that eclipses the story. The effect is often proximal to wonder: it’s playful, giddy, strange. A disjunct arises between the heavy themes of eroticism, violence, and loss expressed in the content, and the joy felt pinching mediated text into innovative transitions. Ironically, the writing must swim against the tide of joy generated by the interface’s success; it is difficult to allow entry into darkness if the door is a playground” (2014). An immersive and affective experience, in Johnston’s view, may be occluded by the interface despite its own (net positive) contribution to affects. Our experiences were similarly poised on a kind of frustrated wonder—­holding open James’s eyes (see figure 6), for example, becomes something of a physical chore, though it is by no means arduous. Holding one’s forefinger and thumb apart, watching for development or denouement with interphalangeal joints firmly locked as we maintain contact Caution, Simulation Ahead  61

Fig. 6. Prying open the text and holding our fingers splayed, we see James’s memories.

with an oleophobic screen: these are not the means by which we might typically consume narrative. Yet patience is a virtue, and we find ourselves following James’s lead to “go back. To recapture what you lost, to understand when you lost it” (Gorman and Cannizzaro 2014, chap. 5). Empirical Complexities It may be a given that narrative complexity is inversely proportional to the kind of familiarity we gain through literary theoretical practice, that is, the less familiar—­at least in terms of cognitive schemata we bring to literary texts—­the more complex. The same relationship is, however, troubled by digital fiction. In 2016 at the University of Otago, we taught Pry in an English course on digital media and literary practice called Digital Literature: Technologies of Storytelling. It is a senior-­level, third-­year course, with the majority of students in their early twenties (and with a fairly even gender split). Pry was an assigned text, with loaner iPads provided to students, who were encouraged to read and interact with the application in their own time, as well as in class workshop sessions with their peers. Our initial assumptions about teaching a text with a touchscreen interface—­its potential complexity for the unfamiliar user primary among them—­proved unfounded. Indeed, the interface was by far the least challenging part of interacting with the text. The majority of our 62  Ciccoricco and Large

students are regular touchscreen users, accustomed to an interface that mirrors scaled-­down abstractions of what are now rather routine physical movements—­the swipe, the pinch, and the tap. Students quickly embraced Pry and the corporeal familiarity of its controls. The comparative ease of its interface meant that many students felt more readily immersed in the world of the text, diving into James’s mind with little hesitation or consideration of subtext. Such immersion altered their perceptions of the text: for many of them the immersion afforded by a familiar control scheme in turn afforded an appreciation of the text’s narrative payload, as opposed to a celebration of the technology itself. Reflecting on Pry, some anticipated the possibility for authors of touchscreen texts to add “new meanings” to familiar movements: one might “drag backwards to go back in time, or back into someone’s memory” or, as Pry demonstrated, “zoom into people to look deeper into [their] thoughts and their side of the story” (Margie Anderson).4 Touchscreen technology might, the same student suggested, allow readers to “approach a story from many different angles and perspectives, reading in a way that allows you to go back and look closer at the things you want more information on.” (Such was the promise of the home-­release dvd, its storage capacity and asynchronous viewing options endeavoring to provide the viewer—­with complicated remote control in hand—­a lternate endings, scenes shot from different angles, or even branching narratives. Hollywood studio economics and the consumer’s general preference for a home-­viewing experience indistinguishable from the cinematic largely brought an end to that teasing possibility, though the promise for narrative choice still remains.) A common sticking point for other students was that they felt themselves to be too conscious of their participation in the text (Hal Fish). By nature of Pry’s chapter-­based structure and fractured narrative discourse, information is drip fed, offering small glimpses of context that the reader must piece together and reread “to gain,” as one student put it, “some semblance of story” (Erin Lee). Was story the prize of interactivity, an achievement to be gained after repetitive work? Or was interactivity the price of the story? Pry’s achievement system further complicated the students’ experience. One student reported that on his first read-­through of Pry he had taken on the role of a film director, choosing cohesive narrative elements for an imagined audience. But he added that he felt pressured by Pry’s linCaution, Simulation Ahead  63

ear chapter system not allowing him to undo or redo his decisions without restarting the chapter. He wrote that this desire to make the “best” or most cohesive choice reflected on his preconceptions about the text; he reacted to it as though it were a video game where one choice was “correct” or better than the other rather than a narrative that embraces its multilinearity (Peter Croft). Another student decided that Pry risked becoming a “chore” for the reader, that there was too much content to find and to read, much less comprehend (Hal Fish). Made too conscious of their participation simply by virtue of the presence of the road not taken, readers might view their encounter with the text as a failed effort. This same student, nonetheless, concluded that the drive to a “total comprehension” of a complex text such as Pry could “in the long run” reward readers with an “all-­encompassing” sense of being engaged by or immersed in it. (Here, albeit at the risk of preempting their experience, we occasionally prompted students to consider possibilities other than total comprehension.) Ultimately, the practice of rereading to combat complexity or incomprehensibility was recognized as more important than ever, even if it was still seen as a necessary evil or a chore. The reader’s relationship with the text was another focal point identified by our students, more specifically, the degree to which they were willing and able to indulge in competition. Whether or not one chooses to interact with a text as though it were a game, certain of Pry’s visual elements direct the reader’s attention to their “progress”: in particular, the four faint diamonds on each chapter heading, which are replaced with a reddish polygonal glaze upon completion of story material within the individual chapters. Exactly which major narrative elements must be experienced to unlock each diamond is unclear, as chapters may be completed with as few as one diamond displaying on the heading. Perhaps, though, the word achievement is misleading: one might well argue (and several of our students did) that these diamonds symbolize the level of comprehension a reader has attained. But for one student the diamonds came to symbolize only the paths taken or the potential comprehensibility of the work as experienced by the reader in that particular reading (Robyn Pickens). The comprehensibility changes depending on the path—­choosing the text’s “interior” paths when possible results in a markedly more textual and less cinematic presentation of James’s narrative and potentially a more difficult narrative strand to follow, re64  Ciccoricco and Large

plete as it is with associative meanings, untethered referents, and rapidly flickering strings of thought. Most students retained something of a surface-­oriented view with regard to interactivity. For these readers the text remained bound to its technology and its artifactuality: they were primarily conscious of their physical interactions with the screen and not always with the text “behind” it. Thus, for all their appreciation of the interface as intuitive, they often remained immersed in the navigational interface at the expense of immersion in the narrative. Our students tended more often than not to gravitate toward a position that a perceived simple interface heightens the appeal of a complex narrative structure—­or even that it might ultimately be an affordance of one. As familiarized end users of touchscreen devices, we have been afforded the silent benefits that allow us to navigate texts such as Pry, in the same way that, as readers of print texts, we readily engage with the interface of covers, endpapers, and indices, not to mention the peculiar gravity (upper left to lower right in our case) that reading dictates. Though Pry’s progress indicators might convey a sense of gaminess, so to speak, the reader’s hand-­eye coordination is moot, and skill is reduced to the sensitivity of each (re)reading: at this juncture might the interior or exterior branch offer a more compelling narrative, or a more comprehensible one? Though the presence or absence of the chapter-­ heading diamonds might suggest otherwise, Pry has no winning state. It is advertised and sold alongside interactive fictions in the App Store on ios devices, but neither “interactive fiction” nor “game” would categorize it. If we consider the only alternative nominalization offered by Apple, that of “application” (or app), we may begin to consider most productively just what Pry offers by way of its imbricated layers of complexity. In (reductive) computational terms, an application is a program or piece of software designed for a specific purpose. As far as definitions go, this one is perfectly adequate for a database program or for software that performs a discrete and replicable function. Intentional fallacies notwithstanding, the purpose of Pry as a work of literature must be more complex than successfully compiling and executing its code on an Apple device. We might well ask, then, just what is “applied” within the software package of the Pry application: perhaps an experiment into the limits of transmedia and narrative coherence or a theory of the (traumatized) mind? Caution, Simulation Ahead  65

Keeping It (Metaphorically) Real At the very least, we know that this iPad novella is not a full-­blown attempt to simulate consciousness, on any level. In fact, it is the very messiness and imprecision of the text’s governing metaphor of prying that makes it a rather delimited experiment in representing minds. That is, in this digital narrative, metaphorical complexity ultimately constrains and countermands simulational complexity. Simulation has long endured suspicion. We can trace its conceptual lineage all the way back to Plato’s concerns with false images in the Sophist and his “Allegory of the Cave” in the Republic. The term simulation itself does not enter English usage until the mid-­fourteenth century (from Latin and Old French), and when it does it carries with it only negative baggage: as pretense, imitation, pretend, or counterfeit. The same skepticism put forth by Plato is picked up by Jean Baudrillard in the 1980s to indict media-­saturated culture (1988), and his critique (which began with those familiar targets of television and theme parks) clearly anticipated the even more pointed polemic that simulation would generate as a result of the unbridled increase in computational power. On the one hand, we tend to see an ethically charged critique of simulation in humanities discourse that all too often starts and ends with Baudrillard. On the other hand, we tend to see an overly narrow conception of simulation in computer culture that is dominated by visual and proprioceptive fidelity. Simply put, the more realistic it looks and feels, the better (Alexander Galloway’s [2006] writing on “social realism” in games is a notable exception.) The fears and the fetishes about simulation are unsurprising. After all, it is arguably digital culture’s greatest asset and greatest liability: it is the source of the compelling verisimilitude we derive from gameplay and, according to its critics, also the source of the pernicious attitudes and behaviors inculcated through that same activity. Jorge Luis Borges (1999) saw the perils of simulation early on, evidenced by his famous one-­paragraph short story, “On Exactitude in Science” (1946), in which the map literally becomes the territory. But if artists such as Borges sounded a cautionary note, then so too did scientists such as John von Neumann, namely at the Macy Conferences, which ran from 1946 to 1953 and are widely considered to be the origin of cybernetics and classical cognitive science. The principal task of the con66  Ciccoricco and Large

ference was an interdisciplinary understanding of the human mind, and many of its participants were turning toward simulation in the form of experimental systems modeling so as to reach that end. But as the meetings wore on, a marked confusion arose at the core of their task. As Jean-­ Pierre Dupuy writes, “Object and model exhibited an awkward tendency during these discussions to constantly switch roles. The ambivalence of the very word ‘model’ . . . was thus revealed: the model may be either that which imitates or that which is, or deserves to be, imitated” (2000, 138). Still, notably, on the eve of designing and building the first electronic computers, von Neumann put forth the notion of a critical threshold with regard to simulations: the idea is that there is a point at which the complexity of a dynamic model of a system takes on a life of its own, and past this point it no longer refers to a target in a useful way. Rather, it tells us more and more about itself—­about its own complex autonomous behavior (141–­43). Rather than perpetuating the panic about crossing beyond the threshold that marked much cultural theory of the late twentieth century, however, artists and scholars might instead recognize the many ways in which cultural production can fulfill a vital role just under it. Metaphor in particular can act as a counterpoint to simulation in this respect: metaphor maintains a distance between source and target domains, whereas simulation attempts to elide that very distance through the faithfulness of its one-­to-­one mappings. Maintaining that distance through metaphor accomplishes the kind of cultural work that preserves rich hermeneutic complexity, which in turn can cultivate the practice of critical and ethical analysis that grows out of it. The move is instructive in that it carries philosophical implications beyond literary domains. Writing in the late 1990s about interface design and the pervasive desktop metaphor, for instance, Steven Johnson opposed metaphor (that which maintains distance) and simulation (that which elides it) and considered the shortfalls of simulation in the same light. “Deference to real-­world conventions has its limits,” he writes. “More limber, loose-­ fitting metaphors seem to work better than meticulous simulations, if only because the world of atoms is subject to so many restrictions that have no purchase over the world of bits” (1997, 232). In creative narrative media, the loose fit of metaphor not only preserves the conceptual richness we value in literary complexity but also reminds us that simCaution, Simulation Ahead  67

ulation is not and should not be the end-­a ll goal in narrative media, even in works where it performs a compelling and conspicuous role.5 Pry demonstrates what we can gain through the strategic deployment of metaphor. The application is bound to its ios medium: while the reader’s touchscreen input is limited to swipes, pinches, and taps, the seeming simplicity of the input device provides a suitably loose metaphor for embodied, cognitive, and affective simulation. When readers swipe across rendered Braille in chapter 3 (see figure 7) or pinch out to open the protagonist’s eyes from chapter 1 onward, they are physically mirroring embodied motions, if not the ability to, for example, translate the bumps of Braille to letters and then to words. Rather, James’s voice reads in sync with our swipes. In the absence of its textures, the Braille is reduced to a simulation, albeit one that removes its most basic functional essence. This said, simply seeing Braille invites touch to such an extent that no instructions are supplied—­or necessary—­in chapter 3. After all, a textured simulation of Braille would in effect be Braille itself: a totalizing simulation with no separation between model and target. In any case the haptic metaphors at work here are intuitive and familiar, though their conceptual complexities escape those of a totalizing simulation. The metaphor of prying, present in the title and ever-­present in our interactions with the application, multiplies the meanings we seek—­in the mutable truths behind James’s words, in the tangled chronology of his memories, or even in locating his occluded present—­just as it constrains our operational input. As Pry reaches a narrative crescendo in chapter 6, for example, and we begin to understand the complex relationship dynamics between James, Luke, and Jessie—­not quite sibling rivalry, not quite love triangle—­our input is reduced to prying and pinching. We begin with two lines, seemingly unrelated: “Back in Schwetzingen, I saw her eye him during pt. My chances were pretty much screwed. / The world moves like a video game. Luke carries me, dodges. The exit flickers, flattens before us.” In prying open these lines and literally reading between them, both distant and recent past begin to unfold. As those first two lines blossom (explode?) into a fully fledged textual narrative with rapid serial visual presentation and video interstices, the reader is left with two primary courses of action: reread each instance as new lines are pried apart (and risk a fractured or partial understanding) or pry apart everything possible and begin again, at the start. As James had done in the previous chapter, 68  Ciccoricco and Large

Fig. 7. Simulated Braille used as a haptic metaphor.

we once more returned to recapture something—­this time not what was lost, but that which was deliberately hidden from our view. The narrative complexity offered by Pry is thus a function of the creative constraints imposed by its interface, which simultaneously allows the reader novel possibilities to access alternate readings of the same narrative in potentia. Lies to Children and the Lying Liars Who Tell Them Conceptual complexity has virtuous consequences. A critically and ethically charged engagement with literary complexity is chief among them. Whether the virtue of patience is a product or a prerequisite of such engagement remains a matter of some debate. But the fact that the allure of simulation profoundly reshapes, and even threatens to occlude, our literary encounters is beyond doubt. Our overall note of caution is resonant with a childlike enthusiasm for the promise of simulation and all of its (nontotalizing) rewards. With that, however, comes an equally sonorous call for the place of metaphor in twenty-­first-­century technologies of storytelling. Pry rewards the reader for enacting both a literal and figural pulling apart of its narrative, but the closer we examine any text, the greater its complexity will appear—­more and more Byzantine, Gordian, Daedalian; the labyrinthine analogies are as elaborate as they are endless. Such analogies, and their allusive heritage, serve to confuse, to lock, to hide, or to conceal. They turn the mind inward and away from its initial intent or Caution, Simulation Ahead  69

purpose. We tie our own knots; we build our own labyrinths. So too, however, might we choose to slice through the knot and escape complexity or, by engaging, act out the roles of both Theseus and Ariadne and spin our own guiding thread. A curious blurring of vision is required to do so. As readers of complexity who nonetheless desire a degree of closure, we find ourselves in a position similar to the student of physics who, seeking a fundamental if particulate truth, must reconcile the elegant simplicity of Newtonian physics with the occluded intricacies of quantum interactions—­complexity at the subatomic level, visible only to the most particular of observers. All of this is to address, in the kindest of terms, both our propensity for self-­serving myopia and a necessary fondness for the reductive simplicity of lies to children, those elegant fictions that reduce the world’s infinitesimal complexities to a structure comprehensible by human minds (Cohen and Stewart 1994). How else to frame or encompass the most diverse of minutiae? We engage in healthy self-­deception whenever we are confronted with complexity. As psychologist Timothy Wilson (2004) suggests, it is to our benefit—­even highly adaptive—­that our nonconscious mental processes take part in positive illusions: those with such innate optimism, he suggests, are more likely to persist at length on difficult tasks and almost always more likely to complete them. The narrative salvo of chapter 6 notwithstanding, Pry anticipates in its readers something of the same self-­ deception. At the same time we must be patient, with James and the proverbial still waters in which we can see his clear reflection: he lies to himself and thus, initially, to us; he relates the same event seemingly from multiple perspectives; he, or the application itself, replays the same scene ad infinitum until, exhausted by repetition, we turn away or pry further onward. In perhaps his most vexing tendency, he changes his mind even just as we glean its inner workings. Nonetheless, as we indulge the technical trappings of simulation to pursue a complex narrative and a complex mind, the text ensures that we remain mindful of metaphor with every touch. Notes The intellectual, compositional, and editorial labor for this chapter was divided equally between the two authors. 1. Theory of mind is best explained either as an endowment that works like an in-­built intuitive theory (innate or developed over time, through trial 70  Ciccoricco and Large

and error) or as a full-­blown imaginative projection—­offline simulation—­ of another’s situation. These positions are known as the “theory-­theory” and the “simulation theory,” respectively. 2. It is this computational sense of simulation that played heavily into the opposition posed by some early game studies scholars between narratology and ludology in what was a short-­lived and at times misguided polemic (see Rovner 2009 for a succinct summation). 3. Despite its common and erroneous attribution to the psychoanalytic tradition, the term subconscious is for the most part homeless with regard to the technical, psychological, and cognitive scientific lexicon. Its widespread adoption is typically traced back to the hype surrounding subliminal advertising from the 1970s. We follow its usage in the text of Pry when necessary. 4. The direct quotations are taken from student responses posted to a class-­ discussion forum on the Blackboard learning management system, which was internal to the class and not publicly available online. Responses are cited with student permission, and we gratefully acknowledge their contribution. 5. There are alternative conceptions of simulation and metaphor’s relationship, especially in game studies, where they have been employed interchangeably in terms of games conveying rhetoric (see Möring 2013). Here we retain the literary and cognitive linguistic denotations of metaphor as, respectively, nonliteral figuration and the act of experiencing or understanding one domain in the context of another; despite the potential parallel between functional metaphors (which index the similarity between what two things do) and simulations (which model what systems do), we treat the two as sufficiently differentiated and productively opposing entities. References Baudrillard, Jean. 1988. Jean Baudrillard: Selected Writings. Edited by Mark Poster. Cambridge: Polity. Bell, Alice, Astrid Ensslin, David Ciccoricco, Jessica Laccetti, Jessica Pressman, and Hans Rustad. 2010. “A [S]creed for Digital Fiction.” Electronic Book Review. March 7, 2010. www​.electronicbookreview​.com​/thread​ /electropoetics​/DFINative. Borges, Jorge Luis. 1999. “On Exactitude in Science” (1946). In Collected Fictions, translated by Andrew Hurley. New York: Penguin Books. Ciccoricco, David. 2007. Reading Network Fiction. Tuscaloosa: University of Alabama Press. —. 2012. “The Materialities of Close Reading: 1942, 1959, 2009.” Digital Humanities Quarterly 6 (1). www​.digitalhumanities​.org​/dhq​/vol​/6​/1​/000113​ /000113​.html. Caution, Simulation Ahead  71

—. 2015. Refiguring Minds in Narrative Media. Lincoln: University of Nebraska Press. Cohen, Jack, and Ian Stewart. 1994. The Collapse of Chaos: Discovering Simplicity in a Complex World. London: Penguin. Dupuy, Jean-­Pierre. 2000. The Mechanization of the Mind: On the Origins of Cognitive Science. Translated by M. B. DeBevoise. Princeton: Princeton University Press. Eliot, Thomas Stearns. 1921. “The Metaphysical Poets.” Times Literary Supplement, October 1921. Galloway, Alexander. 2006. Gaming: Essays on Algorithmic Culture. Minneapolis: University of Minnesota Press. Gorman, Samantha, and Danny Cannizzaro. 2014. Pry. Tender Claws. http://​ prynovella​.com. Hayles, N. Katherine. 2000. “Open-­Work: Dining at the Interstices.” Riding the Meridian. www​.heelstone​.com​/meridian​/templates​/Dinner​/hayles​.htm. —. 2002. Writing Machines. Cambridge ma: mit Press. Johnson, Steven. 1997. Interface Culture: How New Technology Transforms the Way We Create and Communicate. San Francisco: HarperEdge. Johnston, David Jhave. 2014. “Jhave on Pry.” Los Angeles Review of Books, December 29, 2014. https://​lareviewofbooks​.org​/review​/prying​-­­jhave​-­­on​ -­­tender​-­­claws​-­­new​-­­app. Kirschenbaum, Matthew. 2008. Mechanisms: New Media and the Forensic Imagination. Cambridge ma: mit Press. Möring, Sebastian. 2013. “The Metaphor-­Simulation Paradox in the Study of Computer Games.” International Journal of Gaming and Computer Mediated Simulations 5 (4): 48–­74. Rovner, Adam. 2009. “A Fable, or How to Recognize a Narrative When You Play One.” Journal of Gaming and Virtual Worlds 1 (2): 97–­115. Stefans, Brian Kim. 2005. “Privileging Language: The Text in Electronic Writing.” Electronic Book Review. November 5, 2005. www​ .electronicbookreview​.com​/thread/ firstperson/databased. Tabbi, Joseph. 2003. “The Processual Page: Materiality and Consciousness in Print and Hypertext.” nmediac: Journal of New Media and Culture 2 (2). www​.ibiblio​.org​/nmediac​/fall2003​/processual​.html. Thomson, H. J., trans. 1949. Prudentius. Cambridge ma: Harvard University Press. Wilson, Timothy D. 2004. Strangers to Ourselves: Discovering the Adaptive Unconscious. Cambridge ma: Belknap Press of Harvard University Press. 72  Ciccoricco and Large

3

The Wave-­Crest Narrative Complexity and Locative Narrative Emma Whittaker

We live, as it were, upon the front edge of an advancing wave-­crest, and our sense of a determinate direction in falling forward is all we cover of the future of our path. —­W illiam James, Essays in Radical Empiricism

It is a cold and bright morning in 1995. White frost sparkles in the light of the yellow winter sun, crunching under foot as I cross the lawn of Hanley Park, carrying a six-­by-­two-­foot sheet of glass. Leaning this transparent page against a tall fir, my reflection, well wrapped in duffle coat and gloves, provides the contrast required to reveal the screen-­etched text on its surface. In the short piece of prose, the narrator recalls the poignancy of this tree and its neighbor; they were once “loving-­trees,” but now “clipped into slight points, they are two wobbling chunks of green echoing the breeze” (Whittaker 1995). Artists and writers have often felt the outer world reflect back their inner landscape, and, remade as metaphors, they construe their characters’ inner commentaries. In the aforementioned work, the character’s and the reader’s location is identical. The boundary between worlds, tangible and described, is uncertain, as the surface of the page returns the image of the onlooker. This situated narrative experience, re-­sited in bus stops, demolished buildings, and gallery spaces, played with the relations between the site, the story, and the reader-­participant, implicated by their presence and the mode of writing that moved between first and second person. Ben Russell’s Headmap Manifesto (1999, 17, 28) foretold the possibilities for “location aware devices” to create dynamic networks that could connect object and place. He suggested strategies such as attach73

ing histories to places and animating objects with sound, games, and narrative; leaving notes “without a visible sign”; binding visual, textual, and oral information to places that can be found by others; “supplementing” signs, clothes, and advertising; and tracking people, animals, and things. Tactics that he suggested would change relations between physical, digital, and “augmented space,” affecting perception and behavior. In the past two decades research in navigation and mobile technologies has opened up many opportunities for situated narratives. A range of hardware and software capabilities, including positioning techniques and programmatic triggering of media files and sound playback, together with low-­tech props and tools, have been utilized in diverse forms of location-­based play, theatrical and storytelling experiences. City streets, remote clifftops, and public buildings have been transformed into fictional places. Terminology in the Field A variety of terms have been used to describe site-­specific narratives. Jeremy Hight (2006, 2) claims “locative narrative” as his own, with reference to his 2002 collaborative project with Jeff Knowlton and Naomi Spellman, 34 North 118 West, an early example of the use of gps to trigger historically inspired audio narratives of Los Angeles on portable computers. Hopstory (2002) is a “day in the life” narrative, in which wireless “iButtons” launched audio and video character vignettes on personal devices inside a Dublin hop store (Nisi and Davenport 2011, 210). This work is described by Linda Doyle, Glorianna Davenport, and Donal O’Mahony (2002, 348) as a “mobile context aware-­narrative” and as “a story that responds to the physical and social context of a mobile user.” Martin Rieser (2005, 8, 11) describes “spatial narratives” and “locative media narratives” as storytelling in which “diegetic space” is “mapped” onto geographic locations. Marie-­Laure Ryan, Kenneth Foote, and Maoz Azaryahu (2016, 1, 141, 207) use “spatialized narratives” and “narrating space” to refer to the intersections between narratology and geography and the representational, linguistic, methodical, and contextual roles that space plays in narratives and that narrative performs in cartography and toponymy. And in Jason Farman’s (2014, 4) survey of the field, he chooses “mobile media storytelling” to capture a variety of practices where locative media connects story to place and space. 74  Whittaker

Rita Raley’s (2008, 135) vocabulary distinguishes between works of locative media that link individual participants to places and “locative narratives” that have geographic and narrative spaces in common, which are “well positioned for the literary work of imaginative transport.” Her term “mobile narratives” (2010, 302) specifically refers to literary experiences created for mobile platforms where the platform itself holds meaning for the narrative. For Brian Greenspan (2011, 5), however, “locative narrative” is distinguished from other mobile media projects precisely because of its literary dimension. Jon Dovey’s (2015, 141), Tom Abba’s (2016), and Kate Pullinger’s (2016) term “ambient literature” denotes both a lens for place-­orientated literary practices and a “genre of writing”. There is an orientation toward the text, histories of the book, and the authorial intention to create situated literary works, with an emphasis on interpretation and meaning rather than configurations of media content and navigation tech. Nomenclature’s exquisite corpse presents varying emphases on place, technology, politics, aesthetics, performance, and literariness. Works that mediate relations between the text, the reader, and their location, such as walks created by artists Janet Cardiff and George Bures Miller, pervasive games by Blast Theory, and participatory performances by Duncan Speakman, I collect within the broader genera of locative narrative. Narratives intentionally situated in the reader’s location do not have the same signification as eReaders or podcasts that happen to be read or heard in the park or on the bus. Locative narratives are not just experienced in the mind’s eye, on a screen, stage, or play space, but, using text, sound, images, haptics, and apis, they integrate the surroundings of the individual participant. While there is variance in name and descriptors, I argue that locative narratives share three common elements: emphasis on the place of reading, listening, and watching; the mode of interaction or participation; and movement and navigation. A Framework for the Narrative Complexity of Locative Narrative The focus of this chapter is the formal and experiential narrative complexity of locative narrative, which I argue is complex on the grounds that (1) the authorial intention is to situate and integrate the story world within an existent location; (2) the structure of the narrative is linked with the mode of interaction; (3) the reader-­audience becomes a participant whose actions take place within the fictional world and the existent The Wave-Crest  75

environment; (4) the spatial-­temporal complexity of partaking is inherent; (5) ontological ambiguity may result when perceptual cues are not readily attributable to the existent place or the fictional world of the story; and (6) the prompting of experiential and epistemological questions arise, such as “What does it feel like to be a participant in a story?” and “What is the truth-­value we attach to our experience when our actions take place in narrative worlds?” Underpinning this framework is a conception of experience informed by an experiential pragmatism, drawing on the writings of late nineteenth-­and early twentieth-­century philosopher and psychologist William James and contemporary proponents of embodied cognition.1 James (1907, 201) made the case that we predominately believe our perceptions and ideas to be real or true, despite them resting on little verified information; we perceive a sound outside our window to be a car horn, or footsteps to belong to a live human being, while their source remains unconfirmed. When we scrutinize our beliefs, for example, about science, politics, or even what someone means when they converse with us, they rest, it is often revealed, on untested assumptions that function by bringing us to our next thought or action: “We live, as it were, upon the front edge of an advancing wave-­crest” (1912, 69). The plasticity of how we perceive truth and belief in our everyday experiences presents an opportunity for locative narratives to intentionally blur distinctions between “truths” and “fictions” by their story worlds being set within, or synonymous with, the participant’s environment.2 Readers are participants moving in the world of the story and their surroundings. Misperception, categorical ambiguity, or confusion can result when the borders between what’s inside and outside the narrative are not neatly delineated, for example, “I heard [recorded] footsteps coming up behind me and thought there was someone there”; “I thought the museum guard was part of the story”; or “I forgot I was taking part; it felt like I was really there.” Narrative complexity is also evident when the structure of the narrative is linked with the mode of interaction; for instance, a mystery-­story structure based around a central enigma may use a treasure-­hunt interaction mechanism involving a physical search for objects, places, or information. The integration of the story world in participants’ immediate or broader environment also activates questions concerning the spatial-­ 76  Whittaker

temporal complexity of partaking—­is the participant an observer or a protagonist, can they change the course of story events, and is the presentation of time in the story synonymous with the present moment or with past or future events? In James’s philosophy of radical empiricism, he argues that the categorization of experience is dependent on our intentional directedness, our “mental pointing” toward an object.3 Experientially, thoughts and things are differentiated by ways of speaking, “subjectivity and objectivity are affairs not of what an experience is aboriginally made of, but of its classification. Classifications depend on our temporary purposes” (1912, 141). The idea that thoughts point toward objects, which are themselves thoughts, could lead to solipsistic conclusions. How can things be known? From James’s (1909, 210–­11) pragmatist perspective, perceptions terminate in existent objects, and concepts can potentially arrive, through a sequence of thoughts and their motor consequences, at a physical object that we could point to and say, “Here’s the crow to which I refer.” This knowledge is verified, but its truth value is limited and particular to the context of its use. In a work of locative narrative, a passage may refer to a bird, and somewhere in the distance a (recorded) seagull calls. The potential to verify its existence seems possible to the participant in the congruent context of the garden in which they walk. But while the source of this sound is not actually available, the recorded sound may be misattributed to existent gulls in the location. James’s pragmatist method treats beliefs as hypotheses that can be subject to empirical investigation (1907, 53). Testing and verification take place within a particular context, and the results are provisional truths and further hypotheses.4 Their meaning is a value attributed within the stream of experience. We have the textual suggestion of the presence of the bird, the recorded audio of the bird, and a congruent context. There is the potential that we will form the hypothesis that the seagull in the story is present in our location, unless other evidence overrides our assumption. All mental states, which James defines as thoughts and feelings, are motor in their consequences (1892, 5), leading to some bodily change; the functioning of the body for James is for the production of action. Classical pragmatists, particularly James and Charles Sanders Peirce, are acknowledged by Harry Heft (2001, 15) and Mark Johnson and Tim Rohrer (2007) The Wave-Crest  77

and others as antecedents of contemporary extended, embodied, and enactive theories of cognition. Relations between action and perception are key tenets of James J. Gibson’s ([1979] 1986, 244) theory of embodied cognition, which describes the animal acting in its surroundings, its body, head, and eyes adjusting as part of the perceptual system that directly perceives invariants and affordances in the environment. Concurring with Gibson, Kevin O’Regan and Alva Noë (2002, 567) question whether sensory stimulation and neural activity are sufficient to give rise to perception. Their sensor-­motor approach describes vision as an exploratory process within the environment. Our experience of the world is one that appears fully rendered and within an expected range accessible to us; however, as Noë (2004, 49) notes, our field of vision is narrow and blurred at the edges. Turning our head moves this focused vision, enabling us to see (and hear) more, giving the impression of continuity. Our susceptibility to misperceptions does not necessitate that there are representational “sense data,” which are in turn faulty, or that our perception is fundamentally illusionary, but that the misperceptions reveal “our context-­bound performance limitations” (2009, 142). “Perceptual consciousness” for Noë (2012, 24) is “relational,” a type of intentional directness toward objects. For James and Noë it is the percipient’s intentional relations to objects (above and beneath the level of awareness) that affect perception and interpretation. Aesthetic value and sentiment can also affect what we perceive and conceive. When we see the moving curtain, we may perceive it as a ghostly figure and feel apprehensive. Love, anger, and fear are not merely affections of the mind but also of the body (James 1912, 142). Feelings ascribed to physical objects can “produce immediate bodily affects” (150), such as an increased heartbeat, quickened breathing, sweating, salivating, and so forth. In the sentences “I feel the cold snow” or “I feel that the snow is cold” there is slippage in the assignment of adjectives to the subject or object. The ambiguity of classifying effects and the potential to evoke emotional responses to objects in the physical world can be harnessed in the writing and experience design of locative narratives—­the wasteland between buildings is described as passage between worlds, becoming a physical and an emotional challenge for the participant to negotiate. In the classifying of objects or events as “real” or “true,” there is an assigning of a value that, as James (1896, 75–­76) points out, can result 78  Whittaker

in a circularity that may seem to be legitimated by evidence accompanied by the subjective feeling of being objectively right. Perception is not an unqualified correspondence or copy of the world; it is mediated by our “experienceable environment” (James 1909, 41), the evolutionary evolved structure of our brain ([1878] 1920, 63–­68), and cognitive processing above and beneath the level of our awareness. Locative narratives can harness the potential of our “context-­bound” perceptual functioning to perceive more or less than is actually present, by changing relations to objects and affecting interpretation, by directing movement and attention, and by using language, sound, and images to prime participants’ expectations of what they may perceive in the environment. When participants describe their experiences of locative narrative, reoccurring themes are truth and belief and their uncertainty in classifying the real, the represented, or the imagined. Arguably, any narrative defined as a way of telling (White 1981, 1, 4) has the potential to experientially unsettle the ontological status of objects. Our imaginings may have a “reality-­feeling” (James 1902, 58), and what we read may shape how we interpret the world, within and beyond a text. I make the case, however, that locative narratives have additional formal and experiential features that can affect how participants assign ontological status to objects and events: these include linguistic, sonic, and visual methods of priming perception, misperception, and interpretation; reframing of the participant’s surroundings as a story world; and the relationships created between the physical instantiation of the story world and the participant’s body, which is often situated within and moving around the story world. In this framework, underpinned by experiential pragmatism, the six features of the narrative complexity of locative narrative outlined earlier are discussed in relation to four experiential thematics (action, perception; feeling and sentiment; intentional content of thoughts; and truth and belief) and applied in the analysis of the case studies that follow. Case Studies LociOscope: The Letters LociOscope: The Letters (2014) is a locative narrative based on an archive of travel correspondence sent between prospective fiancé and fiancée in 1925 and sited in the garden of their future home, Dartington Hall, Devon, The Wave-Crest  79

United Kingdom.5 The smartphone app purports to be a device for “tuning into” the past that responds to the emotional resonances in the garden, depicted as “clouds” on an onscreen map. The map shows a parallel fictional landscape on which your position is also plotted. Wearing headphones, you search while listening to radio tuning sounds that indicate proximity to the clouds. When you enter, binaural spatial sound portrays one of seven travel destinations, such as Rome or Yosemite National Park. Inside these locations you hear characters voicing fragments of their letters, sent to each other via banks and hotels, often arriving out of order, resulting in protracted misunderstandings played out across continents, by pen and telegraph. Will they or won’t they meet? What is the mysterious “experiment”? Will their friends and family intervene? Different perspectives are suggested according to the order in which they are encountered. Narrative descriptions, auditory representation, and movement in the environment are used to extend the travel destinations into the existent world. Voices populate the simulated places, local inhabitants, and friends, present and absent. You hear laughter—­is it a party? The fictional places make visual and auditory connections with the existent garden’s ambient sounds, for example, “San Francisco Hills” is triggered within the redwood trees in the garden and “San Francisco Harbor” near the fountain. Consumer in-­ear headphones can allow both recorded and ambient sounds to be heard, while attenuation and silence obscure the status of their existence. Walking, searching, listening, and making sense of the fragments are acts in the story’s construction and its conjuring trick. The Lost Index: natmus This situated narrative, experienced on headphones, is distributed across two locations in Copenhagen, at the National Museum of Denmark and the Dieselhouse Museum. The story begins, “In the near dystopian future . . . privacy no longer exists; nothing is wholly personal or secure, all is within the domain of the state, monitored across multiple and intersecting data streams. Activists are pushing back, secretly developing a distributed intelligence system. . . . Traces of this ‘Body-­Mapping’ technology have been detected by government agencies and other interested parties” (Whittaker and Brocklehurst 2015). When you enter the imposing Dieselhouse Museum, the former source of the city’s power, 80  Whittaker

your phone begins to ring. The first of a series of calls is from an old acquaintance, and another is from someone, who it seems, is being forcibly held. What should you believe? You hear disturbing information about your role in an experiment. You need to rendezvous with a contact at the National Museum of Denmark to find out the truth. You explore the first floor of the museum. The narrative structure has an overarching linear plot built from a series of chapters delivered through phone calls at set points over the twenty-­minute duration of the experience. Each chapter is selected at random from a collection of possible recordings and associated with a particular plot point, providing differing perspectives on the story each time the app is launched. natmus’s (2015) interaction mechanism involves searching in the museum. A tracking device appears on the participant’s phone, indicating their relative position to artifacts. They are asked to locate numbers associated with objects and input them into the on-­screen index within a time limit. Logged objects trigger chapters; some of these provide clues to locations, while others introduce obstacles that slow them down, sending them to different rooms in the museum (see figure 8). Low-­energy Bluetooth beacons seamlessly trigger a 3d sound environment in each room that emulates and enhances the ambient sounds. As time ticks, “electromagnetic interference” occurs, and the dimensions of the rooms seem to change. Binaural recordings of the ambient sound are gradually replaced with the spatial dimensions of another subterranean environment. natmus is not a trail between two sites; rather, it integrates two places and their collections within a story world that may be visited in any order, motivated by character interactions. Journeyer’s Guidebook Launching the app at Jubilee Library, Brighton, United Kingdom, you put on headphones and are invited to go on a journey that begins either indoors or outside. These two parallel guided imaginary experiences take the form of a shamanic medicine walk (Harner 1992, 68). The outside journey begins with entering and moving through a tunnel, depicted in binaural sound, as you walk to the Pavilion Gardens. Your attention is drawn to your body moving through space and your relations to objects in your field of vision. Arriving at the garden signals entering another world, to exThe Wave-Crest  81

Fig. 8. The Lost Index: natmus (Whittaker and Brocklehurst 2015). National Museum of Denmark, Copenhagen.

perience “nonordinary reality,” suggested by heightened ambient sounds speckled with tropical birds and animals. A series of attention-­focusing techniques embedded into the story encourage you to carefully notice your surroundings and choose a route around the garden, before engaging in a personal divination ritual and the contemplation of its meaning. Arriving back at the library, you go inside and collect a copy of the illustrated book Journeyer’s Guidebook (2016). Sitting down, you embark on the journey to the garden again, this time via the symbolic representation on the page that takes you through the tunnel, focusing attention, as you transition into the illustrated garden. Now you turn the pages in any order, selecting different paths. Spatial audio surrounds you with the sounds of the tunnel, the garden, and the events and people encountered within (see figure 9). The narrative addresses participants in the second person and is structured as a series of directions and ellipses designed to invite concentration. The emphasis is first on their body in the chair or in the street, then 82  Whittaker

Fig. 9. Journeyer’s Guidebook (Whittaker and Brocklehurst 2016). Jubilee Library, Brighton, United Kingdom.

toward objects and other people in the garden, and later beyond the immediate surroundings to the walk of the Lakota Sioux medicine man. Finally, the narrative gives instructions for the divination activity. The mirrored narratives invite comparisons between the physical and imaginative journeys. In the following section I discuss the six formal and experiential parameters of the narrative complexity of locative narrative in relation to these case studies: The Wave-Crest  83



1. The authorial intention is to situate and integrate the story world within an existent location.

Place-­based writings of those such as Annie Dillard (1983) or James Attlee (2007) vicariously take the chair-­bound reader on a journey via their recounted ambulation. The content of thoughts is an act of “conscious reference,” not of representation, in Noë’s terms (2012, 25–­27). In the locative narrative case studies, existent features of the environment, people, objects, and occurrences are featured within the story, while fictional things, characters, and events are situated within the reader’s vicinity, through the narrative description and recorded sound. The story affects the participants’ interpretation of the surroundings, and live events occurring in the place can impact the interpretation of the story; both types of relations can influence how participants act. In natmus the enemy agent is actually present in the museum. It is the story world that allows a movement between what is taken as the real, the represented, and the imagined. Existent and fictional objects are subject to the logic of the story world and its social norms, physical laws, histories, geography, and so forth. Participants are invited to feel the effects of the fictional electromagnetic radiation by pressing their hand against the museum’s walls. Ryan describes a “storyworld” as a “cognitive construct,” “not encoded in material signs” (2006, 8, 7). For David Herman (2009, 193) the storyworld is a “global mental model of the situations and events being recounted” that are distinct from its representation in the medium of a book, film, or game. Ryan and Thon (2014, 5) describe the storyworld as “medium free” to account for its operation across media forms. I argue, however, that when the characteristics of a story (place, characters, events) occur in different media, including our retelling in thought or speech, it is the instantiation of specifiable content that we assign to Middle-­earth, Wonderland, or Hogwarts. The question can become a traditional philosophical problem of substance: what qualities are essential to an object’s identity? We could say, “We identify a depiction as Hogwarts because it appears to have consistent qualities with those we associate with our prior experiences of that place, whether through the books, conversations, or film advertising.” In James’s (1907, 85, 86) terms, an object is what it is “known-­as.” There are only modes of appearing: a table occupies space and has brownness or hardness in the particular context of a happening 84  Whittaker

event. (Modes of appearing can be taken as the reader-­participant’s interpretation of a story world, in contrast to “ways telling” which arguably emphasizes the intended meaning of the book, film, etc.) It is precisely the particular instantiation of the story world, the combination of specifiable content and how its qualities appear to the interpreter, that is important for locative narrative. The experiential story world concept emphasizes the event of the participant and their environment and how they can be conceptually and physically integrated within a story, from which locative narratives’ complexity extends.

2. The structure of the narrative is linked with the mode of participant interaction.

Situated narratives that are composed as a series of parts can be described as nodal. The medium (voice, soundscape, screen, location); segregation of narrative content (themes, events, characters, chronology); and quantity of content within a node is determined by the overarching structure of the narrative. LociOscope has seven large nodes, characterized in this work as the characters’ travel destinations. They are encountered by participants in any order. Within each node are a series of subnodes, which are experienced sequentially; however, they can be interrupted by leaving one place and entering another. Riot! 1831 (2003) (Reid et al. 2004) is a dramatic recreation of the battle that took place in Queen Square, Bristol. Walking freely within the public space, participants hear a soundscape on headphones that depicts the unfolding scene, while the narrative nodes (individual character vignettes) are triggered by gps in any order. Locative narratives whose events are experienced in a particular sequence, for example, when delivered as a single audio file, are necessarily structurally linear. In Cardiff and Bures Miller’s Villa Medici Walk (1998) the interaction mechanism involves wearing headphones, and listening to the main character’s first-­person narration, in which the route and actions are embedded. Although the underlying narrative structure is linear, it doesn’t preclude a play with time and space within the writing itself, which contains accounts of memories and contemporaneous events set in other locations. Nodal narratives can be structured with multiple story trajectories. The variables that describe the combinational logic of the nodes can be The Wave-Crest  85

temporal, sequential, or thematic. They may affect all nodes or only certain classes; for example, participants can encounter x, y, and z character nodes only after they have heard story event nodes a, b, and c. The structure can employ various logics and mechanisms; for example, an overarching narrative may have different subplots or parallel story lines within the same world. In Machine to See With (Blast Theory 2010) the drama of the heist is staged between participants through their mobile phones. A series of core nodes, experienced by all, are delivered as voicemails that contain story background, navigation, and instructions for activities. At set points the participant is asked to respond to a psychometric test and make various decisions that result in different sets of subnodes, which return to the core nodes and resume the temporal sequence of the main narrative events. Unlike on-­screen computer games, where there can be many algorithmically generated narrative paths or dialogue, locative narrative can be constrained by the practical plotting of story in terms of the existent location’s characteristics, distance to be traversed, and proximity of media files to one another, when triggered by gps or beacons.6 Structure of the narrative refers to the order of events, whereas the interaction mechanism is concerned with ways the participant engages with the narrative, such as listening, searching and discovering, puzzle solving, responding to skillful and creative challenges, participating in thought experiments, meditating, walking, running, dancing, and traveling. The mechanism may, although not necessarily, relate to the story’s genre (e.g., a detective drama is likely to involve puzzle solving); it can have a number of implications for the structure of the narrative. In natmus logging the searched-­for items in the on-­screen index resulted in consequences explicit to the participant. Walking seamlessly triggered crossfades between binaural soundscapes when participants entered different galleries and also phone calls that delivered narrative nodes. The structure and the interaction mechanism can pose challenges for sense and aesthetics. That a narrative is coherent can be taken as the expectation that causal relationships between story events retain the logics of the plot and that of the story world, without which the story can seem implausible. Plausibility, as a criterion for coherence, presents a number of questions that can be thought of as internal to the narrative structure, the logical connections between events and content; and those external to the narrative, the style and the execution of representation and its re86  Whittaker

lations to its referents in the world. (Would people in x scenario exhibit these types of behavior and is the depiction of the causal relations between events logically possible in the “real world?”) Coherent relations between the logic of the internal fictional world of the representation and the logic of the external “real world” can be a question of aesthetics or sentiment, a feeling of coherence. Relations between sentences infer causal links between events within a narrative. In an early iteration of LociOscope, letters extracts faded out and others faded in when the participant walked away from the assigned location. Sentences “floated” free from their letter and its anchoring reference points, reducing the potential for participants to construct a coherent narrative. The story, plot, and the context of engagement can affect the participant’s performance of supposition. Participants’ supposition, “it’s as if the lovers are going to meet” can tip over into belief that the characters (or those performing the role of the characters) may actually be in the vicinity when they interpret people and events in their surroundings as part of the story: participants reported, “I overheard their conversation behind me.” The aesthetic dimension of coherence has the potential to bridge the internal and external relations to the text.

3. The reader-­audience becomes a participant whose actions take place within the fictional world and the existent environment.

The term participation has been used in cognitive psychology to describe the processes of interpretation involved in engaging with representations. Richard Gerrig (1993, 14), after Frederic Charles Bartlett (1932), extends the metaphor of reading as a performance. Participation in locative narrative involves the situated body operating in a place and engaging in acts from listening and active interpretation to decision making and improvisation. “Perlocutionary acts” are defined by J. L. Austin (1962, 2–­8) as utterances that bring something about or achieve something by saying them. In Journeyer’s Guidebook and natmus the status of “you” is important. It turns fictional statements into declarative performative utterances: “You are observed.” James (1890, 1:245) argues that there is hardly a linguistic utterance that does not express the subjective or objective relations between objects and our thought, for example, “The entrance is just ahead.” The degree of participation is affected by the attitude adThe Wave-Crest  87

opted by readers and their willingness to engage in the narrative premise and the undertakings demanded of them. Attention creates a context for our thoughts, and our perception is affected by what we attend to. If an object is not attended to, it can be in the perceptual field without us necessarily being aware of its presence. Colin Cherry’s (1953) study of selective attention describes the “cocktail party problem,” our ability to identify our friend’s voice on the other side of a noisy room. Studies (e.g., Simons and Chabris 1999, 1059) in change and inattention blindness demonstrate that we fail to perceive those things before our eyes that our attention is not focused on. The potential of focused attention and suggestion to prime the listener for perceiving have long been known. James’s own experimental psychological research (1890, 2:600) observes that hypnotism involves focused attention coupled with the disassociation of background ideas. Recent neuroscience research offers a similar interpretation, “a state of highly focused attention coupled with a suspension of peripheral awareness” (Spiegel 2012, 181), which has been demonstrated in clinical trials to affect visual auditory, somatosensory, and pain perception (Müller et al. 2012).7 For John F. Kihlstrom, (2012, 23) hypnosis is a process involving social interaction and cognitive changes in “experience, thought and action.” Distinct from simply listening, guided imagining employed in locative narrative can be used to direct attention and engagement, as the visitor or reader becomes a protagonist, envisaging or performing actions and influencing the interpretation of happening events in the location. In Journeyer’s Guidebook the walk to the garden and the imaginary journey through the tunnel functions to prime participants for the suggestion of moving from ordinary to nonordinary reality. They are directed to notice bodily sensations, sounds, and environmental features. The book uses attention-­focusing diagrams with soundscapes that reflect and reinforce the verbal descriptions. Arriving in nonordinary reality is also accompanied by binaural sound that subtly heightens the perceptual cues in the environment, preparing participants to receive an answer (in their mind) to their divination question. If perception can be altered through focused attention and suggestion, situated narrative experiences can potentially affect the phenomenal experience of represented and imagined locations and extend the imaginative and affective power of storytelling toward perceiving and acting. 88  Whittaker

4. The spatial-­temporal complexity partakes in locative narrative. The spatial-­temporal design of LociOscope works on a number of levels: the chronology of the story; the duration of the writing and reading of letters; the time taken to traverse the garden and walk “around” each sonically created location; the narrative arc and depiction of events inside each of these places; and the historical duration of the events described within a letter. In natmus the story world extends into the city of Copenhagen, with narrative events centered in two museums linked by participants’ real-­time journey between them, with other locations inferred in the phone calls’ background sounds. Journeyer’s Guidebook plays with the experiential expansion and contraction of space and time. The exploration of the garden occurs physically and imaginatively, through the graphic illustrations in the printed book. The duration exploring the gardens is identical, while the tunnel transition that takes place outside adjusts to the walking speed of the participant.

5. Ontological ambiguity may result when perceptual cues are not readily attributable to the existent place or the fictional world of the story.

Playing with context is a strategy that creators of fictional places have long since recognized, from trompe l’oeil painters who attempted to erase the presence of paint to “living history” museums’ desire for authenticity. The story world manifested in the existent environment in which participants act creates complex relations between the moving body and the narrative. Participants’ of LociOscope reported, “There were a few moments where I jumped; I couldn’t distinguish between which sounds were and weren’t recorded, like the birds and the motorbike” (Whittaker 2017, 308). Binaural soundscapes can bleed into ambient sound, masking or multiplying auditory cues and creating the potential for ambiguity and the misattribution of sound sources, according to their congruency with the environment. Imaginings, or nonperceptual experiences, can be considered first as subjective, in that they are not directly felt or seen but are also objective experiences of present thought. Participants of Journeyer’s Guidebook reported that they asked a stone their question and received an answer. The Wave-Crest  89

There is a “parallelism” between the “presently felt and the remotely thought,” which are coordinate realms for James (1912, 16). Both percepts and concepts are types of thoughts that occur in experience, thoughts, and feelings of objects in our field of awareness. The object and the idea are both parts of reality. The attribution of the status of “the real” can be ambiguous when there are slippages of relations to context. Some ideas have a “reality-­feeling” despite being vague or “almost unimaginable” (James 1902, 58). That thoughts of things not present can feel real is their evocative potential. The normative function of language is to “carry a particular meaning” (Wittgenstein [1953] 2009, xi, 267). We “see as” (xi, 203) a particular linguistic-­perceptual instantiation of an object in the changing environment. Our subjective interpretation, as thoughts or speech, function in terms of the consequences that follow from the particular use of language within an “experienceable environment” (James 1909, 41). What something is classed as, what it means for us, is how it functions. We say that we believed that there was someone behind us when we heard footsteps because we turned round to see who was there. We questioned our belief when we could not see the source of the sounds. Our interpretation is either that there is someone behind us (we mistake the source of the footsteps for an existent person in the environment) or we say there is not someone behind us or we say our interpretation is uncertain, yet to be verified in our experience. Our thoughts are intentionally directed toward objects that, in our lived experience of hypothesis forming and testing, may be verified as true or remain “virtually true.” However, “the immensely greater part of all our knowing never gets beyond this virtual stage” (115–­16). We live, as it were, on a wave-­crest.

6. Particular experiential and epistemological questions are prompted: “What is it like to feel like a participant in the world of a story?” and “What is the truth-­value we attach to our experience as we act in narrative worlds?”

Can our belief that an object is existent depend on our relations to it (our attention directed toward it, the sensations and emotions we connect to it) and its particulars (qualities of vividness and liveliness, its congruence with related things)? Reality is a belief relative to particular circum90  Whittaker

stances, James argues (1890, 2:311). Participants of LociOscope reported that they could see in their “mind’s eye” people traveling and hurrying, people writing letters, and Dorothy meeting new people: “It wasn’t like it replaced what I could see with my eyes but that what I could see wasn’t as important as what I was thinking about.” James characterizes individuals’ categories of experience as “worlds” (1890, 2:291–­92).8 These are classes whose members share a logic of acting and influencing one another, the logic of physics, ethics, fantasy, or fiction, for example. Each world is real “while it is attended to” (2:293). Reality is that which we take to be external to us, yet it is mutable. In natmus participants are museum visitors, users of the app, and, if they adopt an attitude of supposition, characters in the story. When answering the phone, they do so as visitors and characters in the fictional world. If the phone call is from “outside the story,” the relations to the device change to that of the individuals’ web of sociopersonal relations. James argues that which we designate as real occupies real space along with other “reals” (1890, 2:290n). What happens when a fictional object occupies a “factual space”? A phone call from an agent in natmus advises participants to look out for someone in a black shirt. Wearers of black shirts are not uncommon occurrences, and if real bystanders or objects can be incorporated into fictional scenarios there is the potential to extend the narrative into the existent environment. The status of factual and fictional oscillates. If we decide the black shirt is not nearby, that explanation seems more logically satisfying when we have little evidence to support the alternative perspective; however, we may remain on our guard, poised to alter our view. Rational interpretation involves logical reasoning but also aesthetics and sentiment; we feel our rational interpretation to be true if it fits satisfactorily with our existing ideas. An Experiential Pragmatist Approach to Locative Narrative Experience Locative narratives don’t aim to detach or transport readers from their current situation; instead they fashion it, thematically, historically, atmospherically, or perceptually, by directing modes of engagement and framing their interpretation. The existent place, supplemented with a story world in which narrative events occur and readers inhabit, presents complex formal possibilities for narrative structure and story content. The Wave-Crest  91

In this account of the complexity of locative narrative, an experiential pragmatist approach to experience informs the analysis of relations between the percipient, the narrative event, and the environment. Experience, for James, is being happening. Being requires an “experienceable environment, as the vehicle or medium connecting the knower with known, and yielding the cognitive relation” (1909, 41). It is the context in which its and other entities’ qualities can be individuated and identifiable (18–­19). In our experiential awareness the meanings attributed to perceptions and conceptions are what they are known as. They are constructed and context-­dependent, spatial, temporal, social, and personal relations to ourselves. While an idea or perception is not contradicted, it is potentially or virtually true (115–­16). Our beliefs about how the world is may seem to have a rational basis, perhaps supported by evidence, but they are first a hypothesis, our faith or hope in which is driven by feelings or sentiment (1896, 64). Beliefs become true when they are verified as such (1907, 206). The truth of an idea is determined by the practical consequences (45), conceptual or physical, that follow from it, leading to further thoughts or physical activities. Locative narratives can be interrogated as a site where the fallibility of perception becomes evident—­when events, people, and objects in the story and occurring in the outside world may be not only conceptually confused but also bodily felt, sensed, heard, and seen in the participant’s surroundings. These effects can be created using conflicting sensory stimuli, priming, and suggestions embedded within the writing, mode of interaction, and experience design of locative narratives. The malleability of our beliefs is what narratives, fiction or otherwise, trade on; however, locative narratives can take this pliancy further because the medium does not bracket the narrative from the world, as do the covers of a book, cinema, or game screen, and therefore their reality status is categorically ambiguous. Locative narratives also create conceptual and physical contexts for supposition that direct us to adopt suppositions in which we may not only pretend narrated events are existent but also believe, perceive, act, and report them to be so, and when this occurs a locative narrative’s representations have a “reality-­feeling” (James 1902, 58). Notes 1. While James’s writing has continually inspired thinking on a range of topics, since the turn of the millennium a cross-­disciplinary emphasis on sub92  Whittaker

jective experience, in what may be referred to as the “experiential turn,” or “affective turn” (Gregg and Seigworth 2010), has brought new analyses of his work within literature (Levin 1999), aesthetics (Shusterman 2011), philosophy of mind, and neuroscience (Johnson 2007; Massumi 2011). 2. These affects, manifesting in a variety of practices, have been captured by what I refer to as “blurred theories” (Whittaker 2017, 21). Markus Montola, Jaakko Stenros, and Annika Waern (2009, xxi) describe pervasive game worlds that blur boundaries, extending Katie Salen and Eric Zimmerman’s (2003, 94) notion of the “magic circle,” a metaphor for the frames that games create, after Johan Huizinga’s ([1938] 1955) cultural analysis of play. Jon Dovey and Constance Fleuriot (2011, 99) describe “convergences” between media and the environment that can lead to “magic moments” and “synthetic confusion” in locative arts. For Steve Benford and others (2006, 435) “blurring the frame” occurs between fictional and real worlds in hybrid performances, drawing on Gregory Bateson (1972) and Erving Goffman (1974). 3. The content of our thoughts may be perceptually available objects, or they may be memories or imaginings of objects present in their absence, in their “intentional inexistence” (James 1909, 44), after Franz Brentano ([1874] 2009). 4. Ernst Gombrich ([1960] 2002, 24) sketches the lineage of the thesis of interpretation as hypothesis forming from Immanuel Kant to its later use in the twentieth century by Karl Popper. 5. The case studies discussed here (2014, 2015, 2016) are from a series of published apps by Emma Whittaker and James Brocklehurst, part of research (2010–­) investigating situated narrative experience. We extend our thanks to the many contributors and volunteers. 6. Ryan’s (2001, 246–­55) diagrammatic representations of narrative paths most usefully demonstrate a range of configurations for nonlinear narrative. Many additional layers of complexity are introduced when considered as potential structures for locative narratives. 7. An effect of hypnosis is a “top-­down resetting of perceptual response,” Fumiko Hoeft and others claim (2012, 1064). Tiara Dillworth, Elena Mendoza, and Mark Jensen’s (2012) review of research in the neurophysiology of hypnosis concludes that clinical trials indicate the efficacy of hypnosis; however, this is not necessarily correlated with subjects’ susceptibility status. 8. James’s idea was developed by Nelson Goodman (1978, 94). References Abba, Tom. 2016. “Ambient Literature with Tom Abba: ‘What the Dickens Is Ambient Literature?’ Lunchtime Talk Write-­Up.” Pervasive Media Studio, The Wave-Crest  93

January 29, 2016. www​.watershed​.co​.uk​/studio​/news​/2016​/01​/29​/ambient​ -­­literature​-­­tom​-­­abba. Attlee, James. 2007. Isolarion: A Different Oxford Journey. Chicago: University of Chicago Press. Austin, J. L. 1962. How to Do Things with Words. London: Oxford University Press. Bartlett, Frederic Charles. 1932. Remembering: A Study in Experimental and Social Psychology. Cambridge: Cambridge University Press. Bateson, Gregory. 1972. Steps to an Ecology of Mind. Northvale nj: Aronson. Benford, Steve, Andy Crabtree, Stuart Reeves, Martin Flintham, Adam Drozd, Jennifer Sheridan, and Alan Dix. 2006. “The Frame of the Game: Blurring the Boundary between Fiction and Reality in Mobile Experiences.” In Proceedings of the sigchi Conference on Human Factors in Computing Systems, edited by Rebecca Grinter, Thomas Rodden, Paul Aoki, Ed Cutrell, Robin Jeffries, and Gary Olson, 427–­36. Montréal, Canada, April 22–­27, 2006. doi​.org​/10​.1145​/1124772​.1124836. Blast Theory. 2010. Machine to See With. Pervasive game. Presented in Brighton Festival, United Kingdom, September 1–­24, 2011. Brentano, Franz. (1874) 2009. Psychology from Empirical Standpoint. Edited by Oscar Kraus and Linda L. McAlister. Translated by Antos C. Rancurello, D. B. Terrell, and Linda L. McAlister. London: Routledge. Cardiff, Janet, and George Bures Miller. 1998. Villa Medici Walk. Audio walk. Villa Medici, Rome, Italy. Accessed October 23, 2016. www​.cardiffmiller​.com​ /artworks​/walks​/medici​.html. Cherry, Colin E. 1953. “Some Experiments on the Recognition of Speech, with One and Two Ears.” Journal of the Acoustical Society of America 25 (5): 975–­79. Dillard, Annie. 1983. Teaching a Stone to Talk. New York: HarperCollins. Dillworth, Tiara M., Elena Mendoza, and Mark P. Jensen. 2012. “Neurophysiology of Pain and Hypnosis for Chronic Pain.” Translational Behavioral Medicine 2 (1): 65–­72. https://​doi​.org​/10​.1007​/s13142​-­­011​-­­0084​-­­5. Dovey, Jon. 2015. “Ambient Literature: Writing Probability.” In Ubiquitous Computing, Complexity and Culture, edited by Ulrik Ekman, Jay David Bolter, Lily Díaz, Morten Søndergaard, and Maria Engberg, 141–­54. New York: Routledge. Dovey, Jon, and Constance Fleuriot. 2011. “Towards a Language of Mobile Media” In The Mobile Audience: Media Art and Mobile Technologies, edited by Martin Rieser, 97–­108. Amsterdam: Rodopi. Doyle, Linda, Glorianna Davenport, and Donal O’Mahony. 2002. “Mobile Context-­ Aware Stories.” In Proceedings of the 2002 ieee conference on Multimedia and Expo (icme), edited by SuviSoft oy, Institute of Electrical and Electronics Engineers, 2:345–­48. Swiss Federal Institute of Technology, Lausanne, Switzerland, August 26–­29, 2002. https://​doi​.org​/10​.1109​/ICME​.2002​.1035702. 94  Whittaker

Farman, Jason, ed. 2014. The Mobile Story, Narrative Practices with Mobile Technologies. New York: Routledge. Gerrig, Richard. 1993. Experiencing Narrative Worlds: On the Psychological Activities of Reading. New Haven: Yale University Press. Gibson, James. J. (1979) 1986. The Ecological Approach to Visual Perception. Boston: Houghton Mifflin. Goffman, Erving. 1974. Frame Analysis. New York: Harper Colophon. Gombrich, Ernst. (1960) 2002. Art and Illusion. London: Phaidon. Goodman, Nelson. 1978. Ways of World Making. Indianapolis: Hackett. Greenspan, Brian. 2011. “The New Place of Reading: Locative Media and the Future of Narrative.” dhq: Digital Humanities Quarterly 5 (3). www​ .digitalhumanities​.org​/dhq​/vol​/5​/3​/000103​/000103​.html. Gregg, Melissa, and Gregory J. Seigworth, eds. 2010. The Affect Theory Reader. Durham: Duke University Press. Harner, Michael. 1992. The Way of the Shaman. New York: HarperCollins. Heft, Harry. 2001. Ecological Philosophy in Context: James Gibson, Roger Barker and the Legacy of William James’s Radical Empiricism. Malwah nj: Erlbaum. Herman, David. 2009. Basic Elements of Narrative. Chichester, England: Wiley-­ Blackwell. Hight, Jeremy. 2006. “Views from Above: Locative Narrative and the Landscape.” Leonardo Electronic Almanac 14 (7–­8): 1–­9. Hoeft, Fumiko, John D. E. Gabrieli, Susan Whitfield-­Gabrieli, Brian W. Haas, Roland Bammer, Vinod Menon, and David Spiegel. 2012. “Functional Brain Basis of Hypnotisability.” Archives of General Psychiatry 69 (10): 1064–­72. https://​doi​.org​/10​.1001​/archgenpsychiatry​.2011​.2190. Erratum 2013 in jama Psychiatry 70 (1): 97. Huizinga, Johan. (1938) 1955. Homo Ludens: A Study of the Play Element in Culture. Boston: Beacon Books. James, William. (1878) 1920. “Remarks on Spencer’s Definition of Mind as Correspondence.” In Collected Essays and Reviews, 43–­68. New York: Longmans, Green. —. 1890. The Principles of Psychology. 2 vols. New York: Holt. —. 1892. Psychology: Briefer Course. New York: Holt. —. 1896. The Will to Believe. New York: Longmans, Green. —. 1902. The Varieties of Religious Experience: A Study in Human Nature. New York: Longmans, Green. —. 1907. Pragmatism. New York: Longmans, Green. —. 1909. The Meaning of Truth. New York: Longmans, Green. —. 1912. Essays in Radical Empiricism. New York: Longmans, Green. Johnson, Mark. 2007. The Meaning of the Body. Chicago: University of Chicago Press. The Wave-Crest  95

Johnson, Mark, and Tim Rohrer. 2007. “We Are Live Creatures: Embodiment, American Pragmatism, and the Cognitive Organism.” In Body, Language, and Mind, edited by Jordan Zlatev, Tom Ziemke, Roz Frank, and René Dirven, 1:17–­54. Berlin: De Gruyter. Kihlstrom, John F. 2012. “The Domain of Hypnosis Revisited.” In Nash and Barnier 2012, 21–­52. Levin, Jonathan. 1999. The Poetics of Transition: Emerson, Pragmatism, and American Literary Modernism. Durham: Duke University Press. Massumi, Brian. 2011. Semblance and Event. Cambridge ma: mit Press. Montola, Markus, Jaakko Stenros, and Annika Waern. 2009. Pervasive Games, Theory and Design. Burlington: Kaufmann. Müller, Katharina, Katrin Bacht, Stephanie Schramm, and Rüdiger J. Seitz. 2012. “The Facilitating Effect of Clinical Hypnosis on Motor Imagery: An fmri Study.” Behavioural Brain Research 231 (1): 164–­69. Nash, Michael R., and Amanda J. Barnier, eds. 2012. The Oxford Handbook of Hypnosis Theory, Research, and Practice. Oxford: Oxford University Press. Nisi, Valentina, and Glorianna Davenport. 2011. “Hopstory: A Study in Place-­ Based, Historically Inspired Narrative.” In The Mobile Audience, Media Art and Mobile Technologies, edited by Martin Rieser, 205–­16. Amsterdam: Rodopi. Noë, Alva. 2004. Action in Perception. Cambridge ma: mit Press. —. 2009. Out of Our Heads. New York: Hill and Wang. —. 2012. Varieties of Presence. Cambridge ma: Harvard University Press. O’Regan, Kevin, and Alva Noë. 2002. “On the Brain-­Basis of Visual Consciousness: A Sensorimotor Account.” In Vision and Mind: Selected Readings in the Philosophy of Perception, edited by Alva Noë and Evan Thompson, 567–­98. Cambridge ma: mit Press. Pullinger, Kate. 2016. “Introducing the Next Generation of Ambient Literature.” Bookseller. June 30, 2016. www​.thebookseller​.com​/futurebook​/introducing​ -­­next​-­­generation​-­­ambient​-­­literature​-­­338141. Raley, Rita. 2008. “On Locative Narrative.” Genre 41 (3–­4): 125–­49. —. 2010. “Mobile Narrative, as Composed Experience.” In Beyond the Screen, Transformation of Literary Structures, Interfaces and Genres, edited by Jörgen Schäfer and Peter Gendolla, 299–­316. Bielefeld, Germany: Transcript Verlag. Reid, Joesphine, Richard Hull, Kirsten F. Cater, and Ben J. Clayton. 2004. “Riot! 1831: The Design of a Location Based Audio Drama.” In Proceedings of 2nd UK-­UbiNet Workshop, 1–­2. Cambridge University, Cambridge. http://​ citeseerx​.ist​.psu​.edu​/viewdoc​/download​?doi​=​10​.1​.1​.60​.2943​&​rep​=​rep1​&​type​ =​pdf. Rieser, Martin. 2005. “Locative Media and Spatial Narrative.” Paper presented at Refresh! First International Conference on the Histories of Media Art, 96  Whittaker

Science and Technology. Banff New Media Institute, Canada, September 28–­October 1, 2005, 1–­21, http://​pl02​.donau​-­­uni​.ac​.at​/jspui​/handle​/10002​/330. Russell, Ben. 1999. “Headmap Manifesto.” Technoccult. Accessed October 23, 2016. http://​technoccult​.net​/wp​-­­content​/uploads​/library​/headmap​ -­­manifesto​.pdf. Ryan, Marie-­Laure. 2001. Narrative as Virtual Reality. Baltimore: Johns Hopkins University Press. —. 2006. Avatars of Story. Minneapolis: University of Minnesota Press. Ryan, Marie-­Laure, Kenneth E. Foote, and Maoz Azaryahu. 2016. Narrating Space/Spatializing Narrative: Where Narrative Theory and Geography Meet. Columbus: Ohio State University Press. Ryan, Marie-­Laure, and Jan-­Noël Thon, eds. 2014. Storyworlds across Media: Toward a Media-­Conscious Narratology. Lincoln: University of Nebraska Press. Salen, Katie, and Eric Zimmerman. 2003. A Rules of Play: Game Design Fundamentals. Cambridge ma: mit Press. Shusterman, Richard. 2011. “The Pragmatist Aesthetics of William James.” British Journal of Aesthetics 51 (4): 347–­61. Simons, Daniel J., and Christopher F. Chabris. 1999. “Gorillas in Our Midst: Sustained Inattentional Blindness for Dynamic Events.” Perception 28 (9): 1059–­74. Spiegel, David. 2012. “Intelligent Design or Designed Intelligence? Hypnotizability as Neurobiological Adaptation.” In Nash and Barnier 2012, 179–­200. White, Hayden. 1981. “The Value of Narrativity in the Representation of Reality.” In On Narrative, edited by W. J. T. Mitchell, 1–­23. Chicago: University of Chicago Press. Whittaker, Emma. 1995. Loving Trees, Winter. Screen-­etched text on glass panels, 6 x 2 ft. New Work exhibition. Shelton Gallery, Stoke-­on-­Trent, UK. January 20, 1996. —. 2017. “Transitions-­Felt: William James, Locative Narrative and the Multi-­stable Field of Expanded Narrative.” PhD diss., Plymouth University. Whittaker, Emma, and James Brocklehurst. 2014. LociOscope, the Letters. ios application. Accessed November 12, 2018. https://​itunes​.apple​.com​/gb​/app​ /locioscope​-­­the​-­­letters​/id778929957​?mt​=​8. —. 2015. The Lost Index: natmus. ios application. Accessed November 12, 2018. https://​itunes​.apple​.com​/us​/app​/the​-­­lost​-­­index​-­­natmus​/id1058419473​?mt​=​8. —. 2016. Journeyer’s Guidebook. ios application. Accessed November 12, 2018. https://​itunes​.apple​.com​/gb​/app​/journeyers​-­­guidebook​/id1146283679​?mt​=​8. Wittgenstein, Ludwig. (1953) 2009. Philosophical Investigations. 4th ed. Translated by G. E. M. Anscombe, P. M. S. Hacker, and Joachim Schulte. Chichester, England: Wiley-­Blackwell. The Wave-Crest  97

4

Complexity and the Userly Text Noam Knoller

Published in print and on the New York Times Magazine’s website in mid-­August 2016, “Fractured Lands” is an ambitious long-­read journalistic story. Intended to make sense of the “broad geography” and “many causes” of “the catastrophe in the Arab world,” thirteen years after the U.S. invasion of Iraq and five years after the Arab Spring, the scale of this piece is unusual. In the words of editor in chief Jake Silverstein: “It is much longer than the typical New York Times Magazine feature story; in print, it occupies an entire issue. . . . It is unprecedented for us to focus so much energy and attention on a single story, and to ask our readers to do the same. We would not do so were we not convinced that what follows is one of the most clear-­eyed, powerful and human explanations of what has gone wrong in this region that you will ever read” (Anderson 2016). The piece is made up of text, over forty-­two-­thousand words long, that took reporter Scott Anderson eighteen months to research. It is organized into five chapters, each told through the perspectives of six recurring protagonists. The text is accompanied by ten portfolios of photographs by Paolo Pellegrin, and the website version also offers a 360-­degree video, which the New York Times, following many others, touts as a virtual-­ reality experience. The call-­out text below the video makes the following ominous promise: “Experience firsthand the battles Iraqi forces endured to retake the city of Falluja from isis.” Fortunately, despite this rhetoric, no reader was literally transported to Falluja to experience the battles firsthand. This unprecedentedly long read, and the accompanying 360-­degree video, raises a number of questions with which we can open a discussion of narrative, complexity, and interactive digital storytelling.1 How 98

are we to make sense of this editorial commitment and of the expectation of a matching commitment by the readers? Why is such a long text required? What is the “virtual-­reality experience” doing there? Let’s begin with the first question: how do we explain the unprecedented scope, commitment, and expectation? The editor mentions that something went wrong, and the explanation needs to span a broad geography and many causes. There’s a catastrophe, and it is overdetermined. We have many interacting elements—­a complex phenomenon, which nevertheless needs to be narrated. The many causes need to be woven into a “clear-­eyed, powerful and human explanation.” The questions multiply: How do you tell a story about such a complex phenomenon? Is it at all possible to narrate such complexity? What is it that properly belongs inside this phenomenon and what is outside of it? How does one “make a cut” (Hayles 1995), to select from all the environmental noise what is or isn’t part of this complex storyworld, and which are the relevant and meaningful elements and processes that constitute it? The New York Times’ approach to this problem of narrating complexity involves, in this particular case, expanded scope and breadth, as well as, to an extent, multiple perspectives (six protagonists, although only one narrator). So far, we are in Habermasian public-­sphere territory. According to this classic model (Habermas 1989), the public sphere, essential to all liberal democracies, functions when citizens have the opportunity to consider sufficient viewpoints on an issue of public concern in sufficient depth. This picture assumes a social system in which communication originates in media that represent and disseminate sufficient viewpoints and reach a citizenry of engaged and critical readers able to dedicate sufficient time and effort to consider these viewpoints in depth. Jürgen Habermas’s (1989) model may have been an adequate description of the public sphere in liberal societies mediated by broadsheet newspapers. At the present moment this vital component of democracy seems to strain under the pull of multiple crises: the media landscape has been radically changed by the appearance of ubiquitous computer-­based networked media; reading habits have changed to the point that the public’s capacity to deeply engage with any story needs to be reappraised; and, most important, the world, about which stories need to be told, has undergone a rapid and profound change, which Mark B. N. Hansen calls “hypercomplexification of the environment” (2009, 114). Complexity and the Userly Text  99

Environment and World Crisis We will return to the changes in the media landscape and in reading habits presently, but first it is useful to understand Hansen’s term. Referring specifically to human systems, Hansen writes, “The complexity of the world has undergone a double transformation sometime in the fairly recent past: first, worldly (environmental) complexity has become so intense and so messy . . . that any effort to reduce it through selection by systems . . . cannot ignore the agency that is wielded by the environment; and second, the operation of this environmental agency is now predominantly and ever increasingly technical, meaning that system function is irrevocably permeated by technicity from the environment.” The challenge for contemporary neocybernetic thinking is to “address a changed world (or environment) that quite simply resists the reduction of complexity so central to both von Foerster’s and Luhmann’s projects” (2009, 113–­14). Hansen thus proposes to develop neocybernetic thinking under the assumption that systems (i.e., humans) and environments (computational, in this particular essay, but other complexities need not be excluded) constitute system-­environment hybrids. Elsewhere, with Udi Ben-­Arie, I analyzed some of the particulars of the experiential locus of this hybridity: the human-­computer interface (Knoller and Ben-­Arie 2015). We observed several current vectors of change that characterize a post-­p c era of computing and, indeed, of human-­computer interaction: the bodily turn; the affective turn; ubiquity and cross-­device integration; prediction by attentive interfaces (all these on the technical level); and the user-­experience economy—­a vector attracting these technological changes through the agency of designers and others, in line with what Michel Foucault (1980) would have called an “urgent need.” In a similar vein to Hansen, we argued that the interface experience is having far-­reaching implications on human agency. The post-­p c era creates a situation in which we are as much controlled by technology as technology affords us control (Knoller and Ben-­Arie 2015, 62). The urgent need for organizing the design of computers and interfaces, we claimed, is the need to harness and control “user experience.” By the middle of the past decade, computers were no longer understood as tools to augment the (cognitive) producer of services (Engelbart [1962] 2001) but rather as media that stage experiences. They have become 100  Knoller

actualized Experience Machines (Nozick 1975), especially suited for the consumption of gaming and (affective) gamified experiences. Hansen (2009) elides these pressures working to subsume both human and environmental agency under the logic of the (user) “experience economy” (Pine and Gilmore 1999).2 I follow this bracketing in the rest of Hansen’s essay in exploring the more optimistic possibility of “imposing some provisional closure, some fleeting reduction of complexity, on a world, a technosphere, increasingly characterized by relentless heterogenesis” (Hansen 2009, 125). Crises in Media and Reading As I claimed earlier, the crisis in human understanding is a result not only of the hypercomplexification of the environment but also of the combination of the failure of current media to reproduce the public sphere, along with changes in human cognitive habits of media consumption, especially reading. Current digital media forms are not yet a solution, as long reads on the web, such as “Fractured Lands,” are at odds with online reading habits. Analyzing the shifts in reading strategies from print to digital (mostly online), N. Katherine Hayles writes that “learning to read complex texts (i.e., close reading)” is increasingly giving way to other forms of literacy, including analysis through machine algorithms (“machine reading”) and hyperreading (2012, 36). From mounting evidence of surveys in the United States, she concludes that “reading skills (as measured by the ability to identify themes, draw inferences, etc.) have been declining in junior high, high school, college, and even graduate schools” (2012, 123). Putting machine reading (which is not employed by the general public) aside, it is worth noting the difference between close and hyperreading: “Hyper reading, which includes skimming, scanning, fragmenting, and juxtaposing texts, is a strategic response to an information-­intensive environment, aiming to conserve attention by quickly identifying relevant information, so that only relatively few portions of a given text are actually read. . . . Close reading, by contrast, correlates with deep attention, the cognitive mode traditionally associated with the humanities that prefers a single information stream, focuses on a single cultural object for a relatively long time, and has a high tolerance for boredom” (Hayles 2012, 35–­ 6). Hayles calls for bridge building between the three different reading Complexity and the Userly Text  101

strategies (close, hyper, and machine) to develop an expanded literacy. But such an expanded literacy is not yet the case. For the time being we are facing a crisis of reading or, more broadly, of reception and reflection. “Fractured Lands” is symptomatic in this respect: the accompanying 360-­degree video signals a recognition that new ways of communication and engagement are needed, and the rhetoric framing the video betrays a desire for “firsthand” experience by the newspaper, and arguably by its audience, mediated by a technology that is (mis)understood to afford presence through a more embodied interface. There are additional aspects of new media consumption that further undermine the Habermasian ideal of a democratic public sphere: “selective exposure” (Lazarsfeld, Berelson, and Gaudet 1944) biases us to avoid exposure to noncongruent opinions and facts (Holbert, Garret, and Gleason 2010). Algorithmic effects appear to amplify this even further, creating “filter bubbles” (Pariser 2011), or a fragmentation of the public sphere, and consequently of society. In Luhmannian terms, filter bubbles disrupt the basic element of social systems: communication (Luhmann [1986] 2008). The three compounded crises outlined here: of the world, of the media, and of reading, result in a general crisis of understanding, with consequences very apparent in the volatile behavior of the political systems in certain societies, forming a substrate for the rise of populist leaders. Their simplistic stories, however cognitively deficient, nevertheless carry rhetorical force since they provide at least a semblance of understanding and a promise of action. But, as Niklas Luhmann observed early on, this action is not likely to follow an optimal path: “Faced with serious problems of understanding and apparent misunderstandings, social systems very often tend to avoid the burden of argumentation and reasoned discourse to reach consensus—­very much to the dismay of Habermas.” What happens instead, Luhmann observes, is a tendency to “embark on a course of conflict” (1986] 2008, 92). Conflict reduces complexity through the elimination of elements and relations and a retreat to a less complex homeostasis. Instead, what is needed and perhaps possible, is the development of a suitably complex embodied-­ cognitive mental model that holds together the dynamic interactions of multiple elements and incorporates flexible, complex cognitive scripts that can adequately guide action. 102  Knoller

The Need to Address Complexity The problem is that of addressing complexity, and it is not limited to the filter bubbles of leisurely online-­news readers. A 2015 report of the Wilson Center, a U.S. nonpartisan policy forum, found that lengthy pdf policy papers by think tanks are seldom downloaded, let alone read (Rajeski, Chaplin, and Olson 2015, 12). The report defines the problem of addressing complexity in the starkest terms. The authors had conducted in-­depth structured interviews with twenty-­five literary scholars, game designers, journalists, media specialists, systems analysts, and people who study policy-­making processes. Their conclusion is bleak: “We were struck by the imbalance between problems and solutions. People were quite articulate about the challenges, but few could offer solutions. Some were realistic about how long it could take for systems thinking to effectively penetrate our problem-­solving strategies—­decades, not years—­and were afraid that the planet cannot wait that long” (19). Thus, the stakes of addressing (hyper)complexity—­not merely representing or mirroring it but furthermore communicating it to both the general public and policy makers, debating and reducing it cognitively rather than through conflict—­are of a planetary order. Merely extending the scope of traditional narratives, as the New York Times attempted with “Fractured Lands,” is insufficient, since “critical stories about complex topics may not be best expressed through traditional step-­by-­ step storytelling. Complexity requires new narratives” (Rajeski, Chaplin, and Olson 2015, 10). New narratives are required, and, to reconnect with Hayles’s thread, these will also mandate a new reading strategy, perhaps a synthesis of close, hyper, and machine readings. The Narrative Scaffolding of Cognition for Complexity To facilitate “some fleeting reduction of complexity,” Hansen turns to cognitive scientist Andy Clark’s Natural Born Cyborgs (2004). Clark assumes that the human system is fundamentally flexible and uses the environment as a resource through cognitive scaffolding, which enables us “to modify our own cognitive operations through our co-­functioning with external, technical scaffolds such as the computer-­cum-­word-­processor.” This also applies to the hypercomplexified environments of today. Indeed, Clark sees environmental complexity as a “crucial source for the cogniComplexity and the Userly Text  103

tive operation and development of the human . . . as embodied brain and as element in a larger distributed cognitive system” (qtd. in Hansen 2009, 122, 124). Likewise referring to Clark, Hayles frames digital technology as “extension of one’s thoughts rather than an external device” (2012, 2–­3). Embodied and extended approaches to cognitive psychology are essential to understanding the potential of narrating complexity using cognitive cultural, technical, and computational scaffolding. Under the conditions of the post-­p c technosphere, the relationship between humans and computers becomes much more embodied and extends not only one’s thoughts (through the keyboard) but also one’s affects (through myriad interface devices). The computer is no longer primarily a cognitive tool, as in Hansen’s not-­yet-­post-­p c example of the computer-­cum–­word processor, but a representational medium that stages experientiality. The Wilson Center report proposes addressing complexity using a particular type of computer-­based artifact to scaffold cognition. The authors refer to a number of prominent scholars from game studies (they specifically cite Henry Jenkins, Katie Salen, Eric Zimmerman and Jim Gee—­but the claim is ubiquitous) to reiterate a common claim that playable models have a potential to help represent and understand complex systems, by modeling those systems and allowing users to play with them. There are several problems with this view, however. In this report the potential is mostly assumed through general explanations rather than argued for in detail. For example, the authors argue that playable models may incorporate diverse perspectives and heuristics that help overcome personal biases and “group think” (Rajeski, Chaplin, and Olson 2015, 11). A concern with this perspective is that “playable models,” rather than helping understand the complexity of a system, merely point it out. The awareness that something is complex does not entail an understanding of its complexity, and in any case this is an achievement that isn’t unique to playable media, as has been shown for complex cinema (see Knoller 2012; Poulaki 2011, 2014) and long-­form tv series (Mittell 2006). Long-­form, traditionally produced narratives, such as “Fractured Lands,” can thus already mirror complexity, whereas what we require is—­at least—­cognitive reduction. Yet the users cited in the report do not report achieving a better understanding of complex issues through playing games such as “budget hero”—­an online simulation of the U.S. budget developed by the Wilson Center—­but rath104  Knoller

er a realization of how complex the issues were (Rajeski, Chaplin, and Olson 2015, 14–­15).3 The report lacks a definition of what understanding complexity actually entails and how it might be described or even measured. Most crucially, playable simulations that do not activate narrative sense making may be comprehensible in a logical sense but run the risk of not being meaningful in personally relevant, anthropomorphic terms. As Richard Walsh warns, “in the absence of narrative, complex systems simulations lack explanatory force” (2016, 273). Modeling the Understanding of Complexity In the field of human-­computer interaction, Matthias Rauterberg has an early proposal for the modeling and measurement of task-­related cognitive complexity. His main concern is to “build up a mental representation of the system’s structure and gain knowledge about the task relevant functions of this system” (1995, 60). Rauterberg distinguishes between a “learning system” (for example, a human computer user) and its “context.” There are four complex constructs in the model: environmental complexity, bodily complexity, memory complexity, and activity complexity. Environmental complexity is input through perception and attention, where it is further affected by bodily complexity, which includes all internal stimuli that might affect mentalization (e.g., arousal, arm and leg movements, motor restlessness) and mental processes like “daydreaming” (56). The user’s internal complexity is reflected in the memory complexity—­essentially the problem-­solving schema, which processes perceptible input and then outputs activity. That complex activity modulates bodily complexity (looping back into perception and attention) and is also the response, or output, of the system or user. The complexity of response patterns, which act back on the context and environment, appears here as an inverse function of the mental complexity of the user. Skill, which increases mental complexity, reduces activity complexity (which is higher for beginners), making observable activity complexity a good (inverse) indication of mental complexity (Rauterberg 1995, 65). Rauterberg’s model suggests a link between environmental complexity and human ability to address it cognitively, through complex mental models that may be improved with skill. It is, however, a model that treats the body as an analytically separate context rather than a resource for meaning. Instead, what is needed is a model that redefines the role of Complexity and the Userly Text  105

the body as an interface for affects and intensities, a locus of additional feedback loops that are part of the narrative experience in this medium. Bodily complexity, through the sensorimotor system, can thus be understood to extend cognition. A meaningful synthesis of playable simulation and narrative is needed. Such a model can be better aligned with more current views on narrative and cognition, such as Richard Menary’s (2008) embodied (and socially extended) narratives of the self; Daniel Hutto’s (2007) situated view of phylogenetic development in his narrative practice hypothesis; and cognitive-­narratological perspectives such as those of David Herman’s (2002) or Marie-­Laure Ryan’s (e.g., 2004). Working at the interstices of narratology and complexity science, Walsh has suggested a diachronic inquiry into the development of narrative form. His thesis is that the most advanced cultural forms of narrative develop in a reflexive process that “can be seen as part of a continual struggle to transcend the limits of narrative sense-­making” (2016, 276). At the core of Walsh’s argument, premised specifically on Herman’s (2002) work in Story Logic and continuing in the vein of Jerome Bruner’s (1986) cognitive-­psychological perspective on narrative, is a perspective on narrative as cognitive practice, a capacity that can be developed phylogenetically, with respect to the diachronic development of narrative forms, as well as ontogenetically, as an individual skill that can be scaffolded by more developed forms. Herman, specifically, views narrative as a cognitive structure, a “frame,” in the cognitive-­scientific sense developed in artificial intelligence (2002, 86), thus describing narrative as furnishing a “forgiving, flexible cognitive frame for constructing, communicating, and reconstructing mentally projected worlds” (49). This description is useful to extend narratology beyond literature or any specific mediatic manifestations. But it could be open to critique that it is still brain-­bound, not accounting sufficiently for both the embodied and the public nature of narrative as a communicative activity or practice. I propose an extension of Herman’s formulation in the concluding section. The Userly Text and Its Userly Performance To understand how digital artifacts can function as complex semiotic scaffolds for narrative meaning production (by their creators) and embodied sense making (by their users), I suggest an understanding of their textuality that incorporates not just the way they organize information 106  Knoller

but also the way they structure the processes of reception. I thus regard them as userly texts, a term that I mean to connote Roland Barthes’s (1974) distinction in S/Z between readerly and writerly texts. This connotation has been suggested once before, without much development, by Daniel Chandler: “In passing, it is worth noting that the extension of Barthes’s notion to other media could be productive, involving a consideration of the extent to which engagement with such media might be regarded as userly or makerly” (1995, 6).4 Barthes’s writerly texts differ from readerly texts in that they require cognitive effort on the part of a reader. Userly texts, on the other hand, require not merely cognitive but also physical, bodily engagement, because they are both constituted and materially—­as well as semiotically—­a ltered by this engagement. Simply put, in a narratological context, a userly text is any object that can be used. Use, here, means more specifically a form of reception that proceeds through embodied-­cognitive performance. Thus, users don’t perceive merely the text’s meaningful form, nor do they construct it merely mentally; they also reconfigure the text through physical enactment. A userly text is not merely navigated by the body, as one might flip the pages of a codex or look around “within” a recorded 360-­degree video. Action and perception are combined here into an authorially structured, proceduralized process of userly performance, in which the perception of the content and point of the work is premised on the simultaneous reflexive perception of the user’s (embodied) activity. There are two constructs, phenomenologically, that the user engages with and that organize the userly text’s form: the first is the encoded storyworld and the second is the interaction model. Users play with the work and need to hermeneutically engage with both, through use or play, interactively, that is, through proceduralized participation. Encoded Storyworld The encoded storyworld (see figure 10a) is implemented as software, a database with some algorithms that together encode a diegesis—­the elements of the storyworld, including locations, characters, and events—­as well as discourse components, which include the way these facts are represented or simulated in media. An important subdistinction here is between two categories of materials: remediated assets such as segments of video, audio, and animation; and natively digital, procedurally generated Complexity and the Userly Text  107

Fig. 10. The userly text. Created by the author.

code assets that may be displayed as text, synthetic images and 2d or 3d animation, or tactile feedback patterns, depending on the display hardware available. And, finally, the encoded storyworld includes all the possible relations between these various elements—­spatial, temporal, causal, formal, ordinal—­as deemed necessary by authorial considerations. Grouping some such relations together creates a higher level of organization of diegetic materials, which I call a discursive strategy. Some discursive strategies are familiar from traditional media. In film, for example, we can organize diegetic materials according to narrative form (as in most fiction and documentary films) or list them thematically or alphabetically or according to formal-­abstract qualities such as rhythm and composition and so on (Bordwell and Thompson 1997). Thus, the encoded storyworld encodes the diegetic elements, the way these are represented or simulated, and the metadata required for these elements to be combined according to some such organizational principle. In relation to our discussion of (hyper)complexity, the encoded storyworld, being a digital artifact, possesses several relevant affordances. First, as we saw in our example of “Fractured Lands,” the narrative representa108  Knoller

tion of complex phenomena requires unprecedented scope. Janet Murray (1997) has pointed out that the digital medium has an “encyclopaedic capacity”: the ability to store vast amounts of information. Second, the organization of the encoded storyworld follows the random-­access model of computer data storage, and any further organization, as in any system building, depends on the model imposed on the data by the author of the model. Consequently, elements are encoded and require decoding to enter a semiotic relationship with a user. As has been theorized in Hartmut Koenitz’s (2015) system-­process-­product (spp) model, these relationships are kept in the system as potentials, requiring instantiation contingent on input from a user and further procedural processing, producing a variability of instantiations. This flexible variability of outputs is constitutive of the userly text and characterizes its inherently complex nature. In Koenitz’s terminology the encoded storyworld is a “protostory.” My model attempts to also account for the instantiation of different discursive strategies and for the potential of a text to shift strategy (for example, from chronological recitation to stream of consciousness) in response to patterns of user input.5 Thus, the userly text’s highly flexible, complex ability to encode multiple storyworld elements and their multiple possible relationships allows for the procedural generation of multiple variable plots and discourses, all representing the same underlying systemic model.6 Interaction Model The second construct is the interaction model (see figure 10b). It manifests as the hardware user-­input and user-­output components of the system, as well as software driving the hardware and interpreting its operation. In relation to our discussion of narrating complexity, the authoring of an interactive digital storytelling experience—­and this has largely been absent from literature on interactive narrative–­authoring systems—­entails also the authoring of the interaction model. The interface, which mediates the semiotic and affective feedback loops between the encoded storyworld and the user’s body-­mind, is very much part of the userly text. The creator of such an experience is authoring a complex system of communicative exchanges. Authoring begins with a choice of hardware platform that determines which information can be communicated between the human system (including which parts of the human body are treated as that system’s interface) and the computer system(s) and proceeds Complexity and the Userly Text  109

with structuring the software procedures that model the possibilities of this interaction, with which the creator defines and constrains the potentials of userly performance. Userly Performance The previous section describes the userly text, a form of computational storytelling system. The specific mode of reception it affords, userly performance, is a fourth type of reading strategy: a procedural, embodied cognitive activity that may have the potential to offer scaffolding for cognitively reducing complexity. The term performance appears in several discourses (from speech-­act theory to industrial, technological performance). In the context of the current discussion, the performance-­ artistic and sociological senses are most relevant. Skill, Repetition, and the Hermeneutic “Spiral” Artistic performance connotes the execution of a score or a script and it requires skill, which is developed through repetition. The skill involved in playing a video game is often understood as sensorimotor in nature, but in narrative userly texts this sense is joined by hermeneutic skill, connoting the artistic sense of performance as interpretation (which can also mature into expressive interpretation). According to Alicia Juarrero, “The interlevel tacking of the hermeneutic ‘circle’ reproduces the self-­ organization of complex dynamical processes. By showing the dynamics of complex adaptive systems, hermeneutical narratives are uniquely suited as the logic of explanation of these strange-­loop phenomena” (1999, 223). Narrative userly texts are structured to encourage a hermeneutic “spiral,” making an interpretive, repeated performance a necessary part of reception, as each run of the system will likely produce a new plot and a new reading. The process of narrative sense making thus scaffolded by the userly text is a tacking, driven by engagement and interest, between the parts—­individual performances and the specific plots they instantiate—­and the text as a whole, which is the dynamic system that underlies it, including the interaction model and the way it structures userly performances. Aspects of this view can already be found in hypertext theory of the late 1990s. In her reading of Michael Joyce’s seminal hypertext fiction work “Afternoon: A Story,” Jill Walker writes about the difference between the interpretive performance of codex reading, 110  Knoller

as described by Wolfgang Iser, and the reading of hypertext literature (a relatively simple early form of userly text) in which the hermeneutic process of interpretation becomes obvious (1999, 112). We can recall here Walsh’s (2016) thesis about the ontogenetic development of narrative sense making and join these together into a view of userly performance as a sense-­making activity, a performantial (Ryan 2004), epistemic, embodied-­cognitive skill that can be cultivated through repetition. A relatively simple recent example of a text affording repeated performance is Breaking Points (Koenitz, Sezen, and Sezen 2013), a visual interactive narrative for the ios platform. The user enacts and reenacts the daily routine in which a young woman feels trapped and from which she would like to escape. The interaction model consists of menus of conversation choices and hot spots. Every repeated performance can differ from the previous one, depending on specific choices the user makes on behalf of the hero, some of which seem trivial, such as where to start with her makeup in the morning; others are likely to produce the expectation of greater significance, such as whether to accept an invitation to go on a date with a colleague. Each playthrough thus produces a different, complete plot through the narrative space of possibilities, differing in interim as well as final outcomes. As a userly text, Breaking Points affords repetition or replay, and interpretation, foregrounding playing not for the plot but for a higher-­order systemic understanding of the complex interrelations between different userly performances and their variable outcomes. The design of the system is such that some of the outcomes are immediately apparent, while others are delayed: the impact of some decisions and choices becomes significant only retrospectively, further encouraging the player to replay and try a different strategy. In this manner a userly text scaffolds a hermeneutic spiral, where attention shifts between several levels of meaning: particular choices and their bifurcating outcomes, one protostory and its instantiated outcome, and subsequently the totality of possibilities of the constituent plot and its performance. Attention also shifts between a self-­reflective attitude to one’s own performance in constructing scripts, and the storyworld within and on which these scripts take effect. Multimodal Embodiment Brenda Laurel, whose theatrical metaphor for human-­computer interaction has been foundational, reminds us of another sense of perforComplexity and the Userly Text  111

mance, invoking the Aristotelian distinction between the visual spectacle and the multisensory performance (1991, 59). The fact that, in the post-­p c era, the previously mentioned bodily and affective turns point userly texts toward greater multimodal embodiment is another motivation for my use of the term userly performance. Writing about self-­narrative and the embodied agent, Menary objects to the notion that we might normally use narratives implicitly as mental scripts to follow. Using the example of driving, he claims, “I enact the skills without thinking about them, the fluid and flexible sequence . . . is open ended and not easily captured as a narrative sequence” (2008, 70). In terms of Rauterberg’s model, a skilled driver’s mental complexity reduces the bodily complexity required to perform the sequence of actions. Such scripts become appropriated and are no longer narrative when they are not consciously narrated to anyone, including to one’s self. That is, however, precisely where a userly text can function as a positive disruptor: one may sometimes have to rethink, even unlearn, certain behaviors that are automated, in a self-­reflexive, narrativized way, which links such automated scripts to causes, consequences, and a thematic context of personal relevance. Skilled behavior could thus be deskilled and made accessible anew to narrativizing—­a Verfremdungseffekt, as Walker correctly observed for hypertext, applied to hitherto habituated performance. Conversely, a narrative could be used to frame the skilling of behavior and habituate it, as is common practice in video games and other simulations that aim to train users in specific performances. Indeed, Menary concedes (crediting Jordan Zlatev for pointing this out) the occasional need for “conscious linguistic interventions . . . to break the stream of embodied experience and skilled behavior” to “remind ourselves of what we should be doing, or how we could do things differently and there looks to be a role for narratives here” (2008, 70). I take this to also mean a distinct role for embodied userly performance of the type afforded by userly texts, especially if the “linguistic” can extend to a multimodal semiotic and affective understanding of embodied performance. We can now see how bodily complexity can serve as a locus of additional feedback loops that are part of the narrative experience. The linear relation Rauterberg observed between behavioral complexity and memory complexity is likely more complex. When interfaces are not merely trans112  Knoller

parent and are allowed to reflect userly performance and create a hermeneutic loop, patterns of interpretive and expressive embodied performance beyond mere skill might appear as more, rather than less, complex, despite being a manifestation of the human system’s mastery over complexity. Nevermind, a biofeedback-­enhanced adventure horror video game (Reynolds 2015), employs this hermeneutic spiraling for therapeutic purposes. Occupying an excluded middle position within the dichotomy suggested by Ryan between “full-­body interaction” and “manipulation of a control pad or keyboard” (2009, 49), the game uses a standard webcam and a heart-­rate sensor to pick up on indications of fear and adapts the difficulty of tasks within the game to those indications in an attempt to teach players to become more aware of their internal responses to stressful situations and learn to control their anxiety levels, hoping that in-­ game coping scripts can be applied to real-­world situations (Hillegas, n.d.). Apart from changing the difficulty of the game in response to userly performance, the game also adjusts its display by filling the screen with static to feed back the player’s affective state (Reynolds 2016), thus potentially creating a loop that dishabituates and rehabituates userly performance, replacing behavioral scripts in the context of a projected storyworld. Here self-­reflective attention is focused on the embodied fodder of prenarrative affects rather than any propositionally articulable linguistic narrative content, and yet narrative action takes place, by extended cognition, within a mentally projected storyworld, scaffolded by the material affordances of a video game. Audience and Agency Nevermind’s powerful embodied interaction model also points toward the importance in the interface experience of the intersection of embodiment and extended sociality. Erving Goffman’s distinction in The Performance of Self in Everyday Life (1959), between signs that are given intentionally and signs that are given off subconsciously, becomes particularly relevant when all of these signs emitted by the user at the interface can (and will) be used as input by a networked system. Communicative userly performance may not all be intentional and may, in fact, lack agency. This captures something that the dominant discourse on agency in human-­computer interaction, and specifically in the field of interactive digital narrative, misses: there’s no reason to take it for granted that computers accept as input Complexity and the Userly Text  113

only what users intend them to.7 In the post-­p c era, especially, the subintentional aspects of userly performances are valid input too.8 Furthermore, userly performance—­intentional, unintentional, or subintentional—­is not just a performance of (a score or script) but also a performance for an audience, whether that audience is the self, the computer (as in the case of Nevermind), or—­through the mediating computer—­other computers, humans, corporations, or governments. Laurel’s theater of the computer, mentioned earlier, still lacked an account of an audience—­an omission symptomatic of a more general myopia in early human-­computer interaction scholarship, when it typically occurred offline. In the current post-­p c context, and in particular in relation to the vector of predictive and attentive interfaces, these audiences cannot go unaccounted for. I am not aware of a critical interactive narrative that addresses both embodiment and signs that are given off. Do Not Track (Gaylor 2015), an interactive documentary, does make reflexive use of information given off by the viewer’s browser (ip address and location). As it intercuts relevant images in real time from online sources, the adaptive narration declares, “I know that this is the country where you live. I know it’s a nice night. I know you’re on a pc,” exposing both the limited agency of online userly performance and the often invisible—­in this case algorithmic—­ audiences of that performance.9 Narrative Userly Texts Might Address (Hyper)Complexity Cognitive scripts guide our understanding of events and therefore also our planning and performance in all situations. In particular, rich and flexible cognitive scripts are important for our ability to adequately respond to new situations, including complex situations, as skilled improvisational performers in theater or in music do. To extend Herman’s description of the narrative frame (2002, 49), userly performance of an interactive narrative is a communicative process of repeated instantiation and interpretive, expressive, embodied performance, which furnishes both a cognitive narrative frame and cognitive scripts for possible and probable performances in mentally projected worlds. Done well, the phylogenetic development of narrative form that is the userly text, and the ontogenetic development of individual embodied and hermeneutic narrative skills in the form of cultivated userly perfor114  Knoller

mance, can spiral like a double helix to cognitively reduce hypercomplexification. The variable repetition constitutive of userly performance can both change the selection and feed the change back to the user so that the contingency of a specific outcome on a specific aspect of userly performance (whether intentional, agential, or otherwise) can be integrated into an ongoing, developing internal model. Such a model can furthermore relate the systemic network of relations in the encoded storyworld to cognitive performance scripts. Thus, as every performance instantiates a discourse and a plot, the resulting variability gradually reveals the underlying dynamics of the system that produced it, of the encoded storyworld and the interaction model. In receiving a userly text through userly performance, it becomes possible to hold together several sequential logics and reflect on their interrelationships, and even more so when these all relate back to the userly performances that enact them. The performing user can experience the implied causality of narrative as contingent on various userly performances within a narrative interactive system, in which the user enacts chains of events as a cause. The consequences of such performances can ripple through networks of interconnected characters and events and differ between variable repetitions so that various plots (causal chains) emerge. These plots are themselves interconnected, even when they would be incompossible in the told diegesis of a noninteractive narrative system such as a book or film. Different userly performances, thus, can elicit a more complex “structure that can embrace contradictory emotional and perhaps factual outcomes” (Bolter 2001, 126). A userly text can hold a greater number of plot elements than the most complex film, television, or book series. It can resolve into a greater number of alternative organizations. Multiple endings are possible and even likely, but, unlike in a linear narrative, no single ending determines the meaning of the text. The meaning of a userly text resides not in a single performance of a single discourse and plot combination but in the systemic comprehension, not only of the encoded storyworld, but rather of the reflexive relation between, on the one hand, variable complementary and contradictory discourses and plots and, on the other hand, the variability of userly performances as structured by the interaction model. When, delayed as they may be, outcomes can systematically be related back in the subject’s internal model to certain intentional actions, choicComplexity and the Userly Text  115

es, or decisions, the feeling of narrative agency can be established. When they can be systematically traced back to aspects of userly performance that were not previously perceived as causative of outcomes (aspects that were unconscious, marginal to attention, or otherwise on the outside or periphery of intentional action), a self-­reflexive sense of userly performance itself emerges—­as in a text such as Nevermind. Finally, a userly text is not able to merely represent or mirror complexity. It adds to the representation an ethical dimension, enhanced not only by the augmented experience of presence typical of 360-­degree videos such as the one that accompanied “Fractured Lands” (an immersion whose effects may wear off once habituation to the sensory augmentation kicks in) but rather by the ability to cultivate courses of action across all levels of embodied cognition, from high-­level goals to fine-­motor activity. Userly texts can offer us an extension of cognitive complexity with embodied scenarios that can be useful in predictively processing complex, environmentally situated events (in all complex domains). Sense making proceeds procedurally, through userly performance, enacting multiple related scenarios and plots, resulting in a dynamic embodied-­mental model capable of holding together contingencies and action possibilities. This model relates a signified hypercomplexified reality of system-­ environment hybrids through the mechanisms of a signifying userly text to knowledge about performing within and acting on that reality, explicitly exploring the ethical agency as well as the experientiality of userly performance. Notes Thanks go to Hartmut Koenitz and to the editors for their helpful comments. 1. A person reading between 200 and 250 words a minute would require three to four hours to get through the story. 2. The appearance in postclassical narratology of a conceptual constellation around narrative experience and experientiality suggests dispositival entanglement, which requires further interrogation. 3. Furthermore, the report seems, in its final section, to adhere to the view that, if applied to journalism, playable models can be “objective” and less “authoritative,” “letting readers draw their own conclusions based on the facts” (Rajeski, Chaplin, and Olson 2015, 18), more so, supposedly, than linear narratives. This is fallacious. As I’ve argued in Knoller (2010) and as 116  Knoller

has been pointed out already by Hayles (1995), the act of modeling a system and determining ways to interact with it cannot somehow be liberated from authorial “bias.” It is wrong to ignore the authoriality of the domain of the observer. In Knoller (2010) I also address the fallacy of elevating the user to the position of an author of an interactive text. The user can be an author either in a trivial sense when using a word processor or other types of authoring software or in a very limited sense, within parameters already defined by the author(s) of a narrative userly text. 4. I’m grateful to Marina Grishakova for pointing Chandler’s (1995) earlier use out to me. 5. For a description of the concrete storytelling systems developed in relation to this model, see Dekel and others (2003) and Knoller (2004). 6. For an in-­depth discussion of plot and discourse variability in interactive digital storytelling, see, for example, Ryan (2004). 7. For critiques of the agency discourse in the aesthetics of interactive digital storytelling, see Knoller (2010) and Harrell and Zhu (2009). 8. Pia Tikka’s (2008) enactive-­cinema approach demonstrates this aspect too, as her system allows a film to adapt its real-­time montage to the viewer’s physiological output, without requiring any intentional userly performance: it’s an interaction model based on interactivity without agency, an implicit interactivity. 9. For an extended analysis of Do Not Track see Koenitz and Knoller (2017). References Anderson, Scott. 2016. “Fractured Lands: How The Arab World Came Apart.” Edited by Jake Silverstein. New York Times Magazine, August 11, 2016. www​ .nytimes​.com​/interactive​/2016​/08​/11​/magazine​/isis​-­­middle​-­­east​-­­arab​-­­spring​ -­­fractured​-­­lands​.html?​_r​=​0. Barthes, Roland. 1974. S/Z. New York: Hill and Wang. Bolter, Jay David. 2001. Writing Space: Computers, Hypertext, and the Remediation of Print. 2nd ed. Mahwah nj: Erlbaum. Bordwell, David, and Kristine Thompson. 1997. Film Art: An Introduction. 5th ed. New York: McGraw-­Hill. Bruner, Jerome. 1986. Actual Minds, Possible Worlds. Cambridge ma: Harvard University Press. Chandler, Daniel. 1995. The Act of Writing: A Media Theory Approach. Aberystwyth: University of Wales. Clark, Andy. 2004. Natural Born Cyborgs: Minds, Technologies, and the Future of Human Intelligence. New York: Oxford University Press. Complexity and the Userly Text  117

Dekel, Amnon, Noam Knoller, Udi Ben Arie, Maya Lotan, and Mirit Tal. 2003. “One Measure of Happiness: A Dynamically Updated Interactive Video Narrative System Using Gestures.” In Human-­Computer Interaction, edited by Matthias Rauterberg, Marino Menozzi, and Janet Wesson, 1011–­12. interact’03: ifip tc13 International Conference on Human-­Computer Interaction, September 1–­5, 2003. Zurich: ios Press. Engelbart, Douglas. (1962) 2001. “Augmenting Human Intellect: A Conceptual Framework (1962).” In Multimedia: From Wagner to Virtual Reality, edited by Randall Packer and Ken Jordan, 64–­90. New York: Norton. Foucault, Michel. 1980. “The Confessions of the Flesh.” In Power/Knowledge: Selected Interviews and Other Writings, edited by Colin Gordon, 194–­228. New York: Pantheon. Gaylor, Brett, dir. 2015. Do Not Track. Paris: Upian. https://​donottrack​-­­doc​.com. Goffman, Erving. 1959. The Performance of Self in Everyday Life. New York: Anchor Books. Habermas, Jürgen. 1989. The Structural Transformation of the Public Sphere. Cambridge ma: Polity/mit Press. Hansen, Mark B. N. 2009. “System-­Environment Hybrids.” In Emergence and Embodiment: New Essays on Second-­Order Systems Theory, edited by Bruce Clarke and Mark B. N. Hansen, 113–­42. Durham: Duke University Press. Harrell, D. Fox, and Jichen Zhu. 2009. “Agency Play: Dimensions of Agency for Interactive Narrative Design.” aaai Spring Symposium: Intelligent Narrative Technologies II, 44–­52. Hayles, N. Katherine. 1995. “Making the Cut: The Interplay of Narrative and System, or What Systems Theory Can’t See.” Cultural Critique 30:71–­100. —. 2012. How We Think: Digital Media and Contemporary Technogenesis. Chicago: University of Chicago Press. Herman, David. 2002. Story Logic: Problems and Possibilities of Narrative. Lincoln: University of Nebraska Press. Hillegas, Greg. n.d. “Press Kit.” Flying Mollusk. Accessed December 28, 2016. www​.flyingmollusk​.com​/press​/sheet​.php​?p​=​nevermind​#description. Holbert, Lance R., Kelly R. Garret, and Laurel S. Gleason. 2010. “A New Era of Minimal Effects? A Response to Bennett and Iyengar.” Journal of Communication 60 (1): 15–­34. Hutto, Daniel D. 2007. “The Narrative Practice Hypothesis: Origins and Applications of Folk Psychology.” Royal Institute of Philosophy Supplement 60:43–­68. 118  Knoller

Juarrero, Alicia. 1999. Dynamics in Action: Intentional Behavior as a Complex System. Cambridge ma: mit Press. Knoller, Noam. 2004. “InterFace Portraits: Communicative-­Expressive Interaction with a Character’s Mind.” In Proceedings of the 1st acm Workshop on Story Representation, Mechanism and Context, edited by Barbara Barry and Kevin M. Brooks, 63–­66. New York: Association of Computer Machinery. —. 2010. “Agency and the Art of Interactive Digital Storytelling.” In Joint International Conference on Interactive Digital Storytelling, edited by Ruth Aylett, 264–­67. Berlin: Springer. —. 2012. “The Expressive Space of ids-­as-­Art.” In International Conference on Interactive Digital Storytelling, edited by David Oyarzun, Federico Peinado, R. Michael Young, Ane Elizalde, and Gonzalo Méndez, 30–­41. Berlin: Springer. Knoller, Noam, and Udi Ben-­Arie. 2015. “The Holodeck Is All Around Us: Interface Dispositifs in Interactive Digital Storytelling.” In Koenitz et al. 2015, 51–­66. Koenitz, Hartmut. 2015. “Towards a Specific Theory of Interactive Digital Narrative.” In Koenitz et al. 2015, 91–­105. Koenitz, Hartmut, Gabriele Ferri, Mads Haahr, Diğdem Sezen, and Tonguç İbrahim Sezen, eds. 2015. Interactive Digital Narrative: History, Theory and Practice. New York: Routledge. Koenitz, Hartmut, and Noam Knoller. 2017. “Interactive Digital Narratives for itv and Online Video.” In Handbook of Digital Games and Entertainment Technologies, edited by Ryohei Nakatsu, Matthias Rauterberg, and Paolo Ciancarini, 1097–­126. Singapore: Springer. Koenitz, Hartmut, Tonguc Ibrahim Sezen, and Digdem Sezen. 2013. “Breaking Points: A Continuously Developing Interactive Digital Narrative.” In 6th International Conference on Interactive Storytelling icids 2013, Proceedings, edited by Hartmut Koenitz, Tonguç Ibrahim Sezen, Gabriele Ferry, Mads Haahr, and Digdem Sezen, 107–­13. Istanbul, November 6–­9, 2013. Switzerland: Springer International. Laurel, Brenda. 1991. Computers as Theatre. Boston: Wesley. Lazarsfeld, Paul Felix, Bernard Berelson, and Hazel Gaudet. 1944. The People’s Choice: How the Voter Makes Up His Mind in a Presidential Campaign. New York: Columbia University Press. Luhmann, Niklas. (1986) 2008. “The Autopoiesis of Social Systems.” Journal of Sociocyberntics 6:84–­95. Menary, Richard. 2008. “Embodied Narratives.” Journal of Consciousness Studies 15 (6): 63–­84. Mittell, Jason. 2006. “Narrative Complexity in Contemporary American Television.” Velvet Light Trap 58 (1): 29–­40. Complexity and the Userly Text  119

Murray, Janet H. 1997. Hamlet on the Holodeck: The Future of Narrative in Cyberspace. Cambridge ma: mit Press. Nozick, Robert. 1975. “The Experience Machine.” In Anarchy, State and Utopia, edited by Robert Nozick, 44–­47. New York: Basic Books. Pariser, Eli. 2011. The Filter Bubble: What the Internet Is Hiding from You. New York: Penguin. Pine, B. Joseph, and James H. Gilmore. 1999. Welcome to the Experience Economy: Work Is Theater and Every Business a Stage. Cambridge ma: Harvard Business Press. Poulaki, Maria. 2011. “Before or Beyond Narrative: Towards a Complex Systems Theory of Contemporary Films.” PhD diss., University of Amsterdam. —. 2014. “Network Films and Complex Causality.” Screen 55 (3): 379–­93. Rajeski, David, Heather Chaplin, and Robert Olson. 2015. Addressing Complexity with Playable Models. Paper, Science and Technology Innovation Program. Washington dc: Wilson Center. Rauterberg, Matthias. 1995. “About a Framework for Information and Information Processing of Learning Systems.” In Information System Concepts, edited by Eckhard D. Falkenberg, Wolfgang Hesse, and Antoni Olivé, 54–­69. New York: Springer. Reynolds, Erin, dir. 2015. Nevermind. Glendale ca: Flying Mollusk. —. 2016. “Nevermind.” In icids Art Exhibition: 9th International Conference on Interactive Digital Storytelling, edited by Ido Iurgel, Jörg Petri, and Andrew Gordon, 48–­49. Los Angeles: Rhine-­Waal University of Applied Sciences and University of Southern California. Ryan, Marie-­Laure. 2004. “Multivariate Narrative.” In A Companion to Digital Humanities, edited by Susan Schreibman, Ray Siemens, and John Unsworth. Malden ma: Blackwell. —. 2009. “From Narrative Games to Playable Stories: Toward a Poetics of Interactive Narrative.” Storyworlds: A Journal of Narrative Studies 1:43–­59. Tikka, Pia. 2008. Enactive Cinema: Simulatorium Eisensteinense. Helsinki: University of Art and Design. Walker, Jill. 1999. “Piecing Together and Tearing Apart: Finding the Story in Afternoon.” In Proceedings of the Tenth acm Conference on Hypertext and Hypermedia: Returning to our Diverse Roots, 111–­17. New York: acm. Walsh, Richard. 2016. “Narrative Theory at the Limit.” In Theory Matters: The Place of Theory in Literary and Cultural Studies Today, edited by Martin Middeke and Christoph Reinfandt, 265–­79. London: Palgrave-­Macmillan. 120  Knoller

5

The Complexity of Informative Autobiographies Ulrik Ekman

In the network societies in the Western Hemisphere, now into their second phase in the first two decades of the twenty-­first century through an extension and intensification of their network logic, it is easy to recognize a massive increase in autobiographical production, dissemination, and consumption. In the broad sense of a self-­organizing, ongoing experience and enactment of “life writing” that articulates a living present, leaves traces, and opens futures in the face of contingencies and complexity, autobiography was always socially and individually pervasive. Current everyday cultures, however, demonstrate a huge increase in the autobiographical kinds of expressivity, modes of mediation, types of networked feedback loops, and preoccupation with a self-­affectation of self-­affectation (as with selfies). In this chapter I am in a heuristic search for a new concept—­or, more precisely, the chapter is in the process of generating such a concept. My initial assumption is that autobiography must be outlined and defined in relation to an expanded field, for historical and theoretical developments imply potential changes of an entire range of key aspects of autobiography: its material forms and their movements, its potential order and organization, its spaces and environments, its events and temporalizations, its technologies and mediations, and its data and sense, as well as its narrating agents. Proceeding to write out the new concept for autobiography, I assume that the initial functional move would be defining it as a self-­organizing, ongoing experience and enactment of “life writing.” I assume that autobiography involves the traces of experience integrated and exteriorized in the project of an open, living system and its dynamic mediatory and technical relations with the environment. I assume that autobiography is in ongoing production, blending multiple 121

interactions of material, technological, and human narrating agents. I also assume that autobiography refers to a situated and eventual system, a whole in the process of emergence, hence something that includes the emergent enactment of synchronic, diachronic, and crosscutting complexities. I hope the remarks here outline what is at stake in making these assumptions and the reasons why they may help to revise existing narratological projects. Everyday autobiographical production of narrative effects today coexists with but also departs from the heritage of classical and modern autobiographies of the written and printed book, from Saint Augustine through Michel de Montaigne and Jean-­Jacques Rousseau to Gertrude Stein, Samuel Beckett, Vladimir Nabokov, and Roland Barthes, which has previously preoccupied scholars.1 Already in the 1990s, it reached beyond the grasp of the classic late modern narratology from 1966 onward aimed at delivering a systematic study across its thematics and modalities.2 Autobiography qua contemporary everyday life writing in network societies is thus to be approached primarily as postclassical, open-­ended, and ongoing (Herman 1999; Nünning 2003). Today most autobiographical projects unfold in an expanded everyday technocultural field of cognition that includes, but also extends beyond, high culture, fine literary writing, accounts of the life story of a significant individual, and the mediation of the printed book. Existing studies on autobiography as an everyday cultural practice attest to this (Campbell and Harbord 2002; Ochs and Capps 2001). A contemporary autobiography will most likely exacerbate earlier narratological problems with respect to defining the autobiographical narrator, story, plot, work, and genre. It is, however, also very likely to problematize traditional blindness to the complexities of mediation, technics, and information. Such blindness is still at stake if narrative studies hold on to the centuries of naturalization to print tradition. Technics, mediation, and the materialities of information are still issues calling for further development, although media blindness is on its way to being replaced by media sensitivity (Ryan 2004, 2006; Grishakova and Ryan 2010; Page 2012; Page and Thomas 2011; Ryan and Thon 2014; Thon 2015; Verstraten 2009, among others). An ideal typological case study might begin to indicate the technical, mediatory, and informative complexity at stake as a challenge. Even a partial description of likely components in the everyday cultural au122  Ekman

tobiographical practice for young to middle-­aged working adults with a graduate education in some part of the Western world suggests the challenge for postclassical narratology. On a given day they could memorize a set of little activities, events, and places they continue to care about; write half a page in a journal with their pen; post quite a number of things on their Facebook wall along with “likes” in various other places; take a set of pictures with their smartphone (selfies included) and print some of them while uploading and manipulating with filters several others on social and mobile media platforms such as Instagram and Snapchat; read and write a series of tweets on their Twitter account; revise their LinkedIn profile; listen and respond to voice messages and leave a couple of their own with friends; send and receive two dozen texts with plenty of abbreviations and emoticons; record a couple of memos and to-­do lists on their smartphone; spend two hours on their personal blog; check their email account and write five to eight messages concerning work matters and activities with family and friends; swipe their credit card in a handful of physical and online supermarkets, stores, cafés, and restaurants; return three books to the library and reserve a handful of others online while also downloading a small set of e-­materials of interest; buy and download two e-­books to their Kindle and iPad; and get tracked and perhaps recognized by a dozen surveillance systems and their databases at work, in their hometown, or on their way through stations and the airport. A contemporary autobiographical project is necessarily incomplete, operating with a differentiated, delayed, and distributed I on the move along with an environment. Emergent and temporally and relationally open, it develops while facing a complex and uncertain future in the living present. It involves complex dynamics of remembering and forgetting, conscious and subconscious neurocognitive restructuration (Brockmeier 2015) and a variety of mediations and technical exteriorizations of life-­ story bits. The contemporary autobiographical process forms a relational bundle, a dynamic assemblage, a palimpsest of inner and outer traces of the complexity of information and noise, some of it becoming a meaningful narrative. It is not obvious that current narratology can address this type of autobiographical project, although good resources have been accumulating. Both the processes of globalization and a range of efforts in an incrediInformative Autobiographies  123

bly varied politics of identity during the past three decades have paved the way for narratologies in the expanded field. Valuable resources exist that bear witness to a cultural anthropological widening of narratology, permitting an approach to autobiography as a continuously varying set of embodied everyday cultural practices and experiential performances of the anthropos (Fludernik 1996; Bruner 1986; Herman 2013).3 In addition, transdisciplinary contributions to the “narrative turn” as well as significant developments in the cognitive sciences have led narratologies to emphasize narrative as a primary cognitive mode for human beings, which organizes their world and time while framing experience in meaningful ways. In recent varieties that go beyond the cognitivist mind-­ narrative nexus and quite disembodied mental representations, they offer approaches to autobiography as a narrative extremely well tuned in gauging the felt quality of lived experience (Fludernik and Alber 2010).4 This would include an approach to contextual and situated cognition of an embodied kind—­considering the elements of the social, biological, technological, and material environments that partake in constituting and shaping the autobiographically cognizing mind (Herman 2011, 2015).5 The spread of the web, the growing import of new media, and the efforts in digital narratology and the digital humanities have led to growing recognition that narrative is far from media neutral, that existing language-­oriented narratology has a limited reach, that transmedial and intermedial approaches are needed, and that the embodiment of the media and technics at play matters for the meanings that can be produced in narratives. Such recent studies make it possible to approach autobiography as relating constitutively to printed text, photography, film, sound, hypertext, blogs, social media, mobile media, and locative media (Stiegler 2015; Livingstone 2012; Rettberg 2014; Schleser 2014).6 All this seems to indicate that a solid array of relevant resources exists for the study of narrative, including the kind of study of autobiography at the everyday plane of human culture that would be sensitive to both situated and embodied cognition and the role of the kinds of media and mediations at play in contemporaneity. Unfortunately, however, this is a partially misleading assessment. These new approaches are still incomplete and in need of further development, which means that they tend to represent minor positions in the research community. Second, the complexity of contemporary autobiographical projects is in several important 124  Ekman

respects still not met as a challenge. Notably, this challenge concerns engaging with the complex conditions of autobiographical projects in the context and epoch of co-­development of networked, post-­traditional experience societies (Schulze 1992) and ubiquitous computing as the third wave of information technology with big data and big databases as its “ontology” (Kitchin 2014; Schneier 2015). Billions of individuals carry out autobiographical projects in networked experience societies, each person increasingly feeling the uncertainty and complexity of attempts at spontaneous life storytelling. They increasingly feel called on to make their lives an aesthetic project, thus encountering the uncertainty and complexity of creating their own beautiful and meaningful life stories in nonnormative, free, and highly variable fashions. The intensified development of network logic in ubiquitous computing generates and maintains an artificial or second-­order information-­rich environment in which more and more autobiographical projects tend to unfold, overt social and mobile media usage being the easiest to point out and the infrastructural and embedded systemic handling of life data being more difficult. Autobiographies developing with ubiquitous computing unfold in interaction with near real-­time big-­data generation, including the data from the operations of profiling systems and myriads of networks of sensors and actuators. The big data and their relations and organization in databases are latent in the history of the present, a potential to be actualized in individual narrative projects, but, as databases and as complexities of data, they are at the same time in tension with narrativity. Big data thus pose important questions to narratology and narrative studies of autobiography: What are the relations between databases and narrative? Can one make narrative autobiographical art of databases? Interesting new narratological footholds could be found by considering both the technical side of databases and the way in which scholars in the digital humanities such as Katherine Hayles (2012) and Lev Manovich and colleagues (2003) have tried to work with the relation of databases and narrative (see also Date 2004). Increasingly, current autobiographical projects are simultaneously humanly cognitive life writing and technically cognitive data posting. From a human perspective, such autobiographies are simultaneously active and passive, intentional and unintended, conscious and prior to conscious awareness. Narratology should investigate the ways Informative Autobiographies  125

in which this is so, noting in each case the relative weighting of the respective sides in these conceptual pairs. Insofar as narrative studies do not meet the challenge of the generative intertwining of experience societies and ubiquitous computing as two key conditions, previously productive ways for narratology to perform historical and theoretical reductions of complexity are becoming less functional. More and more frequently current developments lead both to the reopening of earlier problems and to a pressing need to meet their complexity differently. In what follows I try to map out in more detail what is at stake in this development, indicating along the way both the problems and potential ways for narratology to move toward integration of the complexity of contemporary autobiographies. Since such integration is very difficult, if not impossible, and very wide-­ranging, it has important implications for the concept of autobiography. Historically, the more recent autobiographical developments are not the least concerned with the production of emergent social and individual life stories. Emergence as a narratological problem, particularly in autobiographies, has only recently become a center of attention. Cognitive narratology could thus be revised and strengthened by developing both existing narratological contributions and theoretical work on the notion of emergence (Abbott 2008; Bedau and Humphreys 2008; Clarke and Hansen 2009; Juarrero 1999; Protevi 2006; Ryan 2006; Walsh 2011).7 The emergence at stake, however, is intimately connected with recent developments in media and technology. Autobiographical projects since the late 1990s include not only the relatively high-­level medial concerns that have been, to some extent, addressed in narrative studies. They involve not only new media, social media, mobile media, and locative media. They include as well the complexity of the implementations of the third wave of information technologies and the ways in which individuals and social groups now live with this. Autobiographies develop along with a massive expansion and intensification of the paradigm of decentralized, distributed, and dynamically connective networks of computational entities. For narratology it makes a difference that autobiographies and ubiquitous computing coevolve and mutually specify each other (Ekman 2013; Ekman et al. 2016).8 From the early 1990s onward work on interactive and electronic narrative, cyberfiction, and gaming and narrative, and subsequently on so126  Ekman

cial, mobile, and locative media, has made a first set of valuable inroads on the role of technology and information in narrative production. This work is only partially completed, however, and it does not engage with some of the most significant developments after 2000. Here one can observe the historical tendency in narratology to delay and displace an engagement with media and technology. Narratology, especially in some of its literary humanities quarters, is relatively late to be informed by science and technology developments and to inquire into its constitutive and deconstitutive relation to media and technics as a systemic and environmental membrane. Currently, narratology seems to delay and defer being informed by the effects and implications of the developments of ubiquitous computing, pervasive computing, ambient intelligence, and the Internet of Things. Therefore, narratology would benefit from being revised and strengthened by a new set of studies informed by the complexity of living and individuating with ubiquitous computational technics as well as by the complexity of the technics of life-­writing characteristics of the epoch seeing network societies 2.0 emerge. Specifically, such a revision can take off from the assumption that autobiographical projects today increasingly emerge with the third epoch of computing (after the mainframe and the personal computer). This would permit narratology to acknowledge that an autobiography tends toward mediations in which computing is involved, now less as distinctly recognizable units and more as mediatory processes and technical devices pervasively integrated with physical and information-­intensive environments. Very likely only the tip of this ict iceberg is above the surface of conscious human awareness. The major part is infrastructurally and invisibly embedded. Autobiographies are individual and sociocultural projects that contribute to the specification of ubiquitous computing. But they are also in part defined and delimited as a technical thrust to integrate and embed computing pervasively, to have information processing thoroughly integrated with or embedded into everyday intentions, practices, sensations, and objects, including parts of human bodies. Living with ubiquitous computing implies the production of life writing that only marginally and momentarily engages consciously with a single device or application for some definite autobiographical purpose. More and more, it concerns engaging with, leaving traces with, and figuring out things with multiple computational devices and sysInformative Autobiographies  127

tems simultaneously during more or less ordinary activities, without users necessarily being aware of doing so. The autobiographical projects pursued along with ubiquitous computing involve storage, operationality, environmental feedbacks, and feed-­forwards through something like Mark Weiser’s (1991) vision of a myriad of small, inexpensive, robust, networked information-­processing devices. Perhaps these devices are mobile, but they are certainly distributed at all scales throughout everyday life and culture, most often turned toward commonsensical and commonplace ends. Narrative studies of autobiographies would be therefore more capable if they could account for the entwinement with the complexity of the ongoing generation and processing of big personal, social, and environmental data through infrastructures of networks, devices, software, and relational databases. Autobiographies are increasingly exteriorized and distributed in decentralized ways in an expanded computational field. Consequently, narratology should revise privileges accorded to the intentional and conscious narrator. The autobiographical I’s, both the narrators and the biological individuals involved, bear less and less resemblance to modern liberal subjects—­those clearly delimited, centered, coherent, and unified kinds of intentional subjectivity with which the majority of us still tend to identify quite strongly. Here narratology would do better by assuming that for the most part the autobiographical I has a rather weak and vague outline and is decentered and distributed, just as its relative incoherence and fragmentation is only sometimes moving further toward unity and coherence. Moreover, considerable parts of this life writing unfold at very low levels of human awareness, perhaps as informational registrations and actuations temporally or structurally prior to any clear and reflective conscious intention. It may be out of reach of human awareness. Otherwise, more and more of autobiographical production is taking place either at the exterior limit, where perception and articulation mesh with registration and storage of posts, or closer to where affect and basic sensorimotor operations engage with the movement of signaletic materials and the operations of dynamic networks. From moment to moment, if they do not break down, individuations keep producing autobiographies, as traces and projects, increasingly involving nonsubjective subjectivities and system-­environment hybrids. They are emerging in mixed realities composed of real and virtual en128  Ekman

vironments, as well as augmented reality and augmented virtuality layers. Developing the ways of studying the complex narrative effects characteristic of such mixed-­reality individuations, including those blurring clear distinctions between real and virtual everyday environments, would considerably strengthen narratology. One particularly demanding task concerns equipping narratology for an account of an autobiographical project whose environment and set of actors include intelligent agents. Narratology must be able to address and evaluate the complexity of narrative effects emanating from the presence of technical types of cognitive structuration and processing that have hitherto typically been associated with human beings, such as self-­organization and awareness. Increasingly, narrative effects in autobiographical productions include those deriving from ubiquitous computing operating with a certain context awareness. Autobiographical projects increasingly include narrative effects from relations and interactions with computational systems that have, and perhaps display, a situated context awareness of their own, often in the form of an “intelligent” anticipation of human actions or intentions in given practices and settings. For instance, someone engaging today in developing life logging and life casting—­operative at least from Steve Mann’s experiments in the late 1980s with wearable computing and streaming video through Justin Kan’s life casting in the late 2000s (Mann 2005; Coyle 2007)—­would be likely to generate autobiographical narratives marked by the ways in which mobile and social media are context aware. Life-­logging apps such as Reporter, Journey, HeyDay, and Moves in some ways precode what is to become meaningful. The motion processors and gps capabilities of the mobile devices used attune autobiographies beforehand to what a wireless technics of location, event, and movement can do and present. Some autobiographies have been strongly marked by the kinds of technical context-­awareness characteristic of surveillance systems. The Tracking Transience sousveillance project undertaken by professor and media artist Hasan Elahi (2005) after his mistaken placement on the U.S. federal terrorist watch list in 2002 has defined at least a decade of his life narrative. The capacities for information gathering proper to context-­aware self-­surveillance technics (gps devices, mobile phones, mobile photography, credit card transactions, website logs, relational databases, etc.) played constitutive roles in this narrative. Informative Autobiographies  129

Today a narratological study thus includes an analysis of the everyday setting for emergent autobiographies—­an analysis that traces the ways in which this setting itself includes technical context awareness and operates with a live artificial intelligence of sense. For example, such analysis would include considerations of the ways in which smart software agents project what is situationally meaningful on the web, based on co-­profiling citizens and their browser. This analysis would then be capable of describing and discussing how systems embed such profilings and projections more or less preconsciously into everyday actions, and how they inform autobiographical narrative on a protosemantic plane. Today autobiographies also increasingly involve interaction with stronger, more overtly personalized systemic agents. Therefore, cognitive narratology would be more capable if it included a conceptual armature to help account for the complex narrative effects deriving from interactions with exteriorizations of quasi-­aware and quasi-­articulate liveliness in personalized artificial intelligence. These are no longer prototype experiments in research labs, nor just singular projects from media artists (e.g., Rokeby 1990). They are much more pervasive everyday cultural phenomena making their way into human life stories. This is easily demonstrated by pointing to the roles played in everyday culture by personalized gps navigation systems. It is perhaps even more interesting to point to everyday interactions and dialogues involving the expanding range of intelligent assistants and conversational agents, which draw on models of artificial neural networks and operate with algorithms for deep learning, since they can engage in dialogue with an automatic understanding of natural spoken language (Ekman, forthcoming). Since these have broken the thresholds for commercial products and broad public usage, the production of life writing increasingly includes what happens in interaction with agents such as Amazon’s Alexa Voice Service, Apple’s Siri, Microsoft’s Cortana, Google Now, Braina, Echo, or Voice Mate. More generally, personalized and mobile media intimacy makes a difference for emergent autobiographies, for example, with respect to telling environmental movements and navigations, spacings of narrative worlds, temporalizations of events, and configurations and emplotments of individual and social experiential encounters and meetings. This does not just make a difference in terms of better instrumentalization. Rather, mobile intelligent assistants increasingly 130  Ekman

feed forward the potentials for movements, spacings, temporalizations, and sensations, and these tend to be integrated into a human individual’s more or less articulate and conscious autobiographical narrative. Such assistants specifically present questions to narratology regarding their role as agents in evolving narratives. They are increasingly difficult to distinguish clearly from narrators and from either integrated or distributed parts of the authorial I. Cognitive narratology would be thus more functionally complex while being able to examine this kind of interactivity. It would make analytical sense to operate from the start with a distinction between the narrative affordances belonging to the human biological individuation and those belonging to a technological individuation with artificially intelligent traits. Alongside such complex challenges from the history of the present, narratology is facing a set of issues concerning its theoretical complexity. As the major valuable contributions to both classical and postclassical narratology bear out, narratological theory was always complex. But certain kinds of reduction still seem taken for granted in ways that may be evaded today, generally and with respect to autobiography. I engage here with some of the issues, attempting to identify some ways in which a revised narratology may alter and improve its theoretical complexity. Generally, this means that a narratologist could be better off considering whether and then how reductionism may be replaced by appeals to holism and emergence and by consideration of more or less irreducible complexity. More particularly, it means that cognitive narratology could well advance toward a functional complexity by critically examining existing approaches for reductions to organized structural and spatial closure to one linearly organized temporality to an anthropocentric narrative cognition and a semantics of narrative. Perhaps not least due to the key heritage from formalism and structuralism, a majority of existing studies assume that a narrative can be approached systematically and scientifically as an organized whole with closure, an identifiable form, and a structuration as a work. Interestingly, recent work increasingly questions such assumptions, at least to some extent and implicitly, as one may find, for instance, when considering treatments of online storytelling or complex cinema and television. Nonetheless, such assumptions are often still operative in autobiographical studies, despite the well-­k nown impossibility to present a complete Informative Autobiographies  131

life story from death or from a transcendent afterlife. One might justly claim that it is still implicit in a great many approaches that narrative is a problem of organized complexity that can, with the right kind and amount of theoretical care, be reduced to a simple problem with two or fewer variables (Weaver 1948).9 It is also implicitly taken for granted that quite some emphasis should be placed on space, stasis, structure, and closure—­the modes of reduction that prove inadequate if one is inquiring into autobiographies. Instead, narratology needs to be able to operate with notions of life times, dynamics of development, genesis, and open-­ended processes and relationalities of further individuations. Accordingly, I think narratology as well as autobiography must be approached as questions of complexity in which organization figures importantly but not solely. Disorganized complexity necessarily plays constitutive and deconstitutive roles. In that vein, for a revised narratology, disorder and disorderly orders of narratives would be the more interesting points of entry into what is informatively telling. Autobiographies are perhaps primarily to be approached as an issue of potentially continued or discontinued processes of self-­ organization in a certain life form, implying that the complexity of autobiographies is a matter of emergence and disappearance. Instead of paying heed to existing preferential treatment of spatial configuration and an already organized structuration of the form and content of a narrative work, narratology might do better by pursuing the question of structural emergence as pressing and uncertainly open. Structural emergence is a side effect of living on, extraordinarily important but a marvel not to be taken for granted. As a consequence, narratology may well improve its notion of structural emergence by integrating existing enactive and embodied approaches to engage in a more functionally complex manner with contemporary everyday cultural autobiographical thought, practices, and experiences (Varela 1996; Noë 2004; Thompson 2007).10 In addition, this can be developed by adopting a topological approach to life narrative. Such an approach would try to affirm an autobiographical project as a continuously enacted information and deformation of an individuation that is in birth through the process of living. All along and at each point it is engaging with a multiplicity of contingencies with the potential to introduce breaks, tears, and interruptions that would 132  Ekman

discontinue a topology qua a formation of life (Ekman 2015).11 This approach would allow us to make sure that the complexity of an everyday mixed-­reality autobiography includes synchronic emergence, that is, the coming to presence of an entire system-­environment mix, a whole functional human-­technical structure that is more and other than a number of computationally and biologically cognitive parts and their properties. Accordingly, a narratology studying the world space of an autobiographical project should engage with the ways in which spatial complexity arises from a vast number of distinguishable relational regimes and their associated state spaces. An emergent autobiographical space is occasionally approachable as a uniform, homogenous, Cartesian space, but it is more and more likely to introduce the idea of the complexity of polycontextual and heterogeneous productions of space involving many individual, social, biological, and technical processes that foster or impede self-­organization. This is already to hint at the complexity of temporalization that is becoming more of a pressing issue but tends to be parenthesized in the narratological works building on the linear time of narrative, a universal container through which events pass.12 In contemporary network societies, however, autobiographical projects increasingly involve events qua encounters with multiple contingent transition states. This emphasizes the necessity of existing and living on with temporal nonlinearity and bifurcations. Current projects build on post-­traditional and therefore more obviously contingent individuation processes in the plural. Here human encounters with the need to develop other structures of temporal consciousness emerge along with the developments of a great many technical and computational kinds of temporalization. The narrative times of autobiography are thus increasingly likely something that narratology would approach better by thinking of them as beyond or before traditional linear chronology. They are produced by many and varied human and technical systems attempting to manage complexity by temporalizing it. Autobiographical temporalization might imply encounters with nondecomposable contingencies. Temporalization is becoming more complex and heterogeneous, and therefore narratology increasingly has to address diachronic emergence. Can temporalizations be grasped as one process on one time scale with one homogenous time of development, or not? Informative Autobiographies  133

The complexity of human and technical temporalizations, their transversal interrelations of time scales as well as of both homogeneous and heterogeneous times, is increasingly an issue that has to be addressed. It would be advantageous for narratology to consider this as something prior to the structural and synchronic processual outcomes of human and technical systems. They coevolve temporalizations, perhaps in diachronically emergent ways. Here, many temporalizing self-­(and other) organizations develop to yield emergent times beyond the summation of those of the partaking individuations. Such human and technical coemergences are more than wholes insofar as they cut across the individual ongoing complexifications of temporalization to relate them in symbiogeneses or mutations. They relate new kinds of coevolution of ways to temporalize. Francisco Varela’s evolutionary path making and natural drift are interesting approaches toward multiple, heterogeneous enactments of living times (1996, 185–­216). David Bohm’s (1986) work and the extended conversation between Michel Serres and Bruno Latour (1995) offer other interesting approaches to complex, heterogeneous temporalizations. In addition, it would be rewarding to delve deeper into the work by Latour (2005) and John Law and John Hassard (1999) on actor-­ network theory to see how it can offer narrative studies an approach to the coevolving times of many productive life-­writing actors in material, technological, and social systems. Quite contrary to the import assigned to technical and nonhuman temporalization here, it is a regularly recurring assumption in narratology that narrative and cognition are human. Both are taken to be concerned with a recognizable organization of the time, space, causality, and characters of decidedly human experience of the world. But an autobiographical project today increasingly depends on its relations to the information-­intensive environment, and many of its narrative affordances, including forms and contents, derive from operative interaction designs, algorithmic codings, and exchanges with intelligent personal assistants as well as a multiplicity of smart software agents. In that case, humanist and social scientific anthropocentrisms are too narrow and must be expanded to capture the complexity at stake. Science and technology studies and actor-­network theory, for example, can inform narratology as regards the capacities of technics for movement, agency and operation, registration and memory, and prosthetics and inven134  Ekman

tion. That would be sufficient to prompt complexifying revisions of narratology as regards the import for life narratives of the nonhuman, the technological, and the environment (Whitlock 2012).13 Moreover, current developments blurring the distinctions between animality, organism, and recursive technical process in forms of life are now informing autobiographies constitutively and demand further nonanthropocentric narratological takes.14 The co-­developments and mutual imbrications of physical environments, second-­order environments such as ubiquitous computing and big databases, other life forms, and human cultures increasingly displace, defer, and complicate the subject-­object divide and the distinction between the narrative cultures of human beings and their world. A narratology for contemporary autobiographies is thus led toward affirming that autobiographical art is a second-­order technics of life; that humans as well as other forms of life exist technologically; and that technology is not simply external, marginal, rigidly mechanical, and indifferent but is how environmental membranes and relationality are generated, maintained, and complexified (Simondon 1964, 18–­22; 1992; Esposito 2001).15 In spite of the developments and new challenges mentioned earlier, classical and postclassical narratology are alike in demonstrating that reductionism is a paradigm not easily shaken. It is still a widely shared assumption that a reduction to base holds: narrative and narratology are concerned with semantics and take meaning as their basic concept. This anchors all the hermeneutic efforts in narratology in the human sciences, and all recourse in narrative studies to a meaningful cultural anthropology and a sociological semantics, along with all cognitive science work on narrative meaning. The influence and the value of this can hardly be overestimated, but it is nonetheless an approach slowly but increasingly becoming problematic. Let me try to illustrate this. An autobiographical project emerging with ubiquitous computing today most often tilts and develops asymmetrically: the major part belongs to the complex dynamics of an information-­intensive environment. It belongs to an informational ecotechnics or to the unfolding of computing in an expanded and intensely networked mixed reality. Most of this mixed reality field tends toward becoming embedded and infrastructural, and, consequently, humans find it harder and harder to distinguish this from real and natural environments. This implies at the very least an information-­technological Informative Autobiographies  135

displacement of narrative meaning: narrative sense appears as late, local, secondary, partial, and abstract side effects in the more general distributed system fields and times of complex informational network operations. If narratology keeps a strict, privileged focus on pursuing meaning, it addresses a small part of this autobiographical project. It will be in danger of remaining blind not only to the constitutive role of the development of computational entities as nonmechanical, nontrivial, and open machines but also as regards the key role played by information. In view of this, I think narratology would be strengthened by going backward in order then to move forward. It would be a potential advance to go back over the developments since the late 1940s, using the insights to be had by recognizing how Claude Elwood Shannon and Warren Weaver’s (1949) general and mathematically based communication theory became the hegemonic paradigm. It might be both uncomfortable and productively instructive to consider how the (in)famous statistical informational turn in this theory was accompanied by a deliberate relegation of semantics to irrelevance. In addition, narratology might advance by taking note of two moves: the integration of this question of information in the first wave of German media theory and Friedrich Kittler’s (1997) adoption of communication theory along with a provocatively resolute antihermeneutic stance. Combined with the recognition of the import of cybernetics, mediatechnics, and information for Derridean deconstruction and Luhmannian systems theory (Ekman 2016), these observations would improve narratology by way of complicating the discipline.16 A revised narratology would be capable of considering whether and then how meaning is always already technically informed. In some parts of postclassical narratology one can already find narratives approached sometimes as matters of meaning and sometimes as matters of information gathering, and these approaches vary according to the context at stake. It would be very interesting to see a more extensive and thorough narratological account and evaluation of such oscillations between meaning and information. Specifically, an approach to autobiographical projects today calls for an integrated treatment of the role of information for emergent life stories. It seems too broad and inoperative to adopt a stance that would right away abolish meaning and narratives that make sense, but some studies of narrative nonsense and noise are very interesting in an informative respect (Deleuze 1990; Serres 136  Ekman

2000).17 It would seem too broad ontologically and reductive semantically to move through parenthesizing both meaning and mediatechnics toward a nonmodern sensation of what is born and what comes to presence with life stories (Gumbrecht 2004, 2014). Perhaps narratology would do better by trying to trace the ways in which current autobiographical projects undertake complex transversal movements that have narrative meaning cultures and how information cultures coexist and co-­develop in ways that are not necessarily anthropocentric (Hayles 2002, 2012; Hörl 2011).18 Just as technics, environment, and information tend to be parenthesized, likewise, narratology does not explicitly address the question of complexity. Narratological theories try mostly to reduce complexity to simple problems, but quite often it is made clear that narrative is mainly an organizational problem, hence one of organized complexity in Weaver’s (1948) sense. This is the assumption behind the frequent claims in narratology that narrative organizes, frames, and scaffolds human experience and behind the notion that narratology may study this as a system of logic. Although this is implicit, one can assume that the hegemonic approach to the complexity of narrative is a middle-­of-­the-­ road compromise that holds on to reductionism while making a little bit of room for weak emergence. On such a view autobiographies are approached through a quite pragmatic modular building of near-­independent theories for each level of complexity—­combined with bridging theories that demonstrate how each higher level can be accounted for in terms of the elements and relation of the next level. In other words, autobiographies are taken to be organized as complex systems in the sense that they are made up of a large number of narrative parts that have many interactions. They are weakly emergent because, given the properties of the parts and the rules of their interaction, it is not a trivial matter to infer the properties of the whole narrative. This kind of approach permits crosscutting nonhierarchical interactional relations but assumes that these are mostly insignificant and may well be reduced away in favor of affirming the import of structure and hierarchy for the development of narrative complexity. Like narratives in general, autobiographies are thus taken to affirm that their complexity most often takes the form of organizational hierarchy in a broadly formalized sense (not necessarily with relations of master-­slave authority). They are assumed to form narrative systems composed of interrelatInformative Autobiographies  137

ed subsystems, each of which is in turn hierarchic in structure until we reach some lowest level of elementary subsystem (Simon 1981, 183–­85).19 This is an enormously powerful approach, extremely influential across a wide set of disciplines, and it has the additional merit of keeping the paradigm of objectively oriented reductionism that anchors major parts of modern science intact. It should be clear from my earlier remarks that I question this kind of approach in several ways. Specifically, this type of approach risks missing the import of crosscutting relations in synchronic or structural emergence. It also risks certain blindness to the import of diachronic and transversal emergences in the living enactment of life narratives. Not least of all, it tends to downplay the mutual specification ongoing between a life to be told and an environment that remains the larger source of narrative complexity. I think these issues indicate that a more holist view of narratology would be an advantage. Narratology would be stronger and more adequate insofar as it could confront those contingencies, fragile structures, momentary temporal orders, and passing relational equilibria in environments and niches, which are characteristic of experiments in living on and evolving while trying to tell about it. This chapter contributes to the critical and revisionist argument of the volume by suggesting that studies of autobiography would benefit from opening into a more complex integration of the deep structures and the deep times of lives evolving with environments, engaging with the contingencies of natural selection. In autobiographies, natural selection and continuous variation are perhaps the key stories to be told and plotted while also being impossible to tell and plot. Evolution by natural selection is necessarily at the heart of the autobiographical process and product but it also resists and withdraws from it, informing about a problem of the complexity of nonlinear dynamical and living systems (Abbot 2003). Autobiographical projects are therefore to be approached as enactments in the game of life, always already involving a preindividual dimension with the potentials for other individuations. This would suggest that one considers autobiographical narratology a science of the “naturally” artificial. This includes transversal emergences of narrative configurations, emplotments, and experiences of lives lived with technics as environmental membranes that inform—­meaningfully so only sometimes and in some places. This makes of narratology a discipline that every so often only has the capacity to simulate the life narratives at 138  Ekman

stake on a one-­to-­one basis because these narratives are of irreducible complexity (Wolfram 1984, 1986).20 Such autobiographies must remain complexly moving targets, as points or as waves. They cannot be defined, grasped, and articulated—­at best, they can momentarily be traced and locally placed as fluctuations somewhere between orderings of structures and dissipations into disorder (Nicolis and Prigogine 1977; Prigogine and Stengers 1984). Notes 1. Helga Schwalm (2014) provides a good overview of modern and late modern autobiography. 2. I refer to modern narratology studying narrative systematically, standing on the shoulders of not least the Russian formalists and Vladimir Propp. 3. Monika Fludernik’s (1996) work was an early effort to draw on discourse analysis, reader-­response theory, and cognitive linguistics to sketch a “natural” narratology encompassing all narrative genres, across the fictional and the nonfictional, where narrativity is constituted by the evocation of anthropomorphic real-­life experience. The strong emphasis in Fludernik’s study on text and discourse might be complemented by the kinds of cognitive insights found in Jerome Bruner’s (1986) work on narrative modes of thought and in Herman’s (2013) work on narrative and mind. 4. The anthology edited by Fludernik and Jan Alber (2010) testifies to the recent influence of cognitive science in narrative studies, including a treatment of autobiography. 5. David Herman’s (2011, 2013, 2015) work presents some resources for a cognitive narratology after the second cognitive turn. It includes attempts to pursue the implications for narratology of embracing an embodied and enactivist approach. Interestingly, Herman’s work opens the issue of the complexity of narrative effects through treatments of emergence, environment, and the nonhuman. 6. There is now a small but steadily growing body of research articles and chapters in this area. 7. I assume that the complexity of autobiographical projects is emergent. In narratology the engagement with emergence is growing. Philosophy and science, complexity and systems theory, and cultural theory might offer additional resources for narratologies concerned with emergence. 8. These two edited volumes (Ekman 2013, Ekman et al. 2016) offer more extended treatments of the contemporary co-­development of cultures and ubiquitous computing. Informative Autobiographies  139

9. Weaver (1948) distinguishes first between “problems of simplicity” (largely related to two variables) and “problems of complexity.” He then distinguishes between problems of “disorganized complexity” on the one hand (billions of variables, as in physics and math, to be handled by probability theory and statistical mechanics) and problems of “organized complexity” on the other (the middle region concerning ways to deal simultaneously with a sizable number of factors interrelated into an organic whole). 10. I draw in a broad sense on the notion of enactment. 11. I present a much more detailed discussion of topological approaches in this chapter. 12. A narratological cornerstone such as Paul Ricoeur’s (1984) work on time and narrative still has its dialectics moving according to this paradigm. 13. Critical inquiries into humanist and anthropocentric notions of autobiography are rare. Gillian Whitlock’s (2012) work concerning posthuman life stories is an interesting example. 14. Recognition of this problem spurs the current interest in the Anthropocene and a notable expansion of the notion of the living mind. The rekindling of interest in Alfred North Whitehead’s (1929) thought is one example; another is the return of panpsychism in Evan Thompson’s (2007) work. 15. I am drawing on the Simondonian notion of transductive technical relationality to question the general tendency in systems theory to solidify as rigorous and impenetrable the inside-­outside or system-­environment distinctions. Gilbert Simondon (1964) thinks of transduction as constitutive relationality and individuation in process. In my view technology is not outside a system nor outside system-­environment couplings nor reducible to structural couplings nor is it just that indifferent part helping to form the environment of systems. Technics are nonneutral and capable not only of destroying systems but also of irritating, forming loose couplings, and triggering system-­internal complexification or restructuration through internal processes of self-­determination. For an interesting but minimal agreement from a Luhmannian, see Elena Esposito’s (2001) argument that interactions with computers should be approached as a kind of self-­irritation of society under conditions of high complexity. 16. In this interview, conducted by Roberto Simanowski during email exchanges in 2015, I comment on the approach to information in Jacques Derrida (1976, 1981) and Niklas Luhmann (1995). Both Derrida after the 1973 publications and Luhmann after the adoption of autopoietic theory could be read as more radical because they pursue less radical approaches to meaning vis-­á-­vis information and technology. Both thinkers keep meaning but complexify it informationally, each in his own way. Perhaps 140  Ekman

cybernetic technology is a constitutive and deconstitutive blind spot for both, calling silently for technical complexification before and beyond second-­order cybernetic paradigms. 17. I am thinking in particular of Gilles Deleuze’s (1990) eleventh series of nonsense. Serres (2000) suggests in his work on Lucretius that, in information-­ theoretical terms, narrative meaning emerges as an aleatory, local deviation in between white noise and monotony as two modes of chaos. 18. Broadly speaking, I agree with points made by Erich Hörl. Hayles (2002) offers an interesting approach to narrative as moving between energy and information, reminding me of other publications of hers demonstrating ways to be affected by what murmurs informatively at the limit of narrative meaning. I tend toward granting asymmetrical privilege to noise and information culture as the broader and more constitutive and deconstitutive. But one might well engage in further debate today over the relation and relative merits of information and meaning, database and narrative, as does Hayles. 19. This is a compressed version of the approach to architectures of complexity taken in Herbert Simon’s (1981) influential treatment of the sciences of the artificial. 20. Stephen Wolfram’s (1984, 1986) writings provide an interesting example of a computer scientist, entrepreneurial engineer, and physicist at work on irreducible complexity. References Abbot, H. Porter. 2003. “Unnarratable Knowledge: The Difficulty of Understanding Evolution by Natural Selection.” In Narrative Theory and the Cognitive Sciences, edited by David Herman, 143–­62. Stanford ca: Center for the Study of Language and Information. —. 2008. “Narrative and Emergent Behavior.” Poetics Today 29:227–­44. Bedau, Mark, and Paul Humphreys. 2008. Emergence: Contemporary Readings in Philosophy and Science. Cambridge ma: mit Press. Bohm, David. 1986. “Time, the Implicate Order, and Pre-­space.” In Physics and the Ultimate Significance of Time, edited by David Ray Griffin, 176–­208. Albany: State University of New York Press. Brockmeier, Jens. 2015. Beyond the Archive: Memory, Narrative, and the Autobiographical Process. Oxford: Oxford University Press. Bruner, Jerome S. 1986. Actual Minds, Possible Worlds. Cambridge ma: Harvard University Press. Campbell, Jan, and Janet Harbord, eds. 2002. Temporalities, Autobiography, and Everyday Life. Manchester: Manchester University Press. Informative Autobiographies  141

Clarke, Bruce, and Mark B. N. Hansen. 2009. Emergence and Embodiment: New Essays on Second-­Order Systems Theory, Science and Cultural Theory. Durham: Duke University Press. Coyle, Jake. 2007. “Justin Kan Vlogs 24/7 at Justin​.tv.” Washington Post, March 28. 2007. Date, Christopher J. 2004. An Introduction to Database Systems. 8th ed. Boston: Pearson/Wesley. Deleuze, Gilles. 1990. The Logic of Sense. New York: Columbia University Press. Derrida, Jacques. 1976. Of Grammatology. Baltimore: Johns Hopkins University Press. —. 1981. Dissemination. Chicago: Chicago University Press. Ekman, Ulrik, ed. 2013. Throughout: Art and Culture Emerging with Ubiquitous Computing. Cambridge ma: mit Press. —. 2015. “Design as Topology: U-­City.” In Media Art and the Urban Environment, edited by Frank Marchese, 177–­202. Berlin: Springer International. —. 2016. “Network Societies 2.0: The Extension of Computing into the Social and Human Environment.” In Digital Humanities and Digital Media, edited by Roberto Simanowski, 148–­83. London: Open Humanities. —. Forthcoming. “The Complexity of Coding Conversational Agents.” In Uncertain Archives: Big Data Environments, edited by Kristin Veel, Nanna Bonde Thylstrup, and Daniela Agostinho. Cambridge ma: mit Press. Ekman, Ulrik, J. David Bolter, Lily Diaz, Morten Søndergaard, and Maria Engberg, eds. 2016. Ubiquitous Computing, Complexity and Culture. New York: Routledge. Elahi, Hasan. 2005. Hasan Elahi. Accessed November 22, 2018. http://​elahi​.umd​.edu. Esposito, Elena. 2001. “Strukturelle Kopplung mit unsichtbaren Maschinen.” Soziale Systeme 7 (2): 241–­52. Fludernik, Monika. 1996. Towards a “Natural” Narratology. London: Routledge. Fludernik, Monika, and Jan Alber, eds. 2010. Postclassical Narratology. Columbus: Ohio State University Press. Grishakova, Marina, and Marie-­Laure Ryan. 2010. Intermediality and Storytelling. New York: De Gruyter. Gumbrecht, Hans Ulrich. 2004. Production of Presence. Stanford ca: Stanford University Press. —. 2014. Our Broad Present. New York: Columbia University Press. Hayles, N. Katherine. 2002. “Escape and Constraint: Three Fictions Dream of Moving from Energy to Information.” In From Energy to Information: Representation in Science and Technology, Art, and Literature, edited by

142  Ekman

Bruce Clarke and Linda Dalrymple Henderson, 235–­54. Stanford ca: Stanford University Press. —. 2012. “Narrative and Database.” In How We Think: Digital Media and Contemporary Technogenesis, 175–­98. Chicago: University of Chicago Press. Herman, David. 1999. Narratologies: New Perspectives on Narrative Analysis. Columbus: Ohio State University Press. —. 2011. “Storyworld/Umwelt: Nonhuman Experiences in Graphic Narratives.” SubStance 40 (1): 156–­81. —. 2013. Storytelling and the Sciences of Mind. Cambridge ma: mit Press. —. 2015. Creatural Fictions. New York: Palgrave Macmillan. Hörl, Erich. 2011. “Die technologische Bedingung: Zur Einführung.” In Die technologische Bedingung, edited by Erich Hörl, 7–­53. Berlin: Suhrkamp. Juarrero, Alicia. 1999. Dynamics in Action: Intentional Behavior as a Complex System. Cambridge ma: mit Press. Kitchin, Rob. 2014. The Data Revolution, Big Data, Open Data, Data Infrastructures and Their Consequences. Los Angeles: Sage. Kittler, Friedrich A. 1997. “There Is No Software.” In Literature, Media, Information Systems: Essays, edited by John Johnston, 147–­55. Amsterdam: G+B Arts International. Latour, Bruno. 2005. Reassembling the Social: An Introduction to Actor-­ Network-­Theory. Oxford: Oxford University Press. Law, John, and John Hassard, eds. 1999. Actor Network Theory and After. Oxford: Blackwell. Livingstone, Sonia. 2012. “Taking Risky Opportunities in Youthful Content Creation.” In Self-­Mediation, edited by Lilie Chouliaraki, 39–­54. London: Routledge. Luhmann, Niklas. 1995. Social Systems. Stanford: Stanford University Press. Mann, Steve. 2005. “Sousveillance and Cyborglogs: A 30-­Year Empirical Voyage through Ethical, Legal, and Policy Issues.” Presence: Teleoperators and Virtual Environments 14 (6): 625–­46. https://​doi​.org​/10​.1162​ /105474605775196571. Manovich, Lev, Jake Brouwer, Arjen Mulder, and Susan Charlton. 2003. Making Art of Databases. Rotterdam: v2/nai. Nicolis, Gregoire, and Ilya Prigogine. 1977. Self-­Organization in Nonequilibrium Systems. New York: Wiley. Noë, Alva. 2004. Action in Perception. Cambridge ma: mit Press. Nünning, Ansgar. 2003. “Narratology or Narratologies?” In What Is Narratology? Questions and Answers Regarding the Status of a Theory, edited by Tom Kindt and Hans-­Harald Müller, 239–­75. Berlin: De Gruyter.

Informative Autobiographies  143

Ochs, Elinor, and Lisa Capps. 2001. Living Narrative: Creating Lives in Everyday Storytelling. Cambridge ma: Harvard University Press. Page, Ruth E. 2012. Stories and Social Media: Identities and Interaction. New York: Routledge. Page, Ruth E., and Bronwen Thomas. 2011. New Narratives: Stories and Storytelling in the Digital Age. Lincoln: University of Nebraska Press. Prigogine, Ilya, and Isabelle Stengers. 1984. Order Out of Chaos. Boulder: New Science Library. Protevi, John. 2006. “Deleuze, Guattari and Emergence.” Paragraph 29 (2): 19–­ 39. https://​doi​.org​/10​.3366​/prg​.2006​.0018. Rettberg, Jill Walker. 2014. Seeing Ourselves through Technology. Basingstoke, UK: Palgrave Macmillan. Ricoeur, Paul. 1984. Time and Narrative. 3 vols. Chicago: University of Chicago Press. Rokeby, David. 1990. The Giver of Names. Accessed November 22, 2018. www​ .davidrokeby​.com​/gon​.html. Ryan, Marie-­Laure. 2004. Narrative across Media. Lincoln: University of Nebraska Press. —. 2006. Avatars of Story. Minneapolis: University of Minnesota Press. Ryan, Marie-­Laure, and Jan-­Noël Thon, eds. 2014. Storyworlds across Media: Toward a Media-­Conscious Narratology. Lincoln: University of Nebraska Press. Schleser, Max. 2014. “Connecting through Mobile Autobiographies: Self-­ Reflexive Mobile Filmmaking, Self-­Representations, and Selfies.” In Mobile Media Making in an Age of Smartphones, edited by Marsha Berry and Max Schleser, 148–­58. New York: Palgrave Macmillan. Schneier, Bruce. 2015. Data and Goliath: The Hidden Battles to Collect Your Data and Control Your World. New York: Norton. Schulze, Gerhard. 1992. Die Erlebnisgesellschaft: Kultursoziologie der Gegenwart. New York: Campus. Schwalm, Helga. 2014. “Autobiography.” In Handbook of Narratology, edited by Peter Hühn, 14–­29. Berlin: De Gruyter. Serres, Michel. 2000. The Birth of Physics. Manchester: Clinamen. Serres, Michel, and Bruno Latour. 1995. Conversations on Science, Culture, and Time. Ann Arbor: University of Michigan Press. Shannon, Claude Elwood, and Warren Weaver. 1949. The Mathematical Theory of Communication. Urbana: University of Illinois Press. Simon, Herbert A. 1981. The Sciences of the Artificial. 2nd ed. Cambridge ma: mit Press.

144  Ekman

Simondon, Gilbert. 1964. L’individu et sa genèse physico-­biologique. Paris: Presses Universitaires de France. —. 1992. “The Genesis of the Individual.” In Incorporations, edited by Jonathan Crary and Sanford Kwinter, 296–­319. New York: Zone. Stiegler, Christian. 2015. “Selfies and Selfie Sticks.” In New Media Culture, edited by Christian Stiegler, Patrick Breitenbach, and Thomas Zorbach, 67–­ 82. Bielefeld, Germany: Transcript. Thompson, Evan. 2007. Mind in Life. Cambridge ma: Belknap. Thon, Jan-­Noël. 2015. Transmedial Narratology and Contemporary Media Culture. Lincoln: University of Nebraska Press. Varela, Francisco J. 1996. Embodied Mind. Cambridge ma: mit Press. Verstraten, Peter. 2009. Film Narratology. Toronto: University of Toronto Press. Walsh, Richard. 2011. “Emergent Narrative in Interactive Media.” Narrative 19 (1): 72–­85. Weaver, Warren. 1948. “Science and Complexity.” Scientific American 36:536–­44. Weiser, Mark. 1991. “The Computer for the 21st Century.” Scientific American 265 (3): 94–­104. Whitehead, Alfred North. 1929. Process and Reality: An Essay in Cosmology. Cambridge: Cambridge University Press. Whitlock, Gillian. 2012. “Post-­Ing Lives.” Biography 35 (1): v–­xvi. Wolfram, Stephen. 1984. “Cellular Automata as Models of Complexity.” Nature 311:19–­424. —. 1986. “Approaches to Complexity Engineering.” Physica D: Nonlinear Phenomena 2 (1–­3): 385–­99.

Informative Autobiographies  145

Part 2

Cognition and Narrative Comprehension

6

Sources of Complexity in Narrative Comprehension across Media Joseph P. Magliano, Karyn Higgs, and James Clinton

We experience narratives in a variety of formats (auditory, print, electronic) and media. We can hear, read, view (static visual narratives, graphic novels, and comics), and watch them (film). These different ways of experiencing narratives raise questions about the commonalities and differences in how we comprehend them across different media. Many theories of narrative comprehension proposed by discourse psychologists have claimed that there are common mechanisms supporting comprehension across the ways we can experience narratives (Gernsbacher 1990; Kintsch 1998; Magliano, Radvansky, and Copeland 2007). Although there are empirical bases for this claim (Magliano et al. 2012, 2014; Zacks, Speer, and Reynolds 2009), further research is warranted. A central thesis of this chapter is that the possibility of common mechanisms for comprehending narratives across media does not preclude the possibility that there are also important differences. First, the interplay between the cognitive and perceptual processes involved in consuming a narrative in the moment and building the durable mental models that are the basis for comprehension and memory may vary in nontrivial ways across media (Magliano et al. 2013). Second, narratives in different media may have different affordances (Loughlin et al. 2015; Magliano et al. 2018b) that make some aspects of comprehension more or less challenging. Third, some processes may be either unique or relatively more or less important across media (Loughlin et al. 2015). In this chapter we explore each of these issues to better understand the complexities of narrative experiences across different types of media. We refer to complexities as aspects of a narrative that potentially 149

create challenges for the understander from the perspective of theories of narrative comprehension. We first define what constitutes narrative comprehension according to theory and research in discourse psychology. We then turn to each of these factors in our exploration. What Is Comprehension? To explore the impact of media on the complexity of comprehension, one must first describe what is meant by comprehension. In the context of discourse psychology, comprehension refers to the process of constructing a coherent, mental model for a specific information product (McNamara and Magliano 2009b). A mental model consists of a network of interrelated propositions that reflect the explicit information within narrative, and the inferences that establish how the explicit content is interconnected and related to background knowledge (Kintsch 1988, 1998; Zwaan and Radvansky 1998). The network is constructed through inferences that establish relationships between propositions (i.e., bridging inferences) and knowledge-­based inferences that elaborate on the propositions (McNamara and Magliano 2009a, 2009b). Moreover, understanders should dynamically engage in inference processes given the demands of the narrative, that is, generate bridging inferences when causal or logical connections need to be made (2009a). It is widely accepted that mental models index situational content and relationships that bind and separate the events that comprise a narrative. More specifically, according to the event-­indexing model (Zwaan and Radvansky 1998), readers monitor a number of situational dimensions as they read, update their mental model as these dimension change, and ultimately organize narrative events around these dimensions in episodic memory (see also Gernsbacher 1990). First, mental models contain representations of salient entities that contribute to the story plot. These entities may be agents (people or animals), objects, or abstract concepts; however, narratives at their core are about sentient entities and are arguably structured around them (Ozyurek and Trabasso 1997; Scott Rich and Taylor 2000). Second, narrative events take place in a spatial-­temporal framework, which constitutes the time and place where the events take place in a narrative world (e.g., Radvansky et al. 1998). Third, narrative events are typically connected by causal relationships

150  Magliano, Higgs, and Clinton

and are usually structured around hierarchically organized goal episodes (Graesser, Singer, and Trabasso 1994; Trabasso, van den Broek, and Suh 1989). For example, when watching a James Bond movie, the viewer experiences Bond being given a case to solve near the beginning of the film and the subsequent scenes depict his efforts to accomplish subordinate goals that lead to solving the case. Understanding why Bond behaves as he does in the subsequent scene requires one to infer how his behaviors are causally connected to his goals (Magliano, Taylor, and Kim 2005). But understanding why characters behave the way they do often requires one to monitor a range of internal psychological states, such as characters’ beliefs about the narrative world (Graesser et al. 1999) and how they affectively react to narrative events (Gernsbacher, Hallada, and Robertson 1998; Komeda and Kusumi 2006; Rapp, Komeda, and Hinze 2011). When watching the first exchange between Bond and the primary villain in any James Bond movie, one must understand that the characters have different goals, beliefs, and emotions to appreciate the discrepancies between what is outwardly said and how the characters actually feel about one another. While the multidimensional nature of mental models may be equally important across different media (Gernsbacher 1990; Magliano, Radvansky, and Copeland 2007; Magliano et al. 2012), we contend that one source of narrative complexity arises from differences in the ease of monitoring or inferring change in situational continuities (continuity in time, space, causality, etc.). This is a point that we elaborate on in greater detail in the following section. Differences in Front-­End and Back-­End Processes Joseph Magliano and colleagues (2013) provided a framework to elucidate the basic perceptual and cognitive processes that operate when comprehending narratives in different media and, in particular, in text, sequential graphic narratives, and dynamic visual narratives (film, video, and animation). The framework delineates between front-­end and back-­end processes. Front-­end processes are involved in the moment-­to-­moment processing of information (e.g., attentional selection and information extraction), and back-­end processes are involved in building mental models of an experience (e.g., inferences, structure building).

Complexity in Narrative Comprehension  151

Front-­End Processes and Narrative Complexity First consider front-­end processes, which support the extraction of information from sensory input. The primary product of these processes is a representation (or simulation in the spirit of grounded cognition; Zwaan, 2004, 2014) of what is happening “now” in the narrative world, represented in working memory (Magliano et al. 2013). While the underlying nature of the resulting representation may be similar regardless of the media, there are obvious important differences in front-­end processing across media. Text-­based (and spoken) narratives are entirely consumed through language processing, but visual narratives (comics and film) are typically conveyed in a multimedia format that involves both language and visual content. Language in visual narratives is most frequently used to convey internal thoughts or dialogue, but the events that move the plot forward are often conveyed in visual content (Cohn 2016). As such, perceptual processes are critical for interpreting the events conveyed in the narrative. These processes include gist processing (recognizing the type of environment depicted), object recognition (identifying agents and prominent objects in a scene), motion processing (for film), and attentional selection (the extent to which internal or external forces affect where the eye moves in a text, on a panel, or on a screen) (Magliano et al. 2013). Another potentially nontrivial difference in front-­end processes is the extent that the understander has control over the pace of experiencing the narrative. Text and static visual narratives afford control, whereas control is external to the understander for spoken and dynamic visual narratives. Different forms of media, however, may utilize techniques to direct attention, reducing challenges related to external control of pacing. Filmmakers craft films to help the viewer detect the most salient regions of the screen in terms of conveying the narrative events (Smith 2012). These techniques can exert a strong influence on viewers’ attention. In a recent study exploring the relationship between front-­and back-­end processing when viewing commercial films, we have discovered a phenomenon that we have called the tyranny of film (Loschky et al. 2015). Lester Loschky and colleagues (2015) found that when viewing a commercially produced film, viewers tend to look at the same regions of the frames at the same times, regardless of whether they saw the prior scenes that provided the context to make those regions relevant to the story. 152  Magliano, Higgs, and Clinton

In contrast to dynamic narratives, consumers of static narratives (texts and comics) do have control over pace. Accordingly, they have to decide where to move their eyes, and eye movements are closely linked to comprehension (Reichle et al. 1998). There has been relatively little research exploring eye movements in graphic narrative (Carroll, Young, and Guertin 1992; Chiba et al. 2007; Foulsham, Wybrow, and Cohn 2016). Some studies show that manipulating panel arrangements in a manner that potentially changes the meaning affects eye movement and comprehension. For example, there tends to be increased number of fixations and more regressive eye fixations in random order than in original order, indicating increased processing effort (Foulsham, Wybrow, and Cohn 2016). Given that analogous manipulations lead to similar effects with narrative texts, these findings suggests that eye-­movement data may be similarly useful for studying comprehension processes in graphic narratives. Clearly, more research is needed to understand how eye movements are coordinated to support the comprehension in visual narratives (both graphic narratives and film). There are at least two reasons why these differences in front-­end processes may lead to differences in narrative complexity across media. First, it is well documented that individual differences in the ability to engage in the mechanics of reading affects one’s ability to comprehend what they read (e.g., Gough and Tunmer 1986). As such, proficiency in reading may create a greater barrier for comprehension for narratives that involve texts than those that don’t require reading. This may be one reason why proficiency in comprehending text is weakly correlated with proficiency in comprehending visual narratives, at least in children (Pezdek, Lehrer, and Simon 1984). That is, proficiency in front-­end processes of reading may be a greater source of individual differences in comprehension ability than proficiency in front-­end processes associated with visual narratives. This is an untested hypothesis. Second, differences in the extent that the understander has to devote resources to directing attention could have implications for the computation of the narrative’s representation in working memory. One implication of the tyranny of film is that the viewer does not have to devote much effort to directing where the eyes move and therefore should have more resources to construct a representation of what is happening in the narrative relative to static narratives. We can only speculate as to whether Complexity in Narrative Comprehension  153

this is the case because only a few studies have attempted to explore the impact of media on comprehension (Baggett 1979; Magliano et al. 2012), but none have explored this aspect of front-­end processing. There is a potential tradeoff, however, when one does not have to exert effort to direct attention in dynamic narratives. When control of pacing is internal (e.g., static narratives) and a comprehension failure occurs, readers can direct their eyes to locations in the text (or sequential narrative) that may be useful in repairing the comprehension problem (Reichle et al. 1998). But this is not possible with dynamic films, unless one is watching on a device that affords rewatching. Back-­End Processes and Narrative Complexity Next consider back-­end processes. Magliano and colleagues (2013) specified back-­end processes as involving event segmentation (e.g., Zacks, Speer, and Reynolds 2009), inference generation (e.g., Graesser, Singer, and Trabasso 1994), and the basic processes involved in mental-­model construction (e.g., Gernsbacher 1990). Event segmentation refers to a process of recognizing the boundaries between events, which happens when we experience both real-­world events (Zacks et al. 2007) and narrative events (Magliano, Miller, and Zwaan, 2001; Zacks, Speer, and Reynolds 2009). Inference generation refers to processes that involve drawing on existing knowledge to establish how narrative events are connected or to elaborate on narrative events, for example, inferring traits of characters or instruments used by characters to perform actions and inferring internal states of characters that are not explicitly described (e.g., Graesser, Singer, and Trabasso 1994). Mental-­model construction involves the processes involved in updating episodic memory for the narrative based on the relationship of the current event to prior events (e.g., Gernsbacher 1990; Zwaan and Radvansky 1998). As noted earlier, there are very few empirical investigations in comprehension processes across media, in particular when controlling for content (e.g., Baggett 1979; Magliano et al. 2012). A seminal study by Patricia Baggett (1979) suggested that memory representations for text-­ based and visually based narratives are similar in structure and content. Across several experiments participants either read a text-­based version of a story or viewed a picture-­based version (e.g., static presentation of pictures from the film) and then performed a variety of com154  Magliano, Higgs, and Clinton

prehension tasks. For example, one task involved identifying the salient goal episodes of the primary character in the story. Participants parsed the story into the same episodes regardless of format. As another example, participants were asked to recall the narratives; the content of the recall protocols was remarkably similar in content and structure across the story formats, suggesting mental models are similar across media. Magliano and colleagues (2012) conducted a study showing similarity in event segmentation across visual and text-­based narratives. They had participants view either static picture-­based narratives or text-­based adaptations of them and then engage in an event segmentation task that involved identifying shifts in narrative events. The stories were coded for shifts in particular dimensions of situation continuity (e.g., space, time, and goal episodes). Magliano and colleagues found that participants’ judgments of event boundaries were similarly correlated with the situational shifts regardless of the format in which the stories were presented. But there was one interesting difference. Participants were more likely to judge that there was an event boundary after a shift in narrative space in the text version than in the picture version. One could have argued that the opposite should have been the case since there should be overt perceptual evidence of a spatial shift in the picture versions. However, upon examination of the materials the spatial shifts often occurred in similar regions of the narrative space—­for example, a character moving through different parts of a wooded park. As such, the perceptual cues to a shift in space may not have been as salient to the participants as they were to the researchers who developed the theoretically motivated analysis to determine situational shifts. On the other hand, the text versions always explicitly marked the shift in narrative space with a spatial proposition (e.g., moved to the pond, moved to the bank of the pond). As such, the spatial shifts were more salient in the text version. One source of narrative complexity in back-­end processes may stem from how salient different narrative content is across media during front-­ end processing and how that impacts back-­end processes. That is, the quality and nature of back-­end processing is closely linked to what happens at the front end (Magliano et al. 2013). While there can be dissociation between front-­and back-­end processes (Loschky et al. 2015), back-­end processes typically operate on the products of the front-­end Complexity in Narrative Comprehension  155

processes. Different types of media may have different affordances that may affect front-­end processing and in turn mental-­model construction at the back-­end. Conclusions about Front-­and Back-­End Processes and Narrative Complexity In this section we discussed differences in front-­and back-­end processes across text, comics, and film. There are important differences in the front-­end processes responsible for information extraction across these media. While there are likely common back-­end processes that support mental-­model construction, the differences in front-­end processes may have nontrivial implications for the information that feeds into the back-­ end processes. More research is needed to explore the implications of the differences in front-­end processes on back-­end processes. Affordances As we suggested earlier, different narrative media may have different affordances (Loughlin et al. 2015; Schnotz 2005). Different media allow a storyteller to make different aspects of the narrative more or less salient, which may have implications that affect the information that is represented in mental models for narratives. Additionally, media differ in terms of the extent that there are multiple sources of information that can be used to convey the story (text, dialogue, static and moving images). Additionally, there may be differences in the extent that experience with a given medium is needed to be able to make sense of narratives conveyed in that medium. In the following section we explore the implications of these issues on narrative complexity. Salience of Situational Content and Narrative Complexity There are arguably differences in the extent that storytellers can make the internal thoughts of characters explicit to the audience in different media. For example, in text, authors can use an omnipresent narrator, indirect speech (i.e., internal monologue of a character), and direct speech (i.e., overt dialogue) to indicate exactly what a character is wanting, thinking, and feeling. Filmmakers, however, have fewer mechanisms to explicitly state internal thoughts and typically rely on dialogue. Filmmakers have other conventions for conveying a variety of psycho156  Magliano, Higgs, and Clinton

logical states (e.g., dream states), but these conventions do not necessarily afford one to explicitly state what is going on in a character’s head. Viewers can infer emotions and goals from behavioral cues or facial expressions, and filmmakers have developed techniques to help viewers to make these inferences. For example, a close-­up of an actor’s face followed by a camera shot positioned to imply the character’s view (called an eyeline match) is a convention used to help make salient a character’s reactions and what they are reacting to (Pudovkin [1929] 1970). It is well documented that we can infer a complex range of emotions from facial expressions (Ekman 1992); thus, we by no means want to suggest that viewers can’t readily infer the internal states of characters. But having to infer internal states versus having them specified explicitly certainly affects how strongly these internal states are represented in memory, albeit this is a largely unexplored issue. While there may be differences across media in terms of the extent that dimensions of a narrative can be made more or less salient, there may also be aspects of the narrative processing that may be less influenced by differences in front-­end processes across media. For example, it has been argued that monitoring shifts in narrative time is an essential aspect of mental-­model construction (Zwaan and Radvansky 1998). There is evidence that readers monitor changes in time more closely than changes in narrative space in the context of text (Zwaan, Magliano, and Graesser 1995). Readers tend to monitor changes in space when they have a high amount of preexisting knowledge of the narrative space, when they have the goal to do so, or when they are rereading a narrative (Zwaan and Radvansky 1998). Magliano, Miller, and Zwaan (2001), however, found evidence that temporal shifts in film are also monitored more closely than shifts in spatial regions, despite the fact that film is a visual medium in which these shifts are explicitly conveyed. They concluded that time is more important for conveying shifts in narrative events than space, regardless of narrative medium. There are at least two reasons why shifts in time may be monitored more closely than shifts in spatial regions. Magliano, Miller, and Zwaan (2001) argued that narrative scenes often depict two sets of concurrent events that are happening in two different locations but are still coherently part of the same scene (e.g., switching between the bridges of two spaceships engaged in a battle). As such, shifts in region do not necessarComplexity in Narrative Comprehension  157

ily convey a shift in narrative episode. In contrast, shifts in time almost always indicate a transition from one narrative episode to the next. As such, shifts in time are more predictive of shifts in story episodes than shifts in spatial regions. Additionally, time may be monitored closely because the temporal distance between events is important for processing causality in narratives (e.g., van den Broek 1990). Inferring causal relationships is important for mental-­model construction (e.g., Zwaan and Radvansky 1998), which in turn makes monitoring time an important source of situational coherence. Multimedia Narratives and Complexity Another aspect of the affordances of narrative media that warrants some consideration is the extent that narratives offer multimedia experiences. Oral and text-­based narratives are unimodal experiences, whereas visual narratives are typically (but not exclusively) multimodal. Sequential visual narratives typically involve both text and images, and dynamic narratives involve language, nonlinguistic sound (part of the story and external to it), and images. While it is well documented that combining images with texts can support learning of the text (Paivio 1990), much less is known regarding the extent that multimodal and unimodal narratives are processed differently in terms of supporting comprehension (i.e., back-­end processes), other than the fact that they will necessarily involve different sets of front-­end processes. Text and visual content can have different relationships with respect to the narrative plot. Neil Cohn (2016) recently proposed a system for identifying the different kinds of relationships that can exist between text and images in terms of narrative structure. Text and images in comic panels can be dominant or coassertive. Dominance happens when one information source carries more informational content than the other with respect to the narrative. Consider a set of panels depicting a conversation between two characters. The images make it clear who is delivering a line, but the narrative plot is likely moved more by the text than the images in these types of panels. In contrast, for action-­oriented scenes the dialogue may be less important to the plot than the images. Coassertive panels occur when both the language and the images support the narrative structure to a relatively equal extent. They can do so redundantly (i.e., convey similar semantic content), or they can con158  Magliano, Higgs, and Clinton

Fig. 11. Sequential two-­panel excerpt from the graphic story The Fantom of the Fair (Gustavson 1939).

vey different narrative content. Consider the two-­panel sequence from Fantoman (Gustavson, 1939) shown in figure 11. Linguistic and visual content would be relatively redundant in panel 1 because they would both convey similar information (i.e., the characters are rising to the surface of the water). In contrast, the linguistic content and images are less redundant in panel 2. The images show Fantoman flying away from the woman he rescued, whereas the text makes clear the motivation underlying why he is leaving in such haste. There is considerable research on how to manage overlap in text and images in informational diagrams (e.g., R. Mayer 2009), but relatively little is known about how these different kinds of relationships affect front-­and back-­end processes. One additional source of complexity that can arise when sequential narratives contain both text and images is that the reader has to decide when to direct attentional resources to processing the text and images. As noted earlier, there is paucity of research exploring eye movements in the context of reading sequential narratives (Chiba et al. 2007; Foulsham, Wybrow, and Cohn 2016), and much more research is needed on this issue. There is some evidence that panels with text are processed more closely than those without, and those without text that follow text rich panels tend to be processed in peripheral vision (Chiba et al., 2007). It may be the case that readers of comics implicitly understand that panels that contain text warrant relatively more attention because of the effort required to read text and that textual information is semantically rich (Nakazawa 2016). Magliano and colleagues (2018a) have recently conducted a study that suggests that differences in how language and images are processed and whether people have control over the timing and the rate at which they consume this information may have implications on memory. In this study participants read a graphic novel and then viewed an animated adaptation of it, or vice versa. They were asked to identify discrepancies across versions while processing the second version. We found that participants more frequently reported discrepancies in the linguistic content (i.e., dialogue or internal monologues) when reading the graphic novel second than when viewing the animated version second. Conversely, discrepancies in the narrative events were more frequently reported when the animated version was viewed second than when the graphic novel was second. 160  Magliano, Higgs, and Clinton

How can one explain these differences? First, if text attracts relatively more attention when processing sequential narratives (Chiba et al. 2007), then linguistic discrepancies may have been more salient when the graphic novel was read on the second exposure because relatively more attention was directed to it than to the visual content. This may have led to the text serving as a more effective retrieval cue for linguistic content from the first exposure (the animated version) than when the film was viewed second. Moreover, when participants felt there was a discrepancy, they could reread. In contrast, when the film was viewed second, once the language was spoken, participants had to rely on what was in their working memory to serve as a retrieval cue, as the format did not afford reprocessing the dialogue. Why were the discrepancies in narrative events more frequently identified when viewing the animated version upon second exposure? This may have to do with the tyranny of film phenomenon that was discussed earlier, which Loschky and colleagues (2015) propose arises from practices of filmmakers that create strong exogenous forces on visual attention. The animation utilizes the same conventions that are intended to make narrative action salient across camera shots (Bordwell 1985). As such, the animated version directed viewers’ eyes to those portions of the screen that were most germane to the narrative plot. When reading the graphic novel, readers have their visual attention divided between text and the images, with the images likely processed relatively more shallowly than the text (Chiba et al. 2007). The animated content served as a relatively stronger retrieval cue for content from the text version than vice versa. This would then lead to a higher likelihood that discrepant events from the two versions would be coactivated in working memory when viewing the animated version second than when reading the graphic novel second. Experience with Media and Narrative Complexity Taking advantage of the affordances across media may be learned, and therefore the level of expertise with narrative conventions in a particular media can be a source of complexity. For example, filmmakers have developed a complex set of conventions for managing space and time to create a coherent narrative film (Bordwell 1985). Stephan Schwan and Sermin Ildirar have conducted a series of studies that suggest that viewers Complexity in Narrative Comprehension  161

need to learn how to respond to these conventions (Schwan and Ildirar 2010; Ildirar and Schwan 2015). For example, one convention for starting a scene involving an interior setting is to show the outside of the building in the first shot of the scene, followed by a shot of an interior room. The intention is to convey that the interior location is inside the building shown in the first shot. But viewers with no experience in film do not interpret these kinds of sequences in this fashion (Schwan and Ildirar 2010). Novice viewers are able to comprehend individual camera shots but have trouble processing conventions that help transition from one shot to another (Ildirar and Schwan 2015). In a similar vein, Cohn and colleagues have argued that sequential narratives follow a language like syntactic structure (Cohn et al. 2012). He argues a strong position that sequential storytelling follows a visual language and has been making an empirical case for this position. It is beyond the scope of this chapter to explain the nature of that language or discuss in detail evidence for a visual grammar, but we refer you to his chapter in the present volume. The point here is that if there is a visual language, then it must be learned to fully support the comprehension of sequential narratives. An interesting question arises as to how different the comprehension of a sequential narrative would be between novice and expert viewers? Are there circumstances where comprehension would be similar between readers of differing level of expertise or very different? Schwan and Ildirar’s (2010; Ildirar and Schwan 2015) recent research on film suggests that there is merit to this line of research. Conclusions about Affordances and Narrative Complexity In this section we conveyed the point that different narrative media and conventions associated with them have different affordances for making aspects of the narrative more or less explicit, which should have implications on mental-­model construction. Consumers of these media may have to learn how to process these conventions. Finally, how we divide our attention between multimodal information may be different across multimedia formats. We want to underscore, however, that considerably more research has to be done to fully understand the implications of the differences in affordances on front-­and back-­end processes. 162  Magliano, Higgs, and Clinton

Identifying Trans-­Symbolic and Symbolic-­Specific Processes In this final section we consider the extent that processing and comprehending narratives across different types of media involves overlapping and unique processes. Sandra Loughlin and colleagues (2015) use the term trans-­symbolic processes to refer to those that are universal and likely operate across media and the term symbolic-­specific processes to refer to those that are unique to a form of media. An argument for trans-­symbolic processes in narrative stems from research stating that comprehension rests on extracting information that leads to a durable mental model (Kintsch, 1988, 1998) that is organized around situational dimensions (space, time, causality, etc.; Gernsbacher 1990; Zwaan and Radvansky 1998). A case for symbolic-­specific processes stems from the fact that there may be differences in conventions or functions of different types of literature that may emphasize the importance of certain types of processing in one form of literature that is not relevant to others. For example, processing alliteration is a critical aspect of evaluating lyrical poetry (Lea et al. 2008), but this is typically not the case in other forms of literary discourse (e.g., narrative proses; Loughlin et al. 2015). There has been relatively little research conducted to identify trans-­ symbolic or symbolic-­specific processes, albeit we have discussed a few studies that have explored the extent that comprehension processes (i.e., aspects of back-­end processes) are similar across different types of media (Baggett 1979; Magliano et al. 2012; Pezdek, Lehrer, and Simon 1984; Zacks, Speer, and Reynolds 2009). As we have discussed earlier, there are obvious differences in front-­end processes across media that reflect symbolic-­specific processes. While there be many back-­end processes that are trans-­symbolic across media (Magliano et al. 2013), the possibility that different front-­end processes can affect the information that feeds back-­end processes warrants more research. In the remainder of this section we discuss challenges to empirically identifying trans-­symbolic and symbolic-­specific processes. Controlling for Narrative Content One challenge to studying this issue is controlling for narrative content so that the only aspect of the materials that differs is media. Some studies have attempted controlling for content (Baggett 1979; Magliano et al. Complexity in Narrative Comprehension  163

2012; Magliano et al. 2018a), whereas others have not done so (Pezdek, Lehrer, and Simon 1984; Zacks, Speer, and Reynolds 2009). While there are certainly circumstances in which controlling for narrative content is not possible, we contend that it is optimal to do so. That said, creating materials in which the narrative content is equated can be very challenging. One approach we have adopted is to take an existing narrative in one medium and create a version of it in another. This is arguably easiest to do with visual formats. Magliano and colleagues (2012) created a text-­ based version of Mercer Mayer’s Boy, Dog, Frog series of wordless picture stories. Consider figure 12, which shows three panels from A Boy, a Dog, and a Frog (M. Mayer 1967) and the corresponding text that we created. The text-­based versions were created to convey the major events depicted in the images (e.g., boy running downhill, net raised, frog scared) rather than all the details of the pictures, which would be strange to describe. But there were some details that we determined to be important to describe because they are germane to understanding what has happened or what will likely happen. For example, it was determined that describing the tree branch in the first picture was important because that sets up the prediction that boy would trip over it, which does happen in the next picture. The important point here is that while some information in the pictures could be readily conveyed, other information either could not be easily described in text or it did not make sense to do so to tell the story. Even when controlling for the content in terms of the major story events, it was impossible to have alternative versions that have complete semantic equivalence. Identifying Appropriate Experimental Tasks What kinds of tasks have been used to explore the extent that there are trans-­symbolic or symbolic-­specific back-­end processes? In her seminal study on trans-­symbolic back-­end processes described earlier, Baggett (1979) used a story-­recall task. This task is sensitive to how memories for narratives are organized, and Baggett (1979) found considerable overlap in the narrative content that was mentioned and how it was structured across both media (text and visual narratives). One approach that we have found useful for studying mental-­model construction in visual and text-­based narratives is an event segmentation task (Magliano, Miller, and Zwaan 2001; Magliano et al. 2012; Magliano 164  Magliano, Higgs, and Clinton

Fig. 12. Example picture and text-­based version of three pictures from A Boy, a Dog, and a Frog (M. Mayer 1967).

and Zacks 2011; Zacks, Speer, and Reynolds 2009). This task involves having participants identify when readers perceive changes in narrative events. Participants are instructed to think of the meaningful events that comprise a narrative and indicate when they perceive a change in the events. The participants can be instructed to do so at different sizes, such as the smallest meaningful events or the largest meaningful events (e.g., Zacks, Speer, and Reynolds 2009). Event-­segmentation judgments are robustly correlated with changes in the situational dimensions specified by the event-­indexing model (Magliano, Miller, and Zwaan 2001; Magliano, Taylor, and Kim 2005, Magliano et al. 2012; Zacks, Speer, and Reynolds 2009). As discussed earlier, Magliano and colleagues (2012) used an event-­segmentation task to demonstrate that viewers and readers segment narratives similarly regardless of medium, indicating that segmentation may be trans-­symbolic. Figure 12 conveys sample materials from that study. Another promising technique is a think-­a loud procedure (Ericsson and Simon 1994; Trabasso and Magliano 1996). In a think-­a loud procedure participants are asked to report whatever thoughts come to mind as they engage in a cognitive task. To our knowledge no study has used Complexity in Narrative Comprehension  165

a think-­a loud approach to explore trans-­symbolic and symbolic-­specific processes as a function of narrative media. Loughlin and colleagues (2015), however, demonstrated the viability of this approach. They had participants think aloud while processing paintings. While they did not compare processing paintings and processing prose within their experiment, they were able to draw on a large literature on processing prose that has used a think-­a loud methodology. They found a high number of trans-­symbolic processes associated with meaning making, but a small set of symbolic-­specific processes that suggest that the front-­end processes that support the back-­end processes may be different from those involved in prose processing. That is, participants articulated a set of processes that involved how they were searching the paintings for content associated with mental-­model construction (e.g., entities and events). These types of searches guided by processes associated with scene perception (e.g., Loschky and Larson 2010) are likely unique to static art (paintings and photographs) and therefore not involved in comprehending text-­based narratives. This finding suggests an important possibility. There may be many trans-­symbolic processes across narrative media, but symbolic-­specific processes on the front end support those processes. For example, we have recently conducted a series of studies that show that viewers of sequential narratives generate bridging inferences to connect explicit narrative events (Magliano et al. 2017; Magliano et al. 2016b), which is a phenomenon well documented in text (e.g., Graesser, Singer, and Trabasso 1994). Specifically, we had participants process the same stories shown in figure 12. We identified episodes in the stories that had a three-­panel structure that showed a character beginning an action (starting to run down the hill), a transition event (tripping), and an end state (falling in pond). We varied whether the transition state was present or not and found that viewing times on the end-­state pictures were longer when the transition-­state picture was missing than when it was present, which suggested that participants generated a bridging inference in the missing condition. A recent study found that there were more eye fixations on the end-­state pictures in the missing condition than the present condition (Magliano et al. 2016a). This suggests that participants were engaging in a visual search of the end-­state pictures to find content that would support the computation of the bridging in166  Magliano, Higgs, and Clinton

ferences that are important for establishing coherence. Readers of text-­ based narratives would obviously have no scene to visually search, albeit they may reread segments of a text to support inference generation. But rereading and visual searches of scenes involve different front-­end processes. As such, many back-­end processes may be trans-­symbolic and fundamental to mental-­model construction, but that does not preclude the possibility that the front-­end processes that support them are unique to different media and thus symbolic-­specific. Conclusions regarding Identifying Common and Unique Processes We contend that the exploration of trans-­symbolic and symbolic-­specific processes in front-­and back-­end processes is essential to explore the complexities of narrative comprehension across media. There are certainly challenges for conducting this research and doing so with the level of control typically required in discourse psychology. Moreover, the field needs to identify tasks that afford exploring this issue. In this section we have identified some challenges and some promising tasks. Conclusion In this chapter we have explored sources of narrative complexity that can arise from different narrative media (text-­based narratives, comics, film, video). At the outset of this chapter, we defined complexities as aspects of a narrative that potentially create challenges for the understander from the perspectives of theories of narrative comprehension. We have argued that different media may lead to different complexities for the understander, despite the fact that back-­end processes may be largely trans-­ symbolic (e.g., Gernsbacher 1990; Magliano, Radvansky, and Copeland 2007; Magliano et al. 2012). These complexities stem from differences in front-­end processes involved in moment-­to-­moment selection and extraction of information (what is happening now). While back-­end processes that help understanders construct a mental model of a narrative may be largely the same across media (e.g., monitoring or inferring change in dimensions of situational continuity such as time, space, and causality), these trans-­symbolic processes operate on the input of front-­ end processes, which may be symbolic-­specific. One aspect of front-­end processes that was discussed involved the extent that the comprehender has control of the pacing of how a narrative is Complexity in Narrative Comprehension  167

processed (Loschky et al. 2015). Static narratives (text and comics) afford more control than dynamic narrative (video, film, and auditory narratives). There are tradeoffs here. On the one hand, control allows understanders to focus their attention on narrative content at their own pace and allows them to reread or reprocess content to support mental-­model construction (Magliano et al. 2016b). On the other hand, carefully crafted film and video directs attention to exactly that portion of the narrative that the producers (writers, directors, editors) deem important (Loschky et al. 2015; Smith 2012), potentially making it less challenging to the viewer. A second source of complexity stems from differences in the affordances of narrative modalities in terms of making information more salient or explicit and available to front-­end processes. What is available to front-­end processes will impact back-­end processes because it will affect the knowledge that is activated and available as the building blocks for a mental model (e.g., Myers and O’Brien 1998). The more salient or explicit the content, the more strongly that aspect of the narrative will be represented in a mental model. A final source of complexity is the extent that processing narratives across media involves trans-­symbolic or symbolic-­specific processes (Loughlin et al. 2015). Differences in complexities across modalities of narrative may arise to the extent that symbolic-­specific processes support trans-­symbolic processes. As an example, bridging inferences (i.e., inferences that connect explicit narrative events) are arguably a trans-­ symbolic back-­end process that may be differentially supported by symbolic-­specific processes (Magliano et al. 2016b). One conclusion that the reader can draw from this chapter is that there is too little research exploring how narrative media affects front-­ and back-­end processes. As such, the implications of the potential sources of complexity raised in this chapter are not well understood at this juncture. We hope that this chapter serves as both a motivation and a guiding resource for researchers to systematically address narrative complexities as they are manifested across different narrative media. The study of the comprehension of visual narratives is a burgeoning interdisciplinary field. While it makes perfect sense to draw on research on texts as a basis for answering and asking questions in this domain, there may be interesting and nontrivial differences in how we process, represent, comprehend, and perhaps appreciate narratives across media. 168  Magliano, Higgs, and Clinton

However, as noted in the previous section, there are nontrivial challenges to conducting this research, controlling for semantic equivalence in narrative content being one of them. We may simply have to accept that one can never completely control for content by virtue of the fact that modalities do have differences in affordances in terms of what content can be easily conveyed. This concern set aside, we have identified several tasks that have proven to be fruitful in exploring complexities in narrative processing across media. As such, we hope that the reader is intrigued enough to work through these challenges. References Baggett, Patricia. 1979. “Structurally Equivalent Stories in Movie and Text and the Effect of the Medium on Recall.” Journal of Verbal Learning and Verbal Behavior 18 (3): 333–­56. Bordwell, David. 1985. Narration in the Fiction Film. Madison: University of Wisconsin Press. Carroll, Patrick J., Jason R. Young, and Michael S. Guertin. 1992. “Visual Analysis of Cartoons: A View from the Far Side.” In Eye Movements and Visual Cognition: Scene Perception and Reading, edited by Keith Rayner, 444–­61. New York: Springer. Chiba, Shinichi, Takamasa Tanaka, Kenji Shoji, and Fubito Toyama. 2007. “Eye Movement in Reading Comics.” In Proceedings of the 14th Annual International Display Workshops, 1255–­58. Red Hook ny: Curran. Cohn, Neil. 2016. “A Multimodal Parallel Architecture: A Cognitive Framework for Multimodal Interactions.” Cognition 146:304–­23. Cohn, Neil, Martin Paczynski, Ray Jackendoff, Philip J. Holcomb, and Gina R. Kuperberg. 2012. “(Pea)nuts and Bolts of Visual Narrative: Structure and Meaning in Sequential Image Comprehension.” Cognitive Psychology 65 (1): 1–­38. Ekman, Paul. 1992. “Facial Expressions of Emotion: New Findings, New Questions.” Psychological Science 3 (1): 34–­38. Ericsson, K. Anders, and Herbert A. Simon. 1994. “Verbal Reports as Data.” Psychological Review 87:215–­51. Foulsham, Tom, Dean Wybrow, and Neil Cohn. 2016. “Reading without Words: Eye-­Movements in the Comprehension of Comic Strips.” Applied Cognitive Psychology 30 (4): 566–­79. Gernsbacher, Morton Ann. 1990. Language Comprehension as Structure Building. Hillsdale nj: Erlbaum. Complexity in Narrative Comprehension  169

Gernsbacher, Morton Ann, Brenda M. Hallada, and Rachel R. W. Robertson. 1998. “How Automatically Do Readers Infer Fictional Characters’ Motional States?” Scientific Studies of Reading 2 (3): 271–­300. Gough, Philip B., and William E. Tunmer. 1986. “Decoding, Reading, and Reading Disability.” Remedial and Special Education 7 (1): 6–­10. Graesser, Arthur C., Cheryl C. Bowers, Brent Olde, Katherine White, and Natalie K. Person. 1999. “Who Knows What? Propagation of Knowledge among Agents in a Literary Storyworld.” Poetics 26 (3): 143–­75. Graesser, Arthur C., Murray Singer, and Tom Trabasso. 1994. “Constructing Inferences during Narrative Text Comprehension.” Psychological Review 101:371–­95. Gustavson, Paul. 1939. The Fantom of the Fair. Public domain comic strip. http://​goldenagecomics​.co​.uk. Ildirar, Sermin, and Stephan Schwan. 2015. “First-­Time Viewers’ Comprehension of Films: Bridging Shot Transitions.” British Journal of Psychology, 106 (1): 133–­51. Kintsch, Walter. 1988. “The Role of Knowledge in Discourse Comprehension: A Construction-­Integration Model.” Psychological Review 95 (2): 163–­82. —. 1998. Comprehension: A Paradigm for Cognition. New York: Cambridge University Press. Komeda, Hidetsugu, and Takashi Kusumi. 2006. “The Effect of a Protagonist’s Emotional Shift on Situation Model Construction.” Memory and Cognition 34 (7): 1548–­56. Lea, R. Brooke, David N. Rapp, Andrew Elfenbein, Aaron D. Mitchel, and Russell Swinburne Romine. 2008. “Sweet Silent Thought: Alliteration and Resonance in Poetry Comprehension.” Psychological Science 19 (7): 709–­16. Loschky, Lester C., and Adam M. Larson. 2010. “The Natural/Man-­Made Distinction Is Made Prior to Basic-­Level Distinctions in Scene Gist Processing.” Visual Cognition 18 (4): 513–­36. Loschky, Lester C., Adam M. Larson, Joseph P. Magliano, and Tim J. Smith. 2015. “What Would Jaws Do? The Tyranny of Film and the Relationship between Gaze and Higher-­Level Narrative Film Comprehension.” Plos One 10 (11): e0142474. Loughlin, Sandra, Emily Grossnickle, Daniel Dinsmore, and Patricia Alexander. 2015. “‘Reading’ Paintings: Evidence for Trans-­Symbolic and Symbol-­Specific Comprehension Processes.” Cognition and Instruction 33 (3): 257–­93. Magliano, Joseph P., James A. Clinton, Edward J. O’Brien, and David N. Rapp. 2018a. “Detecting Differences between Adapted Narratives: Implication

170  Magliano, Higgs, and Clinton

of Order and Modality on Exposure.” In Empirical Approaches to Comics Research: Digital, Multimodal and Cognitive Methods, edited by Alexander Dunst, Jochen Laubrock, and Janina Wildfeuer. 284–­304. London: Routledge. Magliano, Joseph P., John L. Hutson, Adam M. Larson, Karyn Higgs, and Lester C. Loschky. 2016a. “Understanding the Moment-­to-­Moment Processing of Sequential Narratives.” Paper presented at the Thirty-­Eighth Annual Conference of the Cognitive Science Society, Philadelphia, August 10–­13, 2016. Magliano, Joseph P., Kristopher Kopp, Karyn Higgs, and David N. Rapp. 2017. “Filling in the Gaps: Memory Implications for Inferring Missing Content in Graphic Narratives.” Discourse Processes 54 (8): 569–­82. Magliano, Joseph P., Kristopher Kopp, M. Windy McNerney, Gabriel A. Radvansky, and Jeffrey M. Zacks. 2012. “Aging and Perceived Event Structure as a Function of Modality.” Aging, Neuropsychology, and Cognition 19 (1–­2): 264–­82. Magliano, Joseph P., Adam M. Larson, Karyn Higgs, and Lester C. Loschky. 2016b. “The Relative Roles of Visuospatial and Linguistic Working Memory Systems in Generating Inferences during Visual Narrative Comprehension.” Memory and Cognition 44 (2): 207–­19. Magliano, Joseph P., Lester C. Loschky, James Clinton, and Adam M. Larson. 2013. “Differences and Similarities in Processing Narratives across Textual and Visual Media.” In Unraveling the Behavioral, Neurobiological, and Genetic Components of Reading Comprehension, edited by Brett Miller, Laurie E. Cutting, and Peggy D. McCardle, 78–­90. Baltimore: Brookes. Magliano, Joseph P., Matthew T. McCrudden, Jean-­Francois Rouet, and John Sabatini. 2018b. “The Modern Reader: Should Changes to How We Read Affect Research and Theory?” In Handbook of Discourse Processes, edited by Michael F. Schober, Anne Britt, and David N. Rapp, 343–­61. 2nd ed. New York: Taylor and Francis. Magliano, Joseph P., Jason J. Miller, and Rolf A. Zwaan. 2001. “Indexing Space and Time in Film Understanding.” Applied Cognitive Psychology 15 (5): 533–­45. Magliano, Joseph P., Gabriel A. Radvansky, and David E. Copeland. 2007. “Beyond Language Comprehension: Situation Models as a Form or Autobiographical Memory.” In Higher Level Language Processes in the Brain: Inference and Comprehension Processes, edited by Franz Schmalhofer and Charles Perfetti, 379–­91. Mahwah nj: Erlbaum. Magliano, Joseph P., Gabriel A. Radvansky, J. Christopher Forsythe, and David E. Copeland. 2014. “Event Segmentation during First-­Person Continuous Events.” Journal of Cognitive Psychology 26 (6): 649–­61.

Complexity in Narrative Comprehension  171

Magliano, Joseph P., Holly A. Taylor, and Hyun-­Jeong Joyce Kim. 2005. “When Goals Collide: Monitoring the Goals of Multiple Characters.” Memory and Cognition 33 (8): 1357–­67. Magliano, Joseph P., and Jeffrey M. Zacks. 2011. “The Impact of Continuity Editing in Narrative Film on Event Segmentation.” Cognitive Science 35:1489–­517. Mayer, Mercer. 1967. A Boy, a Dog, and a Frog. New York: Dial. Mayer, Richard E. 2009. Multimedia Learning. 2nd ed. New York: Cambridge University Press. McNamara, Danielle S., and Joseph P. Magliano. 2009a. “Self-­Explanation and Metacognition: The Dynamics of Reading.” In Handbook of Metacognition in Education, edited by Douglas J. Hacker, John Dunlosky, and Arthur C. Graesser, 60–­81. New York: Routledge. —. 2009b. “Towards a Comprehensive Model of Comprehension.” In The Psychology of Learning and Motivation, vol. 51, edited by Brian J. Ross, 297–­ 384. New York: Elsevier Science. Myers, Jerome L., and Edward J. O’Brien. 1998. “Accessing the Discourse Representation During Reading.” Discourse Processes 26 (2–­3): 131–­57. Nakazawa, Jun. 2016. “Manga Literacy and Manga Comprehension in Japanese Children.” In The Visual Narrative Reader, edited by Neil Cohn, 157–­84. London: Bloomsbury. Ozyurek, Asli, and Tom Trabasso. 1997. “Evaluation during the Understanding of Narratives.” Discourse Processes 23:305–­35. Paivio, Allan. 1990. Mental Representations: A Dual Coding Approach. New York: Oxford University Press. Pezdek, Kathy, Ariella Lehrer, and Sara Simon. 1984. “The Relationship between Reading and Cognitive Processing of Television and Radio.” Child Development 55:2072–­82. Pudovkin, Vsevolod I. (1929) 1970. Film Technique and Film Acting. New York: Grove. Radvansky, Gabriel A., Rolf A. Zwaan, Todd Federico, and Nancy Franklin. 1998. “Retrieval from Temporally Organized Situation Models.” Journal of Experimental Psychology: Learning, Memory, and Cognition 24 (5): 1224–­37. Rapp, David N., Hidetsugu Komeda, and Scott R. Hinze. 2011. “Vivifications of Literary Investigation.” Scientific Study of Literature 1 (1): 123–­35. Reichle, Erik D., Alexander Pollatsek, Donald L. Fisher, and Keith Rayner. 1998. “Toward a Model of Eye Movement Control in Reading.” Psychological Review 105 (1): 125–­57. Schnotz, Wolfgang. 2005. “An Integrated Model of Text and Picture Comprehension.” In The Cambridge Handbook of Multimedia Learning, edited by Richard E. Mayer, 49–­69. Cambridge: Cambridge University Press.

172  Magliano, Higgs, and Clinton

Schwan, Stephan, and Sermin Ildirar. 2010. “Watching Film for the First Time: How Adult Viewers Interpret Perceptual Discontinuities in Film.” Psychological Science 21 (7): 970–­76. Scott Rich, Shannon, and Holly A. Taylor. 2000. “Not All Narrative Shifts Function Equally.” Memory and Cognition 28 (7): 1257–­66. Smith, Tim J. 2012. “The Attentional Theory of Cinematic Continuity.” Projections 6 (1): 1–­27. Trabasso, Tom, and Joseph P. Magliano. 1996. “Conscious Understanding during Text Comprehension.” Discourse Processes 21 (3): 255–­88. Trabasso, Tom, Paul Van den Broek, and Soyoung Suh. 1989. “Logical Necessity and Transitivity of Causal Relations in Stories.” Discourse Processes 12:1–­25. Van den Broek, Paul. 1990. “The Causal Inference Maker: Towards a Process Model of Inference Generation in Text Comprehension.” In Comprehension Processes in Reading, edited by David A. Balota, Giovanni B. Flores d’Arcais, and Keith Rayner, 423–­35. Hillsdale nj: Erlbaum. Zacks, Jeffrey M., Nicole K. Speer, and Jeremy R. Reynolds. 2009. “Segmentation in Reading and Film Comprehension.” Journal of Experimental Psychology: General 138 (2): 307–­27. Zacks, Jeffrey M., Nicole K. Speer, Khena M. Swallow, Todd S. Braver, and Jeremy R. Reynolds. 2007. “Event Perception: A Mind-­Brain Perspective.” Psychological Bulletin 133 (2): 273–­93. Zwaan, Rolf A. 2004. “The Immersed Experiencer: Toward an Embodied Theory of Language Comprehension.” In The Psychology of Learning and Motivation, vol. 44, edited by Brian H. Ross, 35–­62. New York: Academic Press. —. 2014. “Embodiment and Language Comprehension: Reframing the Discussion.” Trends in Cognitive Sciences 18 (5): 229–­34. Zwaan, Rolf A., Joseph P. Magliano, and Arthur C. Graesser. 1995. “Dimensions of Situation Model Construction in Narrative Comprehension.” Journal of Experimental Psychology: Learning, Memory, and Cognition 21:386–­97. Zwaan, Rolf A., and Gabriel A. Radvansky. 1998. “Situation Models in Language Comprehension and Memory.” Psychological Bulletin 123 (2): 162–­85.

Complexity in Narrative Comprehension  173

7

Structural Complexity in Visual Narratives Theory, Brains, and Cross-­Cultural Diversity Neil Cohn

Narratives are a fundamental aspect of human communication and extend across several domains of expression. While the structure of verbal narratives has been studied extensively, only recently has cognitive science turned toward investigating the structure of drawn visual narratives. Yet drawn sequential images date back at least to cave paintings and appear throughout human history, most popularly in contemporary society in comics. This chapter surveys a research program investigating the structure and cognition of visual narratives across three intersecting domains: narrative structure, cross-­cultural variation, and cognitive psychology and neuroscience. Though targeted at research on visual narratives, this approach is proposed as ultimately applying across domains to various types of narrative systems, including verbal and filmed discourse (Cohn 2013b; Magliano, Higgs, and Clinton, this volume). Visual Narrative Grammar The most prominent beliefs about sequential-­image understanding hold that readers connect the linear meaningful relations between images. The most popular codification of this approach emerged in theorist and artist Scott McCloud’s (1993) proposal of “transitions” between panels—­the image units of a visual sequence—­w ith relations like changes in actions, moments, characters, or scenes. Such an approach is similar to models from psycholinguistics that argue that a reader monitors for various types of semantic information throughout a discourse—­verbal or visual—­such elements as time, space, or characters (Magliano and Zacks 2011; Zwaan and Radvansky 1998). When

174

a reader encounters a change in any of these dimensions, a processing cost is incurred for incorporating that information into an altered mental model of the scene (Zwaan and Radvansky 1998). Such a process likely extends across domains, given modality-­specific affordances (Magliano, Higgs, and Clinton, this volume). Visual Narrative Grammar (vng) has argued that, in addition to updating semantic information, sequential-­image comprehension requires a narrative structure to be understood. This narrative grammar assigns panels to categorical roles and then organizes them into hierarchic constituents at a discourse level of meaning, analogous to how syntactic categories organize words into constituents at a sentence level (Cohn 2013b). That is, because images typically contain more information than individual words, the semantics of the units are closer to whole sentences and thereby operate at a discourse level of meaning. However, despite this variance in the semantic structures, the principles that combine these panels remain similar to those between the sentence (syntax) and narrative levels. In essence, narrative structure acts as a “macro-­level syntax” that belongs to the surface “textbase” that a comprehender accesses to subsequently construct a semantic mental model (van Dijk and Kintsch 1983). vng is not the first approach to narrative structure that has drawn on an analogy with syntactic structure. Other grammatical approaches have proposed formal structures for verbal stories (e.g., Mandler and Johnson 1977) and for film (e.g., Carroll 1980; Metz 1974). But vng differs from these precedents in that it is based on, and integrated into, contemporary theories of linguistics using construction grammar (Culicover and Jackendoff 2005), not traditional Chomskyan phrase-­structure grammars (Chomsky 1965). In a construction grammar, sequencing does not arise out of inserting memorized lexical structures (like words) into rules for phrases, but rather these “rules” exist as schemata stored into long-­term memory as lexical items unto themselves, along with interface rules specifying privileged mappings to semantics (A. Goldberg 1995; Jackendoff 2002). While novel narrative schemata can exist, vng uses several primary sequencing patterns: a canonical narrative schema, a conjunction schema, and a head-­modifier schema (table 1). We address each of these structures in turn.

Complexity in Visual Narratives  175

Table 1. Basic constructional patterns in Visual Narrative Grammar (a) Canonical narrative schema

[Phase X (establisher–­initial–­peak–­release)]

(b) Conjunction schema

[Phase X X1–­X2–­ . . . Xn]

(c) Head-­modifier schema

[Phase X (modifier–­X–­modifier)]

First, vng argues that the semantic cues within the image content of panels can map to narrative categories, which are organized into a canonical narrative schema. This schema is similar to, albeit more operationalized than, traditional notions of narrative arcs (e.g., Freytag 1894; see Cohn 2013b for review). These basic narrative categories include the following: Establisher (E) sets up an interaction without acting on it, often as a passive state; Initial (I) initiates the tension of the narrative arc, prototypically a preparatory action or a source of a path; Peak (P) marks the height of narrative tension and point of maximal event structure, prototypically a completed action or goal of a path but also often an interrupted action; Release (R) releases the tension of the interaction, prototypically the coda or aftermath of an action. These descriptions of narrative roles outline their prototypical correspondences to meaning, that is, how semantic content (the visual cues within images) may influence a panel’s structural role in a sequence. Nevertheless, identification of a narrative category uses both a panel’s bottom-­up content and its top-­down context in a global sequence (Cohn 2013b, 2014). Syntactic categories are assigned in a similar way: though syntactic categories (like nouns and verbs) prototypically correspond to the semantics (like objects and events) of words (Jackendoff 1990), they also rely on context within a sentence. For example, the word dance (semantically, an event) can play a role either as a noun (the dance) or a verb (they dance) depending on context. The basic narrative schema in vng, as in table 1, thus places these narrative categories into a narrative constituent in this particular order. In actualization not all constituents must contain all categories, meaning 176  Cohn

Fig. 13. A narrative sequence with two narrative constituents. Created by the author.

that most elements are nonobligatory (as notated by parentheses). Only peaks are marked as obligatory, because they motivate a sequence as its “head.” However, peaks too can be omitted under specific constrained, inference-­generating contexts (Cohn and Kutas 2015; Magliano et al. 2015, 2016). Consider figure 13: it begins with a boxer, who reaches back to punch an adversary. This preparatory action is prototypical of an initial. The full punch occurs in the next panel, a peak in relation to that initial. An establisher then resets the actions in panel 3, by setting up a new situation with the boxers passively standing facing off again. Another initial in panel 4 again shows a preparatory action. The subsequent peak in the penultimate panel does not depict a completed action (as in the second panel) but rather shows an interruption of the other boxer’s action: he slips. In the final panel release shows the coda of this action, with the victor standing over his opponent. A first layer of complexity in sequential images can be captured by vng because narrative categories apply both to individual images and to constituents of images. If taken as a surface string, the narrative roles in this sequence (i-­p -­e -­i -­p -­r ) do not conform to the canonical narrative schema. But combining its segments into constituents allows for groupings that introduce complexity into a narrative. These constituents then also play narrative roles relative to one another at a level above that of individual panels (i.e., groupings of panels). Figure 13 uses two constituents: the first two panels form an initial that together set up a peak constituent of the remaining four panels. Internally, each constituent maintains the caComplexity in Visual Narratives  177

Fig. 14. Different basic narrative schemata within Visual Narrative Grammar. Created by the author.

nonical narrative schema, with peaks forming the “heads” that motivate the primary meaning of their superordinate constituent (i.e., each grouping is an expansion of its peak, indicated by double-­barred lines). Thus, both individual panels and whole constituents take on narrative roles. This recursion also can extend further upward, since the principles guiding short sequences also apply to higher plot-­level narrative structures. An arc is simply a maximal node or a constituent that plays no role in a larger sequence. Other basic constructs in vng capture further complexity of sequenc178  Cohn

es by elaborating on the canonical narrative schema. For example, conjunction allows categories to repeat within a constituent of the same category (Cohn 2013b, 2015), similar to how syntactic conjunction repeats grammatical categories (like multiple nouns in the noun phrase “the butcher, baker, and candlestick maker”). Thus, narrative conjunction also repeats narrative categories within a common constituent. For example, figure 14b uses conjoined panels at the outset of the sequence. The opening panels depicting the same information as the single first panel in figure 14a suggests that they all function to introduce the scene as establishers (and, indeed, the three panels in 14b could be substituted for the single panel in 14a). But because each character is now individuated, nothing shows us that they occupy the same spatial location (as in the panel in figure 14a). Thus, this spatial environment must now be inferred across these panels. This type of construction is called environmental-­ conjunction or e-­conjunction since it is a conjunction (repetition of panels playing the same narrative role) that leads to an inference of a broader spatial environment (notated with subscript e). Scene construction is only one type of meaningful relation possible using conjunction. The conjunction schema specifies only that narrative categories repeat within a constituent, and thus it allows for various semantic mappings to this narrative schema (Cohn 2013b, 2015). Figure 15 depicts several semantic correspondences for a three-­panel conjunction constituent, all initials. The left tier of figure 15 shows different three-­ panel sequences that can operate as conjoined initials of (a) actions or events (a-­conjunction), (b) characters within a scene or environment (e-­ conjunction), (c) parts of a single entity or character (n-­conjunction), or (d) disparate semantically associated elements (s-­conjunction). Together the conjoined images in the left tier are semantically equivalent to (and may create inferences of) the single panels in the right tier. In other words, the single nonconjoined image in the right tier can substitute for the three conjoined images in the left tier (and this substitution serves as a diagnostic test for assessing conjoined panels). In addition, because conjunction is recursive, these forms can also embed within one another. For example, the three panels in figure 15c could replace the first panel in 15b, which would create an n-­conjunction constituent embedded within an e-­conjunction constituent. Thus, the same narrative conjunction can have several types of semantic mappings. Complexity in Visual Narratives  179

Fig. 15. Different ways that narrative conjunction uses repeated narrative categories (here “initials”) to show various semantic information (actions, characters in a scene, parts of an individual, or semantically associated elements). This information could also be framed by a single image (right tier). Created by the author.

vng can also characterize complexity that occurs when panels modify other panels. The third panel in figure 14c depicts the same information as its preceding panel, zooming in on only the puncher’s fist. This is a refiner (Cohn 2013a, 2015), which modifies the information in another “head” panel (again, double-­bar lines) using a narrower viewpoint of the same information. Both of these panels play the same role in an overall narrative arc (in figure 14c, as an initial). Unlike conjunction, 180  Cohn

the narrative role here is not distributed across units, but rather the refiner modifies the head panel with added focus, while the head retains its wider viewpoint and more fundamental role in the sequence (as in 14a). Refiners can go either before or after their head. Because refiners modify their head, they should be able to be deleted without recourse on the sequence (unlike in conjunction). If a head is deleted, the modifier takes on the role of the head (e.g., in figure 14c the refiner would become the initial). This is analogous in language to phrases like “I’ll take the white,” where the adjective white takes on the role of a noun in the otherwise more complete white wine. Structurally, these constructs are similar in that both are grammatical modifiers that take the role of their heads when those heads are deleted, despite involving different types of, and levels of, semantics. Both conjunction and refiners are basic schemata within vng (table 1) that can expand sequences into more complicated structures, often as regularized patterns. One such pattern arises in figure 16a, where the alternation between two characters results in an a-­b -­a -­b pattern (see page 183). This surface pattern is composed of sets of conjoined panels (Cohn 2013a), such that each pairing forms a constituent using e-­conjunction (i.e., [a-­b ]-­[a-­b ]). In figure 16a the first pairing creates an establisher constituent and the second pairing forms an initial constituent. Again, this can be confirmed because each pairing of panels can be substituted by the single panels in figure 14a (Cohn 2013b, 2015). This pattern is a subtype of the “crosscutting” or “multitracking” (Bateman and Schmidt 2012; Bordwell and Thompson 1997) found in films, as well as in drawn visual narratives (Cohn 2013a). Another pattern uses both conjunction and refiners. In figure 16b the refiner is separated from the head it modifies by an intervening panel. This intervening panel unites with the head using e-­conjunction, forcing the refiner to connect across a distance. This pattern is called refiner projection, because the refiner is “projected” away from its head (Cohn 2013a). Thus, the refiner must connect across a distance rather than with its juxtaposed images. The canonical narrative schema, conjunction schema, and head-­ modifier schema constitute three core narrative patterns and are stored in long-­term memory as “constructions” in line with abstract principles of combination found at the syntactic level, as detailed in linguistic theoComplexity in Visual Narratives  181

ries of construction grammar (Culicover and Jackendoff 2005; Jackendoff 2002). However, as in human languages, narratives do not use just basic schemata. Rather, thousands of constructions appear in regularized ways that employ or depart from those canonical patterns in spoken language. Thus, vng allows for conventionalized constructional patterns that may or may not use these basic schemata, so long as they are systematic within or across authors. In sum, vng posits a basic narrative schema composed of categorical roles, which can expand into hierarchic constituents because of its recursive nature. This narrative schema can also be expanded by basic modifying schemata (conjunction, refiners), which alter and enrich the framing of attention on a scene. Together these schemata allow for complex patterning like alternation and refiner projection. Note that in all patterns presented in figures 13–­16, the basic narrative arc persists at the top level of all structures. Formal complexity merely expands from this basic structure. Thus, narrative structure does not simply use a uniform process of updating semantic relations across units of a (visual) discourse (Magliano, Higgs, and Clinton, this volume) but rather stitches together structured patterns in complex hierarchic embedding that frame and organize that meaning. Given the formal complexity that can arise in the structure of a narrative, this may also have consequences on processing and comprehension. Greater complexity in the structure of a narrative grammar—­for example, center-­embedded clauses, conjunction, and refiners—­may in turn yield more challenging comprehension (e.g., Magliano, Higgs, and Clinton, this volume). Such comprehension differences are foreshadowed by the growing empirical work on visual narrative processing but would reflect similar findings at the syntactic level, where greater structural complexity demands processing costs. Typological Complexity vng is embedded within Visual Language Theory, which posits that graphic systems manifest in different “visual languages” used by cultures around the world (Cohn 2013a). Just like verbal languages differ around the globe, so do visual languages, and this hypothesis of cross-­ cultural diversity includes their grammars. Thus, U.S. mainstream superhero comics are drawn in an American Visual Language (specifical182  Cohn

Fig. 16. Narrative patterns created by the interaction between narrative schemata within Visual Narrative Grammar. Created by the author.

ly the Kirbyan dialect), which are predicted to differ from the Japanese Visual Language used to create manga. These visual languages combine with writing to create a larger multimodal discourse (Cohn 2016b). In that vng outlines a narrative grammar in the structure of visual languages, we can now investigate: (1) Do all systems of sequential images use the same narrative grammar, or is there cross-­cultural diversity? (2) Are there general trends for complexity in narrative grammars that transcend diversity in individual cultures? The first question relates to variation between the systems used by different cultures’ comics, and indeed claims to this extent go back several decades. McCloud (1993) observed that differences between panel-­to-­ panel semantic relations arose between U.S. comics and Japanese manga, but not between comics from the United States and Europe. In particular, McCloud noted greater amounts of character changes and fewer tempoComplexity in Visual Narratives  183

ral changes in Japanese manga than European and U.S. comics. Similarly, we also found differences between the structure of manga and comics from the United States with regard to how a scene is framed (Cohn 2011; Cohn, Taylor-­Weiner, and Grossman 2012), which can have consequences on narrative structure, as in e-­conjunction or refiners. The second question pertains not to cross-­cultural differences but to the overall trends for structural patterning across narrative systems. We can characterize our aforementioned narrative patterns into three levels of complexity. First, the simplest structures depict a full scene in each panel (a macro viewpoint) with narrative state changes between each panel, as in figures 14a and 13. This basic narrative progression (bnp) should be the most similar to the iconic perception of viewing events and makes no modulation on framing or narrative pacing. It thus constitutes level 1 of narrative complexity. Level 2 modifies this structure, as in conjunction and refiners. These patterns still use basic schemata within the narrative grammar, but they add complexity beyond a simple progression of a fully depicted scene. Finally, level 3 consists of patterns that combine these modifiers: alternation and refiner projection. These patterns arise from the interaction between modifying structures and thus use more complexity than the basic schemata alone. If narrative patterns arise systematically for different visual languages across the world, it provides evidence that authors store structures in memory for a narrative grammar. In vng all three levels of complexity are, in fact, patterns encoded in memory, be they the basic schemata (table 1) or constructions derived from them. This position contrasts with theories of sequential images focusing on the semantic relationships between images, where an ongoing inductive process updates meaning without using stored schematic structures in memory (Bateman and Wildfeuer 2014; Magliano and Zacks 2011). Under this semantic panel-­to-­panel view, narrative arises because of “storytelling choices” (e.g., McCloud 2006), and differences between cultures must amount to broader “cultural” factors operating on those choices (McCloud 1993). However, if narrative patterns vary in a consistent way across different cultures, then it would provide evidence for a narrative grammar operating in the minds of these authors (which would thereby be accessed by readers of those works). To examine these issues, I drew a sample from the Visual Language 184  Cohn

Research Corpus (n.d.) examining these patterns in 10,521 panels across ninety comics from various cultures (see table 2). I here report on a sample of ten books each from three populations of comics from three parts of the world: Europe (Sweden, France, Germany); the United States (mainstream, indie, original English language [oel] manga); and Asia (Japan, Korea, China). These data collapse across most distinctions of genres and demographics explored in other publications. The exception to this is variance in books from the United States, which contrasts both publishing genres (mainstream, indie) and oel manga. oel manga are comics created by English speakers but ostensibly drawn using the Japanese Visual Language from manga. Inclusion of this sample provides a way to investigate whether the structures used by these authors may be closer to their culture (United States) or their adopted visual language (Japanese Visual Language). Many authors from the contemporary U.S. indie genre are also influenced by manga, though they may not claim that label overtly (Mazur and Danner 2014). Table 2. Data from books analyzed from the Visual Language Research Corpus Group

Dates

Total panels

Panels per page

Swedish

1980–­2011

893

6.91

French

1992–­2014

1607

7.42

German

1987–­2009

1525

5.63

U.S. mainstream

1990–­2014

1016

4.75

U.S. indie

2002–­2014

1336

5.36

oel manga

1991–­2006

1101

4.57

Japanese

2003–­2014

960

5.28

Korean

1987–­2010

837

3.90

Chinese

2002–­2015

1246

4.90

10,521

5.41

Total

Note: Ten books were analyzed from each group.

This corpus analysis of ninety comics is the largest reported in any study of the structure of drawn visual narratives. Nevertheless, data collection on this larger project work is ongoing, and we consider these data as merely preliminary. These reported data were coded by eight indepenComplexity in Visual Narratives  185

dent researchers, with repeated annotations for approximately 70 percent of the books. Before contributing to the corpus, all coders needed to reach a threshold of over 85 percent agreement with other annotations performed on preliminary practice examples. Results from this corpus analysis are provided in figure 17. In general, the overall proportion of patterns decreased across levels (note the differences in scale in figure 17a–­c), with the basic narrative progression constituting the most used of all patterns. This finding reinforced that this pattern provides a fundamental base for most visual narratives. As evident in figure 17a, narrative systems in Europe used the bnp more than those from the United States, which in turn used it more than books from Asia or those imitative of Japanese manga (Japanese, Chinese, Korean, oel manga). The lower proportion of the bnp for Japanese manga and those imitative of Japanese manga appeared to coincide with increases in level 2 patterns (figure 17b) and level 3 patterns (figure 17c) for these same books. Within level 2 e-­conjunction was used more than refiners (17b), and within level 3 alternation was used more than refiner projection (17c). The overall similarity between both U.S. mainstream books and European comics echo McCloud’s (1993) observation of similarities between semantic panel relations in U.S. and European comics, despite fairly independent narrative traditions (Mazur and Danner 2014). These works were characterized by greater quantities of basic narrative progression and fewer complex patterns from level 2 and level 3. Exceptions arise in the greater usage of alternation by German comics. These patterns of European and U.S. narrative styles distinctly differed from Japanese manga and works imitative, inspired, or influenced by them (Korean, oel manga, U.S. indie). Again, this aligns with McCloud’s (1993) observation that semantic panel relations differed between Japanese manga and European and U.S. comics. In particular, McCloud found that manga used greater numbers of changes in characters, consistent with our findings of more e-­conjunction in manga. In fact, in the Japanese manga from our sample, e-­conjunction appears nearly at the same proportion as the bnp. These observations also confirm the interpretations drawn from corpus studies of the framing of panels (Cohn 2011; Cohn, Taylor-­Weiner, and Grossman 2012) that increased proportions of mono (single character) panels implied more e-­conjunction (Cohn 2013a). 186  Cohn

Fig. 17. Mean panels per book between groups for each narrative pattern across the three levels of complexity. Note the difference in scales between each level. Created by the author.

In general, works imitative of, or inspired by, Japanese manga used the bnp less often and used complex narrative patterns more often. These works often share surface features in their drawing style of graphic structure. However, these data suggest that they share structures extending beyond surface forms to consistent narrative grammars. Whether acquisition of this Japanese Visual Language grammar by non-­Japanese authors (Korean manhwa, oel manga) is explicitly or implicitly learned is an important follow-­up to exploring this consistency. The trends shown by oel manga were ostensibly created using the Japanese Visual Language by English speakers outside of Japan. These works demonstrated patterns closer to books using their intended visual language (Japanese manga and Korean manhwa) than to books from their culture of origin (the United States). The similarities of these works’ narrative patterns to those in other manga-­inspired books (Korean) indicate that narrative grammars are not contingent on cultural divisions (i.e., Asian vs. European or American) but rather are a facet of the visual languages themselves. That is, despite their varying countries of origin, Japanese manga, Korean manhwa, and oel manga all appear to use the Japanese Visual Language grammar or at least variants (dialects) of it. In addition, U.S. indie comics use patterns trending in the direction of Japanese manga, away from U.S. mainstream comics. This perhaps validates their status as a categorically different type of visual narrative in the United States. Their similarities toward books drawn in the Japanese Visual Language is also noteworthy, as the contemporary indie movement has emerged in the wake of the mass importation of manga into the U.S. comic market in the 1990s (Mazur and Danner 2014; W. Goldberg 2010), and many authors may have been influenced by manga more than U.S. mainstream comics. Altogether, these preliminary results suggest patterned ways that different visual languages use narrative structures. Just like the diversity of grammatical structures found in the syntax of the world’s spoken languages, the overall trends between comics of the world likely reflect characteristics of particular narrative grammars for these diverse visual languages. Such findings go against the idea that narratives are structured simply through uniform semantic juxtapositions, whereby no patterned information would be encoded in long-­term memory (Bateman and Wildfeuer 2014; Magliano and Zacks 2011). Such cultural trends—­ 188  Cohn

Fig. 18. Correlation coefficients between each narrative pattern, collapsing across groups. The dashed lines represent a negative correlation; the solid lines represent a positive correlation, and the gray lines are not significant. The line weight is equal to r-­value. *p