Beyond the Screen: Transformations of Literary Structures, Interfaces and Genres [1. Aufl.] 9783839412589

Table of contents :
Contents
Acknowledgments
Introduction
Part One. Beyond the Screen: Reconfiguring Space and Time in Literature
Performance and the Emergence of Meaning
Reassembling the Literary. Toward a Theoretical Framework for Literary Communication in Computer-Based Media
Epistemology of Disruptions. Thoughts on the Operative Logic of Media Semantics
RFID: Human Agency and Meaning in Information-Intensive Environments
Memory and Motion. The Body in Electronic Writing
Event and Meaning. Reading Interactive Installations in the Light of Art History
Literature between Virtual, Physical and Poetic Space
Why Digital Literature Has Always Been “Beyond the Screen”
From Concrete to Digital. The Reconceptualization of Poetic Space
The Gravity of the Leaf. Phenomenologies of Literary Inscription in Media-Constituted Diegetic Worlds
Beyond the Complex Surface
Hyperlinking in 3D Interactive, Multimedia Performances
Entering Urban Space: Using Locative Media for Literature
Framing Locative Consciousness
Walk This Way. Mobile Narrative as Composed Experience
Locative Narrative, Literature and Form
A Town as a Novel. An Interactive and Generative Literary Installation in Urban Space
The Global Poetic System. A System of Poetic Positioning
Part Two. Beyond Genre: Perspectives of Literariness in Computer-Based Media
“No Preexistent World” On “Natural” and “Artificial” Forms of Poetry
How to Construct the Genre of Digital Poetry. A User Manual
The Reader, the Player and the Executable Poetics. Towards a Literature Beyond the Book
Beyond Play and Narration. Video Games as Simulations of Self-Action
Part Three. Beyond the Library: Preservation, Archiving and Editing of Electronic Literature
Archivability of Electronic Literature in Context
On Reading 300 Works of Electronic Literature. Preliminary Reflections
Classification vs. Diversification. The Value of Taxonomies for New Media Art
Dispersal and Renown. An Investigation of Blogs, Listservs and Online Journals
Digital Editions in the Net. Perspectives for Scholarly Editing in a Digital World
Contributors

Citation preview

Jörgen Schäfer, Peter Gendolla (eds.) Beyond the Screen

The series “Medienumbrüche | Media Upheavals” is edited by Peter Gendolla.

Jörgen Schäfer, Peter Gendolla (eds.)

Beyond the Screen Transformations of Literary Structures, Interfaces and Genres

Medienumbrüche | Media Upheavals | Volume 44

Bibliographic information published by the Deutsche Nationalbibliothek The Deutsche Nationalbibliothek lists this publication in the Deutsche Nationalbibliografie; detailed bibliographic data are available in the Internet at http://dnb.d-nb.de

© 2010 transcript Verlag, Bielefeld All rights reserved. No part of this book may be reprinted or reproduced or utilized in any form or by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying and recording, or in any information storage or retrieval system, without permission in writing from the publisher. Cover layout by: Kordula Röckenhaus, Bielefeld Cover image: Noah Wardrip-Fruin et al.: Screen (2002); © 2010 Noah Wardrip-Fruin Copy editing by: Jörgen Schäfer and Patricia Tomaszek Layout & Typeset by: Georg Rademacher, Jörgen Schäfer, Patricia Tomaszek Druck: Majuskel Medienproduktion GmbH, Wetzlar ISBN 978-3-8376-1258-5

Distributed in North America by

Transaction Publishers Rutgers University 35 Berrue Circle Piscataway, NJ 08854

Tel.: (732) 445-2280 Fax: (732) 445-3138 for orders (U.S. only): toll free 888-999-6778

Contents Acknowledgments ................................................................................................... 9 Jörgen Schäfer and Peter Gendolla Introduction ............................................................................................................11

Part One Beyond the Screen: Reconfiguring Space and Time in Literature Performance and the Emergence of Meaning Jörgen Schäfer Reassembling the Literary Toward a Theoretical Framework for Literary Communication in Computer-Based Media.......................................................................................25 Ludwig Jäger Epistemology of Disruptions Thoughts on the Operative Logic of Media Semantics.......................................71 N. Katherine Hayles RFID: Human Agency and Meaning in Information-Intensive Environments ..............................................................95 Maria Angel and Anna Gibbs Memory and Motion The Body in Electronic Writing ...........................................................................123 Roberto Simanowski Event and Meaning Reading Interactive Installations in the Light of Art History...........................137 Literature between Virtual, Physical and Poetic Space Andrew Michael Roberts Why Digital Literature Has Always Been “Beyond the Screen” ............153 Anna Katharina Schaffner From Concrete to Digital The Reconceptualization of Poetic Space ...........................................................179

John Cayley The Gravity of the Leaf Phenomenologies of Literary Inscription in Media-Constituted Diegetic Worlds .................................................................... 199 Noah Wardrip-Fruin Beyond the Complex Surface........................................................................... 227 Dene Grigar Hyperlinking in 3D Interactive, Multimedia Performances ................... 249 Entering Urban Space: Using Locative Media for Literature Francisco J. Ricardo Framing Locative Consciousness................................................................... 261 Rita Raley Walk This Way Mobile Narrative as Composed Experience....................................................... 299 Jeremy Hight Locative Narrative, Literature and Form...................................................... 317 Jean-Pierre Balpe A Town as a Novel An Interactive and Generative Literary Installation in Urban Space .............. 331 Laura Borràs Castanyer and Juan B. Gutiérrez The Global Poetic System A System of Poetic Positioning............................................................................ 345

Part Two Beyond Genre: Perspectives of Literariness in Computer-Based Media Peter Gendolla “No Preexistent World” On “Natural” and “Artificial” Forms of Poetry ................................................ 365 Friedrich W. Block How to Construct the Genre of Digital Poetry A User Manual........................................................................................................ 391

Giselle Beiguelman The Reader, the Player and the Executable Poetics Towards a Literature Beyond the Book...............................................................403 Jochen Venus Beyond Play and Narration Video Games as Simulations of Self-Action.......................................................427

Part Three Beyond the Library: Preservation, Archiving and Editing of Electronic Literature Beat Suter Archivability of Electronic Literature in Context........................................443 Joseph Tabbi On Reading 300 Works of Electronic Literature Preliminary Reflections ..........................................................................................465 Katja Kwastek Classification vs. Diversification The Value of Taxonomies for New Media Art ..................................................503 Ravi Shankar Dispersal and Renown An Investigation of Blogs, Listservs and Online Journals ................................521 Fotis Jannidis Digital Editions in the Net Perspectives for Scholarly Editing in a Digital World .......................................543 Contributors ..........................................................................................................561

Acknowledgments This book is based on the conference Beyond the Screen: Transformations of Literary Structures, Interfaces and Genres that took place on November 20-21, 2008 at the University of Siegen (Germany). We would like to thank the German Research Foundation (DFG) for its funding of our research project on Net Literature, which is part of the research center Medienumbrüche (‘Media Upheavals’) and the Alexander von Humboldt Foundation that—in the context of its TransCoop Program—generously supported the German-American research collaboration that enabled us to establish many transatlantic contacts and thereby made this event possible. This volume has benefited from the work of many people to whom we are immensely grateful for their efforts regarding this volume. We particularly owe thanks to Brigitte Pichon and Dorian Rudnytsky, who translated with expertise and accuracy several texts into English and checked several others for their linguistic correctness and to Patricia Tomaszek, who provided invaluable assistance in unifying quotations and bibliographic information. We are also indebted to her for proof-reading the manuscript and for her assistance in finalizing the typesetting of this book. We are also very grateful to Georg Rademacher, who has expertly finalized the graphic elements and the layout of this book. And last but certainly not least, our special thanks go to Noah WardripFruin for giving permission to use a capture of his Cave installation Screen on the book cover. Siegen, February 2010 Jörgen Schäfer and Peter Gendolla

9

Jörgen Schäfer and Peter Gendolla

Introduction Regarding our specific interest in “electronic literature,” “digital literature,” “net literature” or whatever the subject matter might be called, this is the fourth book in a series of publications resulting from German-American cooperations that have addressed the subject from different perspectives and in consecutive steps. The first book, The Aesthetics of Net Literature (2007), had already made clear that it seems necessary to make revisions in the traditional models analyzing literary communication. In particular, the triad of “author,” “work,” and “reader” had to be extended into the technical aspects of media: Literary processes emerge from techno-social networks (Gendolla and Schäfer). The second book, Literary Art in Digital Performance (2009), very deliberately concentrated on the close reading of individual literary projects (Ricardo), whereas Reading Moving Letters (2010) addresses research approaches, institutional and curricular frames as well as didactic questions in various scholarly environments (Simanowski, Schäfer and Gendolla). The traditional models of literary communication in computer-aided and networked media have undergone considerable changes; notably they have to be supplemented with the “autonomous” part of the technical medium and, for quite a while, also other output and input devices than monitor, keyboard and mouse have to be included. But what does this easily said proposal really mean? In what way does the reading of a printed text differ from the reception of a computer-generated poem, of a hyperfiction, of a collaboratively written story or—which is at the center of our interest in this book—of literary works or processes in spatially defined media like the Cave (e.g., Noah Wardrip-Fruin et al.’s Screen). We have to include into this question also Locative Narratives where recipients are navigated by GPS through real space reading fragments of a story. And—despite all changes in the dispositives of mediality brought on by the present upheaval in media—in what way do all these projects carry on the long history of literature that has already survived several media upheavals? Therefore, developing a theory of literature in computer-based media requires critically recapitulating the basic terminology of literary studies as well as the history of reception of established literary theories and their usefulness, for example, of formalism or reader-response criticism. We believe that this cannot be avoided in debating our issues if we do not want to remain within the realm of text or communication theories—as unfortunately is often the case— which may be used just as well in the analysis of non-literary communication

11

Jörgen Schäfer and Peter Gendolla | Introduction

or if we do not want to continue using computer sciences or software design approaches that persist in drawing on aspects of programming technology. Literature in computer-based media, as well as every literary text, activates expectations that then are broken and continued in imaginary form. Here the attributions of meaning have to be conceived of as interactive processes between man and machine. Especially when such literature expands into the physical realm are the ruptures activated technically and therefore they are by no means only cognitively processed. Rather, they invite direct physical activity with the sign processes, i.e., they incite the immediate “writerly continuation” with the whole body, thereby filling the “gaps.” Therefore, the discussions regarding the prevalent literary theories are by no means finalized. On the contrary, they still need to be reappraised much more closely regarding the following problems: -

whether and to what extent they are able to explain the practical integration of established literary ideas, subjects, forms and procedures into the intentions and activities of authors/readers;

-

questions of the technical “support,” translations, and automations addressed in a narrative, or poetical manner or through staging them;

-

and in what way they have to be revised or dismissed.

1

Beyond the Screen: Reconfiguring Space and Time in Literature

This correlation of performativity, performance and meaning is the focus of the essays in Part One. Jörgen Schäfer, in his essay, outlines a theoretical framework for the analysis of literature in computer-based media, especially in spatial environments. He links approaches from literary studies, particularly from reader-response theories, with considerations from the Actor-Network Theory as well as from semiotics, computer and cognitive sciences, thereby confronting questions of the changes in media dispositives and those of the authors’/ readers’ reactive possibilities with that of the literariness of the observed objects. In doing so, he is referring to the views of the distinguished linguist Ludwig Jäger, among others, who for the first time puts his ideas up for discussion in the electronic literature community. In cultural studies and their diverse disciplines, he argues, it is a familiar phenomenon that media refer to each other and that they intramedially allude to themselves in recursive loops. Cultural semantics are generated, conserved, disrupted and modified in a field in which communication unfolds as a symbolic game of interacting and interwoven me-

12

Jörgen Schäfer and Peter Gendolla | Introduction

dia, as an assembly of different methods of “cultural reconceptualization” (Manovich 47). Behind these diversities of communicative processes in oral, written, and visual media—no matter how varied they are regarding their medial and aesthetic aspects—we can identify a fundamental approach of cultural semantics that we can describe as logic of transcriptive reference. Jäger’s contribution, going beyond the idea that transcription is a fundamental procedure of cultural semantics, reveals some of the principles that underlie the practices of cultural reconceptualizations attempting to show that and how they are characterized by an epistemology of disruptions. N. Katherine Hayles to a certain extent links with these thoughts, as her article focuses on the impact of so-called “ubiquitous computing” on human cognition. She analyzes the consequences of “reality mining” by RFID (radio frequency identification) tags that are currently being embedded in product labels, clothing, credit cards, and the environment. The amount of information accessible through and generated by RFIDs is so vast that it may well overwhelm all existing data sources and become, from the viewpoint of human time limitations, essentially infinite. Hayles argues for understanding the constitution of meaning as a “multi-layered distributed activity,” as a result of “context-specific processes of interpretation that occur both within and between human and non-human cognizers.” This is exemplified by Maria Angel and Anna Gibbs who explore the new materialism of the corporeal body in electronic writing and online environments. They argue that electronic environments have a strong relationship with affective modes of communication highlighted by their appeal to sensory novelty through technological innovation—new media platforms proliferate the potentials for combining visibility with aural and tactile modes. Their essay argues for a new materialism in electronic culture, one that has serious implications for the way that we understand memory. Roberto Simanowski demonstrates in a close reading of two interactive installations that they do not simply create an event as “a period of time to be lived through” (Bourriaud 15). Looking at Still Standing by Bruno Nadeau and Jason Lewis and Zachary Booth Simpson’s Mondrian, Simanowski maintains that these pieces do not only offer two different concepts of the interactors’ actions and hence body experiences; they also engage in a very difficult way with the issues of inter- and transmediality and thereby refer to the history of the avant-garde movements of the early 20th century. These essays illustrate that the recipient of literature in computer-based and networked media cannot just sit in front of a computer screen and use his mouse and keyboard for navigating through the text. On the one hand, mobile media devices such as mobile phones, GPS and PDAs are being used, while on the other literature finds its way into mixed reality environments in museums, gal-

13

Jörgen Schäfer and Peter Gendolla | Introduction

leries or research labs. Thus, new combinations of physical, virtual and symbolic spaces emerge. We might even say metaphorically that literature, after it has passed through the needle’s eye of book culture, seems to revert to the multimodal patterns of action and forms of antiquity, of the Middle Ages or of the Renaissance: to ritualized space-body-text stagings, to forms of the Living Theater, or to spontaneous street happenings. This, however, is taking place on a completely changed media-technological level which involves the whole body in the media activity. When sensors and effectors mediate between the recipients’ movements of the body, their gestures and mimic and the linguistic signs, then the spaces of the “real” world and the poetic or narrative spaces of literature enter into a completely new relationship with each other. The second section is opened by Andrew Michael Roberts, who demonstrates that digital literature has always been beyond the screen. In many of the practices and framing ideas of electronic literature, he identifies recurrences of key conceptions of modernism and postmodernism such as literalization, enactment, difference, movement, etc. Nonetheless, as he argues, literature is embracing new forms of expression influenced by the evolving mediatechnological possibilities and the increased involvement of the recipient’s whole body. Anna Katharina Schaffner, in a wide and historically dense survey, traces the reconceptualization of poetic space from Mallarmé’s Un Coup de dés [A Throw of the Dice] via the word experiments of Concrete Poetry to their “mobilization” by means of digital technologies. By doing this, she explores the relationship between “concrete” and “digital poetry” more closely by analyzing how two of the main concerns of the concrete poets—the poetics of space and the exploration of the concrete materiality of the medium—translate into the digital domain. Two doubly gifted persons are presented next who are able to combine their practical work in an exciting way as authors and programmers of projects in electronic literature; recently especially of Cave writing projects with the theoretical analysis of these processes. John Cayley reports on writing and the practice of literary art in the immersive 3D audio-visual environment of the Cave at Brown University, addressing the use of text-as-surface in a three-dimensional space. He develops a conception of new media as “complex surfaces” based on Cave writing courses to confront the relationship between language and embodiment, language and materiality—always attempting to develop a specific literary aesthetics. Noah Wardrip-Fruin analyzes how the relations between audience experience and underlying processes apply to interactive works. Referring to Cayley’s conception, he focuses on such works that turn the recipient’s attention to the complexity of their “complex surfaces.” While most authoring of electronic literature has so far focused on data and processes, Wardrip-Fruin argues for

14

Jörgen Schäfer and Peter Gendolla | Introduction

using innovations at the surface levels to allow for new literary and artistic experiences. Dene Grigar discusses ways in which hyperlinks are utilized in three-dimensional multimedia performance works that offer a narrative or poetic focus. In the new spaces of three-dimensional performance environments, hyperlinking can be incorporated as a performative element into the work and therefore always makes a purposeful act necessary for the performance to unfold. Grigar argues that hyperlinking may denote a change of scene, the progression of a poem’s instantiation or the evocation of musical notes comprising a composition. The following part systematically discusses completely new forms of “mobile” literature, the so-called Locative Narratives, i.e., the expanding field of literary projects in streets and parks, in exterior locations or those of the city. They are using the previously-mentioned locative media: GPS-tools, PDAs or others, aestheticizing each of them in a quite unexpected turn that inverts the traditional processes of literarization from the “head” back to the “feet”: They adapt literary patterns like travel-, adventure-, love-, or detective narratives, returning their imaginary movements back into real ones again. Francisco J. Ricardo analyzes the practices of layering narrative, image, and sound onto existing architecture and geography in locative art. Using many examples from the 20th and the beginning of the 21st century, he identifies an important conflict regarding aesthetic practices, their framings and conceptualizations; namely, the difference between “place” and “space.” Using this differencei.e., the necessarily limited local conditions and the endless imagination intended in the architectural construction or installationhe shows us how and at what point a “locative narrative” emerges from the “locative consciousness”or could emerge. Rita Raley deals with experimental narrative that employs mobile and locative technologies interfacing with the geo-spatial Web. She argues that in such works the recipient is situated as a “participant” or an “experiencer” rather than a “reader.” Therefore, the use of mobile and location-aware devices for literature challenges both the instrumental use of such services and the mode of literary experience. The essays of Jeremy Hight and Jean-Pierre Balpe introduce the readers to the theoretical premises and artistic practice of two leading artists using various locative media for literary purposes. Laura Borràs Castanyer and Juan B. Gutiérrez present the so-called Global Poetic System and propose a framework for the design and application of locative media for literary projects.

15

Jörgen Schäfer and Peter Gendolla | Introduction

2

Beyond Genre: Perspectives of Literariness in Computer-Based Media

If it is true that semantics is always the result of transcriptions between media, then this development affects all human behavior concerning linguistic signs and therefore also the aesthetic processes of perception and self-perception. The question thus has to be asked whether we can continue talking of a specific migration of traditional literary forms into computer-based and networked media. Can we continue analyzing such examples as literature? Can we still correlate the examples mentioned above with the three traditional genres? In what way can the semantics of literary terminology, concepts and systems be retained or does it have to be revised? A second partial goal of this book is the accumulation of the proof of both the conservation and the complex differentiation of literary forms, structures and qualities in the above-mentioned developments of media. Based on early ideas of Max Bense on the difference between “natural” (traditional) and “artificial” (computer-aided) poetry, Peter Gendolla pursues a paradox accompanying the literary avant-garde from Romanticism to the most current electronic installations; namely, that they want to bring back the cold, dead culture into “natural” life and that they are doing this with the most advanced technological procedures. They become more and more “technical” with the impulse not only to dissolve the division of the genres but also to transfer art at least by way of literary means into “natural” forms of life; thus, they are continually developing new forms of aesthetic difference that have to be differentiated from either nature or culture. Friedrich W. Block looks at the systematic and historical conditions of the emergence of a genre like “digital poetry.” He argues that it has been necessary to communicate and spread schemes of invariance and identification to tie together a high variety of artistic practice. For this purpose, concepts and names of genres have been connected with different forms of institutionalization. From this perspective, his essay considers the conceptual and cultural development of “digital poetry” as well as its relation to historical filiations and their transformation. In conclusion, his considerations lead to an abstract reflection of a more general concept of “poetry.” Like him, Giselle Beiguelman underlines that it is essential to be aware of the historical continuities as well as of the discontinuities that materialize in electronic literature or art. This is particularly true in Brazil where multimedia poets combine videotext and video with their texts. Jochen Venus tackles the aesthetics of video games by utilizing a morphological approach that addresses the interplay between the pictorial elements on the screen and the player’s actions beyond the screen. Hence, he considers

16

Jörgen Schäfer and Peter Gendolla | Introduction

video games as “simulations of self-action” that allow for unique aesthetic experiences of immersive and remote-controlled role-playing.

3

Beyond the Library: Preservation, Archiving and Editing of Electronic Literature

Part Three is reserved for a different thematic focus: The performative projects previously mentioned intensify the already difficult problem of the documentation/archiving of, as well as the access to, processes of electronic literature. Therefore, we would also like to address the problem of archiving and editing this rather transitory electronic literature, thereby attempting to advance the coordination between current and planned databases, archives and editions. Designing and building databases for electronic literature, however, does not only imply compiling a compendium that is as extensive as possible of the area in question; it also requires continually developing typological categories and criteria. The book therefore contains several articles focusing on archiving, on categorizing and on editing as three thematic aspects of electronic literature. While there is a large community of video game fans and developers who constantly work on emulating “old” games for changing platforms and thereby safeguarding the access to these games, there are no comparable activities in the field of electronic literature, as Beat Suter regrets. The community is too small, so that often even the access to simple browser-based projects which are only a few years old turns out to be impossible. From an archival point of view, the situation is getting even worse as an increasing number of works or projects of electronic literature contain interactive, collaborative or dynamic elements that, in principle, cannot be archived. Thus, Suter discusses the impact of writing in computer-based and networked media and—with a particular focus on the situation in Germany—describes the legal, institutional and mediatechnological challenges. Joseph Tabbi, who currently is the Director of the Electronic Literature Organization (ELO), gives an account of the work on the Electronic Literature Directory 2.0, while at the same time—apart from the technical and structural aspects of such a directory—looking at possible ways of transforming institutional practices within the Humanities if scholars would make consistent and collaborative use of the affordances of network technology in gathering literary works and initiating online discussions around them. He argues “that the oftnoted ‘obsolescence’ of works published in perpetually ‘new’ media is an institutional and cognitive problem as much as a technological challenge.” His

17

Jörgen Schäfer and Peter Gendolla | Introduction

proposal, originally a blog entry posted at onthehuman.org, brought up an intensive online discussion that is documented in the present volume as well. The discussion on archiving electronic literature can definitely benefit from the experiences made in adjacent disciplines. From the viewpoint of art history, Katja Kwastek investigates how the development of a detailed vocabulary can help to grasp and distinguish formal characteristics as well as aesthetic processes of interactive art forms. A thorough distinction of formal and technical characteristics allows for a more detailed examination of works commonly subsumed under quite global catchwords such as “(New) Media Art” or “Interactive Art,” “Electronic Literature” or “Digital Literature,” “Net Art” or “Net Literature.” To actually grasp the aesthetic characteristics of such works, it is further necessary to develop a method for describing the aesthetic processes at stake. This is a challenge for the ontological approach represented by classification systems, as aesthetics are subject to individual interpretation. Kwastek, however, argues that taxonomies can help to identify potential aesthetic processes and to promote research towards an aesthetics of interaction. While the previous essays deal with the fundamental problems of preserving and archiving electronic literature, Ravi Shankar, the founding editor of the online journal Drunken Boat for many years and an expert for the presentation of electronic literature and arts, investigates how blogs, listservs and online journals help to build literary communities. Shankar claims that they achieve this by providing open access to pieces of literature and art that perform in ways that could not happen in print. They also encourage potentially unrestricted critical communication as well as active artistic collaboration. And, last but not least, Fotis Jannidis provides an insight into the use of the computer in the humanities. Digitalization and networking have fundamentally changed the working process of creating literary editions. Therefore, Jannidis weighs the possibilities and problems that computer hardware, storage formats, algorithms for accessing the data and strategies for visualizing the data present for the work of textual critics. The aim of this book is outlining the most significant developments from within the wide field of literary projects resulting from the technological possibilities of the most current computer-aided, space oriented and orienting artistic practices, providing important examples. Currently, this field is just beginning to open up. Since literature for centuries has (almost) only moved in the “flatland” of the books’ pages, a new Odyssey is now embarking. Readers of literature have to get out of their “armchairs of tradition”; they have to move into the uncertain outer terrains and experience new adventures, guided by ever-smaller electronic helpers that are becoming more and more indiscernible,

18

Jörgen Schäfer and Peter Gendolla | Introduction

headed for goals unnamed and unknown. This is what the readers have to discuss with each other. Translated by Brigitte Pichon and Dorian Rudnytsky

Works Cited Bourriaud, Nicolas. Relational Aesthetics. Paris: Presses du réel, 2002. Gendolla, Peter, and Jörgen Schäfer, eds. The Aesthetics of Net Literature: Writing, Reading and Playing in Programmable Media. Bielefeld: Transcript, 2007. Manovich, Lev. The Language of New Media. Cambridge, MA: MIT P, 2001. Ricardo, Francisco J., ed. Literary Art in Digital Performance: Case Studies and Critical Positions. New York: Continuum, 2009. Simanowski, Roberto, Jörgen Schäfer, and Peter Gendolla, eds. Reading Moving Letters: Digital Literature in Research and Teaching. A Handbook. Bielefeld: Transcript, 2010.

19

Part One Beyond the Screen: Reconfiguring Space and Time in Literature

Performance and the Emergence of Meaning

Jörgen Schäfer

Reassembling the Literary Toward a Theoretical Framework for Literary Communication in Computer-Based Media “New topics, that’s what you need ANT for.” Bruno Latour (Reassembling the Social 142)

1

Literary Processes: Continuities and Discontinuities

Literature does not appear from nowhere. It is usually created by human beings using certain social practices, falling back on cultural traditions and employing certain technical means. In these hybrid arrangements one finds that on the one hand anthropological dispositives and on the other historically variable roles, literary procedures and structures like genres and media dispositives and infrastructures (ranging from the printing press and the book to modern computers, networks and interfaces) are connected in a very complex way. These constellations determine and simultaneously restrict the possibilities of literary communication. They create a flexible connection between “technological-mediated” and “literary-aesthetic” moments that actually upholds all literary processes in all media. However, it has been taken almost for granted until the present that research in literary studies largely concentrates on those texts that have been written by known authors, and then afterwards edited by publishers and distributed as literary “works” in printed books or periodicals in order to be read by anonymous readers in silence. Even though on the one hand almost all aspects of this literary system have been problematizedat times even radically challengedboth in theoretical considerations and in detailed historical studies during the last decades, on the other hand, the subject areas of the disciplines in literary studies have been successively widened within the framework of an expanded concept of literature, including the analysis of theater performances, radio plays, audio books, movies and TV broadcasts to computer games and works of “electronic literature.” Nevertheless, one has held on to the traditional personal and institutional attributions as a rule for (quite understandable) reasons of research-pragmatics and -strategies in everyday discourse as well as in practical research. And this is the case even though such dissimilar analyses like Friedrich Kittler’s media-archeological studies, Gérard Genette’s ideas on paratexts, or the works by Elizabeth L. Eisenstein and Michael Giesecke on

25

Jörgen Schäfer | Reassembling the Literary

the history of book-printing have shown that book culture as we know it today is a phenomenon built on a multitude of requirements, a culture that in no way is prevalent in all cultural and social contexts, and that is at the same time historically quite young and possibly alsoat least as a dominant mediuma transitory one. With the far-reaching reorganization of a large part of our everyday private and professional production, transmission and reception of messages to the latest computer-based and networked media technologies, however, the conditions have become all the more complex since by way of these media no one element of the series of activities of production, distribution and reception can be changed or replaced without leaving the others untouched. Rather, with the new media-technological appliances and infrastructures, all the social possibilities of action and cultural practices of the human actors are shifted. Specifically, the question of the agency of non-human actors raised by the ActorNetwork Theory (ANT)originally in a much more general reference to all non-living objectsacquires considerable significance:1 Computer systems and networks are not mere channels for the transmission of messages. In contrast to print media that only aim at storing and transmitting its input, computers are able to process signifiers according to a program and thus to generate an output that can neither be predicted nor kept fully under control by writers or by readers. Dispositives of media emerge that were unknown beforehand, encompassing an unfamiliar and insufficiently tested spatial and temporal organization, demanding of its users quite different ways of behavior dealing with technical apparatuses and in communication with other human actors. But this is only one side of the story: As the science historian Olaf Breidbach correctly underlined, technical innovations “stellen meist nicht den gesamten Prozeßablauf in Frage; sie werden an vorhandenen Strukturen angesetzt oder modifizieren einzelne Prozeßschritte” (‘usually don’t challenge the whole process; they are applied to existing structures or they modify individual steps of the process’) (39) within contexts that otherwise remain mostly unchanged. This means that regarding the literary pieces I am focusing on, we are always dealing with both complexly interwoven persistent chains of tradition and with discontinuous moments.

2

New Associations: Actor-Network Theory in Literary Studies

Below, following Breidbach’s thesis, I will therefore deal with shared literary “structures” and “processes” on the one hand; on the other, however, with the momentous change of “steps of processes” that are influenced by the use of

26

Jörgen Schäfer | Reassembling the Literary

programmable and networked media using ever-changing interface and display technologies. In the course of this, I will also raise the question where the common interests in concrete cases lie that still connect genuinely literary communications in all media dispositives. In other words, the question will be asked in what way the literarythat has been analyzed as a phenomenon of a quite specific experience of difference for literature in book culturecontinues to be valid for literary processes in computer-based media. Not only does this seriously challenge traditional literary studies that have been forced to take the role of the latest media technologies into consideration, but also the electronic literature and new media art communities that until now have mostly tried to avoid the difficult task of joining both aspectsthe question of the changed media dispositives and possibilities of acting, just as much as the question of literariness of the observed objects. Instead, they often amount to nothing more than limiting themselves in the analyses of the conditions of interaction and communication. Therefore I believe that some approaches from (interdisciplinary) directions of research—such as the ActorNetwork Theory (e.g., Bruno Latour, Michel Callon, Antoine Hennion, John Law) or the latest approaches from semiotics, linguistics, neurosciences, computer sciences, game studies, etc.—should also be used for questions about literary studies, thereby playing a decisive part for the further development of theories and methods in the future.2 If one wants to understand how moments of historical continuity concur with those of discontinuity, then several factorsor more precisely: above all the “translations” between these factorshave to be considered in order to reconstruct the new associations resulting from this. Below I would like to provide some cornerstones of a theory of literature in computer-based media and foreshadow some promising perspectives of further research. We will have to considerand this will be a constitutive part of my suggestionswhether analogous to Latour’s attempt of newly defining the social we can also attain a similar definition of the literary by attempting to define it “not as a special domain, a specific realm, or a particular sort of thing, but only as a very peculiar movement of re-association and reassembling” (Reassembling the Social 7).3 Literature then would result from “an association between entities which are in no way recognizable as being [literary]4 in the ordinary manner, except during the brief moment when they are reshuffled together” (65). In order to do this, I will proceed heuristically considering the following factors that in reality cannot be clearly differentiated from each other and of which each of them is not sufficient to define “literature.” My contention is that nevertheless the association of these factors sharpens our knowledge of the literary. My further contention is that this shows especially clearly with the example of literary processes in computer-based media:

27

Jörgen Schäfer | Reassembling the Literary

-

Language and text: Any form of literature appears in texts, i.e., in principle it operates with letters in the medium language; that is, with discrete, discontinuous and arbitrary alphanumeric signs und with the combination of these signs.5 I hold on to this important premise specifically to continue delimiting “literature” from other forms of artsomething that in no way precludes that intermedial references occur. With Ludwig Jäger’s theory of “recursive transcriptivity,” which in the transcriptive logic of language is the basic principle of any production of meaning, different forms of intra- and intermedial references can be analyzed.

-

Media technologies and infrastructures: Linguistic signifiers must be inscribed into some kind of material medium in order to be communicated. To understand the difference between media operations in computer-based and in other technical media, the relationship between human language and computer codequite literally as a rule for translating a piece of informationhas to be especially considered. Computers can process digital data in a program-controlled way, they allow for interactive interventions by users via different interfaces and they can be networked via data lines with other computers. By way of these characteristics, the medializing of cognitive processes has changed and a new logic of the processing of linguistic signs within and between different media has been establishd that so far for literature has only been analyzed rudimentarily with regard to changes.

-

Social practices: Thereby also the traditional activity-roles in literary communication change. In place of the translations between temporally successive, spatially separated and clearly delimited steps of production, distribution and reception of communication via the medium book, now chains of translations enter in which the agency of human and non-human actors is distributed in a completely different way. This also changes the physical practices in dealing with linguistic signs.

-

Literariness: As a final point, I will concentrate on the blind spot of these notions of agency, namely the literary characteristics of such chains that has to emerge from the interplay of “author(s),” mediated “works” and “reader(s)” in computer-based media.

28

Jörgen Schäfer | Reassembling the Literary

3

Theory of Mediation: The Emergence of the “Work of the Work”

3.1

Chains of Translations

Even though ANT has not been designed as a theory of media, it nevertheless contains an implicit theory of media (cf. Thielmann, Schüttpelz and Gendolla). The increasing interest in ANT in cultural and media studies thus can be mainly substantiated with the fact that it does not define media through the traditional media-technological apparatuses alone. Rather, ANT’s ideas are focused on the concept of mediation that takes its point of departure from recursive processes in social and technical chains of translations and operations. In principle, all mediating factors of the socio-technological world are seen as so-called mediators in these chains.6 This makes it possible to reconstruct the steps of mediacy made by way of chains of translations between persons, artifacts, and signs, by which the forms of agency between the factors involved are being built, connected and redistributed (cf. Schüttpelz 236f.). With these translations, people, material artifacts and signs are per se mediated; one could say just as well: they are constantly referring to each other recursively.7 Thus, during each process of translation all factors included and thus their relationship to each other are changed. Such a concept avoids a media-terminological determinism by exposing every seemingly decisive factor as a factor of translation which through its “in-between” is always at the same time an effect of antecedent linkage as it is a cause for the following. Therefore, the agency is to be found in the operative process itself to which all actors coordinated by it are adapted, i.e., every kind of unit has to be seen no longer as a precondition, but always as a result of media operations as well. This concept of chains of translations was first developed by Latour in Science Studies and finally taken up again in his critical revision of sociology.8 By contrast, aesthetic questions have rarely been of interest to ANT. For example, Antoine Hennion has taken up the concept for music studies; musical mediation serves him as a model for the “collective” production of aesthetic processes. This is of interest in our context insofar as in the case of music the traditional subject/object-divisions as well as the ideas of the “closed work” do not at any rate function in the same way as they seem to function in the realm of the arts or literature. The “reality” of music, according to Hennion, lies in its time-dependent and fleeting event-character, or rather in its appearance; it can only be found when different media, instruments, scores, languages, physical techniques, competences and tastes etc., are mobilized in a common space:

29

Jörgen Schäfer | Reassembling the Literary

. . . speaking of mediation is acknowledging that something effectively “happens” in this process, which transforms the ways things were before; an “event” occurs which has a positivity of its own that cannot be limited to its origins and determinants, no more than to its effect. (Hennion and Grenier 346) Thus, the theory of mediation transgresses the traditional work-model, replacing it with a processual description of the object in which chains of translations are conceived of as “circularité des médiateurs” (‘circularity of mediators’): “une théorie de la médiation qui la définisse non par un état, mais par une oscillation entre deux états” (‘a theory of mediation that defines it not by a state of being, but by an oscillation between two states’) (Hennion, La Passion musicale 369f.). If we still want to speak of a “work” as a unit that can be addressed, then we find it only in the chains of translations themselves, i.e., the “work” is the mediation or ratheras Hennion and Grenier are expressing it in an illuminative bon motthe “work of the work”: The mediations are neither mere means of the work, nor substitutes which dissolve its reality, and their revelation is not an act of unveiling which leaves the king naked. . . . And then, at certain moments, on top of it all—that is to say, in addition to this set of mediations— something might happen. Something may emerge from this mix and that may be the “work of the work” of art. (Hennion and Grenier 348)

3.2

From Language to Cybertext: Recursive Transcriptions

Latour’s recourse to semiotic theories has been repeatedly criticized and with partly convincing arguments.9 Since in literature we are dealing with a linguistic subject area, we can initially put aside these concerns since they can be dispelled by considerations on the intra- and intermedial linguistic form of the kind of mediation as it was developed by the German linguist Ludwig Jäger with the term recursive transcriptivity. As he illustrates in his contribution to this book, a theory of the literary human-machine communication can also profit from considering the transcriptive logic of language as a fundamental procedural logic of mediality for the creation of cultural semantics. Language is the anthropological archetypal medium of cognitive integration. Thus it is only by semiological processing that meaningful inner representations are created in people’s minds. This happens by forming networks of signifiers from multisensory experiences. These networks are the starting point for the creative invention of mental episodes, which then need to be externalized by communicating them

30

Jörgen Schäfer | Reassembling the Literary

through media, whereby technical media, i.e., “things” in Latour’s sense, are explicitly included. Linguistic signs then are not to be understood as storage- and transfermedia for contents independent of language and media-indifferent; rather, as operative media they are themselvesquite in the sense of the mediation approachthe condition for the possibility of mental form-creations. Carrying on neuro-biological descriptions of the encoding of perceptions, of intermodal integration, and of the parallel processing of information in neuronal networks, Jäger is describing the process of semiological transformation as a synthesis of linguistic signs. The human apparatus of cognition connects the cross-modal products of association with the modality of signs and thus positions them in a semiological horizon. Preliterate language then is the semiological processual form that in the very first place makes the building up of inner mental episodes in the network of linguistic signs possible (cf. Jäger, “Zeichen/Spuren” 18f.). These interreferentialities between “prescripts” and “scripts” in the process of transcription in all media dispositives develop in a specific way and I would argue that they distinctly appear specifically in computer-based and networked media. Therefore, it is not by chance that with Espen Aarseth’s cybertext theory one of the most effective approaches is also based on the (even though quite schematized) idea of translation between different linguistic “levels.” The crucial point of Aarseth’s theory is his idea of regarding any text in a very literal— and not only metaphorical—sense as a machine for the production, transmission and reception of verbal signs. This machine consists of a material medium, a user, and strings of signs that are divided into so-called “scriptons” (defined as strings as they appear to readers on some material surface) and “textons,” which are “strings as they exist in the text” (62). Hence such a text does not consist of one single syntagma but of two layers, which—and this is crucial—are recursively related to each other by what Aarseth calls the “traversal function,” i.e., the “mechanism by which scriptons are revealed or generated from textons and presented to the user of the text” (62). I could name some more theories of translation that confirm the central importance of such a “cybernetic” thought for theories of media studies. It is the twist of all such theories, however, that recursive loops are not just a simple means of reproduction. Rather, they combine repetition and variation in a very specific way with the objective of creating something new that cannot be predicted in advance. This is due to the fact that recursions allow the repeated application of a processing instruction onto a variable, which has already been the result of the same instruction itself. The variable value varies with each passing of the loop; but this repetition does not result in the production of identity but in pre-defined variation (cf. Winkler 173).

31

Jörgen Schäfer | Reassembling the Literary

If we thus see “meaning” as a result of such recursive linguistic transcriptions, then this also has far-reaching consequences for the understanding of literature. With reference to Jäger and Aarseth, my point of departure is that the function of language does not lie in the task of representing reality alone; ratherin order to be able to achieve this at allthat it has to reserve a certain potential going beyond the function of representation that is expressed in literary texts. Literature can thus be described as a specific form of the use of language that activates a surplus of the possibilities of language. And so, Andreas Kablitz takes his starting point from the paradox that even though language can say more than one can assert, it can at the same time also say nothing without asserting anything: “Das Potential, das aus diesem Paradoxon erwächst, ist der Fundus, aus dem alle Fiktion schöpft” (‘The potential arising from this paradox is the fundus from which all of fiction draws’) (272) and thus it is an essential source of all literature. Literary texts, then, mediate the assertive statements of a “sense of reality” with the “sense of possibility” of language.10

4

New Media for Literature: From “Immutable Mobiles” to “Permanent Mutability”

4.1

“Techno-Semiosis”: The Computer as Medium of Transcription

I have discussed the linguistic approach by Jäger and the approach from literary studies by Kablitz because they underline that the mental, and therefore the literary as well, are the result of linguistically mediated transcriptions, and I have referred to Aarseth because he makes clear how mechanical or media-technological processes collaborate in the exteriorization of the “Ur-mediality” and thus already point out the important difference between the hidden (computer) code and the text legible by humans. If we assume the constitutive role of the linguistic sign’s mediality that in the first place is founded in its communicative use of recursively combining production and reception of linguistic utterances, then the history of media technologies and practices can be described as “technosemiosis” (Jäger, “Zeichen/Spuren” 32), i.e., as a technical differentiation of this original mediality of signs within the changing dispositives of media. In other words: The very basic semiosis in the mind of a person is to be extended in time and space by media—and these media and their relations to each other have been complemented again and again in the course of media history: from spoken language in face-to-face communication to written inscriptions, from printed books to modern computer systems:

32

Jörgen Schäfer | Reassembling the Literary

Historically, the most successful method of this externalization of “inner” linguistic signs is script, allowing for the storing and transporting of information over spatial and temporal distances. Book printing marks another decisive caesura for the history of media; according to Latour “a device that makes both mobilization and immutability possible at the same time” (“Drawing Things Together” 31) by producing so-called immutable mobiles. In ANT, this concept had been developed originally in order to explain how symbolic representations on paper materialize from scientific observation or everyday experience. Writing and the printing press located or locked down within constant strings of fixed symbols what in fact were originally performative processes, thereby securing a high degree of textual stability. However, if performative processes can be arrested then such arrests can in principle be reversed again. This is precisely what happens in computerbased media, which quite conversely are characterized by the “permanent mutability” of data that can be made temporarily visible on all kinds of displays by way of complex steps of translation: In der elektronischen Welt . . . ist der Augenblick des “Druckens,” in dem ein bestimmter Datenzustand eingefroren wird, lediglich ein Punkt auf einer fortlaufenden Zeitachse; der Datenabruf bietet nicht mehr als eine Momentaufnahme des permanent wandelbaren Datenflusses. (Chaouli 68) In the electronic world . . . the moment of “printing” in which a certain state of data is “frozen” is only a point on a progressive temporal axis; the data recall provides no more than a snapshot of the permanently variable data stream. Due to these properties, the computer additionally affords (and forces) its users (into) a technically controlled interaction. In contrast to the printed book it does not only store and transmit data, it also processes them. In this respect the human-machine interaction has to be understood as a transcriptive and technical process in which the program-determined signal processing (on an operative “subface” level) inside the machine is linked on the performative “surface” level of the interfaces with the sense-generating semiosis and its affiliated activity of the user as “author” and “reader” (Nake 104ff.). Since such “algorithmic signs” per se mediate between human and machine processes, the media logic is inherent to the computer. This was true even at that time when there was no talk at all of the “computer-as-a-medium,” and in general this logic is also valid if computers are not ostensibly being used as media but as control automata:

33

Jörgen Schäfer | Reassembling the Literary

Im Prozessieren des Programms vollzieht der Computer Übersetzungen zwischen Notationen. Dies ist eine allgemeine Beschreibung des Wirkens des Computers. Sie trifft auch zu für den Computer als Steuerungsautomat, der direkt in die Wirklichkeit eingreift, bei der Schaltung von Maschinen, der Steuerung von Fabrikanlagen oder bürokratischen Verwaltungen. . . . Als Medium verändert er die Wirklichkeit vermittelt, als Steuerungsautomat direkt. Als Steuerungsautomat enthält er seine Steuerung medial und als Medium enthält er eine wirkliche Steuerung. (Robben 54) In processing the program, the computer executes translations between notations.11 This is a general description of the operations of computers. It is also applicable for the computer as a control automat that directly interferes with reality: operating machines, controlling factory plants or bureaucratic administrations. . . . As a medium it indirectly changes reality, as a control automat it does this directly. As a control automat it contains its control in a mediated way and as a medium it contains a real control. In order to enable human users to perceive these mediated translations between sub- and surface levels with their senses and to influence them physically, and in order to enable computers to actually control the appliances connected to them, interfacesdisplays as well as input devices12with different spatial-temporal relations between human and non-human actors have to be made possible. Here the theory of chains of translations offers a frame as well; in ANT, it had been developed originally in order to explain how scientific observations or everyday experiences are converted into symbolic representations on paper, i.e., “immutable mobiles.” But these superpositions can occur on all displays, so that this model is suited very well to describe “den Datenwandel AnalogDigital/Digital-Analog und den Formenwandel als Teil einer Operationskette” (‘the data change analog-digital/digital-analog and the change of form as part of one single chain of operation’) (Thielmann 206).

4.2

Reflecting Translation: Stephanie Strickland’s slippingglimpse

In computer-based processes therefore we have to differentiate quite clearly between the “surface” or “interface text” and the computer code. A computer code is a command that translates a character-set into signal sequences within the machine controlling state transitions which can be made accessible in the surface text. As was underlined for example by John Cayley, this is a difference

34

Jörgen Schäfer | Reassembling the Literary

of fundamental importance since only if the code is executed (which is always directed at the machine), can something become visible on the surface, e.g., a text legible by humans as a representation of the code as it is running: These processes are the work. The writing is not the record of an inscription or prior composition. It is a program running. It is the sum of all the phenomena which occur when a program—a “prior writing” in anticipation of performance—is set in train. (“Screen Writing” 609) For example, in pieces like Cayley’s translation or riverIsland, the recipient is confronted with so-called “transliteral morphs”—coded transformations of letters from source texts into target texts. Accordingly, the letters of the source texts are replaced step by step with programmed algorithms, making it possible for the recipient to observe the performance of the appearance by “monitoring a ‘runtime performance’” of the “text as a complex, temporal object,” as “everchanging, ambient manifestations of writing on complex surfaces” (“Writing on Complex Surfaces”). Still more complex steps of translation can be observed in the “digital poem” slippingglimpse that the poetess Stephanie Strickland produced in collaboration together with both the programmer Cynthia Lawson Jaramillo (who was responsible for the flash programming and the interface design) and the video artist Paul Ryan, who contributed video recordings of water movements that seem to run chaotically, yet reveal recurring patterns, so-called “chreods.” slippingglimpse consists of recursive feedback loops between these various human and non-human actors: On the screen, the recipient watches moving images of “chreods” in which a video tracking software looks for color changes. As soon as the algorithm localizes such a change, the program matches the position with words and phrases from Strickland’s poem-texts. The piece thereby allows the aesthetic experience of very complex chains of translations between human actors as programmers, writers and readers as well as non-human actors such as natural processes, video tracking technologies, computer hard- and software, poetic text fragments, and so on. Thus it exemplifies how meaning in a computer-aided and networked environment is created through the interplay of various mediators. In a certain way this is the case as well in the “reading” of the piece by the human recipient as it is in the “reading” of the poem-text by the water: “water reads text; text reads imagecapture technology; and image-capture technology (that is, videography and video-editing) reads the water, thereby coming full circle” (Strickland).

35

Jörgen Schäfer | Reassembling the Literary

Fig. 1. In the scrolling text view of slippingglimps, the poem text “reads” image/capture technologies by sampling and recombining words of visual artists describing their use of digital technologies. Courtesy of Cynthia Lawson Jaramillo.

It can in no way be unequivocally differentiated who or what is “reading” whom or what in these chains of translations. It is precisely this automation of the mutual observation of human and non-human actors that is made available to literary aisthesis as a distinct and reflexive perception of perception. In a piece like slippingglimpse, literature maintains its function by creating an aesthetic distance—that is to say, a deliberate rupture of the increasingly densely “fedback” and (partially) automated social and technical interactions. However, now far more complex media-technological and social conditions have to be reflected in theory. If Jäger describes disruption as “any state in the course of communication producing the operative loss of transparency of a medium” (“Epistemology of Disruptions” 83), then this certainly holds true for a general concept of the emergence of meaning. Literary pieces like slippingglimpse clearly show that in current media dispositives technical “disruptions” of the transparency of media also participate in the constitution of meaning.

36

Jörgen Schäfer | Reassembling the Literary

4.3

Spatially Defined Media for Literature: Mixed Reality Environments and Mobile Media

Even though slippingglimpse is still viewed on a conventional display, it has already made a further question imperative; namely, in what way the computer configures new spatial organizations in media in which mental spheres, real three-dimensional realms of action, computer-generated “virtual” environments and the telematic spaces of the computer nets are dynamically correlated to each other. The spectrum of media dispositives for literary processes in such spaces evidently has widened considerably in recent years and it can be expected that this will continue: Displays are no longer limited to monitors and also the interactions with linguistic signs are now not only implemented through keyboard or mouse. Rather, with the application of mobile media devices such as mobile phones, GPS and PDAs and the development of literary mixed reality environments in museums, galleries or research labs, new combinations of physical, virtual and symbolic spaces are realized. On such “complex writing surfaces,” as Cayley argues, spatial “depth” not only emerges in the literary virtuality of the reader’s imagination; it also comes to the fore on the intrinsically temporal materializations of transcriptions resulting from “complex, recursive interrelations of writing surfaces and surfaces that are, literally, formed by writing, at least in so far as the graphic surfaces of letters are ‘formed by writing’” (“Writing on Complex Surfaces”). Thus, in these informational spaces changes of the temporal-spatial organization occur, i.e., of the culturally conventionalized coordinate systems in which communicative interaction is taking place with linguistic signs. The recipient “must be able to see and read what the screen presents rather than recasting what passes before our eyes as the emulation of a ‘transparent’ medium” (“Writing on Complex Surfaces”). Texts, objects, bodies and spaces combine in a largely uncharted way, electronic media take the interplay of embodied language and “body language” to a new level since more and more the whole body is involved in the media activity. Increasingly complex sensors (integrated into vehicles, clothes and environments) “realize”—“hear,” “see,” “feel,” in other words: measure and translate—the movements of the body, its mimics and gestures. This “multimodal” body itself then also exchanges information with the “products” of this kind of technology. Such mediated couplings and framings enable the co-operation of non-symbolic activities, symbolic language activities and algorithmic processes of computer systems. The central discussion regarding processes of the spoken performance of communicative acts and the performing arts is turned into another direction by pieces such as Camille Utterback’s Text Rain or Noah Wardrip-Fruin’s Screen as

37

Jörgen Schäfer | Reassembling the Literary

well. If the related studies, e.g., those by Erika Fischer-Lichte, once concentrated the attention especially on non-textual or non-literary aspects, i.e., on nonlinguistic physical activities, now the text surprisingly catches up with performative elements again. With the latest media technology it becomes possible “to do things with words” in a way that speech-act theory could not foresee. Especially for the spatially defined literature, the text is an integral, physically present part of the performances: As a three-dimensional projection of light in the Cave or as the directive for movement in locative narratives, it becomes a practically acted out reflection or critique of habitualized activities, rituals or techniques of social control. Attention is challenged in another, precisely aesthetic way: It is directed to the links between texts and bodies and to the transcriptions taking place between them in the realm of the linguistic signs of the projects. In Camille Utterback’s interactive installation Text Rain the recipients, situated in front of a projection screen, use their bodies for playing with falling letters. The installation initially disassembles the elements of its own basis— namely “Talk, You,” a poem by Evan Zimroth on the difficulties of communication and physical nearness—at the outset dissolving it into letters and words falling from up above with which the recipients then can “play” with their hands, arms, legs, and the silhouettes of dark objects: they can catch them, gather them, divert them and hold onto them. Occasionally they succeed in catching an entire word or a phrase, ephemeral successions of signs that for a short while seem to have “meaning” that, however, can directly dissolve again. Francisco J. Ricardo, in his subtle close reading of Text Rain, notes that the readerly function is strongly challenged: When the baseline wanes, text appears visually, not lexically. . . . But in its dual existence as text and visual sign, this type of work conveys a multimodality that is unsettling and refuses reduction to singular classifications. . . . That Text Rain is thoroughly transmodal is additionally evidenced by considering it ontologically, as it exists, or perceptually passes, through a series of more or less distinct phenomenological stages, lives, or moments. (59-61) Traversing these different phases, the relationship between image and language, for which all media dispositives delineate specific realms as stages of their semiological performance, becomes a question. By referring to the phylogenetic antecedence of sign language to spoken language, Jäger points out that there is no categorical opposition between image and language. Rather, language can occur both as spoken language (vocal-auditive communication, temporal-sequential) and as sign language (gestural-visual communication, tem-

38

Jörgen Schäfer | Reassembling the Literary

poral-sequential and spatial): “Der Raum des Bildlichen exkludiert Sprachlichkeit keineswegs, wie umgekehrt Sprachlichkeit sich gerade auch als BildSprachlichkeit realisieren kann” (‘Visual space by no means excludes language, as vice versa, language can also be realized visual-linguistically’) (“Bild/Sprachlichkeit” 8). In Text Rain, from the conflict between the writing surface and the imagination, between the two-dimensional medium of letters on a surface and the multidimensional imaginative realm of the reader, develops aesthetic experience. This conflictindissoluble in the traditional space of the medium booknow, in the three-dimensional space of the installation is solved in a quite specific way by returning the words back to the bodies. However, the conflict on this level is also renewed: The body or the bodies may move as they like; they are unable to reassemble the poem as a whole. Aesthetic difference as a perceptional conflict or tension between the senses and sense in this installation has been transcribed into the electronic-organic coupling. In locative narratives as well, with location-based media the relationship between physical and discursive spaces and the interplay between the physical action of the recipients with the fictional characters of the narrative spaces become the theme. In his “geopoetical” story Wasser [Water] (2004), Stefan Schemat integrates real places and landscapes into his narrative spaces.13 Recipients, equipped with “augmented reality outfits,” a backpack with a notebook, a GPS device and headphones are sent through a town or a landscape on a one-hour walk; i.e., the recipient has to fulfill a mission under constraints of time. The concrete narrative sequencei.e., the sequence in which the computer system connects the narrative fragments depending on the locationand the duration and speed of the narrative depend on the choice of the route the recipients take. While they are on their way, the story of a blind detective searching for a missing person is told to them. Schemat here is taking up a known motive from literary historyespecially in crime fiction, but for example also in one of the key texts of the avant-garde, André Breton’s Nadja (1928). However, the imaginary “co-search” of the reader in the fictional space of a novel here is expanded into a hybrid “real-imaginary” search. Thus, admittedly, the space in the real world and the narrated space are connected to each other“text and environment merge” (Kwastek, Ohne Schnur 203)but then again, with a diegetic break a paradoxical division between the two realms is staged, because the recipients take over the fictional role of a blind detective. In the space of the real world, they are part of a performance in which as seeing persons who nevertheless themselves are “directed” and controlled by way of the media technologies usedthey “embody” a blind character and “lead” him

39

Jörgen Schäfer | Reassembling the Literary

Fig. 2. Recipients experiencing Stefan Schemat’s Wasser at the beach of Cuxhaven, Germany. Courtesy of Cuxhavener Kunstverein.

through a space in which the missing person, however, cannot be found; i.e., they act just as “blindly” as the blind detective does. In the narrated space, a peculiar literary organization of space is effective: despite the fact that the recipients are the ones whofirst of all by their movements in real space and the technical steps of translation initiated by thisconstitute the narrated space, and even though they are thereby technically “controlling” the narrative voices, in the intradiegetic story they are nevertheless dependent on the “control” of just these narrative voicesthe more so as the narrators “are sometimes distanced observers, sometimes the protagonists of the plot and they get into conflict with each other (‘Don’t listen to the voices . . .’), wavering between fiction, reality and dream and between present and past” (203); i.e., they jump back and forth between homo-, hetero- und autodiegetic voices. Although Schemat assumes that “the location of the narrative can be found in reality” (203), the relationships between author, recipient, narrators and literary characters in narrated space are still to a great degree subject to narratological parameters like narrative perspective, focalization, temporal structures etc.; i.e., the analyses of the chains of translations, by which locative narratives are constituted, has to include the categories of literary studies (cf. par. 6.2 in this text).

40

Jörgen Schäfer | Reassembling the Literary

5.

Aesthetic Engineering:14 Actor-Networks and Distributed Agency

The different aspects of a recipient’s activity with Text Rain or Wasser—the cognitive processes activated by reading or listening and the physical movements in space, the interaction with the computer-generated graphic or acoustic “language objects”—and the computer-controlled activities of the technical system show that in such computer-based media dispositives it is all the more necessary to focus our considerations on the investigation of the interaction of different human and non-human actors as hybrid socio-technical “collectives.” This can be illustrated with theories of socio-technical, self-reflexive networks as well as techno-sociological thoughts on distributed agency or distributed cognition. For the analysis of literary processes, the interesting element in these approaches lies in the fact that theydiffering from the ordinary models of sociology of literaturetake their point of departure on the one hand not from the option that the agents are clearly discernible, nor, on the other, that they necessarily have to be human beings (e.g., as “writer,” “editor,” “lector,” “publisher,” “reader”). Instead, they point to associations that are “made of concatenations of mediators where each point can be said to fully act” (Latour, Reassembling the Social 59). Thus, the term “network” acquires a special significance since it stresses specifically the uncertainty about sources of action, i.e., the mediations or transcriptions are not to be ascribed to clear causes, particularly not unequivocally to the intentions of human actors. An actor-network is a heterogeneous network of different actors who are, however, unidirectional. This means that as many “causes” as possible are replaced by way of actor chains of translations. All observable events are not attributed to simple causalities; such local interactions are rather always the result of the interference of circulating heterogeneous entities, or, in Latour’s own words, of an “assemblage of all the other local interactions distributed elsewhere in time and space, which have been brought to bear on the scene through the relays of various non-human actors” (194). What is observable therefore is always restricted to fleeting effects of long chains of processes of mediation or transcription: In most situations, actions will already be interfered with by heterogeneous entities that don’t have the same local presence, don’t come from the same time, are not visible at once, and don’t press upon them with the same weight. The word “interaction” was not badly chosen; only the number and type of “actions” and the span of their “inter”-relations has been vastly underestimated. Stretch any given inter-action and, sure enough, it becomes an actor-network. (202)

41

Jörgen Schäfer | Reassembling the Literary

Latour’s principles of symmetry and hybridity, which in a far-reaching sense assume the equality of all actors, have often been criticized.15 It cannot be decided here whether this criticism is justified; my only concern is retaining the claim that actor-networks are always socio-technological “collectives” made up of human and non-human elements and that in these collectives the machine elements accrue a rising degree of autonomy. Specifically in computer-based systems or networks such an agency distributed into many activities and instancesappears more often. “Action” emerges here as a result of mutual influence, delegation or even substitution within socio-technological constellations. This means that action is not only changed by “adapting” technologies to the needs of human beings; rather, activities are distributed between human and “artificial intelligences”16 that in no way are entirely determined but become increasingly reflexive and in part also indeed able to learn. Thus, computer-based action continues a long story of human self-invention in technical systems in which social and technological abilities are amalgamated into “hochkomplexe anthropotechnologische Netzwerke” (‘highly complex anthropotechnological networks’) (Hagner 34). It has consequences also for literary studies when in the place of a subjectcentered notion of actiontaking its point of departure from the intentional acts of autonomous individualsthe concentration is on the collective dimension of creativity and artistic production. One can regard this quite explicitly as a turn against the very idea of artistic creation, indeed against “creativity” as an activity of particularly gifted individual persons. They are being supplemented or replaced by different forms of distributed agency or by actor-networks, in which creativity, on the one hand, is either distributed between various human beings or between human and non-human actors and, on the other, can be distributed across time and space. To illustrate this with my previous literary examples:17 In John Cayley’s translation the agencies of the “author” are assembled with that of the software but also with that of the hardware used and of the interfaces, with the author of the source text (in this case Walter Benjamin, and strictly speaking also the translators of the English edition of his text), as well as with the interactive interventions of the recipients. In slippingglimpse, for example, the recorded movements of water, the intentions of the writer of the poem or different navigational decisions of the recipients are added to this. In Right as Rain by the artists collective 34 North 118 West (i.e., Jeff Knowlton, Naomi Spellman, and Jeremy Hight), the current weather in the different places has its own agency. In Wardrip-Fruin Screen one could nameapart from the factors already namedalso the physical movements in three-dimensional space or the diverse steps of translation of the VR machine. The Breathing Wall by Kate Pullinger reacts to the recipient’s rhythm of breathing, thereby influencing the

42

Jörgen Schäfer | Reassembling the Literary

narration of the story of a prisoner who is communicating through the wall of his cell with the spirit of his girlfriend. In these examplesand with regard to the distributed agencies this is initially a contradictionthere is nonetheless always talk of an author: of “John Cayley,” “Stephanie Strickland,” “Noah Wardrip-Fruin,” “Kate Pullinger” or of an author-collective like the group “34 North 118 West.” In computerbased media collective works regularly appear since programmers, interface designers, artists, or musicians also take part; Strickland, for example, works together with Lawson Jaramillo and Ryan; Pullinger with Stefan Schemat and babel, and Wardrip-Fruin has even more numerous collaborators. Apart from this, still and all titles of the works exist for these literary pieces. Therefore, the question should be asked to what extent such open aesthetic processes can continue to be (or maybe even have to be) studied with the same categories of observation—even when important activities have been delegated to “intelligent” machines—in order to be understood by the recipients (and especially by those who are interested in literary studies). For these attributions or framings, ANT has developed the concept of blackboxing, by way of which a complete network itself becomes a node in another, more extensive network: . . . if a network acts as a single block, then it disappears, to be replaced by the action itself and the seemingly simple author of that action. At the same time, the way in which the effect is generated is also effaced: for the time being it is neither visible, nor relevant. So it is that something much simpler . . . comes, for a time, to mask the networks that produce it. (Law, “Notes on the Theory of the ActorNetwork” 385) Of course, this is in principle true for every actor-network. In transferring this to literary communication, this would mean that author names such as “James Joyce” or “Thomas Mann” are hiding an extensive network that was necessary for the writing and publishing of Ulysses or Buddenbrooks as well. Clearly there are conventions, forms of co-ordination or “translation regimes” (Callon 147) that regulate to whom a superordinated agency is attributed. In such “convergent and irreversibilised networks” (155) behind the author function, however, all other factors have disappeared and the multiple chains of translations can only be made visible again with philological finesse. By contrast, in literary works in computer-based and networked media, such “framings” or “indurations” cannot succeed in the same way. “Translation regimes” drawing on general conventions exist here as well, but as I have tried to show, the conditions of translation fundamentally change between

43

Jörgen Schäfer | Reassembling the Literary

authors and readers as human actors, as well as between computer-based media and their interfaces as non-human actors, since they are incorporated into significantly more divergent and reversible networks. Thus, for example, Simon Biggs remarks on his early piece The Great Wall of China (1996) that as an author he had not written the text but a “metatext”—or rather a “parent text” from which “children texts” develop according to principles of inheritance: The author has written a “parent” text, from which innumerable other texts are written through the act of reading. The reader does not write these texts, but participates in an ecology of behavior (involving themselves, the author, and the semi-autonomous text itself) from which emerges a particular instance of the text. At this level, once the meta-writing is done, the author becomes just another reader of the work. (191) When the “work” is not a closed object but is itself a processing entity, the activity-roles enter new constellations and dependencies. Normally, a text in literary studies is understood as a linear arrangement of words and punctuation and it becomes a work by being used communicatively as a fixed, completed and reusable unit, as an “immutable mobile.” In computer-based media these two definitions are not tenable: the signifiers of a text are not stored in linear order in data files, and a text or a “work” only becomes visible on a display as an ephemeral materialization of an ongoing process. This process that not only contains data files but also running programs and hardware devices18 therefore is dependent on the respective spatial and temporal contexts: “The materiality of an embodied text is the interaction of its physical characteristics with its signifying strategies” (Hayles, My Mother Was a Computer 103). And thus, a technical processing is inserted that brings to the forequite in Latour’s sensean assemblage of various local interactions of human and non-human actors in place of the work, directed at the cognitive processes of the recipient.19 This illustrates that the traditional concept of “work” is not only based on the physical and social characteristics of an object, but that it also presupposes a quite definite understanding of the subject, namely that of the intentionally acting author. Within the terms of a distributed agency, this changes since “the computer is also a writer, and the software programs it runs to produce the text as process and display also have complex and multiple authorship (not to mention the authoring done by hardware engineers in configuring the logic gates that create the bit stream)” (Hayles, My Mother Was a Computer 107).

44

Jörgen Schäfer | Reassembling the Literary

More and different things are also demanded of (human) “readers” than with the quiet reading of a printed text. “Ergodic literature,” as Aarseth calls it, requires more than just interpreting what the reader or “user” reads in order to understand a text’s meaning. In addition, he needs to perform in an “extranoematic sense”; this means that, as Aarseth puts it, a “nontrivial effort is required to allow the reader to traverse the text” (1). These additional functions are “the explorative function, in which the user must decide which path to take, and the configurative function, in which scriptons are in part chosen or created by the user.” The so-called textonic function demands from the user to add textons or traversal functions to the text (64). Aarseth then assumes that users have several functions with specific activity options at their disposal influencing the translations between textons and scriptonsand that they also have to use these options in order to make the text appear. This already leads to highly complex chains of translations when using normal personal computer systems. In spatially defined media even disproportionately more complex steps in translation and interpretational relations develop. Specifically, the sensorimotor interactions of the recipients with their social and technical environmentincluded via sensors and effectorshave to be taken into consideration. It is decisive that the human body is the connecting link of all interactions. As Thomas Fuchs has shown in a convincing critique of the reductionist-naturalist neurosciences, the brain contributes to all interactions as an “organ of transformation or translation, which translates the relations between single elements of a given situation (‘stimuli’) into wholes or Gestalt units” (“Embodied” 227), i.e., it processes the arriving sensorial stimuli and translates them into physiological and motoric reactions. Even though rational processes are controlled by the brain, nevertheless consciousness only emerges as a result of complex chains of translations that are mediated by way of sensual perceptions, physical activities and neuronal and physiological processes, as well integrating events in the environment via (also technical) media, especially the activities of other people or events imparted by media: Human subjectivity is embedded in the world, with the body acting as its mediator. . . . Miraculously, our body, a solid and material object, is capable of a transformation that turns matter into mind and lets the world appear. By multifarious assimilations, sensorimotor interactions and their further processing, the body becomes transparent to the world we are living in and allows us to act in it. The meaning of this transparency of the body should be noted carefully: It implies that consciousness is not the final link of a chain of deanimated physical processes. . . . The mind is not a transmundane asylum of pure subjectivity, but it is the integration of all these living bodily processes, which render themselves transparent to the world. . . . But in perceiv-

45

Jörgen Schäfer | Reassembling the Literary

ing the subject embodies or enacts these processes. Their invisibility precisely means their transparency. It is through them that we perceive, and they are implicitly present in our act of perceiving, in a way similar to the single letters through which we read a word without being aware of them. (Fuchs, “Corporealized” 95f.) The connection of the brain to the human body is also overlooked in literary theories when they mask the multifarious steps of translation in order to directly refer the text and mental processes to each other. However, only the living being as a wholeeven more so: the living being, eternally “networked” via its senses with its environment20commands a consciousness with which it can perceive or act. In literary communication in principleeven when reading a book quietly, even though to a lesser degree“transparent” physical activities are necessary. Computer-based mediathat via diverse interfaces make the technical control of human-machine interactivity possible and that also extend the human-human interaction via the networking of computers demand a considerably greater degree of physical activity and thus let us perceive that reading has always been “a whole-body-activity that involves breathing rhythms, kinaesthesia, proprioception, and other conscious and unconscious cognitive activities” (Hayles, “Distributed Cognition” 16). Apart from that, other non-human actors are included whose “antiprograms”21 (Akrich and Latour 261) disrupt and restrain the actions of the human actors.

6

The Literary in Chains of Translations

Even though these observations are important and expandable for the discussion of literature in computer-based media, strictly speaking they describe every form of human transactions with linguistic artifacts. However, one decisive aspect is missing, and that is the question of the specific literariness or the specific aesthetic experience. This holds true explicitly for ANT: even the studies by Hennion or those by Jean-Paul Fourmentraux on “Net Art”to say nothing of the few remarks by Latour on aesthetic questionspursue approaches of a sociological kind regarding music or art; they are interested in how human actors communicate artifacts by referring back to cultural practices and by using instruments and technical media to whom different degrees of autonomy are (or have to be) conceded. Neverthelessfor the most part lacking are considerations which, from a point of view of literary studies, deal with the decisive questions: How does the aesthetic, or more concretely the literary, get into these chains of translations at all, or how does the operative process of a translation become a literary one?

46

Jörgen Schäfer | Reassembling the Literary

How, when, and through what does a specific aesthetic experience develop for the recipient? How does the translation between the agencies of human actors acting in “reality space” and those of literary characters in the fictional space of a story or a drama or those of an avatar in a computer game take place?22 Can moments of lyrical subjectivity still be achieved in the communication between human and machine (cf. Gendolla)? And what role does it play that one resorts to the traditional system of literary genres? Which intertextual references to other literary texts become effective in the chains of translations?

6.1

Non-Translatability: From Reader-Response Theories to Literary Pragmatics

Jens Schröter has made a quite promising suggestion for the use of ANT for aesthetic questions. He is asking for “konkrete und lokale Verfahren der Produktion erstens der ästhetischen Wahrnehmung . . . und zweitens des selbstbezüglichen Erscheinens” (‘concrete and local production methods of, firstly, aesthetic perception . . . and, secondly, self-referential appearance’) (Schröter 68) looking for the answer in the recursiveness of the chains as effect of their non-translatability: “Kunst” entsteht immer dann, wenn eine Kette aus Übersetzungen dazu führt, dass das spezifische materielle So-Sein eines . . . singulären Artefakts durch nichts übersetzt werden kann. . . . Das selbstbezügliche Erscheinen wird durch die zyklische Struktur der Kette von intransparenten Mediatoren produziert. (76f.) “Art” always emerges when a chain of translations results in the specific material Sosein (‘being thus and not other’) of a . . . singular artifact not being translatable. . . . The self-referential appearance is produced by the cyclical chain-structure of non-transparent mediators. On the one hand, Schröter’s ideas can be connected to traditional paradigmata of literary theory identifying aesthetic experience in disruptions of recursive loops: The point of departure of Russian Formalism, for example, was that it is one of the functions of literature to perform a deliberate, sensually perceived disruption of automated reactions (“defamiliarization”) (Shklovsky). Wolfgang Iser’s reader-response theory developed the notion of the “gap” or “blank,” that always develops at that point where “there is an abrupt juxtaposition of segments . . ., breaking the expected order of the text” (The Act of Reading 195). On the other hand, however, Schröter’s suggestion to examine local and concrete practices point at a general desideratum of the approaches mentioned

47

Jörgen Schäfer | Reassembling the Literary

that intensifies in computer-based processes of communication: Literariness cannot be understood as a merely text-controlled phenomenon as this is seen, for instance, by Formalism and in Iser’s early approaches,23 whose “implied reader” expressly is not an empirical reader, but denotes first of all an aesthetic effect that emerges between readers and texts. Therefore, the “implied reader” as a kind of readerly role model within the text does not primarily establish an agency of the reader but rather one of the text. But neither can it be explained as a result of a process merely controlled by knowledge privileging each reader’s subjective experience in which the socio-cultural normalizations by “interpretive communities” (Fish) show up. Below I can only indicate with one specific example how, to my mind, the traditional methods of literary studies have to be updated in order to be able to use them for the analysis of literary reception in computer-based media. To do this I will go back to existing reader-response theories, especially those by Wolfgang Iser and Hans Robert Jauß. Both of them already have developed a concept of literary studies taking into consideration action- and communication-theory so as to (re-)construct the processes of literary production and receptionand thus the different creations of meaning for one text by readers with different reception-tendencies.24 But also these approaches cannot fully describe the exchange between the textual structures and the recipients’ subjective level of knowledge as a dynamic, interactive control loop. Ratherthis is the justified objection by Sven Strasenall theories of aesthetic reception attempted to analyze the complex interdependencies of the agencies involved in this processone could also call it the tracing of the chains of translations“indem sie jeweils einer dieser Instanzen ihre Dynamik austreiben, um sie dann zum stabilen Ausgangspunkt einer linearen Ursache-Wirkungs-Kette zu machen, die den Prozeß der Bedeutungszuweisung beschreiben soll” (‘by muting the dynamics of one of these agencies in order to make it into a stable starting point of a linear chain of cause and effect that is supposed to describe the process of establishing meaning’) (22). Already in Iser’s The Act of Reading the constitution of meaning was attributed to the “interaction between the textual signals and the reader’s acts of comprehension” (9); i.e., semantic attribution in the production of meaning had already then been conceived of as an interactive process of mediation between textually and knowledge-controlled processes. According to Iser, an entire process is manifested in the textfrom the author’s world-view to its becoming noticeable by the readerthat is, a process in which, however, the processes of establishing meaning are only just selective realizations of the text. Against the backdrop of the “sense of possibility,” the fundamental plurality of the text is selectively disambiguated.

48

Jörgen Schäfer | Reassembling the Literary

Central building blocks of Iser’s theory are the so-called “blanks” or “gaps”: They are hinges between different schematized perspectives of representation of a text and the (mental) activities of the reader. They introduce a disruption into the act of reading that causes an “impeded process of ideation” (188). Thus, we are dealing with “potential connections” (182) in which the earlier attributions of meaning by the reader are disrupted; he therefore has to test and possibly revise them in order to make the text coherent again. For this, Iser saw two possibilities: the range of semantic horizons either can be “narrowed down” or “modified” (111) so that they establish on the temporal axis of reading a “dialectic . . . between illusion-forming and illusion-breaking” (127).25 The specific oscillation between involvement and distance, i.e., that which Jauß calls “Selbstgenuss im Fremdgenuss” (‘self-enjoyment in the enjoyment of something other’) (Ästhetische Erfahrung und literarische Hermeneutik 84) in the recipient calls up the specific aesthetic experience. Iser describes these interactions of the reader with the literary text and its blanks as a cybernetic mechanism: If we view the relation between text and reader as a kind of selfregulating system, we can define the text itself as an array of sign impulses (signifiers) which are received by the reader. As he reads, there is a constant “feedback” of “information” already received, so that he himself is bound to insert his own ideas into the process of communication. . . . The dynamic interaction between text and reader has the character of an event, which helps to create the impression that we are involved in something real. . . . In literature, where the reader is constantly feeding back reactions as he obtains new information, there is just such a continual process of realization, and so reading itself “happens” like an event, in the sense that what we read takes on the character of an open-ended situation, at one and the same time concrete and yet fluid. . . . The text can never be grasped as a whole—only as a series of changing viewpoints, each one restricted in itself and so necessitating further perspectives. This is the process by which the reader “realizes” an overall situation. (The Act of Reading 67f.) This long quote is illuminating because with the example of the reading of a printed text Iser designs a scenario thatbecause of its emphasis on feedbacks between the text and the reading processalso provides some theoretical building blocks for the literary processes in computer-based media. In changed media dispositives the mediation and the change of different horizons as fundamental conditions of aesthetic experience will have to be proven, e.g., the initial horizon of expectations “als paradigmatische Isotopie, die sich in dem

49

Jörgen Schäfer | Reassembling the Literary

Maße, wie die Aussage anwächst, in einen immanenten, syntagmatischen Erwartungshorizont umsetzt” (‘as paradigmatic isotopy, which is transposed into an immanent syntagmatic horizon of expectations to the extent that the utterance grows’) (Jauß, Literaturgeschichte als Provokation 175) will eventually have to be mediated with the reader’s horizon of experience. Nonetheless, at least two problems remain: Firstly, the dynamics of the text leaves quite a bit more to be desired than becomes visible at first glance. Iser’s text is an object that is fixated on the medium book whose materiality as a printed “immutable mobile”even though it is “dynamized” in the act of readingnevertheless cannot be questioned. Secondly, this concept seemingly is aimed at the actions of the reader, while the elements controlling the reading process in reality are hidden in the text.26 This is also true for the famous “implied reader,” a textual figuration to which the empirical reader has to adapt: The constituting of meaning and the constituting of the reading subject are therefore interacting operations that are both structured by the aspects of the text. However, the reader’s viewpoint has to be prearranged in such a way that he is not only able to assemble the meaning but also to apprehend what he has assembled. (152) However, physical, cognitive, psychological, social, cultural and media-technological factors of the empirical situation during the act of reading remain unheeded. In computer-based media, however, the “blank” between expectation and aesthetic realization sets in just as much as in the reading of a book; only the outline of a different semantic horizon triggered off in the process does not remain only in the imagination of the reader. Rather, the recipients are invited to physically interact directly with the sign processes, to promptly “continue writing” or to fill “gaps” which they themselves continue to create. If we speak of aesthetic reception as a frame of reference for expectations that can be objectified in texts and that then are varied, corrected and changed, thereby disrupting the horizon of expectation, then the here announced scope of change and reproduction definitely remains observable. However, it has emigrated from the relation between texts (e.g., courtly romances and their parodies in Cervantes’ Don Quijote) into a connection between texts, their dynamic projection onto diverse displays and the activities of their readers, including bodily activities. This can be illustrated using the example of Wardrip-Fruin et al.’s Cave installation Screen. At first, this installation surprisingly does not require anything more than reading three introductory texts, which are projected onto the three walls of the Cave, accompanied by a reading from the off. The texts talks of the virtuality and (in)stability of memories that “seem at times more there than

50

Jörgen Schäfer | Reassembling the Literary

the there we daily inhabit, fixed and meaningful texts in the indecipherable flux of the world’s words, so vivid at times that we feel we can almost reach out and touch them” (Coover 13). Continuing, we read: But memories have a way of coming apart on us, losing their certainty, and when they start to peel away, we do what we can to push them, bit by bit, back in place, fearful of losing our very selves if we lose the stories of ourselves. But these are only minds that hold them, fragile data, softly banked. Increasingly, they rip apart, blur and tangle with one another, and swarm mockingly about us, threatening us with absence. (14) It is precisely this experience of peeling away or losing memories that is the subject of the installation. The introductory texts are followed by three narrated memories, which are short poetic descriptions of the moment of awakening, of the transition between a state of dreaming and being awake. In the meantime, the words separate in increasing speed from the projection walls, whirling around the recipient in the three-dimensional space; they remain legible but, as graphic and voluminous objects, they are also situated in space or, as Cayley notes, paradoxically “inscribed on the background that surrounds us” (“The Gravity of the Leaf” 213). By “batting” with the data glove, i.e., with playful movements of the body, words can be manipulated and moved to different positions on the walls: They literally fill the gaps in the text, thereby filling (and eliminating) the “memory gaps” that the text makes into a subject.27 Unlike the printed text, this filling of the blanks happens in a peculiar way: on the one hand, as a result of a sensually perceptible and machine-based activity, as closings of spatial gaps on the surfaces of the projection walls by which the recipients are “starkly, literally, confronted with the diegetic break on which language depends and by means of which we make it and it makes us as we write and read” (216). On the other hand, these gaps also mark blanks in Iser’s sense, i.e., those enclaves in the text that are offering themselves to the imaginary fillings by the recipients’ prompting them “to supply what is meant from what is not said” (The Act of Reading 168): Communication in literature . . . is a process set in motion and regulated . . . by a mutually restrictive and magnifying interaction between the explicit and the implicit, between revelation and concealment. What is concealed spurs the reader into action, but this action is also controlled by what is revealed; the explicit in its turn is transformed when the implicit has been brought to light. (168f.)

51

Jörgen Schäfer | Reassembling the Literary

Figs. 3-4. The texts of Screen are projected onto three walls (top). In increasing speed, words are separating from the walls (bottom). Courtesy of Noah Wardrip-Fruin.

52

Jörgen Schäfer | Reassembling the Literary

Fig. 5. Finally, the words are collapsing into the center of the Cave. Courtesy of Noah Wardrip-Fruin.

In a media-technologically quite specific way this oscillation between what is saidthat itself is changing (and indeed depending on the interactivity made possible by the media dispositive)and that which is made to appear in the imagination of the recipient is only stopped when the texts in the Cave finally collapse, i.e., when the basically unlimited semioses are broken off. On the basis of the actions of the recipients, a changed text is building up on the projection walls and with this also a changed memory is retained. Finally, the following text is read aloud: We stare into the white void of lost memories, a loose scatter about us of what fragments remain: no sense but nonsense to be found there. If memories define us, what defines us when they’re gone? An unbearable prospect. We retrieve what we can and try again. (14) In comparison to the reading of the identical text from a printed page, the experience in the Cave is fundamentally different: In the interplay of physical activities, computer-controlled processes, voices from the off reading parts of the text, and the continued reading by oneself, the recipient’s expectation is disrupted in a double sense: On the one hand, Screen does not attempt to create a virtual reality environment but instead confronts the recipient with a literary text. Thus, theadmittedly not yet widely spreadexpectations in the medium Cave are jarred. However, a new text, a new environment, a new readingexperience are simultaneously also interactively delineated which are made possible only by way of the multiple recursive co-operations of human intentions

53

Jörgen Schäfer | Reassembling the Literary

and mechanical processes. It is a mode of reception that Wardrip-Fruin calls “peripheral reading” (“An Interview and New Work”) of a variable text, that in all probability cannot be generated in identical form, not even after training for months. In order to describe these literary processes the traditional reader-response approaches do not suffice. Therefore their approaches must be supplemented in terms of an empirically verifiable literary pragmatics that focuses on the description of concrete literary chains of translations between human writers and readers (as well as additional actors such as programmers, interface designers, etc.), medialized texts and literary conventions. Such a pragmatic approach then has to conceptionally and methodically measure up to the agency distributed between human and non-human actors, also taking the physical activities of the recipients into consideration. The strength of pragmatic approaches lies in the fact that they are aiming at mediating between textually controlled processes and those that are controlled by knowledge when creating textual meaning in concrete, spatially and temporally situated relationships of communication. Notably in computerbased media, in which posits of an endless semiosis seem particularly convincing and thereby complicate the engagement of literary studies with the objects, Roger D. Sell’s assumption that in practice semiosis “probably never continues without interruption” but inevitably leads to a point “at which we momentarily freeze semiosis in its tracks” (38) can possibly open up new paths for exploring the potentially endless and subjective attributions of meaning or the contextualizations of the empirical recipients in the moment of reading. Hans-Jörg Schmid has pointed out that the generation of a mental representation prompted by a literary text depends on a range of contextual factors encompassing six types of “activated knowledge”28 as well as the recipient’s specific emotional state in the given situation: The interaction between text and context is characterized by a constant reciprocal updating of the mental representation of the text under construction on the one hand, and the context on the other. . . . The only limit to the amount or kind of knowledge that can have some bearing on the mental representation a person construes in response to a given utterance is his or her total cognitive environment, which is potentially infinite. (Schmid 442f.) In order to expand these approaches that locate the openness of literary texts mainly in the indeterminacy of the contexts constituting meaning and that therefore include the material and social dimensions of these contexts, the existing approaches of literary pragmatics have to be expanded including the ear-

54

Jörgen Schäfer | Reassembling the Literary

lier ideas on the agency of non-human actors. If, for example, Sell defines the writing of a literary text as “a deed with an interpersonal valency across time and space, which can only be realized, furthermore, by a second kind of human act, an act of reading” (107), then he focuses only on the interpersonal relationship between a (human) author and a (human) reader. By contrast, he refrains from opening up the black boxes in which the various media and bodily steps of translation are enclosed that for example in the case of slippingglimpse or Screen have to be considered. But this is necessary if one wants to widen these approaches into a pragmatic theory of literary communication between human and machine (cf. par. 7 in this text).

6.2

Literary Knowledge: The Persistence of Genre Conventions

Before concluding with a methodical suggestion how this broadening of the reception theory could be realized, I would like to bring up an important factor that organizes the restriction of the fundamentally open contexts of meaning mentioned above: Literary communication is not possible without the internalized literary knowledge and the previous experiences of the recipients. This means that in the (individual and collective) history of literary reception, readers inevitably create horizons of expectation developing between work and work, in which basic generic patterns are both taken up and at the same time modified. These patterns have been expanded today anyhow through experiences with other non-biblionomic media like film, video, computer games, etc., the knowledge of which will be taken for granted. With this in mind, contextual presuppositions as bodies of “extratextual information, which is encoded in the text as literary convention” (Randall 420), necessitate certain historically developed possibilities of action between author, text, and reader that also continue influencing literary communication in computer-based media. Literary knowledge is integrated within chains of translations in the works of electronic literature as well, thus becoming elements of an actor-network. But the agencies of the human actors are thereby transferred to machines integrating the users into technically controlled inter-actions. Here, three forms become apparent which I believe can still be correlated with the traditional literary genres, insofar as I do not understand genres as ontologically fixed, but as historically changing, dynamic systems of classification for communications.29 A possible solution to the dilemma of applying genre definitions to digital pieces has recently been foreshadowed by Joseph Tabbi who argues that more generic and more qualitative terms are needed in a time of transition: “narrativity or fiction more generally than novel, poesis more generally than poem,

55

Jörgen Schäfer | Reassembling the Literary

conceptual writing more generally than essay”—we could add “dramaticity” or “dramaticness” more generally than “drama.” Current media systems, however, isolate, assess or confront these forms in another way which we have to analyze carefully, especially regarding aspects of space and time: -

Narrativity: We experience narrative lines notably in the already mentioned locative narratives, i.e., in environmental, neighborhood and city projects with GPS-based media following literary patterns. Included are projects such as Jean-Pierre Balpe’s Fictions d’Issy or 34 North, 118 West by Jeff Knowlton, Naomi Spellman, Brandon Stow, and Jeremy Hight. Worldwatchers by Susanne Berkenheger and Gisela Müller is an example for projects on the Internet that locate the increasingly massive social control through video-monitoring (or other sensor systems) in the long literary discussion from perspectivism to dystopian concepts, updating them critically.

-

Poeticity: If we do not understand the poetical only as an effect of literary procedures—in Roman Jakobson’s sense of “poetic function” (358)— but as a re-projection of socio-historical and technical conditionings of the body to its sensual self-perceptions, then it is materialized in mixed reality environments such as the Cave or interactive camera-projection systems. This means that it is transferred from the imaginary realm into the immediate haptic, acoustic and visual realm of perception. Apart from Cave projects like Noah Wardrip-Fruin et al.’s Screen or John Cayley’s lens, literary installations such as Text Rain by Camille Utterback and Romy Achituv, Daniel Howe’s text.curtain (2005) or Simon Biggs’ reWrite (2007) and reRead (2009) also belong here.

-

Dramatic art: In stagings of inner realms and environments, real characters (from simple users to trained actors) and artificial ones (from avatars, software agents, etc., to complex AI-programs), following quite classical dramatic patterns of activity, are involved in dialogues (cf. Schäfer, “Looking Behind the Façade”). Among these belongs for example the interactive drama Façade by Michael Mateas and Andrew Stern for which an augmented-reality version has been developed (Dow et al.).

7

Opening the Ethnographer’s Toolbox: Interactivity Experiments and Participant Observation

The findings described above demand drawing methodological consequences. I believe that in order to be able to describe quite concrete, distributed aes-

56

Jörgen Schäfer | Reassembling the Literary

thetical processes with computer-based and networked media, particularly those using site-specific media in physical space or in computer-aided mixed reality environments, instruments have to be developed that also include methods from media ethnography, “workplace studies” and qualitative social research. Therefore my own thoughts regarding future work are aimed at directing the results of the above-mentioned theoretical and methodological considerations into concrete pragmatic activities, combined in a further step with questions of literary studies. One could, for example, on the one hand think about conducting media ethnographic observations of locative narratives as systematic combinations of participant observation, recording and surveying computer logs—thereby identifying patterns of interaction in concrete contexts of action and communication. Media ethnographyespecially “Technography” (Rammert and Schubert) and “Virtual Ethnography” (Hine)has developed methods that can also be used for the analysis of literary activities of human and non-human actors with symbolic artifacts. Empirically detailed studies for all components have to be pursued recognizing concrete literary actor-networksand in computer-based and spatially determined mixed environments this includes the physical movements of human actors, their literary knowledge, the media-technological and spatial conditions, and the institutional parameters, etc. In contrast to the fieldwork of classic ethnography, the approaches of a “focused ethnography” are well suited for the integration of questions of literary studies since they are based on short-term but data-intensive field trips in areas known to the scholars. Therefore, they can concentrate on the specifically literary frames of activity and communication and present the knowledge of their discipline in a targeted manner. In participant observation both the activity of other actors can be observed and recorded while one’s own experience of the analyzed practices can be described. The meanings and subjective perspectives of other human actors can be determined through interviews. And since we are dealing with a spatially and temporally assessable field, the intersubjective observation of data (specifically in transdisciplinary research groups, cf. Knoblauch) can be enabled through the use of the currently available technical methods of recording. On the other hand, working with the Cave could allow for conceptualizations and realizations of experiments of structured interactions in which concrete human-machine interactions can be recorded and documented. In this sense, the Cave can be utilized as an “experimental system” (Rheinberger), i.e., as a hybrid configuration in which aesthetic experiences of the interaction between literary factors such as genre traditions, roles of literary plots, etc., and technical conditions such as computers, networks and displays are all made possible.

57

Jörgen Schäfer | Reassembling the Literary

This would demand active participation putting the recipient—and especially the literary scholar—into a situation in which he has to introduce his historical and systematic knowledge just as much as his pragmatic competence in the interaction with media interfaces. At the same time he has to continually revise his own horizon of expectation, adjusting it to technically determined constellations the combinatorial possibilities of which as a rule he cannot grasp. In exemplary studies it could be examined in what way literary knowledge, physical action and technical systems are related to each other and how they reciprocally shape each other. Besides, in such experimental settings individual factors within different experimental sequences can be varied in a controlled way. Additionally, the micro-level of concrete human-machine interaction can be made accessible for description by analyzing computer-protocols (cf. Hahne et al. 279). This means that by observing all human and non-human actors, recurring or typical patterns of action of the actorse.g., spatial patterns of distribution, temporal routines and patterns in the sequences of interaction, i.e., in the sequence and distribution of acts of activitycan be identified.30 For example, an analysis of Screen would initially attempt to register how the bodily movements of different recipients and the movements of the letters are related to each other and which textual variations emerge from this interplay. Added to this with video recordings, the physical activities of the recipients can be recorded and documented by analyzing the computer protocols of the VR system’s performance. In this way possibly intersubjective patterns can be determined. In a further step, it could then be established which different semantic representations in the episodic memory were created in these concrete acts of reception. In structured interviews, the attributions of meaning that the different recipients create would have to be approximately determined, taking their contextual literary knowledge into consideration.

8

Conclusion

I have attempted to sketch a theoretical framework that allows integrating the specificities of literary communication in computer-based and networked media—in particular in spatially determined media dispositives—in such a way that it relates the distributed agency between human and non-human actors, the transcriptivity of language and the “artificial intelligence” of computers, the bodily experience of recipients and the mental representations and, last but not least, the literariness of these processes to each other.

58

Jörgen Schäfer | Reassembling the Literary

This framework is understood as an attempt to open up a pathway upon which research can possibly live up to the demand that N. Katherine Hayles has rightfully made: Nothing works well when the focus narrows to the solitary individual considered in isolation; everything works when things are situated in relation to one another. (“Distributed Cognition” 27) Translated by Brigitte Pichon and Dorian Rudnytsky

Notes 1

The important differentiation between actants and actors in ANT is undecided. Therefore I have decided on a use that is based on different states of figuration. An actant is the smallest prefigurative unit that becomes an actor when it is networked and therefore obtains action potential of its own within this network: “any thing that does modify a state of affairs by making a difference is an actor—or, if it has no figuration yet, an actant” (Latour, Reassembling the Social 71).

2

The Actor-Network Theory is in no way a systematic, closed theoretical construct. Between its main representatives Bruno Latour, Michel Callon, John Law, or Antoine Hennion the important points can indeed emerge with differences and disagreements, even in the definition of the key terms. In my references to ANT below, I will proceed at first heuristically analyzing different aspects separated from each other that in reality are connected to each other by the logic of translation.

3

Of course Latour does not talk of “the literary” but of “the social.”

4

Latour here says “social” and not “literary.”

5

The etymological origin of “literature” can be traced back to the Latin “littera” (‘letter’) which initially denoted all kinds of written texts (cf. Schäfer, “Sprachzeichenprozesse”).

6

To be exact, ANT differentiates between “intermediaries” and “mediators”: “An intermediary . . . is what transports meaning or force without transformation: defining its inputs is enough to define its outputs. For all practical purposes, an intermediary can be taken not only as a black box, but also as a black box counting for one, even if it is internally made of many parts. Mediators . . . transform, translate, distort, and modify the meaning or the elements they are supposed to carry” (Latour, Reassembling the Social 39).

59

Jörgen Schäfer | Reassembling the Literary

7

Clearly, ANT to a large extent is based on semiotic theories, especially on the works of Algirdas J. Greimas, e.g., on his Structural Semantics. Cf. Latour: “It would be fairly accurate to describe ANT as being half Garfinkel and half Greimas: it . . . has found ways to tap the inner reflexivity of both actor’s accounts and of texts” (Reassembling the Social 54n54).

8

The idea of the circulating reference was described by Latour quite concisely in his famous ethnography of geological studies from the Brazilian jungle. According to Latour, the sciences are dealing with reality always only “in the form of two-dimensional, superposable, combinable inscriptions” (Pandora’s Hope 29), i.e., with “representations, that seem always to push it [the world] away, but also to bring it closer” (30). These representations (diagrams, texts, etc.) are connected to their original context only by way of chains of translations; here, tools like the pedocomparator represent in a manner of speaking hybrids of thing and signs allowing the creation of these chains of translations, i.e., to substitute thing and sign. This, according to Latour, is an expansion of the semiotic models or rather a generalization and a transfer to things/objects, as a “study of order building or path building and may be applied to settings, machines, bodies, and programming languages as well as texts; . . . the key aspect of the semiotics of machines is its ability to move from signs to things and back” (Akrich and Latour 259).

9

Thus, regarding Latour’s assumption of the hybridity of thing and sign, Georg Kneer criticizes that the links of the chain of signs do not appear as hybrids. According to him, it is not an ontological differentiation, but one that is always done by observers. Therefore signs function only as signs and specifically the fulfillment of its functioning as sign presupposes an abstraction from all material characteristics (295).

10 Since all speech creates such a surplus, it still has to be cleared up more specifically where the specifics of literary language use can be located. First ideas regarding this will be reflected upon in paragraph 6.1. 11 Bernard Robben calls “notation” the relationship between the code and the perceivable representation, i.e., “die Form der Über-Setzungen des Mediums Computer, eine prozessierende Relation zwischen Kode und Darstellung” (‘the form of the translations by the medium computer, a processing relation between code and representation’) (12f.). 12 An interesting variant of this conception has been pursued by Adam Parrish, who creates so-called “New Interfaces for Textual Expression,” i.e., textual instruments as hardware devices.

60

Jörgen Schäfer | Reassembling the Literary

13 For an in-depth discussion of Schemat’s project cf. Kwastek, “Geopoetics.” 14 Renate Grau has coined this term for the appearance of belletristic (print-) works as aesthetic artifacts that emerge in literary publishing houses via the “translations” between the different human actors of the “literary business” (like authors, lectors, literary agents, distributors, book sellers, etc.), different technologies of communication and production (paper, computer, software, printing machines, etc.), sales channels, institutions and competences (book stores, interim storage facilities, etc.) and aesthetic contextual knowledge (98). Here, she is following John Law’s “heterogeneous engineering” who argues “that the stability and form of artifacts should be seen as a function of the interaction of heterogeneous elements as these are shaped and assimilated into a network” (“Technology and Heterogeneous Engineering” 113). 15 In my opinion the most important points of criticism are that on the one hand it does not always become clear why different actors should be included in a network with the same intensity and with the same degrees of freedom, while on the other the two principles are defined so broadly that one cannot always differentiate enough whether an agency of their own is implied to technical systems, or whether an effect of agency is attributed to an entity from the perspective of an observer (Schulz-Schaeffer 21). 16 In the early poetry and story generators, the programmable, combinatorial procedures were delegated to the (mainframe) computers that processed the programs in batch mode. Only with interactive computing have the possibilities of distributed agency decisively widened. 17 The following overview does not intend to furnish a complete list of the actors involved; I simply want to point out the heterogeneity of the agencies. 18 Hayles notes: “An electronic text does not have this kind of prior existence. It does not exist anywhere in the computer, or in the networked system, in the same form it acquires when displayed on screen. After it is displayed, of course, the same kind of readerly processing may occur as with print. . . . In this sense electronic text is more processual than print, it is performative by its very nature, independent of whatever imaginations and processes the user brings to it, and regardless of variations between editions and copies” (My Mother Was a Computer 101). 19 Latour once has made use of the terms “worknet” or “action net” in order to underline this processuality: “If I believed in jargon and if worknet or action net had any chance to hold, I would offer it as a substitute so as to

61

Jörgen Schäfer | Reassembling the Literary

make the contrast between technical networks and worknets, the latter remaining a way for social scientists to make sense of the former. Work-nets could allow one to see the labor that goes on in laying down net-works: the first as an active mediator, the second as a stabilized set of intermediaries” (Reassembling the Social 132). 20 It is just this “networking” of the body via the senses that is currently technologically enhanced with the help of sensors and effectors, i.e., the natural limitations of the physical interior are made permeable in a new way. 21 Akrich and Latour define “antiprograms” as “all the programs of actions of actants that are in conflicts with the programs chosen as the point of departure of the analysis” (261). 22 I cannot here deal with this aspect for reasons of space. In my analyses of the interactive drama Façade I have considered this question (“Looking Behind the Façade”). On the question of “agency in relation to the fictional worlds of games and other playable media” cf. Wardrip-Fruin, Mateas, Dow and Sali. 23 This holds for the reader-response approach of Iser’s early books The Act of Reading and The Implied Reader, which he extended toward literary anthropology in his later seminal studies Prospecting: From Reader Response to Literary Anthropology and The Fictive and the Imaginary. 24 Other established theories of literature such as hermeneutics, formalism, systems or discourse theories or other versions of reader-response theory should also be critically reviewed: How do they conceptualize literariness? What do they regard as specific aesthetic qualities of texts? Are any of their key terms and conceptions such as “defamiliarization” (Shklovsky), “interdiscourse” (Jürgen Link), “autopoiesis” and “communication” (Luhmann), and so on relevant for analyzing literature in computer-based media? 25 Iser argues that “every moment of reading is a dialectic of protention and retention, conveying a future horizon yet to be occupied, along with a past (and continually fading) horizon already filled; the wandering viewpoint carves its passage through both at the same time and leave them to merge together in its wake” (The Act of Reading 112). 26 Here Iser’s theory regains a certain precision: Strasen’s main criticism is that Iser’s theory privileges the control of the reader by textual structures. In computer-based media, however, this question of control has to be asked on quite a different technical foundation and in an expanded version: namely, the additional question has to be asked in what way the text

62

Jörgen Schäfer | Reassembling the Literary

and therefore also the activities of the reader are controlled by running programs and by interfaces that demand not only interpretative but quite concrete physical additional activities. 27 Apart from this some “struck” words dissolve into their syllables and can afterwards converge in neologisms. 28 These types of activated knowledge encompass “mental representations in the immediate past” that have been kept in an activated state; knowledge about other discourse participants; individual goals and expectations in the given situation; knowledge of the situation; “knowledge about the speech act in which one participates”; and, last but not least, “general world knowledge” (Schmid 437ff.). 29 Iser notes: “Literary texts contain a range of signals to denote that they are fictive. . . . More important than the repertoire [of these signals] is the fact that these signals are not to be equated exclusively with linguistic signs in the text. . . . For these signals can become significant only through particular, historically varying conventions shared by author and public. Thus the signals do not invoke fictionality as such but conventions, which form the basis of a kind of contract between author and reader, the terms of which identify the text not as discourse but as ‘enacted discourse.’ Among the most obvious and most durable of such signals are literary genres, which have permitted a wide variety of contractual terms between author and reader. Even such recent inventions as the nonfiction novel reveal the same contractual function, since they must invoke the convention before renouncing it” (The Fictive and the Imaginary 11f.). 30 An interesting co-operation of literary scholars and cognitive scientists is conducted by the British research project Poetry Beyond Text: Vision, Text and Cognition located at the universities of Dundee and Kent. Research teams try to find out how readers respond to the visual aspects of poetry by combining reader-response theories with psychological methods such as eye-tracking and pupil dilation: “We will assess how reading strategies affect memory, interpretation and perceived aesthetic value, using both quantitative measures and reader-response theories. We have developed, on the basis of previous work in this area, a strategy of a ‘reflective feedback loop,’ in which participants in experiments are regarded as co-researchers. Their cognitive processes will be assessed, using various experimental methods, while they are reading the various types of poetry. . . . Crucially, these results will also be presented to participants, who will be asked to write their own responses, allowing us to explore their aesthetic experience and interpretation of the poems, before and after receiving

63

Jörgen Schäfer | Reassembling the Literary

such feedback. Furthermore, participants on appropriate degree programmes, together with poets and artists will be invited to create works in response to the investigations” (Roberts et al.).

Works Cited Aarseth, Espen J. Cybertext: Perspectives on Ergodic Literature. Baltimore: Johns Hopkins UP, 1997. Akrich, Madeleine, and Bruno Latour. “A Summary of Convenient Vocabulary for the Semiotics of Human and Non-Human Assemblies.” Shaping Technology, Building Society: Studies in Sociotechnical Change. Ed. Wiebe E. Bijker and John Law. Cambridge, MA: MIT P, 1992. 259-264. Berkenheger, Susanne, and Gisela Müller. Worldwatchers. 2003. 28 Aug. 2009 . Biggs, Simon. reRead. 2009. 28 Aug. 2009 . ———. reWrite. 2007. 28 Aug. 2009 . ———. “On Navigation and Interactivity.” p0es1s: Ästhetik digitaler Poesie/The Aesthetics of Digital Poetry. Ed. Friedrich W. Block, Christiane Heibach and Karin Wenz. Ostfildern-Ruit: Hatje Cantz, 2004. 181-192. ———. The Great Wall of China. 1996. 28 Aug. 2009 . Breidbach, Olaf. Neue Wissensordnungen: Wie aus Informationen und Nachrichten kulturelles Wissen entsteht. Frankfurt a.M.: Suhrkamp, 2008. Breton, André. Nadja. Trans. Richard Howard. New York: Grove, 1994. Callon, Michel. “Techno-Economic Networks and Irreversibility.” A Sociology of Monsters? Essays on Power, Technology and Domination. Ed. John Law. London: Routledge, 1991. 132-161. Cayley, John. “The Gravity of the Leaf: Phenomenologies of Literary Inscription in Media-Constituted Diegetic Worlds.” In this book. 199-226. ———. “Screen Writing: A Practice-based, EuroRelative Introduction to Electronic Literature and Poetics.” Third Text 21.5 (2007): 603-609. ———. “Writing on Complex Surfaces.” Dichtung Digital 7.2 (2005). 28 Aug. 2009 .

64

Jörgen Schäfer | Reassembling the Literary

———. riverIsland. 2004. 28 Aug. 2009 . ———. translation. 2004. 28 Aug. 2009 . ———. lens. 2006. 28 Aug. 2009 . Chaouli, Michel. “Was bedeutet: Online lesen? Über die Möglichkeit des Archivs im Cyberspace.” Digitale Literatur. Ed. Roberto Simanowski. Munich: Text + Kritik, 2001. 65-74. Coover, Robert. “Cave Writing: New Adventures in Mot-Town.” 2004. 28 Aug. 2009 . 1-14. Dow, Steven, Manish Mehta, Annie Lausier, Blair MacIntyre, and Micheal Mateas. “Initial Lessons From AR Façade, An Interactive Augmented Reality Drama.” Proceedings of the 2006 ACM SIGCHI International Conference on Advances in Computer Entertainment Technology, Hollywood. 2006. 2 Oct. 2009 . Eisenstein, Elizabeth L. The Printing Press as an Agent of Change. Cambridge: Cambridge UP, 1979. Fourmentraux, Jean-Paul. Art et Internet: Les nouvelles figures de la création. Paris: CNRS, 2005. Fischer-Lichte, Erika. Ästhetik des Performativen. Frankfurt a.M.: Suhrkamp, 2004. Fish, Stanley. Is There a Text in This Class? The Authority of Interpretive Communities. Cambridge, MA: Harvard UP, 1980. Fuchs, Thomas. “Embodied Cognitive Neuroscience and its Consequences for Psychiatry.” Poiesis and Praxis 6 (2009): 219-233. ———. “Corporealized and disembodied minds: A phenomenological view of the body in melancholia and schizophrenia.” Philosophy, Psychiatry & Psychology 12.2 (2005): 95-107. Gendolla, Peter. “Artificial Poetry: On Aesthetic Perception in ComputerAided Literature.” Literary Art in Digital Performance: Case Studies and Critical Positions. Ed. Francisco J. Ricardo. New York: Continuum, 2009. 167-175. Gendolla, Peter, and Jörgen Schäfer, eds. The Aesthetics of Net Literature: Writing, Reading and Playing in Programmable Media. Bielefeld: Transcript, 2007.

65

Jörgen Schäfer | Reassembling the Literary

Genette, Gérard. Paratexts: Thresholds of Interpretation. Trans. Jane E. Lewin. Cambridge: Cambridge UP, 1997. Giesecke, Michael. Der Buchdruck in der frühen Neuzeit: Eine historische Fallstudie über die Durchsetzung neuer Informations- und Kommunikationstechnologien. Frankfurt a.M.: Suhrkamp, 1991. Grau, Renate. Ästhetisches Engineering: Zur Verbreitung von Belletristik im Literaturbetrieb. Bielefeld: Transcript, 2006. Greimas, Algirdas J. Structural Semantics: An Attempt at a Method. Trans. Daniele McDowell, Ronald Schleifer, and Alan Velie. Lincoln: U of Nebraska P, 1984. Hagner, Michael. Der Geist bei der Arbeit: Historische Untersuchungen zur Hirnforschung. Göttingen: Wallstein, 2006. Hahne, Michael, Erik Lettkemann, Renate Lieb, and Martin Meister. “Going Data in Interaktivitätsexperimenten: Neue Methoden zur Analyse der Interaktivität zwischen Mensch und Maschine.” Technografie: Zur Mikrosoziologie der Technik. Ed. Werner Rammert and Cornelius Schubert. Frankfurt a.M.: Campus, 2006. 275-309. Hayles, N. Katherine. My Mother Was a Computer: Digital Subjects and Literary Texts. Chicago: U of Chicago P, 2005. ———. “Distributed Cognition at/in Work: Strickland, Lawson Jaramillo, and Ryan’s slippingglimpse.” Frame 21.1 (2008): 15-29. Hennion, Antoine. La Passion musicale: Une sociologie de la médiation. Paris: Éd. Métailié, 1993. Hennion, Antoine, and Line Grenier. “Sociology of Art: New Stakes in a PostCritical Time.” The International Handbook of Sociology. Ed. Stella R. Quah et al. London: Sage, 2000. 341-355. Hine, Christine. Virtual Ethnography. London: Sage, 2000. Howe, Daniel C. text.curtain. 2005. 28 Aug. 2009 . Iser, Wolfgang. The Act of Reading: A Theory of Aesthetic Response. Baltimore: Johns Hopkins UP, 1978. ———. The Implied Reader: Patterns of Communication in Prose Fiction from Bunyan to Beckett. Baltimore: Johns Hopkins UP, 1978. ———. Prospecting: From Reader Response to Literary Anthropology. Baltimore: Johns Hopkins UP, 1989.

66

Jörgen Schäfer | Reassembling the Literary

———. The Fictive and the Imaginary: Charting Literary Anthropology. Baltimore: Johns Hopkins UP, 1993. Jäger, Ludwig. “Epistemology of Disruptions: Thoughts on Some Principles of Cultural Semantics.” In this book. 71-93. ———. “Bild/Sprachlichkeit: Zur Audiovisualität des menschlichen Sprachvermögens.” Sprache und Literatur 37.2 (2006): 2-24. ———. “Zeichen/Spuren: Skizze zum Problem der Sprachzeichenmedialität.” Schnittstelle: Medien und kulturelle Kommunikation. Ed. Georg Stanitzek and Wilhelm Voßkamp. Cologne: Dumont, 2001. 17-31. Jakobson, Roman. “Linguistics and Poetics.” Style in Language. Ed. Thomas Sebeok. New York: Wiley, 1960. 350-377. Jauß, Hans Robert. Ästhetische Erfahrung und literarische Hermeneutik. Frankfurt a.M.: Suhrkamp, 1982. ———. Literaturgeschichte als Provokation. Frankfurt a.M.: Suhrkamp, 1970. Kablitz, Andreas. “Kunst des Möglichen: Prolegomena zu einer Theorie der Fiktion.” Poetica 35 (2003): 251-273. Kittler, Friedrich. Discourse Networks, 1800/1900. Trans. Michael Metteer. Stanford: Stanford UP, 1992. Kneer, Georg. “Hybridizität, zirkulierende Referenz, Amoderne? Eine Kritik an Bruno Latours Soziologie der Assoziationen.” Bruno Latours Kollektive. Ed. Georg Kneer, Markus Schroer and Erhard Schüttpelz. Frankfurt a.M.: Suhrkamp, 2008. 261-305. Knoblauch, Hubert. “Videography: Focused Ethnography and Video Analysis.” Video Analysis: Methodology and Methods. Qualitative Audiovisual Data Analysis in Sociology. Ed. Hubert Knoblauch et al. Frankfurt a.M.: Peter Lang, 2006. 35-50. Knowlton, Jeff, Naomi Spellman, and Jeremy Hight. Right as Rain: A Weather Dependant Love Sonnet. 2002. 28 Aug. 2009 . Knowlton, Jeff, Naomi Spellman, Brandon Stow, and Jeremy Hight. 34 North, 118 West. 2002. 28 Aug. 2009 . Kwastek, Katja. “Geopoetics: Aesthetic Experience in the Works of Stefan Schemat and Teri Rueb.” Literary Art in Digital Performance: Case Studies and Critical Positions. Ed. Francisco J. Ricardo. New York: Continuum, 2009. 84-105.

67

Jörgen Schäfer | Reassembling the Literary

Kwastek, Katja, ed. Ohne Schnur: Kunst und drahtlose Kommunikation/Art and Wireless Communication. Frankfurt a.M.: Revolver, 2004. Latour, Bruno. Reassembling the Social: An Introduction to Actor-Network-Theory. Oxford: Oxford UP, 2005. ———. Pandora’s Hope: Essays on the Reality of Science Studies. Cambridge, MA: Harvard UP, 1999. ———. “Drawing Things Together.” Representation in Scientific Practice. Ed. Michael Lynch and Steve Woolgar. Cambridge, MA: MIT P, 1990. 19-68. Law, John. “Notes on the Theory of the Actor-Network: Ordering, Strategy, and Heterogenity.” Systems Practice 5.4 (1992): 379-393. ———. “Technology and Heterogeneous Engineering: The Case of the Portuguese Expansion.” The Social Construction of Technological Systems: New Directions in the Sociology and History of Technology. Ed. Wiebe E. Bijker, Thomas P. Hughes and Trevor Pinch. Cambridge, MA: MIT P, 1987. 111-134. Mateas, Michael, and Andrew Stern. Façade. 2005. 28 Aug. 2009 . Nake, Frieder. “Surface, Interface, Subface: Three Cases of Interaction and One Concept.” Paradoxes of Interactivity: Perspectives for Media Theory, HumanComputer Interaction, and Artistic Investigations. Ed. Uwe Seifert, Jin Hyun Kim and Anthony Moore. Bielefeld: Transcript, 2008. 92-109. Parrish, Adam. “New Interfaces for Textual Expression.” 2008. 28 Aug. 2009 . Pullinger, Kate, Stefan Schemat, and babel. The Breathing Wall. CD-ROM. London: Sayle, 2004. Rammert, Werner, and Cornelius Schubert, eds. Technografie: Zur Mikrosoziologie der Technik. Frankfurt a.M.: Campus, 2006. 11-22. Randall, Marilyn. “Context and Convention: The Pragmatics of Literariness.” Poetics 14 (1985): 415-431. Rheinberger, Hans-Jörg. Toward a History of Epistemic Things: Synthesizing Proteins in the Test Tube. Stanford: Stanford UP, 1997. Ricardo, Francisco J. “Reading Discursive Spaces of Text Rain, Transmodally.” Literary Art in Digital Performance: Case Studies and Critical Positions. Ed. Francisco J. Ricardo. New York: Continuum, 2009. 52-68. Robben, Bernard. Der Computer als Medium: Eine transdisziplinäre Theorie. Bielefeld: Transcript, 2006.

68

Jörgen Schäfer | Reassembling the Literary

Roberts, Andrew Michael, et al. Poetry Beyond Text: Vision, Text and Cognition. 2009. 28 Aug. 2009 . Schäfer, Jörgen. “Looking Behind the Façade: Playing and Performing an Interactive Drama.” Literary Art in Digital Performance: Case Studies and Critical Positions. Ed. Francisco J. Ricardo. New York: Continuum, 2009. 143-161. ———. “Sprachzeichenprozesse: Überlegungen zur Codierung von Literatur in ‘alten’ und ‘neuen’ Medien.” Analog/Digital: Opposition oder Kontinuum? Zur Theorie und Geschichte einer Unterscheidung. Ed. Jens Schröter and Alexander Böhnke. Bielefeld: Transcript, 2004. 143-168. Schmid, Hans-Jörg. “An Outline of the Role of Context in Comprehension.” Proceedings of the Anglistentag 2002 Bayreuth, 29 Sept-2 Oct. Ed. Ewald Mengel, Hans-Jörg Schmid and Michael Steppat. Trier: WVT, 2008. 435445. Schröter, Jens. “Maßverhältnisse der Medienästhetik.” Medien in Raum und Zeit: Maßverhältnisse des Medialen. Ed. Ingo Köster and Kai Schubert. Bielefeld: Transcript, 2008. 63-84. Schüttpelz, Erhard. “Der Punkt des Archimedes: Einige Schwierigkeiten des Denkens in Operationsketten.” Bruno Latours Kollektive. Ed. Georg Kneer, Markus Schroer and Erhard Schüttpelz. Frankfurt a.M.: Suhrkamp, 2008. 234-258. Schulz-Schaeffer, Ingo. Zugeschriebene Handlungen: Ein Beitrag zur Theorie sozialen Handelns. Weilerswist: Velbrück, 2007. Sell, Roger D. Literature as Communication: The Foundations of Mediating Criticism. Amsterdam: Benjamins, 2000. Shklovsky, Viktor. Theory of Prose. Trans. Benjamin Sher. Normal: Dalkey Archive P, 1991. Strasen, Sven. Rezeptionstheorien: Literatur-, sprach- und kulturwissenschaftliche Ansätze und kulturelle Modelle. Trier: WVT, 2008. Strickland, Stephanie. E-Mail communication with author. June 9, 2009. Strickland, Stephanie, Cynthia Lawson Jaramillo, and Paul Ryan. slippingglimpse. 2007. 28 Aug. 2009 . Tabbi, Joseph. “Toward a Semantic Literary Web: Setting a Direction for the Electronic Literature Organization’s Directory.” 2007. 28 Aug. 2009. . Thielmann, Tristan. “Der ETAK Navigator: Tour de Latour durch die Mediengeschichte der Autonavigationssysteme.” Bruno Latours Kollektive. Ed.

69

Jörgen Schäfer | Reassembling the Literary

Georg Kneer, Markus Schroer and Erhard Schüttpelz. Frankfurt a.M.: Suhrkamp, 2008. 180-218. Thielmann, Tristan, Erhard Schüttpelz, and Peter Gendolla, eds. Akteur-Medien-Theorie. Bielefeld: Transcript, 2010. Forthcoming. Utterback, Camille, and Romy Achituv. Text Rain. Installation. 1999. 28 Aug. 2009 . Wardrip-Fruin, Noah, et al. “An Interview and New Work.” The Iowa Review Web 6 (2004). 28 Aug. 2009 . Wardrip-Fruin, Noah, Michael Mateas, Steven Dow, and Serdar Sali. “Agency Reconsidered.” 2009. 28 Aug. 2009 . Wardrip-Fruin, Noah, Josh Carroll, Robert Coover, Shawn Greenlee, Andrew McClain, and Ben “Sascha” Shine. Screen. 2002. 28 Aug. 2009 . Winkler, Hartmut. Diskursökonomie: Versuch über die innere Ökonomie der Medien. Frankfurt a.M.: Suhrkamp, 2004.

70

Ludwig Jäger

Epistemology of Disruptions Thoughts on the Operative Logic of Media Semantics 1

Semantics of Media and the Logic of Transcriptions

In recent times, the different disciplines of cultural studies have paid growing attention to the phenomenon that media intermedially refer to each other, that they intermedially allude to themselves in recursive loops—in other words that they consistently make their scriptures into the beginning point of resemantizations, of reformulations and of transcriptions. In Aesthetics and History of Art terms like “meta-art” (cf. Christoph Zuschlag) or “metapeinture” are wellestablished categories for 20th century art. For example, in his study The SelfAware Image: An Insight into Early Modern Meta-Painting, Victor Stoichita has considered self-referentiality a characteristic of modern painting; regarding modernism he speaks of an “autoreferential discourse in painting” (Zuschlag 177) and Mitchell in his Picture Theory stresses that “self-reference is a central issue in modernistic aesthetics and its various postmodern revisions” (35). According to Zuschlag, quotation and paraphrase can be seen exemplarily as “central characteristics of art in the 20th century,” representing forms of interpictorial allusion and reference as in “variation, parody, travesty, persiflage, pastiche, allusion and hommage” (172). References of this kind also characterize other media and cultural practices: remakes in film, appropriations in photography1 or cover-hits and samplings in popular music. It is maybe music like no other art form that is marked by these methods of processing self-reference, procedures of arrangement, contrafacture, parody, re-orchestration, revision, variation and improvisation (cf. Leopold; Buschmeier, Konrad, and Riethmüller). The non-scriptural medium of language as well seems to be characterized by the ability of recursive self-reference. It is able to quasi silence sections of speech and to edit them transcriptively2 by reformulating3 them—an autoreferential self-reading that Almuth Grésillon has identified as a characteristic operative constituent of literary writing (8). Beyond the self-referentiality of the writing-act as well—for example in Julia Kristeva’s term intertextuality or in Gérard Genette’s category of transtextuality—literary references can be found by which the texts acquire “a manifest or secret relation to other texts” (1) thereby generating the dense interconnections by which the literary universe is semantically characterized.

71

Ludwig Jäger | Epistemology of Disruptions

And lastly, according to a publication by Bolter and Grusin4, the logic of intermedial relations can be described as logic of “remediation” also in the era of “New Media.” According to Lev Manovich media spur on processes of “cultural reconceptualization” (47) both on the level of content and on the level of form by way of translation, transfiguration and transforming.5

2

A Small Epistemology of Disruptions: Semiological and Epistemological Explorations

No matter how diverse they may be regarding media and aesthetics, we now can identify a fundamental procedural logic of cultural semantics behind the multiplicity of “practices of self- and other-references”6 that shape procedures of language, of the arts, of communicative7 and new media (cf. Manovich) and therefore also significant aspects of cultural communication. In several of my works I have suggested understanding this procedural logic as logic of transcription (cf. Transcriptivity and Transkriptive Verhältnisse). My proposal is that into all these procedures of intra- and intermedial connections and references a wider operational logic of media is inscribed which can be applied just as much to media of speech, writing and image as to the so-called new media: For all these systems of media and signs, for music as well as for its phonographic recording or scriptural notation and transcription, for oral language and writing, for the digital images of hypermedia and the analogous images of the iconographic tradition it is characteristic that they organize the production and transformation of meaning in semiological procedures of inter- and intramedial references. I am hoping that in the course of my considerations here it will become clear that these procedures owe their operative meaning essentially to one dominant feature that I should like to characterize as epistemology of disruption. Guided by five heuristic hypotheses regarding intra- and intermedial references, in the following I will look more closely at this feature inscribed in the procedures of media, simultaneously attempting to outline it as epistemology of disruption.

2.1

Assuming Traces8

For my model of media semantics sketched out here, Brandom’s thesis is central that the ability of subjects to refer to objects of a transsemiotic world with signs has to be explained with the ability to refer to signs with signs. We could formulate this maxim also in the following way that “the idea of privileging in-

72

Ludwig Jäger | Epistemology of Disruptions

ference over reference in the order of semantic explanation” is necessary (Brandom, Articulating Reasons 1). By making inferential and referential references a priority, Brandom connects the representational content of concepts, of contentions and the convictions contained in these, connects in other words the “internal” sphere of the mental—semantics, if you will—to the external realm of media discursivity (cf. Brandom, Articulating Reasons 207ff).9 His term “articulating reasons” insofar can be seen as discursive reason or more generally reason of media: For Brandom cognitive operations as mental processes allowing for references are dependent on the expressive trace of their mediated appearance. Only here, in the mediated realm of communicative references develop subjects of possible referential activities and worlds to which reference can be made. The fact that Brandom privileges inferential references over referential ones (which still has to be discussed more closely later with regard to the principle of interpretation) has to be understood considering the simultaneously epistemological and semiological basic assumption that has developed in philosophy of language in the early 19th century on the grounds of the CartesianKantian ideas of subject and language. The essence of this hypothesis can be voiced in the following way: In the triplet “subject of perception” (res cogitans)—“sign”—“object of perception” (res extensa), the sign insofar plays a central role as it is a necessary condition for the origin and continued existence of the two other categories. Phrased with Stegmeier, . . . philosophy of signs shakes off the shadows of ontological and epistemological doubles by changing perspective: it neither takes its point of departure from the world nor from the self interpreting it but from the signs through which the world and the self and their difference can only show themselves. The sign is the absolute. (123124) Both the differentiation of the world and the emergence of consciousness referring to it are not possible without the mediated “detour” of semiological self-reading and interaction with others through signs; in other words they are not possible without intra- and intermedial references. Only the detour of a semiologically mediated self-lecture—which at the same time is interconnected in a complex network of cultural textures—allows the subject to constitute itself regarding its understanding of the world.10 Concerning our focus we could also say that for the subject the sign to a certain degree lodges itself in the epistemological relation of self and world as a constitutive moment of “disruption.”

73

Ludwig Jäger | Epistemology of Disruptions

This figure of a mediated redirection of the subject to itself, constituting its consciousness, was developed with an epistemological aim for the first time by Wilhelm von Humboldt in his philosophy of language and signs. Compared with the Cartesian-Kantian idea of the subject he maintained that the “inner intellectual work” of a solipsistic subject would be, in a manner of speaking, “transient without a trace” if it would not unite with the outward materiality of sound in speech. Without such a mediation of mental elements according to Humboldt “thought could not become lucid, idea could not become a concept” (7; 53). The outward materiality of speech, the performative element of signs in the horizon of networks for Humboldt is the only media realm in which it is possible to “bring thought recursively face to face with the subject from inside itself and confronting itself” (5; 455). “In all instances of language, without this consistently occurring previous transfer into an objectivity returning to the subject, no formation of concepts, no true thought would be possible” (5; 377).11 Only when reading traces, only during the process of rereading during which the mind meets with the mediated traces of its own mental acts, only in the transcription of mental acts into the semiological registers of mediality can terminological distinction make itself felt so that a subject of possible terminologically differentiated activities can be constituted. As Luhmann formulated later, the “constant differentiation of self-reference and reference (to the other: ‘Fremdreferenz’) in all operations of consciousness requires sign-structures” (Die Kunst der Gesellschaft 18). Already Humboldt had delineated the mediated autoreferential meeting of the mind with itself as a gesture of repetition. Only in the act of this iteration the repeated element is brought forth: The self-lecture of the mind takes on the form of naming of concepts in which, according to Humboldt “it is the naming which completes the emergence of the element to be named in the presence of the mind” (5; 436). In a metaleptic figure the antecedent element, the element to be signified, is constituted only in the retroactive act of signifying, as it were in a transcriptive act of rewriting. Insofar we could also say that in all forms of semiological reference a basic figure of original doubling is inscribed by way of which the element to be signified (the address of reference) only forms in the course of being signified (in the transcriptive act of reference). It might also be possible to connect this theory of traces back to Derrida, even though for him Humboldt would be the most improbable reference for these ideas. Derrida sketches, as did Humboldt before him, the theoretical outlines of a possibly to be called media transcendental condition of mentality that positions the exteriority of external sign-processes against the metaphysics of the “interiority of the soul” (Grammatology 34): He is especially opposed to the idea of a “transcendental signified” as “the face of pure intelligibility” (13), opposed to the “self-presence of the soul within the true logos.” (13).12 In a quasi

74

Ludwig Jäger | Epistemology of Disruptions

mediatheoretical gesture vis-à-vis the classical philosophies of signs and philosophies of the mind he insists on the fundamental importance of the signifier for the constitution of the signified; the antecedence of meaning of the sign and the mind creating it for him is always only to be made up in the retroactive processing via material signs. The privileging of the signifier for him is illegitimate because a signified cannot be existent independently of the phenomenality of the sign: “There is thus no phenomenality reducing the sign or the representer so that the thing signified may be allowed to glow finally in the luminosity of its presence” (Grammatology 49). Derrida—like Humboldt before him—then is attacking the metaphysics of presence, the idea of the direct “self-presence of the cogito” (12) and its significative accomplishments. He exposes the fact that the “deferring”13 through media needing “the exteriority of the signifier” (14) cannot be obliterated, a deferring that is absolutely indispensable for the process of the mind’s self-constitution and its production of meaning. Therefore, the antecedence of meaning—just like the mental system that generates it—for Derrida always necessarily refers to the belatedness of activity traces of the medium: The simple “anterior idea” or the “interior design” vis-à-vis its mediated processing is a prejudice (Writing 11). With Freud Derrida is looking at the transcription of the subconscious into a text of consciousness not as a process of a belated “translation” of an “original”: Rather, already the “originally” subconscious text is formed of archives which are always already transcriptions. . . . Everything begins with reproduction. Always already: repositories of a meaning which was never present, whose signified presence is always reconstituted by deferral, nachträglich, belatedly, supplementarily: for the nachträglich also means supplementary. The call of the supplement is primary, here, and it hollows out that which will be reconstituted by deferral as the present. (Derrida, Writing 211-12) The transcription is executed retroactively with a signifier that—as Humboldt had formulated—only completes the emergence of the element to be named in the presence of the mind. This means that meaning for Derrida, as for Freud and Humboldt, is not conceivable “in an originary or modified form of presence” (Writing 211). It is “always already” the result of iterative processes of “transcription” which “constitutes the past present as such” (214). In the “play of signification” the signifier does not substitute an antecedent meaning because this substitute of the sign, according to Derrida, does not substitute anything “which has somehow existed before it” (280). We could say in Husserl’s terminology that

75

Ludwig Jäger | Epistemology of Disruptions

meaning is existent in its semantic evidence not as “having something-itself” but as the result of a discursive accomplishment of a belated “intentionality that consists in the giving of something-itself”14 (157-58). In short it is only the result of a transcription, of a belated semiological reference that is simultaneously constitutive for the antecedent one. As Cassirer formulates: “The content of the spirit is disclosed only in its manifestations” (86). At the end of my little digression into theories of traces we can make a note of a first result with regard to the semiological and epistemological frame of a theory of reference: Transcriptive references are procedures of media constituting mentality; they are the operative mode in which the mind metaleptically discovers its own traces. As modes of epistemological disruption they suspend the immediacy of self- and world-references into which they inscribe themselves as procedures of media.

2.2

Assuming Principles of Interpretation

It points to a another significant epistemological semiological requirement for a theory of transcriptive reference that cultural semiosis is the systematic area in which subjects constitute their mental identity by way of the traces of mediated acts of signification. To be precise, semiosis is not only the social field of subject-constitutive achievements. It is at the same time the semiological stage on which these cultural worlds constitute themselves in their terminological order within which and with reference to which the subjects are acting. A premedial world of possible references, or better a premedial “language of the mind” then can exist just as little, as there can be subjects who pre-exist their acts of signification as present cogitos. For the genesis of cultural semantics this means: Constituting and verifying meaning cannot be achieved by way of the particular referential comparison of signs-systems with a real world that is media transcendent. Rather, the semantic ratification of meaning can on the one hand have its place only within the horizon of mediated and semiological systems of representation and on the other hand only in the reciprocal reference between them. As Niklas Luhmann (Wissenschaft der Gesellschaft 75) has observed: A “world that is invariant to observation” is not available for any assessment of the degree of adequation of semiological systems of representation: “. . . the world cannot be observed from the outside, only from within itself. This means that it can only be observed in accordance with conditions that it provides itself” (75). Validity and semantic evidence of cultural meaning generated by linguistic and non-linguistic media therefore are owed to a principle that we could call the principle of interpretation. Using signs referentially in order to refer to a world

76

Ludwig Jäger | Epistemology of Disruptions

transcending signs is not conceivable as a fundament for sign-systems constituting meaning.15 From an ontological “world-honeycomb” we will collect just as little semantic honey as from the mental activities of a pre-linguistic spirit. Systems of language and media that remain restricted to a function of semantic reference cannot be definitorial systems—as we have seen from Brandom’s argument. The paradigmatic elements of natural languages seem to be a characteristic of media systems as a whole: namely that they have to be able in a constitutive sense to focus the semiological means through which their users interact and refer to the world making these means into a subject as the very media of interaction and reference thereby arranging and adapting them autoreferentially or in interactive semiological activities in the interest of communication.16 The ability to quote, to paraphrase, to explicate, to explain or to interpret scriptures that one created oneself or obtained in communicative acts in order to reassess its purpose has to be seen as a capacity that is constitutive for the operative knowledge of actors moving in the sphere of mediality. Before they are able to refer to the world with signs, these actors have to be able to play the game of referring to signs with signs. This means that the meaning of signs has to be at their disposal in such a way that their linkage with other signs in the system of a language or of a non-linguistic media-system can be called upon, i.e., that the semantic networked knowledge17 can be activated transcriptively in referential activities if necessary. The representative-referential relation of sign and world would in itself be entirely insufficient to ensure semantic references since references require the ability to refer signs to each other inherently within the sign-system. According to Deacon “symbols do not directly refer to things in the world, but indirectly refer to them by virtue of referring to other symbols” (99). The media-immanent genesis of meaning then follows a semiological law that Peirce has formulated in the following way: “From this proposition that every thought is a sign it follows that every thought must address itself to some other, must determine some other, since that is the essence of a sign. . . . every thought must have been interpreted in another thought” (Writings 173). According to Peirce there is “no exception, therefore, to the law that every thought-sign is translated or interpreted in a subsequent one” (224). For language this means: To be able to refer to the world with language requires that the semantic contents of the verbal comments with which speakers create reference can be explicated in their implication, illustrated in their obscurity, paraphrased in their incomprehensibility and legitimated and justified with regard to doubts, etc. In other words it requires that the speakers are able to enter a discourse of transcriptive—to a certain extent interpretive—reference to their own usage.18 Such a transcriptive ability is a fundamental principle of

77

Ludwig Jäger | Epistemology of Disruptions

processing cultural semantics. Even though this certainly still has to be analyzed more closely, this is valid also for other systems of media and signs—like certain forms of imagery or music—and especially their intermedial interplay. Epistemologically, semantics then are not owed to a realm of media-free cognition or any ontological order of “the world itself.” Their function in the different media then cannot be limited by the task to provide ways in which a premedial (“ontological”) world can be represented, depicted, or mirrored. We do not, according to Richard Rorty, dispose of any “transcendental standpoint outside our present set of representations from which we can inspect the relations between those representations and their object” (293). There is then no excentric Archimedean point of reference allowing us to judge the adequacy of our reference to the world independently of media-systems of representation. Rather, we have to rely on a semantics for which the principle of interpretation is constitutive. Semantics of this type are owed to sign-systems that to a certain extent only allow carrying out further semiological inscriptions into a world that has already been always semiologically structured.19 The genesis, adaptation and validity of meaning then operates by way of different means of reference that in an epistemological sense do not antecedently take place between sign-systems and the world20 but that mainly on the one hand are executed between diverse (mediated) sign-systems and on the other also within the same sign-system. Transcriptions are the basic procedures that on both levels of reference keep cultural semantics in motion,21 and this is the second result that with regard to a theory of reference I would like to make a note of. A second constitutive suspension, namely that of the possibility of the direct reference to a world of possible references corresponds to the suspension of a direct reference of a subject to itself that needs the “detour” of symbolic expression. The present epistemology of disruption then unfolds its striking force in a double suspension of epistemological directness that is replaced with procedures of mediated references.

2.3

Assuming Mediality

Assuming traces and interpretation then inevitably inserts the indirectness of mediality into the sphere of epistemological directness. An epistemology of disruption rehabilitates the “materiality of communication” (Gumbrecht and Pfeiffer) that was ignored and suppressed in the Aristotelian-Kantian tradition of theories of language and semiotics and just as much in its recent offshoots like, e.g., cognitive linguistics. This means that an epistemology of disruption

78

Ludwig Jäger | Epistemology of Disruptions

theoretically re-acknowledges the right of the materiality, performativity and aesthetics of sign systems and media systems. If it is correct that signs in the course of semiosis can neither resort directly to the variety of attributes of a world transcendent to signs nor directly fall back on the mental emanations of a prelinguistic “language of the mind,” then they themselves become the discursive realm of sense-production: Their function is then no longer limited to representation, to the transport or the transmission of contents because they constitutively contribute to the genesis of these contents. Therefore, each transmission/transfer of meanings as it were registers silenced semantic processes of constitution that, however, can be activated at any time. The elements to be transmitted first have to be determined, i.e., they have to be constituted in semiological activities of reference and can at any time—initiated by disruption—be repealed, denied, deferred or affirmed—in short, they can be transcribed in subsequent communicative activities. The generation of meaning insofar is closely connected with the mediality of the sign systems producing it. Indeed in linguistic and non-linguistic media the specific types of meaning are closely connected with the specific material sign substrata; they do not precede them as “neutral” cognitive forms.22 Assuming—as Wittgenstein still had done in his Tractatus using an Aristotelian model of mentality and mediality— that the logical form of a thought is identical both in thought and in a spoken or written sentence, re-uttering or replicating this thought now has to be refuted. Under conditions of transcriptivity the identical replication of a “cognitive original” cannot be achieved in different sign formats. As Danto expresses it, we can no longer assume “that the same form is quasi embodied in three different media: the medium of thought, the medium of writing and the medium of speech” (134). For example the semantics of images—or of other non-linguistic media—therefore does not consist in the fact that they say something in images that could also be said in language or some other way. Neutral contents/information that can be transferred unscathed between media cannot be imagined because there are only mediated variants of contents for which no premedial original exists. Any form of content-transfer from one medium into another therefore necessarily takes on the form of transcription, i.e., it becomes a new production under medially changed conditions. So far my remarks were supposed to give details of the theoretical consequences that result from assuming traces and interpretation leading us to a concept of media that is not compatible with the notion that media are technical means of information transfer. For semiological media and especially for language the concept of transfer23 through a “medium” is certainly inadequate. Peter Koch and Sybille Krämer correctly have underlined that “mediality for signs is not only a prerequisite for their transferability, but rather also for the generation of

79

Ludwig Jäger | Epistemology of Disruptions

meaning” (12). This observation is valid for the whole universe of discourses on mediality. As I have said before, the semantic contents of signs as a cognitive reservoir do not precede their transfer by way of signs. They are not fed into the discourse as transcendent signifiers; rather the discourse is the generic place where meaning is generated. The double suspension of directness (as introspection and reference) produced by the epistemology of disruption is then at the same time connected to a rehabilitation of the indirectness of mediality.

2.4

Assuming Recursion

As should have already become clear in the analysis of assuming traces and interpretations an important procedural mode of mediated discourse consists in the transcriptive reworking of mediated or symbolic utterances that were already generated beforehand in these discourses: As Bolter and Grusin have pointed out, “all media depend on other media in cycles of remediation” (55). Individuals and cultural systems organize semantic processes essentially through forms of autoreferential self-processing, redirecting the system recursively back to itself. It is this recursivity then, this notion of—as Niklas Luhmann calls it—“utilizing procedures for the results of these procedures”24 to which the inner coherence25 of media systems generating meaning immanently is owed (cf. Gesellschaft der Gesellschaft 217-218; 614; 888). The “utilization of system formation for the results of system formation” (614), in other words the utilization of communication for the results of communication according to Luhmann generates an “eigenbehavior,” a “production of ‘eigenvalue’” of the system that results in its relative stability and closure vis-à-vis other systems. The “eigenbehavior” of the system creates “relatively stable attitudes that arise when an operation is utilized for its own results” (1102). With regard to media- and communication systems the point of Luhmann’s idea26 therefore could possibly be summarized in the following way: Systems tend to generate eigensinn (i.e., significance of their own)27 as the profit from their characteristic behavioral procedures of recursive self-processing. Their eigensinn can thus be regarded as a direct effect of their operative logic. This is especially true for language which paradigmatically disposes of an indispensable characteristic to turn back recursively into itself, thereby making its own use of signs into the subject matter of further thematizing, commenting, paraphrasing, explicating or quoting with the use of signs, viz. making this use of signs into the object of self-referential semiological operations in which that element becomes visible that I would like to call the recursive transcriptivity of language.

80

Ludwig Jäger | Epistemology of Disruptions

Already speech operating outside of the procedural realm of writing has the capacity to isolate and silence parts of speech and to modify the silenced elements using a metaleptic form of autoreferentially referring language to language (cf. Jäger, “Versuch über den Ort der Schrift”). With script then new forms of intramedial references emerge; they can be called hypoleptic references28 of texts to texts. According to Bolter and Grusin here we also have to consider the remediation of a medium through another as the “defining characteristic” (45) of new digital media. All artifacts of silencing of our media cultures—images, texts, scores and all other storage—can then be understood as addresses of possible reference, of potential transcriptive addressing in the archives of cultural memory. Self-referentiality would then be one of the foundational sources of cultural semantics, a procedure in which by redirecting a symbolic system back to itself scriptures—i.e., media—are released from conditions of their antecedent discursive circulation and are temporarily silenced, quoted, paraphrased, or explicated for the adaptation or reintroduction into the semantic management under new conditions. In other words, they are deand then recontextualized. This means that if symbolic strings are released from the transitoriness of their performativity for any shorter or longer time this is no privileged characteristic of writing or other artifacts of silencing that are continuously generated by the procedures of media; it rather is a specific (transitory) state in communicative processes in which media temporarily become inflexible insofar as one can refer to them communicatively, can generate and process meaning. It is obvious that without the mediated traces of semiological activities such a process is not imaginable. Only in the aesthetic horizon of linguistic and non-linguistic semiological procedures does the realm for a possible constitution of meaning open up. Regardless whether the shorter or longer silencing of symbolic chains takes on a form of writing or of oral traditions ossified in “fixed texts” (Vansina 145) or that of paraphrase, explication or quotation in linguistic or pictorial media: In all these forms we are dealing with a recursive alternation of duration or silencing as well as intra- and intermedial movement that in principle is a foundation for the adaptation and constitution of meaning in the semantic economy of cultures. At the present state of my considerations we can thus sum up that transcriptive procedures are those procedures of metaleptic (autoreferential), hypoleptic or remediating referentiality of symbolic systems to symbolic systems by way of which the system of sense-making in cultural semantics is kept alive as a mechanism of continual de- and reconceptualization. The constitution of meaning always contains a movement of mediality in which, when executed, media refer to themselves and other media in recursive gestures: There can be no constitution of meaning without recursion. There are always traces of past mediations

81

Ludwig Jäger | Epistemology of Disruptions

to which we have to refer recursively in an interpretive manner, so that in the methods of cultural semantics meaning can be generated. Insofar we could say that recursive transcription is the mediated form of movement within which epistemological disruption occurs.

2.5

Assuming Disruption

By sketching assumptions with regard to the logic of reference I have attempted to outline an epistemological realm in which the epistemology of disruption could be located as a media-critique of the Cartesian-Kantian paradigm of subject and communication and as critique of immediacy of self- and worldreference. Of course it is the idea of communication in particular that is marked by the concept of disruption. To a certain extent therefore the assumptions outlined so far can be seen as requirements for a fifth hypothesis that I would like to call “assuming disruption.” If the processes of the cultural production of meaning are necessarily performed in the mode of intra- and intermedial referentiality of signs to signs, or of media to media then it is evident that in the language games of cultural semantics the states of unproblematic recognition of meaning have to be differentiated from those in which the game of transcriptive references, of “cultural reconceptualizations” is constantly renewed. However, also the states and moments of “undisturbed” semantics are marked by this irredeemable moment of “disruption” in the form of continuous fragility that can at any time initiate processes of remediation. As I have argued elsewhere in more detail, I would like to differentiate these two states of mediated communication—the one of unproblematic recognition of meaning and the other of its recurrent reconceptualization—as states of “transparency” and “disruption” (Jäger, “Störung und Transparenz”). And I understand disruption and transparency as two polarized functional conditions of media performance that are constitutively marked by the procedure of transcription. Transcription then could be described as the respective transition from disruption to transparency, of de- and recontextualization of the signs/media in focus. While disruption as the starting point of transcriptive procedures implements remediation, thereby focusing on the sign/medium as the (disrupted) operator of meaning, transparency can be looked at as that state in the process of media performance in which the respective sign/medium disappears, becoming transparent regarding the contents it mediates. Therefore transparency is not supposed to be (as has been variously assumed) seen as an almost ontological characteristic of media that in the words of Engell and Vogl tend to “become almost unperceivably anaesthetic” (10; cf. also Krämer, Sprache 157) regarding the aspects they are mediating; rather they

82

Ludwig Jäger | Epistemology of Disruptions

have to be seen as a functional state characterizing mediated communication in those cases in which the semantics argued by them are unproblematic regarding their validity. We could therefore say that disruption and transparency mark two modes of visibility that stand out against each other: the visibility of the medium and that of the mediated,29 as a rule reciprocally excluding each other. Since in this condition its fragility is not a subject, the transparency of the sign/medium enables the undisrupted impression of “realism” of the mediated whereas the fact that the medium becomes visible, i.e., the fact that the habitualized contexts of usage are obscured, is a sign for the approaching crisis of ontological semblance that fades with the mediated objects. As Goodman has remarked, realism is relative to the medium: “[It is] determined by the system of representation standard for a given culture or person at a given time” (Languages of Art 37) respectively by the media dispositive in which communication normally is taking place. The more familiar and transparent the used media are, the stronger the impression of realism presented by the symbolic media will be. Therefore, mediated presentations seem realistic when “practice has rendered the symbols so transparent that we are not aware of any effort, of any alternatives, or of making any interpretation at all” (36). In communicative situations of this kind the mediation of the “real” will be “obscured by our tendency to omit specifying a frame of reference when it is our own” (37). Here we can see that—as Bolter and Grusin have assumed—the desire “to get past the limits of representations and to achieve the real” (53) fundamentally is inscribed into our systems of mediation. It is only the erosion of habitualized frames of reference that allows making the mediated relativity of the real and thus the symbolic system of presentation visible again as “ways of worldmaking” (Goodman, Ways of Worldmaking). Of course, crises of this kind as disruptions of semantic balance are not only epochal turning points haunting cultural perspectives and world views from time to time; rather, they are also familiar transitional states for communicative processes that these universally go through according to their own logic of transcriptivity and that they also leave again when communication reenters phases of transparency, i.e., phases of an undisrupted validity of meaning. I would therefore like to call disruption any state in the course of communication producing the operative loss of transparency of a medium, thereby being perceived in its materiality. I would like to call transparency any state in which communication is not “disrupted,” i.e., in which the medium itself is not in the focus of attention in the sense in which Luhmann assumes that in the

83

Ludwig Jäger | Epistemology of Disruptions

interdependent relationship of medium and form the form becomes visible and the medium remains invisible.30 If we were to transfer the model of disruption and transparency to Luhmann’s distinction between medium and form, the disruption would be that state of communication in which we would not observe the form through the (invisible) medium but would observe with the medium the “contingency of creating forms” (Kunst der Gesellschaft 168) or rather “the free capacity of the media substratum to create ever new combinations” (200). And if we were to resort to Alfred Schütz we could also describe the state of disruption as the emergence of relevance of the medium and the state of transparency as its re-entry into the mode of familiarity.31 In other words, here my contention is that the transparency of the medium is not a “characteristic” of the medium but a condition that the medium takes on when the mediated semantics as silent knowledge is “undisturbed” and vice versa disruption is not a parasitic defect of communication but it is that communicative condition in which the sign/medium as such is visible and therefore becomes semantically operative. It is that state then which—once it occurs—is always connected to the need to remediate or transcribe without ever being able to interrupt this process of constant transformation of a fragile semantic balance. This means that the epistemology of disruption in the area of media communication marks a field of mediated operations that constantly oscillates between the fragile validity of unproblematic semantics on the one hand and that of processes of transcriptive reconceptualizations on the other.

3

Brief Summary

So far, my probings in the field of epistemological and semiological requirements for a theory of transcriptive referentiality can be tentatively summarized in the following way: The systems of signs and media that feed the semantics of cultures do not have any system-transcendent sources at their disposal in order to constitute or to assign evidence to meaning. This makes for their fragility and shakiness; here lies also their need to continuously enter into processes of remediating referentiality. In cases of confusion or disruption of established semantics the procedures of cultural semioses therefore necessarily have to refer back to previously constituted meaning and to reconceptualize (cf. Manovich) it culturally under specific discursive conditions thereby subjecting it to the procedural conditions of transcriptive referentiality. Intra- and intermedial transcriptions thus react to discontinuities of resonance, to displacements in the economies of individual and cultural awareness (cf. for this Franck), i.e., to confusions in those “overriding cultural options, standard val-

84

Ludwig Jäger | Epistemology of Disruptions

ues, and institutions that account for specific forms of attention” (Assmann and Assmann 17).32 However, they also react to the breaks by which political and cultural frameworks of media dispositives are continually affected. At this point the game of media referentiality begins to be stimulated: In the discourse of cultural communication now the unproblematic validity of traditional semantics is eroded, its evidence33 is blurred and in this blurred state is now fed into the media game of the transcriptivity-machine. One could therefore say that an epistemology of disruption irredeemably is at work when cultural semantics are remediated and reconceptualized. Translated by Brigitte Pichon and Dorian Rudnytsky

Notes 1

Regarding the problem of the transcriptive relation between original and copy in cases of samlings or appropriation art cf. Fehrmann et al., “Originalkopie: Einleitung.” Regarding the problem of appropriation art cf., e.g., Amelunxen; cf. also Aigner.

2

I speak of transcriptive editing regarding self-explicative and cooperative activities of communication like they have been analyzed in ethnomethodology, e.g., repairing sequences “in which one partner of the interaction —often in a secondary sequence . . . is correcting or specifying or editing a problem of communication signalized by the recipient of the statement” (cf. Selting). Regarding this problem cf. generally Jäger, “Störung und Transparenz.”

3

For this cf., e.g., Gülich and Kotschi.

4

Cf. Bolter and Grusin: “. . . we call the representation of one medium in another remediation, and will argue that remediation is a defining characteristic of the new digital media” (45). Cf. also (55): “It would seem, then, that all mediation is remediation. . . . No medium, it seems, can now function indepedently and establish its own separate and purified space of cultural meaning.”

5

Cf. Manovich: “In new media lingo, to ‘transcode’ something is to translate it into another format. The computerization of cvulture gradually accomplishes similar transcoding in relation to all cultural categories and concepts. That is, cultural categories and concepts are substituted, on the level of meaning and/or language, by new ones that derive from the computer’s ontology, epistemology, and pragmatics. New media thus acts as a

85

Ludwig Jäger | Epistemology of Disruptions

forerunner of this more general process of cultural reconceptualization” (47). 6

Cf. for these “practices of secondary degree” (Fehrmann et al. Originalkopie).

7

For this cf., e.g., Luhmann, Gesellschaft der Gesellschaft (190-412).

8

On the term “trace” cf. the publications in Fehrmann, Linz, and EppingJäger; also Krämer, Kogge, and Grube.

9

Cf. also Brandom: “Sapience of the sort distinctive of us is a status achieved within a structure of mutual recognition. . . . The specifically discursive character of that normative social structure . . . consists in the inferential articulation of those recognitive practices. We are the ones who give and ask for reasons for what we say and do” (Making it Explicit 275).

10 Even though the epistemological model in the semiology of the 19th century cannot directly be referred back to Bruno Latour’s Actor-NetworkModel (e.g., Latour’s “actors” do not necessarily have to be “human actors”) both approaches nevertheless correspond regarding the definition of the relationship actor-network: “No net exists independently of the very act of tracing it, and no tracing is done bei an actor exterior to the net” (378). 11 Cf. also (6; 155 and 7; 55); regarding this cf. in detail Jäger, “Über die Individualität von Rede und Verstehen.” 12 For the reasons already mentioned, the fact that according to Derrida the voice always acts as an accomplice of utter self-affection will not be considered here. 13 Cf. for this Derrida, Speech and Phenomena 88. 14 In German the terminology is the following: “intentionale Leistung der Selbstgebung.” 15 Regarding this remark cf. also de Saussures ideas in his Writings in General Linguistics. He argues against the idea that with words associated with “a material object, it might be said that the obejct’s very essence is likely to bring the word a positive meaning.” He continues: “In fact, I know of no object whose denomination is unaffected by the addition of one or more ideas said to be accessory, but which are ultimately just as important as the main idea—whether the object in question is the Sun, Water, Tree, Woman, Light, etc. This means that in fact all names are uniformely negative, signify only in relation to ideas put into other terms (also negative), never seek to qualify an innately specific object, and in fact only ever approach the ob-

86

Ludwig Jäger | Epistemology of Disruptions

ject . . . askance, through, and in the name of a particular idea of some kind” (cf. de Saussure, Writings 50). 16 Cf. Brandom, “On the side if propositionally contentful speech acts, paradigmatically assertion, the essential inferential articulation of the propositional is manifested in the fact that the core of specifically linguistic practice is the game of giving and asking for reasons. Claiming or asserting is what one must do in order to give a reason, and it is a speech act that reasons can be demanded for. Claims both serve as and stand in need of reasons or justifications. They have the contents they have in part in virtue of the role they play in a network of inferences” (Making it Explicit 161-162). 17 Regarding this cf. de Saussure’s theoretical fragment “Notes Item” in which he theoretically develops the enmeshment in a net of linguistic signs in his Cours de linguistique générale (35ff) (3306-3324 = N 15.1-19); regarding this cf. also Jäger, “Der saussuresche Begriff.” 18 The application of concepts as one of the central preconditions in referring to transitorial circumstances according to Brandom is “a linguistic affair—not in the sense that one must be talking in order to do it, but in the sense that one must be a player of the essentially linguistic game of giving and asking for reasons” in order to arrive at “conceptual contentfulness” (Making It Explicit xxi). Brandom’s point of departure here is that “propositional contentfulness must be understood in terms of practices of giving and asking for reasons. A central contention is that such practices must be understood as social practices—indeed as linguistic practices” (141). 19 As Peirce has shown, of course languages have to have means available that allow distinguishing a “real” world from an “imaginary” one: “The actual world cannot be distinguished from a world of imagination by any description. Hence the need of pronoun and indices, and the more complicated the subject the greater the need of them” (Collected Papers: Exact Logic, “Philosophy of Notation” 3.363; 212); regarding this cf. also Pape (7). However, as Brandom has shown with Sellars, for indexical sentences it is also necessary “that even such noninferential reports must be inferentially articulated” (Articulating Reasons 47); “essentially indexical beliefs have a special sort of object-involving content that other beliefs do not, that object-involvingness should not be thought of as a nonconceptual element in their content; rather, the special sort of access . . . is a special kind of conceptually articulated access” (Making it Explicit 551). Therefore, Brandom considers the intralinguistic reference (word-word reference) more “fundamental” than the word-world reference (306).

87

Ludwig Jäger | Epistemology of Disruptions

20 Of course one can refer to the world with sign-systems but not to a world that is structures without sign-systems. 21 Cf. Jäger, “Transkriptive Verhältnisse/Transcriptivity Matters.” 22 As de Saussure formulates for language, there is “no positive dissociation between the idea of the word and the idea of the idea that is in the word” (57). He insists that “neither idea nor sign, nor different signs, nor different ideas may ever represent, innately and individually, a given term. The only given consists of different signs inseparably combined with different ideas, in a high degree of complexity. These two states of chaos, when brought together, lead to a state of order” (31-32; similarly 46). 23 Hartmut Winkler has shown that the term “transfer” can mediatheoretically be adequately constructed beyond the sender-receiver-model (“Übertragen—Post, Transport, Metapher”). 24 Cf. Luhmann, Gesellschaft der Gesellschaft: “It [the term “eigenbehavior”] denotes a relative stability emerging from the recursive procedure of utilizing an operation for its own results” (213-214). 25 The term “coherence” resembles the terminology by Assmann of (ritual, textual) cultural coherence, without taking over the differentiation. Cf. Assmann (89ff; 97ff). 26 However I don’t want to take over Luhmann’s differentiation of medium and form, nor its application to language. Luhmann’s thesis that language is a “non-system” “through which alone the formation of systems in the realm of consciousness and communication is made possible” cannot be discussed here (cf. Luhmann, Wissenschaft der Gesellschaft 51). Also his contention that language is not a sign system (51-52) relies on a rather flawed knowledge of de Saussure’s theories, to whom he ascribes representational conceptions. 27 Cf. for this, e.g., Jäger, “Vom Eigensinn des Mediums Sprache.” (In German the word “Eigensinn” can have the double meaning of “obstinacy” or also “individualized meaning produced by the speaker.”) 28 For the term “Hypolepsis” cf. Assmann 282; 286ff. 29 The differentiation of these two modes of visibility approximately corresponds to the difference made by Bolter and Grusin referring to Lanham between “looking at” und “looking through” (cf. 41). 30 Regarding this cf., e.g., Luhmann, Gesellschaft der Gesellschaft (190-202). For example for media of perception he observes: “We do not see light, but things. . . . We do not hear air, but sound” (201). Cf. for this also Krämer, Medium in which she says, following Luhmann “that wherever we en-

88

Ludwig Jäger | Epistemology of Disruptions

counter media we do not perceive media, only forms” (76). Cf. also Krämer, Sprache: “Besides, it is form that is visible—the medium itself, however, remains invisible” (157). Research has interpreted this kind of invisibility (transparency) since Fritz Heider’s dictum in 1926 that a real medium is one by way of which one can see through without restraint as a constitutive characteristic of media (cf. Heider). Engell and Vogl formulate that media have the tendency to become “imperceptible as it were, unaesthetic” (10) with regard to those elements they mediate. According to the proposal made here, transparency is not a characteristic of media but a state in which signs/media are “undisrupted.” 31 Cf. Schütz; cf. for this Jäger, “Zeichen/Spuren.” 32 The confusion of semantic evidence of meaning already on the microlevel of interpersonal communiation is a catalyst for transcriptive operations. Cf. for this Jäger, Störung. 33 For the term of semantic evidence cf. Jäger, “Indexikalität und Evidenz.”

Works Cited Aigner, Carl. “Diskurse der Bilder: Photokünstlerische Appropriationen kunsthistorischer Werke.” Diskurse der Bilder: Photokünstlerische Reprisen kunsthistorischer Werke: Katalog Kunsthistorisches Museum Wien. Ed. Wilfried Seipel. Vienna: Kunsthistorisches Museum, 1993. 9-22. Amelunxen von, Hubertus. “Aneignung und Simulakrum: Die Einholung des Anderen in der Fotografie.” Diskurse der Bilder: Photokünstlerische Reprisen kunsthistorischer Werke: Katalog Kunsthistorisches Museum Wien. Ed. Wilfried Seipel. Vienna: Kunsthistorisches Museum, 1993. 23-37. Assmann, Jan. Das kulturelle Gedächtnis: Schrift, Erinnerung und politische Identität in frühen Hochkulturen. Munich Beck, 1992. Assmann, Aleida, and Jan Assmann, eds. Aufmerksamkeiten: Archäologie der literarischen Kommunikation VII. Munich: Fink, 2001. Bolter, Jay D., and Richard Grusin. Remediation: Understanding New Media. 4th ed. Cambridge, MA: MIT P, 2001. Brandom, Robert B. Making it Explicit: Reasoning, Representing, and Discursive Commitment. Harvard: Harvard UP, 1994. ———. Articulating Reasons: An Introduction to Inferentialism. Cambridge, MA: Harvard UP, 2000.

89

Ludwig Jäger | Epistemology of Disruptions

Buschmeier, Gabriele, Ulrich Konrad, and Albrecht Riethmüller, eds. Transkription und Fassung in der Musik des 20. Jahrhunderts. Stuttgart: Steiner, 2008. Cassirer, Ernst. The Philosophy of Symbolic Forms. Vol 1: Language. Trans. Ralph Manheim. New Haven: Yale UP, 1953. Danto, Arthur C. “Abbildung und Beschreibung.” Was ist ein Bild? 2nd ed. Ed. Gottfried Boehm. Munich: Fink, 1995. 125-147. Deacon, Terrence W. The Symbolic Species: The Co-Evolution of Language and the Brain. New York: Norton, 1997. Derrida, Jacques. Speech and Phenomena and Other Essays on Husserl’s Theory of Signs. Trans. David B. Allison. Evanston: Northwestern UP, 1973. ———. Of Grammatology. Trans. Gayatri C. Spivak. Baltimore: Johns Hopkins UP, 1976. ———. Writing and Difference. London: Routledge, 1978. Engell, Lorenz, and Joseph Vogl. “Vorwort.” Kursbuch Medienkultur: Die maßgeblichen Theorien von Brecht bis Baudrillard. Ed. Claus Pias et al. Stuttgart: DVA, 1999. 8-11. Fehrmann, Gisela, et al. “Originalkopie: Praktiken des Sekundären: Eine Einleitung.” Originalkopie: Praktiken des Sekundären. Ed. Gisela Fehrmann et al. Cologne: DuMont, 2004. 7-17. Fehrmann, Gisela, et al., eds. Originalkopie: Praktiken des Sekundären. Cologne: DuMont, 2004. Fehrmann, Gisela, Erika Linz, and Cornelia Epping-Jäger, eds. Spuren—Lektüren: Praktiken des Symbolischen. Munich: Fink, 2005 Foucault, Michel. The Archaeology of Knowledge & The Discouse on Language. New York: Pantheon, 1972. Franck, Georg. Ökonomie der Aufmerksamkeit. Munich: Hanser, 1998. Genette, Gérard. Palimpsests: Literature in the Second Degree. Trans. by Channa Newman, Claude Doubinsky. Lincoln: U of Nebraska P, 1997. Goodman, Nelson. Languages of Art: An Approach to a Theory of Symbols. Indianapolis: Bobbs-Merrill, 1968. ———. Ways of Worldmaking. Hassocks: Harvester, 1968. Grésillion, Almuth. “Über die allmähliche Verfertigung von Texten beim Schreiben.” Kulturelle Perspektiven auf Schrift und Schreibprozesse. Ed. Wolfgang Raible. Tübingen: Narr, 1995. 1-36.

90

Ludwig Jäger | Epistemology of Disruptions

Gülich, Elisabeth, and Thomas Kotschi. “Textherstellungsverfahren in mündlicher Kommunikation: Ein Beitrag am Beispiel des Französischen.” Ebenen der Textstruktur: Sprachliche und kommunkiative Prinzipen. Ed. Wolfgang Motsch. Tübingen: Niemeyer, 1996. 37-80. Gumbrecht, Hans Ulrich, and K. Ludwig Pfeiffer, eds. Materialities of Communication. Stanford: Stanford UP, 1994. Heider, Fritz. “Ding und Medium.” Kursbuch Medienkultur: Die maßgeblichen Theorien von Brecht bis Baudrillard. Ed. Claus Pias et al. Stuttgart: DVA 1999. 319333. Humboldt von, Wilhelm. Gesammelte Schriften. Ed. Leitzmann, Albert, Bruno Gebhardt, and Wilhelm Richter. Berlin: de Gruyter, 1936. Husserl, Edmund. Formal and Transcendental Logic. Trans. Dorion Cairns. The Hague: Nijhoff, 1978. Jäger, Ludwig. “Der saussuresche Begriff des Aposème als Grundlagenbegriff einer hermeneutischen Semiologie.” Zeichen und Verstehen: Akten des Aachener Saussure-Kolloquiums 1983. Ed. Ludwig Jäger and Christian Stetter. Aachen: Rader, 1986. 7-33. ———. “Über die Individualität von Rede und Verstehen: Aspekte einer hermeneutischen Semiologie bei Wilhelm von Humboldt.” Poetik und Hermeneutik XIII, Individualität. Ed. Manfred Frank, Munich: Fink, 1988. 76-94. ———. “Zeichen/Spuren: Skizzen zum Problem der Zeichenmedialität.” Schnittstelle: Medien und kulturelle Kommunikation. Ed. Georg Stanitzek and Wilhelm Vosskamp. Cologne: DuMont, 2001. 17-31. ———. “Transkriptivität: Zur medialen Logik der kulturellen Semantik.” Transkribieren—Medien/Lektüre. Ed. Ludwig Jäger and Georg Stanitzek. Munich: Fink, 2002. 19-41. ———. “Störung und Transparenz: Skizze zur performativen Logik des Medialen.” Performativität und Medialität. Ed. Sybille Krämer. Munich: Fink, 2004. 35-74. ———. “Vom Eigensinn des Mediums Sprache.” Brisante Semantik: Neuere Konzepte und Forschungsergebnisse einer kulturwissenschaftlichen Semantik. Ed. Dietrich Busse, Thomas Niehr, and Martin Wengeler. Tübingen: Niemeyer, 2005. 45-64. ———. “Versuch über den Ort der Schrift: Die Geburt der Schrift aus dem Geist der Rede.” Schrift: Kulturtechnik zwischen Auge, Hand und Maschine. Ed. Gernot Grube, Werner Kogge, and Sybille Krämer. Munich: Fink, 2005. 187-209.

91

Ludwig Jäger | Epistemology of Disruptions

Jäger, Ludwig. “Indexikalität und Evidenz: Skizze zum Verhältnis von referentieller und inferentieller Bezugnahme.” Deixis und Evidenz. Ed. Horst Wenzel, and Ludwig Jäger. Freiburg: Rombach, 2008. 289-315. ———. “Transkriptive Verhältnisse: Zur Logik intra- und intermedialer Bezugnahmen in ästhetischen Diskursen.” Transkription und Fassung in der Musik des 20. Jahrhunderts. Ed. Gabriele Buschmeier, Ulrich Konrad, and Albrecht Riethmüller. Stuttgart: Steiner, 2008. 103-134. [To be published in English in: Media, Culture, and Mediality. Ed. Erika Linz et al. “Transcriptivity Matters: On the Logic of Intra- and Intermedial References in Aesthetic Discourse.” Trans. Brigitte Pichon and Dorian Rudnytsky.] Koch, Peter, and Sybille Krämer. “Einleitung.” Schrift, Medien, Kognition: Über die Exteriorität des Geistes. Ed. Peter Koch, and Sybille Krämer. Tübingen: Stauffenburg, 1997. 9-26. Krämer, Sybille. “Das Medium als Spur und als Apparat.” Medien—Computer— Realität: Wirklichkeitsvorstellungen und neue Medien. Ed. Sybille Krämer. Frankfurt a.M.: Suhrkamp, 1998. 73-94. ———. Sprache, Sprechakt, Kommunikation. Frankfurt a.M.: Suhrkamp, 2001. Krämer, Sybille, Werner Kogge, and Gernot Grube, eds. Spur: Spurenlesen als Orientierungstechnik und Wissenskunst. Frankfurt a.M.: Suhrkamp, 2007. Kristeva, Julia. Sēmeiotikē: Recherches pour une sémanalyse. Paris: Seuil, 1969. Latour, Bruno. “On Actor-Network-Theory: A Few Clarifications.” Soziale Welt 47 (1996): 369-381. Leopold, Silke, ed. Musikalische Metamorphosen: Formen und Geschichte der Bearbeitung. Kassel: Bärenreiter, 1992. Luhmann, Niklas. Die Wissenschaft der Gesellschaft. Frankfurt a.M.: Suhrkamp, 1992. ———. Die Gesellschaft der Gesellschaft. Frankfurt a.M.: Suhrkamp, 1997. ———. Die Kunst der Gesellschaft. Frankfurt a.M.: Suhrkamp, 1997. Manovich, Lev. The Language of New Media. Cambridge, MA: MIT P, 2001. Mitchell, W. J. Thomas. Picture Theory. Chicago: U of Chicago P, 1995. Pape, Helmut. “Indexikalität der Erfahrung oder Objektivität des Wissens.” Zeitschrift für Semiotik 21.1 (1999): 3-14. Peirce, Charles S. Writings: A Chronolocical Edition. Vol. 2. 1867-1871. Bloomington: Indiana UP, 1984. ———. Collected Papers of Charles Sanders Peirce. 6 Vols. Vol. 3: Exact Logic (Published Papers) and Vol. 4: The Simplest Mathematics. 1933. Ed. Charles Harts-

92

Ludwig Jäger | Epistemology of Disruptions

horne and Paul Weiss. Cambridge, MA: The Belknap P of Harvard UP, 1967. Rorty, Richard. Philosophy and the Mirror of Nature. Oxford: Blackwell, 1980. Saussure de, Ferdinand. Cours de linguistique générale. Wiesbaden: Harrassowitz, 1974. ———. Writings in General Linguistics. Trans. Carol Sanders, Matthew Pires, and Peter Figueroa. New York: Oxford UP, 2006. Schütz, Alfred. The Problem of Relevance. Ed. Richard Zaner. New Haven: Yale UP, 1970. Selting, Margret. “Reparaturen und lokale Verstehensprobleme oder: Zur Binnenstruktur von Reparatursequenzen.” Linguistische Berichte 108 (1987): 128-149. Stegmeier, Werner. “Weltabkürzungskunst: Orientierung durch Zeichen.” Zeichen und Interpretation. Ed. Josef Simon. Frankfurt a.M.: Suhrkamp, 1994. 119-141. Stoichita, Victor I. The Self-Aware Image: An Insight into Early Modern Meta-Painting. Cambridge: Cambridge UP, 1997. Vansina, Jan. Oral Tradition: A Study in Historical Methodology. Harmondsworth: Penguin, 1973. Winkler, Hartmut. “Übertragen—Post, Transport, Metapher.” Rhetorik: Figuration und Performanz. Ed. Jürgen Fohrmann. Stuttgart: Metzler, 2004. 283294. Wittgenstein, Ludwig. Tractatus logico-philosophicus: Werkausgabe Band 1. Frankfurt a.M.: Suhrkamp, 1984. Zuschlag, Christoph. “Vom Kunstzitat zur Metakunst: Kunst über Kunst im 20. Jahrhundert.” Wettstreit der Künste: Malerei und Skulptur von Dürer bis Daumier. Ed. Ekkehard Mai and Kurt Wettengl. Wolfratshausen: Minerva, 2002. 171-189.

93

N. Katherine Hayles

RFID: Human Agency and Meaning in InformationIntensive Environments From the beginning, RFID technology has been entangled with politics. It pinged Western consciousness (not for the first time [Pruett]) when Leon Theremin’s listening device was discovered hidden inside a wooden replica of the Great Seal of the United States that had been presented to the U.S. Embassy in Moscow by a group of Russian schoolchildren and installed in the Embassy’s conference room (the original is on display in the National Security Agency Museum) (Glinsky). Mystified by the device, Embassy personnel discovered that it backscattered a radio frequency wave after modulating it so it contained new information—in this case, Embassy conversations. Surveillance remains one of the principal concerns raised RFID technology, now so small and cheap that it can be embedded in a wide variety of products and objects. More subtle, but no less important, are the effects of RFID in creating an animate environment with agential and communicative powers. While surveillance issues are primarily epistemological (who knows what about whom), the political stakes of an animate environment involve the changed perceptions of human subjectivity in relation to a world of objects that are no longer passive and inert. In this sense RFID is not confined only to epistemological concerns but extends to ontological issues as well. Combined with embedded sensors, mobile technologies and relational databases, RFID destabilizes traditional ideas about the relation of humans to the built world, precipitating a crisis of interpretation that represents both a threat to human autonomy and an opportunity for re-thinking the highly politicized terrain of meaning-making in information-intensive environments. RFID and associated technologies fundamentally change the rules of the game. There are many who are already at work co-opting RFID technology for military and capitalistic purposes. If our responses remain solely on the level of resisting the spread of the technology—important as that may be in certain respects—we lose the opportunity to seize the initiative and explore the technology’s potential for shedding the burden of long-held misconceptions about cognition and moving to a more processual, relational and accurate view of embodied human action in complex environments. The challenge RFID presents is how to use it to re-think human subjectivity in constructive and lifeenhancing ways without capitulating to its coercive and exploitive aspects. The context in which this challenge presents itself is one of the major developments of intelligent technologies in the 21st century: the movement of

95

N. Katherine Hayles | RFID

computation out of the box and into the environment. Whereas mid-twentieth century research in artificial intelligence focused on trying to create, in a single entity, all the complex capacities of human thought (a project doomed to failure, for reasons that Hubert Dreyfus among others has demonstrated), contemporary research in distributed cognition concentrates on creating complex interrelated systems in which small sub-cognizers that perform within a very limited range of operation are combined with readers that interpret that information, which in turn communicate with relational databases that have the power to make correlations on much wider (and extensible) scales. No one component of these systems comes anywhere close to the complexity of human thought, but combined together, the components constitute a flexible, robust, and pervasive “internet of things” (Gershenfeld; Gershenfeld et al.) that senses the environment, creates a context for that information, communicates internally among components, draws inferences from the data, and comes to conclusions that, in scope if not complexity, far exceed what an unaided human could achieve (for a discussion of “Knowledge Discovery in Databases” (KDD), cf. Fayyad et al.). In this model of distributed cognition, the emphasis shifts from the traditional triad of human/animal/machine to human/animal/thing. While the components of RFID could be considered machines, their small size, ubiquitous presence in the environment, and very limited range of sub-cognition makes them more thing-like than machine-like, a construction in line with moving from the traditional AI model of a single thinking entity to myriad small sub-cognizers. The focus on many tiny interactors (smart dust rather than the Terminator) foregrounds communication between components of a system, relational dynamics between different systemic levels, embodied interactions, and contextual awareness. The traditional Heideggerian progression of humans as world-building, animals poor in world, and stones without world (Heidegger) is brought into question; as a result, the relations between human, animal and thing come up for grabs, functioning as a chaotic nexus in which technological innovations, anxieties about surveillance and privacy, capitalistic and military exploitations, and creative storytelling swirl together in a highly unstable and rapidly changing dynamic. My interest here is exploring the implications of this dynamic by triangulating between technological practices, information-theoretic conceptualizations, and fictional representations of RFID technologies. As I and others have argued, science fiction can be a potent resource for interrogating new technologies, especially when the rapid pace of change outstrips the capacity of social theory to grapple with emerging complexities (Hayles, My Mother Was a Computer; Burrows, “Cyberpunk as Social Theory;” Burrows, “Virtual Culture, Urban Polarization and Social Science Fiction;” Featherstone and Burrows).

96

N. Katherine Hayles | RFID

RFID operates not only in the realm of such technological-managerial practices as the identification and tagging of products but also in what Nigel Thrift has called the technological unconscious, working in subtle ways to change the relation of humans to their environments. It is constituted through “the bending of bodies-with-environments for a specific set of addresses without the benefit of any cognitive inputs” (“Remembering the Technological Unconscious” 177). While epistemological concerns about surveillance and privacy can (and should be) addressed through such tactics as regulation, disclosure and informed consent (Kang and Cuff; Cuff), ontological issues concerning how human subjectivity is being reconfigured by context-aware technologies are more difficult to assess and address. Epistemological issues lend themselves to strategy and tactics (from sophisticated counter-surveillance techniques to brute force methods like smashing RFID tags with a hammer or frying them in a microwave), but how do we understand the ontological effects of animate environments? For these concerns, fictional explorations of near-future worlds not only let us imagine what such societies might look like but also engage us on other levels as well, including embodied affectivity and the unconscious. Precisely because narratives (unlike databases) always mean more than they explicitly state (Hayles, “Narrative and Database”), they can address ontological questions as well as epistemological issues. Moreover, as linguistic artifacts, narratives have a vested interest in the operations of language, including interfaces between computer-mediated symbolic code and the so-called “natural” languages native to humans (Hayles, My Mother Was a Computer). Like computational technologies, narratives operate through decoding and encoding procedures, and these similarities provide potent analogies to layered RFID communication protocols (the importance of protocols for these systems is emphasized by Alexander Galloway in his Protocol). As my tutor texts, I will focus on David Mitchell’s contemporary experimental novel Cloud Atlas (2004) and Philip K. Dick’s visionary and hallucinogenic novel Ubik (1966). Foregrounded in these works are questions about human agency and autonomy when things (seem to) come alive and clamor for attention. While Cloud Atlas emphasizes a politics of epistemology, Ubik presents complexities irresolvable in a traditional worldview and hence forces a confrontation with ontological questions. Both Cloud Atlas and Ubik are deeply concerned with the relation between capitalism and the politicization of animate environments, but whereas Cloud Atlas uses logical extrapolation from present conditions, Ubik makes inferential leaps that work through metaphor and image, as if designed to engage directly with the technological unconscious.

97

N. Katherine Hayles | RFID

In a perhaps surprising conjunction, both works evoke the divine, connected by subterranean flows to animistic environments that, functioning like RFID in some respects, nevertheless resist reductive explanations that would account for them solely through technological mechanisms. The trope of the divine, I will argue, brings to the fore the ontological dimensions of RFID; in these texts it is employed to urge the necessity for accountability and progressive action in the face of changing relationships of humans to their world. In the conclusion, I will return to the challenges that RFID poses to conventional ideas about information, proposing a model of information processing more adequate for the distributed cognition of RFID and better suited to understand human interactions with animate environments. To prepare for these arguments, let us briefly to review how RFID technology works.

RFID: Tags, Sensors, Databases, and Distributed Cognition RFID tags come in two moods, active and passive (for an introduction, cf. Glover and Bhatt; Bhuptani and Moradpour). They are comprised of a computer chip with an integrated circuit, onto which information is encoded, and an antenna for receiving and transmitting radio frequency waves. Passive chips backscatter a signal sent from a reader, after modulating it with a 10-digit identification number (Electronic Product Code, or EPC) that uniquely identifies the object to which it is attached (the 96-bit EPC can generate 296 different numbers, enough to code 80,000 trillion objects, more than sufficient to identify every man-made object on the planet). Active chips have a power source and can send as well as receive radio waves. Passive chips are as cheap as 1-3 cents and can be as small as a grain of rice (now much smaller tags, measuring considerable less than the diameter of a human hair, are being developed by Hitachi). Active chips are larger and correspondingly more expensive, from $3$10. The passive tags have a reading range of a few inches, while the active tags can transmit signals for up to a mile. Currently the primary use for the tags is to trace objects as they move in time and space through integrated systems that include readers, middleware, and backend databases, making them a flexible technology platform (Lenoir and Giannella). Many commentators believe these integrated systems will revolutionize the ways that products are manufactured, delivered, stored, and inventoried. Already two behemoths of supply and demand, Wal-Mart and the U.S. Department of Defense, are requiring their vendors to attach RFID tags to their merchandise (Gilbert; Polsonetti), a practice that virtually ensures the rapid proliferation of RFID technology throughout the U.S. and other developed countries.

98

N. Katherine Hayles | RFID

Fig. 1. Variety of passive tags. Courtesy of Nicholas Gessler and his collection of “Things-That-Think.”

Fig. 2. The circular antenna and chip inside the circle are clearly visible in this transparent passive tag. Courtesy of Nicholas Gessler and his collection of “Things-ThatThink.”

99

N. Katherine Hayles | RFID

Fig. 3. Examples of active tags containing their own power sources. Courtesy of Nicholas Gessler and his collection of “Things-That-Think.”

The insider term for RFID tags is “arphids,” an apt neologism suggesting they can be pervasively scattered throughout the environment. Coupled with sensors, arphids can record and transmit all kinds of information, from temperature to seismic activity to the presence of warm bodies. Since they are both actual physical devices and virtual presences accessed through databases, Bruce Sterling sees them as the leading edge of a SPIME world. SPIMES are virtual/actual entities whose trajectories can be tracked through space and time; as Sterling conceived the term, however, it implies more than the devices by themselves. SPIME connotes the transition from thinking of the object as the primary reality to perceiving it as data in computational environments, through which it is designed, accessed, managed, and recycled into other objects. The object is simply the hard copy output for these integrated processes. The SPIME is “a set of relationships first and always, and an object now and then” (77); it is “not about the material object, but where it came from, where it is, how long it stays there, when it goes away, and what comes next” (109). In this vision RFID participates in a larger transition to a world where human action is coordinated with complex virtual/actual environments characterized by flows and relations between many different agents, including non-human ones, tied together through distributed cognitive networks (Anne Galloway).

100

N. Katherine Hayles | RFID

Both time and space are transformed as they circulate through and between the actual and virtual domains (Anne Galloway). Space is configured by distinctions between observable domains on the one hand, and on the other blind spots that do not report data back for recording and accessing (Crang and Graham; Graham); time changes from uniform clock measurement to digital recording of the always temporary instantiations of material objects (Sterling). Transformed also is the lynchpin of capitalism, private property. Whereas private property is traditionally understood as the legal possession of tangible (and intangible) assets, in an RFID world property is defined by two interpenetrating but distinct systems: one based on possession of the material object, and the other on data about the object. A consumer may buy an object—say, a bottle of wine—without owning the database entries that record the sale, link the consumer’s credit records with that particular bottle through its unique RFID number, track its recycling into another glass bottle, and continue to follow its new instantiation through space and time (for a discussion of the consumer’s “second self” in databases, cf. Andrejevic 137). Like virtual objects sold for real money in Second Life, virtualized data about the object have market values that amount to considerable percentages of the value of the material commodities to which the data correspond (judging by the typical discounts offered by loyalty cards, somewhere between twenty and thirty percent). As Sterling observes, “My consumption patterns are worth so much that they underwrite my acts of consumption” (79).

Fig. 4. RFID devices such as these were coupled with vibration sensors and used by the U.S. military to detect traffic in the Vietnam War. Courtesy of Nicholas Gessler and his collection of “Things-That-Think.”

101

N. Katherine Hayles | RFID

Fig. 5. X-Ray images of the tags shown in Figure 4, revealing their antennae and internal structures. Courtesy of Nicholas Gessler and his collection of “Things-That-Think.”

RFID and related technologies thus enable significant cost savings for corporations and offer flexibility in designing for an open-ended future in which data management facilitates sustainability practices; they also raise serious concerns about surveillance and privacy (Bajc; Albrecht and McIntyre). Tags can be embedded in objects and read without needing a clear line of sight, including cars, clothes, purses, wallets, and shoes, all of which can be used to track people as they move through RFID-embedded terrain. RFID tags are now mandatory for passports in the U.S. and the 27 countries with which the U.S. has visa waiver agreements (Evars); in addition to encoding the passport number and the holder’s name and address, the tags will soon include biometric data such as fingerprints and retinal scans. With RFID spreading to driver licenses and credit cards, unauthorized reading leading to identity theft is also a concern (Lyman). Among the effects of RFID are changing advertising and consumption practices (Andrejevic). Coupled with mobile devices such as RFID-enabled cell phones, tags can deliver location-specific information to a passerby who may, for example, receive on a cell phone the menu of the restaurant she is strolling past (Fitzpatrick). Moreover, RFID tags, coupled with backend databases, can lead to sophisticated “behavior inferences” that predict how people will act in a variety of situations (Albrecht and McIntyre). RFID participates in a paradigm shift in which the focus shifts from present and past actions to the anticipation of future actions—yet another way in which RFID leads to different constructions of time (Beer). Military proponents see RFID as part of a comprehensive strategy of surveillance in which massive amounts of data allow

102

N. Katherine Hayles | RFID

“behavior inferences” about insurgents or terrorists who mix invisibly with urban populations (McCue; Pruett). RFID has biopolitical implications as well. RFID implants are now standard for companion and meat animals; it is estimated that 15 million animals in the U.S. alone have them (as reported by EZID Animal Verification Systems). RFID tags are used in a variety of biological applications, from tagging the semen of a prize bull so that it can be authenticated to human-edible tags used to diagnose gastric disorders (Fox). Scott Silverman of VeriChip Corporation has also proposed that RFID implants be required for immigrants; he confirmed that VeriChip is pitching implants to the military as well (McIntyre and Albrecht; Komp; Jones). RFID bracelets and anklets are already being used to track the movements of prisoners, either under house arrest or in minimum security facilities (Swedberg, “L.A. County Jail to Track Inmates”). In the Yokohama district of Tokyo, RFID bracelets are mandatory for school children, allowing them to be tracked as they go to and from school and move within the school perimeter (Swedberg, “RFID Watches Over School Kids in Japan”). Given these possibilities, it is not surprising that anti-RFID activism is spreading in Europe and gaining momentum in the U.S., as well as in other developed countries (Albrecht and McIntyre). The technology’s scary potential is perhaps most evident in the “Combat Zones that See” program launched by the U.S. military that envisions using video cameras, RFID and related technologies to monitor civilian populations (cf. DARPA for the call for proposals; Crang and Graham; Shactman; and Crandall for analyses and critique). Although the military proclaims that the technology is intended for use abroad, notably Iraq, its usefulness for tracking and surveillance makes it likely that similar applications will also be deployed against domestic populations, as was clearly the case in Scotland Yard’s monitoring of major highways in the recent search for terrorist suspects (Dana Cuff reports that by 2001 the average British citizen was photographed 300 times a day, “Immanent Domain” 47). Any car with an EZPASS tag that enables it to pass through freeway gates without stopping is already RFID responsive, making it vulnerable to surveillance for other purposes as well. Important as these concerns are, and as necessary as it is to craft well-designed legislation to curb potential abuses (Kang and Cuff), we should not over-state the danger. Passive tags can be read only within distances of a few inches, limiting their usefulness for surveillance purposes. Moreover, for every technology counter-technologies are likely to emerge; already such products as wallets lined with aluminum foil to defeat unauthorized reading of RFID-encoded cards are on the market. Other proposals are being advanced to protect

103

N. Katherine Hayles | RFID

Fig. 6. RFID tag in glass capsule intended for implantation in domestic animals (or, as Silverman suggests, in humans). Courtesy of Nicholas Gessler and his collection of “Things-That-Think.”

consumer privacy, for example by having the chip embedded in a tear-off label that can detached at the point of purchase. The technology can also be used to “bite back,” for example by having information compiled by activist groups sent to RFID-enabled cell phones, so that as the consumer strolls down the aisle she can see warnings about unsafe, environmentally unfriendly, or otherwise undesirable products whose RFID labels are detected as she passes them (Kang and Cuff). Many art projects in Europe and the U.S. perform resistance by redeploying the technology or changing its signification through interventions and subversive practices (cf. Crang and Graham for a discussion of some of these projects). My focus here will be on literary narratives and what they suggest about the possibilities for ethical action in environments made animate through embedded sensors, communicators, and actuators. As indicated earlier, the issues are both epistemological and ontological, affecting not only what we know and how we know it but also what we conceive ourselves to be. As Sterling comments in an optimistic reading of an RFID world, humans are viewed “as processes: a process of self-actualization based not on what you are but what you are becoming” (52). David Mitchell’s Cloud Atlas, especially the powerful section entitled “An Orison of Sonmi-451,” presents a darker view. In this future world, the dystopian possibilities of an RFID world are fully realized, whereas the utopian potential is a fragile seed desperately trying to grow. Let us turn now to explore this representation of this future world.

104

N. Katherine Hayles | RFID

Sonmi’s Orison: The Politics of Epistemology Extrapolated from present trends, Sonmi-451’s world represents the convergence of corporate capitalism, government, and theocracy, a fusion denoted by the neologism “corporacy” (with a transparent pun on hypocrisy). In Nea So Copros (which we gradually realize is the future instantiation of Korea), “Enrichment Statues” dictate that consumers must “spend a fixed quota of dollars each month, depending on their strata. Hoarding is an anti-corpocratic crime” (Cloud Atlas 227). The official government creed, significantly called Catechisms, includes the pronouncement that “A Soul’s value is the dollars therein” (325), where the traditional meaning of “soul” as an indwelling spirit is overlaid by the denotation it has in Nea So Copros, an implanted RFID chip that identifies a citizen with a unique ID number. A person’s Soul bestows on him certain rights such as the ability to operate automated machinery like elevators (a development anticipated by Roger Burrows and Nick Ellison when they write about using “a recast definition of social citizenship [that] points[s] to potential new categories of social inclusion and exclusion” [334]). A citizen’s Soul also registers whatever his bank balance is at the moment, enables him to purchase goods, and of course, makes him vulnerable to pervasive surveillance by Eyes, the RFID readers ubiquitously employed at checkpoints. The central insight informing the narrative is its portrayal of the contradictions that riddle the corpocracy. Many of the previous era’s developed countries have become “deadlands,” wastelands rendered uninhabitable by environmental toxins, predatory capitalism, and resource depletion. We can infer that the U.S. is one of these from a reference to the “Merican Boat-People Solution” (220); so is Britain and most of Europe. So far Nea So Copros has survived, but only because it started further behind on the (over)development curve. Perhaps the most devastating of its internal contradictions is the distinction the official ideology draws between “purebloods,” womb-born citizens, and “fabricants,” clones tailored for specific tasks in the economy and destined for execution as soon as their usefulness for work has been exhausted. They are fed chemicals designed to keep them largely without memory (“amnesiads”), devoid of curiosity, and with a strictly limited consciousness so repressed it is not capable of generating an interior monologue. Male fabricants are used up working as “militiamen” and “disastermen,” genetically engineered to be relatively toxin- and fire-resistant. Female fabricants are destined for work as domestics and laborers in the service industry; such is Sonmi-451, a server at “Papa Song’s” (which we gradually realize is the future instantiation of McDonald’s). In a pointed satire underscoring how the corpocracy works, the semi-deity delivering each day’s sermon to the fabricants is a hologram of their “Logoman” (Ronald McDonald), who preaches that loafing is “time

105

N. Katherine Hayles | RFID

theft,” and that a disobedient fabricant “denies Papa Song’s love for us and cheats His Investment” (191). The naming system for fabricants is significant. The given name (“Sonmi”) refers to a genotype of supposedly identical units, whereas the number (“451”) indicates where the fabricant comes in the manufacturing order. The play between the name (traditionally connoting an individual) and number (indicating a mass-produced object) points to how the human/thing dynamic is destabilized in an RFID world. When each object has a unique identity, objects begin to seem more like individuals, and individual people become susceptible to being constituted as objects. Officially, all the clones of a given genotype are identical, but as Sonmi later asserts (and makes good in her actions), “even same-stem fabricants cultured in the same wombtank are as singular as snowflakes” (187). The official ideology holds that fabricants lack Souls, an assertion rendered tautological by denying them Soul implants and controlling them instead with electronic collars and identity chips implanted in the neck designed to explode on contact with air, thus ensuring that any fabricant who attempts to alter her identity will not survive. In a performance that may or may not be scripted by unseen powers, Sonmi gradually begins to “ascend,” rising to full consciousness and experiencing for the first time an interior monologue. As she reports in her Testimony to the Archivist, “A voice spoke in my head. It alarmed me greatly, until I learned that no one else could hear this voice, known to purebloods as ‘sentience’ . . . my language evolved . . . my curiosity about all things grew acute . . . my sense of futility grew . . . but most of all, I was afraid” (198). After her ascension and escape from the disciplinary spaces that controlled her, Sonmi becomes part of an elaborate plot masterminded by the Union, the resistance movement trying to foment a revolution against the governing party, Unanimity. When she is captured, Unanimity displays Sonmi in a show trial as an aberration of nature and denounces her Declarations (counter-Catechisms laying the ground for ethical actions capable of resisting the evils of corporacy) as “the ugliest wickedness in the annals of deviancy” (347). To make good this claim, however, they must reveal the Declarations’ content, thus inadvertently publicizing the very reasoning that is the most potent weapon poised against them. Sonmi’s Testimony (the question-and-answer dialogue constituting the narrative we read) is given to an Archivist and electronically recorded for future generations; as such, it embodies a similar contradiction. Torn between wanting Sonmi consigned to oblivion (her execution awaits her when she finishes her Testimony) and needing to publicize her as an object lesson, Unanimity bows to pressure from genomicists to allow her Testimony to be recorded. The Archivist himself embodies another kind of contradiction. Shocked to the core by some of Sonmi’s revelations, he nevertheless insists upon an accurate

106

N. Katherine Hayles | RFID

record, for “a duplicitous archivist wouldn’t be much use to future historians” (189). Sonmi’s ascension was catalyzed in part by her exposure to Yoona 939, who opened a crack in the seamless world of mindless time and disciplinary space that was Sonmi’s lot in the corporacy, destined for a life of slavery twelve stories down in an underground Papa Song’s. The insidious tool Yoona used to pry open this world was a concept so potent, Sonmi tells the Archivist, that the corporacy would do well to fear it: a secret. Thus the text initiates a politics of epistemology. Actually Yoona reveals a series of secrets to Sonmi, starting with the revelation that Seer Rhee, the overseer who governs their servitude, is anything but a Panoptic all-seeing presence. Addicted to Soap, the food/soporific that keeps the fabricants alive and repressed, Seer Rhee pigs out on it every night, falling into a sleep so deep it resembles a coma. With the Seer seeing nothing, Yoona slips past him and discovers a storeroom, and within it a fairy tale book some child has left behind. She mistakes the book for a “broken sony,” as she mistakes its images of princesses and dwarfs, castles and elves, for an accurate picture of the world outside. Poignant and pathetic, these misapprehensions initiate a series of events that leads to Yoona’s doomed escape attempt. Without resources, knowledge, or a tactical plan, she grabs a pureblood child and runs for an elevator, only to return as a bullet-ridden corpse. The incident unleashes the nightmare specter the corpocracy had been denying and repressing all along: the possibility that the supposedly docile fabricants may ascend to full consciousness and rebel. Echoing how many Americans felt after 9/11, Sonmi pushes the Archivist’s buttons by articulating his reaction to Yoona’s rebellion: You felt the corpocratic world order had changed, irrevocably. You vowed never to trust any fabricant. You knew that Abolitionism was as dangerous and insidious a dogma as Unionism. You supported the resultant Homeland Laws dictated by the Beloved Chairman, wholeheartedly. (195) The politics of fear thus reinforces the politics of consciousness, in which the degree of consciousness a subject possesses becomes a basis for discrimination so violent that it amounts to slavery. Moreover, the politics of consciousness is based on a tautology. Fabricants “deserve” their enslavement because they have limited consciousness, and because they are enslaved, they can be forcibly fed the suppressants that deny them full consciousness. Sonmi’s ascension enables her fully to comprehend these tautologies and contradictions. Spirited away by Union operatives, she is covertly taken to witness the atrocities at Papa Song’s Golden Ark, the vessel that supposedly

107

N. Katherine Hayles | RFID

transports fabricants who have fulfilled their twelve years of servitude to “Xultation,” a life of ease and happiness in Hawaii. Actually it is a killing ship, as Sonmi tells the scandalized Archivist, bound to the economics of corpocracy. The genomics industry demands huge quantities of liquefied biomatter, for wombtanks, but most of all, for SoaWhat cheaper way to supply this protein than by recycling fabricants who have reached the end of their working lives? Additionally, leftover “reclaimed proteins” are used to produce Papa Song’s food products, eaten by consumers in the corp’s dineries all over Nea So Copros. It is a perfect food cycle. (343) This horrific revelation, which can be taken as a viciously satirical demonstration of Alliez and Feher’s claim that “individuals are enslaved by, or rather, incorporated into capital[emphasis added]” (317) is superseded by yet another secret, Sonmi’s gradual realization that her “rescue” by Union operatives is in fact part of Unanimity’s strategy to control the pureblood population as well as the enslaved fabricants. “Union preexists me,” she tells the Archivist, “but its reasons d’être are not to foment revolution” (348). Reminiscent of tactics employed by the CIA during the Vietnam (and Iraq?) war, Union activists operate as provocateurs, attracting social malcontents and keeping them “where Unanimity can watch them,” meanwhile providing the corpocracy “with the enemy required by any hierarchical state for social cohesion” (348). Against this strategy, the tactics of evasion, identity counterfeiting, running and hiding that Sonmi and her supposed protectors practiced are of limited usefulness, suggesting that the dystopian aspects of RFID technology cannot be defeated by evasive tactics alone. Something more is needed—something with the heft and potency sufficient to set the world on another track, an idea so compelling that it can contest and defeat the fused religious, political, and economic ideology of the corporacy. Sonmi, realizing this all too well, initiates what she calls “the game beyond the endgame” (359). She knowingly follows the script that has been prepared for her because it gives her an opportunity to set her Declarations, their “logic and ethics” (347), free in the world. Mitchell mostly withholds from his readers the text of Sonmi’s Declarations, leaving us to construct for ourselves their content by considering what could best counter the corpocracy’s ideology. We are given instead Sonmi’s Testimony, a dialogic narrative that vividly and compellingly challenges us to imagine what a better future might be and speculate on how we can help bring it about. As a literary text, the narrative works both on the explicit level of plot events and implicitly through structure and language. Somni’s “Orison” is nar-

108

N. Katherine Hayles | RFID

rated in two parts, placed so that it bookends a narrative set in the far future, “Sloosha’s Crossin’ an’ Ev’rythin’ After.” This narrative, strategically placed to intervene between Sonmi’s escape and her subsequent adventures, reveals that Nea So Copros and indeed almost the entire world has fallen into darkness, with civilization hanging by a thread in the gentle trading culture of the Valley people in Hawaii, the promised land that Sonmi would never reach. Although the record of Sonmi as an actual historical person has been lost to the Valley culture, she survives for them as a local deity inhabiting certain places where, their religion holds, they are able to communicate with her and receive visions. Thus the RFID technology of Sonmi’s world (and ours), embedded pervasively to create context-aware and animate environments, is transformed through a series of historical contingencies back into an animistic religion that, for the Greeks and early Western culture, was perhaps the original version of animate environments. The Valley people get from the Prescients, precarious inheritors of high technology who themselves are about to fall into oblivion, a “silv’ry egg” that they learn to call an “orison” (309). When they warm it in their hands, a “beautsome ghost-girl” appears “in the air an’ speaks in an Old-Un tongue what no un alive und’stands nor never will, nay” (309). Unable to decode Sonmi’s Testimony and helpless to make the connections that would enable them to realize this is the historical instantiation of their local deity, they nevertheless are inspired by the ghost-girl’s “hov’rin’n’shimm’rin’” (309). The message as language is lost, but the mediating technology allows the Valley people tenuous contact with the historical precedents for the ideals they hold dear—support one another, prefer trade to murder, hope for a better life. At once a prayer and a technology, the orison bequeaths to us, readers who have the context the Valley people lack, the urgent necessity for imagining the strategies that will open for us and our descendents a different kind of future. This is made very clear by the performative gesture that concludes the dark vision of “Shoosha’s Crossing’ an’ Ev’rythin’ After,” when the narrator invites us to “Sit down a beat or two./Hold Out your hands./Look” (309). When we turn the page, we encounter the second half of “An Orison of Sonmi-451,” which concludes, as we have seen, in an open-ended fashion that challenges us to arrive at a different place than the corporacy, the impetus toward which is all too apparent in the ideology of George W. Bush’s New World Order. Realizing the utopian possibilities of RFID and minimizing its dystopian features thus requires more than regulation, evasive tactics, and progressive legislation, useful as these may be in the short term. Also necessary are analyses that probe the technology’s deep entanglement with economic structures and political ideologies, and strong counter-visions that articulate a future

109

N. Katherine Hayles | RFID

worth fighting for. Without such inspirations, we are left in defensive postures that can respond to the technology’s abusive uses but are helpless to imagine how it might be directed in other, more positive ways. The better we understand how RFID changes the rules of the game, the more we can imaginatively engage with it to exploit its positive potentials. An example is Bruce Sterling’s Shaping Things, which I admire (notwithstanding its significant short-changing of the technology’s abusive potential) because it succeeds in articulating a positive agenda for designers that effectively challenges them to use data-intensive environments to actualize a better future.

Ubik: The Technological Unconscious and the Ontology of an RFID World For Joe Chip, an animate environment is nothing but trouble. Perpetually short of cash, Joe must negotiate with the coffee pot, toaster, and even the door of his “conapt” to get them to perform routine services for which they demand instant payment (Ubik 19-24). The demands of these animated devices are all the more annoying to Joe because they can talk (and reason to a limited extent). Breaking the monopoly of human “natural” language, the animate devices go beyond the context-awareness of RFID and become characters in their own right (necessarily so, because characterization is implicit in the vocabulary, syntax, and content of the utterances that Dick fashions for them— in the case of the door, quite a litigious and testy character). Although there is no explicit connection between the scenes that establish the animate environments and the novel’s main events, the scenes are much more than windowdressing for a futuristic world. They connect to a deep-seated fear that Dick inscribed into many of his most powerful works: a fear that as things became animate, people tend toward the inanimate. The fear has deep roots in Anglo-American culture, from Karl Marx’s evocation of a dancing table in his discussion of commodity fetishism to Donna Haraway’s prescient comment that “Our machines are disturbingly lively, and we ourselves frighteningly inert” (152). For Dick, the fear is overlaid with the private iconography of the “tomb world,” a fictional/literary representation of the clinical depression that plagued him throughout his life. The tomb world is depicted in several of his novels as a state of living death in which time slows to a crawl, decay is pervasive, and the poor unfortunate soul stuck there must endure eons of purgatory before he is able slowly and painfully to crawl out. At the same time, the tomb world is not merely personal, for it is consistently linked with destabilizing the dynamic between human/animal/machine (notably in Dick’s best-known novel, Do Androids Dream of

110

N. Katherine Hayles | RFID

Electric Sheep?, on which the film Blade Runner was based). In Ubik, the emphasis shifts to the dynamic between human/thing, making this text especially appropriate for an exploration of the ontological implications of RFID. At the novel’s heart is the idea of vitality: who (or what) has it, who can steal it, who is losing or gaining it. Embedded in a capitalist economy, the things that make Joe Chip’s life miserable are the tip of the iceberg; the problem that looms much larger is the overall destabilization of the boundary between human and thing, accelerated in our world by the of distributed cognitive systems such as RFID. Although the date of Ubik’s publication (1966) pre-dates the huge explosion of personal computing then in its infancy, Joe Chip’s name evokes the computational technologies with which Dick likely had at least some familiarity. The first silicon chip was invented in 1958; by 1961 it was used in Minutemen missiles and subsequently in television electronic circuits. IBM introduced its 1400 series computers in 1961, and the first chip to be used solely in computational devices, Intel’s 4004, was released in 1971. More relevant to Dick’s text is Boris Artzybasheff‘s famous illustration of an animated computer, Mark III, oozing personality on the cover of Time magazine (23 Jan. 1950), which Dick almost certainly would have known. Unlike many hard science fiction writers, however, Dick’s interest was not primarily in the technology as such but rather in its ontological and epistemological effects. When he needed a certain technology to carry out an idea, he inscribed it with hand-waving pseudo-explanations that often sounded technically impressive but had no actual scientific basis. Such is the invention of half-life in Ubik, the cryogenic suspension of people (housed in institutions appropriately called moratoriums) who have only a spark of life left but have been revived from complete oblivion and hooked up to an apparatus that allows them to communicate with the outside world. Every time a half-lifer is revived, some of his small remaining stock of life force is depleted. As a result, families who have put their loved ones in half-life face a cruel dilemma: they can access the half-lifers and so for a brief time again have contact with them, but at the cost of accelerating their final demise. During the communication sessions, the half-lifers remain frozen and inert, communicating thoughts without apparent bodily engagement. They have effectively become data, living in an illusory dream-world constrained by real-world energetics but otherwise disconnected from reality. From this context arises the novel’s central mystery. Joe Chip is employed by the Glenn Runicter, who runs a business lending out employees who have psionic capabilities: precognition, telepathy, psychokinesis and, most importantly, inertials who can offset other psionic abilities. Lured to Luna by a business rival, Runciter and his entire crew are blown up by a bomb. Joe Chip thinks that Runciter was killed in the explosion and rushes him back to earth

111

N. Katherine Hayles | RFID

before it is too late to revive the small spark of life that is half-life; gradually the suspicion grows, however, that it was actually Joe Chip and his associates who were killed, and Runciter who is trying to communicate with them while they are in half-life. Insofar as the novel engages in the politics of epistemology, questions about who knows what center on finding out who is really alive, and who is already in the deep twilight of the half-life world. More pressing are ontological questions about the nature of the strange world in which Joe Chip finds himself. Traveling with his colleagues back to Des Moines, Iowa, ostensibly to attend Runciter’s funeral, Joe begins to encounter a temporal regression of objects. First manifesting as stale cigarettes and curdled coffee cream, the regression soon takes the form of objects transforming back to previous instantiations. Push-button elevators regress to cages requiring operators; spray cans to medicinal salves; automobiles to versions manufactured twenty, thirty, and then forty years earlier; airplanes to biplanes. In a world of data where objects have only virtual existences, the temporal trajectory tracing their movements through time and space can arc backward as well as forward. Half a century before Sterling observed that in a SPIME world the emphasis falls not on the object as a material entity but rather on its always-changing instantiation, Dick intuited that in a world of data, time (and space) would be radically destabilized. Objects would cease to be stable matter and become instantiations vulnerable to time’s backward flow. Time’s destabilization affects the humans as well, although ironically in another way, moving them not toward infancy but toward the inanimate state that, in Freud’s reading of the death drive, is the ultimate origin to which the psyche secretly yearns to return. The impersonal pronoun used to describe Wendy Wright, first of the group to die from the regression, testifies to this destabilized boundary of the human/thing: “On the floor of the closet a huddled heap, dehydrated, almost mummified, lay curled up. Decaying shreds of what seemingly had once been cloth covered most of it, as if it had, by degrees, over a long period of time, retracted into what remained of its garments [emphasis added]” (97). Already frozen in the deep inertia of half-life, Joe Chip and his companions experience an irresistible force moving them toward thinghood. The remainder of the novel centers on the question of what—or who—is causing this regression. It would have been easy for Dick to explain it through the natural ebbing of the life-force to which Joe and his half-life companions are inevitably bound. Interestingly, he chose to insinuate that the cause is not simply the second law of thermodynamics but a conscious, volitional agent. At first Joe suspects Pat Conley, whose scary psionic talent is an ability to travel back into the past and create a new present by changing a single decisive event. Linked by her talent to time’s destabilization, Pat is nevertheless shown to be

112

N. Katherine Hayles | RFID

as much in thrall to the regression as her compatriots. The culprit, revealed at the novel’s climax, is the teenaged Jory, another half-life resident who eats up the life-force of others to feed his own voracious appetite. Again, however, Dick declines the obvious route of making Jory the villain; rather, the struggle between him and Joe Chip is cast as an epic battle between the thinghood of the tomb world and the forces of vitality. The nature of this battle becomes clear when Joe encounters Ella Runciter, Glen Runciter’s wife who has been consigned to half-life; her vitality almost exhausted, she is preparing to be re-born into another life cycle. When Joe tells her “Maybe I can defeat Jory,” she comments, “Maybe in time you can learn ways to nullify him. I think that’s really the best you can hope to do; I doubt if you can truly destroy him—in other words consume him—as he does to half-lifers placed near him at the moratorium” (207). When Joe objects that he can denounce Jory’s predations to Glen and have him moved, Ella explains that Jory’s relatives pay handsomely for the moratorium owner to keep him near the others. “’And—there are Jorys in every moratorium,’” she adds. “This battle goes on wherever you have half-lifers; it’s a verity, a rule, of our kind of existence” (207). The fight, in other words, cannot be waged through strategy and tactics alone; rather, it is primarily and fundamentally an ethical struggle that continues as long as life in any form exists. “You’ll have to take charge, Mr. Chip, after I’m reborn,” Ella tells him. “Do you think you can do that? It’ll be hard. Jory will be sapping your strength always, putting a burden on you that you’ll feel as—She hesitated. The approach of death” (207). The talismanic object Ella creates to aid in this struggle is Ubik, the mysterious substance for which Joe Chip desperately searches. If he can find it in a spray can before it regresses into a useless salve, he can douse himself with it and stabilize, for the moment, his own regression into thinghood. Half-life, as a phantasmatic space, can thus be understood as playing out in metaphoric fashion the complex relation between the technological unconscious and the destabilized dynamic between human/thing. Dick’s fine insight is to make Ubik profoundly ambivalent, associated both with the worst kind of hucksterism for predatory capitalism and with the divine force that its name suggests, ubiquitous and eternal. These characterizations of Ubik come in the epigraphs of the final chapters. The penultimate chapter, for example, begins with an advertisement for “Ubik toasted flakes, the adult cereal that’s more crunchy, more tasty, more ummmish. Ubik breakfast cereal, the whole-bowl taste treat!” The invitation to eat more and more is vitiated by the follow-up warning (no doubt mandated by regulation): “Do not exceed recommended portion at any one meal,” suggesting that this “taste treat” is also a poison. In sharp contrast is the epigraph of the final chapter:

113

N. Katherine Hayles | RFID

I am Ubik. Before the universe was, I am. I made the suns, I made the worlds. I created the lives and the places they inhabit. I move them here, I put them there. They go as I say, they do as I tell them. I am the word and my name is never spoken, the name which no one knows. I am called Ubik, but that is not my name. I am. I shall always be. (215) The startling contrast between these two versions of Ubik suggests that the force it represents can be appropriated for good or evil. Ella, after all, does not simply discover Ubik but invents it, enrolling it in her efforts to enhance life over death through strenuous and unremitting effort. If Ubik is eternal, as the final epigraph suggests, its declaration of ultimate agency must nevertheless be seen in terms of human interpretation and human ethical action. At stake is how the destabilization of time and space by data-intensive environments will be interpreted and employed; as time and space become more malleable, will this flexibility be used for to enhance and amplify human life, or to drive humanity closer to thinghood? Capitalism alone, Dick’s novel suggests, cannot be trusted to bring about salutary results on its own. In a concluding master-stroke, the narrative performs a final inversion that throws the struggle back to us. Joe Chip first suspects that he might be in half-life when his money displays not the conventional historical figures but the profile of Glenn Runciter. In the novel’s final moments, just when we think we have everything figured out, Glenn Runciter discovers that his money is now displaying the profile of Joe Chip, a development designed to subvert the neat closure otherwise achieved by drawing a firm line between the phantasms of half-life and the “real” world that Glen Runciter (and we) inhabit. The implication, of course, is that normal life has begun to operate by the same rules as half-life; with the boundary between them destabilized, we are left to draw the obvious conclusion. Our world is no more secure from the threat—and promise—of data-intensive environments than Dick’s fictional creation. It is up to us to face the epistemological and ontological challenges they represent and imagine how they can be used to fashion a better world. Like Joe Chip, we inherit the ethical imperative of Ubik: will ubiquitous computing be co-opted as a stalking horse for predatory capitalism or can we seize the opportunity to use it for life-enhancing transformations?

A Modest Proposal: Reconnecting Information and Meaning As William Mitchell comments, RFID and related technologies “change the fundamental mechanisms of reference—the ways in which we establish

114

N. Katherine Hayles | RFID

meaning, construct knowledge, and make sense of our surroundings by associating items of information with one another and with physical objects” (Mitchell, Me++ 120). A framework is needed capable of building bridges between human agency and an RFID world without collapsing distinctions between them. Such a framework would allow us to shed the misconception that humans alone are capable of cognition (a proposition already deconstructed with respect to animals and growing shaky with regard to distributed cognitive systems). The way forward, I want to argue, should not be to beat a retreat to traditional liberal humanism but rather to re-think the ways in which human cognition is like RFID technologies in that it is multi-layered, context-aware, and capable of generating novel meanings and interpretations. Nevertheless, human cognition remains distinct from thinghood because it arises from embodied contexts that have a biological specificity capable of generating consciousness as an emergent phenomenon, something no mechanical system can do. The politics of consciousness dramatized in “An Orison of Sonmi-451” shows that the traditional view equating the human with the ability to formulate conscious thoughts can lead to radical social inequalities. Moreover, the politics of consciousness is not confined to fictional scenarios; whenever a group of people is stigmatized and repressed, the charge is leveled that the people are deficient in consciousness (or reason, consciousness’s handmaiden). The challenge Cloud Atlas and Ubik together present is to arrive at a fuller, richer, and more adequate view of human cognition without making humans vulnerable to being reduced to thinghood. As a contribution to this challenge, I want to conclude by proposing a modification of information theory suggested when I attended a seminar given by Edward Fredkin and heard him utter the following sentence: “The meaning of information is given by the processes that interpret it” (Fredkin). Although Fredkin did not develop this idea beyond suggesting that it could, for example, be used to understand the operation of an MP3 player interpreting a digital file to produce music, I think it has great potential for contributing to an understanding of information that is contextual, processual, and embodied. Such an understanding is crucial for constructively integrating human relationships with the new kinds of situations created by RFID technologies. As we know, when Claude Shannon formulated information as a probability function, he declared that information in this technical sense had nothing to do with meaning (Shannon and Weaver). The problem he faced was how to quantify information reliably so that it was suited for calculation, a preeminent concern of electrical engineers. Traditionally meaning has been closely linked with context. However, if information is defined so that it is contextdependent and hence tied to meaning, Shannon understood that its quantification would change every time it was imported into a new context, making cal-

115

N. Katherine Hayles | RFID

culation an engineer’s nightmare. The problem Shannon faced can thus be understood as defining an appropriate context without sacrificing quantifiability. Donald MacKay, in his embodied version of information theory, boldly proposed that information should be understood in the context of the embodied receiver (MacKay), thus reconnecting information with meaning, but at the cost of failing to solve the problem of quantification. Fredkin’s formulation breaks new ground by crucially changing the meaning of “interpretation” and (tautologically) the meaning of “meaning.” Information in this view is inherently processual and contextual, with the context specified by the mechanisms of interpretation. These processes take place not only within consciousness but within sub-cognitive and non-cognitive contexts as well, both biological and mechanical. A computer, for example, gives information one kind of meaning when voltages are correlated with binary code; another kind of meaning is generated with high-level programs such as C++, much easier for humans to understand than ones and zeros; still another when C++ commands are used to generate screen displays and behaviors, which have yet more general meanings to humans. Human cognition, for its part, arises from contexts that include sensory processing, which interprets information from the environment and gives it meaning within this context; the meaning that emerges from these processes undergoes further interpretation and transformation when it reaches the central nervous system; these meanings are transformed yet again as the CNS interacts with the neo-cortex, resulting in conscious thoughts. MacKay had already envisioned a series of hierarchical and interrelated contexts that included sub-cognitive processes when he insisted that the meaning of a message “can be fully represented only in terms of the full basicsymbol complex defined by all the elementary responses evoked. These may include visceral responses and hormonal secretions and what have you” (MacKay 42). Fredkin’s formulation adds to this vision a way of understanding meaning that extends it to mechanical non-human processes. Indexed to local sub-cognitive and non-cognitive contexts, “interpretation” ceases to be solely a high-level process that occurs only in consciousness. Rather, interpretation becomes a multi-layered distributed activity in which the “aboutness” of intentionality (traditionally used by philosophers as the touchstone of cognition) consists of establishing a relation between some form of input and a transformed output through context-specific local processes. By breaking the overall context of reception into many local contexts, Fredkin’s formulation makes the processes at least partially amenable to reliable quantification. Many of these local contexts already have metrics that work: measuring voltages, processing speeds, and bits/second in computers; in humans, measuring neural responses, fatigue rates and the like. The important point is a shift of vision that

116

N. Katherine Hayles | RFID

enables us to see these sub-cognitive and non-cognitive processes as not just as contributing to conscious thought but themselves acts of interpretation and meaning. This vision of meaning-creation is especially well suited to understanding human cognition in the context of RFID technologies, in which context and relationality play central roles. Context awareness is achieved when RFID tags are connected with embedded sensors and location-specific technologies such as GPS-enabled cell phones; relationality is achieved through the communications of the tags among themselves and, more widely, also through the relationality central to the operation of relational databases. When human cognition is identified solely with consciousness, it seems to operate in a qualitatively different way than these technologies. If we understand high-level consciousness to be emergent from lower-level distributed cognitive processes, as Daniel Dennett argues (Darwin’s Dangerous Idea 401-427; Kinds of Minds), we have a way to connect human cognitive and sub-cognitive processes to distributed mechanical cognition in a number of ways. Everything does not have to go through the needle’s eye of conscious awareness. This means that RFID can interface with human cognition well below the threshold of consciousness through embodied actions such as gesture, posture, and the habituated motions that give rise to and embody unconscious presuppositions, a proposition Nigel Thrift has explored in positing the technological unconscious (“Remembering;” Knowing Capitalism). Moreover, these interactions can now properly be said to be meaningful, in the precise sense of that term: full of meanings generated by context-specific processes of interpretation that occur both within and between human and non-human cognizers. Consciousness in this view loses its prerogative to be the sole arbiter of meaning; but this loss (if it is such) is more than offset by a richer contextual and processual view of how meanings are generated. RFID technologies in this view cease to be alien to the human condition and instead become part of the distributed cognitive systems that have, for millennia, extended and amplified human cognition (as demonstrated by Clark and Hutchins). Of course, this conceptual configuration of RFID technologies does not alone guarantee that they will be used to enhance human lives rather than diminish, coerce, and endanger them. That is why the ethical imperatives that emerge from the epistemological and ontological explorations of texts like Cloud Atlas and Ubik are so important. The idea that meaning and interpretation can occur across and between human and mechanical phyla contributes to an expanded sense of ethics necessary when the contexts for human actions are defined by information-intensive environments and include relational and context-aware technologies such as RFID. Such an ethics would emphasize context over generalization, processes over static objects, embodied and dis-

117

N. Katherine Hayles | RFID

tributed systems over hierarchical abstract ones, and a full range of cognitions over a sole focus on consciousness. Not coincidentally, this ethics has much in common with the Deleuzian ethics that has catalyzed contemporary work in the social sciences and humanities. When we understand that humans are not the only cognizers who can interpret information and create meaning, we are free to imagine how a world rich in embodied contextual processes might be fashioned to enhance the distributed cognitive systems that surround us and that we ourselves are. The final, definitive version of this paper has been published in Theory, Culture and Society 26 (2/3): 47-72 by Sage Publications Ltd. All rights reserved. © Theory, Culture and Society Ltd., 2009. It is available at .

Works Cited Albrecht, Katherine, and Liz McIntyre. Spychips: How Major Corporations and Government Plan to Track Your Every Purchase and Watch Your Every Move. New York: Penguin, 2005. Alliez, E., and Feher, M. “The Luster of Capital.” Zone 1-2 (1987): 314-359. Andrejevic, Mark. “Monitored Mobility in the Era of Mass Customization.” Space & Culture 6.2 (2003):132-150. Bajc, Vida. “Introduction: Debating Surveillance in the Age of Security.” American Behavioral Scientist 50.12 (2007): 1567-1591. Bhuptani, Manesh, and Sharam Moradpour. RFID Field Guide: Deploying Radio Frequency Identification Systems. Upper Saddle River: Prentice, 2005. Burrows, Roger. “Cyberpunk as Social Theory: William Gibson and the Sociological Imagination.” Imagining Cities: Signs and Memories. Ed. W. Westwood and J. Williams. London: Routledge, 1997. 235-248. ———. “Virtual Culture, Urban Polarization and Social Science Fiction.” The Governance of Cyberspace. Ed. B. Loader. London: Routledge, 1997. 38-45. Burrows, Roger, and Nick Ellison. “Sorting Places Out? Toward a Social Politics of Neighborhood Information.” Information, Communication and Society 7.3 (2004): 321-326. Clark, Andy. Natural-born Cyborgs: Minds, Technologies, and the Future of Human Intelligence. London: Oxford UP, 2000.

118

N. Katherine Hayles | RFID

Crang, Mike, and Stephen Graham. “Sentient Cities: Ambient Intelligence and the Politics of Urban Space.” Information, Communicaton and Society 10.6 (2007): 789-817. Crandall, Jordon. “Anything that Moves: Armed Vision.” CTheory 28 Oct. 2009 . Cuff, Dana. “Immanent Domain: Pervasive Computing and the Public Realm.” Journal of Architectural Education 57.1 (2003):43-49. Dick, Philip K. Ubik. 1966. New York: Vintage, 1991. DARPA. “Combat Zones that See (CTS).” Board Agency Announcement 0315, 2003. Dennett, Daniel C. Darwin’s Dangerous Idea: Evolution and the Meanings of Life. New York: Simon and Schuster, 1995. ———. Kinds of Minds: Toward an Understanding of Consciousness. New York: Basic, 1996. Dreyfus, Hubert L. What Computers Still Can’t Do: A Critique of Artificial Reason. Cambridge, MA: MIT P, 1992. Evers, Joris. “RFID Passports Take Off,” C/NetNews.com (2006). 28 Oct. 2009 . EZID. Animal Verification Systems. 28 Oct. 2009 . Fayyad, Usasm, Gregory Piatetsky-Shapiro, and Padhraic Smyth. “Knowledge Discovery in Databases.” Proceedings of the 2nd International Conference on Knowledge Discovery and Data Mining. Ed. Evangelos Simoudis, Jiawei Han, and Usuma Fayyad. Menlo Park: American Association for Artificial Intelligence, 2006. 37-54. Featherstone, Michael, and Roger Burrows, eds. Cyberspace, Cyberbodies, Cyberpunk: Cultures of Technological Embodiment. London: Sage, 1995. Fitzpatrick, Michael. “Tagging Tokyo’s Streets with No Name.” The Guardian (10 May, 2007). 28 Oct. 2009 . Fox, Barry. “Edible RFIDs.” New Scientist (12 Feb., 2007). 28 Oct. 2009 .

119

N. Katherine Hayles | RFID

Fredkin, Edward. “Informatics and Information Processing vs. Mathematics and Physics.” Presentation at the Institute for Creative Technologies, Marina Del Ray, CA. 25 May, 2007. Galloway, Anne. “Intimations of Everyday Life.” Cultural Studies 18.2 (2004): 384-404. Galloway, Alexander R. Protocol: How Control Exists After Decentralization. Cambridge, MA: MIT P, 2004. Gershenfeld, Neil, Raffi Krikorian, and Danny Cohen. “The Internet of Things.” Scientific American. (2004). 28 Oct. 2009 . Gershenfeld, Neil. When Things Start to Think. New York: Holt, 1999. Gilbert, Alorie. “Wal-Mart Tagging Fuels RFID Market.” News.com (22 Dec., 2004). 28 Oct. 2009 . Glinsky, Albert. Theremin: Ether Music and Espionage. Urbana-Champaign: U of Illinois P, 2000. Glover, Gill, and Himanshu Bhatt. RFID Essentials. New York: O’Reilly, 2006. Graham, Stephen. “Software-sorted Geographies.” Progress in Human Geography 29.5 (2005): 562-580. Haraway, Donna. “A Cyborg Manifesto: Science, Technology, and SocialistFeminism in the Late Twentieth Century.” Simians, Cyborgs and Women: The Reinvention of Nature. New York; Routledge, 1991. 149-181. Hayles, N. Katherine. My Mother Was a Computer: Digital Subjects and Literary Texts. Chicago: U of Chicago P, 2005. ———. “Narrative and Database: Natural Symbionts.” PMLA: Publications of the Modern Language Association 122:5 (2007): 1603-1608. Heidegger, Martin. The Fundamental Concepts of Metaphysics: World, Finitude, Solitude. Trans. William McNeill and Nicholas Walker. Bloomington: Indiana UP, 1995. Hutchins, Edwin. Cognition in the Wild. Cambridge, MA: MIT P, 1996. Jones, K. C. “VeriChip Wants to Test Human Implantable RFID on Military.” Techweb (23 Aug., 2006). 28 Oct. 2009 .

120

N. Katherine Hayles | RFID

Kang, Jerry, and Dana Cuff. “Pervasive Computing: Embedding the Public Sphere.” Washington and Lee Law Review 62 (2005): 93-147. 28 Oct. 2009 . Komp, Catherine. “Electronic Tags Used to Track Immigrants.” The New Standard (5 Sept., 2005). 28 Oct. 2009 Lenoir, Tim, and Eric Giannella.“Technological Platforms and the Layers of Patent Data.” National Science Foundation Grant No. SES 0531184, 2007. Lyman, Jay. “Hacker Cracks, Clones RFID Passport.” TechNewsWorld (7 Aug., 2006). 28 Oct. 2009 . MacKay, Donald. Information, Mechanism, Meaning. Cambridge, MA: MIT P, 1969. McCue, Colleen. “Data Mining and Predictive Analytics: Battlespace Awareness for the War on Terror.” Defense Intelligence Journal 13.1-2 (2005): 47-63. McIntrye, Liz, and Katherine. “Company Pushes FRID Implants for Immigrants, Guest Workers.” The Sierra Times (24 May, 2007). 28 Oct. 2009 . Mitchell, David. Cloud Atlas. New York: Random, 2004. Mitchell, W. J. Me++: The Cyborg Self and the Networked City. Cambridge, MA: MIT P, 2003. Polsonetti, Chantal. “US Department of Defense Issues Updated RFID Policy.” ARC Wire (11 Aug., 2004). 28 Oct. 2009 . Pruett, Richard. “Identification—Friend or Foe? The Strategic Uses and Future Implications of the Revolutionary New ID Technologies.” USAWE Strategy Research Project: US Army War College, Carlisle Barracks PA, 2006. 28 Oct. 2009 . Shactman, Noan. “Big Brother Gets a Brain.” Village Voice (9-15 July, 2003). 28 Oct. 2009 . Shannon, Claude, and Warren Weaver. The Mathematical Theory of Communication. Urbana: U of Illinois P, 1949. Sterling, Bruce. Shaping Things. Cambridge, MA: MIT P, 2005.

121

N. Katherine Hayles | RFID

Swedberg, Claire. “L.A. County Jail to Track Inmates.” RFID Journal (16 May, 2005). 28 Oct. 2009 . ———. “RFID Watches Over School Kids in Japan.” RFID Journal (16 Dec., 2005). 28 Oct. 2009 . Thrift, Nigel. Knowing Capitalism. London: Sage, 2005. ———. “Remembering the Technological Unconscious by Foregrounding the Knowledges of Position.” Environment and Planning D: Society and Space 22 (2004):175-90. Williams, Martyn. “Project to Tag Tokyo Neighborhood with RFID.” Computerworld (16 May, 2007). 28 Oct. 2009 .

122

Maria Angel and Anna Gibbs

Memory and Motion The Body in Electronic Writing “For images can never be anything but things, and thought is a movement” Henri Bergson, Matter and Memory “The world can only be grasped by action, not by contemplation. The hand is the cutting edge of the mind”. Jacob Bronowski, The Ascent of Man, 1974.

This paper explores the “new materialism” of online writing environments, and electronic writing—we define materiality in terms of Henri Bergson’s conception of materiality as inextricably caught between images and things, between scales and orders of experience. Referring to Jay David Bolter’s comment that the largest aircraft carriers are still infinitely closer to human scale than the simplest micro-computer, Anthony Dunne writes that “the electronic object is a confusion of conceptual models, symbolic logic, algorithms, software, electrons and matter. The gaps between the scale of electrons and objects is most difficult to grasp” (emphasis added) (7). As Serge Bouchardon, and Barbara Bolt have both noted in different contexts, to grasp or to handle a concept, an object or text, is to introduce a human dimension through the figure of the hand and its capacity for manipulation and gesture. In this paper we attempt to handle or grasp some ways of conceiving the new scales and economies of online experience and electronic writing in terms of what Brian Rotman has termed, to cite the title of his paper, “Going Parallel”. For Rotman, distributed computing is not only changing the way that we communicate but the way that we act and think. Based on the explosion of the digital image as a means of information storage, Rotman argues that the serial individual is being replaced by “the emergence of a larger—collectivized, distributed, pluralized—‘intelligence’” (74). We argue that this has implications for the way that we understand memory, both cultural and subjective. Central to Rotman’s thinking is the apparent transition from alphabetic modes of writing and communication to those that are based on emergent technologies of the image, motion capture and gesture. While this transition may not mean the effacement of writing and alphabetic modes, but rather their remediation in new contexts, we propose (like Rotman) that electronic environments have a strong

123

Maria Angel and Anna Gibbs | Memory and Motion

relationship to affective modes of communication highlighted by their appeal to sensory novelty through technological innovation—new media platforms proliferate the potentials for combining visibility with aural and tactile modes. As we argue in an earlier paper, a common factor in forms of electronic writing is its hybridization with forms of imaging (Angel and Gibbs). This “becoming-image” of writing appears to have at least two vectors. On the one hand, it intensifies what Walter Ong noted as a tendency most clearly marked in typographic and electronic cultures “to reduce all sensation, and indeed all human experience to visual analogues” (75). On the other hand, the increasing pictographic quality of electronic writing treats the alphabetic word as a thing to be explored, as a kind of material architecture, rather than as a representation of speech. John Cayley has written on Jeffrey Shaw’s early work, The Legible City as exemplary of a movement towards what he calls “Concrete Poetics” by which he means “a poetics in which a written language—letters and words—take on characteristics of other kinds of objects in order to generate rhetorical and aesthetic affect and significance” (4). This treatment of words as primarily images to be explored or that “exhibit behaviour” (5) while it has precursors in Futurism, Lettrism, concrete poetry, and artist books, profoundly rewrites the written function of language, its connection to and transformation of forms of knowledge that Ong associates with literacy and particularly typographic cultures. For Ong, the stability and linearity of alphabetic writing (the ability to “backscan” writing [Goody qtd. in Ong 98]), its semantic and spatial relationship with speech (the capacity of writing to extend speech beyond the body of the writer and to “fictionalise” the reader), creates a “psychodynamics” of interiority (152; 170). Reading, as part of this dynamic, produces the effect of an interior or “inner voice”, and knowledge is predicated on the process of abstracted and reproducible speech. Knowledge becomes characterized as the product of reflection and ideation and is conceived as absorption and contemplation rather than as the product of one’s active relationship and interaction with an exterior environment. For Ong, and others like David Abram, all of whom were writing in the unacknowledged wake of Marcel Jousse, in primarily oral cultures knowledge and memory are reproduced through mnemonic formulas that exteriorize speech in aural patterns (such as complexes of rhyme and rhythm, and the standardization of narrative and figure) or else in relation to landscape (such as the function of Australian aboriginal songlines). For Jousse especially, memory is less a purely cognitive capacity than the result of the body’s organization of expression in relation to its action in and active proprioceptive apprehension of its relation to the world (the up/down, left/right, forwards/backwards, which ultimately produces parallelism in oral style, for example). Drawing on Bergson’s idea of the body as a transducer, which releases imprisoned energy

124

Maria Angel and Anna Gibbs | Memory and Motion

by its movements, Jousse theorizes gesture as the body’s “direct resonance” with the world, or an “objective registration” (15) of it. For him, gesture comprehends all corporeal activity, including that associated with affect (laughter, tears), and even states of mind, while what we ordinarily call “intellectual” is simply an abstraction from gesture (15), that is, it can become dissociated from the real. Writing increases the risks of this, as language itself is an abstract form of gesture. Gesture in this sense is independent of consciousness, but the body’s movements are of a piece (18), all related to each other by the energetic rhythm of alternation between rest and activity. Rhythm, in turn, is intimately related to memory, which is therefore gestural, and inherently somatic, a matter of material relation rather than abstract idea or representation. One of the differences to emerge between oral, typographic and electronic cultures is the distribution of what Merlin Donald calls external memory systems. For Donald, the developments of the human mind and of material culture have co-evolved in a relationship where human knowledge is reliant on forms of externalised memory. Cultural artefacts, (and for Donald these include the built environment, monuments, objects and machinic technologies, as well as symbolic media, custom, and ritual), are complex storage systems for knowledge which “serve as shared cognitive maps” shaping behaviour, and what and how we know. He writes: Material culture externalizes memory and greatly amplifies the permanence, and power of distributed cognition. In advanced societies external symbolic storage entails highly complex storage media that require extensive training of the young. Such training can actually change the operational architecture of cognition in the individual by influencing the developing brain. The continuing interplay between material culture and cognition creates new cognitive opportunities, changing how members of a society represent reality, both individually and collectively. (“Material Culture and Cognition” 181). Here Donald provides an elaboration of the idea propounded by Ong that “artificiality is natural to human beings”, which is useful in deconstructing the opposition between what Vicki Kirby has called the “purely natural” and “purely cultural” that doggedly pursues epistemologies of technics. Perhaps, to use N. Katherine Hayles’ term, we have always been “posthuman.” Strangely—because Donald is keenly sensitive to issues of the nature-culture divide, and the sophistication of so called “primitive” cultures—he appears to privilege, to a certain extent, the complexity of the storage media of what he terms “highly advanced societies”. We would argue however that complexity is a differential value rather than a hierarchical one. The songlines of aboriginal cultures in Australia have a highly structured and complex mnemonics developed

125

Maria Angel and Anna Gibbs | Memory and Motion

in parallel conjunction with landscape and in serial transmission over generations. Knowledge of these songlines, which incidentally involves the elaborate and highly skilled initiation of the young, enables movement (often hundreds of kilometres) through, and existence in, some of the harshest physical environments on the planet—no Western individual could survive these conditions without a battery of technological life support systems (bottled water, packaged food, maps, compass, appropriate clothing and 30+ sunscreen). The sophistication of this kind of material culture, where memory is externalized in landscape and operant in the extensive parallel relationship between story and place is at least as dynamic as any interactive networked event. The performed oral forms and stories of the songlines render the landscape, or rather perform it, the landscape in turn informing and instituting speech. As Abram writes, places are not “passive settings” but animate environments, and he gives a vivid example of this indigenous technology at work. Writing of the indigenous songlines associated with the “dreamtime” (Jukurrpa, or Alcheringa), he argues that “song is thus a kind of auditory route map through the country; in order to make her way through the land, the Aboriginal person has only to chant the local stanzas of the appropriate Dreaming, the appropriate Ancestors song” (166). As Abram argues, “the land and the language . . . are inseparable,” they are “reciprocally mnemonic” (177). The place name affects a “direct sensorial bond between . . . persons and particular places” (155). We argue that some electronic locative writing works with aspects of this kind of oral mnemonics—in fact it would not be a stretch of reason to see oral songlines as forms of GPS technology. Mark B. N. Hansen writes of this latter technology as “nothing less than a mediation of consciousness and matter that takes form as a technical extension of movement, that modality of life that lies between and conjoins—that composes—space and time” (1211). The Algo Mantra Blog in their locative work Cellphabet work with an interesting variation of walking through country as a form of composition. As we have described elsewhere (“On Moving and Being Moved”), the blog uses the locative technology of cell (mobile) phones to convert the activity of walking into what one of the blog members, Rohit Gupta calls “linguistic form.” Although, as Gupta purports, it takes four hours to walk three words, by hacking into the code systems of towers around Mumbai and walking around the city the blog members convert signals into sent text without touching the phone pad. Gupta cites James Joyce’s Leopold Bloom walking the streets of Dublin in the figure of a question mark as inspiration, and in terms of accounting for the work’s usevalue, he argues that it has potential application in countries with regimes that install repressive censorships. In this instance perhaps Cellphabet describes the possibility for a kind of ephemeral performative graffiti. In a similar manner, the African-american practice of “gangwalking” which spells out tags on Los

126

Maria Angel and Anna Gibbs | Memory and Motion

Angeles’ sidewalks through the motion and emotion of dance makes this fusion of writing and the body manifest in effusive practice (Phillips). Walking “is a mode of making the world as well as being in it,” writes Rebecca Solnit: it enables us to know “the world through the body and the body through the world” (29). Central to the poetics of Rousseau and Wordsworth, who theorized the practice of solitary rambling as a contemplative mode of being and thinking, and to the Lettrists (later the Situationists) who developed it in the form of the dérive as an aesthetic practice (a kind of poetry without traces), it is perhaps not surprising that walking should return in the poetics of digital writing. For, as Solnit clearly sees when she distinguishes between walking as movement and the motion of travel: “It is the movement as well as the sights going by that seems to make things happen in the mind” (6). Once again, the body marks the site and sign of the intimate relationship, rather than the absolute divide, between digital and print media. As Merleau-Ponty long ago realized, writers don’t just deliver messages: they produce gestures (60), for even forms of sedentary writing involve incipient gesture. As Deleuze and Parnet argue, writing is made less of ideas than of “motor agitation and inertia” (75). From this perspective, the writing body rehearses and recalls, in abstract attenuated form, active relations with the world. Writing of the evolution of human cognition, Merlin Donald, like Jousse, has theorized this mimetic capacity as one that involves using the body as a complex storage and retrieval device for memory. Arguing that “a mimetic act is a manifestation of a highly abstract modelling process,” Donald writes that: Mimesis is based in a memory system that can rehearse and refine movement voluntarily and systematically, guided by a perceptual model of the body in its surrounding environment, and store and retrieve products of that rehearsal. . . . Purposive rehearsal reveals the presence of a unified self-modelling process, and most importantly, the whole body becomes a potential source of conscious representation. Retrievable body-memories were thus the first true representations, and also the most basic form of reflection, since the mimetic motor act itself represents something: systematic rehearsal refers to the rehearsed act itself, comparing each exemplar with a sort of idealised version of itself. (“Precis of Origins of the Modern Mind” 6) For Donald, this mimetic system is “supramodal” in that the human body can employ various senses or perceptual systems for modelling input and output (a visual event can be translated into speech, or a tactile sensation rendered by an image). In their description of writing as a form of gesture, Deleuze and Parnet take this capacity for mimicry further in a manner that recalls the “totemism” of indigenous oral cultures in its reference to “animism”. Deleuze and Parnet

127

Maria Angel and Anna Gibbs | Memory and Motion

describe writing as a “strange ecology” (75) that either “conforms to a set of dominant utterances, to a territory of established states of things”, or else it is a “becoming something other than a writer, since what one is becoming at the same time becomes something other than writing” (74). For them, writing is gestural and always relational with other bodies: one writes “as a rat traces a line or as it twists its tail, as a bird sends out a sound, as a cat moves or else sleeps heavily” (75). Similarly, reading involves the reception of gesture via the human mimetic capacity, so that reading activates the body in response to the gesture of writing. In reading, these figures of writing (for example, Deleuze and Parnet’s scurrying rat, its twisting tail, birdsong, the moving or sleeping cat) become diagrams, they become “figures of contemplation and rumination” which “literally embody thought” in immobilized gestures (Chatelet qtd. in Rotman, “Gesture in the Head” 18). Handwriting retains the indexical trace of gesture (hence the importance we accord to the signature), and typography and the production of graphical logos both represent an attempt to reproduce the gesture inherent in writing and in visual signs. To a certain extent then, we see in forms of electronic writing a return to relations exteriority that have characterized aspects of older oral cultures, and that were transformed by the advent of literacy. Both Ong and Abram argue that with the development of alphabetic writing, and typography in particular, the multisensory, synaesthetic relation with the more than human world shifts to the written word and the privilege of visual analogues. Attention is looped inward to produce that sense of interior “voice” and sensation that we associate with acts of reading. Brian Massumi describes reading as an act of “incipient perception”: Through the letters, we directly experience fleeting visionlike sensations, inklings of sound, faint brushes of movement. The turning in on itself of the body, its self-referential short-circuiting of outwardprojected activity, gives free rein to these incipient perceptions. In the experience of reading, conscious thought, sensation, and all the modalities of perception fold into and out of each other. Attention most twisted. (139) What Massumi points to here is the creation of interiority, the manner in which writing inflects the outside world, produces it as virtual and incipient interior experience. On the other hand, it seems to us that modes of electronic writing address, foreground, and externalize the “animism” associated with systems of symbolic perception that remain interiorized and muted (or, on in Jousse’s terms, are “dissociated”) in typographic reading and writing. As we mentioned earlier, the animism of electronic form can be seen in the treatment

128

Maria Angel and Anna Gibbs | Memory and Motion

of words as things to explore, things that move and that make or respond to sound. In fact, Ong’s proposition, made back in 1982, that electronic media usher in an age of secondary orality is highly significant in this respect (3). We see John Cayley’s work as seminal in the treatment and exploration of some of these issues. At the level of analogic reception, if not electronic programmatic code, his work windsound seems to us to work with a form of animism that performs the dynamics and reciprocity, or différance (to use Derrida’s term), between phonics and alphabetic writing so familiar to typographic literacy. We read windsound as a text about code, one that performs code at the level of micro-process but also at the level of text, at the level of classical literacy where the word is bound up with alphabetic/phonetic writing. windsound restores to “natural (literate) language” its operation as code that involves a displacement of the spoken word as an oral event phenomenologically experienced co-extensively with an external environment (as with oral cultures) to the word experienced as a cluster of letters (or writing) becoming sound. As you play the work it opens to a black screen with the sound of wind, backgrounded by the murmur of voices. A sprinkling of vowels appear onscreen, the letters “e”, added to by other vowels and a few consonants. These letters begin to morph and cluster in seeming random fashion, taking the shape if not sound of words. Eventually, the morphing produces readable and speakable words, a computer generated voice appears to read these words which begin to disintegrate into meaningless clusters of letters and sounds (phonemes). This disorder morphs back into readable and speakable texts. The three voices that play the piece each start as language but then dissolve into nonsense as the text morphs on screen. Once the text becomes readable again, it becomes language. windsound plays out the concept of language as the relationship between writing and speech. Speech is a windsound. Vowels are sounded breath with the set of consonants providing stops or modulations of windsounds that become coded, decoded, and recoded. In Deleuze and Guattari’s terms, writing as an abstract machine deterritorializes and reterritorializes speech. We argue that with electronic writing the word returns somewhat to performed (social) event (but also to the magic and ritual associated with animism, rather than the inanimate typographic image associated with forms of privatized (book) reading (and also distinct from the forms and practices of television, freeway driving and shopping in the mall that Margaret Morse characterizes as practices of distraction rather than practices of absorption and interiority, of which reading provides the template). We could take the practice of texting (SMS) as an example here. Much like old Hebrew, texting works through an economy, which omits (mainly) vowel sounds. We add them as we read to produce the word. According to Abram, the twenty-two letters of the Hebrew aleph-beth were consonants (historically, the Greeks added vowels to

129

Maria Angel and Anna Gibbs | Memory and Motion

the alphabet and to writing). “Thus,” he argues “in order to read a text written in traditional Hebrew, one had to infer the appropriate vowel sounds from the consonantal context” (241). In this sense it is the consonants that code sounded breath to produce speech, and Abram argues that with Old Hebrew the reader had a choice to make in terms of vowel sound, the difference say between “cat” and “cot.” This difference is not merely at the level of abstract sound (as Saussurian linguistics posits) but is driven by contextual choice and sensible/sensate/perceptual participation. When I text you, I expect you to paint the “cot” and not the “cat.” Cayley has written extensively about the concept of code and code work in programmable literature and media, famously proposing “that the code is not the text unless it is the text.” He makes the compelling case that we need to pay attention to the distinction between code and language. It seems to us that we can use this distinction to denature natural language at work in alphabetic text. As both Ong and Abram make clear, alphabetic typography is a programmable medium that involves the co-evolution or involution of technology and human subject. One of the interesting and pressing questions for us is to ask “how will the new becoming-image of writing in electronic contexts recode what it means to be human? What are the new protocols or algorithms?” Cayley argues that to reduce the program (code) to natural language, the signifier on the screen, is a simplification of a set of issue to do with “the emergence of new or less familiar rhetorical strategies” that would construct a different set of object-rhetorics” (“The Code is not the Text” 2). Writing of the electronic “text,” he asks “how such work is to be grasped as an object: is it text, process, performance, instrument? If code is treated distinctly, as an aspect of writing with its own structures and effects, then we gain in the potential to articulate more appropriate classes of literal objects, with instrument, for example, forming one class . . . [to] explore” (2). For Cayley the “surface” text is an “interface” between the underlying code and the user: this makes for a complexity of address which complicates relations between human and technological orders at the same times as it fractures temporalities and generates new tropes (e.g., “playability”). He writes that mediation “can no longer be characterized as subsidiary or peripheral; it becomes text rather than paratext” (2). In other words, it is this productive but not completely determinate relationship between code and text that constitutes the artwork as an object and which must be read in relation to specific instances rather than cited as a general truth. This calls, as Cayley argues, for a more formalist criticism—but in our view it has to be one which addresses the recursive nature of experience as object, rather than text as object, if it is to be adequate to work like JODI’s described by Cayley as “never the same on successive visits.” This implies the need for reception studies in addition to formalist criticism. But this feature of

130

Maria Angel and Anna Gibbs | Memory and Motion

JODI’s work also opens questions about the implications of the relative instability of artworks as memory storage systems, given that rereading will always produce a “plural” text, as Barthes long ago pointed out (16). Where this becomes complicated is on the level of reception (a cultural act) rather than that of reading (an interiorized private act). This is where the figures of mutation (Hayles; Cayley) and loss of (cultural) grasp (Bouchardon) become relevant. What we are witnessing with electronic writing is a movement towards the exteriorisation of writing through its relationship to gesture, a form of body motion capture and entrainment rather than speech capture. If electronic text is increasingly being treated as a visual landscape to be explored rather than a mnemonics for the reproduction of inner speech, then this landscape is subject to the manipulation of the reader/user. Serge Bouchardon writes of electronic work as “moveable, actable or explorable text” (5). He argues that the materiality of electronic text “cannot be dissociated from the action of the reader. The text on the screen is not only another materialisation of a meaningful form. It is the gesture of the reader which reveals the materiality of the text. One can wonder if the nature of a digital text is not to be manipulable more than to be readable” (5). The significance of this lies less in outdated debates about interactivity and its limits in artworks, or in discourses of the power, control or freedom of the reader, than in the way electronic works solicit the body’s involvement with reading via gesture, and in the nature of the experience this generates. Bouchardon’s work suggests that this involves not only the reader’s active capacities, but also their failure: for him, the experience of manipulation is characterized not only by grasp, but also, and crucially, by its loss. (Here he draws on the way in which Bessy and Chateauraynaud use the concept of “grasp” to describe the embedding, in both persons grasping and things grasped, of moments when the body is engaged in noncognitive experience.) As Ong writes, then, “technologies of the word do not merely store what we know, they style what we know” (152). In describing the differences between oral and literate cultures, Ong develops an earlier proposition of Marshall McLuhan’s: that the naturalization of media alters the ratio among our senses and changes mental processes (24). We would argue that the naturalization of media alter not just the ratio of senses and mental processes but that fundamental to these changes is the manner in which media mediate and alter physical activity, what we do with our bodies in acts of mediation. Brian Rotman writes about these changes in terms of an economy of parallel distribution, the becoming-image of writing, and the capture and reflection of gesture in motion capture technologies the mouse has been the historical basis of this. As Paul A. Harris argues, following Rotman, the technologics of our environment end up as pathways through which we “install bodily regimes and recon-

131

Maria Angel and Anna Gibbs | Memory and Motion

figure . . . mappings between our neuro-physiology—the inside of our heads and the technological milieu in which those heads operate” (3). The networked body art of Stelarc suggest that online experience is a movement away from classical interiority where thought resides within a singular body. In “Ping Body” Stelarc is wired to the net and through random pinging of Internet domains, ping values are transformed into electrical volts which activate a muscle stimulator. Here Stelarc’s body is motivated by the external activity of online users. Ben Denham, writing of the relation and coevolution of speech, gesture, and thought, argues that what is interesting about this situation is that . . . the body must learn-to-make sense of its involuntary responses to this external system. Here the struggle to think operates on a kind of meta-level where the autonomy of the system demands an observational mode of thought that might then attempt to make sense of the programmed “randomness” of the thoughts of the system. (42) But this sense is intimately imbricated with the feel of the gestures produced by the “ping body”: it cannot be separated from their rhythm and timing, their slowness or speed, or the qualities of smoothness or jerkiness. They are what must be “observed” through proprioceptive reception. For Denham, Stelarc limits the body’s “capacity for self-initiated action while transferring a set of fullness to the external . . . [electronic] system” (42). One of the implications of this transfer is a re-corporealisation of the electronic network in which distance and density of internet activity emerge from untraceable bodies and quickly disappear into an electronic flux” (43). For Denham, Stelarc’s work trys to “think” the realization of electronic media as external but relational gesture. In conclusion, our gestures, Jousse argues, have become atrophied and inexpressive—almost algebraic: dissociated from the subtle soul of things (49). But electronic writing aims to revivify gesture (though perhaps much of it rather comprises an elegy to the lost relationship between gesture and the animate world). To revivify gesture is not to restore control or power to individual readers, but rather to restore the links between gesture and the world. It is perhaps too early to draw final conclusions about the gestural economy of electronic writing, but there is no doubt for us that it is one that will also radically alter the relationship between bodies and their environments, as Stelarc’s body art suggests.

132

Maria Angel and Anna Gibbs | Memory and Motion

Works Cited Algo Mantra Blog. “Cellphabet 1.0: How I can walk in English!” 2007. 27 Oct. 2009 . ———. “Cellphabet on BBC Radio 5.“ 2007. 27 Oct. 2009 . Abram, David. The Spell of the Sensuous: Perception and Language in a More-ThanHuman World. New York: Vintage, 1997. Angel, Maria, and Anna Gibbs. “On Moving and Being Moved: The Corporeality of Writing in Literary Fiction and New Media Art.” Literature and Sensation. Ed. Anthony Uhlmann, Helen Groth, Paul Sheehan, and Stephen Mclaren. Newcastle-upon-Tyne: Cambridge Scholars Publishing, 2009. 162-172. Barthes, Roland. S/Z: An Essay. Trans. R. Miller. New York: Hill and Wang, 1974. Bergson, Henri. Matter and Memory. Trans. N.M. Paul and W.S. Palmer. New York: Zone, 1991. Bessy, Christian, and François Chateauraynaud. Experts et faussaires: Pour une sociologie de la perception. Paris: Editions Métailié, 1995. Bolt, Barbara. “A Non Standard Deviation: Handlability, Praxical Knowledge and Practice Led Research.” Speculation and Innovation: Applying Practice Led Research in the Creative Industries. Brisbane: Queensland U of Technology, 2006 . Bouchardon, Serge. “The Aesthetics of Materiality in Electronic Literature.” Electronic Literature in Europe. Bergen: September 11-13, 2008. 27 Oct. 2009 . Bronowski, Jacob. The Ascent of Man. Boston: Brown, 1974. Cayley, John. windsound. 1999. 27 Oct. 2009 . ———. “The Code is not the Text (unless it is the Text)”. electronic book review. 10 Sept. 2002. 27 Oct. 2009 . Cayley, John and Dmitri Lemmerman. “Lens: The Practice and Poetics of Writing in Immersive VR (A Case Study with Maquette).” Leonardo Elec-

133

Maria Angel and Anna Gibbs | Memory and Motion

tronic Almanac. 14.5-6 (2006): 1-19. 27 Oct. 2009 . Chatelet, Gilles. Figuring Space: Philosophy, Mathematics, and Physics. Trans. Robert Shaw and Muriel Zagha. Dordrecht: Kluwer, 2000. Deleuze, Gilles, and Claire Parnet. Dialogues. Eds. Hugh Tomlinson and Barbara Habberjam. New York: Columbia UP, 1987. Denham, Ben. Gestural Sense: Art, Neuroscience and Linguistic Embodiment. Diss. Sydney: U of Western Sydney, 2009. Derrida, Jacques. Of Grammatology. Trans. G.C. Spivak. Baltimore: Johns Hopkins UP, 1976. Donald, Merlin. “Precis of Origins of the Modern Mind: Three Stages in the Evolution of Culture and Cognition.” 1997. 27 Oct. 2009 . ———. “Material Culture and Cognition: Concluding Thoughts.” Cognition and Material Culture: The Archaeology of Symbolic Storage. Eds. Colin Renfrew, and Chris Scarre. Cambridge: The McDonald Institute for Archaeological Research, 1998. 181-187. 27 Oct. 2009 . Dunne, Anthony. Herzian Tales: Electronic Products, Aesthetic Experience, and Critical Design. Cambridge, MA: MIT P, 2005. Hansen, Mark B. N. “Movement and Memory: Intuition as Virtualization in GPS Art.” MLN 120.5 (2005): 1206-1225. Harris, Paul A. “Fictions of Globalisation: Narrative in the Age of Electronic Media.” PhiN 7 (1999): 26-39. 27 Oct. 2009 . Hayles, N. Katherine. How We Became Posthuman: Virtual Bodies in Cybernetics, Literature and Informatics. Chicago: U Chicago P, 1999. Jousse, Marcel. Études de psychologie linguistique: Le Style oral rythmique et mnémotechnique chez les Verbo-moteurs, Paris: Gabriel Beauchesne, 1925. Kirby, Vicki. Telling Flesh: The Substance of the Corporeal. New York: Routledge, 1997. Massumi, Brian. Parables for the Virtual: Movement, Affect, Sensation. Durham: Duke UP, 2002. Merleau-Ponty, Maurice. Phenomenology: Language and Sociology. Ed. J. O’Neil. London: Heinemann, 1974.

134

Maria Angel and Anna Gibbs | Memory and Motion

McLuhan, Marshall. The Gutenburg Galaxy: The Making of Typographic Man. 1962. Toronto: U Toronto P, 2002. Morse, Margaret. “The Ontology of Everyday Distraction: The Freeway, The Mall, and Television.” Logics of Television: Essays in Cultural Criticism. Ed. Patricia Mellencamp. Bloomington: Indiana UP, 1990. 193-221. Ong, Walter. Orality and Literacy: The Technologizing of the Word. 1982. London: Routledge, 2002. Phillips, Susan A. “Physical Graffiti West.” Migrations of Gesture. Eds. Carrie Noland, and Sally A. Ness. Minneapolis: U Minnesota P, 2008. 31-66. Rotman Brian. “Gesture in the Head: Mathematics and Mobility,” Mathematics and Narrative Conference. Mykonos: July 12-15, 2005. 27 Oct. 2009 . ———. “Corporeal of Gesturo-haptic Writing.” Configurations 10 (2002): 423438. ———. “Going Parallel.” Substance 91 (2000): 56-91. Shaw, Jeffrey. The Legible City. 1989-1991. 27 Oct. 2009 . Solnit, Rebecca. Wanderlust: A History of Walking. New York: Viking, 2000. Stelarc. Ping Body. 1996. 27 Oct. 2009 .

135

Roberto Simanowski

Event and Meaning Reading Interactive Installations in the Light of Art History While theoreticians of digital media have stated that AI-controlled environments take the “body language” to a new level by controlling and reacting to its movement, mimics and gestures, theoreticians of interactive art have conceptualized “behaviorist” or “relational” art as a shift from content to event (Roy Ascott), from “private symbolic space” to the “realm of human interactions” (Nicolas Bourriaud). It is important, however, not to ignore the content of the event and the symbolic of the interactions. My paper explores the meaning of two interactive installations, which require very difficult bodily actions. While Still Standing by Bruno Nadeau and Jason Lewis demands the participant’s body to be immobilized as a condition for the reading and contemplation of its linguistic content, Mondrian by Zachary Booth Simpson and his collaborators allows the audience to generate Mondrian-like images by drawing lines on a screen with one’s hand and coloring sections with one’s finger. Theses pieces do not only offer two different concepts of the interactors’ action and hence body experience but also engage in a very complex way with the issues of inter- and transmediality as well as avant-garde. While Still Standing uses new technology in order to enhance the cultural practice of reading endangered since the arrival of electronic and digital media, Mondrian promotes craftsmanship and parodies the aesthetics Mondrian represents. Both interactive installations, I will argue, do not simply create “a period of time to be lived through” (Bourriaud) but have to be understood in the context of art history and as a specific contribution to it.

1

The Problems of Code, Body and Close Reading

There are two problems in researching digital aesthetics: Scholars emphasize too much either the code or the body.1 With no doubt, code is an indispensable aspect in every discussion of digital arts. Since it is code what makes everything happen on the screen or on the scene, everything happening is subject to the grammar and politics of code. In many cases and in many regards it is important to understand what can be done and what has been done on the level of coding in order to understand the semantic of a digital artefact, just as

137

Roberto Simanowski | Event and Meaning

with film one needs to understand the potential of the technology (camera, sound, cut) in order to determine the value and meaning of a given action. There is also no question that the body is an important element in any analysis of interactive installations. With respect to the perception of traditional arts such as literature and painting theorists focusing on the reader or spectator acknowledge that the physiological specificity of the lived body (gender, race, age, weight, health) contributes to the way an artifact is perceived. If the audience is physically engaged in the art and the body becomes the central subject of the aesthetic experience (i.e., as agent and subject matter), the body obviously plays an even more significant role. However, a preoccupation with code and body threatens to override our attention to the deeper meaning of an artifact and may produce declarations that are hardly helpful if not completely misleading. Such declarations are for example the notion that everything in digital media is actually literature since everything is based on alphanumeric code or the proclamation that digital spaces represent a strong desire for control over the messiness of bodies and unruliness of the physical world, since everything in digital media is coded and computed.2 While such proclamations are not principally wrong, they are not very helpful for their formalistic approach and focus on the technology behind the interface, which neglects the actual experience of the audience. If we look at the scenery of an interactive installation such as David Rokeby’s Very Nervous System we easily realize that this closed-circuit-installation in which the physical action of the interactor alters the acoustic information sent from the system neither is literature nor intends to control the messiness of bodies but presents an interactive performance that entices the interactor to “produce” a body completely different from the controlled body of everyday life. The other problem is the focus on the body’s action at the expense of the meaning of these actions. Such perspective is put forward when interactive art is described as a shift from content to event, from the communication of a certain message to the production of a space that inaugurates dialogue (Ascott 110ff.; Bourriaud 166). When the British artist and theorist Roy Ascott, in his 1989 essay “Gesamtdatenwerk: Connectivity, Transformation, and Transcendence,” writes the audience no longer can be at the window looking in on a scene composed by another, but instead is invited to enter the doorway into a world where interaction is everything (226), he suggests that there is no longer a scene one could look at. When the French theorist and curator Nicolas Bourriaud in his 1998 book Relational Aesthetics conceptualizes interactive art as “a period of time to be lived through” in contrast to the private symbolic space traditional art provides (14f.), he neglects that this lived through time itself embodies a symbolic space to be reflected on.

138

Roberto Simanowski | Event and Meaning

The celebrated openness and formlessness of interactive art has been subjected to strong criticism. Hal Foster for example notes in reaction to Bourriaud: “for all its discursivity, ‘relational aesthetics’ might be sucked up in the general movement for a ‘post-critical’ culture—an art and architecture, cinema and literature ‘after theory’” (195).3 Foster’s conclusion may sound conservative or at least anti-avant-garde and reminds us of Adorno’s aesthetics. It is remarkable that it comes from a left-wing intellectual known for his criticism of the contemporary aesthetic of spectacle and for his insistence on the political engagement of art. The charge of post-criticisms refers to the conceptualization of interactive art as an unstructured event to be experienced rather than to be analyzed; it refers, to quote Hans Ulrich Gumbrecht quoting Susan Sontag, to the “farewell to interpretation” advocated in some contemporary aesthetic theories.4 To be sure, even an open interactive system like Rokeby’s work has an underlying structure and generates a specific symbolic of the interactions. In this case the underlying message of Rokeby’s “systems of inexact control,” as he calls his installations, is the deconstruction of the illusion and fantasy of total control set up by the medium computer. Rokeby considers the fetishization of control an unhealthy paradigm for real-world encounters and thus creates systems which do not allow the interactor to gain complete control. The question, however, is, how scholars and art critics approach such open interactive systems. The question is how we understand or read the experience of such interactions, which brings me back to my notion at the beginning that code and body are emphasized too much in the discussion of digital arts. While I consider it important not to ignore the general role of digital technology on digital art, I am afraid the focus on this technology obstructs the access to each artwork’s aesthetic singularity.5 Hence, I advocate a close reading of the artwork, which on one hand sticks to the “surface” of the code, i.e., to its materialization as text, sound, image, and action on the screen or scene.6 On the other hand, such close reading should look behind the surface of the body, i.e., it should inquire the meaning of the body’s actions within the framework provided by the code. Such reading must take technology into account where it is important but also discuss the connections to philosophy and art history where they are obvious or helpful. To give an example of such reading I am going to discuss two interactive installations which require rather diverse bodily actions and refer in quite different ways to technology. While the first work contains text and thus can be seen as an example of digital literature, the second does not and rather seem to belong to the genre of digital painting. However, my purpose is not only to present a close reading of digital literature but also to exemplify an approach to digital art that carries out the main principle of literary-critical training, i.e., to

139

Roberto Simanowski | Event and Meaning

follow up on each aspect of a text that is unfamiliar and strikes us as significant. My intention is to carry out such follow-ups on linguistic as well as nonlinguistic aspects of two digital artworks that strike me as significant.

Détournement and Inter-Media-Competition: Still Standing

2

Still Standing (2005) by Bruno Nadeau and Jason Lewis from OBX Labs Concordia University puts the interactor in front of a big screen and features letters reacting to her movements (fig. 1). Nadeau explains on his web site:7 the installation consists of an amalgam of characters projected on the wall as if they were resting on the floor. when a participant walks in front of the projection, the first reaction of the text is to act as if it was being kicked, pushed by the person’s feet. when the participant stops for a short moment, the text is attracted towards his position and moves up, like water soaking his body. the participant can then enjoy a motionless moment and contemplate the textual content that becomes more and more legible. when the user is done and decides to start moving again, the text falls back to the floor and wait for a new interaction. The “grammar of interaction,” i.e., the modus of interaction the artist made possible within the interactive environment,8 insists on the immobilization of the interactor, making this a key for accessing the text which reads: five chapters of addiction for my perpetual commotion bring my brain to a stop. the inception of sedation is needed for the waves to break and the spin to reduce. letters to literal the motionless moment hides for my sight to seduce. The message of Still Standing is quite evident and directly expressed at Nadeau’s web site: nowadays, designs are created to be decrypted and enjoyed at a glance, requiring no attention span. the piece evolved as a response to the “collapse of the interval.” a phenomenon of fast pace culture that rarely allows us a moment to stop and observe.

140

Roberto Simanowski | Event and Meaning

Fig. 1. Bruno Nadeau and Jason Lewis: Still Standing (2005).

The underlying subject of Still Standing is the “cannibalistic” relationship between the semiotic systems of text on one hand and visual art or interactive installation on the other hand: The “consumption” of text by replacing it with images or by transforming it into image, sound, action depriving it of its linguistic value.9 In contrast to such consumption, Still Standing applies sophisticated digital technology not to marginalize text but to demand attention to it by commanding the viewer to stand still and concentrate on text. Paradoxically, the piece using new technology enhances the cultural practice of reading, otherwise endangered since the arrival of electronic and digital media. New technology turns out to be a kind of Trojan horse containing an old-fashioned paradigm of communication. The letters standing in line inside the user’s silhouette in Still Standing resemble the Greek soldiers lined up within the Trojan horse. This strategy reminds us of the use of cinema by Guy Debord in the 1950s to protest the transformation of the world into a society of images. After World War II image production increasingly occupied the conscious and unconscious processes by means of which the subject sensed, desired and understood the world. According to Debord the cinema had become the cathedral of modernity, reducing mankind, previously an autonomous, contemplative subject, to an immobile, isolated, passive viewer, sitting in the dark and fixed in front of the shining screen. In reaction to this voyeuristic fixation Debord declared war against cinema not, as his 1964 film Contre le cinéma shows, by renouncing film but by appropriating it and freeing it from the dominance of the spectacle. An example of this iconoclastic reappropriation of film is Debord’s

141

Roberto Simanowski | Event and Meaning

Hurlements en faveur de Sade (1952), an eighty minutes long film without pictures and with almost no sound; only from time to time three voices recite, without any expression, fragmentary sentences taken from bodies of laws, modernistic literature and newspapers during which the screen changes from black to white. With this film Debord temporally occupied the cinema and interrupted the circulation of false images with the intention to use the suspended film to create critical awareness. Such “hijacking” of the new medium in favor of the old is part of Debord’s concept of détournement, a subversive “turnabout” of the meaning of an object, space, image, or idea. It comes as no surprise that the audience was not interested in spending eighty minutes this way in the cinema. The premiere on June 30, 1952 ended in chaos and scandal, the film was stopped after less than ten minutes. In Still Standing Debord’s iconoclasm translates into the critique of bustling activity in front of the screen. Nadeau and Lewis interrupt the business of action and interaction which not only has become the new religion in art but also an integrated element of the Society of Spectacle that Debord described with regard to image production. Part of this trend is the abandonment of reflective, contemplative reading. Forcing the audience to stand still in order to read text on the screen of an interactive installation is similar to having the audience watch an empty screen in the movie theater. The irony, however, is that Still Standing sends an almost empty horse into Troy. It does not, for instance, refresh the text once the interactor has finished reading it. Tracking the eyes should be no problem, nor replacing one text sequence by another. But could Still Standing have kept the interactor still standing still after three, ten or even a hundred text fragments have been presented? Could it have told an entire story this way? Could it capture its audience longer than Debord’s film did? It doesn’t dare to try. For good reasons. The prospects of a text’s survival in a “hostile” environment such as an interactive installation are limited. In fact by not refreshing the text, by not testing the patience of the audience, this installation actually utters exactly this belief and essentially portraits its own undertaking as futile. To put it this way: the almost balanced proportion between the time the text needs to build up and the time one needs to read the text allows to experience this moment of standing still as an action in its own right. By abstaining from requesting a longer period of immobilization and thus requesting a long attention span, Still Standing undermines its own agenda and contributes to the fast pace culture it criticizes.

142

Roberto Simanowski | Event and Meaning

3

Avant-garde and Recuperation: Mondrian

The interactive installation Mondrian by Zachary Booth Simpson and his collaborators allows the audience to generate Mondrian-like images by drawing lines on a screen with one’s hand and coloring sections with one’s finger. Simpson’s “company,” Mine-Control, claims “art can be both playful and thought-provoking.”10 Mondrian is certainly playful. To what extent is it thought-provoking?

Fig. 2. Zachary Booth Simpson et. al.: Mondrian (2004).

The accompanying text on the web site states: this work “permits participants to simply sketch out and edit compositions in the style of the great abstractionist Piet Mondrian. Create your own composition in 10 seconds!”11 There are two interesting aspects to this statement: The homage to Mondrian as a great abstractionist and the promise to imitate him in 10 seconds. As for the homage, it should be noted that Mondrian is named a great “abstractionist,” not a great painter or artist. While this may be a random, insignificant detail, Mondrian’s position as artist is certainly undermined and trivialized by suggesting the audience could do in 10 seconds a painting that would have taken Modrian considerably longer.

143

Roberto Simanowski | Event and Meaning

While we don’t know how long it took Mondrian to create his paintings or “compositions” as he called them, we know that it took him a while to overcome his naturalistic and impressionistic style and find his own voice. His style became an important milestone in the history of art, together with other movements in painting that signaled the shift from the mode of representation to pure abstraction. Mondrian’s work contains—in a similar manner to the manner Duchamp carried more than just a “readymade” into the museum—a complexity that belies its apparent simplicity. What makes Mondrian’s, and even more so Duchamp’s, work art is not something that meets the eye but something that happens in the intellect. This art is not perceptual, in terms of sensual perception, but conceptual. It can no longer fully be appreciated on the visual level—though spectators did and do visually appreciate Duchamp’s urinal as well as Mondrian’s compositions—but requires an understanding of its historical and conceptual references. Nonetheless or rather for that reason, Mondrian’s work was often parodied and even trivialized. The Rock band Botch released a song entitled “Mondrian Was a Liar” that speaks of the hardship of life and work with your hands. Mondrian only appears in the title of this song. However, the lyrics imply that he was not a hard manual worker, in contrast, say, to the painters of the trompe l’œil school who aimed at such virtuosic craft that their paintings could fool the audience into taking them for reality. Does Simpson think Mondrian was a liar? By allowing the audience to paint just like Mondrian with a few clicks and flicks of the hand and finger, Simpson’s piece unavoidably lends weight to the idea that Mondrian’s work is so simple a child could do it. The fact that this installation effortlessly generates artifacts that “look just like it did in the museum,” as participants may think, mocks the famous artist. But it goes further, because it is not a canvas on which people “unchain” the inner Mondrian in their soul but an interactive screen with highly sophisticated programming behind it. The programmer is Simpson and his team. While the participants may change their conception of Mondrian’s talents, they will appreciate the achievement of the creators of the Mondrian machine. This achievement is based on excellent coding. It is based on craftsmanship, which is what Mondrian was denied when he was parodied, mocked or even called a liar. Simpson’s work not only mocks Mondrian, it actually offers a better candidate for homage. It establishes the fame of the programmer at the expense of the painter or rather it praises “real” painting (based on virtuosity) over conceptual art. Simpson does not really honor Mondrian with his Mondrian machine, but uses Mondrian’s abstractionism to position himself against it and to celebrate his superiority over Mondrian with respect to craftsmanship. Simpson’s work turns out to be an example, deliberately or subconsciously, of

144

Roberto Simanowski | Event and Meaning

deconstructing the avant-garde. In the terms of Debord we may speak of recuperation which is the commodification and incorporatation of the subversive results of a détournement within mainstream society. Simpson’s Mondrian machine functions as a toy, which is fun to play with for a brief time. The discussion, however, reveals that it is more than simply a toy and raises the question: Can it also be art? According to Danto, for the evaluation of an artifact as art or non-art neither craftsmanship nor visible quality is important but “aboutness.” He explains this aboutness with respect to Andy Warhol’s Brillo Boxes, an arrangement of the branded cardboard cartons containing scouring pads that are sold in supermarkets. In Warhol’s installation these boxes contain more than the scouring pads. They are not only about their content but also about the boxes themselves, about their visual pleasure, about their possible status as art. Warhol’s boxes comment on the original boxes and thus become art, as Danto states in his essay “Art and Meaning.” Can we say the same about Simpson’s Mondrian? The answer may be Yes and No. It is No if we see this work only as an example of coding, i.e., artistry and virtuosity. Coding, understood in terms of craftsmanship, would be a very problematic foundation for art and could certainly not bear comparison with the conceptual complexity of Mondrian’s work. However, the answer may be “Yes, Simpson’s Mondrian machine is art,” if we see it as a comment on Mondrian’s lack of craftsmanship. Simpson’s work could be understood as being about (and against) a concept of art that approaches art only from an intellectual perspective rather than on the basis of technical virtuosity. This model of art is adequately described by Sol LeWitt who, in his Paragraphs on Conceptual Art (1967), holds that in conceptual art “the idea becomes a machine that makes the art” (12). LeWitt, in his Wall Drawings, consequently delegates the execution of the idea to a draftsperson. The “aboutness” of Simpson’s work may be a critique of this liberation of art from any skill of the artist as a craftsman. It is a critique of the “philosophical disenfranchisement of art,” as Danto entitles one of his books. It is the end of the expulsion of the technical. While Mondrian presents his intellectual sophistication by reducing painting to its utmost simplicity, Simpson presents, in a kind of counterstrike, his technical sophistication by creating a machine that imitates that simplicity: an objection to the intellectualization of art which is itself conceptual and intellectual.12 Of course, it is the simple shape of the grid in Mondrian’s work that allows its imitation within the digital medium as matrices of discrete values. And it is exactly this paradigm of the grid which allows us to eventually see Simpson’s turn against Mondrian as continuation of what Mondrian represents. As Rosalind Krauss points out in her essay “Grids”: “The grid functions to declare the modernity of modern art” and announces “modern art’s will to

145

Roberto Simanowski | Event and Meaning

silence, its hostility to literature, to narrative, to discourse” (9). Similar to the database in Manovich’s account (Manovich 225), the grid is the enemy of the narrative and represents, as Krauss continues, “a naked and determined materialism” (10). As Lutz Koepnick asserts in his essay “[Grid Matrix]: Take II” grids, with their conceptualization of the world as a predictable system of horizontal and vertical lines, “energize sensation of control, power, and omnipotence” and thus “arrange the perceptible world as something that more or less excludes the possibility of chance and surprise” (53). As Koepnick continues, the grid “removes the messy, strange, and the mysterious and entertains the modern subject with sensations of unmitigated presence and wholeness” (54). His words recall the notion of coding in digital media as control over the messiness of bodies and unruliness of the physical world. In fact, the grid is seen as having exactly the same effect as the code when Koepnick states: “To ‘go digital’ is to reconstruct the world within a grid of discrete values” (48).13 Insofar as the grid signifies control and represents one of the “most stunning” myths in modernity—as Koepnick states, in line with Krauss (54)— Simpson’s Mondrian does not actually oppose Mondrian but reconfirms his practice. Simpson radicalizes the concept of Mondrian’s grids by expanding it from the surface (of the canvas) to the level of code thus applying the modus of “discrete values” even to the process of constructing the grids on the screen. In light of this doubling of the emblem of control and its “declaration” of modernity we may understand the code as first-degree relative of the grid and consider Mondrian the inheritor to the avant-garde represented by Mondrian. The irony of this view is the return of craftsmanship (as software engineering) in art which had been fundamentally disregarded by the avant-garde.

4

Conclusion

As has become clear, both interactive installations do not simply create “a period of time to be lived through,” as Bourriaud holds for artworks belonging to relational aesthetics (14f.), but create a symbolic space which can and needs to be made to the object of interpretation. This is especially the case since in both installations the grammar of interaction makes the interactors doing exactly what the artist expects them to do. In addition, the use of text in Still Standing and the direct reference to art history in Mondrian contain a statement by the artists that already calls for interpretation. However, as Still Standing demonstrates, interpretation is needed also beyond the text, namely with respect to the experience of the body. Interpretation is even needed beyond this experience of the body considering the fact that the text does not refresh and thus the body’s experience of immobilization

146

Roberto Simanowski | Event and Meaning

is not extended into an area where this immobilization doesn’t feel like an action anymore. If we want to understand the deeper meaning of a work, we have to factor in the experiences the work does not allow the body to encounter. In the case of the Mondrian machine I consider the role of the body rather neglectable. One does not really need to physically experience the creation of one’s own Mondrian in 10 seconds in order to understand the allusion and implications of such parody. To put it this way: Who does not get the allusion and implications hearing how this work functions will not do better having played with the piece in person. As far as the focus on code is concerned, in the case of Mondrian the discussion of the general nature of digital technology does not impede the access to the artwork’s aesthetic singularity but provides it on the ground of a comparison of code and grid as well as coding and craftsmanship. In the case of Still Standing I take coding only into account insofar as I assume it is feasible to refresh the text, though, I don’t consider it unlikely that an exhaustive investigation of the code underlying Still Standing and Mondrian would generate different results. I would certainly welcome such readings. However, I will wait until they arrive and prove that such investigation allows new insights into the nature of these artworks, before I renounce my claim that code is overrated in the discourse of digital arts.

Notes 1

“Code” here is understood as alphanumeric code used on the operational level of computers. “Body” refers to the physical entity in contrast to the mind as conceptualized in dualistic philosophy. While here is not the place to engage in a discussion of the mind-body dichotomy (debating for example to what extent mental processes are the result of sensory organs or on the contrary physical experiences may result from mental properties), for the discussion at hand it is useful to differentiate between the perception of an artifact which makes the body itself the ground of immersive participation and the cognitive reflection (interpretation) of this very interaction and immersion.

2

For the notion of any artifacts based on software as literature cf. the essay “Software Art and Writing” by Florian Cramer and Ulrike Gabriel: “If ‘literature’ can be defined as something that is made up by letters, the program code, software protocols and file formats of computer networks constitute a literature whose underlying alphabet is zeros and ones. By running code on itself, this code gets constantly transformed into higher

147

Roberto Simanowski | Event and Meaning

level, human-readable alphabets of alphanumeric letters, graphic pixels or other signifiers. These signifiers flow forth and back from one aggregation and format to another. Computer programs are a literature in a highly elaborate syntax of multiple, mutually interdependent layers of code. . . .” For a critical response cf. Trevor Batten’s annotations at: . For an account of the arguments on body and control cf. Munster (1ff.). 3

Foster maintains that politics are sometimes ascribed to such art on the basis of a shaky analogy between open work and inclusive society: “as if a desultory form might evoke a democratic community, or a non-hierarchical installation predict an egalitarian world”; Foster notes that the artists and theorists in question frequently cite, without justification, the Situationists who valued precise interventions and rigorous organization above all things (193).

4

For a critical discussion of the announced farewell to interpretation in contemporary aesthetic theory in the context of digital art cf. Simanowski, Against the Embrace and Simanowski, Digitale Medien in der Erlebnisgesellschaft 246-275.

5

In the same spirit John Zuern concludes in his discussion of the appropriate methods in the study of digital literature: “Special pleading for the digital impedes our access to each artwork’s ‘literary singularity’” (238).

6

For the concept of surface cf. also Noah Wardrip-Fruin: “The surface of a work of digital literature is what the audience experiences: the output of the processes operating on the data, in the context of the physical hardware and setting, through which any audience interaction takes place” (48).

7

The web site also provides a video of the installation.

8

“One could say that interactivity is the field for constructing sentences. This field is regulated by a kind of grammar which is not same as the grammar for writing sentences, but rather a grammar that tells you how to use it.” (Fujihata 319)

9

I have discussed this phenomenon elsewhere as competition between the new media and the old with reference to the concept of remediation and cultural anthropophagy (Simanowski, “Textual Objects”).

10 . 11 . The web site also provides a video of the installation. 12 This notion raised the question whether Simpson really intended this kind of statement and points to the issue of the relationship between meaning

148

Roberto Simanowski | Event and Meaning

and intention, which cannot be discussed here. No matter whether we insist that the “aboutness” is intended by the artist or accept that it is simply present in the artwork, the Mondrian-machine is exactly what Simpson’s team states art can be: “both playful and thought-provoking.” 13 Koepnick further discusses the relationship between grids and digitality on pages 60-63.

Works Cited Ascott, Roy. Telematic Embrace: Visionary Theories of Art, Technology, and Consciousness. Ed. Edward A. Shanken. Berkeley: U of California P, 2003. Botch. “Mondrian Was a Liar.” We Are the Romans. Hydran Head, 2000. Bourriaud, Nicolas. Relational Aesthetics. Paris: Les presses du réel, 2002. Cramer, Florian, and Ulrike Gabriel. “Software Art and Writing.” American Book Review 22.6 (2001). DIY Media: Kunst und digitale Medien: Software, Partizipation, Distribution. Ed. Andreas Broeckmann and Susanne Jaschko. Berlin: BKV, 2001. 29-33. Danto, Arthur C. “Art and Meaning.” Theories of Art Today. Ed. Noël Carroll. Madison: U of Wisconsin P, 2000. 130-140. ———. The Philosophical Disenfranchisement of Art. New York: Columbia UP, 1986. Foster, Hal. “Chat Rooms.” Participation. Documents of Contemporary Art. Ed. Claire Bishop. Cambridge, MA: MIT P, 2006. 190-195. Fujihata, Masaki: “On Interactivity.” Takeover: Who’s Doing the Art of Tomorrow. Ed. ARS Electronica Vienna. New York: Springer, 2001. 316-319. Gumbrecht, Hans Ulrich: “A Farewell to Interpretation.” Materialities of Communication. Ed. Hans Ulrich Gumbrecht and K. Ludwig Pfeiffer. Palo Alto: Stanford UP, 1994. 389-402. Koepnick, Lutz. “[Grid Matrix]: Take II” [Grid Matrix] Ed. Sabine Eckmann and Lutz Koepnick. Washington, DC: Mildred Lane Kemper Art Museum, 2006. 47-75. Krauss, Rosalind E. “Grids.” The Originality of the Avant-garde and Other Modernist Myths. Cambridge, MA: MIT P, 1986. 8-22. LeWitt, Sol. “Paragraphs on Conceptual Art.” Conceptual Art: A Critical Anthology. Ed. Alexander Alberro and Blake Stimson. Cambridge, MA: MIT P, 2000. 12-17.

149

Roberto Simanowski | Event and Meaning

Manovich, Lev. The Language of New Media. Cambridge, MA: MIT P, 2001. Munster, Anna. Materializing New Media: Embodiment in Information Aesthetics. Hanover: UP of New England, 2006. Nadeau, Bruno. Still Standing. 2005. 30 Oct. 2009 . Rokeby, David. “Very Nervous System and the Benefit of Inexact Control. Interview with Roberto Simanowski.” Dichtung Digital 5.1 (2003). 30 Oct. 2009 . Simanowski, Roberto. Against the Embrace: The Recovery of Meaning Through the Reading of Digital Arts. Minneapolis: U of Minnesota P, 2010. Forthcoming. ———. “Textual Objects: Refashioning Words as Image, Sound and Action.” Leonardo 43.2 (2010). Forthcoming. ———. Digitale Medien in der Erlebnisgesellschaft: Kultur—Kunst—Utopie. Reinbek: Rowohlt, 2008. Simpson, Zachary B. Mondrian. 2004. 3 Nov. 2009 . Wardrip-Fruin, Noah. “Five Elements of Digital Literature.” Reading Moving Letters: Digital Literature in Research and Teaching. A Handbook. Ed. Roberto Simanowski, Peter Gendolla, and Jörgen Schäfer. Bielefeld: Transcript, 2010. 29-57. Zuern, John. “Figures in the Interface: Comparative Methods in the Study of Digital Literature.” Reading Moving Letters: Digital Literature in Research and Teaching. A Handbook. Ed. Roberto Simanowski, Jörgen Schäfer, and Peter Gendolla. Bielefeld: Transcript, 2010. 59-80.

150

Literature between Virtual, Physical and Poetic Space

Andrew Michael Roberts

Why Digital Literature Has Always Been “Beyond the Screen” 1

Introduction

My title is, of course, something of an exaggeration. I do not wish to deny that the screen has been a dominant mediating technology for much digital literature. Furthermore, the locative narratives and immersive virtual reality (VR) works produced by artists such as Jean-Pierre Balpe and John Cayley do represent new forms of aesthetic experience, in which bodily movement and the human relationship to existing or constructed environments become central to the experience of the work in ways which are new to literature (though they may appear less radically new in the context of installation and media art). However, I do wish to argue that the impulse to go “beyond the screen” pervades many digital works—even those which are technically screen-based. So the current movement towards “aesthetic processes . . . in the physical realm between the interfaces of technical sensors or effectors and the human body” (Beyond the Screen), which the present book identifies and analyzes, springs not only from new technical possibilities offered by interactive technologies, but from a long-standing dynamic in digital literature. This dynamic in certain ways replicates and continues impulses within literary modernism, both in its responses to new technologies and in its enactment of ideas about authorship, readership and interpretative processes. As is often the case, the move “beyond” is also a return. Many of the practices and framing ideas of the move “beyond the screen” are articulated in terms which have also been used to define, refine and question conceptions of modernism and postmodernism: in particular time, space, and the body. I shall also suggest that the move to “more complex interfaces” (Beyond the Screen) continues a strategy of “literalization” or “enactment” which is a pervasive feature of digital literature, and which connects it to print literature and the critical theory which it generated. I shall therefore offer a provisional “yes” to the question posed by the editors of this volume, “whether we can continue talking of a specific migration of traditional literary forms into computer-based and networked media” (Schäfer and Gendolla 16), although that migration no doubt requires us to rethink terms such as “literary” and “forms.”

153

Andrew Michael Roberts | Digital Literature

2

Digital Literature Has Always Wished to Go Beyond the Screen

Arguably, digital literature has always wished to go “beyond the screen” (if the screen is defined as a flat display surface for digitally-enabled works): to augment what is presented on the screen and to make the experience of looking at the screen like an experience of something else. While the ability to achieve this move, with the flexibility, richness and speed allowed by locative, immersive and virtual reality technologies, represents a radical break with print literature, the impulse to go beyond the screen has continuities with an impulse to go beyond the page, so as to engage the faculties of the reader as fully and intensely as possible. This impulse is apparent in print literature in a wide range of features: the invocation of bodily experience; the attempt to convey visual, haptic, aural and other sensory simulation in words; the thematization on multiple discursive levels (fictional, formal and critical) of “depth” (the dimension absent form the physical properties of the page); the value attributed to multiple and varied readings of a text; effects of movement within a text and the valorization of these in literary criticism; the role in literary culture of performance and the physical presence of the reader. Many of these features, while detectable in print literature of diverse periods, emerge with particular intensity in literary modernism. If modernism was (or is) a response to the development of mass urban society, to rapid social and technological change and to globalization (all of which might seem to threaten assumptions of a defined, consensual discursive space shared by author and reader), then the impulse to draw the reader into multiple forms of intense engagement with the literary work might be seen as a compensatory response to these aspects of modernity. For example, Andrew Bowie writes of a loss of the sense of general or shared meaning: “Modernity both creates space for the proliferation of individual meaning and tends to destroy the sense that such meaning really matters in terms of the general goals of society” (12). The experience of modernity may involve a sense of loss of contact with the past, as described by Matei Calinescu, who writes that the modern artist is cut off from the normative past with its fixed criteria. . . . What we have to deal with here is a major cultural shift from a time-honoured aesthetic of permanence, based on a belief in an unchanging and transcendent ideal of beauty, to an aesthetics of transitoriness and imamnence whose central values are change and novelty. (3) One might think here also of Baudelaire’s description of modernity as “the transient, the fleeting, the contingent” (403). Such an aesthetics is likely to

154

Andrew Michael Roberts | Digital Literature

privilege intensity in the moment, as expressed in writers such as James Joyce, Virginia Woolf and Imagist poets. What is immanent in the transitory moment both comes from beyond the page, and reaches out beyond the page. Some characteristics of the impulse to go beyond the page derive from specifically modernist practices: notably from formal strategies and declarations of poetic principle which seek to efface the writer in favor of reader autonomy, and from the prominence of the wanderer in the city (or flâneur) within modernist literature. The terms in which I have described this dynamic (and even the word “dynamic”) may seem to imply a teleology in which new technologies allow digital literature to achieve in substantial form effects to which print literature could only aspire. In fact there is clearly potential for both loss and gain in such changes. Furthermore, I have so far simplified the process for purposes of exposition, making it sound unidirectional, when in fact there are cross-currents and reactions of various sorts. One of these emerges powerfully in the debate between Mark Hansen and N. Katherine Hayles around questions of embodiment in relation to digital art works. While the parameters of this debate far exceed the scope of the present article, a central tension is between Hayles’s stress on an equality between human and machine, and Hansen’s wish to assert the priority of the human. So, for example, Hayles describes Cave works as “an interface in which a human user cooperates and competes with intelligent machines” (Electronic Literature 14), whereas Hansen writes that aesthetic experimentations with VR ultimately demonstrate . . . the capacity of new media to accord the body new functionalities . . . the virtual in VR lies on the side of the human even when it is placed in direct correlation with digital information: at stake in our encounter with the digital is nothing less than an opportunity for us to carry out an embodied virtualization. (194-195) The move beyond the screen may be conceived as a reassertion of the human and affective against a dominance of the (potentially machinic) visual faculty in screen-centred artistic work; part of what Hansen terms “a [paradigm] shift from a dominant ocularcentrist aesthetic to a haptic aesthetic rooted in embodied affectivity” (12). Alternatively, it may signal the accelerated emergence of the “posthuman” in a “co-evolutionary” process shared between humans and machines, which gives these two equality in that “technologies . . . have their own material specificities as central to their understanding as human physiology, psychology and cognition are to understanding how (human) bodies work” (Hayles, Electronic Literature 12).

155

Andrew Michael Roberts | Digital Literature

3

Literalization/Enactment

What I have referred to as literalization or enactment is the process whereby formal and stylistic effects, as well as implicit or explicit conceptions of authorship, readership and interpretation, which appear in print literature, literary criticism and literary theory in metaphorical or abstract form are given literal form or are physically enacted in digital literature. The move towards the literal, enacted or performed appears as a move from the represented, evoked, metaphorical or imagined towards that which is presented, carried out, nonmetaphorical and physical. This trajectory from the static, single-media page to the mobile inter-medial screen/loudspeakers would seem logically likely to continue towards complex environmental interfaces (mixed, virtual, or cultural-geographic). The term “literal” is of course potentially problematic, and should perhaps be used only “under erasure.” But the same applies to many of the crucial terms in the field of digital literature and new media, such as “digital,” “virtual,” “body,” “actual” and “reality.” I should note here John Cayley’s important conception of “literal art,” defined as “[p]oetic practice informed by the materiality of language” (“Literal Art” 210).1 However, I am using the terms “literalization” with reference to the more familiar sense of “literal” which opposed it to “metaphorical,” although given what we know about the pro foundly metaphorical nature of much natural language, this should be understood in terms of relative degrees of the metaphorical and literal, as well as the inter-relations of “dead” and “live” metaphors. Various critics have noted such a “literal” aspect to digital literature: for example Bachleitner writes of kinetic poetry that “words are let loose in a physical, quite literal sense” (303). “Actualization” would be another possible term, but if “literal” is successfully defamiliarized by Cayley’s distinction between “literal” and “digital” art, then “actual” is similarly destabilized by its potential opposite, “virtual.” However, one does see “actual” (and “realization”) used in a corresponding manner. For example, Rita Raley, in her Editor’s Introduction to a “Writing 3D” edition of TIR Web, comments that “we will see a collection of texts in which three-dimensionality is suggested [on a screen] and some in which it is actually realized” (1). Of course so-called “virtual reality” is one way in which such “actual” realization takes place. What we see here is the displacement of categories as the dead metaphors embedded in such terms return to haunt them. In the face of this instability, a simple example of what I mean by “literalization” or “enactment” may be helpful. Concrete poetry often presents the idea of movement, such as the idea of convergence in Emmett Williams’s “Like Attracts Like” (Rothenberg and Joris 307). In digital kinetic poetry, the words literally move. The implications of this shift are ambivalent. It could be seen as a satis-

156

Andrew Michael Roberts | Digital Literature

fying achievement of an implicit aspiration in print poetry (cf. Schaffner). On the other hand, it could be seen as a shift of power from reader to work: the ideas of movement produced by “Like Attracts Like” are cognitive processes on the part of the reader, which digital kinetic poetry appropriates and performs in accordance with computer algorithms. Reflection on this process of development from print to digital literature offers some perhaps unexpected perspectives. We are used to connecting the “digital” with the “virtual,” but what I am calling “literalization” or “enactment” could also be seen as a shift from the virtual to the literal: from virtual movement and depth in print literature to actual movement and depth in digi tal literature.2 Similarly, we associated the words “digital” and “code,” but this shift could be seen as one from the encoded to the performed. Here the instance of the body is particularly salient. N. Katherine Hayles, alluding to the work of Mark Johnson, comments that “embodiment is encoded into language through metaphoric networks” (Posthuman 206). Through multi-sensory stimulation and the invitation physically to interact via the mouse and keyboard, much digital literature elicits a performance of interpretation from the reader/ viewer, radically changing the mode of attention involved in the interpretative/ aesthetic experience. Like an actor in a play, we to some extent perform a work of digital literature rather than reading it, and in this way are both closer to it and less able to reflect on it while it is in process (there is a loss of reflective distance which has both positive and negative results). Locative and immersive works take this process further, with greater bodily activity on the part of the reader, and stronger forms of immersion.

4

Movement

Literary criticism has often invoked ideas of movement or dynamism within a text, especially poetry, in which the effects of rhythm, rhyme and other formal features, as well as the interaction of form with syntax, are frequently conceptualized in dynamic terms. Often there is an interplay between the “movement” of syntax and the formal structures (such as line breaks) across which the movement takes place (cf. Bradford). As already noted, in many works of digital literature, words literally move on a screen. Kinetic poetry is especially likely to use movement literally to figure meaning, in continuity with the use of shape in some forms of concrete poetry—a continuity humorously made ex plicit in Johannes Auer’s net poem worm applepie for döhl. For example, in Robert Kendall’s Faith, as Norbert Bachleitner observes

157

Andrew Michael Roberts | Digital Literature

the phrases “walking out”, “leave taking”, “forgoing-going-gone” and “stride out” . . . walk out of the text. . . . The phrase “Off the rocker (Yippee!)” bends down to the right, and finally the word “leap” jumps into the foreground. (313) As Hayles notes, the movement in Faith signifies at the level of consciousness as well as at the level of individual word or phrase, as the work “visually performs the vacillations the lyric voice verbally articulates between logic and faith” (Electronic Literature 29). In this case I feel that some of the negative connotations of the word “literal” are applicable to this literal performance: the work’s binary opposition between “Faith” and “logic” seems literal-minded and simplistic in a way which the kinetic effects at times reinforce rather than mitigate. Similarly, Bachleitner interprets the “disruption and fragmentation of hypertext” in Deena Larsen’s “Stained Word Window” as “a perfect performance” of idea of splitting between subjects (rather as the structure of T.S. Eliot’s The Waste Land is conventionally interpreted as mimetic of a supposedly fragmented culture). Stefans’s The Dreamlife of Letters is another well-known kinetic work in which words perform their meaning, sometimes in a quite literal way, although often leavened with humor or irony, and there are comparable effects in other of Stefans’s works, such as Concatenation, which seeks to thematize “a nexus between language and violence” and in which the word “explode,” in the phrase “our thoughts explode,” spreads out into a circle, mim icking an explosion (Stefans, The Dreamlife of Letters;” Concatenation). Here is a critic commenting on a print poem by the British poet Barry MacSweeney: the real triumph of these poems is that they “move” the reader—in both senses of the work. Yet that “movement” of the poems, the celerity of the text, resists that static aestheticization of the feeling, the comforting, introspective notion, of having been “moved.” (Sheppard 70) Some works of concrete poetry such as “Extension 2: Soleil” by Ilse and Pierre Garnier (Rothenberg and Joris 309), or many of the poems in Ottar Ormstad’s collection bokstavteppekatalogen, use the qualities of font, print and layout to create an illusion of letters moving “in front of our eyes” (in fact, presumably, “behind” our eyes, since the “movement” must be the product of our brain’s processing of visual stimuli: these works would have no effect on an automated motion detector). Although one could not say that affect is absent from such works, a common response to them might be that they are striking, intriguing or disconcerting rather than especially “moving” in an emotional sense. If we then turn to a digital kinetic work such as La série des U [The Set of

158

Andrew Michael Roberts | Digital Literature

U], by Philippe Bootz and Marcel Frémiot, the letters on the screen literally or actually move. What does this mean for the “movement” of thought, feeling or rhythm within the work (the commonest literary critical meaning of “movement”), and what might it mean for the affective power of the work? One way of making the latter connection is offered by Mark Hansen’s redefinition of “affectivity” (based on Bergson): “the capacity of the body to experience itself as “more than itself” and thus to deploy its sensorimotor power to create the unpredictable, the experimental, the new” (7). The emotional seems to have been rather left behind by this definition, however. The definition is very much in accord with Philippe Bootz’s expressed intentions for The Set of U, which invoke the concept of the “metareader”: when we take into account that rereading is a part of the reading of such kinds of works, we can conclude that this work is interactive, not by managing input devices, but through meta-rules. Finally, meta-rules are not “technical rules”, but the expression of a complex aesthetical intention that lies in programming and can only be perceived by looking at the program. This intentionality is not addressed to the reader, but to a meta-reader: reading is a limited activity that is unable to give a complete knowing of the work. (“about the work”) In certain ways the “obvious” next step in this progression is the move “beyond the screen”—for the body to “deploy its sensorimotor power” by moving and touching as part of aesthetic experience. (Perhaps one should rather say “around the screen,” since locative, immersive and multi-media works often still include screens). The dynamic effects common in kinetic screen-based digital works can migrate relatively easily to an immersive environment, as in Noah Wardrip-Fruin et al.’s Cave work Screen. Here, as Hayles describes it, the theme of memory loss is “enacted in a startlingly literal way when words suddenly begin peeling away from the walls and moving in the three-dimensional space” and, despite the user’s attempts to bat them back into place, end up “jumbled on the floor” (Electronic Literature 13). Similarly, Camille Utterback and Romy Achituv’s Text Rain has evident continuities with concrete poetry, such as “rain” by Seiichi Nĩkuni (Rothenberg and Joris 308). Interactive Poetry Garden: Stream of Consciousness, by Tom White and David Small, goes one better by literalizing a long-standing concept in criticism of print fiction (stream of consciousness as a narrative technique) to create as a sensuously-appealing work. Here the element of capturing words has some affinities with, say, Jim Andrews’s screen-based Arteroids, but with a very different mood and aesthetic from the latter’s video “shoot-up game” format.

159

Andrew Michael Roberts | Digital Literature

Digital works with cinematic affinities, such as the rapid text-sequence Flash works of Young-Hae Chang Heavy Industries, use movement in rather different ways.3 Steven Connor invokes, as a metaphor for the “contingent flow of temporality” in postmodern poetry, Ezra Pound’s idea of the “periplus,” a term which he used “to describe the technique of his Cantos.” The periplus is “a map which projects the stages of a journey as they succeed each other for the traveller,” rather like a Satellite Navigation system (Connor 125), and provides a further link between Dakota and Pound’s Cantos, a work which Dakota references thematically (cf. Pressman). Dakota is both a form of “road poem” (on an analogy with the road movie genre), and deploys a temporality which discloses sequentially and resists the perception of transcendent pattern viewed from a distance. Hayles references a discussion by Pressman of Bill Brown’s “Readie” machine of the 1920s as a precursor of Young-Hae Chang Heavy Industries, adding that “Brown imagined that reading could be brought up to speed, so to speak, by displaying it as a linear stream of words flashing by, much as a highway unfolds at night” (Hayles, Electronic Literature 125). Here the resemblance to Satellite Navigation, and the link to “beyond screen” locative technologies is apparent. In this way Dakota can be seen as pointing the way towards locative works such as Stefan Schemat’s Augmented Reality Fictions. Inanimate Alice, by Kate Pullinger and babel, does something comparable by rather different means, with its themes of travel and the interplay between the domestic sphere, global business and technology, refracted through the Baxi on which Alice plays games but also maps locations, combined with the filmic sections of rapid, often blurred moving images. In these respects Inanimate Alice approaches a simulation of a locative work, with a perspective recalling the Poundian periplus, and thus looks both back to modernism and forward to “beyond screen” works. Works such as Jean-Pierre Balpe’s Fictions d’Issy extend the modernist theme of the flâneur into the socio-geography of the city (cf. Balpe). The role of mobile personal technology, thematized and simulated in Inanimate Alice, is actualized in Fictions d’Issy, where readers make use of their mobile phones to interact with installed screens in a city environment, while sniff_jazzbox by AND-OR and Johannes Auer creates music via Nintendos or iPhones from ambient wireless networks, making sound a specific privileged medium of locative interaction.

5

Depth

The concept of depth has been pervasive in much Western philosophy and literature in many different guises, from philosophical depth and “depth psychology” to the hermeneutics of deep meaning and the reactive postmodern

160

Andrew Michael Roberts | Digital Literature

critique in the name of surfaces. Depth is one of those “orientational metaphors” which “organizes a whole system of concepts with respect to one another” (Lakoff and Johnson 14). In literary critical discourse, depth is frequently a metaphor used to ascribe both complexity (layers of hidden meaning) and value (profundity or insight). Digital literature can enact or perform depth through the simulation of three-dimensional movement of words and letters, notable examples being found in Eduardo Kac’s “Holopoetry.” This is, of course, a simulation on a more or less flat screen: it seem closer to literal depth than the “deep” meaning of a printed poem, but remains a simulation. It also draws on metaphorical and imagined forms of “depth,” including metaphors arising from the hardware on which digital works run as well as the layers of encoding which are necessary for them to be created and to operate. (Of course a sceptic might regard simulated three-dimensional movement of the material forms of words as a “shallow” or superficial form of depth in comparison to “depth” of meaning.) John Cayley, probably more than any other digital author, has explored ideas of depth and surface. I have discussed elsewhere responses in a range of digital poetry to the various depth models of Western cultural thought as well as the postmodern idea of depthlessness (Schaffner and Roberts). In his piece “Writing on Complex Surfaces,” Cayley suggests a reciprocity between the metaphorical “depth” of authorial presence in print literature and its tendency to negate material depth: Addressed to writing, “depth” is rarely conceived as material depth. Depth is even more abstracted when it is applied, critically, metaphorically, to writing than when, for example, it is applied to painting. Generally speaking, rather than any aspect of material depth, it signifies access, through a symbolically marked but dimensionless and transparent surface (paradoxically, it is the marks that render the surface transparent) to the interiority of a remote author, an author whose very authority is guaranteed by institutions of publication which are, in a circular, bootstrap logic, predicated on flatland delivery, with all traditionally perceived material characteristics of language intact, or rather, collapsed, resting, flattened, on paper-thin media, ready to be read and passed through. (“Writing on Complex Surfaces”) He identifies too the engagement with material depth in some radical poetic traditions: A related argument, that practices of writing are constrained by actual physical media—paper and the book—is often resisted by poetic

161

Andrew Michael Roberts | Digital Literature

writers, those, that is, who produce work which challenges flatland authority and engages with language-as-material . . . these “old” media allow many ways to indicate, if not perform, a text’s material depth, its temporality, its constitution as process. Books can be programs. Because deep, time-based poetic practice has a history, including a tradition of serious intellectual exposition and commentary, poetic practitioners often also demonstrate their suspicion of so-called “new” media. They resist work in new media which reads as “thin” despite its explicitly, overtly complex surface; and they resist a potential future of overdetermination by unproven writing machines. In agreement with many active poets, I do not and would not argue that print-based textuality is incapable of delivering writing with a complex surface, but I do say that in so far as this is achieved it is achieved as concept, in the familiar and comfortable realm of literary virtuality, in the “mind” and in the “imagination,” but not in the material experience of the text and its language. (“Writing on Complex Surfaces”) Here he points towards the “virtual” or metaphorical nature of depth in print literature traditions which nevertheless seek to engage with the materiality of writing, and so, potentially, with material depth. This situation both implies the dynamic towards performed or literal depth, and, as Cayley points out, raises the suspicions of (philosophically) “thin” literal materiality. The trajectory of Cayley’s own work stands as a notably clear example of the dynamic I am dis cussing, in part because of the consistency with which he has directed his creative attention to particular themes, tropes and techniques. As compared with what he calls “flatland” literature, Cayley’s screen works such as overboard and translation create a strong impression of material depth, reinforced by theme and allusion. One reason for his interest in the rhetoric of depth and surface is apparent from his claim that:

the iterative transliteral morphs between related texts—texts that might be seen, for example, as rewrites in differing styles—will reveal abstracted underlying structures supporting and articulating the “higher-level” relationships between the texts. (Cayley, “OVERBOARD”) Thus the sense of depth has a conceptual as well as a thematic function, suggesting ideas of “deep” underlying linguistic structures (especially in translation), as well as mythological and symbolic ideas associated with water, drowning and rebirth (in overboard).

162

Andrew Michael Roberts | Digital Literature

To describe the effects of overboard and translation as a more “literal” evocation of depth would be appropriate in one sense, given Cayley’s claiming of that term for an art focused on the materiality of the letter. But “literalization” only partly conveys the difference here. The depth is both more “actual” and more “virtual” than the depth of a print poem which has different “layers” of meaning.4 What Hansen terms “the body-brain’s capacity to generate impressions of ‘virtual totality’ through selective filtering of information” (13) allows us to interpret the disappearance of letters on a screen as a form of submersion. The effect is to foreground materiality, because we are invited to interpret (attribute meaning to) the changes in the material status of the signifier (its visibility/invisibility or presence/absence), in a line of continuity with concrete poetry, though with a time-based dimension. At the same time, however, the effect is to foreground the immateriality of the “flickering signifier,” since it is precisely this immateriality that enables such a fluid appearance and disappearance. Relevant here is Cayley’s comment in the current volume, that Questions concerning materiality and embodiment have been very much to the fore in contemporary analyses and theories of artistic and critical practice. In so far as the trope of immateriality is associated with the realm of computation—along with the cultural production which computation has appropriated through remediation or has, itself, generated—there is a proportional effort to restate and critically reinstate its necessary materiality, an effort that is presumed to have aesthetic traction in critical discourse and is sometimes implicitly or explicitly heralded as part of a return of the repressed. (“The Gravity of the Leaf” 204) The play between materiality and immateriality in Cayley’s work functions as another level of the works’ thematization of presence and absence, surface and depth, the preservation and loss of meaning, visibility and invisibility, readability and unreadability, the representable and the unrepresentable. It is Cayley’s ability to embed these conceptual themes in multi-layered formal procedures that makes his work so aesthetically satisfying. The process is also self-mirroring, since the homology between these sets of philosophical and conceptual binaries produces a thematic multi-layering effect, which itself reflects and is reflected by the “complex surfaces” of the works’ formal processes. Without reducing creative development to a deterministic teleology, it is not hard to see why the creation of works in an immersive 3D audio-visual environment was the next step for Cayley, intensifying both the materiality and the immateriality of the signifier, by introducing a more powerfully-realised third dimension, and bring the reader’s body explicitly into play. On one perspective overboard has only metaphorical or imaginative “depth” itself: when the

163

Andrew Michael Roberts | Digital Literature

words disappear they are not literally “behind” the screen, and this may be one motive for Cayley’s embrace of immersive technology. The ways in which the presence/absence thematics extend into Cave works is suggested by the focus, in Cayley’s chapter in the current volume, on the “other-worldly” quality of aesthetic uses of language (“The Gravity of the Leaf” 200). Interestingly, Cayley seems here to conceptualize his work as resistance to a general cultural “phenomenological and accumulative” force, which he sees as pushing the opposite way to the dynamic I have been describing within digital literature: “an underlying cultural force that draws graphic linguistic materials to the two-dimensional surface” (203). However, he also observes that this “gravity of the leaf” may be about to break down (203), and sees the Cave as contributing to this process.

6

Re-reading and Difference

Re-reading is both an imperative and a source of value in the literary criticism of highly-regarded print literature. To appreciate fully rich and complex novels and poems we need to read them more than once, in a cumulative process of interpretation, with memory enriching each new reading experience. To be worthy of, and stand up to, repeated re-readings is often held to be a criteria of value: if we read a work once and are finished with it, that seems to imply a certain superficiality. Of course, when digital works appear different on each occasion, it is not rereading per se which is made more “literal” in comparison with print literature. Re-reading is almost always literally possible. What is literalized is the idea of the work being different each time it is read. When applied to print literature, this idea might be termed metaphorical, or more simply a loose way of saying that the experience of reading it is different on each occasion. While, as noted, this quality is often seen as a sign of value, one might argue, in accordance with the idea that “novelty” can be a product of repetition, that any work produces a different experience when re-read, although that difference may or may not be of substantial interest.5 Espen Aarseth distinguishes carefully between the experience of reading a literary work, and the “object” which he defines as the “material entity that determines [a literary work] . . . in a way the individual readings (or all of them put together) do not” (46). On this account, the work as a totality presumably includes both the material object and its readings, since Aarseth says that “[w]e need not think of this object as identical to the work” (46). In the case of a print or other nontime-based literary work, the object would then be considered as spatial and the readings as temporal. Aarseth also refers to Ingarden’s schema, in which the literary work is again spatial, while temporality is an attribute of its “con-

164

Andrew Michael Roberts | Digital Literature

cretizations,” or readings. Ingarden raises the possibility of divergence between work and readings, stating that whether “the given concretization can still be considered a concretization of the same work, or whether it then expresses an entirely new work, is a matter that requires a separate, extensive analysis in each concrete instance” (qtd. in Aarseth 45). When a work such as La série des U [The Set of U] or Urbanalities (by babel and escha) changes materially each time it is read, this variability is presumably to be considered a property of the “object” (in Aarseth’s terms) or “work” (in Ingarden’s terms). In Barthes’s terms, it is a figure of the text, not a figure of reading (The Pleasure of the Text, qtd. in Aarseth 47). Yet its function is surely to generate a greater variability in the reading experience. As Aarseth points out, it is possible to read a book in different orders on different occasions, and the same could be said of the lines of a poem. In this instance the variation is determined by the action of the reader, not of the object or work. In the case of works of digital literature which are different each time they are viewed, such reader-driven variability may be prevented, or constrained (in Urbanalities one can review “scenes,” but only having let them run first). But the more subtle sense in which every re-reading is new (corre sponding to Popper’s “novelty”) still operates, because the reader’s memory changes the nature of the experience. So there is an interaction between the changes produced by the work and those produced by the reader. The question is whether these two processes enrich each other or cancel each other out, and this clearly may depend on the specific processes and effects of the work.

7

Interactivity

Interactivity is a contested and problematic term in the creation, analysis and criticism of digital literature. Exaggerated claims for the empowerment of the reader have produced a reaction against trivial or superficial forms of interaction, evident in statements against interactivity by artists such as Young-Hae Chang Heavy Industries, while artists such as Chris Joseph (“babel”), who use it in some works, find it necessary to warn readers of its absence in other (Urbanalities), to prevent futile attempts at mouseovers and clicks.6 The idea that the reader is a creative partner, with an active role in the reading process goes back at least to Romantic ideas of the active role of the imagination in perception, and becomes a literary and theoretical program in Barthes’s concept of the “lisible” text (S/Z ) and in the poetics of 20th century experimental poetry: A poem is an object, but it is also part of the reader’s responses, since he or she must complete it . . . in certain kinds of open work offered by some writers of the British Poetry Revival and Linguistically In-

165

Andrew Michael Roberts | Digital Literature

novative persuasions, who work in deliberate collusion with this fact, this is crucial. (Sheppard 5) Digital literature has the potential to literalize this process, for example by inviting the reader to type in words (e.g., Jim Andrews’s Arteroids or Melinda Rackham’s carrier (becoming symborg). The basic forms of interaction found in digital literature, such as recombining elements, mousing over to reveal varying texts, changing speed and direction of movement and so on, while they can powerfully act out ideas of textual materiality and interpretative contingency, are open to the objection that they replace a rich imaginative interaction with a more banal physical one. Espen Aarseth criticizes spurious “claims that digital technology enables readers to become authors, or at least blurs the (supposedly political) distinction between the two, and that the reader is allowed to create his or her own ‘story’ by ‘interacting’ with ‘the computer’” (14). However, as already noted, the idea of the reader as author is a standard one within more experimental or avant-garde forms of print literature, as well as having become something of a cliché of literary theory in the wake of Roland Barthes’s work in S/Z and “The Death of the Author.” So claims that interactivity in digital literature enables greater creativity on the part of the reader (whether or not justified), could indicate an underlying continuity of theoretical assumption, even if conceptualized in terms of radical change. The move “beyond the screen” extends the scope of interaction, making literal (or “virtually” literal) ideas such as the “space” of reading, and the “presence” of the reader in the text. The active reader becomes the physically mobile reader in both VR and locative processes. Furthermore, some forms of interactivity point “beyond the screen,” not because they are radically interactive, but because they foreground the role of the computer in social and other networks, rather than as a source of a visual display. For example, carrier (becoming symborg), a work which “investigates the fluid boundaries of the body and the self via viral symbiosis in the biological and virtual domains by weaving an intimate love story between the viewer and the hepatitis C virus” (Rackham) incorporates allusion to both online support groups and e-mail communication. Clicking on certain elements of the work will open a new message folder in the user’s e-mail program, apparently to a real address, while elsewhere personal support-group style messages on experiences of living with the hepatitis C virus open. This is a clear example of the way in which the integration of different programs possible with contemporary operating systems and applications can connect (or seem to connect) what is primarily a work of art (with critical and didactic elements), into circuits of communications beyond the aesthetic. Relevant here are Simon Biggs’s comments on “participatory” art:

166

Andrew Michael Roberts | Digital Literature

We can ask whether creativity might be regarded as a form of social interaction rather than an outcome. How might we understand creativity as interaction between people and things, as sets of discursive relations rather than outcomes? Whilst creativity is often perceived as the product of the individual artist, or creative ensemble, it can also be considered an emergent phenomenon of communities, driving change and facilitating individual or ensemble creativity. Creativity can be a performative activity released when engaged through and by a community and understood as a process of interaction. (“littlepig”) The allusion to such ideas in carrier must be seen as in part ironic and transgressive. I take it that part of the point of the work is a somewhat troubling effect in which experience of trauma or distress caused by illness, and the use of the internet in attempts to respond to these, becomes a feature of an aesthetic “technique.”

8

The Body and Embodiment

The invocation of “the body” in critical and literary theory of the 1970s and 1980s became so prevalent that Terry Eagleton was moved to write that “[a] recovery of the body has been one of the most precious achievements of recent radical thought” (7). The intention of the “recovery” was to reintroduce the material and physical into what was seen as an excessively dematerialized Cartesian subjectivity, and excessively idealist theories of meaning. However, there was a tendency for the theorization of “the body” to repeat that which it critiqued: in the process of being inserted into theoretical discourse “the body” often became one more abstract counter, and the tactic of pluralizing (as “bodies”) was of limited help in this respect. Hayles notes this problem in relation to Foucault [t]he abstraction of the Panopticon . . . into a system of disciplines . . . diverts attention away from how actual bodies, in their cultural and physical specificities, impose, incorporate, and resist incorporation of the material practices he describes. (How We Became Posthuman 194) Hayles prefers the plurality of “bodies” to “body,” endorsing Elizabeth Grosz’s comment that “there is no body as such, there are only bodies” (Grosz 19), before going on to propose a concept of “embodiment” distinct from “the body”:

167

Andrew Michael Roberts | Digital Literature

Embodiment differs from the concept of the body in that the body is always normative relative to some set of criteria. . . . In contrast to the body, embodiment is contextual, enmeshed within the specifics of place, time, physiology, and culture. . . . Embodiment is . . . inherently destabilizing with respect to the body.” (Hayles, How We Became Posthuman 196-197) As already noted, Mark Johnson shows that print literature encodes embodiment in metaphor. Such literature may represent the bodily in multiple ways, and it evokes bodily responses, though these are not readily open to scrutiny: the reader’s responses to rhythm, the potential sound of words, represented speech, tension, anticipation, pleasure, and so on may include micro-responses of the body (such as subvocalization). Screen-based digital literature brings the reader’s body more explicitly into play through mouse movements and key strokes, and through intermediality brings into play hearing and (to a limited extent) touch directly, rather than via imagination: the reader’s body is engaged in the process in ways distinct from turning the page and reading text. When digital literature engages the individual body of the reader in physical processes, there is again a certain appropriation of control by the machine: the freedom to interact is also the demand to interact within certain constraints. Sound, a key part of most digital literature, does not emanate from a screen; it surrounds and passes through the reader’s body. This includes sound derived from the traditions of sound poetry, as, for example, in Jim Andrews’s Nio and Arteroids. Many works of digital poetry are viewed on the screen, but use strategies which invoke a “beyond” of the screen by using the limitations of human spatial and temporal perception to create an experience which extends in space and time through, round and after the screen moment. In the instance of works which vary on each viewing (already discussed), the cognitive processes involved in watching them generate an “extended mind” effect, so that the true location of the “work” becomes a series of experiences, memories and perceptions, distributed across space by eye-movements and across time by memory and repetition. Works such as Young-Hae Chang Heavy Industries’s Dakota or Jim Andrews’s Stir Fry Texts present disrupted or challenging reading experiences through temporal-spatial dislocations: in the former case the successive words displace each other on the screen at variable speeds, with other effects of movement such as juddery text; in the latter case, moving the cursor over a section such as “Blue Hyacinth” produces, depending on mouse speed, unreadable flickering between alternate phrases, or changing combinations of fragments within a narrative (combining elements of four narratives) which

168

Andrew Michael Roberts | Digital Literature

defy sequential reading. Such effects produce new literary experiences of relations between space and time. N. Katherine Hayles comments in relation to the work of Young-Hae Chang Heavy Industries, in particular Nippon the text’s movement and its interpenetration by sound . . . [creates] a more complex and multimodal production in which embodied response, machine pacing, and transnational semiotics, along with the associated spatialization of temporality, all contribute to construct the relation between text, body, and machine. (Electronic Literature 127) Her phrase “embodied response” captures well the “beyond screen” aspect of this work which I wish to highlight: by making us aware of our embodied perceptual processes, through the difficulty of processing and disruption of normal reading. Young-Hae Chang Heavy Industries’s work precludes the habit of reading in which the page or screen functions as a perceived “external” location, and makes us feel the way in which viewing/reading is a bodily and cognitive process. In terms of “extended mind” theory (cf. Clark and Chalmers), one might say that the underlying programming of the work forms a dysfunctional extension to our cognitive processes. Festivals and conferences, such as the Digital Arts and Culture Conference (initiated in Bergen, Norway (1998); annual/biannual) and E-Poetry (initiated in Buffalo, NY (2001); biannual) have become crucial in the development of digital literature (practice, theory and criticism), and such events usually include a significant element of performance, in which one or more screens form only one part of the performed work, along with music, reading, sound, objects and the performer’s bodily presence.7 Examples would be Aya N. Karpinska’s performed work at E-Poetry 2007 in Paris, or Jörg Piringer’s visual and sound poetry at E-Poetry London in 2005. As Simon Biggs has commented Digital poetry is more and more associating texts, images and sound in a kind of combination of sound poetry, visual poetry, hypermedia poetry and generative poetry in new ways and forms. It is online, off line, in performances, etc. and uses all possible media and forms. (“Poetry in Paris”) There is, as often noted, a certain tendency towards dematerialization to be found in digitally-mediated modes of communication, from e-mail and mobile phones, which reduce the significance of physical location, to virtual worlds such as Second Life, in which code substitutes for the physical body. The tendency to performance of digital literature could be seen as a countervailing

169

Andrew Michael Roberts | Digital Literature

tendency, while drawing on traditions of performance poetry and “happenings.” The presence of the performer’s body is significant and sometimes foregrounded in these performances, as is the physical voice in some cases. The bodily presence of the audience members, and their multi-sensory awareness of both work and venue, are also crucial to the aesthetic experience. The restaging or development of the practices and imperatives of modernism is again in evidence here. As Tim Armstrong has written: The desire to embody and perform is an important part of the story . . . a corporeal equivalent of Modernism’s slogan “No ideas but in things.” In the modern period, the body is re-energized, re-formed, subject to new modes of production, representation, and commodification. (2) In such respects modernism presaged the alleged “co-evolution” of the human body and machines, which leads some, such as Hayles, to argue for “the catalyzing effects of media on the body, on subjectivity, and on the structures of knowledge associated with them” (Electronic Literature 119). From this perspective, virtual worlds and digital communications may be seen as forms of embodiment, or as forms of “extended mind” rather than as a substitution for the body.

9

Impersonality

Impersonality, anonymity and “the death of the author” form a cluster of ideas with a long history in print literary and criticism (and in this respect resemble the ideas around the (inter)active reader, with which the “death” of the author is sometimes correlated). This history runs from Flaubert’s detached and invisible author, via the personae and supposed impersonality of modernist poets such as Pound and Eliot, to Barthes literary theory and the “destabilisation of the lyrical ego” in American Language Poetry and British Linguistically Innovative Poetry. Digital media give it a new impetus and a new dynamic, though here “literalization” is not quite the right term, and the transformation occurs as much at the level of the system as at the level of the individual work. Although anonymously-authored books of course exist, impersonality in print literature has been primarily a matter of rhetorical structures and style, the presence or absence of “voice” (a very slippery term), modes of address to the reader, the deployment of pronouns, the way in which form does or does not image a “stable ego” or “originary subject.” Multi-mediality and the Internet have greatly weakened the general sense of authorship (of words and images as

170

Andrew Michael Roberts | Digital Literature

defined by their “origin” in, or attribution to, a particular person or persons, irrespective of the particular instantiation). This is partly a matter of framing (most books have the author’s name prominently displayed; many web pages don’t) and of economics (income streams from web sites, where they exist, are less linked to authorial copyright claims, while much digital literature is distributed on a creative commons basis). As Young-Hae Chang Heavy Industries comment: It’s pretty obvious that the “tone” or “voice” of Internet literature is more distant and difficult to “locate” than traditional writing. Mere book packaging tells a lot about the book and the author; browser packaging is generic. Internet writers can either see this as a problem or welcome it as a relief from the critical fashion of reading biography into every aspect of literature. (“Distance, Homelessness”) It is also a function of the relatively seamless joins, facilitated by shared digitality, between media, between different work, and between electronic literary works and other forms not traditionally “authored” to the same degree, such as advertisement, public documents, corporate statements, catalogues, maps, etc. Again, we can see here an extension of the process in “beyond screen” works, where there is an even greater distance from notions of voice and intentionality, as the work becomes more like a process or the framing of an activity rather than a created object.

10

Conclusion

The move “beyond the screen” may support Mark Hansen’s idea of “a new, more or less ubiquitous form of aura: the aura that belongs indelibly to this singular actualization of data in embodied experience” (3), as well as Hayles’s idea of embodiment as “contextual” and “enmeshed with specifics” (How We Became Posthuman 196). Works in locative and immersive media are contextspecific on an individualized basis, just as performed works are frequently specific to a social and geographical context. The cost of immersive technology makes works such as the Cave works highly location-specific, at present to the institutional space of resource-rich universities (although this may change soon as the technology develops).8 These processes, of literalization, performance, enactment (whatever term one applies to them) represent a dynamic tending towards a reader/viewer (though perhaps those terms are no longer adequate) who is physically mobile in a dynamic aesthetic environment, which can be immersive simulations, as in the Cave, or “real,” as in locative works which promote the city as an aesthetic environment. In many ways, then, the screen

171

Andrew Michael Roberts | Digital Literature

seems only a staging-post, or an element, in a process which returns art to embodiment and the “real,” even if forms of technology built around the screen are rapidly transforming the environment, reader/viewer and even the “real.”9

Notes 1

Cayley’s view on this issue has evolved since “Literal Art” was written, as described in the following comment: “I’ve moved more and more towards a position in which the ‘language’ or ‘the literary’ of ‘language art’ or ‘literary art’ is not material as such, although its practices and embodiments always are, indeed, must be … By ‘literal art’ I also mean ‘art with letters (and their singular materiality)’” (Cayley, private e-mail, 31 Oct. 2009). Cayley’s more recent thinking on materiality is explored in his chapter in the present volume.

2

It should be noted that the common association between the digital and the virtual has been strongly challenged. Brian Massumi has argued that “[d]igital technologies have a remarkably weak connection to the virtual. . . . Equating the digital with the virtual reduces the apparitional to the artificial” (qtd. in Hansen 161).

3

I am grateful to Kim Knowles for pointing out to me the affinities between Young-Hae Chang Heavy Industries’s work and fluxus films such as Paul Sharits’s Word Movie (1966).

4

This seeming paradox may merely illustrate Hansen’s claim that “the virtual is a quality of the human” (50).

5

Popper notes that “the life-history of an organism is partially conditioned by past events. If such events are repeated, they lose, for the experiencing organism, their character of newness, and become tinged by habit. Yet this is precisely why the experience of the repeated event is not the same as the experience of the original event—why the experience of a repetition is new” (Popper 9).

6

“It’s a simple technique that shuns interactivity” (Young-Hae Chang Heavy Industries, “Distance, Homelessness”); “This tale is not interactive —there is nothing you have to find or click” (babel and escha, Urbanalities).

7

Cf. and .

8

Cf. Hayles, Electronic Literature 14.

172

Andrew Michael Roberts | Digital Literature

9

Transformation of the viewer is of course central to the theorization of the posthuman.

Works Cited Aarseth, Espen J. Cybertext: Perspectives on Ergodic Literature. Baltimore: Johns Hopkins UP, 1997. AND-OR (René Bauer and Beat Suter) with Johannes Auer. sniff jazzbox. 30 Oct. 2009 . Andrews, Jim. Nio. 30 Oct. 2009 . ———. Arteroids. 30 Oct. 2009 . ———. Stir Fry Texts. 30 Oct. 2009 . Armstrong, Tim. Modernism, Technology and the Body: A Cultural Study. Cambridge: Cambridge UP, 1998. Auer, Johannes. worm applepie for döhl. 2000. 30 Oct. 2009 . Balpe, Jean-Pierre. “A Town as a Book: An Interactive and Generative Literary Installation in Urban Space.” In this book. 331-344. babel, and escha. Urbanalities. Electronic Literature Collection. Ed. N. Katherine Hayles, Nick Montfort, Scott Rettberg, and Stephanie Strickland. 2006. Vol. 1. 30 Oct. 2009 . Bachleitner, Norbert. “The Virtual Muse: Forms and Theory of Digital Poetry.” Theory into Poetry: New Approaches to the Lyric. Ed. Eva Müller-Zettelmann and Margarete Rubik. Amsterdam: Rodopi, 2005. 303-344. Barthes, Roland. The Pleasure of the Text. Trans. Richard Miller. London: Cape, 1976. ———. S/Z. Trans. Richard Miller. Oxford: Blackwell, 1990. Baudelaire, Charles. Baudelaire: Selected Writings of Art and Artists. Trans. P.E. Charvet. Harmondsworth: Penguin, 1972. Beyond the Screen: Transformations of Literary Structures, Interfaces and Genres. Conference program. 2008. 30 Oct. 2009 .

173

Andrew Michael Roberts | Digital Literature

Biggs, Simon. “littlepig: Artworks and Writings by Simon Biggs.” 30 Oct. 2009 . ———. “Poetry in Paris: 2 Reviews”. networked_performance. 2007. 30 Oct. 2009 . Bootz, Philippe. “about the work.” Electronic Literature Collection. Ed. N. Katherine Hayles, Nick Montfort, Scott Rettberg, and Stephanie Strickland. 2006. Vol. 1. 30 Oct. 2009 . Bootz, Philippe, and Marcel Frémiot. La série des U [The Set of U]. Electronic Literature Collection. Ed. N. Katherine Hayles, Nick Montfort, Scott Rettberg, and Stephanie Strickland. 2006. Vol. 1. 30 Oct. 2009 . Bowie, Andrew. Aesthetics and Subjectivity: From Kant to Nietzsche. Manchester: Manchester UP, 1990. Bradford, Richard. The Look of It: A Theory of Visual Form in English Poetry. Cork: Cork UP, 1993. Calinescu, Matei. Five Faces of Modernity: Modernism, Avant-Garde, Decadence, Kitsch, Postmodernism. Durham: Duke UP, 1987. Cayley, John. “Literal Art: Neither Lines nor Pixels but Letters.” First Person: New Media as Story, Performance and Game. Ed. Noah Wardrip-Fruin and Pat Harrigan. Cambridge, MA: MIT P, 2004. 208-217. ———. “Writing on Complex Surfaces.” Dichtung Digital 7.35 (2005). 30 Oct. 2009 . ———. “OVERBOARD: An Example of Ambient Time-Based Poetics in Digital Art.” Dichtung Digital 6.32 (2004). 30 Oct. 2009 . ———. E-Mail to the author. 31 Oct. 2009. ———. “The Gravity of the Leaf: Phenomenologies of Literary Inscription in Media-Constituted Diegetic Worlds.” In this book. 199-226. ———. overboard. 2004- . 30 Oct. 2009 . ———. translation. 2004- . 30 Oct. 2009 . Clark, Andy, and David Chalmers. “The Extended Mind.” Analysis 58 (1998). 10-23.

174

Andrew Michael Roberts | Digital Literature

Connor, Steven. Postmodernist Culture: An Introduction to Theories of the Contemporary. 2nd ed. Oxford: Blackwell, 1997. Eagleton, Terry. The Ideology of the Aesthetic. Oxford: Blackwell, 1990. Grosz, Elizabeth. Volatile Bodies: Toward a Corporeal Feminism. Bloomington: Indiana UP, 1994. Hansen, Mark B. N. New Philosophy for New Media. Cambridge, MA: MIT P, 2004. Ingarden, Roman. The Literary Work of Art: An Investigation on the Borderlines of Ontology, Logic, and Theory of Literature. Evanston: Northwestern UP, 1973. Hayles, N. Katherine. How We Became Posthuman: Virtual Bodies in Cybernetics, Literature, and Informatics. Chicago: U of Chicago P, 1999. ———. Electronic Literature: New Horizons for the Literary. Notre Dame: U of Notre Dame P, 2008. Johnson, Mark. The Body in the Mind: The Bodily Basis of Meaning, Imagination, and Reason. Chicago: U of Chicago P, 1987. Kendall, Robert. Faith. 2002. 30 Oct. 2009 . Lakoff, George, and Mark Johnson. Metaphors We Live By. Chicago: U of Chicago P, 1980. Larsen, Deena. “Stained Word Window.” 1999. 30 Oct. 2009. . Massumi, Brian. “Line Parable for the Virtual.” The Virtual Dimension: Architecture, Representation, and Crash Culture. Ed. John Beckman. New York: Princeton Architectural P, 1998. Nĩkuni, Seiichi. “Rain.” Poems for the Millennium: The University of California Press Anthology of Modern and Postmodern Poetry. Vol. 2. Berkeley: U of California P, 1998. Ormstad, Ottar. bokstavteppekatalogen. Oslo: Galleri Briskeby, 2007. Popper, Karl. The Poverty of Historicism. 2nd ed. London: Routledge and Kegan, 1961. Pullinger, Kate, and babel. Inanimate Alice. Electronic Literature Collection. Ed. N. Katherine Hayles, Nick Montfort, Scott Rettberg, and Stephanie Strickland. 2006. Vol. 1. 30 Oct. 2009 .

175

Andrew Michael Roberts | Digital Literature

Pressman, Jessica. “The Strategy of Digital Modernism: Young-Hae Chang Heavy Industries’s Dakota.” MFS 54.2 (2008): 302-26. Rackham, Melinda, and Damien Everett. carrier (becoming symborg). Electronic Literature Collection. Ed. N. Katherine Hayles, Nick Montfort, Scott Rettberg, and Stephanie Strickland. 2006. Vol. 1. 30 Oct. 2009 . Raley, Rita. “Editor’s Introduction: Writing 3D.” The Iowa Review Web. 2006. 30 Oct. 2009 . Rothenberg, Jerome and Pierre Joris, eds. Poems for the Millennium: The University of California Press Anthology of Modern and Postmodern Poetry. Vol. 2. Berkeley: U of California P, 1998. Schäfer, Jörgen, and Peter Gendolla. “Introduction.” In this book. 11-19. Schaffner, Anna Katharina. “From Concrete to Digital: The Reconceptualization of Poetic Space.” In this book. 179-197. Schaffner, Anna Katharina, and Andrew M. Roberts “Rhetorics of Depth and Surface in Digital Poetry.” Ri.L.Un.E.: Review of Literatures of the European Union. 5 (2006). 3 Nov. 2009 . Sheppard, Robert. The Poetry of Saying: British Poetry and its Discontents, 1950-2000. Liverpool: Liverpool UP, 2000. Stefans, Brian K. Concatenation. Electronic Literature Collection. Ed. N. Katherine Hayles, Nick Montfort, Scott Rettberg, and Stephanie Strickland. 2006. Vol. 1. 30 Oct. 2009 . ———. The Dreamlife of Letters. 2000. Electronic Literature Collection. Ed. N. Katherine Hayles, Nick Montfort, Scott Rettberg, and Stephanie Strickland. 2006. Vol. 1. 30 Oct. 2009 . Wardrip-Fruin, Noah, et al. Screen. 2002- . 30 Oct. 2009 . White, Tom, and David Small. Stream of Consciousness: An Interactive Poetic Garden. Installation. 1997. 30 Oct. 2009 .

176

Andrew Michael Roberts | Digital Literature

Young-Hae Chang Heavy Industries. “Distance, Homelessness, Anonymity, And Insignificance.” Interview with Thom Swiss. 2002. 30 Oct. 2009 . ———. Dakota. 2002. 30 Oct. 2009 . ———. Nippon. n.d. 30 Oct. 2009 .

177

Anna Katharina Schaffner

From Concrete to Digital The Reconceptualization of Poetic Space 1

Digital Signifiers and the Concrete Trajectory

It has become a commonplace in the digital discourse to assign digital poetry to the wider trajectory of avant-garde poetry, and to consider it as a third stage, a continuation and further development of a tradition which originated in the orbit of the historical and the neo-avant-garde. Friedrich W. Block; Roberto Simanowski; Christiane Heibach; and Loss P. Glazier, amongst others, point out that most vital concerns of digital poetry can be traced back to its historical predecessors.1 Frequently, the key concepts shared by historical avant-garde, concrete and digital poetry are identified as the reflection upon the language material, the transgression of genre boundaries, and a concern with multilinearity, the exploration of spatial structures, movement and interactivity. However, most of the claims emphasizing the continuity between the different historical stages of the experimental tradition remain rather general and abstract. In this essay I will explore the relationship between concrete and digital poetry more closely by analyzing how two of the main concerns of the concrete poets—the poetics of space and the exploration of the concrete materiality of the medium—translate into the digital domain. For my comparative analysis I will focus on digital works which share vital features with concrete poetry on the following levels: works which operate conceptually with semanticized space, works which explore the “verbivocovisual” qualities of the letter material, and works which focus on structural relationships between linguistic elements and suppress or reduce linear syntactic links in favor of an exploration of multiple dynamic structures. Finally, I will explore works which represent digital signs and symbols in a concrete, non-representational fashion. The concern with the poetics of space and the parameters of surface is arguably one of the main features of concrete poetry. Values such as the topographical position of the signifier material on the page, structural relationships between the linguistic elements and their spatial interaction, and distance, density and size of the letter material, gain conceptual and semantic significance. The German concrete poet Franz Mon, one of the most vocal prophets of the importance of surface, advocates the creative exploitation of the spatial values of the page in various essays. In concrete poetry, he argues, the functions of

179

Anna Katharina Schaffner | From Concrete to Digital

surface replace the functions of grammar and open up new avenues both for poetry and thought. The relationships between spatially arranged words are not fixed and unambiguously predetermined like those of words firmly embedded in syntactical hierarchies, but are open and flexible and subject to continual redefinition during the process of reception. The position of the textual elements on the page, the distance between them and the density of the textual field all acquire potential semantic significance, and serve as extensions of the conventional means of structuring a poem (Mon 170). They become an integral part of the semiotic repertoire, and introduce additional particles and tools of expression. The conceptual deployment of surface values thus constitutes a novel way of charging language with meaning, and allows for the expression of what cannot be expressed within the boundaries of existing grammatical frameworks. The poets of the Brazilian Noigandres group too explicitly propagate new ways of exploring poetic space: “Concrete poetry begins by being aware of graphic space as structural agent,” the group writes in their “Pilot Plan” from 1958 (de Campos et al. 71). Moreover, concrete poetry, Augusto de Campos states, not only represents space, but acts upon it, “proportioning new spatiotemporal modes of apprehension of the text by the reader” (de Campos and Greene). In the “Pilot Plan,” the Noigandres poets explicitly pay tribute to Stéphane Mallarmé, the pioneer of the conceptual, abstract use of poetic space. Already in 1897, Mallarmé urged the readers of Un Coup de dés [A Throw of the Dice] to read above all the blanks. The blanks, he writes in the preface to his poem, “assume importance and are what is immediately most striking” (121). The reading process is spaced out, the paper intervenes, and the spatial arrangement of the free verse prose-poem intimates the movement of thought by means of a “prismatic subdivision of the Idea” (121). The concrete poets tied onto Mallarmé’s innovation and revolutionized spatial conventions by turning space into an integral component of the poem with semantic significance. The flat, two-dimensional surface of the page, however, is fundamentally redefined on the computer screen once again, for the poetic space of the screen is radically different from that of the page on numerous levels. Firstly, it is kinetic and interactive: letters can move and migrate, positions of letters and words are no longer fixed and static, but in flux and transient, they are no longer predetermined but potentially open for creative interventions. mIEKAL aND’s work after emmett (1998) is an homage to Emmett Williams and a reflection upon the ancestry of the concrete poets and the position of digital poets in this lineage. It evokes Williams’ poem The voy age from 1975, which consists of 100 word squares decreasing in size as the poem advances. aND’s digital poem displays fifty-three screens featuring a three-bythree grid of nine letters or punctuation marks each. Each single character in

180

Anna Katharina Schaffner | From Concrete to Digital

those word squares changes typeface continually, switching through a sequence of five to eight different fonts. As a result, the letters seem to dance: they swell and shrink, shimmer and flicker, bloat and shrivel, twitch and shake. They seem to move, to pulsate, palpitate, and a sense of motion and dynamics is evoked. “Eyevoyage,” the first trio of syllables, emphasizes that this piece is appealing primarily to the sense of vision. Here, the concern of concrete poetry with visual gestalt, typography, geometrical word patterns and conceptual movement is transferred into another medium and enriched by a new feature: actual movement, characters in flux—dynamic signs in constant metamorphosis. Ana M. Uribe’s series Anipoems (1997-2003) is a striking example of the change of the conventional function of signs: here, letters fulfil the tasks of images. The graphic gestalt of the letters, their shapes and forms and the visual associations they trigger, are the crucial constituents of meaning. Their dominant function is undermined: Uribe deploys letters iconically rather than symbolically, a strategy which lies also at the heart of Ladislav Novák’s “individualista” (1959-1963), Ronaldo Azeredo’s “velocidade” (1957), and Mon’s “fallen” (1966). The letters perform, and act, and twitch and flicker. They mimic the shape of animals; a “P” exercises and stretches its leg and thus transforms into an “R”; an “i” has a headache which makes its dot rotate furiously. Uribe’s letters are animated linguistic signs which perform an act: program code is used to inscribe a concrete behavior into the textual system. Neil Hennessey’s Paddle is another example of a playful exploration of the iconic dimension of letters, and his piece vowheels 2, a rotating, circular concrete poem, is reminiscent of Ferdinand Kriwet’s “Rundscheiben.” In Uribe’s, Jim Andrews’ and many other digital works, words are equipped with a performative dimension: letters are put on scene like actors, words end up doing something, like floating around, exploding, drifting off the scene, dancing, exercising, changing their size or color etc. Operative, effective program codes, as John Cayley points out, “instantiate a genuinely ‘performative’ textuality, a textuality which ‘does’ something, which alters the behaviour of a system” (293). This also introduces the issue of time: speed and duration of reception can now be programmed, temporal structures can be inscribed into the work and the reception process can thus be carefully stage-managed in advance. The French poet Julien d’Abrigeon equips concrete poetry with an interactive dimension: horde d’ordre et de horreur (2002) allows the user to alter the position of words and thus to reorganize the verbal stock on screen by means of clicking and dragging. In his do-it-yourself piece, the user can construct new spatial and semantic relationships between the given linguistic elements: although this is the case conceptually in concrete poetry too, where different

181

Anna Katharina Schaffner | From Concrete to Digital

combinatory constellations need to be rehearsed mentally, they can be made explicit and visible by actual movement of the linguistic elements here. In Giselle Beiguelman’s recycled (2001), a concern with multilinearity, movement and interactivity becomes manifest: the letters contained in the title word float over the yellow screen, and can be deleted by pointing the arrow symbol at them. The user can determine the movement of the migrating letter material with the movement of his or her mouse. Moreover, Beiguelman recuperates another avant-garde concern here: she only uses existing pages and junks of code, which she equips with new functions and parameters. The title is thus programmatic: recycled is a digital ready-made. In Jim Andrews’ Arteroids (2002), interactivity is taken a step further: here, the boundaries between poem and game are collapsed. The user can navigate an “id-entity” word over the screen, and has to shoot fragmented poetic sequences which float randomly into the field of vision. The poetic flotsam and jetsam descend upon the user’s “id-entity” like asteroids, and, when shot, they explode in circular sprays of atomized letter material underlined by a distorted soundtrack. If it gets hit by antagonistic text elements, the “id-entity” word explodes in turn. The text that glides into the screen can be edited and changed, its speed and color can be altered and it can be shot and destroyed or allowed to keep on drifting across the screen. Arteroids appropriates and mocks the rhetoric of “shoot-em-up” games, and fuses “written, visual, and sound poetry, visual art, computer game conventions” (Andrews, “The Battle”) and generative audio forms into a new hybrid synthesis. “Arteroids is about cracking language open,” Andrews writes (“The Battle”). He thus places himself firmly into the tradition of the avant-garde poets, who have always worked under the aegis of venturing forth into the very heart of language, aiming to uncover the arbitrary and material nature of signs and the codes that govern their usage by means of taking language apart on different levels of linguistic organisation (cf. Schaffner). Moreover, this hybrid between poem and game forces the recipient into activity, and thus evokes the oft-cited “game activity” that Eugen Gomringer has defined as another important feature of concrete poetry. The concrete constellation, Gomringer argues (15; 61), offers a fixed set of parameters, within which the reader is asked to playfully create meaning by combining and relating the given elements in a creative fashion. The second way in which signs on the screen differ from those in print is that they can be present in all their physical aggregates at once, exposing their visual, acoustic and semantic dimension simultaneously. The advantages of spoken and of written language can thus be combined, subtleties that can only be conveyed visually can be explored and at the same time, nuances of tone, pitch, rhythm, volume etc., can be put to use as well.

182

Anna Katharina Schaffner | From Concrete to Digital

Thirdly, digital signifiers come attached with an additional technical dimension. One of the major concerns of avant-garde and concrete poetry alike is the exploration of the medium of usage, the language material, its physically perceptible qualities, its visual and acoustic dimension. In extreme cases, signs are deprived entirely of their representative function and pragmatic use value, referring to themselves and their concrete materiality alone. In Hansjörg Mayer’s series alphabet (1963), or Raoul Hausmann’s poster poems (1918), language itself is thematized and staged: its codes and structures, as well as its aesthetic, epistemological and cognitive dimensions, become the centre of poetic attention. In digital poetry too, attention is frequently directed to the medium and its conventions—one of the reasons why many consider it a continuation of the avant-garde tradition in the first place (cf. Block, Heibach and Wenz 25). However, on the screen, the material is no longer just language, but language with a whole new halo of technical meaning attached to it (cf. Block, “Digital Poetics”). As Florian Cramer has pointed out, language in its specific manifestation in the computer is marked by a paradoxical double function as both message and code: language is not only transmitted as message on the screen, but also controls and generates this transmission behind the screen in the form of codes and programming languages. Self-reflexive digital texts frequently include or reference the processes by which they were generated, thus reflecting upon the technologies that have produced them (cf. Hayles 46). While self-reflexivity in print is limited to an implicit thematization of poetic, linguistic and epistemological conventions, self-reflexivity on the screen can include all of the above, plus a contemplation upon the technological processes involved. This dimension is most obvious in code poetry, which draws all the processes which usually happen behind the screen to the fore, and explores digital codes ranging from basic binary to hexadecimal and ASCII to complex programming languages. Machine and program code is thus turned into poetic material. The title of Beiguelman’s reversion (2001) is programmatic yet again, for she turns surface/background relations on their head: what is usually in the back, namely code, is drawn to the fore and is presented as the actual work itself. Something similar happens in %Location by JODI, a Belgian-Dutch duo consisting of Joan Heemskerk and Dirk Paesman, which features a long continuous string of unintelligible ciphers which look like code. The key to %Location is in the source code window, which, when opened, features a graphic representation of an H-Bomb, made up of spatially arranged ASCII signs. This piece of ASCII art is subsequently interpreted as HTML code and represented on the screen. %Location seems to suggest that an H-bomb, when unleashed, or literally, in this case, when it is executed as code, turns orderly structures into

183

Anna Katharina Schaffner | From Concrete to Digital

chaos. Moreover, by equating code with a bomb, it reflects on the potentially destructive force of code, its latent power to cause destruction. Fourthly, in the sphere of the digital, language is frequently treated graphically and presented visually, as image. N. Katherine Hayles has coined the notion of the “flickering signifier” in her study How We Became Posthuman: text, she argues, morphs into a flickering image, an instable visual display, and is no longer a physical, material object. Beiguelman too emphasizes the imagetic condition of the screen text, and at the same time the essentially textual condition of the Web: on the screen, images perform texts, and behind the screen, texts generate these images, an idea which she explores both visually and textually in the book after the book (1997). The internet, Beiguelman writes, “is no more than a big text. On the front, at the screen, text reveals itself as image” (the book after the book). The fifth way in which text on the screen differs from text in print is that it is transient and changeable. Textual fluidity is one of the main new characteristics of the new signifiers. Flickering signifiers, as Hayles argues, are characterized by “their tendency toward unexpected metamorphoses, attenuations, and dispersions” (30). “When a text presents itself as a constantly refreshed image rather than as a durable inscription,” Hayles, writes, “transformations can occur that would be unthinkable if matter or energy, rather than informational pattern, formed the primary basis for the systemic exchanges” (30). The sixth and final way in which the poetic space of the screen is redefined is the addition of two further dimensions which the page lacks: a temporal one, and the third dimension, the simulation of depth. New spatio-temporal parameters thus become possible. Duration and speed of reception can be dictated and imposed tyrannically upon the viewer, as in Young-Hae Chang’s works. Furthermore, space is no longer flat, and multiple layers of textual organisation become possible. Foreground and background relations can be constructed, letters can be superimposed upon others, and distance and proximity can be simulated, as in John Cayley’s lens (2006). Writing becomes volumetric: letters can suddenly be viewed from all sides, from behind, below, above, they can be rotated and turned around their own axis like real objects in space, as in Mary Flanagan’s “[theHouse]” (2006), or in Dan Waber’s, Jason Pimble’s and Aya N. Karpinska’s works.

2

The Concrete Founding Father of Digital Poetry: On Max Bense

Not only is there significant conceptual overlap between the poetics of concrete and those of digital poetry, but, moreover, digital poetry was born in the

184

Anna Katharina Schaffner | From Concrete to Digital

orbit of concrete poetry in the late 1950s in Stuttgart. In fact, the deployment of computers for the artificial generation of poetry can be considered both as a radicalisation of certain aspects of concrete poetry and as its logical consequence. Max Bense, a German scholar, philosopher and poet, one of the leading figures of the Stuttgarter Gruppe (‘Stuttgart Group’) and an important international mediator between different concrete groups and factions, was a key figure who precipitated this development. Bense worked on the interface between aesthetics, mathematics, technology and theory of science, and was one of the earliest advocates of information theory, cybernetics and semiotics in Germany. In a radically interdisciplinary manner, Bense tried to establish an exact, scientific and objective branch of aesthetics by means of applying mathematical and information theoretical premises to the study of aesthetic texts. Essentially, his objective was to shift the assessment, discussion and ultimately also the production of literature from an emotional basis towards a rational one. He considered his poetic work a theory-based inquiry into the nature of language, signs and communication, and vigorously rejected the notion of the emotionally driven, romantic and intuitive creator. The preoccupation with objectivity seems to be a general concern of concrete poets, though its significance varies in different poetic frameworks. Bense is certainly the most radical pursuer of scientific values, but Gomringer too emphasizes the importance of method, system and structure (19-22). Experiments with stochastic, permutational and combinatorial structures can also be observed in many other concrete oeuvres, such as Mon’s and those of the Wiener Gruppe (‘Vienna Group’). The quest for objectivity can be considered a direct consequence of the notion of literature as experiment, as research, as poetic-scientific investigation into the nature of communication, language and signs. Furthermore, the emphasis on objectivity is in keeping with the focus on the material, autonomous linguistic world and the notion of language as a concrete object with physically perceptible and manipulable dimensions. This conception implies that, as an object, language should be measurable and classifiable according to exact scientific parameters, and that it can be subjected to experiments and tests just like other material or numerical signs too. In his Einführung in die informationstheoretische Ästhetik from 1969, Bense defines “aesthetic states” of texts according to their degree of unexpected, surprising and non-trivial occurrence of words (110). This notion is a direct transfer of Claude E. Shannon’s definition of information as “unexpected, unpredictable news” into the realms of the aesthetic. Bense worked, amongst other things, with frequency dictionaries and stochastic and statistical devices to determine the extent of unexpectedness of a word in a textual set-up. The step, then, from the mathematical description of aesthetic states to the automatic generation of aesthetic texts is not a big one: the purely formal analysis of texts

185

Anna Katharina Schaffner | From Concrete to Digital

can be turned into a set of rules for their synthetic generation, similar to the idea of generative grammars (Bense 109).2 Bense and his students were the first to utilize computers for purely aesthetic purposes to produce stochastic, machine-generated poetry. They deployed the random function for the generation of “unlikely, highly selective and non-trivial” sequences—which is Bense’s definition of what makes a text aesthetic rather than functional. In 1959, in the computer lab of the “Technische Hochschule” (‘Technical University’), Theo Lutz fed vocabulary taken from Franz Kafka’s Das Schloß [The Castle] into a Zuse Z 22, and wrote a program determining several rules of combination, and thus generated the first artificial, chance-determined literary text. One of the first sentences of the experiment read: “NICHT JEDER BLICK IST NAH UND KEIN DORF IST SPÄT.” Because pragmatic selection restrictions are violated, syntactic compatibility clashes contrasts starkly with semantic incompatibility. However, it is precisely this incongruity, the surprise moment and the bizarre imagery which constitute the text’s poetic appeal. The artificial generation of aesthetic texts is a radical attempt to eliminate the subjective dimension from art and to convey aesthetics detached from semantic meaning. The aesthetic interest is transferred entirely into the purely material realm. “Material anstelle von Bedeutung” (‘Material instead of meaning’), Bense proclaims in “Manifest einer neuen Prosa und Poesie” (‘Manifesto of New Prose and Poetry’), the first computer poetry manifesto, written in 1960. “Die Strategie des Sprachspiels digitaler Texte beabsichtigt, der Außenwelt semantische Verluste beizubringen, um ästhetische Gewinne zu erzielen” (‘The linguistic game strategy of digital texts intends to inflict semantic losses upon the exterior world, so as to gain aesthetic surplus’), he writes. Here, chance is effectively deployed as a tool to transgress subjective powers of imagination, to transcend limits of comprehension in an attempt to generate results which surpass cultural, psychological, linguistic and intellectual boundaries. Many concerns both of the historical avant-garde and the concrete poets converge in Bense’s framework, and are pushed to their most radical extremes. Whilst the field of inquiry has expanded into many other directions since these early experiments, permutational works are still a viable sub-genre in digital poetry. Other concrete poets too have explored the new expressive repertoire opened up by computer technology, amongst them Reinhard Döhl, Emmett Williams and Augusto de Campos. De Campos uses the Web primarily as a rather conservative transmission medium for his poems, and exhibits static representations of concrete poems, some of them equipped with a soundtrack, such as tensao (1956) and cidade/city/cité (1975). In these pieces, the visual and the acoustic dimension do not enter into an innovative dialogue, as in more

186

Anna Katharina Schaffner | From Concrete to Digital

complex digital works, but remain separate. However, de Campos also animated some of his poems by adding a kinetic dimension, such as “poema bomba” (1983-97), and “hearthead” (1980). In an interview with Roland Greene from 1992, de Campos too expresses the hope that the new media experiments which are being conducted by the new generation will present the third stage of the avant-garde quest: “It is perhaps . . . the exploration of new technological media, and in their interaction with the spectacular arts or multidisciplinary events that we will find what remains to be done.” Moreover, de Campos, like various other digital poets and critics, argues that computers finally allow for the realisation, the literal or rather virtual putting into effect of ideas of the previous avant-gardes: theoretical movement gives way to actual movement, static structures can be animated and turned into cinematic performances, and the “verbivocovisual” structures explored conceptually by concrete poetry can be graphically and aurally executed. “The ‘wishful thinking’ of the 50s” thus came about with the digital turn and the move of experimental poetry into the digital realm (de Campos, The Yale 388). In fact the vast field of new possibilities opened up by computer technologies, de Campos argues, represents the “ideal space for ‘verbivocovisual’ adventures” (388).

3

Reconceptualizing the Topography of Poetic Space

Rita Raley has dedicated a whole edition of the Iowa Review to what she calls the “spatial turn” of digital writing. In her introductory essay, Raley calls for a new type of “deep reading,” similar to Jamesonian “archisemiotics,” which acknowledges the semantic significance of spatial design and takes into account the extension of poetic space into the third dimension. Raley argues that multidimensional digital works which integrate the z-axis into their repertoire require new critical frameworks for their analysis. A fourth type of volumetric reading becomes necessary, “reading surface to depth and back again.” “The unit of poetic analysis has shrunk from line to word to letter and now we have need of another unit,” she writes: “the three-dimensional projecting plane” (Editor’s Introduction). Once again, the links to concerns of the concretes are striking: de Campos too already called for “new spatio-temporal modes of apprehension of the text by the reader,” arguing that concrete poetry not only drastically redefined poetic space and poetic temporality, but also the reader’s perception of those categories. One of the most striking examples of 3D writing is Dan Waber’s and Jason Pimble’s “five by five” (2006), a cubic poem which can be spun and rotated into multiple directions. The poem can be revolved around its own centre, at the heart of which resides a fixed thematic word, the only stable point of

187

Anna Katharina Schaffner | From Concrete to Digital

orientation for the viewer. Three-dimensionality is suggested by scale here: the words which are conceptually closest are larger, and what is conceptually further away is smaller, as in Cayley’s lens. The American poet Aya N. Karpinska also investigates the dynamic relationships between space and meaning, the “effect of spatial arrangement on the meaning and experience of text” (“Introduction, beeBox) in works such as the arrival of the beeBox (2003) and open.ended (2004). She hopes that the extension of poetry into the third dimension will lead to “novel ways of representing relationships between words, as well as the evolution of new patterns of reading and rhythm” (Introduction, beeBox). In the arrival of the beeBox for example, she focuses on concepts such as “surface versus depth, the use of regions to organize space, the direction of reading, as well as perceptual distance and motion of verses” (Introduction, beeBox). Karpinska also suggests geographic metaphors and its core concepts, such as “location, direction, distance, distribution, spatial interaction, scale, and regions,” as pillars for a grammar of the three dimensions, as new parameters and concepts for dealing with spatial structures—an idea which is very reminiscent indeed of Mon’s reflections on surface (Introduction, beeBox). The default strategy of reading from left to right and from top to bottom is rendered impotent in these 3D works, and Karpinska’s and Waber’s multidirectional, dynamic textual structures are thus substantially akin to those found in concrete poetry. In Karpinska’s work, just as in concrete poetry, the structural, conceptual and functional arrangement of the letter material is one of the most crucial features. The famous “Pilot-Plan” axiom of the Noigandres poets seems very fitting here indeed: the concrete poem, the poets wrote, “communicates its own structure: structure-content” (72).

4

Notions of Concrete on the Web

Conceptually and visually, the parallels between digital and concrete works are striking indeed. But what happens to the notion of “concrete” on the Web? In philosophical terms, concrete is the opposite of abstract, a concrete thing can be perceived by the senses; it is particular and thus occupies a specific place both in space and time. In 3D digital poetry, words are indeed concrete in this sense—they inhabit their own spatial and temporal position and can be turned around like real objects, viewed from behind and from below, a development which adds a further twist to the Noigandres group’s notion of concrete poetry as “tension of things-words in space-time” (“Pilot Plan” 71). Viewed from a different angle, however, digital works are the opposite of concrete, for they are immaterial, merely a transient array of pixels on the screen, a representation of binary data generated by a string of zeros and ones, lacking a permanent,

188

Anna Katharina Schaffner | From Concrete to Digital

physical and material body. Paradoxically, the simulation of concreteness is the result of entirely abstract processes. Many digital poems illustrate the concept of the autonomous linguistic “Eigenwelt” that Gomringer defines in his writings (17). Text is represented for its graphic qualities, or, less frequently, for its acoustic ones, and its representative function is just one amongst other possible textual roles. Andrews’ entirely abstract, visual, interactive sound poem Nio (2001) is a point in case. Concrete poems, Gomringer writes, are not poems “about something but concrete realities in themselves” (17)—this also holds true for the abstract linguistic constellation Untitled by Squid Soup (2000) for instance. Squid Soup generate a three-dimensional orange space defined by walls made of letters, through which the user navigates accompanied by a jazzy sound track and a murmur of space-age voices. Single letters can be clicked to generate images and sounds which float into the constellation. It is one of the numerous examples which literally “fall between the media,” and the simultaneous presence of sounds, letters and shapes, which are represented both aurally and typographically, immerse the user completely in the tension between the different sign systems. Squid Soup also created a textual tool called the Scruncher (2006), which allows people to send e-mail messages to friends. These messages unfold their meaning only gradually, and are virtual, interactive textual sculptures, kinetic 3D animations which can be turned around their own axis and can be “scrunched” by the recipient, like an unwanted piece of paper which one has finished reading—they are thus ironically reinstating a material dimension to the net by means of graphically imitating qualities of paper. Another very concrete piece in that sense is NeEn by Rat, which explores both sound and visuals in a concentrated, minimalist and constructivist manner. The white on black syllables “ne” and “en” are combined in various constellations both visually and acoustically in this work. Just like the visuals, the sounds remain entirely self-referential. Two male voices articulating the sounds relating to the letters seem to act out what the letters perform visually—a tentative, explorative dance around each other, switching through all possible combinatorial clusters. The letters gradually lose their character as linguistic signs, and morph into purely geometrical constructs, lines that signify only themselves. Most digital poets deal graphically with text. Texts are treated as visual objects, no different from other graphic objects, and can be manipulated in a similar fashion, as Andrews points out: “You can move both around, size them, color them, program them, etc” (“DIGITAL LANGU(IM)AGE”). This is a profoundly concrete approach to the letter material, and Roberto Simanowski (“Fighting/Dancing Words”) too rightly characterizes Andrews’ works as extensions of concrete poetry, arguing that he explores “the new pos-

189

Anna Katharina Schaffner | From Concrete to Digital

sibilities of concrete poetry under the conditions of their being digital” in pieces such as Seattle Drift (1997) and Enigma n (1998). Enigma n is an animated anagrammatic poetic game with the words “enigma” and “meaning.” The user can prod, tame, stir and spell the word, and he or she can change the speed of the letter movement, and the color of the letters, and their size. In Seattle Drift, the user can “do,” “stop” and “discipline” the text—the words of the poem start to drift around anarchically on screen, and ask to be disciplined. Temporality, interactivity, motion and changeability are the attributes Simanowski (“Fighting/Dancing Words”) identifies as new here, whereas he designates the concern with anagrammatical structures and permutation as shared common denominator of concrete and digital poetry. Permutation is also the major poetic device of Marko Niemi’s stir-fry-texts (1999-2006). However, it is not letters or words which are reshuffled here, but rather the individual visual components of letters: straight and curved black lines in different sizes, positions and constellations. Niemi’s poems are interactive concrete dissection tools: the user can, by means of moving the mouse over different sections of the presented alphabetical signs, disassemble them graphically and recombine them with fragments from other letters. One can both draw and delete letter elements by means of following the partly invisible letter shapes with the movement of the mouse, which thus functions simultaneously a virtual rubber and a pen. Here too letters perpetually loose and regain their textual character, vacillating between linguistic signs and abstract constructivist constellations. Niemi’s works, reminiscent of poems by Mayer, Mon, Hausmann and Kurt Schwitters, thus operates on the thin threshold between symbolic and concrete signifiers.

5

Futuristic Anticipations and Digital Deconstructions

Interestingly, it is not the concrete poets who were the first aim conceptually for the effects which could be fully realized only in digital poetry, but the Italian Futurists. In 1916, F. T. Marinetti and his comrades in arms already foretold the downfall of the book in their manifesto “The Futurist Cinema.” Moreover, they envisaged the following: Parole in libertà in movimento cinematografate (tavole sinottiche di valori lirici—drammi di lettere umanizzate o animalizzate—drammi ortografici—drammi tipografici—drammi geometrici—sensibilità numerica, ecc.) (Marinetti et al. 194). Filmed Words-In-Freedom in Movement (synoptic tables of lyric values—dramas of humanized or animated letters—orthographic

190

Anna Katharina Schaffner | From Concrete to Digital

dramas—typographical dramas—geometric dramas—numeric sensibility, etc.) (Marinetti et al. 218). The prophetic vision of the Futurists seems, in turn, to corroborate one of Walter Benjamin’s theses. In Das Kunstwerk im Zeitalter seiner technischen Reproduzierbarkeit [The Work of Art in the Age of Mechanical Reproduction], Benjamin posits that artists tend to aim for effects which can be wholly realized and effectuated only with the help of future technologies: Es ist von jeher eine der wichtigsten Aufgaben der Kunst gewesen, eine Nachfrage zu erzeugen, für deren volle Befriedigung die Stunde noch nicht gekommen ist. Die Geschichte jeder Kunstform hat kritische Zeiten, in denen diese Form auf Effekte hindrängt, die sich zwanglos erst bei einem veränderten technischen Standard, d.h. in einer neuen Kunstform ergeben können. (36-37) One of the foremost tasks of art has always been the creation of a demand which could be fully satisfied only later. The history of every art form shows critical epochs in which a certain art form aspires to effects which could be fully obtained only with a changed technical standard, that is to say, in a new art form. (230) Concrete poetry seems to have anticipated the new digital medium, where many of its conceptual premises can finally be realized and effectuated literally: they are enacted externally on the screen, and no longer just mentally during the process of reception. A potentially negative aspect of this virtual enactment is that digital works can appear less open and complex than concrete poetry, which is in some respects closer to conceptual art than to literary art forms, for it relegates the responsibility for most creative and imaginary processes to the viewer. However, the visual, kinetic, interactive, changeable and potentially topographically multifaceted space of the Web allows for many new ways of arranging and manipulating the signifier material, and seems to be the perfect environment for linguistic constellations operating conceptually with the notion of poetic space in all its dimensions. Digital poetry is frequently criticized for its dependence on special effects. It is denigrated as surface spectacle, as a manifestation of the Jamesonian, superficially postmodern “culture of the depthless image” (Darley 192) and dismissed as lacking profundity and complexity. In “Concrete Poetry in Digital Media: Its Predecessors, its Presence and its Future,” Simanowski talks about the conflict between his “meaning driven soul,” which mourns for the loss of hermeneutic depth in digital works, and his “spectacle driven soul,” which enjoys technology for technology’s sake. He thematizes the predicament of the

191

Anna Katharina Schaffner | From Concrete to Digital

“technological ornament” in digital poetry, which is liberated from the burden of representation and essentially signifies itself. Simanowski draws a parallel between abstract, purely visual avant-garde art and self-sufficient non-representational code. He asks: “Is the autonomous self-centered technical effect— the code as a self-sufficient presentation on the screen—the contemporary equivalent of the pure visual?” (“Concrete Poetry”). However, Simanowski never fully answers this question, and also does not distinguish between “digital mannerism,” i.e., effect-based works which deploy technological effects uncritically and do not disrupt the symbolical order of representation, and conceptually much more complex code poetry, which deliberately disturbs representational and perceptual conventions by dissecting the surface screen spectacle into its component parts. The representation of dysfunctionalized, self-referential code can indeed be viewed as the most radical digital interpretation of the conceptual premises of concrete. JODI notoriously confront the viewer with raw, unformatted jumbles of signs, codes, symbols and graphics—the concrete, entirely non- and thus self-referential graphic and textual building blocks of digital works. When opening a work which is programmatically called “TEXT,” the recipient is confronted with a jumble of instructions and commands, executable code, scripts, variables and statements, machine language instructions and text strings consisting of representable signs (ASCII) and non-representable signs (binary ones), as well as basic graphic symbols in the form of colorful blocks. “TEXT” consists of an endless sea of ciphers and colors with no discernible semantic meaning. The viewer clicks his or her way from one page scattered with these symbols to the next, in a hopeless and ultimately frustrated quest for meaning. “TEXT” conjures up the disturbing visual symptoms of system crashes, malfunctions, a graphic program causing havoc, or indeed teletext, the multicolored gap-fillers that featured on television screens when the TV stations had ceased sending programs for the day. In spite of its negative connotations, “TEXT” is strangely appealing aesthetically. “TEXT” could be the simulated result of a memory dump, where raw and unformatted data, often in unreadable form, are copied from the main memory to the screen. This work is in many ways as concrete as it gets in the digital domain: here, the signs and languages of the computer in all their manifestations—as raw binary data, as machine language, as graphic symbols, as colored pixel blocks, as human language and as code—are represented for their own sake alone, referring to nothing but themselves and their aesthetic qualities. JODI very clearly work in the tradition of the concrete poets, who frequently thematize the means, the signs, the tools and channels of communication and the conventions governing their usage, rather than generating readily interpretable messages. They too privilege

192

Anna Katharina Schaffner | From Concrete to Digital

the signifiers over the signifieds, and create rupture, frustrating habitual expectations, responses and strategies on the Web. JODI too neglect the level of content for the exposure of processes and materials. By implication, they question the ideologies latent in representational conventions. They dissect the symbolic order of representation, and deconstruct software conventions, which filter, mediate, organize and structure information in a certain way. Again there are parallels to the concrete poets, who transgressed linguistic boundaries in order to liberate themselves from conceptual, epistemological and social frameworks and preconceptions. JODI put a new, ultimately political spin on the poetics of space on the Web: the smooth, glossy, animated multimedia screen spectacle is deliberately deconstructed and dissected into its constituent parts. It is unmasked as conventionbased optical illusion, and the material generative processes behind it are reinstated in an ironic yet at the same time interventionist gesture. JODI return to and expose the most basic material building blocks of the screen event, just as the concrete poets dismantled and presented the most minuscule particles of language.

Notes 1

Cf. Friedrich W. Block’s “Digital Poetics” and “Innovation oder Trivialität;” Roberto Simanowski’s “Concrete Poetry in Digital Media;” Christiane Heibach’s chapter on “Vorläufer der Literatur” and Loss P. Glazier’s “Digital Poetics.”

2

Bense writes: “Die analytische Beschreibung von Texten mit mathematischen Mitteln statistischer und topologischer Art legte von Anfang an den Gedanken nahe, die exakten Verfahren der Zerlegung in technische Verfahren eines synthetischen Aufbaus der Texte umzukehren. Verstärkt wurde der Gedanke, als es möglich wurde, datenverarbeitende Rechenanlagen mit ihrer Fähigkeit zur programmierbaren Speicherung, Selektierung, Sortierung, Repetierung und Verknüpfung von eingegebenen Daten heranzuziehen. . . . Damit drang die Idee einer künstlichen Poesie in die experimentelle Literatur der Avantgarde ein, die zugleich als synthetische oder sogar als technologische Poesie definiert werden konnte.” (109) (‘From the very beginning, the analytical description of texts with the help of mathematical means of a statistic and topological nature suggested the possibility of turning exact procedures for dissection into technical procedures for a synthetic generation of texts. This idea grew stronger when it became possible to draw upon dataprocessing computers and their capacities for programming memory, selection, sequentialisation, repetition

193

Anna Katharina Schaffner | From Concrete to Digital

and connection of inputed data. . . . At that moment, the idea of artificial poetry penetrated experimental avant-garde literature, which could at the same time be defined as synthetic or even as technological poetry.’)

Works Cited Andrews, Jim. Seattle Drift. 1997. . ———. “DIGITAL LANGU(IM)AGE: Language and Image as Objects in a Field.” 1998. 16 Sept. 2009 . ———. Enigma n. 1998. 16 Sept. 2009 . ———. “Stir Frys and Cut Ups.” 1999. 16 Sept. . ———. Arteroids. 2002. 16 Sept. 2009 . ———. “The Battle of Poetry Against itself and the Forces of Dullness.” 2005. 16 Sept. 2009 . Beiguelman, Giselle. “The Book after the Book.” 1997. 16 Sept. 2009 . ———. recycled. 2001. 16 Sept. 2009 . ———. reversion. 2001. reversion.htm>.

>oh